benchmarking software implementations of 1st round ......benchmarking software implementations of...

Benchmarking Software Implementations of 1stRound Candidates of the NIST LWC Project on MCUs

Sebastian Renner, Enrico Pozzobon and Jürgen Mottok

Laboratory for Safe and Secure Systems, OTH Regensburg

November 3, 2019

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 1 / 19

I GoalsI FrameworkI Test SetupI Test CasesI PlatformsI ResultsI Conclusion & Future Work

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 2 / 19

Table of Contents

I Obtain performance figures for all NIST LWC candidatesI Provide results on popular embedded platformsI Test as many cipher variants as possible (per platform)I Fair comparison, i.e. no change of ref. implementationsI High degree of automationI Easy extensibility

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 3 / 19

Goals

I Written in C, Python and BashI Common test procedure for all platformsI MCU-specific template (platformIO/STM32CubeMX)

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 4 / 19

Framework I

I Fully automated compilation and test routineI Measurements are taken directly on the DUTI Use of NIST test vectors and API

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 5 / 19

Framework II

Test vectors sent and

verified overstdout/stdin

SubmittedAlgorithmsSubmitted

AlgorithmsSubmittedAlgorithms

MCU-specific template

C/C++/Arduinoproject structure

MCU

adapter.py

UART

JTAG / SWD

CompiledBuilds

test.pycompile_all.py

test_all.shGenerates

Compiles

All combi-nations are compiled

LogicAnalyzer

Git

Measurements from probes

TestVectors

Measurements

Logs

Runs

Figure: Core components and data flow of the test framework

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 6 / 19

Framework III

I SEGGER J-LinkI Saleae logic analyzerI GPIO pull up/down for signaling

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 7 / 19

Test Setup I

Figure: Benchmark Test Setup

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 8 / 19

Test Setup II

I Average over 1089 generated NIST test vector en- anddecryptions

I Expected PT and CT are compared to the resultsI nocrypt memcpy “algorithm“I AES-GCM implementation stripped out of mbedTLS for

reference

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 9 / 19

Test Cases: Avg. Encryption Speed

I nocrypt template size as baselineI Compilation with optimization for speed (where possible)I Bash script to determine code size

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 10 / 19

Test Cases: ROM Footprint

I Conducted only on the STM32 F746ZGI RAM gets filled with a known patternI Memory dumped after execution of algorithm(s)I Consecutive untouched memory segments are seen as

“unused memory“I nocrypt is used as a reference

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 11 / 19

Test Cases: SRAM Usage

I Arduino Uno R3: 8 bit ATmega328P MCU, 16 MHz clock, 32 KBflash

I STM32F1 “bluepill“: 32 bit ARM Cortex-M3 core, 72 MHz clock,64 KB flash

I STM32 NUCLEO-F746ZG: 32 bit ARM Cortex-M4, 216 MHzclock, 1MB flash

I Espressif ESP32 WROOM: 32 bit Xtensa LX6 MCU, 240 MHzclock, 4 MB flash

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 12 / 19

Tested Platforms

nocryp

t.ref

sneiken1

28.opt

sneiken1

92.opt

sneiken2

56.opt

tinyjam

bu12

8.op

t

tinyjam

bu19

2.op

t

clx1

28.opt

sneiken1

28.ref

clx1

92q.op

t

clx1

28q.op

t

sneiken1

92.ref

cipher

0.00000

0.00002

0.00004

0.00006

0.00008

averag

een

cryp

tiontim

e[s]

Figure: Ten fastest implementations on the STM32 F746ZG

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 13 / 19

Results: Speed

tinyjam

bu12

8.ref

tinyjam

bu25

6.ref

clx128

.ref

tinyjam

bu19

2.ref

clx128

h.ref

clx128

q.ref

clx192

h.ref

clx192

q.ref

clx256

h.ref

clx256

q.ref

cipher

0

200

400

600

800

1000

ROMusagecomparedto

nocrypt[b]

Figure: Ten smallest implementations on the STM32 F746ZG

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 14 / 19

Results: ROM

knot12

8v1.opt

ascon1

28av12

.bi32_arm

ascon1

28v12.bi32

_arm

tinyjam

bu12

8.opt

tinyjam

bu12

8.ref

ascon1

28av12

.bi32_lowreg

ascon1

28av12

.bi32

clx128

.opt

ascon1

28v12.bi32

clx128

.ref

cipher

0

20

40

60

80

100

RAMusagecomparedto

nocrypt[b]

Figure: Ten least RAM-intensive implementations on the STM32 F746ZG

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 15 / 19

Results: RAM

Figure: Test results for top ciphers per test case on the STM32 F746ZG

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 16 / 19

Results: Comparison

I Use of plain reference implementationsI Some cipher variants have not been tested (e.g. optimized

implementations)I The level of protection against SCA/FI was not taken into

account

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 17 / 19

Conclusion I

I Development of an easily extensible LWC performanceevaluation tool for embedded devices

I Successful test of 200+ cipher variants (enc/dec test case)I AES-GCM is neither best nor worst in all test cases

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 18 / 19

Conclusion II

I Extension of evaluation toolI Determine why some tests failedI SCA/FI attacks & countermeasures on e.g. fastest candidates

Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 19 / 19

Future Work