why java developers struggle with cryptography apis?

“Jumping Through Hoops” Why do Java Developers Struggle With

Cryptography APIs?

Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden

sarahnadi.org@sarahnadi

ICSE ’16 — May 20th, 2016

Sarah Nadi

An Application Developer’s World

2

Application

Sarah Nadi

User accounts

Payment info.


2

ApplicationSensitive user

documents

Sarah Nadi

User accounts

Payment info.

How to encrypt data?

Encryption vs Hashing?

Encryption mode?

Salted hashing?

How to securely connect to a server?


2

ApplicationSensitive user

documents

Sarah Nadi

Application Developers in The Wild

3

83% of 269 Vulnerabilities are due to misuse of crypto libraries [Lazar et al., APSys ’14]

Even Amazon & Paypal misuse SSL certificate validation [Georgiev et al., CCS ‘12]

88% of ~12,000 Android apps misuse crypto APIs [Egele et al., CCS ‘13]

Sarah Nadi 4

So what exactly is a misuse?

Sarah Nadi

Example of an API Misuse

5

Sarah Nadi


5

SecretKey secretKey = …Cipher cipher = Cipher.getInstance("AES");cipher.init(Cipher.ENCRYPT_MODE, secretKey);cipher.doFinal(inputMsg);

Sarah Nadi

Sarah Nadi


5



6

“Algorithm/Mode/Padding”

Sarah Nadi

Sarah Nadi


5



6

“Algorithm/Mode/Padding”In some API implementations,

default mode for AES is Electronic Codebook (ECB) —

which is insecure

Sarah Nadi

Sarah Nadi


5


[https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation]


6

“Algorithm/Mode/Padding”In some API implementations,

default mode for AES is Electronic Codebook (ECB) —

which is insecure

Sarah Nadi 7

Why do Developers Struggle With Cryptography APIs?

Sarah Nadi 7


What obstacles do developers face?

Sarah Nadi 7


What are the common cryptography tasks

developers perform?


Sarah Nadi 7


What are the common cryptography tasks

developers perform?

What tools/methods would help developers use cryptography

more effectively?


Sarah Nadi

Empirical InvestigationStudy 1 Study 2

Study 4Study 3

8

Sarah Nadi

Study 1 (S1) Design

9

…

<java> <cryptography>(Sorted by view count & score)

Goal: Find obstacles

Sarah Nadi

Study 1 (S1) Design

9

…

<java> <cryptography>

top 100

(Sorted by view count & score)


Sarah Nadi

Study 1 (S1) Design

9

…

<java> <cryptography>

top 100

Question topic?Likely obstacle?

(Sorted by view count & score)


Sarah Nadi

Study 2 (S2) Design

10

javax.crypto

Goal: Find tasks

Sarah Nadi

Study 2 (S2) Design

10

random 100

repos

javax.crypto

Goal: Find tasks

Sarah Nadi

Study 2 (S2) Design

10

random 100

reposWhat crypto task

is performed?

javax.crypto

Goal: Find tasks

Sarah Nadi

Study 3 (S3) & Study 4 (S4) Survey Design

11

Goal: Find obstacles, tasks, & suggestions

Sarah Nadi


11

Background


Sarah Nadi


11

BackgroundFrequency of Cryptography Use

Never Rarely …

Occasionally …

Frequently …


Sarah Nadi


11


Never Rarely …

Occasionally …

Frequently …

Cryptography Tasks Used/Needed


Sarah Nadi


11


Never Rarely …

Occasionally …

Frequently …


Frequently-used Crypto APIs/libraries & ease of use


Sarah Nadi


11


Never Rarely …

Occasionally …

Frequently …


Frequently-used Crypto APIs/libraries & ease of use

Obstacles (free-text + rated)


Sarah Nadi

Survey Process & Participant Recruitment

12

Study 4

Study 3

Sarah Nadi

Emailed <java><cryptography> posters


12

Study 4

Study 3

Sarah Nadi



12

Study 4

11 participants

Study 3

Sarah Nadi



12

Study 4

11 participants

Study 3

Refine

Sarah Nadi



12

Snowball sampling

Emailed related

committers

+

Study 4

11 participants

Study 3

Refine

Sarah Nadi



12

Snowball sampling

Emailed related

committers

37 participants+

Study 4

11 participants

Study 3

Refine

Sarah Nadi

Findings

13

TASKS

OBSTACLES

DESIRED SUPPORT

Sarah Nadi

What Obstacles do Developers Face?

14

Sarah Nadi


15

Obstacle % PostsAPI use 57%Domain knowledge 15%Provider & Setup 15%Library Identification 7%Domain knowledge + API use 6%

Sarah Nadi


15

S3 participants mention: poor documentation, bad API design, and missing cryptography knowledge


Sarah Nadi


15

S3 participants mention: poor documentation, bad API design, and missing cryptography knowledge

65% of S4 Participants find the APIs hard to use


Sarah Nadi

Obstacle Category 1: Domain Knowledge

16

11% 24% 38% 27%

Frequently Occasionally Rarely Never Don't know

Identify correct cryptography algorithm

14%41% 43%

3%Frequently Occasionally Rarely Never Don't know

Identify relevant cryptography concepts

Sarah Nadi

Obstacle Category 1: Domain Knowledge

17

11% 24% 38% 27%


Identify correct cryptography algorithm

14%41% 43%

3%Frequently Occasionally Rarely Never Don't know

Identify relevant cryptography concepts

Developers may not always know the correct cryptography algorithm to use

Sarah Nadi

Obstacle Category 2: Setting Up

18

27% 32% 27%11% 3%


Identify relevant Java API

22%41% 30%

5% 3%Frequently Occasionally Rarely Never Don't know

Select provider

22% 27% 32%16% 3%


Setup environment

Sarah Nadi

Obstacle Category 2: Setting Up

19

27% 32% 27%11% 3%


Identify relevant Java API

22%41% 30%

5% 3%Frequently Occasionally Rarely Never Don't know

Select provider

22% 27% 32%16% 3%


Setup environment

Finding the right API/library to use & setting it up is often an obstacle in itself

Sarah Nadi

19%54%

22%5%


32% 43%19% 5%


35% 46%11% 8%


Obstacle Category 3: API Use

20

Identify sequence of method calls

Identify parameters

Understand API error messages

Sarah Nadi

19%54%

22%5%


32% 43%19% 5%


35% 46%11% 8%


Obstacle Category 3: API Use

21

Identify sequence of method calls

Identify parameters

Understand API error messages

The most frequently faced obstacle is identifying the sequence of API method calls

Sarah Nadi

Common Cryptography Tasks

22

Sarah Nadi

Common Cryptography Tasks

23

10% of analyzed GitHub repos Top task by 64% of S3 participants

Avg. Rank of 3.95 by 28 S4 participants

6% of analyzed GitHub repos Top task by 34% of S3 participants Avg. Rank 2.22, 35 S4 participants

64% of analyzed GitHub repos 37% of analyzed StackOverflow posts

Avg. rank 5.03, 25 S4 participants

User Authentication

Secure Communication

Symmetric Encryption

Sarah Nadi

What do Developers Want?

24

Sarah Nadi


25

Better Documentation

“Better documentation with examples and deeper layers of documentations with theoretical knowledge” (S4-P23)

“better examples” (S4-P2)

“More/better example code, access to API source code and high quality JavaDoc with cross references” (S4-P12)

“A real documentation would be very helpful. Some kind of best practice methodology for a crypto API usage could also help” (S4-P18)

Sarah Nadi


26

Higher Abstraction Level

“Make [the] API simpler, e.g., have a class with the name AES or RSA[;] getInstance(‘whatever’) is bad API design” (S4-P36)

“[I want something that] just takes input […]. Short and sweet” (S3-P3)

“Higher level task oriented APIs for things like public key crypto, key exchanges, ..” (S4-P27)

“High level APIs [that] can't be used incorrectly..” (S4-P8)

“A library providing simple API calls (one or two methods and simple parameters) for different use cases.” (S4-P7)

“Standardized task-based API” (S4-P1)

“…Providing higher-level APIs, .. would go along way to making sure developers don't

do something dumb.” (S4-P27)

Sarah Nadi


27

“Test tooling that understands encryption and verifies if encryption is used where it should be. Source scanners that identify configuration mistakes, weak algorithms etc.” (S4-P17)

Tool Assistance

“Some kind of testing tool…” (S4-P13)

“Special CryptoDebugger” (S4-P11)

“Templates for common used patterns” (S4-P7)

“IDE Plugin generating code pattern for specific use cases” (S4-P6)

Sarah Nadi 28



Tool Assistance

Sarah Nadi 29



Tool Assistance

How Can We Move Forward?

Sarah Nadi

The Ideal Tool Assistance

30

COMMON TASKS

TO SUPPORT

STEPS TO

SUPPORT

Sarah Nadi


30

BlockCipher

AES keySize (128, 192, 256) mode (ECB, CBC, …) padding (PKCS5Padding, NoPadding)

DES

COMMON TASKS

TO SUPPORT

STEPS TO

SUPPORT

Sarah Nadi


30

BlockCipher

AES keySize (128, 192, 256) mode (ECB, CBC, …) padding (PKCS5Padding, NoPadding)

DES

COMMON TASKS

TO SUPPORT

STEPS TO

SUPPORTKeyGenerator keyGen = KeyGenerator.getInstance("AES");keyGen.init(256);SecretKey secretKey = keyGen.generateKey();Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7PADDING");cipher.init(Cipher.ENCRYPT_MODE, secretKey);cipher.doFinal(inputMsg);

Sarah Nadi

“The ability to just perform some simple cryptographic tasks in Java without jumping through hoops

would be brilliant…” (S4-P10)

31

“Jumping Through Hoops” Why do Java Developers Struggle With

Cryptography APIs?Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden

sarahnadi.org@sarahnadi

why java developers struggle with cryptography apis?

Technology