final field semantics

Semantics of final fields in java

Vladimir Sitnikov, Valentin Kovalenkositnikov@netcracker.com, @VladimirSitnikv

NetCracker

September 2014

Introduction

Examples

Why final is required inJMM?

String safety

String s = ...

if (checkAccess(s)) {

return readFile(s);

Is this a valid security check?

String unsafety in 1.4

String s = ...

return readFile(s);

The answer depends on the java version, and in java 1.4 the code isinsecure

String s = GLOBAL;

return readFile(s);

HackThread

GLOBAL =

"/tmp/etc/passwd"

.substring (4);

For instance: HackThread executes .substring(4) and transfers itvia data race to the checker thread

String s = GLOBAL;

return readFile(s);

HackThread

GLOBAL =

"/tmp/etc/passwd"

.substring (4);

In java 1.4 result of substring references the same char array, andthe value depends on String#offset and String#size

String s = GLOBAL;

return readFile(s);

HackThread

GLOBAL =

"/tmp/etc/passwd"

.substring (4);

Since no synchronization is in place, reader might observenot-fully-initialized String

String s = GLOBAL;

return readFile(s);

HackThread

GLOBAL =

"/tmp/etc/passwd"

.substring (4);

checkAccess might observe "/tmp/etc/passwd", and even thenreadFile might observe "/etc/passwd"

Even synchronization on s and volatile will not help!

String s = GLOBAL;

return readFile(s);

HackThread

GLOBAL =

"/tmp/etc/passwd"

.substring (4);

checkAccess might observe "/tmp/etc/passwd", and even thenreadFile might observe "/etc/passwd"Even synchronization on s and volatile will not help!

String safety in java 1.5+

String s = GLOBAL;

return readFile(s);

HackThread

GLOBAL =

"/tmp/etc/passwd"

.substring (4);

In java 1.5+ final protects from such non-initialized objects fromHackThread

Зачем нам JMM?

int x = 1;

public int neverTryThisAtHome () {

int i = this.x; // it is 1, isn’t it?

this.setX (2); // just updates x to 2

return this.x - i; // 2 - 1 == ...?

What is the result? 1? 0? -1?

int x = 1;

return this.x - i; // 2 - 1 == ...?

OK, the result is 1

final int x = 1;

return this.x - i; // 2 - 1 == ...?

Let’s add some final

final int x = 1;

return this.x - i; // 2 - 1 == ...?

The specification allows all the cases: 1, 0, and even -1! (see alsoexample 17.5.3-1)

A bit of theory

Program order

I Program order is a total order among inter-thread actions ofeach thread in source code order

I Compiler is forbidden to reorder/alter/ignore operations ifobservable behavior violates program order

I It does not mean the program is executed in program orderI For instance: program order is not defined for operations onlocal variables

Program order

I It does not mean the program is executed in program order

I For instance: program order is not defined for operations onlocal variables

Program order

Partial order

I In section 17 JLS "partial order" is mentioned 8 times

I Partial order ishb−−→ a binary relation that is:

I Reflexive: for each element x , xhb−−→ x

I Antisymmetric: if xhb−−→ y and y

hb−−→ x , then x and y are the same

element

I Transitive: if xhb−−→ y and y

hb−−→ z , then x

hb−−→ z

Partial order

element

hb−−→ z

Partial order

element

hb−−→ z

Partial order

element

hb−−→ z

Partial order

element

hb−−→ z

Happens-before

I General idea of JMM: let’s consider all the executions and forbid"bad" ones

I Happens-before is a partial order that selects allowable readsI A read either sees the latest write (in hb order)I Or any write via data race when multiple writes are

hb -unordered with the readI It must still obey other rules, especially: 17.4.8 executions andcausality requirements

Happens-before

I Happens-before is a partial order that selects allowable reads

I A read either sees the latest write (in hb order)I Or any write via data race when multiple writes are

Happens-before

I Happens-before is a partial order that selects allowable readsI A read either sees the latest write (in hb order)

I Or any write via data race when multiple writes arehb -unordered with the read

I It must still obey other rules, especially: 17.4.8 executions andcausality requirements

Happens-before

hb -unordered with the read

I It must still obey other rules, especially: 17.4.8 executions andcausality requirements

Happens-before

Semantics of final fields

whb−−→ f

hb−−→ a

mc−−→ r1

dr−→ r2⇒ w

−−→ r2

I w and r2 – the read and the write in question

I f – freeze of a final field, that is read in r1

whb−−→ f

hb−−→ a

mc−−→ r1

dr−→ r2⇒ w

−−→ r2

I w and r2 – the read and the write in questionI f – freeze of a final field, that is read in r1

whb−−→ f

hb−−→ a

mc−−→ r1

dr−→ r2⇒ w

−−→ r2

I If the only path from write to read follows all this arrows, thenwe must not see earlier writes

I If multiple paths exist, you need to go deeperI We use hb∗ notation to distinguish it from hb and since "thishappens-before ordering does not transitively close with otherhappens-before orderings"

whb−−→ f

hb−−→ a

mc−−→ r1

dr−→ r2⇒ w

−−→ r2

I If multiple paths exist, you need to go deeper

I We use hb∗ notation to distinguish it from hb and since "thishappens-before ordering does not transitively close with otherhappens-before orderings"

whb−−→ f

hb−−→ a

mc−−→ r1

dr−→ r2⇒ w

−−→ r2

I If multiple paths exist, you need to go deeperI We use hb∗ notation to distinguish it from hb and since "thishappens-before ordering does not transitively close with otherhappens-before orderings"

Freeze action w hb−→ f hb−→ a mc−→ r1 dr−→ r2

Freezes of a final field occur both at the end of the constructor, andimmediately after each modification

Freeze action

Freezes of a final field occur both at the end of the constructor, andimmediately after each modification

Freeze actionField

Dereference chain: what is that? w hb−→ f hb−→ a mc−→ r1 dr−→ r2

T local = GLOBAL; r1

int localX = local.x; r2

I r2 reads a field of an object

I Current thread did not create the objectI Hence, we must read an address of the object somewhere

I This is called r1dr−→ r2

I r2 reads a field of an objectI Current thread did not create the object

I Hence, we must read an address of the object somewhere

I r2 reads a field of an objectI Current thread did not create the objectI Hence, we must read an address of the object somewhere

Dereference chain in two threads w hb−→ f hb−→ a mc−→ r1 dr−→ r2

Thread 1

T localA = new T();

GLOBAL = localA;

Thread 2

T localB = GLOBAL; r1

if (localB != null) {

int localX = localB.x; r2

r1dr−→ r2 (thread 2 reads a field of object created in a foreign thread)

Thread 1

T localA = new T(); a

GLOBAL = localA;

Thread 2

Is there adr−→ r2?

Thread 1

T localA = new T(); a

GLOBAL = localA;

Thread 2

dr does not appear between actions of different threads!

Dereference chain: double-check w hb−→ f hb−→ a mc−→ r1 dr−→ r2

T local = GLOBAL; ra

local = GLOBAL; rb

Is there a dr ?

Dereference chain: double-check w hb−→ f hb−→ a mc−→ r1 dr−→ r2

T local = GLOBAL; ra

local = GLOBAL; rb

One of radr−→ r2 or rb

dr−→ r2 must exist, however no one can tell

Memory chain: what is that? w hb−→ f hb−→ a mc−→ r1 dr−→ r2

Thread 1

T o = new T();

GL = o; w1

Thread 2

T o = GL; r1

GL2 = o;

Thread 3

T o = GL2;

int r = o.x;

If a read sees write, then w1mc−−→ r1

Thread 1

T o = new T();

GL = o; w1

Thread 2

T o = GL; r1

GL2 = o; w2

Thread 3

T o = GL2;

int r = o.x;

If a thread writes the address of an object that was initialized in

another thread, then r1mc−−→ w2

Thread 1

T o = new T();

GL = o; w1

Thread 2

T o = GL; r1

GL2 = o; w2

Thread 3

T o = GL2; r3

int r = o.x;

r3 sees w2 ⇒ w2mc−−→ r3

Thread 1

T o = new T();

GL = o; w1

Thread 2

T o = GL; r1

GL2 = o; w2

Thread 3

T o = GL2; r3

int r = o.x; r4

r3dr−→ r4 (read of a object’s field)

⇒ r3mc−−→ r4

Thread 1

T o = new T();

GL = o; w1

Thread 2

T o = GL; r1

GL2 = o; w2

Thread 3

T o = GL2; r3

int r = o.x; r4

r3dr−→ r4 (read of a object’s field) ⇒ r3

mc−−→ r4

Thread 1

T o = new T();

GL = o; w1

Thread 2

T o = GL; r1

GL2 = o; w2

Thread 3

T o = GL2; r3

int r = o.x; r4

mc is transitive (it is a partial order) ⇒ w1mc−−→ r4

Introduction

Examples

Disclaimer

I All the examples contain data races (why bother otherwise?)

I We do not consider cases when null is observed (those cases arenot interesting)

I Lots of brains were hurt when preparing the slides

Disclaimer

I All the examples contain data races (why bother otherwise?)I We do not consider cases when null is observed (those cases arenot interesting)

Disclaimer

I All the examples contain data races (why bother otherwise?)I We do not consider cases when null is observed (those cases arenot interesting)

Trivial final (1) w hb−→ f hb−→ a mc−→ r1 dr−→ r2

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = l;

Thread 2

T o = GLOBAL;

if (o != null) {

int result = o.fx; r2

Can result become 0?

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = l; a

Thread 2

T o = GLOBAL;

if (o != null) {

int result = o.fx;

Actions in a thread form happens-before:

whb−−→ f , f

hb−−→ a

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = l; a

Thread 2

T o = GLOBAL; r0

if (o != null) {

int result = o.fx;

r0 sees write a :

amc−−→ r0

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = l; a

Thread 2

T o = GLOBAL; r0

if (o != null) {

Thread 2 did not create the object, r1 reads its field, and r is theonly read of the address, thus we have dereference chain:

r0dr−→ r1

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = l; a

Thread 2

T o = GLOBAL; r0

if (o != null) {

r0dr−→ r1⇒ r0

mc−−→ r1

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = l; a

Thread 2

T o = GLOBAL; r0

if (o != null) {

amc−−→ r1 ( mc is transitive)

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = l; a

Thread 2

T o = GLOBAL;

if (o != null) {

int result = o.fx; r1 r2

Let’s pick r2 = r1, then r1dr−→ r2 ( dr is reflexive)

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = l; a

Thread 2

T o = GLOBAL;

if (o != null) {

Here all the requirements for HB∗:

whb−−→ f

hb−−→ a

mc−−→ r1

dr−→ r2⇒ w

−−→ r2

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = l; a

Thread 2

T o = GLOBAL;

if (o != null) {

(whb∗

−−→ r2)⇒ result ∈ {42}

Array in a final field (2) w hb−→ f hb−→ a mc−→ r1 dr−→ r2

T l = new T() {{

int[] u = new int [1];

u[0] = 42; w

fx = u;

GLOBAL = l;

T o = GLOBAL;

if (o != null) {

int[] lfx = o.fx;

int result = lfx [0]; r2

T l = new T() {{

u[0] = 42; w

fx = u;

GLOBAL = l; a

T o = GLOBAL;

if (o != null) {

int[] lfx = o.fx;

whb−−→ f , f

hb−−→ a

T l = new T() {{

u[0] = 42; w

fx = u;

GLOBAL = l; a

T o = GLOBAL;

if (o != null) {

int[] lfx = o.fx; r1

r1 sees write a (it reads a regular final field):

amc−−→ r1

T l = new T() {{

u[0] = 42; w

fx = u;

GLOBAL = l; a

T o = GLOBAL;

if (o != null) {

Thread 2 did not create array, r2 reads an element of the array, r1 isthe only read of the address of the array, thus dereference chain:

r1dr−→ r2

T l = new T() {{

u[0] = 42; w

fx = u;

GLOBAL = l; a

T o = GLOBAL;

if (o != null) {

Here are all the requirements for HB∗:

whb−−→ f

hb−−→ a

mc−−→ r1

dr−→ r2⇒ w

−−→ r2

T l = new T() {{

u[0] = 42; w

fx = u;

GLOBAL = l; a

T o = GLOBAL;

if (o != null) {

(whb∗

−−→ r2)⇒ result ∈ {42}

Array reversed (2.1) w hb−→ f hb−→ a mc−→ r1 dr−→ r2

T l = new T() {{

fx = u; // (!)

u[0] = 42; w

GLOBAL = l;

T o = GLOBAL;

if (o != null) {

int[] lfx = o.fx;

int result = lfx [0];

T l = new T() {{

fx = u; // (!)

u[0] = 42; w

GLOBAL = l; a

T o = GLOBAL;

if (o != null) {

We compose hb∗ exactly as in the previous example

T l = new T() {{

fx = u; // (!)

u[0] = 42; w

GLOBAL = l; a

T o = GLOBAL;

if (o != null) {

(whb∗

−−→ r2)⇒ result ∈ {42}The result does not depend on the order of assignment to final fields!

this leaks (3) w hb−→ f hb−→ a mc−→ r1 dr−→ r2

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = this;

Thread 2

T o = GLOBAL;

if (o != null) {

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = this; a

Thread 2

T o = GLOBAL;

if (o != null) {

Actions in thread form happens-before:

whb−−→ f , a

hb−−→ f

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = this; a

Thread 2

T o = GLOBAL;

if (o != null) {

For hb∗ we need fhb−−→ a!

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = this; a

Thread 2

T o = GLOBAL;

if (o != null) {

If ahb−−→ f and f

hb−−→ a, thus a = f (anti-symmetry of hb )

However a is cannot be a freeze action!

Thread 1

T l = new T() {{

fx = 42; w

GLOBAL = this; a

Thread 2

T o = GLOBAL;

if (o != null) {

There is no hb∗ , thus all the cases are possible: result ∈ {0, 42}

Unintentional this leak (4) w hb−→ f hb−→ a mc−→ r1 dr−→ r2

Thread 1

T l = new T() {{

fx = 42; w

GLb = this;

GLa = l;

Thread 2

T u = GLb;

T o = GLa;

if (o != null) {

Thread 1

T l = new T() {{

fx = 42; w

GLb = this; b

GLa = l; a

Thread 2

T u = GLb;

T o = GLa;

if (o != null) {

int result = o.fx;

whb−−→ f , f

hb−−→ a

Thread 1

T l = new T() {{

fx = 42; w

GLb = this; b

GLa = l; a

Thread 2

T u = GLb;

T o = GLa; ra

if (o != null) {

int result = o.fx;

Suppose thread 2 sees GLa, thus amc−−→ ra!

Thread 1

T l = new T() {{

fx = 42; w

GLb = this; b

GLa = l; a

Thread 2

T u = GLb; rb

T o = GLa; ra

if (o != null) {

There should be dr somewhere: rbdr−→ r1 or ra

dr−→ r1

Thread 1

T l = new T() {{

fx = 42; w

GLb = this; b

GLa = l; a

Thread 2

T u = GLb; rb

T o = GLa; ra

if (o != null) {

If rbdr−→ r1, then we fail, since w

−−→ r1 can not be completed

Thread 1

T l = new T() {{

fx = 42; w

GLb = this; b

GLa = l; a

Thread 2

T u = GLb; rb

T o = GLa; ra

if (o != null) {

Conclusion: if a thread had ever seen object with unfrozen fields,then there is no final semantics for that object!

Reflection in action (5) w hb−→ f hb−→ a mc−→ r1 dr−→ r2

T t = new T() {{

fu = new U();

fu.x = 1; w1

GLOBAL = t;

U w = new U();

w.x = 42; w2

reflectSet(t.fu , w);

Thread 2

T t = GLOBAL;

if (t != null) {

U u = t.fu;

int result = u.x; r2

T t = new T() {{

fu = new U();

fu.x = 1;

GLOBAL = t;

U w = new U();

w.x = 42; w2

Thread 2

T t = GLOBAL;

if (t != null) {

U u = t.fu;

We’d better have w2hb∗

−−→ r2 (otherwise we must not see the writeof 0 in w.x)

T t = new T() {{

fu = new U();

fu.x = 1;

GLOBAL = t;

U w = new U();

w.x = 42; w2

reflectSet(t.fu , w); f2

Thread 2

T t = GLOBAL;

if (t != null) {

U u = t.fu;

f2 does not work for hb∗ : there is no suitable f 2hb−−→ a

T t = new T() {{

fu = new U();

fu.x = 1;

}}; f1

GLOBAL = t;

U w = new U();

w.x = 42; w2

Thread 2

T t = GLOBAL;

if (t != null) {

U u = t.fu;

f1 does not work for hb∗ as well: it must be w2hb−−→ f 1

T t = new T() {{

fu = new U();

fu.x = 1;

}}; f1

GLOBAL = t;

U w = new U();

w.x = 42; w2

Thread 2

T t = GLOBAL;

if (t != null) {

U u = t.fu;

It turns out r2 is not denied to observe 0

T t = new T() {{

fu = new U();

fu.x = 1;

}}; f1

GLOBAL = t; a

U w = new U();

w.x = 42; w2

Thread 2

T t = GLOBAL;

if (t != null) {

U u = t.fu;

Conclusion: do not alter the final field after you publish its contentsresult ∈ {0, 1, 42}

Let’s fix reflection (5.1) w hb−→ f hb−→ a mc−→ r1 dr−→ r2

T t = new T() {{

fu = new U();

fu.x = 1;

U w = new U();

w.x = 42;

GLOBAL = t; a

Thread 2

T t = GLOBAL;

if (t != null) {

U u = t.fu;

int result = u.x;

If we publish the object after all modifications of final fields, thepublication is safe: result ∈ {1, 42}

Final wrapper (6) w hb−→ f hb−→ a mc−→ r1 dr−→ r2

Thread 1

A a = new O() {{

fx = 42; w

GL = a;

Thread 2

A a = GL;

B = new B() {{

fb = o;

GL2 = b;

Thread 3

B B = GL2;

A a = b.fb;

int r = a.fx; r2

Thread 1

A a = new O() {{

fx = 42; w

GL = a; wa

Thread 2

A a = GL; ra

B = new B() {{

fb = o; wb

GL2 = b;

Thread 3

B B = GL2;

A a = b.fb; rb

int r = a.fx; r1

rb sees wb ⇒ wbmc−−→ rb

ra sees wa : wamc−−→ ra

Thread 1

A a = new O() {{

fx = 42; w

GL = a; wa

Thread 2

A a = GL; ra

B = new B() {{

fb = o; wb

GL2 = b;

Thread 3

B B = GL2;

A a = b.fb; rb

int r = a.fx; r1

Thread 2 writes an address of a foreign-created object, thus

ramc−−→ wb

Thread 1

A a = new O() {{

fx = 42; w

GL = a; wa

Thread 2

A a = GL; ra

B = new B() {{

fb = o; wb

GL2 = b;

Thread 3

B B = GL2;

A a = b.fb; rb

int r = a.fx; r1

Therad 3 did not create object A, r reads its field, and rb is theonly read of A’s address, thus dereference chain:

rbdr−→ r1

Thread 1

A a = new O() {{

fx = 42; w

GL = a; wa

Thread 2

A a = GL; ra

B = new B() {{

fb = o; wb

GL2 = b;

Thread 3

B B = GL2;

A a = b.fb; rb

int r = a.fx; r1

rbdr−→ r1⇒ rb

mc−−→ r1

Thread 1

A a = new O() {{

fx = 42; w

GL = a; wa

Thread 2

A a = GL; ra

B = new B() {{

fb = o; wb

GL2 = b;

Thread 3

B B = GL2;

A a = b.fb; rb

int r = a.fx; r1

mc is transitive (since it is a partial order) ⇒ wamc−−→ r1

Thread 1

A a = new O() {{

fx = 42; w

GL = a; wa

Thread 2

A a = GL; ra

B = new B() {{

fb = o; wb

GL2 = b;

Thread 3

B B = GL2;

A a = b.fb; rb

int r = a.fx; r1 r2

Let’s pick r2 = r1, thus r1dr−→ r2 ( dr is reflexive)

Thread 1

A a = new O() {{

fx = 42; w

GL = a; wa

Thread 2

A a = GL; ra

B = new B() {{

fb = o; wb

GL2 = b;

Thread 3

B B = GL2;

A a = b.fb; rb

int r = a.fx; r1 r2

Here are all the requirements for HB∗:

whb−−→ f

hb−−→ a

mc−−→ r1

dr−→ r2⇒ w

Thread 1

A a = new O() {{

fx = 42; w

GL = a; wa

Thread 2

A a = GL; ra

B = new B() {{

fb = o; wb

GL2 = b;

Thread 3

B B = GL2;

A a = b.fb; rb

int r = a.fx; r1 r2

(whb∗

−−→ r2)⇒ result ∈ {42}

Questions?

final field semantics

c copyright

checkaccess s

return readfile s

string safetystring

netcracker technology

program orderi program

string unsafety

quizfinal int x

Documents

natural language semantics - semantics archive

programming language semantics axiomatic semantics

comp 205: comparative programming languages semantics of...

from scalar semantics to implicature: children’s...

hit standards committee semantics standards workgroup...

lexical semantics computational semantics

im2pano3d: extrapolating 360deg structure and semantics...

im2pano3d: structure and semantics beyond the field of...

02 - syntax and semantics - uni-bonn.de · prolog syntax...

semantics final

field assignment final draft

final field work report123

field guide final jacqueline

static and dynamic semantics of the webdylucknow.weebly.com...

chicago field study final

the crop ontology - harmonizing semantics for agricultural...

semantics and computational semantics

cognitive semantics and structural semantics

semantics sense vs. reference phrasal semantics

im2pano3d: extrapolating 360deg structure and semantics...