Transparent Referencesand Garbage Collectionfor Project Darkstar– making it easier to write applications on Project Darkstar Server
Esko LuontolaProgrammer, UI Designerwww.orfjackal.net
Agenda• How Darkstar's persistence model is
currently?• How transparent references and garbage
collection change the persistence model?• Future directions
Agenda• How Darkstar's persistence model is
currently?• How transparent references and garbage
collection change the persistence model?• Future directions
Darkstar requires special handling for ManagedObjects
What happens under the hood when the following is executed?
Database
Memory
Database
Memory
Transaction
BEGIN TRANSACTION
Database
Memory
Transaction
Personname = “Donald Duck”
Person donald = new Person("Donald Duck");
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
donald.setAddress(new Address("Duckburg", "Duck Street 313"));
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
Person mickey = new Person("Mickey Mouse");
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
ess
mickey.setAddress(donald.getAddress());
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
essManagedReference
id = 1
id = 1
id = 1
mickey.setRoommate(donald);mickey.roommate = AppContext.getDataManager().createReference(donald);
roommate
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
essManagedReference
id = 1
ManagedReferenceid = 2
roomm
ate
id = 2
id = 1
id = 1id = 2
donald.setRoommate(mickey);donald.roommate = AppContext.getDataManager().createReference(mickey);
roommate
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
essManagedReference
id = 1
ManagedReferenceid = 2
roomm
ate
id = 2
id = 1
id = 1id = 2
“donald” = 1
AppContext.getDataManager().setBinding("donald", donald);
roommate
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
essManagedReference
id = 1
ManagedReferenceid = 2
roomm
ate
id = 2
id = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1
Write
id = 2
“donald” = 1
COMMIT TRANSACTION
roommate
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
essManagedReference
id = 1
ManagedReferenceid = 2
roomm
ate
id = 2
id = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
Write
“donald” = 1
COMMIT TRANSACTION
roommate
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
essManagedReference
id = 1
ManagedReferenceid = 2
roomm
ate
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
roommate
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
What happens under the hood when the following is executed?
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Transaction
BEGIN TRANSACTION
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Transaction
Person donald = (Person) AppContext.getDataManager().getBinding("donald");
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
ManagedReferenceid = 2
roomm
ate
id = 1
Load
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Transaction
Person mickey = donald.getRoommate();Person mickey = donald.roommate.get();
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
ManagedReferenceid = 2
roomm
ate
id = 1
Addresscity = “Duckburg”
street = “Duck Street 313”
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
roommateid = 2
Load
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Transaction
System.out.println(mickey.getName() + " lives in " + mickey.getAddress().getCity());
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
ManagedReferenceid = 2
roomm
ate
id = 1
Addresscity = “Duckburg”
street = “Duck Street 313”
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
roommateid = 2
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Transaction
donald.setRoommate(null);
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
ManagedReferenceid = 2
id = 1
Addresscity = “Duckburg”
street = “Duck Street 313”
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
roommateid = 2
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”ManagedReference
id = 2
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Transaction
AppContext.getDataManager().removeObject(mickey);
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
ManagedReferenceid = 2
id = 1
Addresscity = “Duckburg”
street = “Duck Street 313”
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
roommate
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1Person
name = “Mickey Mouse”Address
city = “Duckburg”street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Transaction
COMMIT TRANSACTION
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
ManagedReferenceid = 2
id = 1
Addresscity = “Duckburg”
street = “Duck Street 313”
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
roommate
Write
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1
“donald” = 1
Transaction
COMMIT TRANSACTION
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
ManagedReferenceid = 2
id = 1
Addresscity = “Duckburg”
street = “Duck Street 313”
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
roommate
Delete
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1
“donald” = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
ManagedReferenceid = 2
Addresscity = “Duckburg”
street = “Duck Street 313”
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
roommate
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1
“donald” = 1
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1
“donald” = 1
What is the problem?
Too much work for the programmer!Mixed responsibilities! Leaking abstractions! Tight coupling!
. . .
Solution• Make Darkstar itself responsible for taking
care of the technical details of ManagedObjects referring to each other
• Code using an object should not know the implementation details of that object – whether it is a ManagedObject or not
• Refer to ManagedObjects through proxies> Allows lazy loading> Separates object graphs during serialization> And the code using the object will never know it!
Agenda• How Darkstar's persistence model is
currently?• How transparent references and garbage
collection change the persistence model?• Future directions
ProxyType.CLASS- generated proxy extends target class- all fields must be private- all methods must be non-final- the class must be non-final(- an accessible default constructor is not needed, as the constructor will not be called)
ProxyType.INTERFACE- generated proxy implements the same interfaces as the target class- instances of the class must always be used through their interfaces
What happens under the hood when the following is executed?
– with transparent references and garbage collection
Database
Memory
Database
Memory
Transaction
BEGIN TRANSACTION
Database
Memory
Transaction
Personname = “Donald Duck”
Person donald = new Person("Donald Duck");
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
donald.setAddress(new Address("Duckburg", "Duck Street 313"));
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
Person mickey = new Person("Mickey Mouse");
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
ess
mickey.setAddress(donald.getAddress());
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
ess
mickey.setRoommate(donald);
roommate
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
ess
donald.setRoommate(mickey);
roommateroommate
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
ess
id = 1
id = 1
“donald” = 1
AppContext.getDataManager().setBinding("donald", donald);
roommateroommate
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
ess
id = 1
id = 1
“donald” = 1
COMMIT TRANSACTION
roommateroommate
!
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 2
id = 2
id = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 id = 2
“donald” = 1
COMMIT TRANSACTION
roommatePerson Proxy
roomm
ate
Person Proxy
Write
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 2
id = 2
id = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 id = 2
“donald” = 1
COMMIT TRANSACTION
roommatePerson Proxy
roomm
ate
Person Proxy
!
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
ManagedReferenceid = 2
id = 2
id = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
Write
“donald” = 1
COMMIT TRANSACTION
Person Proxyroommate
Person Proxy
roomm
ate
Person ProxyPerson Proxy
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
address
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
ManagedReferenceid = 2
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Person Proxyroommate
Person Proxy
roomm
ate
Person ProxyPerson Proxy
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1 Person ProxyPerson Proxy
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1 Person ProxyPerson Proxy
What happens under the hood when the following is executed?
– with transparent references and garbage collection
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1 Person ProxyPerson Proxy
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
BEGIN TRANSACTION
Person ProxyPerson Proxy
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
ManagedReferenceid = 2
id = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Person donald = (Person) AppContext.getDataManager().getBinding("donald");
Person Proxy
roomm
ate
Person ProxyPerson Proxy
Load
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
ManagedReferenceid = 2
id = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Person mickey = donald.getRoommate();
Person Proxy
roomm
ate
Person ProxyPerson Proxy
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
ManagedReferenceid = 2
id = 2
id = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
System.out.println(mickey.getName() + " lives in " + mickey.getAddress().getCity());
Person Proxyroommate
Person Proxy
roomm
ate
Person ProxyPerson Proxy
Load
Addresscity = “Duckburg”
street = “Duck Street 313”
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
ManagedReferenceid = 2
id = 2
id = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 2
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
donald.setRoommate(null);
Person Proxyroommate
Person Proxy
Person ProxyPerson Proxy
Addresscity = “Duckburg”
street = “Duck Street 313”
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
ManagedReferenceid = 2
id = 2
id = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
COMMIT TRANSACTION
Person Proxyroommate
Person Proxy
Person Proxy
Addresscity = “Duckburg”
street = “Duck Street 313”
Write
Database
Memory
Transaction
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
ManagedReferenceid = 2
id = 2
id = 1
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
COMMIT TRANSACTION
Person Proxyroommate
Person Proxy
Person Proxy
Addresscity = “Duckburg”
street = “Duck Street 313”
Not Modified
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”address
Personname = “Mickey Mouse”
addr
ess
ManagedReferenceid = 1
ManagedReferenceid = 2
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1
Person Proxyroommate
Person Proxy
Person Proxy
Addresscity = “Duckburg”
street = “Duck Street 313”
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1 Person Proxy
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1 Person Proxy
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1 Personname = “Mickey Mouse”
Addresscity = “Duckburg”
street = “Duck Street 313”
ManagedReferenceid = 1
id = 2
“donald” = 1 Person Proxy
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1
“donald” = 1
Database
Memory
Personname = “Donald Duck”
Addresscity = “Duckburg”
street = “Duck Street 313”
id = 1
“donald” = 1
Agenda• How Darkstar's persistence model is
currently?• How transparent references and garbage
collection change the persistence model?• Future directions
Future directions• Include transparent references in the official
Project Darkstar Server distribution• Areas which need more work
> Scalability of the multi-node version> Improved garbage collector, multi-node support,
alternative algorithms, reference counting vs. tracing collectors
> Refactoring serialized data, rolling upgrades> Indexing and querying the database> ...
• It's all open source!
Questions?Project Darkstar Community– official web site, discussion forumshttp://www.projectdarkstar.com/
Darkstar EXP– unofficial distribution of Darkstar Server with experimental features, includes transparent references and garbage collectorhttp://code.google.com/p/darkstar-exp/
Dimdwarf Application Server– same transparent reference implementation, alternative server implementation, primarily designed for embedded standalone modehttp://dimdwarf.sourceforge.net/
