ser321 principles of distributed software systems 2...

cnSerialization.fm Ser321 Class Notes

e Systems

on

n

Principles of Distributed Software Systems© T. Lindquist 2018 January 2018

Ser321 Principles of Distributed Softwar

2. Externalizing with Serializati• Section 2.a Introduction to Serialization• Section 2.b Basics of Java Execution• Section 2.c Basics of Serialization with Java• Section 2.d Customizing Java Serialization• Section 2.e Multi-Language Compatible Serializatio• Section 2.f Serialization in C# with .NET


n

a file for later use? across a network? with the object?now it matches the

ch as a file handle, c transfer format?

methods involved

machine to another alization.


2.a Introduction to Serializatio2.a.1 Motivation

• How can we save an instance of a user-defined class to • How can we migrate an instance of a user-defined class

- Can the class definition and associated methods come- If the class definition is already resident, how do we k

class definition on the remote host?• What if the user-defined object contains information, su

that doesn’t make sense to transfer or we want a specifi• Outcomes

- To be able to use the Java API interfaces, classes andwith Externalization and Serialization.

- To be able to move an instance from one Java virtual via the network or a file. To be able to customize seri


putingText:xternalization,

treams


2.a.2 Readings• In Java Network Programming and Distributed Com

- Chapters 4, especially 4.5: Data Streams and Object ESerialization

• In Java Network Programming 4th Edition Text:- Chapter 2: Streams

• See also Java Serialization in the Java Tutorial- Tutorial -> Essential Classes -> Basic I/O -> Object S- http://java.sun.com/docs/books/tutorial/

• Json (JavaScript Object Notation). see:

http://java.sun.com/docs/books/tutorial/

http://www.davidreilly.com/jnpbook/

http://shop.oreilly.com/product/0636920028420.do


ing Objects XMLbject Notation)

d Output in .NET

t Graphumerable or


2.a.3 Index to Serialization Examples• Serialize Group instance to a file: Section 2.c.3 Serializ• Java XML Serialization: Section 2.e.4 Serialization to• JSON serialization: Section 2.e.3 JSON (JavaScript O

Values• Basic .NET IO: Section 2.f.2 Basics of Stream Input an

System.IO• .NET Serialization: Section 2.f.3 Serializing an Objec• .NET Collections: Section 2.f.7 XmlSerializer for IEn

ICollection



2.b Basics of Java Execution2.b.1 Java Virtual Machines

• Overview of JDK Compilation and Runtime Support.


hine

files for illegal sage.

files.

existing constraints


2.b.2 Basic Elements of The Java Virtual Mac• Functional elements of a JVM.

• Byte-code Verifier - examines the code found in .class pointer usage, data conversions, instructions and stack u

• Class Loader - determines how and when to load class- Loads all classes needed to run a class,- Protects system classes

• Security Manager monitors certain operations based onfor accessing resources (files, ports, system).


zing it into the same of the java runtime

both the serializing

ponents of a he de-serializing ject being de-

s for which no class using the classpath e classloader (and

object-level pports this type of t is a sub-type of the


2.b.3 Java Runtime - Impact on Serialization• Serialization of a Java object and subsequently de-seriali

or another executing Java program crosses the boundaryenvironment (the java command):

- The class of the object being serialized, must match inand deserializing programs.

- When serialization is used to pass objects among comdistributed application, situations can arise in which tprogram has not loaded a class (or subclass) of an obserialized.

- If a program attempts to de-serialize an object of a clashas been loaded, the runtime environment will searchfor a matching class. If found, the runtime will use thbytecode verifier) to load the class.

- Java Remote Method Invocation (RMI) provides aninterface for distributing an application. Java RMI suclass loading when it receives a method parameter thadeclared parameter type.


ava

InputStream bytes) to the output ernal buffer, which stream buffering.

treamta types (int, ble (machine-

bjects)tStreame types to/from a le objects) are

that support the t streams.


2.c Basics of Serialization with J2.c.1 Java Input and Output Streams

• Buffered Streamjava.io.BufferedOutputStream, and java.io.Buffered- Buffered writing and reading single bytes (or arrays of

stream or from the input stream. Data goes into an intcan be marked or reset. The implementation handles

• Data Streamjava.io.DataOutputStream, and java.io.DataOutputS- Lets an application write and read primitive Java da

boolean, double, char, ...) to/from a stream in a portaindependent) way.

• Object Stream (built in support for externalizing Java Ojava.io.ObjectOutputStream, and java.io.ObjectInpu- Serializes and deserializes Java Objects and primitiv

stream. Object graphs (supports nesting of serializabplaced onto or obtained from a Stream. Only objects java.io.Serializable interface can be written to objec


rface can be written mented with the t language support r Serializable.

es stream-ifying all s of the object.

subordinate objects

tor of User objects.

of the Vector,

ble class are

face, so there are no r-defined class


2.c.2 Interface java.io.Serializable• Instances of classes that implement the Serializable inte

to, or read from a stream. No methods need to be impleSerializable interface. Its simply a marker indicating thafor serialization should be included. See the javadocs fo

• Stream-ifying a serializable object automatically includof the (serializable) objects making up instance attribute

- Serialization places an object graph into a stream; allare serialized.

- For example, suppose a Group object contains a Vec

- Serializing the Group will automatically serialize allincluding its User’s.

• All subtypes (classes that extend the class) of a serializathemselves serializable.

• No methods are defined in the java.io.Serializable intermethods that need to be implemented to make your useserializable.


.Time");

out);

e");

hrough:

nd in the sample

RMI


2.c.3 Serializing Objects

• Example, Serializing an object - String “Today” to a fileFileOutputStream out = new FileOutputStream("theObjectOutputStream s = new ObjectOutputStream(s.writeObject("Today");s.writeObject(new Date());s.flush();

• De-Serializing a string from a file.FileInputStream in = new FileInputStream("theTimObjectInputStream s = new ObjectInputStream(in);String today = (String)s.readObject();Date date = (Date)s.readObject();

• Different JVM’s communicate using serialized objects t

- Files - see the example: GroupFileSerialize.java fouproject groupSerialize.jar

- Sockets - see the section on Sockets.

- Remote Object Parameters - see the section covering

http://pooh.poly.asu.edu/Ser321/Examples/Serial/groupSerialize.jar


lasses

jdk tool:

cluding members

cts.pl, as well as its

al machine


2.c.4 Object Serialization for User-Defined C• implement java.io.Serializable (or Externalizable)

- No methods to implement- Object (Input|Output) Stream takes care of serializing

• What gets serialized?- Class and its signature (including serial version - see

serialver ser321.serialize.GroupImpl)- Values of all non-transient and non-static members, in

that refer to other objects- Serialization “stream-ifies” a tree of serializable obje- In GroupFileSerialize.java, we serialize a GroupIm

included User objects. See: groupSerialize.jar• What would you use the transient modifier for?

- fileHandles,- Remote object references- anything you don’t want serialized- anything that doesn’t make sense in another java virtu

http://pooh.poly.asu.edu/Ser321/Examples/Serial/groupSerialize.jar


ion

dObject or

lizable

the

rovided by the

bject's class is

ntents, and it must


2.d Customizing Java Serializat

2.d.1 Writing Objects in a Format you Design• If your class implements Serializable, and provides rea

writeObject, it will be called during (de)serialization.private void writeObject(ObjectOutputStream s) throws IOException {

s.defaultWriteObject();// customized serialization code

}

- These are not in any interface but are used with Seria

• To have complete control over serialization, implementjava.io.Externalizable interface

- writeExternal and readExternal methods must be pimplementing class.

- For Externalizable objects, only the identity of the oautomatically saved by the stream.

- The class is responsible for writing and reading its cocoordinate with any superclasses to do so.


on is generated and

ed with the version

object,

ith:

on of the class? The ompiled class:Guill definition of the hat value can be ersions of the class.

patibility is here:version.html#a6678


2.d.2 Class Versioning• Whenever you compile a Serializable class a Serial versi

imprinted on each instance of the class

• When deserializing, the version of the object is comparof the class- If .class version in JVM is not the one that created the

deserialization generates an exception- Programmers may control the serialization version w

static final long serialVersionUID=...

• How do you know the serialVersionUID of an older versiJDK provides a tool to examine the serial version of a c- serialver -classpath classes ser321.chat.ChatClient- executing the serialver command will provide the fu

serialVersionUID static field -- including its value. Texplicitly specified by the programmer in subsequent v

• A full explanation of the use of serialver and class comhttps://docs.oracle.com/javase/8/docs/platform/serialization/spec/

https://docs.oracle.com/javase/8/docs/platform/serialization/spec/version.html#a6678


alizationes

a) and de-serialize (C++).

inter-operability.ndent on the Object-pecified. What s parent classes? le), formats may be what precision).

programs written in t based.3.org/XML/.w3.org/TR/soap/

g/


2.e Multi-Language Compatible Seri2.e.1 Serialization Among Different Languag

• Inter-Language Serialization. What is it?- Serialize an object of a class from one language (Jav

the object into a running program of another language

• Built-in Language Support? No languages support such- Its not an easy problem. Object representation is depe

Oriented programming language and not commonly sformat and order are used to serialize an object and it

- Even for primitive types (int, bool, double, for exampdifferent between languages (such as, how many bits,

• What formats support moving objects between running different languages? The most accepted formats are tex- XML (Extensible Markup Language) https://www.w- SOAP (Simple Object Adapter Protocol) https://www- Json (JavaScript Object Notation) see: http://json.or

http://json.org/

https://www.w3.org/TR/soap/

https://www.w3.org/XML/


erencese)serialization.

.com/SmileFormat

entation that can be rs (MacOSX

son-1973242.html Ser321 Examples):


2.e.2 JSON (JavaScript Object Notation) Ref• JSON a popular text-based format for inter-language (d

- JavaScript Object Notation Wikipedia page:https://en.wikipedia.org/wiki/JSON

- JSON’s specification. See: http://json.org/- Smile, a binary Json format at: http://wiki.fasterxml

• JSON Implementations.- See the list of implementations at:

http://www.json.org/- C++ Json-cpp. Debian has a package for this implem

install using apt-get. Same for most package manageMacPorts, for example).https://github.com/open-source-parsers/jsoncpp

- Oracle Java Enterprise Edition (javax.json package) http://www.oracle.com/technetwork/articles/java/j

- Douglas Crockford Implementation (distributed withhttps://github.com/eskatos/org.json-javahttps://github.com/stleary/JSON-java

http://wiki.fasterxml.com/SmileFormat

https://github.com/eskatos/org.json-java

http://json.org/

https://github.com/stleary/JSON-java

http://www.json.org/

https://github.com/open-source-parsers/jsoncpp

https://en.wikipedia.org/wiki/JSON

http://www.oracle.com/technetwork/articles/java/json-1973242.html


esre:bles and integers).

:alse ]

2, 23], "id": 1}

and C++ using the


2.e.3 JSON (JavaScript Object Notation) Valu• The basic JSON value types and there representations a

- Asigned decimal number (compatible with floats, dou-125.32

- A string literal“I am a string literal”

- A boolean value: can be either true or false- An array of Json values denoted by encapsulating [ ]

[ 122.55, “Another String”, [“A sub-array”,true], f- A collection of named values (an Object), similar to a

Map<String,JSONObject> and encapsulated in { } {"jsonrpc": "2.0", "method": "subtract", "params": [4

- A null value, denoted as null• Here is an example that serializes a Group to/from Java

implementations referenced above. See: groupJson.jar

http://pooh.poly.asu.edu/Ser321/Examples/Serial/groupJson.jar


for JavaBeans.

riting/index.html

mponents. A well-defined”

mponents, although ewer version of a atible with) prior ports this reuse.g JavaBeans.Decoder which

arort. Covered next.


2.e.4 Serialization to XML• The package java.beans supports serialization to XML• What are JavaBeans?

- http://docs.oracle.com/javase/tutorial/javabeans/w- Or, http://en.wikipedia.org/wiki/JavaBeans- JavaBeans are defined to enable reusable software co

JavaBean is a Java class that defines properties and “accessor and mutator functions for those properties.

- JavaBeans were defined to allow for reusable GUI cothey are now viewable or not. For example, when a nGUI component wants to take advantage of (be comprelease versions of those components, JavaBeans sup

• The package java.beans provides facilities for managin- Among the facilities are an XMLEncoder and XML

object stream-ify a JavaBean using text-based XML.• An example of Java XML serialization: See: userXml.j• The .NET framework has built-in XML and SOAP supp

http://docs.oracle.com/javase/tutorial/javabeans/writing/index.html

http://en.wikipedia.org/wiki/JavaBeans

http://pooh.poly.asu.edu/Ser321/Examples/Serial/userXml.jar


T

t it can be persisted

code (class files).

, Soap, or XML.

en you wish to state.

the stream.

formatter with

r customization.


2.f Serialization in C# with .NE

2.f.1 Basics of Serialization in C#• Converting the state of an object into a stream so tha

or transported.

- Unlike Java, C# does not support transporting the

• Serialization in .NET supports saving objects in binary

• Binary and Soap Serialization of an object

- All properties and fields are serialized. Use binary whreplicate the object and transport or recover its entire

• XML Serialization of an object

- Only the public properties and fields are converted to

• Serialization involves using the Binary, Soap or XML

- Attributes [Serializable] and [NonSerialized]

- Possibly implementing the ISerializable interface, fo


ET System.IO

al

reamIO.jarample.cs

d and write filesrite a string buffer

t file:

the classes

irectory


2.f.2 Basics of Stream Input and Output in .N• Store and retrieve primitive data as binary values

- int, bool, string, byte, short, char, float, double, decim- use classes BinaryReader, and BinaryWriter

• See the example programs (with an Ant build file) in: st- To read and write data to a binary file: BinaryRWEx

• Store and retrieve primitive data as text values- use classes StreamReader and StreamWriter to rea- use classes StringReader and StringWriter to read/w

• See the example program to read and write data to a tex- StreamRWExample.cs

• Directory and File information can be manipulated with- Directory, DirectoryInfo, File, FileInfo- See the example program that lists the contents of a d- DirectoryInfoExample.cs

http://pooh.poly.asu.edu/Ser321/Examples/Serial/streamIO.jar


erialABoat.jarized,

ct up its inheritance

hich are objects are

y other parents.

as_aPrintRender


2.f.3 Serializing an Object Graph• Serializing an object serializes all related objects. See: s

- All parents (which the child class extends) are serial- All contained (fields or properties) are serialized.

• When an object of a derived class is serialized, each objechain is able to write its state to the stream.

• When an object of a class is serialized, any properties wable to write their state to the stream.

• Serializing a Sail object includes its Radio, Boat and an

Boat

Sail Radio

is_a

has_a

is_a

h

Document

Report

lengthdraft

keelmast

statefrequency

http://pooh.poly.asu.edu/Ser321/Examples/Serial/serialABoat.jar


s constructor

cument objects are

bute so they are not

ess on a different

he docIDNumber.


2.f.4 Marking Classes for Serialization• Serializable Attribute

- SerializableAttribute class has a single parameterles

- Attributing the class Document, as below indicates Doavailable for serialization.

• Mark transient properties with the NonSerialized attriserialized.

[Serializable]public class Document : object {

[NonSerializable]private FileStream fileHandle;...private int numberOfSections;...[NonSerializable]private int docIDNumber

}

• Since the fileHandle has no meaning in a different procmachine, this transient value should not be serialized.

• Although it could be, the class chooses not to serialize t


naryFormatter

to be written

s:apFormatter


2.f.5 Using the Formatter Classes• System.Runtime.Serialization.Formatters.Binary.Bi

class used to serialize- Create the stream to which the binary information isFileStream aStream = File.Create(fileName);

- Create the binary formatter objectBinaryFormatter aBinaryFormatter = new BinaryFormatter();

- Call the formatter’s serialize methodaBinaryFormatter.Serialize(aStream, aSailObject);

• Steps to deserialize an object from an binary document- Create the stream from which the object will be readFileStream aStream = File.OpenRead(fileName);

- Create a binary formatter objectBinaryFormatter aBinaryFormatter = new BinaryFormatter();

- Call the formatter’s deserialize methodSail mySailObject = (Sail)aBinaryFormatter.Deserialize(aStream);

• There is also a Soap serialization formatter. See the clas- System.Runtime.Serialization.Formatters.Soap.So


of an object.o serialize to do the serializingm to write the Xml

n XML document.

m (Stream,

lting object to the

perty that follows.bute not an element IsNullable = false)]nt if member is null


2.f.6 Serialization to and from Xml• The XmlSerializer only serializes the public properties• Using System.Xml.Serialization.XmlSerializer class t

- Use the XmlSerializer constructor to create an object- Create an instance of the object to serialize and a strea

to (Stream, TextWriter or XmlWriter)- Call the Serialize method to convert the object into a

• Steps to deserialize an object from an XML document- Create a suitable object to read the document or strea

TextWriter, or XmlWriter).- Invoke the Deserialize method while casting the resu

type of the original object (that was serialized).• Attributes controlling Xml Serialization

- [XmlIgnoreAttribute] - don’t serialize the public pro- [XmlAttribute] - serialize the class member as an attri- [XmlRootAttribute("Sail",Namespace="http://pooh",- [XmlElementAttribute(IsNullable = false)] - no eleme


ection classes differently.

indexer in C#) that erty of type integer.takes a single on.as is returned from e returned from the merator).

be serialized are Enumerator.ble Item property

le.cs


2.f.7 XmlSerializer for IEnumerable or IColl• The XmlSerializer treats IEnumerable or ICollection• IEnumerable

- The class must have a public Item indexed property (takes an integer, and it must have a public Count prop

- The class must implement a public Add method that parameter and adds the argument value to the collecti

- The Add method’s parameter must be the same type the Item property, or one of that type's bases (the typCurrent property on the value returned from GetEnu

• ICollection- For classes that implement ICollection, the values to

retrieved using the Item property, not by calling Get• Hashtables are not XmlSerializable because the Hashta

takes an object key.- For example, several class examples use String keys.- For examples of serialization see SerializeAHashtab

in: serialEgs.jar

http://pooh.poly.asu.edu/Ser321/Examples/Serial/serialEgs.jar


adio

) --> void

--> bool


2.f.8 Class Diagram for the Boat Example• Sail inherits from Boat and includes a Radio property

• See: serialABoat.jar

public class Boat

public class Sail : Boat

public class R

public model

private sailArea

getModel ( ) --> string

getSailArea( ) --> double

public loa, beam, draft, name

private displacement, lengthWL

getName( ) --> string

dispOverLength( ) -- > double

public name

private isOn

switchOnOff(

isRadioOn( )

is_a

has_a

http://pooh.poly.asu.edu/Ser321/Examples/Serial/serialABoat.jar


sclass diagram above

ecute

able

bers (see sail.xml).

the constructor for Radio are detected

ail’s. We need:typeof(ArrayList),

e most specific type.


2.f.9 Serializing Inherited and Nested Object• Here’s how to execute the project corresponding to the

- Download and extract the archive

- Execute with either Ant or Nant; for example: ant ex

• Observations in reading the source code

- Sail, Boat and Radio classes are all marked as serializ

- All are serialized as a part of serializing a Sail object.

- Note that XMLSerializer serializes only public mem

- In TestSerSail’s method writeXMLObject method, XmlSerializer only includes the type for Sail. Boat andby the serializer.

- Suppose we want to XML serialize an ArrayList of S XmlSerializer serializer = new XmlSerializer( (new Type[]{(typeof(Sail))}));

- An ArrayList holds objects, and it does not cast to th


t into the stream or

lues are pathanmes, paths.rties) are saved and

ation.ISerializable

ctx);

ssociations for all

rom the stream


2.f.10 Customizing Serialization• You can provide your own functionality for serialization

- To filter the values of object properties as they are purecovered from the stream.

- For example if the object contains properties whose vayou may want to convert relative paths into absolute

- To control which properties (or components of propesubsequently restored.

• How to implement custom serialization- Classes need to implement System.Runtime.Serializ- ISerializable defines a single methodpublic virtual void GetObjectData(SerializationInfo si, StreamingContext

- GetObjectData is called once to make (key,value) aproperties to be saved to the stream.

- Classes need to have a new constructorpublic MyClassName (SerializationInfo si, StreamingContext ctx);

- Constructor recovers (deserializes) object properties fusing the (key,value) associations.


soperties.inary Serialization.deserializing ry and SOAP.lSerializer to

tData and the m the stream.itives and

constructor:etByte, GetString ...g, Type), which called.d file: serialEgs.jar


2.f.11 Custom Serialization of Nested Objects• Example of Custom Serialization: see: CustomSerial.c• Xml serialization is designed to only work for public pr

- You can serialize private properties using SOAP or B• The XmlSerializer does not call GetObjectData or the

constructor, so custom serialization only works for bina- You cannot use customized serialization with the Xm

serialize a Hashtable to XML.• SerializationInfo class provides methods for GetObjec

deserializing constructor to add/retrieve members to/fro- GetObjectData calls AddValue method to add prim

serializable Objects to the stream.• To retrieve members from the stream, the deserializing

- Gets primitives and strings with methods: GetInt32, G- Gets nested objects using the method: GetValue(strin

causes the appropriate deserializing constructor to be• Download both serialization examples with an Ant buil

http://pooh.poly.asu.edu/Ser321/Examples/Serial/serialEgs.jar


efined serialization

ilt-in serialization

ing how to use the

enerate XML by work for any class


2.f.12 Generating XML• Often its necessary to generate XML without using pre-d

facilities- When there are special XML formatting needs that bu

can not accommodate.- Here are examples in both Java and C# programs show

language’s XML facilities to write and read XML.• The Java example (CourseTitles.java) shows how to g

using Java’s Reflection API, so that it could be made toin general.- See: generateXML.jar

http://pooh.poly.asu.edu/Ser321/Examples/Serial/generateXML.jar


the Language

oc/tutorial.html


2.g Serialization in C++2.g.1 Serialization is not supported as part of

• Packages are available, such as:- http://www.boost.org/doc/libs/1_56_0/libs/serialization/d

http://www.boost.org/doc/libs/1_56_0/libs/serialization/doc/tutorial.html


without the s11n elf supports saving

ouble. This can be m class.

le:

);


2.g.2 Serialize Manually

• C++ provides limited support for file processing, whichlibrary must be used to accomplish serialization. C++ itsonly values of primitive types, such as short, int, char ddone by using either the C FILE structure or C++ fstrea

• To save a value, use the fstream class. Here is an examp#include <fstream>#include <iostream>using namespace std;class Student{public:

char FullName[40];char CompleteAddress[120];char Gender;double Age;bool LivesInASingleParentHome;

};int main(){

Student one;strcpy(one.FullName, "Ernestine Waller");strcpy(one.CompleteAddress, "824 Larson Drv, Silver Spring, MD 20910"one.Gender = 'F';one.Age = 16.50;one.LivesInASingleParentHome = true;ofstream ofs("fifthgrade.ros", ios::binary);return 0;}


alue will be stored. es to a stream, you

e of the variable to

);


2.g.3 Writing to the Stream• The ios::binary option lets the compiler know how the v

This declaration also initiates the file. To write the valucan call the fstream::write() method.

• After calling the write() method, you can write the valuthe medium. Here is an example:

#include <fstream>#include <iostream>using namespace std;class Student{public:


};int main(){

Student one;strcpy(one.FullName, "Ernestine Waller");strcpy(one.CompleteAddress, "824 Larson Drv, Silver Spring, MD 20910"one.Gender = 'F';one.Age = 16.50;one.LivesInASingleParentHome = true;ofstream ofs("fifthgrade.ros", ios::binary);ofs.write((char *)&one, sizeof(one));return 0;

}


am::read() method.

);

e << endl;


2.g.4 Reading From the Stream• Reading an object saved in binary format: call the ifstre#include <fstream>#include <iostream>using namespace std;class Student{public:


};int main(){ /* First student

strcpy(one.FullName, "Ernestine Waller");strcpy(one.CompleteAddress, "824 Larson Drv, Silver Spring, MD 20910"one.Gender = 'F';one.Age = 16.50;one.LivesInASingleParentHome = true;ofstream ofs("fifthgrade.ros", ios::binary);ofs.write((char *)&one, sizeof(one)); /* Second studentifstream ifs("fifthgrade.ros", ios::binary);ifs.read((char *)&two, sizeof(two));cout << "Student Information\n";cout << "Student Name: " << two.FullName << endl;cout << "Address: " << two.CompleteAddress << endl;cout << "Gender: “ << two.Gender << endl;cout << "Age: " << two.Age << endl;cout << "Lives in a single parent home" << two.LivesInASingleParentHomcout << "\n";return 0;

}

ser321 principles of distributed software systems 2...

Documents