apach avro
TRANSCRIPT
![Page 1: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/1.jpg)
Apach Avro
![Page 2: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/2.jpg)
Overview 1
Data serialization system or /and Data Exchange
Resolve Hadoop Writables : lack of portability
Sharing data
Language independent-schema (JSON)
No need for code generation
![Page 3: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/3.jpg)
Overview 2
Supports schema evolution
Supports compression and splitting
Rich data types and schema
![Page 4: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/4.jpg)
Avro Data types and Schemas 1
null
boolean
int
long
float
double
bytes
![Page 5: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/5.jpg)
Avro Data types and Schemas 2
array
map
record
enum
fixed
union
![Page 6: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/6.jpg)
Avro Data types and Schemas 3
Generic Java mapping
Specific Java mapping
Reflect Java mapping
![Page 7: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/7.jpg)
In-memory Serialization and Deserialization
![Page 8: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/8.jpg)
Specific API (avro-tool)
![Page 9: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/9.jpg)
Datafiles
Schema
Avro object
Marker sync
In binary format
![Page 10: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/10.jpg)
Datafiles
![Page 11: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/11.jpg)
Portability
![Page 12: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/12.jpg)
Portability
![Page 13: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/13.jpg)
Schema resolution (Projection)
![Page 14: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/14.jpg)
Sort Order
Every avro object has ordering rule except records
Comparing works directly on the byte streams
![Page 15: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/15.jpg)
Avro MapReduce
Avro offers many API to run MapReduce on Avro data
![Page 16: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/16.jpg)
Avro MapReduce
![Page 17: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/17.jpg)
Avro MapReduce
![Page 18: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/18.jpg)
Avro MapReduce
![Page 19: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/19.jpg)
Avro MapReduce
![Page 20: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/20.jpg)
Avro MapReduce
![Page 21: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/21.jpg)
Avro MapReduce
![Page 22: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/22.jpg)
Avro MapReduce
![Page 23: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/23.jpg)
Avro Sorting MapReduce
![Page 24: Apach avro](https://reader030.vdocuments.us/reader030/viewer/2022021508/5aabf3ba7f8b9a893c8b48c7/html5/thumbnails/24.jpg)
Avro Sorting MapReduce