![Page 1: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/1.jpg)
Presented by
Date
Event
ODPi (Open Data Platform Initiative)
Ganesh Raju & Naresh Bhat Big Data Team, LEG
BKK14-400B March 10, 2016
Linaro Connect BKK16
Standardizing Hadoop Ecosystem
![Page 2: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/2.jpg)
Why ODPi ?● Hadoop is a collection of 47+ components
○ Compatibility issues○ Insufficient documentation○ No proper integrated tests○ Maintenance nightmare
■ Release cycle timelines■ Api changes■ Config files
● Help minimize the fragmentation and duplication of effort within the industry
![Page 3: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/3.jpg)
In the table below, you can see who supports what. Other projects supported by all the vendors include HBase, Hive, Pig, Spark, and Zookeeper – for a total of 8 projects supported by all. Potential for more.
![Page 4: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/4.jpg)
What is ODPi ?● A shared industry effort to advance the state of the Apache
Hadoop and Big Data technologies for the enterprise● Provide a well integrated and tested stable base● Bringing real enterprise demands to align with the
developer community● Configuration optimizations● Best practices and standards● Platform agnostic (ARM, X86, etc.)● Certification programs● Backed by Linux Foundation
![Page 5: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/5.jpg)
Benefits for ISVs● Eliminates fragmentation, reduces costs, accelerates time to market
○ Cost of maintaining external open source components. ○ Insufficient documentation
● Promotes compatibility between distros. ● Members can focus on innovation and differentiated value-add ● Already companies like Pivotal, Hortonworks, IBM and Altiscale have big
data solutions based on ODPi specs
![Page 6: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/6.jpg)
![Page 7: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/7.jpg)
ODPi Components● ODPi consists of 2 stacks
○ Runtime stack○ Management stack
● ODPi currently consists of only 3 components in Runtime stack and Ambari on the management stack
![Page 8: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/8.jpg)
ODPi - Current status● Automated CI loop builds are established. Local Nexus repository is setup to store all artifacts
instead of getting from apache● Utilizing BigTop as single tool for CI build, deployment and automated tests● Defining more smoke tests.● Spec for runtime stack is in draft, publically shared ● Working on certification specs for distros ● Technical progress can be found at: https://github.com/odpi
Challenges:● Concession between members. ● Management tool like Ambari
![Page 9: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/9.jpg)
Linaro is a Member ODPi● Enablement of AARCH64● Tested on Member Platforms● Participate in Technical Spec● Provide feedback to the community● Provide Engineering efforts
![Page 10: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/10.jpg)
Linaro’s contribution● Reference ARM64 build● Patches to Apache Bigtop● ODPi tested against multi node ARM based clusters● Benchmarking and performance tuning for ARM● JDK optimizations● ARM based Developer Cloud for ODPi members to set up CI builds and
tests
![Page 11: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/11.jpg)
ODPi Installation and Run Instructions● ODPi specs can be found here: https://github.com/odpi/specs/blob/v1.0.0-
runtime-draft/ODPi-Operations.md ● ODPi deb and rpm packages can be found on Linaro repositories:
○ Debian Jessie - http://repo.linaro.org/ubuntu/linaro-overlay/ ○ CentOS7 - http://repo.linaro.org/rpm/linaro-overlay/centos-7/
● ODPi Installation, setup and instructions to run○ https://github.com/96boards/documentation/wiki/ODPi-Hadoop-Installation
![Page 12: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/12.jpg)
ODPi Milestones● RC of spec for runtime stack will be delivered on March● First official release of ODPi will be by end of March● Next release will be in October with a cadence of release every 6 months● Voting procedure to be defined by April to let members choose the next
important component to be added into ODPi (Could be HBase / Hive / Spark)
![Page 13: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/13.jpg)
ODPi Roadmap● Add more tests● Certification program with tests for members to validate against them● Expanding tests to applications● Containerize ODPi● Make ODPi more cloud friendly, make Ambari-like management stack
more cloud integrated● Grow the footprint and have as many components included as possible
![Page 14: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/14.jpg)
DemoBy Naresh Bhat
![Page 15: ODPi (Open Data Platform Initiative)ODPi (Open Data Platform Initiative) Ganesh Raju & Naresh Bhat Big Data Team, LEG BKK14-400B March 10, 2016 Linaro Connect BKK16 Standardizing Hadoop](https://reader030.vdocuments.us/reader030/viewer/2022040409/5ec36fcb3daee311045bb7a4/html5/thumbnails/15.jpg)
Q & A