case study50 + microservices and 25+ pipeline services integration of software and hardware...

5
Case Study

Upload: others

Post on 16-Jul-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Case Study50 + microservices and 25+ pipeline services Integration of Software and hardware Involvement of various advance technologies like ReactJS, Angular, Spring boot, Zuul proxy,

Case Study

Page 2: Case Study50 + microservices and 25+ pipeline services Integration of Software and hardware Involvement of various advance technologies like ReactJS, Angular, Spring boot, Zuul proxy,

INDUSTRY

Digital Advertisement

SPECIALITY

Advertisement, Marketing, Content Creation, Interactive Experience

CHALLENGES

● The application is exposed on the Internet, outside trusted on premise boundaries. It prevents malicious or accidental actions which can compromise security.

● The changes made could degrade the performance to which users are accustomed.

● High business complexity to design and maintain all the services

● Advertisement schedule setup and running associated with Hardware setup, So need more resources

Our client is the next-generation technology for digital, out-of-home market. It creates interactive experience of digital marketing anywhere, on any space. The client's solutions allow its customers to elevate their physical locations, turning them into digital, social environments that add value and personality. The client's open and extensible, Experience as a Service (EaaS) platform has five core capabilities: Automation Support, Control Network, Content Lifecycle Management, Ad Lifecycle Management, and Experience Lifecycle Management. Value Additions Customer Delight: With in the interest of customer requirements, ATMECS delivered every sprint on time with quality and consistency Validation Techniques: ATMECS concentrated more on delivering quality output using testing techniques and approaches like BVA, ECP, etc Automation: ATMECS developed scripts to automate regular, manual tasks like backend services and front end scenarios Security: Provided highly secure zone to the application which is the latest in the current market standard, like OATH Auditing & Compliance: Established a separate team for access management and developed processes/reports to support compliance requirements Technology Stack Development: Java Front end: HTML, Angular, React bootstrap, Java Back end: Swagger UI, Rest Database: Cassandra Cloud: AWS, Bitbucket Hardware: Raspberry pi, Odroid OS: Linux, Ubuntu QA: Selenium, Jira, DevOps, Manual

Hardware in IoT Complexity

● Firmware development and modules interaction with Raspberry pi Device and odroid boards ● 20+ hardware modules interacting with IoT application ● Firmware Development and high-level testing ● High-end real-time testing scenarios ● Release the cycle every sprint ● Yocto build with supportable insertion of protocols and modules for every release

Challenges

● Testing Hardware device performance while running core embedded software modules ● Testing connectivity of device switching across all network interfaces to maintain internet connectivity ● Testing device for long run without internet so that logs don't make device memory full ● Testing new hardware interfaces like RTC, External Battery, and Cellular Module functionality ● Working on high bitrate videos and 4K videos for customer support

Page 3: Case Study50 + microservices and 25+ pipeline services Integration of Software and hardware Involvement of various advance technologies like ReactJS, Angular, Spring boot, Zuul proxy,

Achievements

● Maintaining test scenarios, test reports, and use cases for every new release of firmware ● Analyzed embedded software modules to bring negative scenarios. Found at least one corner case in all modules ● Documenting scenarios and test cases in confluence ● Found an issue while testing the module which doesn't handle sound and video together. The client appreciated for

doing R&D level testing for a future-focused feature ● By analyzing the functionalities, found some interesting issues which had not been fixed from so long as those

scenarios are not straight away ● Ability to deliver on time ● Completing tasks within the sprint with increased productivity ● Ability to work on all hardware modules ● Working on high-end features correlating with trending market values ● Planning to implement automation for hardware

DevOps in IoT Complexity

● Three development teams in different timezone ● 50+ microservices and 25+ pipeline services ● Integration of software and hardware ● Collaboration between DEV, QA, UI, Hardware, DevOps and management ● Release cycle every sprint ● Release deployment to Raspberrypi

Challenges

● 0 DevOps ● Downtime, High Availability & SLA ● Collaboration and adoption ● Version control ● Continuous Integration ● Configuration Management ● Continuous deployment ● Operations ● Database Management ● Choosing appropriate resource for microservice ● Services using high memory

Achievements

● Achieved 3.5 DevOps CMM within 12 weeks ● Collaboration improved ● Improving branching strategy ● Streamlined DevOps process (version control, continuous integration, configuration management, release

management) ● Automated release deployment without any human intervention ● Integrated automated test cases with CI ● Reduced downtime by implementing Kubernetes and high availability ● Monitoring ● Backup ● Rollback ● Implemented APM ● Performance testing to get TPS for Cassandra, Kafka, Flink, and other services

Page 4: Case Study50 + microservices and 25+ pipeline services Integration of Software and hardware Involvement of various advance technologies like ReactJS, Angular, Spring boot, Zuul proxy,

● Resolved service memory utilization ● Reduced % cost by implementing Open Source tool

User Interface Complexity

● Development based on wire frames. ● Changes in developed design with changes in requirements ● Involvement of various advance technologies like ReactJS, Angular, BootStrap, Typescript, and npm. ● Working with Elasticsearch for getting data from backend.

Challenges

● If we need to get data from the database, with the help of Elasticsearch, we need to follow query DSL, based on JSON to define queries

● We are using OAuth 2.0; we need to use JWT for managing sessions Achievements At the beginning, the application of TruClear Global was like a skeletal structure with no proper exposure or display of its features. Then we have designed and enhanced an application, with clear exposure of features, which can be handled by any individual. We have also included tool tips so that everyone can easily gain knowledge regarding it. Content Management: Angular Control Network: ReactJS Sales Portal: Angular

Performance Complexity

● 50 + microservices and 25+ pipeline services ● Integration of Software and hardware ● Involvement of various advance technologies like ReactJS, Angular, Spring boot, Zuul proxy, eureka, Apache kafka,

flink, cassandra, MQTT broker, S3 and R-pi

Challenges

● It's a challenging task to do performance testing of various IoT based applications along with various non-GUI elements like Cassandra, Flink, and Apache Kafka

● R-pi continuously communicating with architecture and continuously sending some POP, heartbeat, schedule, asset messages

● It was mandatory to find out: total payload send by individual R-pi devices and calculating approximate R-pi (edge hub) each client is using, number of clients TCG architecture can support in terms of 24*7 load

● Need to give them estimation on number of clients current architecture can support and need to give proof on which resource we can increase to ramp up more clients to fulfill their threshold criteria

Achievements

● Started with 1 client (2 R-pi/Edge hub devices) load test and reached till 400 clients without any challenges ● After 400 clients, started observing data latency in Cassandra, Flink job memory threshold etc. Had continuous

discussions with DEV/QA or DevOps experts for a detailed explanation, to find the root cause, and approach the process to improve performance threshold

Page 5: Case Study50 + microservices and 25+ pipeline services Integration of Software and hardware Involvement of various advance technologies like ReactJS, Angular, Spring boot, Zuul proxy,

● After doing some POC on flink jobs, parallelism was able to achieve the 1200 client load support on existing architecture and started observing the issue with cassandra latency

● Done various kinds of load testing on Cassandra to find bottlenecks. Currently, TCG is moving frequently-used tables/data to Elasticsearch instead of Cassandra to improve the performance

Backend

Complexity

● Application development using Spring Boot, REST Web services, Kafka API, Cassandra Accessors, Elasticsearch, Zuul, Flink, Swagger UI, Java 8, and Azure Authentication

● Creation of design diagrams for analysis and estimation of tasks using Gliffy before development of an individual task ● Role-based authentication of REST Web service endpoints using Group UUID from JWT Token of login user ● Using Elasticsearch which can store huge data and retrieve data very quickly

Challenges

● Analysing application logs using Kibana retrieving from Elasticsearch ● Flink Job error logs from Flink UI ● Cassandra being not completely SQL database; which do not supports and joins other SQL functions

Achievements

● Following new standards with respect to design of new development tasks ● Considerable performance improvement using Cassandra Accessors over Spring Cassandra API

About ATMECS Inc ATMECS is a result oriented full-service engineering and R&D organization. We are Technology Accelerators bringing in visible transformation for our clients through automation, adoption of leading edge integrated development platforms, CI/CD, DevOps, Cloud, and Big Data. Several Fortune 500 customers and exciting next gen start-up companies engage us to partner with them to solve critical business challenges. As Innovation Catalysts, we help clients lead change through AI/ML, AR/VR, IoT, Conversational BOTs & Blockchain.