perform day sg_customer showcase_nie

1 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE

1

Agenda for ISAAC’s APM JourneyAgenda for ISAAC’s APM Journey

Being ISAAC and its expectations

ISAAC’s holistic quality framework

Active Performance Management in ISAAC

APM in production

APM in development

Questions and Answers


2

Being ISAACBeing ISAAC

• ISAAC is a fully-integrated student lifecycle system


3

Being ISAAC - expectationsBeing ISAAC - expectations

• ISAAC needs to be better, faster and more effective than the legacy systems

• ISAAC turnaround time for support needs to be good

• ISAAC implementation has to be smooth

• ISAAC needs to be up almost 24 X 7

• ISAAC needs to be error-free


4

ISAAC – Being AgileISAAC – Being Agile

• ISAAC adopted parts Agile Software Development

– Welcome changing requirements, even late in development. (Agile manifesto)

– Agile processes harness change for the customer's competitive advantage (Agile manifesto)

– Iterative Development Approach

– Just in time documentation

– Evolutionary Architecture


5

Agile has its challengesAgile has its challenges

– Change requirements -> Change in Design

– Change in design -> Change in architecture

– Change in scope but no change in timeline?

– Things are built iteratively

– Regression testing to be done?

– Integration testing to done?


6

Agile – How to make senseAgile – How to make sense

• Agile Testing practices

– Sufficient quality assurance measures need to be built

– Test to be automated (critical!)

– Test coverage to be inspected

– Continuous Integration Cycle


Holistic Quality Framework• Unit Tests

– Unit tests in C#,

– Services, persistence and business logic is functional

• Functional Auto Test

– Automatic UATs, codified test suites

– Browser compatibility

• Performance Tests

– Load test, stress test

– No auto test for performance

– Gap in framework

– What happens when code goes to production?

– How to profile performance in UAT?

• Need an APM tool!


8

Hunt for the perfect APM toolHunt for the perfect APM tool

• Some tools merely monitor the services

• Some tools are difficult to setup

• Some tools are difficult to use

• Some tools are deployed on the cloud

• Dynatrace is chosen because of

– Ease of installation

– Richness of “deep-dive” information

– Vibrant online community


9

The old way – reading server logThe old way – reading server log


10

The new way – APM toolThe new way – APM tool


11

APM In ISAAC ProductionAPM In ISAAC Production


12

Case Study 1 – Student Course RegistrationCase Study 1 – Student Course Registration

• ISAAC Course Registration was launched at 2nd Jan 2014

• 2600 student teachers went online to register their courses over 2 days

• First time ISAAC handled a mission critical function for students

• Courses are registered on a first-come-first-serve basis

• Traffic Spikes are expected in the first moments of the window opening

• Not a bidding system but a real-time constraint-checking system


13

Case Study 1 – Student Course RegistrationCase Study 1 – Student Course Registration

• Major risks identified

– Can system handle the load?

– Will the quota checks be compromised under high load?

– Will the high spikes in traffic affect other ISAAC modules?

• Measures taken to address risk

– Load test done for 200 concurrent users on production environment

– Additional application server added to server farm


14

Case Study 1 - Purepaths as it happensCase Study 1 - Purepaths as it happens


15

Case Study 1 - Deep dive into transactionCase Study 1 - Deep dive into transaction


16

Case Study 2 - Intermittent SlownessCase Study 2 - Intermittent Slowness

• User complain of intermittent slowness in system for some features

• Cannot be reproduced in staging or development servers

• Was working fine for the past week

• No code patches for the past week


17

Case Study 2 - Finding of root causeCase Study 2 - Finding of root cause


18

Case Study 3 - BottlenecksCase Study 3 - Bottlenecks


19

APM In ISAAC DevelopmentAPM In ISAAC Development


20

Agile Development Process with APMAgile Development Process with APM


21

Pre-production Load Testing with DynatracePre-production Load Testing with Dynatrace

• ISAAC Course Registration was loaded tested with 200 concurrent users

• Users were simulated using functional testing frameworks

• The following are done using dynatrace

– Performance Profiling, how performance scale with number of concurrent users

• Monitored server resource such as CPU, RAM

• Monitored transaction throughput

– Stress test, increased until system break or performance degrade dramatically


Performance Profiling of Register Course Function

• Ramp up number of concurrent users

• Check how the timings vs the number concurrent users

• Stress test until performance deteriorate dramatically

• Check Average vs Worst timings for each run


Conclusion

• APM is an essential tool in ISAAC’s quality assurance framework

• It allows developers to deep-dive into their code’s performance easily

• It enables real-time insight for production systems

perform day sg_customer showcase_nie

Education

run22 company confidential

agile4 company confidential

apm2020 company confidential

isaac expectations isaac

time isaac

smooth isaac

isaac modules

case study