the art of testing trading platforms from applabs

AppLabs confidential and restricted circulation 1

The Art of Testing Trading Platforms

23rd July 2008

Ravi Gurrapadi


Agenda

» Exchange/Trading Platform Overview» Key Challenges/Risks» Acceptance Criteria» Test Governance/Management» Functional Testing/Complex scenarios» Integration Testing» Performance Testing» Failover/Recovery Testing» Summary» Questions


Typical Stock Exchange/ Trading Platform Overview


Key Challenges/Risks» Business Challenges

» Developing complex trading scenarios & test cases

» To reflect real time trading

» Integrating different modules (Front, Middle and Back office functions)

» Compliance to evolving regulations like RegNMS, MiFID

» Building & maintaining the domain competence

» Technical Challenges

» Interaction of multiple systems through several interfaces and gateways

» Automation of the complex scenarios & interface gateways

» Performance testing to ensure acceptable levels of latency as per the SLAs

» Risks

» Business – changes in business model, rules & regulatory requirements, introduction of new products, down time (revenue lost), time to market

» Technology – risk of failure, new technology integration, legacy applications, migration to a new application, 3rd party application issues affecting your own system


Acceptance Criteria

» Testing for core functionalities as specified in the functional requirements» User Management» Instrument Management» Quote/Order Management» Session Management» Trade Management» Risk Management» Compliance to Regulations

» Testing for agreed specifications as per SLAs» Performance» Security» Network» Reliability


Governance Structure


Test Governance/Management

» Major stakeholders» Client organization, development organization

» Management » Domain » Technical » Client's clients/vendors

» Adherence to agreements with all the stakeholders» Test Plans

» Deliverables» Schedule» Change management

» SLAs» Metrics» Escalation procedures

» Sub contracts


Test Planning & Design


Level 1 Business Process Flows

LIMIT

STOP LIMIT

DAYGTD / GTC

IsBUY / SELL

Is Price >= BO

BUY

Traded

ORDER BOOK

NO

Is Price<= BB

SELL

NO

YES

Is BUY / SELL

Is STOP Price > LTP

BO

Is Order Qty fully

filledYES

YES

Order traded partially routed to Order Book

NO

BUY RejectedNO

Is STOP Price < LTP

SELL

Order Book (Order in Passive)

Is STOP Price > LTP

Order elected

YES

BUYNOIs

STOP Price < LTPSELL

NO

NO

YES

YES

BB

YES


Level 2 Test Scenarios

LIMIT

STOP LIMIT

DAYGTD / GTC

IsBUY / SELL

Is Price >= BO

BUY

Traded

ORDER BOOK

NO

Is Price<= BB

SELL

NO

YES

Is BUY / SELL

Is STOP Price > LTP

BO

Is Order Qty fully

filledYES

YES

Order traded partially routed to Order Book

NO

BUY RejectedNO

Is STOP Price < LTP

SELL

Order Book (Order in Passive)

Is STOP Price > LTP

Order elected

YES

BUYNOIs

STOP Price < LTPSELL

NO

NO

YES

YES

BB

YES

Test Scenario


Level 2 Test Scenarios


Level 3 Test Cases

Test Case


Integration Testing/Interface Testing


Performance Testing

» Response times are critical

» New generation systems have minimal latency

» Time is literally money

» Highly complex

» Multiple API’s and connections to external systems

» Need to performance test sub-systems

» Variety of scenarios

» Examples: day in life of, daily peaks, fast markets

» Soak tests for memory leaks, etc.

» Performance test data

» Huge volumes of test data required

» Needs to be generated and managed efficiently


Failover/Recovery Testing

» Test system resilience to failure and the ability to continue to provide service during failure & recovery

» Test standard operating procedures to ensure they are robust and comprehensive

» Ensure recovery is with acceptable time scales and performance criteria

» Areas to cover:» Software (operating system, process and application)

» Hardware

» Power Failure

» Network


Summary

» Stock Exchanges have inherent complexity in terms of business flow and business rules – testing is a critical element in ensuring business delivery

» Key areas of interest for stock exchanges:

» Increased volumes

» Speedy execution

» Effective surveillance to safeguard investor and market interest

» These go in as requirements to a trading application and the testing process to ensure the implementation is a business success

» Having a well tested application that doesn't fail in real time gives an edge over the competition – areas of failure in this area become public news


Questions?

» For any questions following the presentation, then please e-mail [email protected]

mailto:[email protected]

the art of testing trading platforms from applabs

Technology

test scenarios test

test cases test case

testing process

business rules testing

acceptance criteria

test planning design

trading application

new application