2 Agenda Latency : Definition and Importance Latency : Definition and ImportanceLow Latency Application Architecture - QA checkpointsRisks involved - QA significanceGlobal QA Team - Effective Test ManagementGlobal QA Team – Key BenefitsBest Practices – QA MethodologyQA Service OfferingAppendix - I : QA MetricsQ & A.
3 Latency : Definition and Importance Latency - a synonym for delay, is an expression of how much time it takes for a packet of data to get from one designated point to anotherWhen talking about latency in the context of capital markets consider the round trip between trade initiation event and trade execution event.Latency involves Processing time and Propagation timeBuy Side ClientBrokerage FirmsTrading Venue/ExchangesABCDHGFEPropagation TimeProcessing TimePropagation TimeProcessing TimeImportanceBuy Side firms want to be closer to the liquidity (exchange matching engine)Faster access to market benefits the Arbitrage Trader.Cancelling Open Orders quickly - those who cancel first will reduce their potential lossesHigh frequency Traders hold the positions for as low as milliseconds employing computerized algorithms and hence very sensitive of the processing speed of markets and of their own access to the market.
4 Low Latency Application Architecture - QA checkpoints ClientExchangeBooking AppsOMSBackfiller ComponentClient Facing Component (FIX Engine)Market Facing ComponentProprietary ProtocolFIX ProtocolNative / Exchange ProtocolProprietaryProtocol12345123a3b45Validating the message received from the client.Validating Proprietary Protocol conversion within the applicationValidating the message sent to the exchange/ destination trading venues.Responses received from destination venues.Reports sent back to the client.Message sent to internal OMS systems for trade reporting and settlement.
5 Risks involved - QA significance Risks/Challenges involved emphasizing strong QA involvementIncorrect messages/orders sent to exchanges can lead to compliance and regulatory breaches which can also imply monetary penaltiesIncorrect report/trade execution not complying with FIX/specific protocol can lead to messages not being delivered to client or even denial of trade executions. This can lead to huge financial losses as the trader can be exposed to unaccounted positionsErrors in trade artifacts sent to downstream order management systems may result in erroneous trade settlement and clearanceDelays in messages sent to exchanges can result in huge direct losses to clients who stand a chance to have substantial gains from arbitrage or even marginal fluctuation in stock price changes hence emphasizing on the importance of low latency.Delays in cancelling the open orders in market can also implicate serious monetary losses on account of gaining unwanted positions. The most famous examples are “6th May Flash Crash” in US markets
6 Case Study: QA Function for Low Latency Platform of a Global Bank
7 Global QA Team - Effective Test Management BeforeUS TeamUSOff-shore QA TeamUK TeamUKAPAC TeamAPACAfterUSGlobal TeamUKOff-shore QA TeamAPACImproving quality of work by building a strong teamRemoving redundancies and duplicity.Leveraging Regional SynergiesKey Objective
8 Global QA Team – Key Benefits Sharing of Issues/ DefectsTraining and Knowledge sharingLeveragingregional best practicesStreamlined reportingTest Case reusabilityGlobal QA TeamEffective ResourceutilizationFIX protocol testingBuilding a strongSME teamPeople Benefits
9 Best Practices – QA Methodology Aligning with Rapid Deployment Process“Follow the sun” modelAutomated regression suiteReal time production scenarios testingBuild tracking mechanismExploratory TestingValue Added Testing
10 Capacity/Load Testing QA Service OfferingRegression TestingRegression testing is performed to ensure any new functionality/bug fixes do not impact the existing functionalityRegression test suites are maintained, updated and validated periodically to ensure that critical bug fixes/changes make it to the test suite which would be executed on future releases.Functional TestingFunctional Testing to test new functionalities/enhancements/ requirements and bug fixes in the new release/ build provided by Development team.Automated TestingIn-house automation tools are used to automate a substantial amount of test cases.The tool utilizes the record and replay functionality, parsing the logs and replaying against simulators.As the FIX logs have similar pattern this helped in reusing the same component for automation efforts in other regions also.Performance TestingPerformance team engaged to provide performance figures to calculate the round trip delays in QA environment against available simulators/exchange sessions.The performance figures even though not realistic provide benchmarks for QA team for further deployments.Capacity/Load TestingCapacity and Load testing performed to identify the benchmark capacity figures for the application and also to test the application behavior under variable loads, like multiple client connections.
11 Appendix -1 : QA Metrics QA Coverage: Exchanges Covered: 25+ exchanges covered across US,UK and APACProtocols Tested:FIX 4.2 protocolCustomized FIX protocolProprietary protocolExchange specific native protocolsIndicative QA Metrics for Global QA team supporting low latency applicationsTotal Team Size: Around 7Total QA Drops: 200+ YearlyTotal Defects Raised: 290+ YearlyTest Case Executed: Around Yearly