Presentation on theme: "Progress Apama – Complex Event Processing for the Financial Markets"— Presentation transcript:
1Progress Apama – Complex Event Processing for the Financial Markets Alexander SirotinDirector of Sales.CIS, CEE Region Apama Capital Markets11 November 2009
2Business Velocity Is Key to Efficiency Strategic Imperative: Companies must continuously find ways to speed up their business to respond to customer demands and competitive pressures.mail express fax Document transferBusinessRequirementAlgorithmic tradingReduceprocessing time100 ms20 ms20 min30 secAirline operationsDesign Strategy8 hr10 secCall center inquiriesSTP, zero-latency enterprise1 day5 minTrack financial position1 day15 minSupply chain updates3 days1 minPhone activation1 week0.5 hourRefresh data warehouseTrade settlement5 days2 hrs.Build-to-order PC4 weeks1 dayThe Business PipelinePipeline Velocity is Accelerating in all businessesE.g.Financial Trading is 10x the volume of 5yrs agoSLAs on most manufacturing is half what it was 10 years agoVirtually all retail fulfillment is next day ship… remember “allow 4-6 weeks for delivery”? It wasn’t that long ago.Note’s from Roy’s presentation:Many business managers are focused on reducing the time it takes to execute their business processes and activities. Speed underlies many modern management strategies, including the zero-latency enterprise (ZLE), "time-based competition," straight-through processing (STP), and "just in time" inventory (note that real-time enterprise and event-driven enterprise generally are synonyms for ZLE). When an order is placed online, the expectation is that the goods will arrive "faster" than in the past. When someone applies for an insurance policy or submits a claim, he or she expects a quicker decision and more visibility into the process. These expectations require remaking supply, distribution, service and marketing infrastructures. A classic example of increased business velocity can be found in PC manufacturing. The entire process — talking to the customer, configuring a system, manufacturing, testing, shipping and transferring funds from the customer's bank account — takes less than 24 hours (in some cases). Short "quote to cash" cycle times require fast, closely knit applications, often implemented using EDA, with or without BPM.Action Item: Business managers and analysts must identify which of their business processes should be accelerated to produce meaningful improvements in business results. Not all processes need to run fast.Typical Business SLAs Seconds
3We’re told that information is not timely enough Survey of 500 European Business and IT Leaders► 83% believe IT has a major influence on the ability of businesses to adapt and change► 77% believe availability and timeliness of information is inadequate to support this change► The velocity of information is not keeping up with the speed of businessSurvey methodology The survey carried out on behalf of Progress Software was conducted by Vanson Bourne Research in March blue chip companies were surveyed in a number of industries across Europe to assess enterprise IT capabilities in today’s fast-moving and complex business environment.The survey was carried out using a mixture of telephone polling and questionnaires among a sample of 500 made up in equal parts of 250 CIOs/ IT Directors and 250 business leads and departmental heads. The surveys were conducted across the following European countries: Belgium, Finland, France, Germany, Italy, The Netherlands, Spain, Sweden, Switzerland, and United Kingdom.The survey covered businesses in the following sectors: Financial Services, Manufacturing, Professional services, Retail, Telecommunications, Travel, Transportation and Logistics.Source: Vanson Bourne Research, 2008
4Conventional Ways of Examining Business Information Always looking in the rear view mirrorConventional BI is about running reports on the past. Valuable but if you want to know what’s happening NOW, it’s not good enough.How can I examine what is happening and react now?
5Market Leadership Industry Recognition: The Forrester Wave™ Leadership in architecture, features, market presence and strategy“Progress Apama earned high marks for its event processing features, its development tools, and it business end-user tools, propelling it into its position as a Leader.”The Forrester Wave™: Complex Event Processing (CEP) Platforms, Q3 2009
6Now: $141M globally 2012: $803M CEP Market Size Compound Annual Growth of 55%Apama is a Leader in CEP From IDC report: “Complex Event Processing Opportunity Analysis and Assessment of Key Products”, February 2009Comment: what proportion outside capital markets?IDC (September 2008):CAGR of 55% between 2008 and 2012Market worth $141M globally in 2008 rising to $803M by 2012
8Equity data rates have been increasing Total number of equity tradesAccording to a recent report from DB Trading volume in S&P 500 is up 50% in 12 months. Average size of trade down 25%Between the end of 2006 and 2007 the number of equity trades increased by about 40% on NasdaqOn NYSE by 60%This is combined with smaller order sizes. Between 2006 and 2007 average value of orders dropped by 40%Picture is of the crab nebulaSource: World Federation of Stock Exchanges
9Equity data rates have been increasing Source: World Federation of Stock Exchanges
10Derivatives data rates are exploding Now let’s look at derivatives.Who are the Options Price Reporting Agency? Some kind of central american body for reporting on options pricing.Between 2004 and 2008 (estimated) the number of orders will have risen by 20 fold.Source: OPRA
11Derivatives data rates are exploding So, not only are equities volumes rising, but those on derivatives exchanges are exploding.Between 2004 and 2008 (estimated) total trades on Euronext Liffe will have nearly doubled.Source: Euronext Liffe
12+ = In general Order sizes Trading volumes Data! Algo trading has caused this.Algo trading is the answer to this too!Trading volumesData!
13Algorithmic trading is now everywhere At least 80% of equity trades in US/EU go through an algoProbably half of FX trades40% of exchange traded derivatives25% of bonds.It’s everywhere in the world.It’s part of the DNA of trading organisations.
14Algorithmic Trading – What & Why Automated tradingWhen to tradeObjective: seek alphaContinuously re-calculate analyticsMonitor thresholds/rulesStat arbSpreadIndex arbHow to tradeObjective: best executionOrder execution policySlice up orders (wave trade)Route across liquidity pools (smart order routing)In market limit + timeoutsOut of market limits (“electronic eye”)Calculate 10 minute moving averageCalculate 5 minute moving averageMonitor for moving averages crossingWave trade order over next 2 hours based on historic volume profileRoute child order to liquidity pool with best priceWhat are the big uses of algo trading?Productivity. Fewer traders covering more of the market. Traders doing the boring, easy trades. Have time to spend on the more advanced stuff.This doesn’t necessarily need HFT, just “good-enough” frequency trading.Then, of course, there’s alpha generation. High frequency is important for this.But for now, Controversy about algorithmic trading dominates the news worldwide, currently under the umbrella term high frequency trading. Coverage has appeared in the US the last few months in the WSJ, NYTimes, Time Magazine, and now even on a Comedy television program called the Daily Show, which is a for the general audience and makes fun of everything that’s in the news.By this point there is no doubt that algorithmic trading has become a key component of the trading arsenal of many firms. But all the hype clouds the issues, so let’s start with what algorithmic trading is, and what it sets out to achieve.In the Apama view, algorithmic trading is the automation two things - Alpha & Execution – specifically like this:<CLICK>The investment decision - What and When to Trade: strategies that leverage computing power and sophisticated mathematical and statistical models to identify alpha investment opportunities.<Click>The execution decision - How and Where to Trade: identification, placement, and execution of orders based on predefined trading goals. Algorithms are designed to capture real-time trading opportunities by identifying tiny market inefficiencies based on various factors, including price, volume, liquidity, benchmarks, and so on- basically, achieving best execution.Some of the drivers for Algo Trading Automation in Capital Markets include:Increased electronic trading, volatility, volumes, and market fragmentationMore complex instrumentsReduced spreadsRegulatory imperatives, e.g. MiFID, RegNMSAnd perhaps most importantly… Increasing competition
15High Frequency Trading Rule !WHENMSFT price moves outside 2% of MSFT Moving Average!FOLLOWED-BY (My Basket moves up by 0.5%!AND (HPQ’s price moves up by 5%!ORMSFT’s price moves down by 2%)ALL WITHINany 2 minute time periodNASDAQNYSEMSFT Moving AverageMy BasketTHENBUY MSFTSELL HPQtimemultiple data streamsLet’s take a look at a high frequency trading ruleThat results from a model such as on the previous slideas an example of how this is done.Starting with the following real-time data streams:…[Animate & Read]temporal sequencingcomplex event sequencesreal-time data streamsreal-time constraintsautomated actions15
16The offering What our customers are doing ApamaThe offeringWhat our customers are doing
18Solution Accelerators Key ApplicationsApplicationsExamplesSolution AcceleratorsCustomersAlgorithmic TradingAlpha Seeking & ExecutionVWAP, TWAP, etcMulti-Asset, Cross AssetBuy Side, Sell Side, Prop TradingAlgorithmic TradingMarket Making & AggregationLiquidity DiscoveryMarket MakingSmart Order Routing & ExecutionAuto HedgingFX AggregationSurveillanceComplianceRisk ManagementMarket Abuse DetectionMarket Maker MonitoringOp Risk, Business ControlRisk Firewall, Auto HedgingMarket SurveillanceBest ExecutionThe solution accelerators are, to use a phrase, Apama’s “Productization of a Solutions-based approach” to solving business problems in trading.< Click through >So here is a vew of key applications in trading, mapped to the Apama accelerators that jumpstart the development and deployment of those applications, and mention of a few customers that use them.We will talk about some more of these acclerators later today in the other sessions.But the point to remember now is that the more you automate in the future, across both trading functions and across asset classes, the more we can help.Smart Order RoutingAggregated ExecutionComplianceSmart Order RoutingMajor Tier 1 BankMarket Making & PricingIntelligent Pricing, Skewing, TieringMarket MakingReal-Time PricingGlobal Tier 1 Bank1818
19Real-time, Event-based Platform for Building Applications Business & IT Tools - Apama WorkbenchGraphical Development Event ModelerDashboard BuilderCode-based DevelopmentAlpha-seeking & execution rules:CrossoverMomentum tradingStatistical arbitrageIndex arbitrageIcebergVWAPMarket participationOrder slicing...and more…InputsMarket dataOrdersAny eventApama’s run-time environment includes the Event Correlator, the technology expression of CEP.The alpha-seeking and execution rules are expressed within the correlator as a set of WHEN/THEN (rather than IF/THEN) conditions, which:Monitor the inputs such as market dataAnalyze events in those data streams according to the WHEN/THEN rules, such as a Stat arb algorithm,And Act – initiating orders to the market in real timeAnd if there’s one unique point about Apama that I want you to remember from this slide here it is… we are talking about the ability to handle both the event stream processing, AND the business logic analysis and action, in the same paradigm, both of which therefore benefit from an advanced architecture. There is no need when creating an application to leave the Apama context after building the event collection and pattern detection, and then implement the business logic and automated action in a different language that doesn’t benefit from the performance characteristics that Louie will describe in just a minute. This offers benefits to your trading applications that others will either struggle to implement in their architectures or struggle to optimize for performance when handing off to an environment that doesn’t benefit from the event based architecture.In this complete architecture everything happens in real time… this is the essence of being able to Run FAST, the speed/latency frontier of algo trading competition.Actions/OutputOrdersDerived dataDashboard interaction1919
20Apama Solution Accelerators Algorithmic TradingMarket SurveillanceFX AggregatorSmart Order RouterReal-time Pricing…. . .Capital Markets FrameworkSub-Systems & ComponentsMarket DataTrade ServicesPosition ServicesOrder BookOrder MgmtStatus UtilitiesRisk Firewall…Apama CEP PlatformSo here is a view of the accelerators and other parts of the tool kit, in the context of the rest of the Apama product.The BLUE platform in the middle is an expression of the event processing architecture and other technology components around it, and we’ll talk a more about that in just a little bit.Underneath that is a framework of over 50 Adapters that is always growing and provides two things:Built-in Connectivity to capital markets destinations and general infrastructure elementsAn open toolkit for modifying existing adapters and building more.On top of the platform is the Capital Markets Framework, a set of capital markets specific subsystems, components, and smart blocks that perform reusable common functions such as trade and position services and risk checkingAnd on top of that is our set of solution accelerators, accelerating the deployment of applications built around key capital markets business areas – and right now we are speaking specifically of Algorithmic trading.But so that you know what the future holds as you tackle more and more trading issues on a global scale with automation, here are a few other accelerators50+ Market & Infrastructure Adapters
21Algorithmic trading accelerator AlgorithmsAlpha-seeking & ExecutionCross-asset supportBacktesting facilityStrategy DesignBusiness & IT ToolsDashboardsPre-builtCustomizableIntegrated CMF ComponentsExchange simulatorRisk & position managementTrader P&LBroad ConnectivityMarket Data & ExecutionInfrastructure
25Apama Connectivity Market Data Equities Markets Analytics ReutersActiv FinancialDow Jones Elementized News FeedKOSPI (Korean Composite Stock Price Index)Nexa TickstreamWombatLime Brokerage (Citrius)Equities MarketsMarket Data Market Data sources (listed above)BovespaOther direct connectivity 2Order ExecutionFIX 4.0 – 4.4EMS/OMS/FIX Gateways (listed right)Direct Bank/Broker Connectivity (listed below)Other direct connectivity 3AnalyticsMATLABQuantlib (select functions)Intel Math Library (select functions)CQGDirect Foreign Exchange MarketsEBS Spot AiHotspot FXCurrenexFXall AccelorLava FX – Spot FXReuters Dealing 3000BGC Direct Futures & Options MarketsEuronext.LiffeIntercontinental Exchange (ICE)Chicago Mercantile Exchange (CME/NYMEX)EurexELXBM&FDirect Bank ConnectivityBarclaysDeutsche Bank AutobahnUBSGoldman SachsITGPenson BrokerboxLime BrokerageMorgan StanelyDirect Fixed Income MarketsICAP BrokertecBGC (eSpeed)Tick DatabasesVhayu/Reuters Tick Capture EngineKX Systems kdb+EMS/OMS/FIX GatewaysTransact ToolsAegis Gateway (Athena Trader)Ulbridge (Odisys)GL TradeCameronTrading Technologies (TT) ProNexa FastpathIRESSFlexTradeFidessaInfrastructure AdaptersJMSJDBCODBCSonicTibco RendezvousTibco Talarian1. If a source or destination is not shown, please inquire with your Apama representative as to its status, as new adapters are always being built and can be built as required.2. Direct connectivity can be provided as an alternative to market data sources listed 3. The Apama FIX adapter can be configured for direct connectivity to any FIX destination
26Speed/latency advantage Trading in the CloudExchange collocation – enables competition in:Cross-geographySpeed/latencySoftware-as-a-ServiceQuick time to marketMinimal investmentHostedEasy accessOptional connectivity to any other trading venues and market data sourcesApama ExamplesCQG – Automated execution for commodity & FX futuresBGC – Algorithmic trading for Fixed Income cash & futuresFFastFill – Automated futures spread trading & global exchange collocationAlgorithmicTradingCross-asset classSpeed/latency advantageRapid customizationCross-geographyCrossing borders with low-latencyTrading in the cloud offers Progress Apama's advanced capabilities to a wider trading community.Through our partnerships, clients can take advantage of hosted, connected, and integrated solutions to accelerate their time to market, minimizing investments in time, space, infrastructure, and integration.And, collocation lowers latency through physical proximity, and further enables trading organizations to cross borders to trade on markets around the world.With a cloud solution, traders can now leverage their own ideas through Apama’s graphical development tools to quickly and easily create, manage, and deploy cross-asset alpha and execution strategies.
27CQG and Progress partnership This illustrates a hosted offering.Charts, quotes and graphs (CQG) provides a hosted market data and trading signals platform. With AutoEx, these trading signals can automatically trigger an execution algo.Hosted means that it can be paid for out of an operational, rather than a cap-ex budget.That it is simple for a client to adopt.Appeals to mid-tier firms that don’t necessarily have IT expertise to build something themselves.
28Building and Deploying an Apama Application Adapters connect the event processing engine to the IT environment. Pre-packaged adapters can be dynamically deployed, configured, and modified. New adapters can be built quicklyMiddlewareDBMSPoint SystemsIntegrationOther…
29Building and Deploying an Apama Application CEP Smart Blocks encapsulate common analytics and domain specific functionality (for example, libraries of statistical analytics)Smart Block AnalyticsMiddlewareDBMSPoint SystemsIntegrationOther…
30Building and Deploying an Apama Application Event based scenarios can be modelled using a GUI tool.Business analysts can develop, configure and deploy scenarios themselvesScenario modellingSmart Block AnalyticsMiddlewareDBMSPoint SystemsIntegrationOther…
31Building and Deploying an Apama Application Customers & partnersEnd-User DashboardsScenario modellingSmart Block AnalyticsMiddlewareDatabasePoint SystemsIntegrationOther…
32What to do About it Now: Use Own Automated IP for differentiation Fears of the Black BoxIf everyone has the same algo then the competitive advantage is reducedLimited scope to use your skills – can only parameteriseMarkets are continually evolving with new opportunities emergingNeed to develop new strategies fastSpot opportunityDesign strategyBacktest and deployCustomization is now kingIf everyone has the same black boxes: no competitive advantageLeverage own intellectual property for competitive differentiationCustom “rules-based” trading is becoming criticalWhy would you want to build trading strategies in this way?Because Markets are continually evolving with new opportunities emergingAnd being able to capitalize on those opportunities not only requires speed of execution, but speed to develop unique strategies.If everyone has the same algorithm – if your trading system is a black box and you cannot see or change what’s inside - then the competitive advantage is reducedNeed to develop new strategies first, which means, toSpot opportunityDesign strategyBacktest and deployCustomization is now kingIf everyone has the same black boxes: no competitive advantageLeverage own intellectual property for competitive differentiationCustom “rules-based” trading is becoming criticalSemejnyj Recept
33Security and Compliance Staff Apama ConnectivityUser Front EndsProp TraderSecurity and Compliance StaffAPAMA PLATFORM at BROKERAGEMARKET DATA AND EXCHANGESECNSExchangesMarket Data Vendors
35All Participants Are Interested In Surveillance and Monitoring Orders (proprietary and client)BrokerTrading Venue (Exchange, ECN etc.)and MonitoringSurveillanceThere are 3 audiences to potentially adopt this type of solution- Regulators- Trading Venues- And BrokersAnd in addition – the brokers clients are interested to know that they are being protected in a fair and efficient market;Or, they need to know to keep their intentions clean, because they are being watchedRegulator35
36What Needs To Be Done - The 4 Facets Of Monitoring Market Abuse SurveillanceProscribed scenarios e.g. front running, fictitious orders, painting the tape etcMarket MonitoringLiquidity and spread monitoring, associated market halts, price /volume spikes, abnormal concentrationsOperational MonitoringThroughput, competitiveness, movements from normal behaviourManagement & ControlBoth sides of book (efficient internal matching), rogue algorithms, best execution, internal controls and limits36
37The Business Driver of FSA’s Sabre II The FSA are concerned that market abuse continues to be a significant issue. For example, FSA researchers have detected price movements, suggesting “informed trading” may have taken place prior to 28.9% of takeover announcements and 21.7% of the FTSE350 trading announcements, and which were identified as being most likely to contain information of use to an insider trader.Sally Dewar Director of Markets Division, FSA May 2006 Speech
38SABRE II Requirements Process Information in Motion! Market events, news events, multiple sources of events, multiple exchangesEasy To Use, User Friendly SystemGraphical display of complex data relationships (web and thick-client)Tools that are easy to use and easy to configureIntuitive Alert Programming LanguageWell developed mature programming languagePowerful multi-exchange and multi-instrument type language featuresTruly integrated alerting across all combinations of integrated dataSatisfy MiFID RequirementsMiFID data feed acquisitionMiFID policy calculations & reportsMiFID inter-regulatory reporting
39Usage Model at FSA Integrated RAD environment Market conduct teams and “Super users”Deep historical analysisCreate and evolve alert rulesDevelop dashboardsIT and data analystsDevelop market connectivityMonitor alertsMonitor companiesAlert analysts & company monitoringManage alertsFully integrated, rapid development environment
40The SABRE II Architecture A next-generation, world-class market surveillance and market abuse detection platformMarket conduct teams, alert analysts, etc.Business Intelligence Historicalabuse analysisHistorical abuse patternsApamaLSEAlert rule InstancesLSE Adaptervirt-x AdapterAbuse detection scenarios
41Example: Fraud Detection The UK Financial Services Authority