Presentation is loading. Please wait.

Presentation is loading. Please wait.

Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf Testing Salesforce at Cloud Scale Thomas Greg

Similar presentations


Presentation on theme: "Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf Testing Salesforce at Cloud Scale Thomas Greg"— Presentation transcript:

1 Jenkins User Conference San Francisco, Sept #jenkinsconf Testing Salesforce at Cloud Scale Thomas Greg

2 Jenkins User Conference San Francisco, Sept #jenkinsconf Agenda History of CI at Salesforce Adding elasticity with public clouds Reducing cycle time with Jenkins-CI Jenkins Everywhere

3 Jenkins User Conference San Francisco, Sept #jenkinsconf Continuous Testing at Salesforce Multi-million line code base Several major branches 200,000 tests o Mostly in-appserver functional tests ~3000 hours serialized runtime Autobuilds - home-grown system 2000 VM's Hundreds of changelists per day (Perforce)

4 Jenkins User Conference San Francisco, Sept #jenkinsconf Luna

5 Jenkins User Conference San Francisco, Sept #jenkinsconf Autobuild System Design luna db Assignment Runner p4 Autobuild Runner luna web

6 Jenkins User Conference San Francisco, Sept #jenkinsconf Test Failure Correlation - Changelists are batched together - Tests that failure are re-executed against the skipped changelists

7 Jenkins User Conference San Francisco, Sept #jenkinsconf VM Provisioning luna db Assignment Runner p4 Autobuild Runner luna web VM Pool Manager VMware API VMware infrastructure

8 Jenkins User Conference San Francisco, Sept #jenkinsconf Autobuilds in the Cloud luna db Assignment Runner p4 Autobuild Runner luna web EC2 Poolboy AWS API EC2

9 Jenkins User Conference San Francisco, Sept #jenkinsconf VM Provisioning Part II Static allocation Dynamic allocation Predictive allocation Elastic allocation

10 Jenkins User Conference San Francisco, Sept #jenkinsconf Selenium Spawns a 2nd VM running Windows RemoteWebDriver -> IE6,7,8,9; Firefox; Chrome; Safari Selenium Browser Compatibility suite 35,000 UI tests; some Selenium1 debt Tricky trifecta: o Chrome releases every 6 weeks o Firefox releases every 6 weeks; 3 week stagger o Selenium library updates Selenium committer on staff (Jim Evans)

11 Jenkins User Conference San Francisco, Sept #jenkinsconf Great coverage at great cost Multimillion dollar HA test infrastructure Cycle time is 6 hours per changelist [Automated] Continuous delivery not possible Some manual "sign off" at release boundary "Butterfly effect" changelists o which tests to run before pushing code?

12 Jenkins User Conference San Francisco, Sept #jenkinsconf Salesforce + Jenkins =

13 Jenkins User Conference San Francisco, Sept #jenkinsconf Jenkins Satellite Builds Automated installer for automation Shelve a changelist -> queue for CI run Run static analysis, coverage reports Faster cycle time, "release ready" mainline

14 Jenkins User Conference San Francisco, Sept #jenkinsconf

15 Jenkins User Conference San Francisco, Sept #jenkinsconf

16 Jenkins User Conference San Francisco, Sept #jenkinsconf Linking src/main and src/test

17 Jenkins User Conference San Francisco, Sept #jenkinsconf Linking src/main and src/test (2)

18 Jenkins User Conference San Francisco, Sept #jenkinsconf Future: Jenkins everywhere Code base is rapidly modularizing o Jenkins CI continuously delivering jars Jenkins as a Service o Option A: Cloudbees hosted o Option B: ad hoc "swarm" o Option C: Salesforce R&D hosted Jenkins for metrics:

19 Jenkins User Conference San Francisco, Sept #jenkinsconf We invented Testing at Scale

20 Jenkins User Conference San Francisco, Sept #jenkinsconf We're not close to done Smarter VM provisioning Build-anything platforms hard to test Pixel perfect UI test framework Headless browser testing (PhantomJS) In memory computing for tests (high cost) Massively parallel testing on Intel Atom HW o low cost, low power, natural isolation

21 Jenkins User Conference San Francisco, Sept #jenkinsconf Q&A on Twitter on Twitter

22 Jenkins User Conference San Francisco, Sept #jenkinsconf Thank You To Our Sponsors


Download ppt "Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf Testing Salesforce at Cloud Scale Thomas Greg"

Similar presentations


Ads by Google