Presentation is loading. Please wait.

Presentation is loading. Please wait.

Testing in Salesforce Going beyond 75% (or even 99%) statement coverage Laurent Poulain

Similar presentations


Presentation on theme: "Testing in Salesforce Going beyond 75% (or even 99%) statement coverage Laurent Poulain"— Presentation transcript:

1 Testing in Salesforce Going beyond 75% (or even 99%) statement coverage Laurent Poulain

2 © 2014 Progress Software Corporation. All rights reserved. 2  This is NOT an exhaustive presentation about testing  Great FREE online courses on testing Udacity: Software Testing edX: CS169.1x / CS169.2x: Engineering Software as a Service  Two developers, three opinions Disclaimer

3 © 2014 Progress Software Corporation. All rights reserved. 3  Because Salesforce requires it?  Because it’s a good practice  Useful for tedious manual tests  Useful after refactoring some code  “This code has been working fine for years. No need to retest it.” Yeah right!  Testing is a key component if you want to pass the Advanced Developer Certification Why testing?

4 © 2014 Progress Software Corporation. All rights reserved. 4  The Ariane 5 rocket launch (1996)  A software exception occurred in legacy code from Ariane 4  An integer conversion turned awry in the stabilization subsystem One of the most expensive bugs in history

5 © 2014 Progress Software Corporation. All rights reserved. 5  The Ariane 5 rocket launch (1996)  A software exception occurred in legacy code from Ariane 4  An integer conversion turned awry in the stabilization subsystem  Cost (rocket + satellite): $370-$500 million  A simple test of the function would have detected the bug One of the most expensive bugs in history

6 © 2014 Progress Software Corporation. All rights reserved. 6 Morale of the story: even stable code can break

7 © 2014 Progress Software Corporation. All rights reserved. 7 Morale of the story: even stable code can break  Changes other than data can affect your code and/or your test Triggers Validation rules Workflows New required fields …

8 © 2014 Progress Software Corporation. All rights reserved. 8 The various types of tests

9 © 2014 Progress Software Corporation. All rights reserved. 9  A test suite can ONLY prove that your code has a bug  You cannot run all the tests possible, so you have to make some choices Searching for a needle in a haystack

10 © 2014 Progress Software Corporation. All rights reserved. 10  Statement coverage (used by Salesforce) Code coverage

11 © 2014 Progress Software Corporation. All rights reserved. 11  Method coverage (S0): are all the methods called?  Method coverage (S1): are all the methods called from all the places where they can be called?  Branch coverage (C1): for each branch, are the tests covering both scenarios where the condition is met and is not met? Other types of code coverage

12 © 2014 Progress Software Corporation. All rights reserved. 12 if (condition) { // block 1... } // block 2... Code coverage example

13 © 2014 Progress Software Corporation. All rights reserved. 13 if (condition) { // block 1... } // block 2... Statement coverage if condition == true: 100%

14 © 2014 Progress Software Corporation. All rights reserved. 14 if (condition) { // block 1... } // block 2... Branch coverage if condition == true: 50%

15 © 2014 Progress Software Corporation. All rights reserved. 15 if (isAdmin) { // block 1... } // block 2... Branch coverage if condition == true: 50% (in theory)

16 © 2014 Progress Software Corporation. All rights reserved. 16 if (isAdmin || isManager || isSuperUser) { // block 1... } // block 2... Branch coverage+

17 © 2014 Progress Software Corporation. All rights reserved. 17  Path coverage (C2): are all the possible paths through the code executed?  Loop coverage: cover every loop with a scenario with 0, 1 and multiple iterations  MC/DC (modified condition/decision coverage): for each condition, test all possible boolean combinations Other types of code coverage (ctn’d)

18 © 2014 Progress Software Corporation. All rights reserved. 18 Are we done now ?

19 © 2014 Progress Software Corporation. All rights reserved. 19 try { if (isAdmin) { // block 1... } // block 2... // bug => crash for non-admin users } catch (Exception e) { // handles the exception } Code coverage example

20 © 2014 Progress Software Corporation. All rights reserved. 20  Use assertions in your test  Add assertions in your code to detect internal inconsistencies  Two approaches Black box testing: given an input, do we get the expected output? –What if you enter invalid data? White box testing: given an input, is the program behaving as expected? –On Salesforce, needs to add code for testing purpose only –Sometimes, looking at Limits.get () is enough –Too many white box tests it might be an indication you need to refactor your code Is the code doing the right thing?

21 © 2014 Progress Software Corporation. All rights reserved. 21  An oracle can tell whether the code is doing the right thing or not  Weak oracle: exception in code  Medium oracle: assertion in code  Strong oracle Compare results with an alternate implementation Function inverse pair (e.g. save and load) Null space transformation (feed two inputs that should be treated exactly the same) The need of oracles

22 © 2014 Progress Software Corporation. All rights reserved. 22 Are we done now ????

23 © 2014 Progress Software Corporation. All rights reserved. 23  Start with acceptance tests Decompose in unit tests  Integration testing: testing with other systems the app is interacting with Use StaticResourceCalloutMock  Random/fuzz testing: sending random data or random input You may not want to deploy such tests Still not finished…

24 © 2014 Progress Software Corporation. All rights reserved. 24  Cost/benefit tradeoff: is the testing worth the time?  Critical parts should be well-tested (e.g. triggers)  It is not unusual to have more lines of test than lines of code itself  SQLite 3.8 has ~84 thousand lines of C code How much testing?

25 © 2014 Progress Software Corporation. All rights reserved. 25 How much testing?  Cost/benefit tradeoff: is the testing worth the time?  Critical parts should be well-tested (e.g. triggers)  It is not unusual to have more lines of test than lines of code itself  SQLite 3.8 has ~84 thousand lines of C code  SQLite tests: 91 million lines of code (x1084 !)

26 © 2014 Progress Software Corporation. All rights reserved. 26  Use the default “seeAllData=false” whenever possible  Use a separate class for tests  Create separate factory classes that will generate your test data e.g. implement a factory method to create n Opportunities with m Contact per Opportunity  Bulkify your tests by feeding your code with 200 records  Use test.startTest() and test.stopTest() to get two sets of governor limits Salesforce Recommendations

27 © 2014 Progress Software Corporation. All rights reserved. 27 Check the documentation

28 © 2014 Progress Software Corporation. All rights reserved. 28 Methodology

29 © 2014 Progress Software Corporation. All rights reserved. 29 Testing is not always considered as a “finishing touch”

30 © 2014 Progress Software Corporation. All rights reserved. 30 Test-Driven Development (TDD) Behavior-Driven Design (BDD)  Both methodologies put writing tests first, coding second  Test early, test often  BDD is about the behavior of the application A black box form of TDD testing A good way to think of some tests: each feature should be tested  TDD can be very convenient when debugging code which is modifying data

31 © 2014 Progress Software Corporation. All rights reserved. 31 Test Examples

32 © 2014 Progress Software Corporation. All rights reserved. 32  A Scorecard written in VF+apex Contains 30+ case metrics Several filters, breakdowns and ways to display the data Some data is precomputed, some is computed on the fly  The challenges: Lots of possible user options increases risk of bugs Detecting bugs in numbers  Four types of tests Code coverage White box testing Black box testing Random testing -> Exhaustive testing Testing a complex VisualForce page

33 © 2014 Progress Software Corporation. All rights reserved. 33  Check that the right function is called and the right dynamic SOQL is generated  Doesn’t care about the results  Use a class called SupportTest that acts as a “debug log” that the test can read Stores data as a key / value pair Use only static functions for simplicity Available on https://github.com/lpoulain/DebugLoghttps://github.com/lpoulain/DebugLog White box test

34 © 2014 Progress Software Corporation. All rights reserved. 34 SupportTest.reset(); … SupportTest.addValue(’compute’, ’Precomputed (6 months)’); SupportTest.addValue(’SOQL’, soql_query); SupportTest.addValue(’error’, e.getMessage()+’ - ’+getStackTraceString()); … System.assert(SupportTest.valueEquals(’compute’, expectedValue)); System.assert(SupportTest.getNbValues(’compute’) == 1); error = SupportTest.getOldestValue(’error’); System.assert(error == null, error); soqlError = SupportTest.SOQLWhoseWHEREEquals(’SOQL’, SOQLFilters) System.assert(soqlError == ’’, soqlError); SupportTest class

35 © 2014 Progress Software Corporation. All rights reserved. 35  Runs some scenarios and compares the result with hard-coded data  The test class contains all the necessary test data: The raw data Scenarios (user selection + expected output)  A VisualForce page to display the test data Use the test as a form of documentation  When a new metric is added, the test is FIRST updated Black box test

36 © 2014 Progress Software Corporation. All rights reserved. 36 A VisualForce page that displays the raw data…

37 © 2014 Progress Software Corporation. All rights reserved. 37 … and the scenarios

38 © 2014 Progress Software Corporation. All rights reserved. 38  Use the drill down capability as an oracle Compared numbers from the Scorecard with numbers returned by the drilldown  Started as random tests  Decided to transform it to an exhaustive test It is now a batch job It is not an official test anymore I did find a bug this way Random test turned exhaustive test

39 © 2014 Progress Software Corporation. All rights reserved. 39 BDD on Salesforce

40 © 2014 Progress Software Corporation. All rights reserved. 40  A BDD tool inspired from Ruby tool Cucumber  Allows to write some tests in plain English (some apex required behind the scenes)  Available on Github (https://github.com/lpoulain/pickle)https://github.com/lpoulain/pickle  You can use Cucumber + Selenium, but it runs outside the Salesforce testing framework BDD on Salesforce: Pickle

41 © 2014 Progress Software Corporation. All rights reserved. 41 Pickle Test Example Given the following Users: u1|Joe Smith Given the following Accounts exist: Account Id|Account Name a2 |Foo Inc. Given the following Cases exist: Case Id|Owner Id|Account Id|Subject |Case Origin|Escalated c3 |u1 |a2 |This is a test|Web |true Given I am on page "My VF Page" (MyClass) with parameters foo=24 When I set "My Field" to "Random Value" and I click on "Update" Then "result" should contain "OK" and "numeric Field" should be >= "5" and "foo" should be equal to "24" and the query [SELECT Case Id, Subject FROM Case WHERE "Case Origin" = ’Web’] should return: c3|This is a test

42 © 2014 Progress Software Corporation. All rights reserved. 42 Web interface

43 © 2014 Progress Software Corporation. All rights reserved. 43 Questions?

44


Download ppt "Testing in Salesforce Going beyond 75% (or even 99%) statement coverage Laurent Poulain"

Similar presentations


Ads by Google