Growing software from examples Seb Rose, Claysnow Limited Blog:

Heavily influenced by GOOS book:

They’re called different things The difference is that one is called Behaviour Driven Development – and some people find that wording useful – and one (or two) is called (Acceptance) Test Driven Development – and some people find that wording useful in a different way. And that’s it. Liz Keogh,

Test Driven Development Acceptance Test Driven Development Behaviour Driven Development Domain Driven Design Specification by Example Example Driven Development Customer Test

PatternGenerally good forGenerally bad for Given/When/Th en Tests that require a lot of setup OR easily forgotten setup Tests that have a non-obvious trigger Tests with few expected outputs Tests that have unimportant/simple/obvious preconditions Tests where there are multiple different inputs and multiple different outputs Tests where a single Given/When/Then only describes one of numerous very similar test scenarios Specification By Example - Conceptual or Concrete Tests that have numerous: Inputs that affect output behavior Outputs/expected behaviors Tests where it’s important to test a lot of different data scenarios Tests where the trigger event is somewhat obvious Any test where it seems like a table would be useful to: describe the test better, or help explore all of the possible inputs and outputs for a test. Simple tests Tests that are more about verifying simple UI behavior For instance – “Test that an error message is displayed when the user enters an incorrect password.” Test where there is really only one input or precondition mmary3.pdf

Outside-in

Understandable Maintainable Necessary Granular Reliable

Understandable

Maintainable

Necessary

Granular

Reliable

Declarative - makes a statement (.) Prefer declarative examples Exclamatory - shows excitement (!) Interrogative - asks a question (?) Imperative - makes a command (.)

Imperative vs Declarative Style Feature: Sign up Scenario: New user redirected to their own page Given I am not logged in And I visit the homepage And I follow "Sign up" And I fill in "Username" with "Seb" And I fill in "Password" with "password" And I fill in "Confirm password" with "password" When I press "Sign up" Then I should be on my feeds page And I should see "Hello, Seb"

Imperative vs Declarative Style Feature: Sign up Scenario: New user redirected to their own page When I sign up for a new account Then I should be taken to my feeds page And I should see a greeting message

Imperative vs Declarative Style Feature: The entire system This feature illustrates what can happen when you take the declarative style too far. Scenario: It works When I use the system Then it should work perfectly

Remember your audience Who should be able to read the examples? Keep things clear. What is their interest in them?

Don’t hide the context [Test]public void asterisk_should_format_to_em(){ String expected = "This is em text"; String actual = f.Format("This is *em* text"); Assert.AreEqual(expected, actual);}

Don’t hide the context Scenario: Increased delivery charges for zip code Given my delivery zip code is in Alaska When I go to the checkout Then I will have to pay the higher delivery charges

Don’t hide the void smoker_requires_manual_referral(){ Referral referral = underwriting.process(smoker); Assert.assertEquals(Referral.Manual, referral);}

Make data void smoker_requires_manual_referral(){ Customer customer = new Customer(“Joe”, “Smith”, “12/12/1980”, “Accountant”, “$300,000”, “Yes”, “No”); Referral referral = underwriting.process(customer); Assert.assertEquals(Referral.Manual, referral);}

Emphasise interesting void smoker_requires_manual_referral(){ CustomerBuilder builder = new CustomerBuilder(); Customer customer = builder.withSmokerFlag().build(); Referral referral = underwriting.process(customer); Assert.assertEquals(Referral.Manual, referral);}

Work collaboratively Create a ubiquitous language. Use examples to discover the domain. Decompose ruthlessly.

Online subscriptions In order to increase subscriptions visitors should be able to subscribe online with a VISA card User / Stakeholder Story

VISA subscriptions In order to increase subscriptions visitors should be able to subscribe online with a VISA card User / Stakeholder Story

VISA subscriptions In order to increase subscriptions visitors should be able to subscribe online with a VISA card Acceptance Criteria Must support VISA Does not need to support MasterCard, Switch... Customers should be prevented from entering invalid credit card details... Credit Card Processing Acceptance criteria:

Really? So tell me... "Customers should be prevented from entering invalid credit card details" What exactly makes someone's credit card details invalid? Can they use spaces? Should we checksum the digits? How do we feed back that the details are invalid?

Avoid workflow style Every journey is made from single steps. Workflows are more brittle. A few workflows go a long way.

aopwork11/TestingPyramid.jpg Exploratory and manual

Workflow style Scenario: Happy path for registration and purchase Given I am on the homepage And I register for an account in Alaska And I go to the most popular item page And I add the most popular item to my basket And I go to checkout And I select my delivery address And I give valid payment details When I confirm acceptance of terms and conditions Then my purchase will be confirmed

Workflow style Scenario: Correct postage price for Alaska Given I am on the homepage And I register for an account in “Alaska” And I go to the most popular item page And I add the most popular item to my basket And I go to checkout When I select delivery to my home address Then I have to pay the higher delivery charge

Focus on a single step Scenario: Correct postage price for Alaska Given I am on the checkout page When I select delivery to “Alaska” Then I have to pay the higher delivery charge

Consider costs and benefits Remove unnecessary examples Exercise the thinnest slice possible Automate when viable

Cost Risk Manual Automate

Don’t let example dictate mechanism Visibility depends on interest not layer. Insulate examples from technical details. Acceptance tests not always end-to-end.

Make technical tests visible Scenario: High Risk rates for Test Pilots Given an applicant with occupation “Test Pilot” When the underwriting engine is invoked Then we will use the “High Risk” rates table

Scenario: Applicant with high risk occupation Given a standard, single-life, male applicant But with a high risk occupation When the application is processed Then it will be referred for manual underwriting Is this end

Categorise in multiple dimensions Faster feedback is better. Other dimensions are also useful. Take advantage of partitioning.

Summary Example-based methods are very similar. Minor variations by target audience. Skills are transferable.

Nomenclature Check Example Specification Test

Seb Rose Blog:

Backup

Drive out traceability - - -

Verify dependencies - - -