Developer Testing Tricks Brian Takita, Pivotal Labs

Abstract This presentation will go over a number of testing technologies and methodologies that will increase the odds of success for Rails projects. The goals of the talk is to, present a set of testing related situations and experience in solving a number of issues, raise the audiences' awareness over effective ways of communicating through tests, and emphasize skills and courage to solve testing issues.

Reasons to Test Executable specifications TDD – Make your design better Verification - Keep your code safe Support experimentation and refactoring Empirical software development

Focus of Tests Method Manual Automated Audience Customer Developer Scope Unit Functional Integration

Granularity of Tests Happy Path Testing Edge Case Testing

Happy Path Testing Tests the piece of functionality in a "normal" situation Quick and dirty way to see if your functionality is working Useful in integration and functional tests

Happy Path Selenium Example describe "A User on the home page" do before do open "/" end describe "clicking the up vote" do describe "when logged in" do @user = users(:quentin) element(“link=Login”).click element("name=login").type(@user.login) element("name=password").type('test') element("name=commit").click it "increments the vote count by 1" do element("css=.score").assert_contains('0') element("css=.up").click element("css=.score").assert_contains('1')

Edge Case Testing Tests the edge cases of the piece of functionality Invalid Data Invalid fixture state Ensures full code coverage Useful in Unit tests

Edge Case Testing VoteSubmissions::UpVoteSubmissionsController POST create when not logged in redirects to SessionsController#new when logged in and passed invalid data does not create a Vote responds with an error and passed valid data and the User does not have a Vote for the PainPoint creates a Vote responds with json representation of the User data of the PainPoint and the User has a Vote for the PainPoint does not create a new Vote sends an up_vote event to the existing Vote

Lifecycle of a Test TDD Tests to drive the design of the software Regression Testing Tests to ensure that your software still works when you make changes Retirement of a Test Tests that are no longer useful

TDD Red -> Green -> Refactor Write the tests first, then implement the software These tests drive design Tests should be refactored You can manually test drive your code, its just slower and you dont get to keep the tests

Goals of TDD Specification, not validation Feedback Developer makes smaller steps Developer knows when the code is finished Developer has an example of using the implementation code Developer maintains focus on the objective Fast Iterations Developer rhythm Confidence Simplicity YAGNI (You Aint Gonna Need It) Make a regression test

TDD – Test your tests Red -> Green -> Refactor Its easy to make a test that does not test anything For example, testing the elements within an empty collection Use Preconditions to set the context of your test

Retroactive TDD Comment out the implementation you want to test Write the tests Watch them fail for expected reasons Uncomment the implementation Watch the tests pass Refactor

Regression Test After the TDD phase, your tests serve to make sure your software still works Tests live longer than the code (the implementation often changes)

Regression Test Goals Verify your software does not break due to changes in implementation or state Courage Support Experimentation & Refactoring Documentation Clarity Fail in the right places Avoid crying wolf due to unnecessary brittleness Keep tests predictable

Regression Test - Documentation Tests should clearly document your system Refactor the test if it is not clear or focused Nested ExampleGroups are very helpful in showing contextual logic Unit tests should match the logical layout of your software

Regression Test – Clarity and Focus User sends messages to friends and receives replies from them ↲ when the friend responds back otherwise no reply is sent User #send_message sends a Message to a friend #receive_message receives a Message from a friend

Monitoring your tests Are you getting good coverage? Are these tests still relevant? How often do these tests fail? How long does your suite take to run? Are your tests easy to run or do they get in the way?

TDD & Regression Testing Their goals are not necessarily aligned Things change What is relevant today is not tomorrow Refactor early and often

Retirement of a Test If the test no longer contributes to logical coverage in its particular scope (unit, functional, integration, etc), then retire it.

Obsolete Tests They add clutter and make your test suite harder to read and reason about They take time to run They take space

Testing Untested Software Working Effectively with Legacy Code - Michael Feathers Seams Places where you can alter behavior without editing the code Dont forget to Retroactively TDD test your tests Rails MVC separation makes it easier to add tests later

Consequences of Refactoring Extract class or module refactoring can be supported by existing tests You may or may not need to test drive the extracted module It depends on your situation If you are unsure, error on the side of writing tests

Why Test Drive your refactorings You can get defect localization The tests help the design of your extracted module

Why Not Test Drive your refactorings TDD may disrupt your "refactoring flow" You may want your refactoring to be a spike to quickly experiment on an idea You can always retroactively TDD your changes You already have test coverage

When finished with the refactoring Refactor your tests Move the tests into the correct places Use abstraction in your tests by verifying that the dependencies are utilized

Test Fixture Fixed state to be used as a baseline for your tests Mock & Stubs Instantiated Fixtures Transactional Fixtures Fixture Scenario Object Mother

Test Drive Fixtures Verify your fixtures are in a valid state Test drive a change to your DB schema by making a fixture test Verifies that your fixtures reflect reality Keeps your fixtures maintainable Discourages your fixture set from getting large describe "users.yml" do attr_reader :user describe "bob" do before do @user = users(:bob) end specify "was in the green and red room" do user.rooms.should include(rooms(:green)) user.rooms.should include(rooms(:red))

Fixture Scenarios http://code.google.com/p/fixture- scenarios/ describe User do describe “enter” do describe “when all of the rooms are full” do scenario :all_rooms_are_full it “raises a NoRoomAvailableError” do user = users(:bob) lambda {user.enter}.should raise_error(NoRoomAvailableError) end

One off cases You do not necessarily need to make a new fixture