1. Ranorex Appium Calabash
UI Automation Tools
Ranorex
Appium
Calabash

2. Tool Evaluation

3. Ranorex - Advantages Cross platform support
Test Automation Library for .Net
Standard programming langauges C# and VB.Net for scripting
Test development environment (Ranorex Studio) features recorder, debugger, object spy and repository
Flexible Test Automation Library Interface Test suites with Ranorex results in .EXE files for simple integration with test management, behaviour driven development and continuous integration server tools
Customizable Test report engine.

4. How does it work? Gherkin Device/Simulator Android + Run Tests
4. Specflow reads bdd
(formatted in Gherkin)
Device/Simulator
3. Team city runs Visual Studio project
6. Test Scripts
runs on RT App.
5. Connects to device through WiFi
Android
Run Tests
(TeamCity Server) +
Visual Studio
RT App
(Our Build)
SpecFlow
Ranorex Library
iOS
2. .Exe is stored in repository
1.BDD Stories are turned into Visual studio projects
3333
Windows
BDD Stories
SVN

5. Ranorex-Disadvantages
Xamarin related automation Issues
Support takes more time for Xamarin related issues
Ranorex agent compatibility with Application is poor and causes crashing as new features are introduced in app
Instrumented application is slower than original Application
Some Touch gestures/Drag and drop operation not supported
In case of crashing/hanging, Automatic launching of closed Application is not supported
Need to provide long waiting times for object identification.

6. Appium - Advantages Active community No extra code into application
Open source – we can make our own fixes and ask for pull requests on github
Easier to learn and implement than Calabash
Element inspector in Appium app

7. Appium - Disadvantages
Over 50 open bugs with iOS
Unable to get hooks on certain elements in complicated apps(programme detail page).
Inconsistent with outcomes(due to no wait/delay)
Currently cannot drag – known logged bug

8. Appium - Demo

9. Calabash - Advantages Now owned by Xamarin Wait/sleep functionality
Can swipe
BDD Layer easily configurable

10. Calabash - Disadvantages
Not tested enough?
No updates in months – maybe due to takeover from xamarin so may become closed source.
Must be written in Ruby and cucumber - Can be tricky

11. Calabash - Demo

12. Conclusion Best tool to use is Calabash:
Owned and supported by Xamarin
More stable and reliable, testing is more consistent.
Easier to setup the BDD layer for QA’s

13. (formatted in Gherkin)
How does it work?
Gherkin
1. Team city runs Cucumber scripts.
2. Cucumber reads BDD
(formatted in Gherkin)
Android
Run Tests
(TeamCity Server)
Cucumber
Calabash
Xamarin
(Our Build)
iOS
3. From the parsed Gherkin files, Cucumber runs the retrieved Ruby files and methods. WP
4. Ruby talks to calabash to run commands on the platforms.
Ruby