As software development complexity is increasing, efficient and reliable testing is critical. Enter Maestro, a user-friendly development testing tool that streamlines testing processes, speeds up release cycles and enhances software quality.
This article explores our journey with Maestro, highlighting challenges, solutions, and benefits as we strive towards enhanced testing and continuous improvement.
Our main goal? To use testing time efficiently, moving repetitive tasks to automation, and freeing up engineering time for activities such as critical and manual testing of a new feature before they are automated or preparing automation for a new feature.
How do we do it?
We use Maestro to automate testing scenarios and flows in our mobile applications - this helps us bring on-demand test results faster, gives us a quick retest option, and saves time.
Even though initial setup and writing tests require time investment, while automation tests run, QA engineers can focus on tasks like manual testing of new features or preparing automation for new features.
Tests are written in YAML files in a semi-codeless manner, using commands like "tapOn" and "scroll". Tests can run on hardware devices or emulators/simulators, making them ideal for end-to-end smoke tests or regression tests during fresh releases.
Time Savings with Automation
Example Scenario:
A hypothetical mobile app with login, logout, dashboard, 2 main features, and a user profile.
Manual Testing:
- Complete smoke test: 1 hour per device
- Development period: 6 months (18 sprints, 4 releases per sprint)
- 3 devices (current and 2 older OS versions)
Approach 1: Manual Testing (First Month)
- 3 sprints, 4 releases/sprint, 1 hour/release, 3 devices/release
- Total: 36 hours/month on manual smoke testing
- Remaining: 134 hours for other testing activities
Approach 2: Automated Testing (First Month)
- Initial setup: 4 hours, fixing: 3 hours
- 3 sprints, 4 releases/sprint, 45 minutes/release, 3 devices/release
- Automated testing time utilized for other activities
- Total: 7 hours (setup + fixing)
- Remaining: 163 hours for other testing activities
- Time saved: 19 hours (29 hours without buffer)
Simulating Growth (Sixth Month, Coefficient: 1.2 per month)
Approach 1: Manual Testing
- Total: 269.2 hours/month on manual smoke testing
- Negative balance: -89.2 hours (not feasible)
Approach 2: Automated Testing
- Initial setup: 28.8 hours, fixing: 21.6 hours, writing: 28.8 hours
- Total: 194.4 hours (automated testing time utilized for other activities)
- Remaining: 119.6 hours for other testing activities
- Time saved: 208.8 hours
Summary:
- First month: 19 hours saved with Maestro
- Sixth month: 208.8 hours saved with Maestro
- Automation maintains 100% test coverage and allows QA engineers to focus on critical tasks, ensuring product quality.
Maestro represents a solution to the often complex issue of mobile application automation testing, offering organizations the tools and capabilities needed to elevate their testing practices to new heights and deliver exceptional software products that exceed customer expectations.
TL;DR
As applications grow more complex, automation helps maintain comprehensive test coverage. Automation frees QA engineers to focus on tasks requiring human insight, ensuring quality doesn't suffer even with tight deadlines.
