Testing Wall @ Smoca
To complete my education as a software developer it was required for me to complete a software development related project over a period of 10 days. My task was to expand the Continuous Integration System, already existing at Smoca, to run tests on real devices. Testing on real devices has many advantages over testing in a simulator. There are many problems that can occur on a real device but are these are often difficult to simulate. For example a bad internet connection can dramatically impact the user experience in a negative way. Bugs could also occur depending on different hardware. This is a problem especially for Android Apps. Finally different versions of operating system might lead to different behaviour within your apps. All these problems can only be reduced to a minimum by testing on the most commonly used phones.
How to: Run Android/iOS tests on multiple devices with Gitlab
To complete my education as a software developer it was required to realise a software development related project over a period of 10 days. My task was to expand the Continuous Integration System already existing at Smoca.
As a version control system Smoca uses Git and all the projects are hosted on a local Gitlab server. For Continuous Integration Gitlab's built-in tools are used. The goal was to create a system that runs UI and unit tests on multiple devices (not simulators) whenever a new version is pushed to the server. This should work for iOS and Android projects. In this blog post I will share how I achieved this goal so that anyone can recreate such a system.
Testing on real mobile devices
Testing on real devices has many advantages over testing in a simulator. There are many problems that can occur on a real device but are often difficult to simulate. For example a bad internet connection can dramatically impact the user experience in a negative way. Bugs could also occur depending on different hardware. This is a problem especially for Android Apps. Finally, different versions of operating systems might lead to a different behaviour within an app. These problems can be reduced by testing on the most commonly used phones.
Testing system setup
Here is a quick summary of all the steps necessary to setup a similar testing system. This guide works specifically for Gitlab though the concepts can be applied to other Continuous Integration systems.
First of all, you will need a Gitlab-Runner. That's a piece of software installed on a computer in your network. This will have to be Mac if you want to run iOS tests since the Apple build tools are only available for macOS. I would recommend a dedicated Mac mini. The guide lists the necessary software you will need to install on it. This mainly consists of all the software needed for building and testing iOS/Android Apps.
In addition to the Gitlab-Runner you will also need to prepare the project of the apps to be tested. In the root of the project at a gitlab-ci.yml file. In this file “jobs” are defined. In this example one for testing. In the end the job executes a bash command.
Technically it would be possible to use the standard build-tools provided by Apple and Google. This can be a bit cumbersome though, especially for iOS since the commands get very verbose quickly. Also code signing can be a huge pain. For these reasons I used a powerful tool named Fastlane, which is a cross platform build tool that provides many useful functions such as automated code-signing, building, testing, releasing and many more.
Setup guide
I have written three guides that provide detailed information about the setup of a Gitlab-Runner and the setup of iOS and Android projects. These guides can be found here:
Gitlab-Runners Setup Guide: https://github.com/smoca-ag/ci-demo-app-readme
iOS Project and Setup Guide: https://github.com/smoca-ag/ci-demo-app-ios
Android Project and Setup Guide: https://github.com/smoca-ag/ci-demo-app-android
As a result of my individual practical work, Smoca now has the opportunity to run automized app tests on several mobile devices. I expect that my efforts will help Smoca to reach an even higher quality of app development.
