Testing Wall @ Smoca

• Development

To complete my educa­tion as a soft­ware deve­loper it was required for me to complete a soft­ware deve­lo­p­ment related project over a period of 10 days. My task was to expand the Conti­nuous Inte­gra­tion System, already exis­ting at Smoca, to run tests on real devices. Testing on real devices has many advan­tages over testing in a simu­lator. There are many problems that can occur on a real device but are these are often diffi­cult to simu­late. For example a bad internet connec­tion can drama­ti­cally impact the user expe­ri­ence in a nega­tive way. Bugs could also occur depen­ding on diffe­rent hard­ware. This is a problem espe­cially for Android Apps. Finally diffe­rent versions of opera­ting system might lead to diffe­rent beha­viour within your apps. All these problems can only be reduced to a minimum by testing on the most commonly used phones.

How to: Run Android/iOS tests on multiple devices with Gitlab

To complete my educa­tion as a soft­ware deve­loper it was required to realise a soft­ware deve­lo­p­ment related project over a period of 10 days. My task was to expand the Conti­nuous Inte­gra­tion System already exis­ting at Smoca.

As a version control system Smoca uses Git and all the projects are hosted on a local Gitlab server. For Conti­nuous Inte­gra­tion Gitlab's built-​in tools are used. The goal was to create a system that runs UI and unit tests on multiple devices (not simu­la­tors) whenever a new version is pushed to the server. This should work for iOS and Android projects. In this blog post I will share how I achieved this goal so that anyone can recreate such a system.

Testing on real mobile devices

Testing on real devices has many advan­tages over testing in a simu­lator. There are many problems that can occur on a real device but are often diffi­cult to simu­late. For example a bad internet connec­tion can drama­ti­cally impact the user expe­ri­ence in a nega­tive way. Bugs could also occur depen­ding on diffe­rent hard­ware. This is a problem espe­cially for Android Apps. Finally, diffe­rent versions of opera­ting systems might lead to a diffe­rent beha­viour within an app. These problems can be reduced by testing on the most commonly used phones.

Testing system setup

Here is a quick summary of all the steps necessary to setup a similar testing system. This guide works speci­fi­cally for Gitlab though the concepts can be applied to other Conti­nuous Inte­gra­tion systems.

First of all, you will need a Gitlab-​Runner. That's a piece of soft­ware installed on a computer in your network. This will have to be Mac if you want to run iOS tests since the Apple build tools are only available for macOS. I would recom­mend a dedi­cated Mac mini. The guide lists the necessary soft­ware you will need to install on it. This mainly consists of all the soft­ware needed for buil­ding and testing iOS/Android Apps.

In addi­tion to the Gitlab-​Runner you will also need to prepare the project of the apps to be tested. In the root of the project at a gitlab-​ci.yml file. In this file “jobs” are defined. In this example one for testing. In the end the job executes a bash command.

Tech­ni­cally it would be possible to use the stan­dard build-​tools provided by Apple and Google. This can be a bit cumber­some though, espe­cially for iOS since the commands get very verbose quickly. Also code signing can be a huge pain. For these reasons I used a powerful tool named Fast­lane, which is a cross plat­form build tool that provides many useful func­tions such as auto­mated code-​signing, buil­ding, testing, releasing and many more.

Setup guide

I have written three guides that provide detailed infor­ma­tion about the setup of a Gitlab-​Runner and the setup of iOS and Android projects. These guides can be found here:

• Gitlab-​Runners Setup Guide: https://github.com/smoca-​ag/ci-​demo-app-readme

• iOS Project and Setup Guide: https://github.com/smoca-​ag/ci-​demo-app-ios

• Android Project and Setup Guide: https://github.com/smoca-​ag/ci-​demo-app-android

As a result of my indi­vi­dual prac­tical work, smoca now has the oppor­tu­nity to run auto­mized app tests on several mobile devices. I expect that my efforts will help smoca to reach an even higher quality of app deve­lo­p­ment.