Test, Double-test, and Triple-test
For some time now, we have been working on improving a location tracking feature for one of the agencies we collaborate with. The QA process for location tracking is expectedly different from that of other features, simply because location tracking must be tested in a real environment and there is no way around it. It is best practice to assess the app in different conditions, to make sure that it works as expected and under all circumstances.
The conditions we test the app in, as a part of our mobile app development services, are determined by user feedback. We add a new test route whenever a new case is reported by users, one that we haven’t thought of before. As you can see, a detailed complaint from users helps a lot in improving the app.
How does testing differ for walking, driving, or the actual skiing?
 | While you’re walking, your speed is constant and slow, QA takes a lot of time and you can have less iterations. Just because the more you ‘test’, the harder it gets to keep up the pace, and after several iterations, you may want to take a seat. :) |
 | While driving your speed is changing from 0 to maximum possible within the selected track. The covered distance can be huge, assessment takes less time obviously, and more iterations can be made. |
  | While skiing or snowboarding, you have a real environment, rather than an emulation of real conditions. The skiing/snowboarding path may be changing, you can lose your ski or snowboard, put them back on again, and check if your app has skipped this path depending on the walking direction. |
Now, the two key terms you should understand about our app are a LIFT and a RUN.
A Lift is a part of the path that is skipped because the user at this point is lifted up the mountain. This part of the road is not included and is not shown in the general statistics.
A Run is a part of the path that our tracker app measures, because the user at this point is running down the mountain. This part of the road is shown in the general statistics.
Paths
To test Location Tracking, we selected five routes that demonstrate 5 different scenarios that we tested in real conditions. We tested this simultaneously on a number of high-end devices and under various weather conditions.
The first three routes were tested on the streets of Lviv while driving. We specifically chose areas with altitude differences, and saturation of buildings and trees, which typically affects the accuracy of GPS user location tracking. The other two routes were tested at the local ski resorts (Bukovel & Trostjan).
This article is made up of five parts, each part revealing a specific test situation. For clarity, you can view our graphic trips with Google Earth, recorded with MyTrack application. There are also detailed graphs, which are additionally explained in tables.
Our device set is formed with iPhone and Nexus 5. Learn how we test our devices for GPS accuracy at `GPS Accuracy. Tools & Tips to Test Your Device`
Story Of Path 1
We started our testing challenge from route 1, Kn. Olha street.
This path is a low-traffic, straight section of the road and it gave us a chance to testing tracking speed. The maximum speed was 42.8 mhp.
First of all we need to set up the devices before the test. When we are reaching a start point of our path I am launching location tracking and location change recording.
During the test it is nice to control the moment of recording a run, to control how the app displays all metrics and at the same time compare with the MyTrack application.
From Google Maps our path will look like on the following image.
Lifts-&-runs scheme of what is expected to be tracked within path 1.
The graph below shows the emulation of one run and a lift. Blue trace shows the downhills, and gray stands for uphills.
Location Change Chart
The graph shows two lines: red (altitude) - shows change in altitude depending on the distance, blue (speed) - shows change in speed depending on the distance. The lift on the graph is labelled as a skipped path, i.e. the distance we skipped because the user at this point is lifted up the mountain.
Path Summary
| Length of runs (mi) | 0.37 |
| Length of lifts (mi) | 0.35 |
| Total Distance (mi) | 0.72 |
| Number of lifts | 1 |
| Number of runs | 1 |
| Altitude change (ft) | 78 |
| Top speed (mhp) | 42.8 |
Story Of Path 2
We tested the second path around the St. George's Cathedral. In this part we will present below two charts of testing when driving versus walking.
This path is so special for testing because this part of the road is surrounded by buildings, making the GPS signal deteriorate due to being reflected off the buildings. This was a very cool way to test under conditions of weak GPS signal.
The graph below shows the emulation of two runs and one lift.
Lifts-&-runs scheme of what was expected to be tracked within path 2.
Location change chart while driving
The graph shows two lines: red (altitude) - shows change in altitude depending on the distance, blue (speed) - shows a change in speed depending on the distance.
Location tracking summary table for driving
| Length of runs (mi) | 1.11 |
| Length of lifts (mi) | 0.37 |
| Total Distance (mi) | 1.48 |
| Number of lifts | 1 |
| Number of runs | 2 |
| Altitude change (ft) | 101 |
| Top speed (mhp) | 28.9 |
Location change chart while walking
The graph shows two lines: red (altitude) - shows change in altitude depending on the distance, blue (speed) - shows a change in speed depending on the distance.
a
Location tracking summary table for walking
| Length of runs (mi) | 0.78 |
| Length of lifts (mi) | 0.34 |
| Total Distance (mi) | 1.12 |
| Number of lifts | 1 |
| Number of runs | 1 |
| Altitude change (ft) | 114 |
| Top speed (mhp) | 5.13 |
Story of Path 3
The third path we tested around the Stryisky Park in Lviv. Originally, a part of this path was included into the Lviv Grand Prix Track.
This section of the route is the longest among what we selected for testing in urban environments. It can also boast the greatest change in altitude in the urban environment compared to previous paths
The below graph shows emulation of two runs and a lift.
Lifts-&-runs scheme of what is expected to be tracked within path 3.
Location change chart while driving
The graph shows two lines: red (altitude) - shows change in altitude depending on the distance, blue (speed) - shows a change in speed depending on the distance.
Location tracking summary table for driving
| Length of runs (mi) | 2.15 |
| Length of lifts (mi) | 0.6 |
| Total Distance (mi) | 2.75 |
| Number of lifts | 1 |
| Number of runs | 2 |
| Altitude change (ft) | 183 |
| Top speed (mhp) | 31.8 |
Story Of Path 4
This now gets even more interesting. The third route we tested in real-life conditions of a ski resort, Bukovel in the Carpathians.
Testing on this path requires additional skills of skiing or snowboarding.
The value of testing Location Tracking in real conditions cannot be denied, because the influence of the level of GPS signal is minimal: it’s all about the weather conditions. We carried out multiple runs and lifts.
Below is a map showing all our uphills and downhills. To emulate three runs and two lifts, we have selected the part of the resort highlighted in yellow and closed up.
Lifts-&-runs scheme of what is expected to be tracked within path 4.
Location changes chart while skiing & snowboarding
The graph shows two lines: red (altitude) - shows change in altitude depending on the distance, blue (speed) - shows a change in speed depending on the distance.
Location tracking summary table for skiing & snowboarding
| Length of runs (mi) | 5.4 |
| Length of lifts (mi) | 3 |
| Total Distance (mi) | 8.4 |
| Number of lifts | 2 |
| Number of runs | 3 |
| Altitude change (ft) | 1476 |
| Top speed (mhp) | 35.9 |
Story Of Path 5
Like the previous route, this one was at a ski resort also. We tested the technology on Trostjan ski resort in the Carpathians. This is Vasyl’s favorite ski resort because it has no infrastructure at all.
Just like the previous route, this one gives us good testing ground because the influence of the level of GPS signal is minimal, it’s only the weather conditions that we have to worry about. Just to be sure, we tested again and again :)
A video of our testing - snowboarding session can be viewed here: TrailTap_QA.
Below is a map which shows all our uphills and downhills. The graph below shows emulation of two runs and three lifts.
Lifts-&-runs scheme of what is expected to be tracked within path 5
Location change chart while skiing & snowboarding
The graph shows two lines: red (altitude) - shows change in altitude depending on the distance, blue (speed) - shows a change in speed depending on the distance.
Location tracking summary table for skiing & snowboarding
| Length of runs (mi) | 3.87 |
| Length of lifts (mi) | 2.71 |
| Total Distance (mi) | 6.58 |
| Number of lifts | 3 |
| Number of runs | 2 |
| Altitude change (ft) | 1481 |
| Top speed (mhp) | 29 |
Summary
All in all, our testing environments were quite diverse: the city streets versus the mountains, different rates of altitude change, uphill and downhill speed variation, and of course, weather conditions. This helped us ensure that our app is as accurate as possible in tracking a location in different environments and under various weather conditions and that the app calculates stats for each position a user is in at any given time, whether a lift or a run.
The graphs were accurate in rendering the test scenarios every time we repeated them to double-check. It is easy to tell apart lifts and runs. For extra explanation, we provided tables summarising each scenario's stats.
Once more, this testing session has proved that: testing the app on a set of different but typical routes helps to properly assess the accuracy of the app algorithms, particularly route tracking algorithms. When in doubt, double-check. Then compare and contrast the test results. This will help you understand if the app location-related functions have improved or if the dev team has really screwed up this update, so to speak.
SEE ALSO:
- How To Determine Location If You have A Roof Over Your Head
- 3 Powerful Stories From Tech Team Behind Rockstar Startups
Authors
 | Anver QA Engineer in Lemberg Solutions Limited
|  | Vasyl QA Lead in Lemberg Solutions Limited
|