Finally, in this article we will show how we were testing our approach with iBeacons. Interested in the results? Let’s check it out then. Short reminder - our task is to find a solution for identifying the current position of a horse rider on the arena and to keep track of the location changes in an accurate way.
It was exciting to receive two packs of estimote beacons and at the same time to dream how everything will be working perfectly.
So after we got the packs with iBeacons, we did the usual and standard procedures, like marking every ibeacon with its ID. This helps us to distinguish different ibeacons while doing the environment setup.
We set up the iBeacons with 2 apps: Estimote and Horse Analytics Dressage.
And here is how we are going to test location tracking accuracy:
Our QA team’s approach:
Use ‘Measure map lite’ application for a rough iBeacons setup
- Set start position of the device
- Observe the location on the device
- Use a laser distance meter HILTI PD-E 01 to check the distance
- Compare data
Generally, we performed 5 different tests:
- on the parking lot & in the room
- in the park
- in a room with obstacles (indoor location SDK)
- on the parking lot (indoor location SDK)
- at the stadium (indoor location SDK)
First test - on the parking lot and in the room
Testing on the parking lot showed very unstable results. Due to the external factors (people, cars) the signals received from iBeacons were impermanent and as a result we were uncertain about the distance from iBeacons to the mobile device. And this leads either to high inaccuracy of the results (the location point is jumping) or to the inability to find a valid point of intersection of the circles. As the circles do not overlap each other, it is impossible to see the position of the device. Testing was conducted with iBeacons placed at a distance of 5 meters from each other.
Testing in the room showed better results: the distance was determined more precisely, as a result the user’s position was defined better. The exact position could be determined if you were standing on the same place for around 15 seconds. The problems we faced during the testing on the parking lot remained, namely - unstable distance from iBeacons to the mobile device and the inability to find a valid point of intersection of the circles. However, the margin of error has dropped. Testing was conducted with iBeacons placed 3m apart.
Testing on an open area should give better results.
Conclusions:
- There is a need to improve the implemented algorithm for determining the position with mathematical methods (screening jumps, jump smoothing) and further testing.
- Additional research of indoor location from Estimote is required - On the open area the results should be fairly accurate, but this should be verified.
Second test - on the open space (parking area)
Before the testing was initiated, we had eliminated minor issues in the algorithm for determining the location and added a filter to screen out unnecessary points.
Testing was conducted on the parking lot with an area of 10m*20m, the distance between the iBeacons was 10m. Testing showed the signal’s strength we got ranges (even at a distance of 1m the fluctuations could be ± 2 dB). Strength fluctuations cause a significant error in the calculation and creates a situation when it is impossible to find a real point of intersection of the circles. We often acquired data, where the distance to the iBeacon was greater than the length of the field.
Smaller fluctuations of the signal would provide more stable result of the algorithm and more opportunities for smoothing the returned results. At the current moment the values that we acquire vary too greatly.
Third test - room with obstacles (indoor location sdk)
Testing sdk in the room shows more stable results, compared with the implemented algorithm. The user’s position doesn’t jump and when we get closer to the iBeacons, the values are displaying accurately. But almost nothing happens in the middle of the room and in motion, or the user’s position moves with a delay. Most likely it happens because the room is full of various objects.
The next step was to test indoor location sdk outdoors, on the open space.
Fourth test - Parking Lot (indoor location sdk)
Testing was conducted on the parking lot, without any interferences. At the testing site, we selected a square area. Each side of the square was 4m, and the iBeacons were located at the center of each side (as shown in the picture above). We received similar results, compared to the previous testing results. The user’s position almost didn’t jump and when you moved closer to the iBeacon, the values were displayed accurately. But a real-time movement does not get displayed, the changes start to appear only after you stop. By using the documentation, we can make a conclusion that the SDK solution was configured for iBeacons distance on a chest height.
We are planning to perform tests on larger areas.
After our discussion, the next thing we saw is a change of the algorithm, namely - to use iBeacons as an entry point and in further to determine the position using the accelerometer and compass.
Fifth test - on the stadium (indoor locations SDK)
Testing was conducted on an open area without any obstacles. iBeacons were situated on the ground, and placed on the corners of the square (4m*4m). The results were again unstable and were very similar to the results on the parking lot. We came to the conclusion, that it is required to have an Internet connection for indoor locations SDK.
Summary
As we see - our assumptions, unfortunately, are incorrect. That’s why we are continuing to explore and search for something that will give us better results. Stay tuned! Our fourth article will be published soon.
If you weren’t with us from the beginning of our journey - please look through our first 2 articles.
