Spyrosoft’s first project: GeoPlace

GeoPlace headquarters are based in London, and the company processes and delivers GeoLocation data. Our main task within this project was to develop and share an API that would be suitable for managing data related to streets and addresses in the United Kingdom.

Main functionalities and additional goals

The system’s main functionalities were centred around downloading certain localisation files from various sources (including email inboxes and FTP), adjusting these files, so they would fit certain formats and then sending them to databases where they would be validated. As a next step, the data can be made available for the client’s third-party partners using endpoints. This is how the information is currently being used by Ordnance Survey in the UK.

In addition to this main goal of creating an accessible and reliable API, our testers were also tasked with training the client’s tester team in Agile and guiding them through the set of best practices for what type of issues to look for, how to grow as QA specialists and how to create a QA plan.

The Spyrosoft testers have also prepared a testing framework using the Cucumber Testing Tool that is written in Java as well as test reports that are generated automatically when starting a new task in Jenkins, an open-source automation server.

One of the most extensive challenges in this project when it comes to testing was the size and the complexity of the procedures that were stored in the databases and the complex SQLs for testing the data. Considering these limits, our testers decided to use an additional tool for creating the database unit tests to validate the data more effectively. The uniqueness of this solution lays in the fact that the data was received from several sources at once and it had to be processed asynchronously.

How was project Aurora created?

The team at GeoPlace needed a more effective way to manage their GeoLocation data and decided to hire an external development team that would be able to create a reliable API for third-party suppliers so they could have an always-up-to-date database of streets and addresses. As the first step, our developers created a plan for the complete solution and its MVP and then added new functionalities to it.

GeoPlace team was entirely responsible for verifying the data added to the database. They were also working with experienced database specialists and were managing the business logic of the product, which was directly tied with the databases. It was particularly important for this project to consolidate various sources of data which were also delivered using different formats, including XML and CSV. Before we started setting up the system, these files had had to be manually managed by the GeoPlace team.

The solution

One of the first improvements that our team implemented was introducing Agile and Scrum workflow to the rest of the project teams. As a result, the team met during short catchups every day (daily scrums) and reviews (retrospectives) after each sprint (a 2-week period where all team members worked on a new iteration of the product). To make it possible, we collaborated very closely with the GeoPlace team showing them how this methodology works in practice and conducted Agile training throughout our initial 4 months working from their headquarters in London. We were also the first outsourced additions to the project team, meaning that our developers were all on board with arranging new processes and adjusting the workflow, so it would fit this new setup.

Our team was also involved in product development, planning for additional features and the technologies to be used in the process. The implementation was handled by our team, excluding the databases. There were two Systems Architects responsible for the project – one from the GeoPlace team and one from Spyrosoft.

The client had their own deployment and infrastructure environment. Our engineers offered to introduce ClayOne Kafka for performance monitoring and processing large files. We also implemented the Apache Camel framework – with the sole purpose of integrating the system with different data sources. On our end, there were also solutions written in Java and receiving the data before passing it on to the database.

One of our QA Specialists was also conducting training sessions for other testers in the project where he talked about working in Agile, how testers could work using this methodology, what to look for, how to create a testing plan and what traits should a great tester have. He also prepared a testing framework using Cucumber Testing Tool, written in Java, and a testing report generated automatically when starting a new task in Jenkins, an open-source automation server.

One of the biggest challenges in this project was the size of the procedures stored in the database and advanced SQL that were testing these data. Here’s where our developers introduced a tool for creating unit tests of the database and for verifying the data more effectively.

The uniqueness of the solution lays in how well it handles files from different sources and how it can work asynchronously. As for the tests, testing 2G files in a format that’s specific for GeoLocation projects.

Our team collaborated closely and built trust and a good working relationship right away, so they genuinely felt like they were part of GeoPlace.

Next steps

Thanks to the implemented system, the team at GeoPlace was able to limit the number of hours spent manually downloading the data, verifying it and uploading it to the database. It not only helped them save time and money but also provided them with the bandwidth to focus on other core tasks. They are now able to offer more services related to the data to their end customers.

We started collaborating back in 2016 and Aurora was the first ever project for Spyrosoft. It was launched in January 2021.