{"id":40120,"date":"2022-03-31T19:37:03","date_gmt":"2022-03-31T17:37:03","guid":{"rendered":"https:\/\/intellias.com\/?p=40120"},"modified":"2024-04-26T12:25:11","modified_gmt":"2024-04-26T10:25:11","slug":"automotive-navigation-software-development-adopting-openstreetmap-data","status":"publish","type":"post","link":"https:\/\/intellias.com\/osm-data-integration\/","title":{"rendered":"Automotive Navigation Software Development: Adopting OpenStreetMap Data"},"content":{"rendered":"

Business challenge<\/h2>\n

As a leading provider of connected car<\/a> and location solution development services, our client is constantly looking to improve their technological solutions. To cut production costs and avoid the restrictions of commercially provided data, the company decided to adopt the OpenStreetMap (OSM) geographic database to start producing maps in-house. This makes our client independent from navigation platform providers and gives them full control over the cadence and quality of the maps delivered for their navigation solutions.<\/p>\n

\"Automotive<\/p>\n

OSM provides a free, editable, and constantly updated database. Considering that the volume of received data is quite high and that location solution development requires timely releases, data processing demands a big data platform and solid expertise in data mapping solutions. Having proven to be an expert in providing data mapping services<\/a>, Intellias was invited to contribute to our client\u2019s shift from the existing system to the free open-source data provided by OSM. The team employed a big data process based on the Spark framework for cleansing data, integrating third-party data, enriching existing data, and providing a proper structure for obtained data.<\/p>\n

OpenStreetMap integration<\/h2>\n

OpenStreetMap has a special tool that allows reviewers to contribute to the database. OSM maps are provided as vector data, but maps get updated only after they get verification from recognized contributors. The project consumes updated data daily, and that data becomes available in the form of a portable screenshot. Then this data can be parsed, converted into Parquet, and uploaded to Hadoop, after which Spark processes it steadily. Unlike data obtained from other platforms, OSM data is not structured and needs to be adapted to fit the existing database.<\/p>\n

The main challenge of the project was structuring the OSM data. Previously, the project used source processed data in Relational Database Format (RDF), given as Oracle-related tables. OSM data is different, and it had to be properly converted before starting the end-to-end data processing.<\/p>\n

Apart from that, OSM data is populated by people, and its model is not restrictive. It allows people to add various attributes without any restrictions. This led to another crucial project task \u2014 a formalization of collected data, making it compatible with the source data previously provided to the project. The Intellias team found the best solution for processing OpenStreetMap integration data within Spark and Java.<\/p>\n

The solution contained:<\/p>\n