- \n
- Optimized datasets for machine learning<\/li>\n
- Data compression methods<\/li>\n
- Use of AI for sensor fusion and data compression<\/li>\n
- Multisensor data fusion for AI-based self-driving cars<\/li>\n
- An in-depth learning approach to data compression<\/li>\n
- How compressed data is sent to the cloud<\/li>\n<\/ul>\n
<\/p>\n
Creating optimized datasets for machine learning: how it works<\/h2>\n
After raw data from multiple sensors is gathered, it must be adequately processed. Keep in mind, the higher the autonomy level, the more sensors are needed. Processing consists of several stages:<\/p>\n
- \n
- data cleaning: finding incomplete or inaccurate input and correcting mistakes in the raw sensor data by applying various rule- or model-based<\/a> techniques<\/li>\n
- data fusion: combining data from various sources, as well as the information from related databases, to ensure high-level accuracy<\/li>\n
- data compression: reducing both storage space and the amount of transferred data by reducing redundancy, i.e., dropping duplicated or invaluable data; using different data representation and approximation techniques, i.e., transferring fewer data without losses and transferring compact models instead of raw data<\/li>\n<\/ul>\n
It can be quite tricky to determine exactly all required and sufficient data that an autonomous car needs at each specific moment. Specific prerequisites help a vehicle\u2019s AI work with sensors, gradually learning what data to use and when. However, prerequisites cannot be updated in real-time. It\u2019s essential for the machine learning engine to recognize the required data for mission-critical actions and analyze it locally.<\/b> Therefore, to run data processing, AI must:<\/p>\n
- \n
- identify data in all formats (from all sensors)<\/li>\n
- implement sensor fusion algorithms for all-encompassing analysis<\/li>\n
- recognize data for mission-critical operations and analyze it locally<\/li>\n
- compress non-critical data<\/li>\n
- schedule the uploading of compressed data to the cloud when less critical communications are available<\/li>\n
- call for legacy data from the cloud when the analysis of non-critical data is needed<\/li>\n<\/ul>\n
To run the processing like this, autonomous cars have to possess powerful and, therefore, expensive machine learning engines. But to enter the mass market, automakers must look at ways to optimize data and reduce the vehicle price.<\/b> To reach this goal, advanced data compression, and data fusion<\/a> techniques are required, as well as efficient two-way communication between vehicle and cloud backend.<\/p>\n
Using AI for sensor fusion and data compression<\/h2>\n
Since an AI-based autonomous car has multiple sensors – cameras, sonars, LIDAR, etc. – techniques for both fusing and compressing the arrays of connected car data<\/a> gathered must be adopted. It\u2019s a vicious cycle: for an autonomous car to function seamlessly, it needs tons of input, which requires hefty computational processing and more processors, together with more storage inside the car. It adds cost, weight, and complexity to the vehicle\u2019s AI system. How can this be dealt with?<\/p>\n
After fusing the data from several IoT devices, you\u2019ll receive a large amount of information pushed forward into the system for AI to then analyze. To deal with this amount of data, various compression techniques are used. With the help of these techniques, information is encoded and undergoes compression, then decoded and uncompressed for use. There are so-called lossless compression and lossy compression approaches:<\/b> in the first case, you get back all the information originally held while in the second case, some data is lost.<\/p>\n
![\"The](\"https:\/\/intellias.com\/wp-content\/uploads\/2020\/04\/lossy-data-info-1.jpg\")
<\/p>\nThe data must be compressed at the fusion center to preserve the communication bandwidth and processing capability.<\/p>\n
Explaining multisensor data fusion for AI-based self-driving cars<\/h2>\n
Let\u2019s revisit sensor fusion and its importance.Sensor fusion presupposes merging data from various sources to develop an accurate and comprehensive perception.<\/b> Sensor fusion is critical for a vehicle\u2019s AI to make intelligent and accurate decisions.<\/p>\n
Sensor fusion in an autonomous vehicle<\/b><\/p>\n
![\"The](\"https:\/\/intellias.com\/wp-content\/uploads\/2020\/04\/lossy-data-info-2.jpg\")
\nSource: Towards Data Science<\/a><\/em><\/p>\nMultisensor data fusion can be both homogeneous – data coming from similar sensors, and heterogeneous – data combined from different kinds of sensors based on its time of arrival. There are also different levels on which multisensor data fusion can be performed:<\/p>\n
- \n
- signal level:<\/b> on this level, the fusion creates a new signal of better quality<\/li>\n
- object level:<\/b> object-based fusion generates a fused image where each signal data type is determined with the help of clustering<\/li>\n
- feature level:<\/b> fusion on this level requires the recognition of objects from different data sources<\/li>\n
- decision level:<\/b> fusion on this level requires the combination of results from multiple algorithms<\/li>\n<\/ul>\n
At the moment, automakers are using both the feature and decision level multisensor data fusion. However, this is not enough to reach a higher level of autonomy. For an extended sense of environment and better contextual input, signal-based fusion techniques must be adopted.<\/p>\n
This fuels a second challenge: the more complicated processing AI has to fulfill, the more power it requires. A self-driving car needs more processors and memory onboard, which results in added cost and bigger energy consumption. What is even more critical, fusing and interpreting data from so many different sensors will take more time, and the AI reaction cannot be altered on the road. That\u2019s why data compression is no less vital for autonomous vehicles than fusion.<\/p>\n
Investigating data compression methods<\/h2>\n
Sensors differ in the types and volume of data they generate. Looking at the estimates given by Stephen Heinrich from Lucid Motors, the difference can be quite drastic:
\n![\"The](\"https:\/\/intellias.com\/wp-content\/uploads\/2020\/04\/lossy-data-info-3.jpg\")
\nWith cameras consuming the biggest part of the data exchange channel, it is the first candidate for compression. Video and LIDAR data compression can be lossless and lossy, as mentioned before. Let\u2019s elaborate a bit.<\/p>\nHow Intellias employs a deep learning approach to data compression<\/h3>\n
Lossless compression methods solve two key problems:<\/p>\n
- \n
- Approximating the true data distribution to the model.<\/li>\n
- Developing a practical compression algorithm called an entropy coding scheme.<\/li>\n<\/ol>\n
We can approximate<\/a> the expected efficiency of lossless video coding from the compression ratio of 100 images from ImageNet:
\n![\"The](\"https:\/\/intellias.com\/wp-content\/uploads\/2020\/04\/lossy-data-info-4.jpg\")
\nAs for LIDAR, data compression approaches are:<\/p>\n- \n
- ilLASCompression<\/li>\n
- LASZip<\/li>\n
- LZMA or Lempel-Ziv-Markov chain algorithm<\/li>\n
- Compressing POINT10<\/li>\n<\/ul>\n
To improve data compression, we implement deep learning techniques. The recent BB-ANS method utilizes latent variable models. This model defines unobserved, random variables used to represent the distribution of original data. For example, in the case of images, pixel distribution may be dependent on the location of edges and textures, which are the latent variables.<\/p>\n
As for lossy compression approaches, unsupervised learning methods are implemented for image modeling. We use Variational Autoencoders (VAEs), PixelCNN, and PixelRNN models to learn about latent image representation. In this case, a smaller encoded vector is exchanged and then decoded.<\/p>\n
To separate valuable and quickly changing information from less valuable and more static, a combination of lossy and lossless approaches is implemented. When we want to optimize data, we use a lossless approach to transfer and compress larger amounts of highly dynamic and critical information. Less precise but more compact data transfers will be applied to images that are describing static surroundings or non-critical backgrounds.<\/p>\n
Backgrounds, in turn, open possibilities for applying deep learning models for object detection and tracking. In this case, objects of interest are transferred in higher quality and rate, while everything else remains in lower quality.<\/p>\n
Other deep learning methods for sensor data compression include attention models, which are used to reduce data size and point out the most valuable information, and Golomb-Rice encoding, a specific data compression method based on entropy, volatility\/persistence, diversity\/uniformity, initial data size, etc. These methods are considered lossy compression and generally refer to multidimensional scaling. They are efficient for numeric multidimensional data.<\/p>\n
While even simple approaches to data compression can provide significant bandwidth savings, advanced ML-augmented dimensionality reduction<\/a> is even more promising. With this reduction, the size of transferred data between connected cars is cut in half. Also, when coded or compressed information is sent to the cloud, the transfer data size is also reduced up to two times.<\/p>\n
Sending compressed data to the cloud<\/h2>\n
Non-critical data, i.e., long reaction time reference data, which is not processed locally, is compressed and sent to the cloud. The cloud platform used for data storage must be capable of hosting the arrays of generated data. Computing costs are rarely mentioned when it comes to autonomous vehicles, but they should be taken into account:<\/b> GM, for instance, spent<\/a> $288 million for two new warehouses to store and process car data.<\/p>\n
This is where products like Google\u2019s Cloud Data Fusion solution come in handy. With this solution\u2019s help, data from different sources can be fused into the central data warehouse. Fusion cloud allows building data pipelines and transforming them without writing any code.<\/p>\n
Challenges Cloud Data Fusion Solves<\/b><\/p>\n
- object level:<\/b> object-based fusion generates a fused image where each signal data type is determined with the help of clustering<\/li>\n
- signal level:<\/b> on this level, the fusion creates a new signal of better quality<\/li>\n
- data cleaning: finding incomplete or inaccurate input and correcting mistakes in the raw sensor data by applying various rule- or model-based<\/a> techniques<\/li>\n
![\"The](\"https:\/\/intellias.com\/wp-content\/uploads\/2020\/04\/lossy-data-info-5.jpg\")