{"id":6968,"date":"2018-02-13T17:40:29","date_gmt":"2018-02-13T16:40:29","guid":{"rendered":"https:\/\/www.intellias.com\/?p=6968"},"modified":"2024-07-29T12:23:39","modified_gmt":"2024-07-29T10:23:39","slug":"computer-vision-keep-sharp-eye-road","status":"publish","type":"blog","link":"https:\/\/intellias.com\/computer-vision-keep-sharp-eye-road\/","title":{"rendered":"Computer Vision for Autonomous Driving: Keep an Eye on the Road"},"content":{"rendered":"

Elon Musk continues to insist that self-driving technology is possible without supportive infrastructure and additional sensors like LIDAR<\/a>. Just as human drivers primarily trust a passive optical sensing system (we call them eyes) when driving, so too can cars rely on computer vision for autonomous driving\u00a0to analyze the road and drive on their own.<\/p>\n

\n\t\t\t
\n\t\t\t\t
\n\t\t\t\t<\/linearGradient><\/defs><\/svg>\n\t\t\t\t\t

We have to solve passive optical image recognition extremely well in order to be able to drive in any environment and in any conditions. At the point where you\u2019ve solved it really well, what is the point in having active optical, which means LIDAR. In my view, it\u2019s a crutch that will drive companies towards a hard corner that\u2019s hard to get out of.<\/p>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\tElon Musk,<\/span> CEO at Tesla Inc.<\/span><\/span>\n\t\t\t\t<\/small>\n\t\t\t<\/blockquote>\n\t\t<\/section>\n

Still, a lot of autonomous car manufacturers are developing their solutions with the use of various sensors fusion. Here you see the comparison of preferred approaches in autonomous technology development from different car makers’ standpoints.<\/p>\n

\"Computer<\/p>\n

By believing in the computer vision car, Tesla\u2019s CEO is one of only a few people who bravely promise to cross the US from coast to coast by the end of 2018 using autopilot<\/a> alone. All Musk needs for that is stereo sensors to collect data, radars, and machine learning to train neural networks how to act.<\/p>\n

\n\t\t\t
\n\t\t\t\t
\n\t\t\t\t<\/linearGradient><\/defs><\/svg>\n\t\t\t\t\t

I am pleased with the progress made on the neural net. It\u2019s kind of like it goes from \u2018doesn\u2019t seem like too much progress\u2019 to wow.<\/p>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\tElon Musk,<\/span> CEO at Tesla Inc.<\/span><\/span>\n\t\t\t\t<\/small>\n\t\t\t<\/blockquote>\n\t\t<\/section>\n

Musk\u2019s promise is ambitious and raises a lot of questions \u2013 especially considering the recent Tesla autopilot incident<\/a> that drew public attention with the screaming headline \u201cOne of the most advanced driving systems on the planet doesn\u2019t see a freaking fire truck, dead ahead.\u201d<\/em><\/p>\n

Let\u2019s figure out why Elon Musk is an adamant believer in computer vision for self-driving\u00a0cars and their ability to drive the road only by seeing it just like a human driver sees.<\/p>\n

\n\t\t\t\n\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
CONNECTED DRIVING WHITEPAPER<\/div>\n\t\t\t\t\t
How Tier 1s can thrive during technology disruption<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t Download now <\/span>\n\t\t\t<\/div>\n\t\t<\/a><\/div>\n

How does\u00a0computer vision in self driving cars work to see the world?<\/h2>\n

\"Computer<\/p>\n

First, let\u2019s define terms. Computer vision is the science of machines, robots, computer systems, and artificial intelligence analyzing images, recognizing objects, and acting accordingly. Computer vision in self driving cars\u00a0relies on images from sensors, image processing, and deep learning to turn data into appropriate actions.<\/p>\n

\n\t\t\t
\n\t\t\t\t
\n\t\t\t\t<\/linearGradient><\/defs><\/svg>\n\t\t\t\t\t

Computer vision is the science of machines, robots, computer systems, and artificial intelligence analyzing images, recognizing objects, and acting accordingly.<\/p>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t <\/span>\n\t\t\t\t<\/small>\n\t\t\t<\/blockquote>\n\t\t<\/section>\n

As for computer vision for autonomous driving<\/a>, stereo sensors continuously collect images of the changing environment while driving on the road to build a picture that\u2019s understandable for a car. But the way a car perceives visible data is quite different from the way a human perceives it. Artificial intelligence systems don\u2019t recognize objects as people do. A person says, \u201chere\u2019s a tree, there\u2019s a cat, and those are people crossing the road on the red light without any apparent reason.\u201d<\/p>\n

Automotive computer vision in cars doesn\u2019t have such insights per se. A computer needs to assign specific features to objects to recognize them and understand what\u2019s happening or what will happen in the next moment. This process of recognition includes semantic segmentation, creation of a detailed 3D map, and object detection in it. After an AI system has recognized objects on the detailed map, deep learning teaches car how to behave in a particular situation based on what it has detected. Further action is a path planning based on a virtual picture with recognized objects and assigned reactions on them.<\/p>\n

\n
\n

Learn more about the main principles of advanced path planning for self-driving cars<\/p>\n

\n
<\/div>\n <\/div>\n <\/div>\n Read more<\/span>\n\t\t <\/a><\/div>\n

Semantic segmentation to assign features to objects<\/h3>\n

You\u2019ve probably seen a ton of pictures reminiscent of the image seen through a night-vision device, looking through which you could delightfully say, \u201cWow! I see the world as a machine.\u201d Here you go with one more.<\/p>\n

\"Computer<\/p>\n

What we see here is called semantic segmentation<\/strong> or color classification for a\u00a0computer vision car. This is the ability to label pixels with certain classes of objects and later assign them particular features such as the ability to move, a potential trajectory, or predictable behavioral patterns. The higher the density of pixels that an AI can label, the better its perception of the environment. Let\u2019s look at the historical progress of pictures with labeled pixels that machines have been seeing over time.<\/p>\n

\n\t\t\t
\n\t\t\t\t
\n\t\t\t\t<\/linearGradient><\/defs><\/svg>\n\t\t\t\t\t

Semantic segmentation is the ability to label pixels with certain classes of objects and later assign them particular features such as the ability to move, a potential trajectory, or predictable behavioral patterns.<\/p>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t <\/span>\n\t\t\t\t<\/small>\n\t\t\t<\/blockquote>\n\t\t<\/section>\n

1990s: 300 MHz \/ 80 ms \/ 1% density<\/strong><\/p>\n

\"Computer<\/p>\n

2000s: 1 GHz \/ 80 ms \/ 20% density<\/strong><\/p>\n

\"Computer<\/p>\n

2010s: 3 GHz \/ 2ms \/ >90% density<\/strong><\/p>\n

\"Computer<\/p>\n

Based on data from stereo sensors, AI can pull out necessary semantics and build a\u00a0<\/span><\/span>3D<\/span><\/span>\u00a0<\/span><\/span>map<\/span><\/span> of the surroundings with great accuracy and robustness. If an AI can recognize what\u2019s in an image at this level of detail, then we have greater confidence in the ability of an autonomous car to correctly identify objects in this virtual map and further react on them faster with ADAS\u00a0functions<\/a><\/span><\/span>. With more accurate object recognition, computer vision for autonomous vehicles becomes much more reliable and driving itself safer, as cars can act precisely as they have been trained to a particular object or situation.<\/p>\n

\n\t\t\t
\n\t\t\t\t
\n\t\t\t\t<\/linearGradient><\/defs><\/svg>\n\t\t\t\t\t

Based on data from stereo sensors, AI can pull out necessary semantics and build a 3D image of the surroundings with great accuracy and robustness.<\/p>\n\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t <\/span>\n\t\t\t\t<\/small>\n\t\t\t<\/blockquote>\n\t\t<\/section>\n

\n
\n

Learn how we helped equip German premium cars with the most advanced driver assistance functions on the market<\/p>\n

\n
<\/div>\n <\/div>\n <\/div>\n Read more<\/span>\n\t\t <\/a><\/div>\n

SLAM\u00a0algorithms\u00a0to build a 3D picture of the unknown environment<\/span><\/span><\/h3>\n

Obtaining real-time images from sensors, a car still needs to build a virtual 3D map from them for an understanding of where it is acting now. For that, every autonomous robotic system relies on Simultaneous Localization and Mapping algorithms, or simply SLAM.<\/p>\n

Originally, SLAM\u00a0algorithms\u00a0have\u00a0been used to\u00a0control\u00a0robots, then\u00a0broadened for a\u00a0computer vision-based online 3D modeling, augmented reality-based computer vision applications<\/a>, and finally, self-driving cars. SLAM algorithms can process data from different types of sensors such as:<\/p>\n