AI in Self-Driving Cars

Autonomous/self-driving vehicles are gaining popularity and are on the verge of becoming a reality. The development of self-driving cars began in the early 20th century, but significant industrial progress only started in the 2010s, with companies like Google and General Motors(GM) heavily investing in the technology¹. These vehicles aim to transform transportation by enhancing decision-making, optimizing fuel consumption, and promoting road safety.

Autonomous vehicles rely on various technologies, including radar, sensors, and artificial intelligence (AI), to navigate roads without human intervention. AI plays a crucial role in enabling self-driving cars to make real-time decisions and adapt to changing road conditions.

While autonomous vehicles promise to reshape the future of transportation, their success depends on overcoming significant technical, regulatory, and societal hurdles. This article explores the main applications of AI in self-driving vehicles, highlighting the benefits and challenges associated with this transformative technology.

Current State and Future Outlook

The last decade has witnessed a surge in the development of self-driving technology, propelled by breakthroughs in artificial intelligence (AI), machine learning (ML), and deep learning (DL). These cutting-edge technologies are empowering engineers to address complex challenges surrounding scene perception, motion control, path planning, and behavior arbitration in autonomous vehicles.

The Society of Automotive Engineers (SAE) has established a framework that defines six levels of driving automation, ranging from level 0 to level 5. As the levels progress, the need for human intervention diminishes².

A Level 1 vehicle has a single feature, such as cruise control, to manage its speed.

At level 2, assistive technologies like adaptive cruise control, lane departure warning, and lane-keeping systems employ AI.

Level 3 relies heavily on AI for navigation and decision systems, allowing vehicle systems more control. It also features a suite of sensors and devices for data collection. Only a handful of companies in Europe and North America, publicly offer this level of autonomy with authorisation to drive³.

Level 4 systems can do almost everything Level 5 systems can do, though they’re limited by geographic and weather conditions, where human drivers need to take over⁴.

Level 5 representing fully autonomous vehicles that require no human input or oversight⁵.

Leading autonomous vehicles(AV) companies

Developments in the self-driving landscape are quickly advancing, with giant automakers, AI companies, and startups all vying for a piece of the pie. All these companies are promising new ways for consumers to experience mobility.

Here are some of the top companies leading in developing self-driving cars.

Zoox: An Amazon-operated company, Zoox’s AVs utilize computer vision to analyze their surroundings. The company’s advanced features can detect pedestrians using their phones and interpret light flashes from other cars⁶.

Waymo: A subsidiary of Alphabet (Google’s parent company), Waymo develops autonomous cars equipped with a suite of sensors and AI-powered software. The company’s vision system and machine learning algorithms enable multiple capabilities, allowing the vehicle to respond safely to real-world traffic situations⁷.

Tesla: Tesla has developed an autopilot neural network that incorporates diverse real-world vehicle simulations. The company’s AI solution can detect objects, estimate depth, and perform semantic segmentation⁸.

Pony.ai: Pony.ai builds autonomous vehicle capabilities for robotrucks, robotaxis, and personal vehicles. The company utilizes AI for various functionalities, including localization, map features, scene perception, and prediction⁹.

Cruise: Owned by General Motors, Cruise employs complex machine learning models in its AI engine to help AVs develop dynamic understanding and decision-making capabilities. The company also uses continuous learning frameworks to enhance safety and reliability¹⁰.

Other notable companies that are leading the way in AI for autonomous vehicles include Baidu, Mercedes-Benz, and Motional.

Key AI technologies in autonomous driving

Here are some of the ways AI is contributing to making vehicles autonomous.

Perception: Lidar, radar, cameras, GPS, and various sensors continuously collect different forms of data. Insights from these data sources help the vehicle understand its surroundings for safe operations. Computer vision, which involves motion detection and calibration, enables the vehicles to detect objects.

DL algorithms, like convolutional neural networks (CNN) and recurrent neural networks (RNN), are applied to the 3D data collected to help the AV understand its scene¹¹. With this information, the vehicle can plan maneuvers, perceive the environment, and make decisions.

Lane detection and changing: Changing lanes is one of the most complex problems for AVs due to the unpredictable nature of road users. Machine vision is a primary component for lane detection capabilities using segmentation techniques to determine lines and curves.

Multiple neural network algorithms, such as line-CNN, spatial-CNN, deep-CNN, and 3D CNN, can work out the temporal and spatial information provided to accurately identify lanes on the road. For instance, Tesla uses a neural network called Hydranet for lane detection and lane-changing capabilities¹².

Prediction: Self-driving vehicles need to project how identified road agents(pedestrians, cyclists, and vehicles) may act. This is another challenging task considering how dynamic each agent is and how their interaction causes different results. DL algorithms like CNN and RNN have been used to process data and forecast trajectories.

Nvidia has trained a predictionNet deep neural network(DNN) that anticipates and predicts the trajectories of road agents¹³. The prediction model output is a series of trajectories/events scored with their probability of happening.

AI in Robotaxis

Riding in a robotaxi has become increasingly popular in cities like San Francisco and Beijing. Although most robotaxis are still in the testing phase, residents in Wuhan and Beijing are already using them for public transport after authorities granted permits to Baidu’s Apollo Go. The tech company aims to deploy 1,000 robotaxis in Wuhan by the end of 2024¹⁴.

Robotaxis utilize cameras and sensors, enabling AI to reconstruct a 360-degree view of the vehicle’s surroundings. Additionally, the AI-based computing engine within the vehicle can plan the journey, avoid collisions, predict various scenarios, and navigate through city streets.

Many companies have developed their own neural network engines to support intelligence and autonomy in robotaxis. Others have relied on third-party AI-driven platforms provided by companies like Nvidia. These powerful AI computing engines are designed to convert data received from sensors and cameras into actionable insights.

Nvidia offers its partners end-to-end solutions for AV development. The company recently unveiled a new centralized computing platform capable of running generative AI applications. By integrating this platform with natural language processing (NLP), robotaxis will be able to have customized personas that engage in dialogue, provide real-time recommendations, and offer assistance to passengers¹⁵.

AI in autonomous mass public transport systems

AI has helped solve some complex problems in mass transport, such as traffic management, route optimization, and optimized scheduling. Researchers are stretching AI’s capabilities for use in autonomous public transport vehicles. Though still in its infancy, this technology promises to shuttle people within specific city zones, such as campuses.

Robobuses are a prime example of mass transport vehicles using AI for different systems. Armed with a suite of sensors and cameras, onboard AI systems are tasked with functions like navigation, precise docking, energy, and traffic management. Integrating V2X capabilities in robobuses will allow optimized fleet management and better response to their environment.

Companies like WeRide and Pix Moving have made significant progress in producing autonomous mini shuttles. Apart from open road testing in designated zones in Beijing, WeRide is operating in Resorts World Sentosa, Singapore. The robobus operates along a 1.2km stretch, carrying and dropping passengers along the way¹⁶.

Benefits of AI in Autonomous Vehicles

Improved road safety and reduced accidents: Human errors, such as fatigue, sleep, and distraction, are the leading causes of most accidents.

AI-powered self-driving vehicles can significantly reduce accidents and incidents resulting from human errors. With adaptive capabilities, visual capabilities, and proximity sensing analysis, AI allows AVs to operate safely and avert dangerous scenarios.

Increased mobility: AVs can change how the disabled, elderly, and people with medical conditions travel. Using speech recognition or mobile apps, the vehicle can safely navigate these groups to their destination. Moreover, ridesharing robotaxis can provide an affordable and convenient alternative to traditional modes of transport.

Reduced traffic congestion and environmental impact: AVs can detect and avoid traffic using dynamic routing capabilities, route planning, and maneuvering. AI in AVs can manage energy by optimizing braking and acceleration, limiting fuel consumption. This eco-friendly strategy reduces greenhouse emissions.

Improved traffic management: Self-driving cars using V2X frameworks can communicate with other road agents to understand traffic conditions. This information can help in route planning and avoiding congested areas.

Challenges

Technical limitations and edge cases: Some of the technical complexities facing the advancement of AVs include limitations in onboard computing power, perception accuracy, localization challenges, and hardware. Simulating rare edge cases that can occur in the real world is also challenging.

Regulatory hurdles and liability concerns: There is a lack of clarity on who is liable if an AV gets into an accident. Additionally, it is challenging to adapt existing laws to work for self-driving vehicles. To mitigate this, stakeholders must develop a comprehensive framework to manage liability. By following standardized frameworks, lawmakers can establish laws prioritizing human safety and encouraging innovation.

Consumer trust and acceptance: Self-driving cars come under sharp scrutiny every time they are involved in accidents. For instance, Cruise robotaxis faced criticism in San Francisco following two separate accidents¹⁷. Such incidents only continue to build mistrust in the community with safety concerns.

Companies must prioritize user safety in design to boost consumer acceptance and perform vigorous testing.

Ethical dilemmas: AI systems in AVs may face morally challenging situations, forcing them to choose between ambiguous situations. For instance, choosing between protecting onboard passengers or pedestrians in case of an accident. Stakeholders must develop acceptable frameworks and standards to tackle some of the more pressing moral dilemmas.

Conclusion

Thanks to breakthroughs in AI, autonomous vehicle technology has seen significant advancements. The sector has widely adopted DL algorithms, revolutionizing the perception, planning, and execution framework. Consequently, AI has enabled various applications in self-driving vehicles, ranging from lane recognition to environmental awareness.

These developments have paved the way for innovative projects like autonomous taxis and robobuses. As we further leverage AI’s potential, companies must maintain a human-centered approach, prioritizing enhanced mobility and safety in their technological implementations.