Researchers at Ulm University in Germany have recently developed a new framework that could help to make self-driving cars safer in urban and highly dynamic environments. It is designed to identify potential threats around the vehicle in real-time. This previous work was aimed at providing autonomous vehicles with situation-aware environment perception capabilities, thus making them more responsive in complex and dynamic unknown environments.
“The core idea behind our work is to allocate perception resources only to areas around an automated vehicle that are relevant in its current situation (e.g., its current driving task) instead of the naive 360° perception field,” Matti Henning, said. “In this way, computational resources can be saved to increase the efficiency of automated vehicles.”
When the perceptive field of automated vehicles is limited, their safety can decline considerably. For instance, if a vehicle only considers specific regions in its surroundings to be “relevant,” it might fail to detect potentially threatening objects in other regions. This could happen if the algorithms underpinning the vehicle’s functioning are programmed to only consider and process a specific area of the road.
“This is where our threat region identification approach comes into play: regions that might correspond to potential threats are marked as relevant in an early stage of the perception so that objects within these regions can be reliably perceived and assessed with their actual collision/threat risk,” Henning explained. “Consequently, our work aimed to design a method solely based on online information, i.e., without a-priori information, e.g., in the form of a map, to identify regions that potentially correspond to threats, so they can be forwarded as a requirement to be perceived.”
To be applied on a large scale, the researchers’ framework should be as lightweight as possible. In other words, it should not need extensive computational resources to continuously scan the environment for threats.
The method proposed by Henning and his colleagues is very straightforward, as it only needs to perform a limited number of computations. In addition, it is highly adaptable, thus it could be tailored for specific use-cases or vehicles.
Essentially, the framework captures model-free representations of the environment, which include velocity estimates for all moving objects in the vehicle’s surroundings. This means that, in contrast with other approaches, it does not rely on a limited, previously delineated map of relevant areas.
“Specifically, we leverage a Cartesian Dynamic Occupancy Grid Map (DOGMa), which provides a velocity estimate for each cell of the rasterized environment,” Henning said. “From this, we use a standard clustering algorithm to identify sufficiently large clusters of cells of similar velocity and then evaluate if, assuming a constant velocity for identified clusters, these clusters would intersect with the movement of the automated vehicle within a set prediction horizon.”
If the moving clusters of cells identified by the team’s clustering algorithm intersect with the vehicle’s motion, a possible collision with the corresponding object could occur. To avoid this, the team’s model marks the clusters’ position as a relevant region that should be processed, so that the vehicle can perceive objects within it and adapt its velocity or direction to avoid accidents.
The key difference between the framework created by Henning and his colleagues and other threat identification approaches introduced in the past is that it tries to identify threats as early as possible. Their approach first identifies regions that contain moving objects and then allocate computational resources to these regions, using a technique introduced in their previous work.
This allows the vehicle to detect where moving objects and potential threats are before they are in its immediate vicinity. Once these are identified, a threat assessment module would assess the risk of collisions with these objects and a planner would delineate actions to avoid these collisions. The team’s paper only focuses on the treat identification model, as the threat assessment system and planner are beyond the scope of their paper.
“Our work is to be seen in the context of regional allocation of resources to parts of the perception data instead of the full 360° field of view,” Henning said. “We outlined the (quite obvious) importance of retaining the capability of reacting to the environment without being restricted to a-priori knowledge. In this context, we have shown that already straightforward and lightweight implementations can significantly improve possible reaction time on potential collision threats.”
Henning and his colleagues evaluated their framework in a series of simulations and found that it could improve the operation of self-driving vehicles in different critical scenarios. These include scenarios in which another traffic participant approaches the vehicle’s lane in different ways.
“The implication that we derive is that safety is not necessarily tied to an all-time, 360° multimodal perception system,” Henning said. “Instead, safety can also be achieved by an efficient perception system that adapts in smart ways and based on context knowledge as well as online information (and possibly even other sources of information) to an automated agent’s situation.”
The new framework could eventually be implemented and tested in real-world settings, to enhance the safety of self-driving vehicles navigating dynamic environments. In the meantime, Henning and his colleagues plan to continue working on their approach, while also devising new models to enhance autonomous and semi-autonomous driving.
“In the future, we aim to follow the path to both efficient and safe perception using introduced methods for situation-awareness,” Henning added. “Early-stage threat region identification is only one of the components required for such a system, and several challenges are still to be handled.”
