Tom Leggett is Lead Research Engineer for Automated Driving at Thatcham Research. This is the fourth blog in a series providing behind the scenes insights into his work developing a world-first consumer rating for Automated Driving Systems.
In my previous blog post, I described the key ingredients required for independent virtual testing, as well as the issues that come with them. The proposed solution is to provide carmakers with the list of scenarios we wish to test and allow them to execute the simulation themselves. This protects their intellectual property and means carmakers can use their own highly verified and validated models.
However, some big questions remain unanswered. Namely:
- How can we ensure that each manufacturer executes the same virtual tests?
- How can we ensure these tests are repeatable?
- How can we verify the manufacturer-provided results are representative?
Scenarios Description Language (SDL) helps to resolve these problems. SDL is a programming language that describes the parameters of the scenarios to be tested. This includes the environment, scenery, and dynamics within each test scenario. SDL was developed with the primary objective of being human-readable so that it can be understood by all users involved in the development of automated systems. Crucially it is also machine-readable, allowing it to be fed into testing toolchains and simulation platforms.
But didn’t you mention in the last blog that manufacturers use many different types of simulation platforms with custom pieces of software? How can SDL be used across all the different variations of simulation software?
Correct, I did raise that as an issue. Thankfully, SDL has an ace in the sleeve; it is simulator independent which means it can be readily translated into any simulator specific code. Warwick Manufacturing Group (WMG) has also developed a toolchain that automatically converts SDL into internationally recognised formats such as ASAM OpenScenario and OpenDrive, allowing it to integrate with already established formats.
Now we have a scenario format that manufacturers can work with; we must share the scenarios with them. The Safety Pool™ Scenario Database (SPSD) is an online library that allows free access to the 250,000 test scenarios already written in SDL. The SPSD has been created as part of a collaboration between WMG, University of Warwick, and Deepen AI. Safety Pool™ allows for the sharing of scenarios across test facilities, developers, legislators, and even manufacturers.
The biggest hurdle to overcome will be instilling consumer confidence in virtual testing. Here traditional physical testing has a major role to play in spot-checking the virtual testing results provided by the manufacturer, and by doing so strengthening trust in the ratings.
Select scenarios based on performance and significance will be chosen as the primary focus to understand the correlation of physical and virtual results. Further research is required to fully understand the best methods of such a comparison, but there is no doubt that physical and virtual testing must work hand-in-hand.
Only then will independent assessment programs like the ALKS Consumer Safety Confidence Framework become a vital tool in ensuring that drivers can use automated systems with confidence and that the full societal benefit of automated and autonomous technologies are realised.