The UK's research and development department has today unveiled its futuristic vision of 'quantitative safety assurance' for AI.
The Advanced Research and Inventions Agency (ARIA) compares this guarantee to the high safety standards in nuclear power and passenger aviation. In the case of machine learning, standards include probabilistic guarantees that a particular action will not cause harm.
At the heart of ARIA's plan is “gatekeeper” AI. This digital sentinel is Other AI agents operate only within the guardrails set for a specific application.
ARIA will commit £59m to the scheme. By the end of the program, the agency plans to demonstrate a scalable proof of concept in one domain. Suggestions include: Power grid balancing and supply chain management.
TNW Conference 2024 – All scale-ups invited to attend June 20-21
Promote your scale-up in front of investors, innovators, and potential customers with our carefully selected scale-up packages.
If effective, this project could protect high-stakes AI applications, such as improving critical infrastructure and optimizing clinical trials.
The program is the brainchild of David 'davidad' Dalrymple, who co-invented the popular cryptocurrency Filecoin.
Dalrymple has also studied the safety of technical AI extensively, which sparked his interest in the gatekeeper approach. As his director of ARIA's program, he was able to put his theory into practice.
Gatekeeper Guarantee
ARIA gatekeepers Utilizes scientific world models and mathematical proofs. Dalrymple said the concept combines commercial and academic concepts.
“The approaches being considered by major AI companies rely on finite samples and do not provide any guarantees about how the AI system will behave when deployed,” he told TNW in an email.
“On the other hand, placing too much emphasis on academic approaches such as formal logic runs the risk of effectively trying to build AI capabilities from scratch.
“The gatekeeper approach gives us the best of both worlds by tuning frontier features as engines for speeding along the rails of mathematical reasoning.”
This fusion requires deep interdisciplinary knowledge Collaboration — This is where ARIA comes into play.
British DARPA?
Founded last year, ARIA funds “high-risk, high-return” research. This strategy has drawn comparisons to DARPA. The Department of Defense's “mad science” unit.
Dalrymple drew another parallel with DARPA. He compares his ARIA new project as follows: DARPA's HACMS program. We created an unhackable quadcopter. This project proved that formal verification can produce bug-free software.
“It is possible to eliminate vulnerabilities, but it requires assumptions about the scope and speed of intervention an attacker can make to the physical embodiment of a system,” Dalrymple said.
His plan is based on the approach advocated by. Yoshua Bengio is a renowned computer scientist. Turing Award winner Bengio also called out: “Quantitative safety assurance. ” But he is disappointed with the progress so far.
“Unlike the way we build bridges, drugs, and nuclear power plants, current approaches to training frontier AI systems (the most capable AI systems that currently exist) do not provide any kind of quantitative safety assurance. ” Bengio wrote in a blog post last year. .
Dalrymple has a chance to change that. This will also be a big boost for ARIA, which has been under intense scrutiny from politicians.
Some lawmakers have questioned ARIA's budget. This organization raised him 800 million pounds in five years. While this is a significant amount of money, it is only a small fraction of other organizations. Government research institute.
ARIA can also point out potential future savings. One of the programs the company launched last month aims to train AI systems at 0.1% of current costs.
One of the themes of this year's TNW conference is “Ren-AI-ssance: Regeneration with AI.” If you want to learn more about all things artificial intelligence, or just want to experience the event (and say hello to our editorial team), we've got something special for our loyal readers. Use code TNWXMEDIA at checkout for 30% off. business pass, investor pass or the startup package (bootstrap & Expanding).