Old domain, ground combat, network/cyber
WASHINGTON – Despite the Pentagon's big bets on big data and artificial intelligence, the Army's software chief says adversaries will “pollute” the well of data that AI engulfs and the U.S. won't be able to fight back in future conflicts. A new warning has been issued that the algorithms used could be subtly tampered with. .
“I don't think our data is contaminated.” now” Assistant Secretary of State Jennifer Swanson emphasized Wednesday at a Potomac Officers Club conference, “but when you're fighting a near-peer adversary, you need to know exactly what those threat vectors are.”
The fundamental problem is that all machine learning algorithms need to be trained on data, large amounts of data. The Department of Defense is making significant efforts to collect, collate, organize, and clean data so that it can be understood by analytical algorithms and infant AI. In particular, the preparation team must discard all erroneous data points before the algorithm learns what it did wrong.
Commercial chatbots, from Microsoft Tay in 2016 to ChatGPT in 2023, are notorious for sucking in misinformation and racism along with all the other internet content they consume. But even worse, Swanson argued, is that the military's own training data could be intentionally targeted by adversaries, a technique known as “data poisoning.”
Related: Department of Defense tested generative AI for creating supply plans in latest GIDE 9 wargame
“Any commercial LLM [Large Language Model] Outsiders learning from the Internet are being poisoned today,'' Swanson said frankly. “[But] To be honest, I'm more worried about so-called “normal” AI. Because these are the algorithms that are actually used by soldiers to make decisions on the battlefield. ”
She argued that building better chatbots is not a big problem for the Pentagon. “I think [generative AI] It’s fixable,” she said. “What really matters is the data.” Instead of training LLMs on the open internet, as OpenAI et al. I would train LLM on set. Specifically, it recommended DoD Impact Level 5 or 6 systems suitable for confidential data (5) or confidential data (6).
“Hopefully by this summer, IL-5 LLM capabilities will be available,” she said. This is useful for all kinds of back-office functions and can summarize large amounts of information to make bureaucratic processes more efficient, she said. “But our main concern is [is] These algorithms will inform decisions on the battlefield. ”
“In these use cases, the impact of bad data, bad algorithms, harmful data, Trojan horses, or all of the above is much greater,” Swanson said. “That’s really where we spend the majority of our time.”
CJADC2, AI testing, and defeating harmful data
Obtaining the right military-specific training data is especially important for the Department of Defense, which aims to use AI to coordinate future combat operations across land, air, sea, space, and cyberspace. The concept is called Joint All Domain Command and Control (CJADC2), and in February the Pentagon said a functioning “minimum viable capability” was already in place at select commands around the world. announced.
Future versions will connect to existing AI combat command projects at the service level, such as the Air Force's ABMS, the Navy's Project Overmatch, and the Army's Project Convergence, to add targeting data and attack plans.
Related: 'The worst day': DISA's future strategy prepares for wartime communications
Project Convergence, on the other hand, uses technology developed by the newly formed Project Linchpin. Swanson explained that the service's “flagship AI program” is designed to be a “reliable and secure ML operational pipeline for the program.”
In other words, the Army is building the “agile” feedback loop between development, cybersecurity, and current operations (DevSecOps) that leading software developers use to rapidly deploy new technology and keep it updated. I'm trying to apply it to my learning.
catch? “At this point, we're not 100 percent sure how we're going to make that happen,” Swanson said. In fact, she argued, no one would. We asked probably 100 different companies, “How do you do it?” And they're like, 'Hmm.' ”
Swanson is no Luddite technophobe. She is an old-school programmer and “in college she learned FORTRAN,” she said. She spent her 30 years working in government technology and in 2022 became the Army's first assistant secretary for software, engineering and data. She first bought a Tesla, and then she turned off the free trial of the self-driving feature because she didn't like the way it handled the tests she set up (she said she “failed” at merging) ). In fact, one of her big concerns about AI is how difficult it is to test.
“How do you actually test AI? AI is not deterministic; [other] software? ” asked Swanson. From the ancient FORTRAN she learned in school to modern favorites like Python, traditional code uses strict IF-THEN-ELSEs that always give the same output for a given input. Modern machine learning, by contrast, relies on neural networks loosely modeled on the human brain and uses complex statistical correlations and probabilities. Asking the same question over and over and getting a different answer each time is a dream come true for students who fake their essays, but a nightmare for reliability testers.
Additionally, machine learning algorithms keep It learns when exposed to new data, essentially reprogramming itself. This makes it much more adaptable than traditional code, which requires humans to manually make changes, but at the same time it can fixate on unintended details or obvious information that deviate significantly from the author's intent. It also means that there is a possibility.
“Learning is continuous, so how do you manage it on the battlefield?” [to] Try not to go completely out of control? ” asked Swanson. “How do I know my data is not contaminated?”
“There’s a lot of work going on right now,” she said. “My hope is that a lot of this research will lead to some answers in the next year or two so that we can lay a good foundation from the beginning.”