The US Air Force says the military's new dogfighting AI fighter jet pilot is making 'dramatic' advances from one test to the next

• The US military has been moving forward on developing AI pilots in recent groundbreaking tests.
• The AI is making "dramatic" progress as it learns from flight and mission data, an Air Force officer said.

Last month, two US Air Force fighter jets battled in the skies above California in an intense dogfight. Air-to-air combat training isn't unusual, but what made this event strange is that one of the planes didn't have a human pilot at the stick.

The fighter, flown by an artificial intelligence agent with the Air Force secretary riding in the back, squared off in a dogfight with a manned F-16, demonstrating the rapid progress of the Air Force and Defense Advanced Research Project Agency's AI fighter pilot program.

An Air Force commander and the CEO behind the company that developed the AI, Shield AI, told Business Insider the moment was a monumental step in the program's development.

The Air Force spoke highly of it as well. Col. James Valpiani, Commandant of the Air Force Test Pilot School, told BI "Secretary [Frank] Kendall's flight marks an important milestone" for the program and speaks to growing "confidence in the state of combat autonomy."

Kendall hopped in the X-62A Variable In-flight Simulation Test Aircraft (VISTA), the Air Force's highly modified F-16, at Edwards Air Force Base on May 2 and flew for about an hour, "witnessing several AI agents as they autonomously maneuvered the X-62 against a manned F-16 in dogfighting engagements," Valpiani said.

Throughout the testing of this emerging technology, the Air Force has been impressed by how quickly the AI can learn new information and change based on hoards of data and flight experiences.

"This interactive process allows the team to rapidly mature AI agents, often generating dramatic performance improvements from one test flight to the next," Valpiani.

Across the Pentagon, defense officials and leadership are pursuing affordable, mass AI capabilities at a breakneck pace. It's been described as a major element of great power competition with China, an arms race of sorts to keep the US at the apex ahead of future conflict.

Kendall himself called his flight in May "a transformational moment," marking the transition from when autonomous air-to-air combat had only been "a distant dream" to becoming reality.

Shield AI CEO and co-founder Ryan Tseng told BI the US should undergo a Manhattan Project approach to getting the resources, personnel, and technology for mass-implementing AI, and the autonomous pilots are just the first step.

"What we see [from Kendall's flight] is excellent progress in the performance, the trustworthiness, and the safety of AI pilots," Tseng said. "There's a tremendous amount of activity and investment on our end toward the application of AI pilots to missions that are relevant to the security challenges that are facing us around the world."

He added that AI pilots answer DoD's call for intelligent, affordable mass AI, solving concerns about the number of pilots and training needed to get them flying, as well as remaining effective in a challenging modern battlespace where GPS and communications are potentially degraded or denied.

Tseng co-founded Shield AI "to bring the best of AI and autonomy technology to the DOD and our allies to protect service members and civilians," per the company's website.

Its work with the Air Force and DARPA on the AI flying the X-62A involves trial and error, massive amounts of data, simulated dogfights — which, of the ones operated by Shield AI, see AI win-rates of 99% — and real-world applications.

But throughout all of this, the AI pilot, named Hivemind, has been maturing quicker than expected. "I think things are going much faster than people probably appreciate," Tseng said.

Lt. Col. Ryan Hefron, the Air Combat Evolution program manager for DARPA, held a similar sentiment back in April, when the Air Force announced that AI had completed a landmark real-world dogfight against a manned aircraft in September 2023. That event was the first time such a test had occurred.

Although officials were mum on whether AI or the human pilot had won the dogfight, they were eager to share a progress report.

"Things are progressing as well or faster than we had hoped," Hefron said at the time.

Much of that may be attributable to how quickly AI agents can be adjusted based on new data or performance.

Training AI for flight begins with basic maneuvers and foundations, like a human pilot; then, new missions, skills, and capabilities enter its repertoire. Many changes to the program can be made within a few hours or overnight if the AI shows "unexpected behavior," Valpiani said.

"These changes often concern the 'sim to real gap,' a common challenge in machine learning research where AI agents perform differently in real-world application than they do in a simulator due to inevitable discrepancies between the simulator and real-world environments," he explained.

Such errors can prompt the team to replicate the dogfighting scenario in a simulator, identify what went wrong, update the scenario, and retrain the AI agent. The interactive process means that the AI system is always learning and relearning.

But there are major questions about what the role of AI will be in the US military. Recent events — such as accusations that the Israel Defense Forces used AI programs to target Hamas operatives in Gaza, indiscriminately bringing civilians into harm's way — have only increased anxieties about a future war fought by AI.

DoD has made explicitly clear that AI won't make life or death decisions and that those will be left to a human being. But there are concerns about how that distinction may change as AI continues to develop at a rapid pace.