AI & Autonomous Systems Dive
This was not written by ChatGPT...
Continuing with some deeper dives with the USD(R&E) critical technology areas, we’ll now turn our attention toward AI & Autonomous Systems. Broadly, with the boom in large language models (LLMs) and progression toward Artificial General Intelligence, AI has been gaining a lot of notoriety recently. But AI is far from a novel concept, with ideas such as autonomous vehicles, computer vision, chatbots, and NLPs now perhaps reaching peak saturation. Yet, innovation in AI is still chugging along and use-cases on AI in defense contexts continue to gain steam. As with all the posts, we’ll look to frame AI/Autonomy by answering two key questions:
· What do we mean when we talk about AI & Autonomy?
· What are some emergent areas and trends that will shape opportunities here?
But, given the maturity of AI and the preponderance of perspectives, this post will try to unpack more on AI in context of the defense mission.
AI/Autonomy
Before diving into subclassifications of AI & Autonomy, I want to underscore the broader context of AI development and its application to the DoD. At this stage, “AI” is less of an emerging tech area and more of a foundational capability that underpins several technologies used in a DoD context. Conversations around AI have been so commonplace that it has turned into a throwaway buzzword. All that said, advancement in AI still has the potential to be transformative to the DoD. Predominantly, this will come from more trustworthy and humanlike AIs, adopted on a broader scale, trained to make decisions beyond simple automation or inference generation, and existing in context-adapted environments. It’s almost the case that AI is so overrated that it might be underrated (like Bryce Harper).
One way to understand this AI/Autonomy space in DoD’s adoption context is by looking at budgetary indicators for AI adoption. And in this realm, the Center for Security & Emerging Technology at Georgetown has done a remarkable job breaking down US Military Spend on AI & Autonomy. One interesting subdivision in the space here is AI spend on Autonomy related applications and AI spend on non-autonomy related applications. Per CSET’s analysis 68% of all “AI” research form the DoD is also classified as “autonomy” while only 26% of “Autonomy” research is also classified as “AI.”
The fact that there is such a disparity between autonomy and non-autonomy research is not too surprising. Given the overlap between autonomy and AI is strongest at the “advanced” research state, DoD seems to be using budgeted research to test implementations and use-cases of autonomy capabilities. In this vein, coordination with commercial innovators on basic and advanced AI research is critical toward shoring up blind spots in DoD AI capability development. These foundational capabilities and commercial technologies can be leveraged to make best use of AI and Autonomous systems during applied research and capitalize on the respective comparative advantages of DoD and commercial industry. As such a fundamental cornerstone of so many technologies, AI is perhaps the most quintessential dual-use capability.
To understand where these fault lines and opportunities lie, let’s break down what we mean by “autonomy” applications and non-autonomy applications.
On the autonomous systems front, the space can be largely divided into non-weapon systems and lethal autonomous weapons (LAWs):
Non-weapon autonomous systems broadly can include unmanned vehicles across each physical domain (land/air/sea/space etc.) of varying sizes and missions. These systems can be used for reconnaissance missions, security functions, or integrated with munitions for weapons. On the size front, the breadth of these systems varies significantly as well from major systems like the Predator to small handheld drones like Darkhive’s Yellowjacket. While the large systems are ones dominated by the primes, in the small drone space, there’s likely to be disruption and opportunity for dual-use products. Here, Chinese drone manufacturer DJI owns greater than 70% of the market share for drones, which while not used in DoD contexts are used by law enforcement and in commercial applications. Recent efforts by lawmakers to ban DJI and Defense Innovation Unit’s Blue UAS seek to establish a trusted network of these unmanned aerial capabilities. As regulatory effort continues to restrict the market, the rise of these players and growth on both the commercial and defense side is something to watch.
On the Autonomous weapons side, LP agreements largely limit major investment from venture capital but nonetheless, opportunity and attention exists in this space. Recently, DoD updates to DoDD 3000.09 redesigned the DoD’s autonomous weapons ethical standards, creating guidelines for commanders to use autonomous weapons and establishing performance standards for these systems. But given LP agreements and limitations on investment, innovation in this area may be a bit more opaque. Several large weapons manufacturers and primes already have advanced AI research and capabilities and through acquisition may be interested in translating innovation by AI startups into weapons capabilities. In this realm, the value-chain strategies will dominate over disruption ones and startups that look for strong partnerships and quicker exit pathways may see opportunity. But for now, at least from a policy perspective, clarifying guidance on the use of autonomous weapon systems paves the way for introduction of innovative capabilities.
The quick closing note I’ll make here is that full autonomy has a large way to go in terms of trust and policy. Particularly on the weapons system side, accountability and eliminating bias in these systems is essential toward their full-scale deployment. Much like human machine teams, which in the last post I chronicled on their spectrum from Evaluator to Automator, the lion’s share of autonomous systems will live in the middle—enabling humans to make better decisions, keeping them safe, and supporting their decision-making—but will keep human operators in the picture.
On the non-autonomy side, several areas of AI capability have risen to prominence for the DoD. This space will be important to view in terms of commercial capabilities and dual-use technology. Since AI systems and the foundational developments here can be scaled quickly across commercial and defense use-cases, and given that this is less of a priority from the DoD budget in terms of applied research as compared to autonomy, leadership in this space will largely come from the commercial-sector with defense use cases growing with either enterprise ones or as a secondary market:
In terms of capabilities that are more mature, information processing, decision support, NLP, and human-machine teaming (which I dove into previously), have seen broader defense-use cases and adoption. These areas are all classic applications of AI that have not necessarily reached peak saturation and use-case viability but ones that we’re relatively familiar with at this stage. Startups like Vannevar Labs, which uses NLP to support decision-markers in gathering and processing intelligence, or Pendulum Systems (formerly Macro-Eyes) which uses AI models to help provide accurate supply-chain insight & forecasting have led the charge in both defense-first and dual-use applications. In this space, iterative innovation, better trained models, and applied use cases such as predictive maintenance, force readiness, or simulation analysis can help scale AI in defense markets. For other applications, such as supply chain, cybersecurity, or AI assistants the market will markedly be more dual-use with commercial products potentially gaining traction in the DoD as it becomes better at deploying commercial solutions.
Obviously, the AI topic gaining most attention these days is generative AI and content creation. And while most of the commercial conversation around generative AI has come around applications of ChatGPT, DallE or Stable Diffusion, from a national security lens, there are a few areas I think will see attention. The first is data generation and model training, where generative AI has a strong potential to be an underpinning capability to advance things like autonomous vehicles, robotics, and other AI applications. Scale AI has obviously been a market leader in this space, developing a strong data generation and tagging platform that’s been used by a variety of federal customers. The remarkable thing about Scale AI is the position it holds in the generative AI value-chain. Using Scale AI’s platform, commercial and federal users have an enhanced ability to create new use cases of AI applications, fostering in a robust ecosystem of AI tools built around this pick-and-shovel capability in data.
The second space I think generative AI has an interesting national security lens in the counter mis/dis/malinformation space. The scary thing about increasing accuracy around AI generated content is its potential to be a tool for misinformation by adding “veracity” to misinformation. The ability of threat actors to rapidly generate misinformation and incept these ideas in the public consciousness is wildly amplified by generative AI tools and countering these potential narratives is an important consideration for national security customers. In the digital age, companies like Dataminr played a critical role in conducting sentiment analyses and monitoring real-time risk. In the age of generative AI, there’s a larger surface area for this misinformation and potential for startups to enter the space as tailored solutions for reducing cyber-risk, information-risk, or decision making risk associated with misinformation.
On a topic as dense as AI and Autonomous systems, I don’t think I’ve even scratched the surface. To be honest, there’s probably a separate substack that someone can write just on all the applications of AI that we haven’t covered: Neuromorphic AI, AI Engineering, Cloud & Edge AI, etc. just to name a few.
The one thing that I do think underscores all these applications is an emphasis on trust. In fact, USD(R&E) calls this technology area “Trusted AI & Autonomy.” Yet, in CSET’s analysis on US Military spending on AI, only 39 out of 1,076 components classified as AI or Autonomy mentioned the word “trust.” While policy guidance on the use of AI has continued to evolve, there is still a lot of work to be done from the government and from the commercial sector on instilling trust in AI systems. Reducing bias and developing fidelity in AI systems is critical toward not only ensuring their adoption but in stewarding responsible use of these platforms.
In the last few weeks, several technology leaders signed on to a letter calling for a pause on AI development while some of these issues surrounding trust and impact of AI systems are evaluated. While perhaps well intentioned, this would be a mistake. Developing standards, not just in coordination with industry, but driven by industry leaders is crucial toward securing unbiased development of AI. Like standards on privacy or cybersecurity, public-private cooperation is required on standards but this must be done without a pause in development, which will only allow adversaries to gain an edge on this foundational capability of the digital age.
I realize there’s still much to be said here but for, as always: What did I miss? What can be better?