Human Machine Interfaces Dive
Exploring and unpacking the intersection of people and robots, and cyborgs
Continuing with some deeper dives with the USD(R&E) critical technology areas, this post will seek to expound upon Human-Machine Interfaces (HMI). As a reminder, these posts intend to dive deeper into these spaces than a small blurb can contain but are still angled toward providing a survey of the space writ-large. Like any of the critical tech areas, HMI has layers and multitudes which require deep knowledge of technical dimensions. So, just like with Advanced Materials & Manufacturing, we’ll look to frame HMI by answering two key questions:
What do we mean when we talk about Human Machine Interfaces?
What are some emergent areas and trends that will shape opportunities here?
Human Machine Interfaces
Based on the USD(R&E) definition of the space begins by dividing the category into two major subcomponents, each of which has its own clear lane: Augmented & Virtual Reality and Human-Machine Teaming. Beginning with the AR/VR space (or even XR as an umbrella term), the interface between the human and the machine can largely be defined by the extent to which digital environments complement physical ones. At the lowest level of interaction is augmented reality where translucent devices can be used to superimpose digital components to physical spaces. Taking this one step further is a little used term, mixed reality, which layers in some user interaction between the virtual and physical environments. And finally at the most digital level is full virtual reality, where the user is transported from the physical to the virtual in an immersive experience.
From a defense context, the importance of these tools largely exist to augment and support decision-making systems/processes, enhance command and control, generate more user-intuitive systems, and support the integration of increasing disparate data into warfighting systems. Several trends and undercurrents in the space can be unpacked to build an investor’s perspective on AR/VR as well as subdivide this area into more digestible applications:
One the VR front, current adoption of VR is largely in its infancy with limited transformative use cases seen in the market so far. Overall, venture investment into AR/VR technologies has been on a downward trends since 2017, leaving the space sensitive to investment risk. However, the promise of AR/VR tech as a value-add capability is still there, though the timing may not be. What this may tell us is that investment in the enabling capabilities and pick-and-shovel opportunities in AR/VR may be an interesting prospect. Google Glass was long derided for being too soon to the market but it’s not unfounded to expect innovation in hardware forms to support AR/VR technology. Better displays, better wearables, better enabling tech each make reduce the potential energy required to supercharge AR/VR adoption. Even on the enabling technology front, innovation such as advanced gesture control from Pison or voice control from Primordial Labs can help operators control the physical and digital environments seamlessly.
One other small segment to observe here is the AR/VR simulation and content space. The use case here is fairly clear with advanced simulation capabilities and training leading to potentially better adherence outcomes. And the spectrum of capability here varies widely from traditional content platforms and metaverse style simulations to advanced neuromorphic AI and brain-pattern mimicking simulations. The viability and outcome in this area is still to be determined with fewer clear competitive moats and a dependence on the hardware that’s currently lagging. However, with breakthroughs in systems, the value proposition can be enhanced and turn into a potential watershed moment.
On the AR side of the house, adoption of mixed reality tools and integration into existing platforms may be a more immediate gate. We’ve already seen the early inklings of this with heads-up displays on cars but innovation on embedded systems can prove to be a critical capability for increasing decision-making effectiveness and information dissemination. Disruptors here, who are innovating in pick and shovel opportunities like displays such as Brelyon or Light Field Labs, can potentially created AR displays embedded into military platforms, enabling warfighters to leverage mission data without losing focus on the operation.
Turning our attention now to the human-machine teaming capabilities the HMT term itself fairly intuitively describes the category. However, like XR, HMT spans a spectrum of capabilities based on the ratio of human to machine being teamed. At the far end—with greatest human control— are “Evaluator” systems which help assess problems, leaving decisions to humans. One step beyond this are “Illuminator” systems which, like Evaluators leave the decision to humans but provide more insight rather than compiling information. “Recommender” systems take these insights to support automation of routine decisions with ease. “Decider” systems take this a step further, making the decision based on its assessment but leave the human to implement. And at the other pole, “Automator” systems fully automate processes with limited human interaction.
The common thread here between all these systems is the motivation and outcome, which much like XR, is intended to support decision making and mission operations. The degree to which machines should be integrated into the decision-making process depends on a variety of factors, including the reliability and trustworthiness of the underlying decision engine, the context of the mission, and the capabilities available to execute upon these decisions. Here, in order to unpack trends, I want to talk about the interaction points between human-machine interfaces and two other critical technology areas as this will illuminate just how advancement in these technologies can augment defense capabilities.
First, looking at advanced manufacturing, industrial IoT applications, advancement in sensor technology, and other digital manufacturing trends along the lines of what I highlighted in my previous post are key enablers of better human-machine teaming. To create more reliable and autonomous human-machine teams, the systems require strong and reliable sets of data to open their aperture across two dimensions: the scope of work that can be performed by a machine and the quality of this work. With better IoT and better sensors, human-machine systems, will have better data across the entire decision-making chain, thereby understanding the impacts and consequences of decisions and enabling smarter HMT. Applications of this type of human-machine teams can be seen in factory settings, where better robotics will enable a transition to Industry 4.0, or in military settings where human-machine teams can facilitate the automation of repetitive tasks.
A second point of interaction with the critical technology areas is AI and Autonomous systems. Autonomous systems and human-machine teams both largely exit on the same spectrum, varying based on the level of automation. Here, innovation in artificial intelligence capabilities can help advance applications on semi-autonomous systems and human-machine teams. On AI and human machine teaming, developments in AI such as large language models, image models, and neuromorphic AI can radically enhance the way scope of work that can be done by machines. An AI that is able to behave and think in human patterns can not only make better decisions but can make more human decisions and more interpretable decisions, enabling more effective teaming and extrapolation of insight.
Closing up this piece, there is one of section of human-machine teaming that I omitted but that I must admit I’m deeply interested in. Frontier technology bets and deep-science plays that are potentially decades away are looming on the horizon to supercharge our human-machine teams. Brain-computer interfaces such as Neurable or Blackrock Neurotech invert the human-machine value chain by bringing the machine to the human rather than the human to the machine. While the idea may be a little sci-fi cyborg, advanced brain computer interfaces can help enhance our cognitive capacity and seamlessly interact with the physical and digital world in tandem. Applications of these capabilities may still be a while out, with several ethical, medical, and technological milestones yet to be accomplished but development in this area excites me as a catalyzing function As always, let me know: what am I missing? What can be better?