Sensor Fusion – The Future of Soldier Vision: War and Peace

Teal Group’s forecasts for Present & Future Soldier Sensor Fusion Programs would make these soldier vision/C4I systems one of the world’s biggest electronics markets – of any kind – over the next decade, worth more than $11.6 billion, with continually increasing capabilities and – likely – funding. Our multiple forecast lines (see Teal Group’s Military Electronics Briefing for full forecasts) are essentially our all-inclusive future soldier vision forecasts. 

NOTE: On 12 October 2021, US Army officials confirmed at the AUSA 2021 Annual Meeting & Exposition in Washington, DC that the Army has “paused” fielding of Microsoft’s new Integrated Visual Augmentation System (IVAS) Heads Up Display (HUD) system. Teal Group’s discussion and forecasts below were largely written before the Army’s “pause” – which has not yet been presented in detail – but our forecasts already included delays in testing and fielding, and considerably less funding over the next decade than Microsoft has claimed ($22 billion). Our background and discussion below (especially for the ENVG) explain why Teal Group’s forecasts still stand – and what will likely happen over the next decade. 

ENVG Development: ENVG & ENVG(D)

The planned future of soldier “night vision goggles” for the past two decades has lain with the US Army’s AN/PSQ-20 Enhanced Night Vision Goggle (ENVG), a helmet-mounted passive device that incorporates image intensification and long wave infrared sensors. The fusion of both technologies results in night vision goggles that merge the strengths of image intensification – a clear, sharp (originally green-tinted) picture – with the advantages of infrared – the ability to see through most environmental conditions and at longer ranges.

The ENVG was designed for use in conjunction with rifle mounted aiming lights and was originally planned as a blanket successor to the image intensification-only AN/PVS-7 night vision goggle and AN/PVS-14 Monocular Night Vision Device (MNVD). In February 2002, ITT and Raytheon announced a strategic teaming agreement to combine image intensification and IR technology for night vision sensor fusion.

Before 2009, the Army Acquisition Objective (AAO) for the ENVG program was estimated at between 125,000 to 150,000 systems. While the PVS-7 and PVS-14 cost about $2,500 to $3,000 each, the ENVG was planned to cost about $13,500 in full rate production. Combined ENVG and ENVG(D) (Digital) production was to have been worth $3 billion over ten years, not counting continuing development and tube replacements (at the time, this was a lot of money…).

However, by early 2009, though planned funding remained high, the Army had stepped back from expectations of a universal procurement. Instead, the bigger, heavier, and more-expensive-than-planned ENVGs were to go to soldiers in leadership positions in brigade combat teams, as well as special forces troops. With so many PVS-14s procured through the 2000s, the Army re-envisioned ENVG as working with the pure image intensifiers, rather than universally replacing them.

Three competitors developed ENVG systems, but ITT Night Vision and Raytheon took the first five-year, $560 million production contract in April 2005, beating out the Insight Technologies team and Northrop Grumman’s fully digital sensor fused system.

In August 2010, following a slow production ramp-up by ITT, the Army awarded a potential $770 million to three companies, to split the next several years’ ENVG production to come after ITT’s initial “optical fusion” ENVG(O)s. ITT, L-3 Insight Technologies, and DRS Systems were each to produce 220 ENVGs – sometimes called Spiral ENVGs – by August 2013, with the first to be delivered in the first half of 2011. Once production and quality goals were achieved, an order for 4,500 systems was to be awarded to each supplier, followed by a bulk order of 12,000 each. The emphasis for the Spiral ENVG was increased production, not increased performance. Raytheon also began competing with ITT, L-3, and DRS.

The next generation follow-on to ENVG was planned as the Enhanced Night Vision Goggle (Digital) [ENVG(D)] (was DNVG). The ENVG(D) would allow importing images and exporting imagery, even from diverse sensors such as radar, allowing the next level of network-centric situational awareness. BAE Systems delivered a prototype system to the Army in September 2008. Low-rate initial production was originally planned for 2009-10, with Army fielding in 2010, but this slid far to the right. In May 2009, MS III was planned for 2QFY13. In early 2012, an RFP for ENVG(D) development was expected in the 2012-14 time frame. But by April 2013, the ENVG(D) had disappeared from DoD budgets (perhaps going classified rather than being eliminated).

ENVG III & ENVG-B: Major Production, Finally?

Instead, in March 2014 funding was planned for near-term production of an upgraded-capability version of the then-current ENVG(O), designated the ENVG III. In March 2016, procurement funding of $132 million was scheduled for FY17, to buy 9,878 ENVG III systems (this funding included ENVG support as well as production). In the FY17 budget, ENVG III unit cost was shown as $9,300 for the 10,000-unit buy in FY17 – less than the $13,500 originally envisioned by the Army a decade earlier.

ENVG plans and schedules have changed so often that it has been difficult to forecast production numbers, and even versions, definitively. But substantial funding has remained in place throughout nearly two decades of teething troubles, with the Army continuing to plan for tens of thousands of ENVGs. And the last few years have shown a remarkable stability in Army plans, ever since the next-generation ENVG(D) dropped from public records.

From 2015-2018, major ENVG production contracts were awarded to Leonardo DRS ($367 million in 2015 for ENVG III), BAE Systems ($97 million in 2018 for ENVG III/FWS-I, under a previous five-year contract), and L-3 Technologies ($391 million in 2018 for ENVG-B, under a three-year contract, with 10,000 pairs of US Army ENVG-Bs to be fielded through 2021).

By mid-2019, initial production was underway for the current most advanced version of the ENVG – BAE Systems’ ENVG III monocular goggle combined with the Family of Weapon Systems-Individual (FWS-I) rifle sight (see Teal Group’s AN/PAS-13(V) TWS & Family of Weapons Sights (FWS) report). The two sensors are connected wirelessly, accelerating the target acquisition and weapon firing process and greatly increasing capabilities.

In March 2019, the US Army also documented plans and funding for L-3’s new ENVG-B (Binoculars) system, also a modular helmet-mounted, passive electro-optical night vision and long wave infrared (LWIR) imaging device, but in a binocular configuration. The dual image intensification (I2) sensors will provide the Soldier with depth perception for ease of low-light level maneuvers and the ability to detect rifle-mounted aiming lights to engage targets. The ENVG-B can also be operated in a monocular configuration by moving one of the two individually rotating monoculars.

FY19 RDT&E was to see ENVG-B Engineering and Manufacturing Development (EMD), with FY20 plans to continue testing and qualification.

In April 2019, the Army announced the ENVG-B would deploy to an armored brigade combat team in South Korea in September. ENVG-B completed a two-week limited user evaluation test that led up to a 36-hour platoon situation training exercise. Over the next two years, 10,000 soldiers and 3,100 Marines were to receive the ENVG-B.

In October 2020, the US Army awarded L3Harris Technologies (was Insight Technologies and L3 Insight [Londonderry, NH]) an initial Other Transaction Authority (OTA) with a potential total value of $442 million, for the ENVG-B Program of Record. Also in October 2020, the US Army awarded Elbit Systems of America (was ITT, Exelis, and L3Harris [Roanoke, VA]) an initial $22.5 million order under the OTA to begin low-rate initial production of the ENVG-B.

In May 2021, the US Army’s FY22 budget provided FY22 funds to complete ENVG-B LRIP and accomplish Full Materiel Release and First Unit Equipped.

IVAS HUD: The New ENVG(D)?

But by 2019, a huge new parallel soldier night vision headset development was also funded – focusing more on the headset, image, and C4I, rather than the ENVG’s focus on sensors and sensor fusion. In November 2018, the US Army awarded Microsoft (yes… Bill Gates, at that time) a $480 million contract to develop an augmented reality system for use in combat and military training by the US Army and other forces. Now called the Integrated Visual Augmentation System (IVAS) Heads Up Display (HUD), formerly Heads Up Display (HUD) 3.0, the project is developing a headset that gives soldiers—both in training and in combat—an increase in “Lethality, Mobility, and Situational Awareness.” The physical system consists of a wearable visor or goggle – but with nothing mounted on a helmet – with an integrated 3D display, digital cameras, ballistic laser, and hearing protection. The goggle/visor will be connected to a small computer on the soldier’s body.

Plans for this initial contract included Microsoft providing up to 100,000 customized HoloLens systems (already developed as a commercial/gaming technology) to the Army. The HoloLens is a composite technology including sensors such as microphones and light sensors, and a range of cameras. It includes a “holographic” display and lenses, creating a view before the wearer that is mapped onto the world around them, as well as speakers, creating an experience in the real world similar to playing a first-person video game. HoloLens weighs slightly more than a pound.

The Army planned to field the system by the end of FY21, but there have often been problems with contracting a major military development with a commercial company. In this case, in February 2019 Microsoft workers asked Microsoft CEO Satya Nadella and president Brad Smith in a letter to cancel the IVAS contract because IVAS will be used as a weapon, with the direct purpose to improve the ability to kill people. Given especial attention in the letter was IVAS’s goal of blurring the difference between video games used for training (as they have been for years) and the same system and interfaces used in real combat to kill – reputedly reducing war to a game.

Though there were question marks, the US Army’s FY20 budget plans allotted more than $1 billion for procurement of ENVG and IVAS in both FY21 and FY22.

And in March 2021, the US Army awarded Microsoft Corporation a fixed price production agreement to manufacture IVAS, transitioning IVAS from “rapid prototyping” to “production and rapid fielding.” In early 2021, Microsoft said the current IVAS contract will be worth up to $21.88 billion over the next decade, with a five-year base agreement, extendable for another five years.

In other reports, Microsoft president Brad Smith reportedly discussed how IVAS could incorporate facial recognition technology, to pull up database information on civilians and other threats and display this information on the Soldier’s HUD. Presumably this could be combined with the IVAS HUD’s weapon gunsight “pipper” (predicted impact point [PIP]) on the IVAS HUD display for immediate prosecution of virtual targets (but resulting in real deaths). And IVAS could certainly replace live target images with “bad guy” avatars, to make targeting more certain (perhaps even to minimize PTSD?). But what if the data are hacked and the digital “bad guy” avatars aren’t really the bad guys?

So… these and other capabilities make this analyst wonder why Microsoft employees gave up their 2019 protests at developing such a system. But just as the expected legal battles against armed, unmanned drones never materialized a decade ago (“because the need was there”), IVAS HUD seems to be moving ahead as another example of the military not looking too closely at potential future consequences, and scientists developing the deadliest possible systems… simply because they can (and I count test-defeating automobile engine diesel emissions chips in this discussion). So, should Microsoft develop the ultimate personal killing/gaming system, or should it not? Perhaps due to the distractions of the Covid pandemic, Microsoft’s IVAS has cruised full-speed ahead with very little discussion or criticism in the media.

In 2020, the US Army said it planned to field more than 40,000 IVAS HUD goggles in 4QFY21 – from July to September 2021. The current contract with Microsoft reportedly covers production of more than 120,000 goggles for the Army’s Close Combat Force.

Teal Group’s Forecasts

Teal Group earlier forecast that development of a fully digital ENVG could be underway as a classified program, and ENVG(D) production might be blended into our funding forecast at some point in the next ten years. We stated that procurement funding might increase well above our forecast.

Now, with Microsoft contracted to develop what is essentially a digital ENVG (though much smaller and lighter, and with different [seemingly much greater] capabilities), our earlier forecast may play out. On the other hand, the IVAS is based on commercial systems, and that has not always worked out to fulfill all military needs. IVAS could go into production as a universal soldier procurement, but still be supplemented by a greater-capability, leader-borne ENVG (III or -B or Digital…). The Army also originally soft-pedaled on the IVAS’s war-function by mostly referring to it as a training system, rather than as a way to give the computer-video-game generation of young soldiers a user-friendly and familiar way to kill real people (but the DoD’s FY22 budget in May 2021 described IVAS capabilities in depth, and the whole point is to seamlessly blend training and reality).

Our Procurement forecast is thus still somewhat speculative, as ENVG has been a very changeable program(s), and the recent success of IVAS HUD throws some doubt on the ENVG’s future. The US Army’s FY20 budget plans allotted more than $1 billion for procurement of ENVG and IVAS in both FY21 and FY22. This seemed an absurd number – unless the Microsoft IVAS was developed on schedule, with no delays, and a universal procurement of 100,000 systems went ahead with a commercial ramp-up speed. Our forecast assumed somewhat less funding up front, and a couple of years of development delays, but we did forecast the IVAS/Digital ENVG procurement going ahead in the next few years.

As it has happened, there was a year of delay and reduced near-term Procurement funding, but otherwise both programs seem to be going ahead in mid-2021, with Microsoft positively crowing about its IVAS HUD production contract potentially worth more than $22 billion.

RDT&E funding is also uncertain, as lately it has seemed to be included in procurement funds – similar to image intensification NVG procurement – allowing the bidding companies to allot development funding as they choose (for continually re-competed production awards). There may also be a substantial, separate, classified RDT&E line for an ENVG(D). Thus, we include separate ENVG, Digital ENVG (which now includes IVAS), and Sensor Fusion RDT&E funding forecasts.

Most of this non-“ENVG” RDT&E funding is likely classified today, and will remain classified. But as the ENVG, IVAS HUD, and follow-on systems may remain the future of soldier night vision indefinitely (or at least for our forecast period of the ten years), there is almost undoubtedly substantial development ongoing. Our RDT&E forecasts are thus also speculative.

Present & Future Soldier Sensor Fusion Programs

Our forecast for Present & Future Soldier Sensor Fusion Programs would make these soldier vision/C4I systems one of the world’s biggest electronics markets – of any kind – worth more than $11.6 billion over the next decade.

Teal Group typically forecasts most Programs of Record (PoR) will slowly decrease in value in the out-years. But, in this case, why should they? Whether the funding goes to Microsoft or L3Harris/Elbit or another company, lightweight, multi-function helmets or visors or eyeglasses or contact lenses or whatever… if this is the future of soldier vision, why would funding decrease?

Rather, capabilities will continue to increase. As our multiple forecast lines (see Teal Group’s Military Electronics Briefing for full forecasts) are essentially our all-inclusive future soldier vision forecasts (now including a new Future Soldier Sensor Fusion Programs forecast line), we will keep our forecasts strong and steady – not decreasing. Indeed, our forecasts could prove to be conservative. Microsoft has already forecast almost double our funding, just for themselves, and they seem very cheerful about it.

What the systems will look like in a decade, we don’t know. Who will be making them, we don’t know. Will we truly enter a science-fiction-like world of soldier vision systems seamlessly merged with movie/game-like killing efficiency, we don’t know. But Teal Group believes the funding will continue. Why shouldn’t it? No one is going to stop it.