When Big Data went to war — and lost

On October 1, 2017, a roadside bomb northwest of Baghdad killed Spc. Alexander W. Missildine. The 20-year-old was the latest American soldier to die in a war that had lasted, in some form or another, since he was in kindergarten. And as much as the Iraq War had changed over the past 14 years, the weapon that killed Missildine — the improvised explosive device, or IED — remains just as potent, and just as vexing, as it was when the U.S. originally invaded Iraq.

For a few years, it looked like the U.S. was about to make headway against IEDs, using a combination of old military insights and new data-processing techniques. The project was called JIEDDO, the Joint Improvised Explosive Device Defeat Organization. Born on Valentine’s Day, 2006, for nine years it remained a central part of America’s wars in the Middle East. In the annals of the pre-surge war in Iraq, few acronyms conjure as many strong feelings among Washington’s national security establishment, or offer as clear a lesson about the challenges of quickly building smart new systems and keeping them on track.

The idea was appealing: IEDs were a new and unpredictable weapon, but the U.S. had had great success using data against an unpredictable weapon before. During World War II, the Allies faced irregular attacks from an unseen foe, the new German submarines preying on shipping in the North Atlantic. By analyzing the attack patterns, and developing new technologies that could spot and then destroy U-boats before they attacked, science and data secured safe passage across the Atlantic, and left the once-feared U-boat fleet strategically sidelined.