A STUDY INTO THE COGNITIVE PROCESSES USED BY PRACTITIONERS
By Edwin A. Bundy, Ph.D. in F.S., Ph.D. in Ed, CIPBI
Program Manager (GS-15), EOD/EXO Subgroup, Irregular Warfare Technical Support Directorate
INTRODUCTION TO THE STUDY
In early 2022, data was collected from current and former military and public safety bomb technicians from 17 countries, to investigate cognitive approaches used in this community for problem-solving and decision making. The study used improvised explosive devices (IEDs) as the catalyst to stimulate cognitive processes in study participants related to decision-making and problem-solving, and various demographic factors were examined to determine if variables could be identified that made some bomb technicians better at performing these cognitive functions than others. Study participants had from 1 to 35 years of experience, and education levels ranged from the equivalent of a high-school education, to several participants with a Ph.D. Additionally, information related to factors like length of initial bomb disposal training, and any specialized IED electronics training, was captured, as well as formal electronics training (i.e., a technical or academic degree in electronics), and whether or not a study participant was an IED electronics trainer for other bomb technicians.
After applying to be considered for the study, a process that was used to verify that applicants were in fact current or former bomb technicians, participants were given access to an online site consisting of 18 scenarios (see Figure 1), with each scenario consisting of two to three images of a potential IED circuit, and checklists of labels to identify components in the circuit; any hazards they felt were associated with the circuit; and the circuit’s type-by-function. All components for the circuit were visible (i.e., no hidden components), and if a nomenclature was stamped into, or silk screened onto a component, at least one of the images included ensured that these markings were clearly visible.
Figure 1: Study circuits and degree of difficulty as rated by the Expert Panel.
For hazards and circuit type-by-function, participants were also asked to fill in a free-text field identifying the factors or features regarding the circuit that drove their decision-making. Participants could also suggest additional components, hazards, and circuit types-by- function if they felt any were inadvertently left out of the relevant checklists. Finally, study participants were also asked to rate their own confidence levels regarding decisions made for each category (i.e., components, associated hazards, and circuit type-by-function). Qualitatively, this helped in understanding if those with more training or experience were more confident in their decision-making capabilities than those with less training and experience, and the degree to which that was warranted.
In addition to the principal investigator being an experienced researcher and a former U.S. Army Explosive Ordnance Disposal (EOD) technician, standard checks and balances were used for this study. As with all human subjects research, approval was sought and granted by an Institutional Review Board, and oversight was conducted by an academic review committee consisting of Dr. John Frucci, a former bomb squad commander and Executive Director of the School of Forensic Sciences at Oklahoma State University; Dr. Kirk Yeager, Chief Explosives Scientist for the U.S. Federal Bureau of Investigations; Dr. CJ Cowlin, a former U.S. Navy EOD technician and current Director at Large for Bomb Techs Without Borders; and Dr. Jason Beaman, Chair of Psychiatry and Behavioral Sciences at Oklahoma State University Center for Health Sciences.
The content for this study was vetted through use of an expert panel. This panel consisted of subject matter experts (SMEs) in the fields of bomb disposal and IED electronics, and selected based on depth and breadth of knowledge and experience. The panel consisted of Chief Superintendent (Ret) Michael Cardash, Israel National Police Bomb Disposal Division; Lt. Col, Adam Modd, New Zealand Special Air Service; Special Agent Bomb Technician (Ret) Robert von Loewenfeldt, South Carolina Law Enforcement Division; Sergeant Robert Epps, Bomb Squad Commander, Riverside County California Sheriff Department; Mr. Jared French, a U.S. DoD Tactical Bomb Technician; and Mr. Rick Hayworth, a Forensic Examiner and Technical Lead for the Technical Exploitation Unit at the FBI’s Terrorist Explosive Device Analytical Center (TEDAC). Mr. Hayworth was the only non-bomb technician in the group, but is an internationally recognized IED electronics expert, and holds an advanced degree in Electronics Engineering.
Because the study was conducted using a web- based platform, the study was naturally limited to participants who had access to the internet, and because the data collection instrument was written in English, it was also assumed that participants had a technical understanding of the English language. One of the final assumptions was that because this was a voluntary study, and participants were self-selected, that they were likely to have an interest in IED electronics, and therefore the results of this study could potentially be skewed in the direction of higher success rates for component identification, hazard assessment, and circuit types-by-function determination.
FINDINGS
It should be noted by the reader, that these findings should not be interpreted either directly or indirectly, as this author, the academic review committee, or the expert panel, suggesting that bomb technicians are not highly capable, or proficient at the jobs they are trained to do. This study simply focuses on what types of cognitive reasoning approaches bomb technicians use in decision making, and looks at how effective these approaches are used for problem solving by bomb technicians in the IED electronics domain. Further research needs to be conducted to see if similar approaches are used by bomb technicians in other domains, such as x-ray interpretation, and if similar correlations exist.
That said, the results of the study were rather eye- opening. Data gathered from study participants suggests that overall, bomb technicians use pattern-matching for over 90% of their decision-making, rather than using a true hypothetico-deductive reasoning process. Cognitive psychologists classify this as System 1 thinking, which is an intuitive mode of thinking, so is conducted almost effortlessly. This system is responsible for spontaneous or immediate reactions to stimuli, and is often driven by pre-existing biases or an emotional response. System 1 thinking relies almost exclusively on heuristics, or mental shortcuts, and allows decision making to occur with very little conscious effort or reflection.
Conversely, System 2 thinking was used by study participants less than 6% of the time while conducting their analyses. System 2 thinking is a more deliberate and analytical process that requires greater levels of attention and cognitive effort. This system is involved in complex problem-solving tasks, and is characterized by deliberate, effortful cognitive processing, and can override automatic responses generated by System 1 thinking, which allows the user to consider alternative paths or options while in use.
Image 1: 555 IC.
Together, System 1 and System 2 thinking are referred to as Dual-Process theory, and it is widely believed that these two systems operate in parallel, interacting with each other, but sometimes competing for control over thought processes. In this study however, System 1 thinking was clearly dominant, and suggests that rather than participants conducting what might be thought of as a true analysis of the circuit, they keyed in on certain components, presumably those they had been taught during training, or had encountered at some other point in their career. They then based the remainder of their decisions only on the presence of those very specific components, and regardless of what other components might have been present, some unquestionably more dangerous to the bomb technician than the ones recognized, these unrecognized components were not factored into further decision making.
An example is probably warranted here. A compo- nent commonly used in bomb disposal training, whether during initial or sustainment training, is the 555 integrated circuit (IC). This circuit is easily recognized by most bomb technicians, and is commonly re- ferred to as a “555-timer.” It has a rather distinct appearance, in that it is small, black, and has 8 pins/legs (see Image 1).
Image 2: PIC12 microcontroller.
This becomes problematic when bomb technicians start to think that any small, black, 8-pin IC is a 555- timer, and therefore by extension, carries the same associated hazards, and always creates a specific circuit type-by-function. As suggested by data collected during the study, if a component with similar physical characteristics is used in a circuit, even though a visible nomenclature is present, clearly indicating otherwise (see Image 2), many study participants defaulted to selecting the 555 IC, and indicated that the circuit was for use as a “Time” type- by-function, as well as carrying a time hazard. This is problematic, not only because 555 ICs are capable of being used for functions other than timing, but there are numerous other components with similar physical characteristics that present many other types of potential hazards.
Quantitatively, the study found that average success rates for study participants were only 20% for component identification, 16% for associated hazards, and 51% for circuit type-by-function. However, study participants rated their confidence in making these decisions rather high (see Figure 2). This would suggest that bomb technicians in general, significantly overrate the accuracy of their own decisions.
Figure 2. Average self-assessed confidence levels.
Figures 3 – 5 provide study results by dependent variables. The reader will likely note that average success rates are higher for circuits that are commonly taught during bomb disposal training. This is in line with the qualitative findings suggesting that bomb technicians perform their circuit analyses using pattern-matching rather than deductive reasoning, as they are likely to have seen the components used in these circuits, taught the hazards, and then explained the circuit type-by-function. While some of the circuits receiving higher success rates may not be taught during initial bomb disposal training, a review of the curricula from “advanced” courses suggest that they are taught there.
Figure 3. Average success rates for component identification.
Figure 4. Average success rates for hazard analysis.
Figure 5. Average success rates for circuit type-by-function determination.
Data from the study also suggests that independent variables like years of experience, specialized training, or even length of initial training, had no effect on a bomb technician being able to successfully identify the components in a circuit, know what hazards that circuit presents, or in making a determination as to circuit type-by-function. Figures 6 – 11 identify average success rates by independent variables.
Figure 6. Component identification success rates by affiliation, length of initial training, and years of experience.
Figure 7. Component identification success rates by self-assessed knowledge level, specialized and formal training, highest education level, and trainer experience.
Figure 8. Hazard assessment success rates by affiliation, length of initial training, and years of experience.
Figure 9. Hazard assessment success rates by self-assessed knowledge level, specialized and formal training, highest education level, and trainer experience.
Figure 10. Circuit type-by-Function determination success rates by affiliation, length of initial training, and years of experience.
Figure 11. Circuit type-by-Function determination success rates by self-assessed knowledge level, specialized and formal training, highest education level, and trainer experience.
IMPLICATIONS
The results of this study have some very serious implications, not the least of which is for how bomb technicians are trained. While most decisions made during an average day, in any occupation or activity, are based on learned, repetitive behaviors, or what cognitive scientists call heuristics, in new or critical situations, relying on heuristics can fail catastrophically. And even though it is probably not lost on anyone in bomb disposal, that to use pattern matching to train a new bomb technician effectively to quickly and accurately identify all of the electronic components or circuits they might encounter during their career, they would have to be shown literally thousands of different components. The fact remains however, that most bomb technicians are trained to rely almost exclusively on heuristics, even after only being shown what amounts to a hand-full of components during initial training, rather than being taught to use deductive reasoning for their principal decision-making strategy. There are many reasons for this of course, such as limited resources and training time, as well as trying to instill in trainees the perceived need to resolve an incident and return a scene to normalcy as quickly as possible. Because of this, initial training generally focuses on tool-use, rather than critical thinking and problem-solving, which are arguably some of the most valuable skills a bomb technician needs when faced with, and having to make decisions regarding any type of hazardous device.
This data also has implications for what we believe to be the types and varieties of IEDs used globally. If we function under the working premise that statistics regarding device use is gathered through bomb data centers, and bomb data centers collect their data from reporting, which is usually done by bomb technicians, and bomb technicians are only recognizing the components used in circuits 16% of the time on average, or circuit type-by-function 51% of the time, how accurate could device reporting possibly be? And by extension, how accurate could information regarding device construction possibly be, unless it has been analyzed at an exploitation lab by properly trained and qualified forensic scientists or examiners?
Finally, the results of this study have implications for the use of bomb technicians as expert witnesses. In the U.S., it is not uncommon for bomb technicians, particularly at the State and Local level, to be called on by prosecuting attorneys to act as expert witnesses regarding the construction and functioning of a device, or even the potential for pieces and parts found at a residence to be used to construct a device. Again, this becomes problematic when the testimony calls for anything beyond recognition of things other than simple circuits or components. Of course, this is not to suggest that some bomb technicians are not sufficiently qualified to provide expert testimony on device construction or functioning, but it would behoove prosecutors to ensure their bomb technician expert witnesses do not stray too far beyond the basic generalized functioning of a device, unless of course, what they are testifying to has been peer reviewed, and confirmed through valid scientific testing. It is just far too easy in this age of fast fact- checking and AI capabilities like ChatGPT, to access what used to be considered arcane knowledge, like that of bomb disposal.
CONCLUSION
Bomb technicians do a series of non-routine tasks that require mental and physical dexterity, complex critical thinking skills, and creative problem-solving…and that’s the easy part of the job. While these brave men and women are experts in the use of tools to defeat IEDs, there seems to be a lack of training on the general cognitive reasoning skills necessary to enhance a bomb technician’s ability to conduct actual problem-solving, which in turn creates down-stream issues with decision-making. As noted by Louis Pasteur, “Chance favors only the prepared mind,” so the community owes it to our bomb technicians to provide better tools and training in the cognitive domain. Not only will this improve survivability, but it will allow our practitioners to keep pace with adversaries, who are more likely than ever, to use not-yet-seen components and circuits in their devices. ■
ABOUT THE AUTHOR
Dr. Edwin A. Bundy is a Senior Program Manager for the U.S. Department of Defense at the Irregular Warfare Technical Support Directorate (IWTSD). He oversees advanced technology development efforts related to Explosive Ordnance Disposal and Explosives Operations (EOD/EXO) and provides technology- based solutions for combating terrorist and criminal use of explosives to operational personnel worldwide. Dr. Bundy serves as a federal liaison to the National Bomb Squad Commanders Advisory Board (NBSCAB) in the U.S., and is an advisor for both the International Association of Bomb Technicians and Investigators (IABTI) and U.S. Bomb Technician Association (USBTA).
Dr. Bundy is also a former U.S. Army Explosive Ordnance Disposal (EOD) Technician, and Certified International Post-Blast Investigator (CIPBI). Dr. Bundy holds Ph.Ds. in both Forensic Science and Education.
Download PDF: Ed Bundy article – COUNTER-IED REPORT, Spring-Summer 2023