“The United States’ technological edge has long been a foundation of our military advantage. . . . Because Joint Force operations increasingly rely on data-driven technologies and integration of diverse data sources, the Department will implement institutional reforms that integrate our data, software, and artificial intelligence efforts and speed their delivery to the warfighter.”
— 2022 US National Defense Strategy
“[Goal #1] The IC [intelligence community] will improve its ability to provide timely and accurate insights into competitor intentions, capabilities, and actions by strengthening capabilities in language, technical, and cultural expertise and harnessing open source, ‘big data,’ artificial intelligence, and advanced analytics. . . . In response to changing mission needs, the IC will foster a culture that embraces innovation and the application of tools, data, processes, and standards necessary to transform labor- and time-intensive work into more efficient and productive human-machine partnerships.”
— 2023 US National Intelligence Strategy
Strategic documents from both the Department of Defense and intelligence community illustrate the importance of harnessing the value of data. But has it, as the National Defense Strategy describes as the objective, reached the warfighter? The short answer is no. For soldiers, a brief look at the courses available in ATRRS (Army Training Requirements and Resources System) shows little data-specific training. The branch schoolhouses (where soldiers learn how to perform their jobs) are also largely barren. Professional military education (which provides training as people move up the promotion ladder) also offers little on “data-driven technologies and integration of diverse data sources” or how to integrate “data, software, and artificial intelligence efforts” to support the warfighter. Indeed, there is a gap between what strategy calls for and what is actually available to soldiers.
In the absence of training, innovate. During the summer of 2023, the United States Civil Affairs and Psychological Operations Command conducted its annual Command Post Exercise–Functional (CPX-F) with Army Reserve civil affairs, psychological operations, and information operations units. The exercise offered an opportunity to experiment—can Army units, psychological operations and information operations battalions in this case, train their soldiers to use data in ways envisioned by strategic documents?
In the months leading up to the exercise, the three of us worked with leadership of the brigade and battalions involved in the exercise to break down this challenge into three key questions:
- The National Defense Strategy aside, how willing is the Army to actually allow outside data, software, tools, and processes anywhere near its IT systems? In other words, just how ardent is our embrace of the innovation ecosystem?
- How capable are Army systems of delivering enough reality in a virtual training environment to effectively assess data-related skills?
- How long does it take to train a unit on basic data skills so unit members can effectively integrate data analysis into operations?
Question 1: Why Should I Allow Your Software Anywhere Near My System?
Months prior to the exercise we identified the need for a capability to visualize and make sense of data representing the information environment. In previous iterations of the exercise this was largely an analog process—a small batch of posts, news articles, and social media messages would be posted in the virtual environment; analysts would find and read the content, take notes, and brief relevant findings using whiteboards and static PowerPoint slides.
For this iteration we wanted an environment more reflective of real-world conditions, where the amount and speed of information flow is too large for an analyst, or even a large group of analysts, to read every post, news article, and message to assess value. For example, monitor all social media posts related to Russia’s invasion of Ukraine is a useful assignment, but hardly possible without the type of big data analytic capabilities mentioned in the National Defense Strategy and National Intelligence Strategy.
Based on these assessed needs, we evaluated several data analysis tools. Ultimately, we selected an existing commercial software tool called Tableau for several reasons. First, unlike many exquisite, purpose-built government systems, Tableau must survive in a competitive commercial environment, forcing it to provide both ease of use and up-to-date capabilities. Second, it was already approved for government use, including the ability to secure multiple user accounts. Third, it is industry-defining software—several soldiers already knew how to use it from their civilian careers. And finally, it was already in use across multiple commands, which meant training on the software could realistically be used in support of real-world missions.
Unfortunately, and to exactly no one’s surprise, there were multiple issues incorporating new software into the exercise. First, we had to convince senior leaders of the value of bringing data analytics into the exercise, both why it was important and how we were going to do it. Next, we had to get the software approved for use during the exercise by those running the exercise. We initially reached out to the 91st Training Division for approval. The request was sent from there up to the 84th Training Command. The 84th was willing to approve but needed approval from the US Army Reserve Command to have the software installed on the exercise network. The process took thirty days from initial request to approval, as the 91st had not gone through a software request before. To restate—it required an entire month and approval from the highest level of the Army Reserve to install government-approved software on government computers. In addition, a division-level training organization had never received a software request prior to ours. Ultimately, embracing the process and winning approval was a clear positive, but the process needs to become quicker and more efficient to better align with the goals outlined by senior leaders in the strategy documents.
Once the software was approved and purchased and licenses were provided, the software had to be installed on the exercise network laptops. Once loaded on the exercise network, the unit’s signal and communications team had to figure out how to incorporate software into the system without documentation or an example to follow. The personnel at Command Post Exercise–Functional were willing to help but did not have knowledge on how to install third-party software and integrate it with exercise systems and services.
These efforts were driven by reservists from the 15th Psychological Operations Battalion, often acting on their own time between drills. Without persistence and a large investment by the unit in planning and preparation, we would not have gotten the software approved, purchased, loaded, and ready for data analysis. Overall, the experience serves as a proof of concept through trial and error.
This pre-exercise process led to three key early findings:
- Across the force there is little understanding of big data and what it can provide.
- Adding capabilities to use during the exercise (i.e., software and data) is a struggle, even when we purchased our own government-approved software and brought our own data.
- There is concern about crossing from data analysis into the intelligence function, and hence operating outside of an ATO (authority to operate).
Question 2: Can Army Training Systems Deliver Enough Realistic Information?
High-quality training requires enough realistic information to truly test data skills. So, can Army systems provide it? Barely, and only if pushed. The virtual information environment used during the exercise was the Information Operations Network, a closed intranet colloquially known as ION. ION can create a notional information environment—with social media messages, video postings, news sites, and a basic search function—but offers only a limited number of online personas and a limited ability to populate an information environment. A dynamic environment (one that changes and adapts based on activities performed by the training audience, not a simple, linear rollout of precooked scenarios) is needed to test soldiers, maintain their interest, evolve the story arc of the exercise, and show how effects in the information environment translate to the operational environment. However, ION offers only limited content created in a dynamic fashion and requires a heavy lift from either contractors or soldiers acting as a red cell to populate the information. In our case, we took several psychological operations soldiers and had them generate content, which all had to be approved by exercise control before going live on ION. That is, to meet our requirement for dynamism, we had to utilize members of the training audience to augment exercise control so that we could meet our training objectives.
This led to three key findings:
- Improved virtual tools (which may require greater investment) are needed to provide dynamic training exercises that more accurately mirror real-world information and operational environments, tempos, and activities.
- It is imperative for leaders to provide guidance on dynamic activities and responses to ensure training objectives are met.
- There must be multiple personnel with the authority to disseminate created content to avoid bottlenecks.
Question 3: Training People on Data Analysis Takes Forever, Right?
How long does it take to train an entry-level data analyst—specifically, one able to sort through large quantities of information (i.e., big data)—to uncover mission-relevant insights? In our case, as psychological operations and information operations officers, how long does it take to train an analyst capable of sorting through tens of thousands of social media messages, online news articles, and other publicly available information to create operationally relevant target audience and narrative analysis?
It took less than twelve hours across two days to train twenty soldiers, ranging in rank from private to lieutenant colonel. The all-star of the exercise ended up being a junior enlisted soldier with zero data analysis experience prior to the training we provided.
As mentioned above, the Army offers little data-related training, so we relied on our civilian skill sets to create the course. We developed the training—the instructional design, scenarios, datasets, Tableau files, and related slides—months before the exercise, content we then used during our campaign to win approval to use outside software on government systems. The training was scenario driven throughout, using real-world data on extremism, Twitter/X data on Russian efforts to influence US elections in 2016 (data originally used by the Senate Intelligence Committee to investigate Russian activities), and two decades of English-language content from the Russian foreign ministry. Each scenario required soldiers to use the data and software to provide operationally relevant findings they briefed to leadership. Soldiers successfully gave their first briefings less than three hours into the training. The concluding briefs were observed by a general officer, who praised the soldiers’ ability to find and visualize answers in the data. Per a brief survey administered after the conclusion of the course, 100 percent of those surveyed found the training valuable and 100 percent thought the Army or individual units should offer similar training outside of the exercise.
After the training, soldiers were successfully able to run analytics on the exercise platform by importing raw data from the ION website to the analysis software (Tableau). Analysts were, for example, able to quickly sort thousands of posts on Twitter/X and Facebook to find key players in the information space. The analysts said without the capabilities provided by the software and training, they would not have been able to piece together network diagrams, identify key leaders, and track who held large influence on social media.
Within hours of the start of the exercise, our soldiers were able to employ the tools and training to provide relevant, data-driven insights (a process that had taken days in the analog era). Based on that initial success, we asked the exercise controllers for more noise (essentially, more junk messages that analysts would have to filter), but the soldiers swiftly overcame the increased flow. We repeated this process several times, but no matter how much we increased the amount and rate of information flow, the pre-exercise training held and the soldiers were able to quickly and effectively analyze data to provide operationally relevant findings to the commander for data-driven decision-making.
This is our key finding and proof of concept: with minimal training and using off-the-shelf commercial tools, soldiers can quickly and effectively produce operationally relevant data analysis.
Conclusion and Recommendations
Data analysis does not require months or even weeks of training to generate useful insights. In two days, our training audience, including nearly every rank from private to lieutenant colonel, was able to develop functional data analysis and visualization skills.
At little cost to the Army, we developed and demonstrated a clear proof of concept for embracing the National Defense Strategy’s “innovation ecosystem” to support data-driven decision-making. We encourage the Army to invest in these efforts, improve them, and then bring the training to additional units. Multiple capability areas need at least a basic level of data analysis and would benefit from this type of training. We also note that refinement and improvement of our training is important—data, tools, and methods are not static.
We also encourage the Army to use common commercial software, not exquisite, military-specific software. Commercial software is often cheaper, easier to learn, and easier to find (both the software and those already trained to use it). In addition, when commercial software is properly selected, it is more likely to receive ongoing development, patches, and updates as part of the license, allowing soldiers to benefit from rapidly evolving, best-in-class software capabilities. A marked contrast to the buggy, difficult-to-use, slow-to-improve software commonly seen in the Defense Travel System and other government-only software systems. The advent of large language models such as ChatGPT provide even more capabilities that could be tested in future exercises and wargames.
Our Army experience, including during deployments, influenced the instructional design and datasets selected for the training, but it is important to note that many of the skills required to create and deliver the training came from our civilian backgrounds, not from the Army. By combining Army experiences with capabilities from our civilian professions, we attempted to create training that would better prepare our units and ourselves for future mobilizations. As the Army and joint force continue to build their data-related capabilities, we strongly urge leadership to build similar data analytic courses, include related training in professional military education, and consider further harnessing the existing civilian data skills offered by members of the reserve components.
Michael Schwille is a political scientist at RAND, where his primary research interest is the integration of information into combined arms warfare. He is also a lieutenant colonel in the US Army Reserve, where he is qualified as a civil affairs, psychological operations, and information operations officer. He has conducted multiple deployments to the Middle East and Africa and is currently the commander of the 15th Psychological Operations Battalion.
Scott Fisher is a professor of security studies for New Jersey City University, where his research focuses on information warfare, US security challenges in East Asia, and open-source intelligence. He is a major in the US Army Reserve, with deployments as an information operations officer to Afghanistan, East Africa, and US European Command in Germany. He is currently special projects officer for the 151st Theater Information Operations Group at Fort Totten, New York.
Eli Albright is the S6 for the 15th Psychological Operations Battalion and an IT professional with five years of experience managing IT infrastructure for large corporations and the US Army Reserve.
The views expressed are those of the authors and do not reflect the official position of the United States Military Academy, Department of the Army, or Department of Defense.