By the Director, National Intelligence Academy
Human cognition is central to all the work we do, whether it’s drafting intelligence reports, policy proposals, decision making or even just planning your next holiday. While people claim to be objective, in reality all our thinking is subject to cognitive biases which are influenced by our experiences, preferences, information availability and any number of other factors. Even how you frame the problem – what we’d call the intelligence question – can shape the conclusions you reach!
The bottom line is all thinking is subjective.
Daniel Kahneman’s book ‘Thinking, Fast and Slow’, discusses two systems of thinking. The first is quick and easy as it’s based on experience and instinct, while the second is slower, harder and more deliberative. Both are valid approaches to making decisions and both are subject to cognitive biases. However, being more systematic and deliberative in your thinking – slowing it down – will give you a better chance of mitigating (not eliminating) your biases.
In intelligence analysis, we use Structured Analytic Techniques (SATs) as tools for slowing down and being more systematic in how we think about intelligence problems. In the rest of this article, I’ll discuss how we use SATs in our analytic processes.
Step 1: Asking the right question
Sometimes a customer (policy officer or decision maker) will provide the question. Other times there is scope to adjust the question to draw out more meaningful insights – think of the difference between ‘Will Candidate X win the election in Country Y?’ and ‘What are the implications for Australia if Candidate X wins the election in Country Y?’
Step 2: Selecting the right SAT
There are a huge variety of SATs, and we chose the techniques based on the problem we’re trying to solve. Some techniques are diagnostic, creating different ways of organising and testing information to draw our insights. Others are designed to promote imaginative thinking or to push analysts to challenge their prevailing judgements. If the question is around the long-term future, we can use a range of techniques to explore alternative futures.
Step 3: Applying the SAT
While analysts can use SATs by themselves, their true power is being used to structure a group discussion. Using a group is a critical way to overcome individual blind spots but to do so, the group needs to contain a diversity of (relevant) knowledge, experience and perspectives to avoid group think. A facilitator plays a critical role in ensuring all voices and perspectives are heard.
Most SATs will have some form of brainstorming as a core component of the process. Brainstorming is essentially an idea generation technique, and we tend to use ‘silent brainstorming’ to prevent the group becoming anchored to the first idea mentioned or ‘band wagoning’ behind the idea that seems most popular.
Once the ideas are generated, they are clustered into themes and the group then decides which ideas should be taken forward for either further refinement or inclusion in final judgements.
Step 4: Recording the SAT
Intelligence failures tend to be very public and often followed by inquiries into what went wrong. Using SATs generate records – these include all the ideas (or evidence) considered and what they were narrowed down to. Having a scribe to record the conversations is a great way to capture how different ideas were weighed and either discarded or included in the final report. Using SATs in analysis is therefore a great way for building in transparency for how conclusions were reached.
None of this is to say that SATs will enable us to avoid intelligence failures – they don’t solve the problems of incomplete information and the ‘unknowability’ of the future. But they do improve our analysis by mitigating biases, enabling a consideration of a wider body of evidence and ideas, and for reframing problems in ways that generate fresh insights.
Explore more
If you’d like to learn more, you can do the free online ‘Introduction to Structured Analytic Techniques’ course at NIA Learn.