An analyst produces a threat assessment in January concluding that a particular criminal organization poses a moderate risk to a client’s operations in a given region. Over the following months, new information trickles in: a leadership change within the organization, a shift in law enforcement pressure, a new alliance with a logistics network. Each piece of information is evaluated individually against the current assessment. None of it, on its own, is dramatic enough to trigger a revision. By June, the analyst is still reporting moderate risk, even though the cumulative picture has shifted substantially. The January assessment was the anchor, and every new piece of information was measured against that anchor rather than evaluated as part of a changing whole. The consumer has no idea the landscape has shifted because the analyst never stepped back to compare the current evidence base against the original conclusion.

ICD 203 requires that analytic products state how their major judgments on a topic are consistent with or represent a change from those in previously published analysis, or represent initial coverage of a topic (ICD 203 2015). Recurrent products such as daily crisis reports should note any changes in judgments; absent changes, recurrent products need not confirm consistency with previous editions (ICD 203 2015). Significant differences in analytic judgment, such as between two agencies or intelligence organizations within the IC, should be fully considered and brought to the attention of customers (ICD 203 2015). DIA guidance reinforces this with a direct instruction: be transparent about how an analytic conclusion is different than previously published analysis. The CIA’s Directorate of Intelligence treated conclusions as the bedrock foundation for estimative judgments in assessments that address future patterns of development, and recommended that analysts use a textbox to explain any major shift from previous assessments or the basis for a contrasting conclusion held by other IC analysts (CIA 1995).

Consumers build plans, allocate resources, and make decisions based on prior assessments. A corporate security director who received a moderate-risk assessment in January may have staffed, budgeted, and planned around that assessment for the entire quarter. If the risk level has actually increased but the analyst hasn’t flagged the change, the security director is operating on outdated analysis without knowing it. A law enforcement commander who was told a gang’s territorial ambitions were limited may have allocated patrol resources accordingly. If the gang has since expanded but the assessment language hasn’t changed, the commander’s deployment is misaligned with the actual threat and nobody has told them.

How Judgments Drift

Information arrives incrementally, and each individual piece gets compared against the analyst’s existing view. No single piece may be sufficient to prompt a change, but the cumulative message inherent in many pieces of information may be significant, and that significance is lost when the information is examined only one piece at a time (Heuer 1999). A corporate intelligence analyst tracking a competitor’s market strategy might receive weekly updates: a new hire here, a patent filing there, a shift in marketing language, a change in supplier relationships. Each update, viewed against last week’s reporting, looks like normal business activity. Viewed collectively over six months, the pattern reveals a major strategic pivot that the analyst’s incremental approach never surfaced.

Assimilation bias compounds the problem. Information that is consistent with an existing mind-set is perceived and processed easily and reinforces existing beliefs (Heuer 1999). Information that is inconsistent tends to be overlooked, distorted, or rationalized to fit existing assumptions (Heuer 1999). An analyst who has assessed a foreign government as cooperative on trade issues will readily absorb reporting that confirms cooperation and will tend to discount or reinterpret reporting that suggests the government is quietly building barriers. Over time, the analyst’s assessment drifts further from the actual situation while the analyst remains confident the assessment is current.

Heuer described analysts proceeding on the basis of the day’s take, hastily comparing it with material received the previous day, producing in “assembly line fashion” items that did not accrue from a systematic consideration of an accumulated body of integrated evidence (Heuer 1999). A private investigator working a long-running surveillance case faces the same trap. Each day’s observations are compared against yesterday’s. The subject’s pattern of behavior shifts gradually, and the investigator’s daily reports each describe minor variations that individually seem unremarkable. Six weeks in, the subject’s behavior pattern has fundamentally changed, but because no single day’s observations were dramatic enough to flag, the change went unreported. Stepping back to compare the current pattern against the original baseline would have made the shift obvious.

Consistency Can Be Misleading

Analysts and consumers alike place too much weight on consistency when evaluating evidence. When multiple pieces of evidence point in the same direction, the natural response is increased confidence that the underlying judgment is correct. But information may be consistent only because it is highly correlated or redundant, drawn from a very small sample, or consistent with a plausible but wrong hypothesis (Heuer 1999). Five media reports about a company’s financial health may all tell the same story because they’re all drawing on the same press release. A law enforcement analyst receiving consistent reporting from multiple informants about a target’s activities may not realize the informants are all getting their information from the same intermediary. The consistency is real, but it reflects a single data point amplified through multiple channels rather than independent confirmation. The analyst who reports “our assessment remains consistent with previous reporting” without examining whether the underlying evidence base has genuinely broadened or merely echoed itself is projecting false confidence.

Once an analyst has constructed a narrative from previous events, new information gets evaluated based on whether it fits the story (Heuer 1999). Information that fits gets incorporated; information that doesn’t fit gets minimized or reinterpreted. The narrative becomes more coherent over time, which increases confidence, but that coherence may reflect the strength of the story rather than the strength of the evidence. The article on argumentation covered this dynamic in detail; for analytic line purposes, the key risk is that a coherent narrative makes drift invisible because every new piece of information gets absorbed into an existing framework that was never re-examined as a whole.

Language That Hides Drift

Ambiguous language creates a particular problem for analytic line. When intelligence conclusions are couched in ambiguous terms, a reader’s interpretation will be biased in favor of consistency with what they already believe (Heuer 1999). If an analyst writes that a threat “remains elevated” in both the March and June assessment, the consumer will likely read both as saying the same thing, even if the analyst’s understanding of the threat has actually shifted. The language is vague enough to accommodate different meanings, which means the consumer can’t detect whether the judgment has changed.

DIA writing guidance warns against overusing the words “continues” and “remains” (DIA 2015). “Drug trafficking continues to provide most funds for the insurgents” and “the insurgency remains concentrated in the south” sound like they’re conveying continuity, but they may be masking an unexamined assumption that things haven’t changed rather than reflecting a deliberate assessment that the situation is genuinely stable. An analyst who defaults to “continues” and “remains” in recurring products can produce months of reporting that reads as consistent when the analyst has never actually re-evaluated the underlying judgment.

Products need not be lengthy or detailed in explaining change or consistency, but they should avoid boilerplate language and should make clear how new information or different reasoning led to the judgments expressed in them (ICD 203 2015). A law enforcement analyst producing weekly gang activity reports for a patrol commander doesn’t need to write a paragraph explaining why this week’s assessment matches last week’s. But when the assessment does shift, the report needs to say so explicitly, explain what changed and why, and connect the new judgment to specific evidence. “Our assessment has shifted from moderate to elevated risk based on three confirmed territorial incidents in the past two weeks” does the work. “Risk remains elevated” does not, because it tells the consumer nothing about whether anything has actually changed.

Pre-Committing to Change

Heuer recommended that analysts specify in advance what would cause them to change their mind, because pre-commitment makes it harder to rationalize contrary developments as not really requiring revision (Heuer 1999). An analyst who has written down “if we see the target company change auditors and simultaneously accelerate debt issuance, we should reassess our financial stability conclusion” has created a tripwire. When those events occur, the analyst has already committed to treating them as significant. Without that pre-commitment, the same events can be rationalized away: the auditor change was routine, the debt issuance was opportunistic, neither one alone warranted a reassessment.

Heuer also recommended identifying milestones that would indicate events are taking a different course than expected (Heuer 1999). A corporate intelligence analyst assessing that a competitor is unlikely to enter a specific market segment could identify milestones: if the competitor hires executives with experience in that segment, files relevant patents, or begins procurement relationships with suppliers in the space, the assessment needs revisiting. Those milestones give both the analyst and the consumer a shared framework for monitoring whether the assessment remains valid. Analytical conclusions should always be regarded as tentative, and specifying in advance what would suggest a significant change in probabilities helps analysts avoid waiting for a single dramatic piece of evidence before they’ll revisit a judgment (Heuer 1999).

Thinking backwards starts with the assumption that a targeted danger or opportunity has already occurred and then generates plausible scenarios for how it could have happened (CIA 1995). For analytic line, this technique forces the analyst to imagine what the world would look like if their current assessment were wrong and work backward to identify the observable indicators that would precede such a shift. If the assessment says a political alliance is stable, thinking backwards asks: what would destabilization look like in its early stages, and are any of those indicators present in current reporting? A private investigator maintaining a long-term assessment of a subject’s risk profile could use the same approach: if this subject were to engage in the conduct we’ve assessed as unlikely, what steps would precede it, and have we seen anything resembling those steps?

More broadly, Heuer argued that analysts should periodically re-examine key problems from the ground up to avoid the pitfalls of incremental analysis (Heuer 1999). Ground-up reviews force the analyst to evaluate the entire body of evidence rather than measuring each new piece against the existing conclusion. The techniques for challenging established views covered in the article on analysis of alternatives, including devil’s advocacy and competitive analysis, serve this function when applied to standing assessments that have been in place long enough to become entrenched.

Interagency Disagreement and Downstream Consistency

When multiple analysts or organizations cover the same topic, their assessments will sometimes diverge. ICD 203 requires that significant differences in analytic judgment between IC elements be fully considered and brought to the attention of customers (ICD 203 2015). Disagreement is diagnostic information. Two agencies reaching different conclusions from the same evidence base may be applying different assumptions, weighting different factors, or interpreting ambiguous information through different frameworks. Making that disagreement visible lets the consumer evaluate which set of assumptions they find more persuasive and plan for the possibility that either assessment could prove correct.

A corporate intelligence department where the competitive analysis team and the geopolitical risk team reach different conclusions about how a regulatory change will affect the company’s market position needs to surface that disagreement rather than resolve it quietly. If one team assesses minimal impact and the other assesses significant exposure, the executive reviewing those assessments needs to see both views and understand why they diverge. Burying the disagreement by splitting the difference or defaulting to the more optimistic assessment deprives the consumer of information they need to make a sound decision.

Analytic line also requires consistency across product formats and audiences. Tearlines, the portions of classified products that can be shared at lower classification levels, must render facts and judgments consistent with the original reports on which they are based (ODNI 2012). Sanitized products should never render facts or judgments in a manner inconsistent with their higher-classified version; facts, judgments, confidence levels, and probabilistic language must be congruent (ICD 208 2017). If the classified version says “we assess with moderate confidence” and the unclassified version says “we assess,” the consumer receiving the unclassified version is reading a more confident assessment than the analyst intended. For practitioners outside the IC, the parallel is straightforward: internal briefings, external client reports, and executive summaries of the same assessment must all convey the same judgments at the same confidence levels. A due diligence report that tells the internal team “significant concerns about financial controls” and tells the client “minor issues identified” has broken its analytic line across products.

Closing

The countermeasures here are straightforward even if they take effort to sustain. Write down what would change your mind before you need to change it. Build periodic ground-up reviews into your workflow so you’re comparing current evidence against your conclusion, not just comparing today’s report against yesterday’s. When your assessment does change, say so explicitly, say why, and connect the new judgment to what drove the revision. When it hasn’t changed, make sure that’s because you’ve genuinely re-evaluated rather than because “remains” and “continues” were easier to write than a fresh look at the evidence.

Your consumers are making real decisions based on what you tell them. A corporate security director is allocating budget. A law enforcement commander is deploying officers. A client is deciding whether to proceed with a transaction. They can’t see the information you’re seeing, and they can’t detect when your underlying assessment has shifted unless you tell them.

References

  • Central Intelligence Agency. 1995. A Compendium of Analytic Tradecraft Notes. Directorate of Intelligence.

  • Defense Intelligence Agency. 2015. Style Manual for Intelligence Production.

  • Heuer, Richards J. 1999. Psychology of Intelligence Analysis. Central Intelligence Agency.

  • ICD 203. 2015. Analytic Standards. Office of the Director of National Intelligence.

  • ICD 208. 2017. Maximizing the Utility of Analytic Products. Office of the Director of National Intelligence.

  • Office of the Director of National Intelligence. 2012. Intelligence Community Directive 206: Sourcing Requirements for Disseminated Analytic Products.