Intelligence analysts rarely know things for certain. The information they work with is incomplete, the sources they rely on vary in reliability, and the situations they’re assessing involve human actors whose decisions resist prediction. Joint doctrine acknowledges this directly: intelligence is the synthesis of quantitative analysis and qualitative judgment, subject to competing interpretation (JP 2-0 2022). CIA guidance goes further, stating that analysts have a professional obligation, where warranted, to conclude that they do not know (CIA Directorate of Intelligence 1995). Analytic Tradecraft Standard 2 exists because that obligation extends beyond admitting ignorance. Analysts must communicate uncertainty in terms precise enough that the people acting on their judgments understand exactly how much weight those judgments can bear.

A military commander deciding whether to commit forces needs to know whether the intelligence supporting that decision is well-corroborated fact or an inference drawn from fragmentary reporting. A corporate security director evaluating whether to evacuate staff needs to understand the difference between “an attack is possible” and “an attack is likely within 48 hours.” A law enforcement analyst briefing a tactical team on a suspect’s probable location needs to convey whether that assessment rests on three independent sources or one unverified tip. When analysts fail to communicate uncertainty clearly, consumers fill in the gaps with their own assumptions, and those assumptions tend to confirm whatever the consumer already believes (Heuer 1999).

ICD 203 defines the standard: analytic products should indicate and explain the basis for the uncertainties associated with major analytic judgments, specifically the likelihood of occurrence of an event or development, and the analyst’s confidence in the basis for this judgment (ICD 203 2015). Two distinct concepts sit inside that requirement: the probability that something will happen, and the strength of the evidentiary basis for that probability estimate. Keeping them separate is foundational to applying the standard correctly, and the distinction creates problems for analysts and consumers alike when it collapses.

Likelihood and Confidence

Likelihood addresses probability: how likely is it that a specific event or development will occur? Confidence addresses the evidentiary and analytical foundation: how strong is the basis for the judgment itself? These operate independently. An analyst can have high confidence in a low-likelihood judgment when the evidence base is robust and the sources are reliable but the event itself remains improbable. An analyst can also assess an event as highly likely while holding only moderate confidence, because the available evidence points strongly in one direction but the sources are thin or the analytical methods have known limitations.

ICD 203 is explicit about keeping these concepts separate in written products. Products that express an analyst’s confidence in an assessment using a confidence level must not combine a confidence level and a degree of likelihood in the same sentence (ICD 203 2015). When likelihood and confidence appear together in the same construction, consumers lose the ability to parse which dimension the analyst is qualifying. “We assess with moderate confidence that an attack is very likely” raises an immediate question: does the analyst think the attack is probable but isn’t sure about the evidence, or is the evidence solid but the probability is harder to pin down? Separating the two into distinct statements eliminates that confusion.

The Likelihood Scale

ICD 203 prescribes a specific set of probability terms tied to numerical ranges (ICD 203 2015). Each tier on the scale pairs a verbal expression with a bounded percentage range, giving both the analyst and the consumer a shared anchor for what the language means.

These ranges exist because without them, the same word means different things to different people. Sherman Kent was one of the first to recognize this problem. He documented how policymakers interpreted the phrase “serious possibility” in a national estimate and found that their interpretations varied wildly (Heuer 1999). Decades later, the problem persists. A report stating there is “little chance” of an attack against an embassy will be read differently by an ambassador who assumes that means a one-in-a-hundred probability than by one who reads it as one-in-four; the term is consistent with either interpretation, and both ambassadors would respond very differently (Heuer 1999). The ICD 203 scale constrains this ambiguity by assigning each term a bounded range, so that “likely” always means 55 to 80 percent and “very unlikely” always means 5 to 20 percent.

Analysts are strongly encouraged not to mix terms from different rows of the scale within a single product (ICD 203 2015). If a product uses “probable” in one paragraph and “likely” in another to describe the same assessed probability, the reader has to guess whether the analyst is making a distinction or just varying word choice. Products that do mix terms must include a disclaimer noting the terms indicate the same assessment of probability. The simpler path is to pick one term per probability level and use it consistently throughout.

Heuer argued for supplementing verbal probability terms with numerical ranges inserted parenthetically to clarify key analytical points (Heuer 1999). Verbal expressions of uncertainty have long been recognized as sources of ambiguity and misunderstanding, and a parenthetical probability range removes any remaining question about what the analyst intends. A sentence that reads “the group is likely (55 to 80 percent) to attempt an attack within the next six months” leaves less room for misinterpretation than one that says “the group will likely attempt an attack within the next six months.” The IC’s adoption of bounded probability ranges in ICD 203 reflects this principle at the institutional level, even if individual analysts don’t always include the numbers in their text.

Confidence Levels

Where likelihood addresses what’s expected to happen, confidence addresses how solid the analytical basis is for expecting it. Defense intelligence defines three levels, each evaluated across four dimensions: the quality and corroboration of sources, the number and weight of assumptions, the strength of analytical inferences and methods, and the extent of intelligence gaps (JP 2-0 2022).

Reading across any row shows what a given confidence level actually means. Reading down any column shows how that single factor degrades from high to low confidence. Confidence is a composite judgment across multiple dimensions, and the table makes that composite structure visible in a way that a bare label cannot. An analyst who says “we assess with moderate confidence” without explaining whether the limiting factor was weak corroboration, untested assumptions, or a significant intelligence gap gives the consumer a label without the information needed to weigh it.

ICD 203 reinforces this by requiring that analytic products note the causes of uncertainty and explain how those uncertainties affect the analysis (ICD 203 2015). The directive identifies specific causes worth surfacing: the type, currency, and amount of information available; knowledge gaps; and the nature of the issue itself. It also requires analysts to explain how uncertainties affect the analysis, including to what degree and how a judgment depends on assumptions. A confidence statement like “moderate confidence, based on limited reporting from a single corroborated source and the assumption that current leadership dynamics remain stable” gives the consumer a path to their own evaluation of the judgment’s weight. The label alone doesn’t.

Language That Fails

The most common language failures in uncertainty communication are vague formulations that feel like analysis but don’t actually commit to a probability or explain what drives the uncertainty. Two words cause disproportionate damage: “possible” and “could.” DIA’s style guide is blunt: avoid using “possible” or “could” to describe an event unless the alternative is impossible, because these words are not analytic judgments but simply statements that the event is not impossible (Defense Intelligence Agency 2015). Saying that a state actor “could” develop a specific weapons capability within five years tells the consumer nothing useful; nearly anything that isn’t physically impossible “could” happen. Heuer reinforced this point: to be useful to the decision-maker, the analyst should narrow the range of uncertainty by stating the probability of the event (Heuer 1999).

Stacking qualifiers compounds the problem. CIA guidance warns against phrases that combine unavoidable uncertainty with unnecessary confusion, citing “real possibility” and “good chance” as examples (CIA Directorate of Intelligence 1995). A sentence like “reporting may suggest that the adversary is probably considering an escalatory response” contains three layers of hedging: “may suggest,” “probably,” and “considering.” Each qualifier dilutes the judgment further, until the consumer has no clear sense of what the analyst actually thinks is going to happen. Words that already express a degree of judgment, such as “imply,” “indicate,” and “suggest,” should not be combined with additional qualifiers like “may,” “likely,” and “probably” (Defense Intelligence Agency 2015).

There’s also a subtler problem with sourcing language that masquerades as analytical judgment. CIA guidance flags formulations where an analyst’s estimative judgment is confused with a source’s opinion. The example: “Country X has turned the corner toward recovery, as indicated by a reliable clandestine source with access to the Finance Minister.” That construction blurs whether the analyst is making a judgment supported by the source, or the source is expressing a view that the analyst is passing along. The correct formulation attributes the opinion to the source and lets the analyst’s assessment stand separately: “According to a reliable clandestine source, the Finance Minister has stated that Country X has turned the corner toward recovery” (CIA Directorate of Intelligence 1995). Analysts should be particularly wary about projecting thin information as an analytic conclusion; when the informational base is limited and conclusions rest on a small number of reports, those conclusions should be attributed to the source rather than presented as the analyst’s own assessment (CIA Directorate of Intelligence 1995).

Cognitive Biases in Uncertainty Estimation

Several well-documented cognitive biases systematically distort how analysts estimate and communicate uncertainty. Overconfidence is the most pervasive. People translating feelings of certainty into probability estimates are frequently overconfident, and expertise makes the problem worse rather than better (CIA 2009). Heuer’s research found that once an experienced analyst has the minimum information necessary to make an informed judgment, obtaining additional information generally does not improve the accuracy of the estimate; it does, however, make the analyst more confident, to the point of overconfidence (Heuer 1999). An analyst who has read twenty reports on a topic may feel substantially more certain than one who has read five, even if the additional fifteen reports contained mostly redundant information that didn’t actually improve the evidentiary base.

Anchoring distorts probability estimates by making them resistant to change once established. Probability estimates are adjusted only incrementally in response to new information or further analysis, which means important analytic judgments can become anchored to weak initial information, and any caveats attached to those judgments in the past can be forgotten or ignored over time (CIA 2009). An early assessment that a threat is “unlikely” will tend to stay assessed as unlikely even as incoming reporting shifts the balance, because each new piece of evidence produces only a small upward revision rather than a fresh evaluation of the full picture. The longer a judgment has been in circulation, the harder it is to move, and the original caveats that accompanied it tend to erode with each repetition.

The availability bias shapes probability estimates based on how easily an analyst can imagine an event or recall similar instances (Heuer 1999). If no reasonable scenario comes to mind for how an event could occur, the event is deemed impossible or highly unlikely, regardless of the actual evidence base. This bias is particularly dangerous in intelligence work, where the most consequential events are by definition unusual, and an analyst’s inability to imagine a scenario doesn’t mean the scenario is implausible. Conversely, events that have vivid recent precedents will be assessed as more probable than the evidence supports, because the ease of recalling similar cases creates an inflated sense of base rate frequency.

The base-rate fallacy operates when analysts have both specific evidence about a case and numerical data summarizing many similar cases; the numerical data is commonly ignored unless it illuminates a causal relationship (Heuer 1999). A threat analyst evaluating a specific warning might focus entirely on the specifics of the current intelligence and neglect the statistical base rate of how often similar warnings have materialized in the past. If historical data shows that this type of threat warning results in an actual attack less than 5 percent of the time, that base rate should inform the probability estimate, but it rarely does when the specific intelligence feels compelling. The same dynamic affects corporate risk analysis: a due diligence team that has uncovered specific derogatory information about a potential partner may overweight that finding relative to the statistical base rate of fraud in the relevant industry, producing a probability estimate driven by the vividness of the evidence rather than its actual predictive power.

The Best-Guess Trap

Before an analyst ever starts writing, a processing shortcut has already distorted the uncertainty they’ll communicate. When working with information of uncertain accuracy or reliability, analysts tend to make a binary yes-or-no decision about whether the information is true, rather than carrying forward the actual degree of uncertainty associated with it (Heuer 1999). This “best guess” strategy treats information that the analyst is 70 or 80 percent certain about as though it were 100 percent certain, which simplifies analysis but guarantees that judgments built on that information will be overconfident. The analyst may later write a carefully calibrated likelihood statement, but if the underlying inputs were already stripped of their uncertainty during the analytical process, the calibration is an exercise in false precision.

If an analyst treats five pieces of partially reliable information as fully reliable and then builds a judgment on all five, the resulting judgment carries a compounded overconfidence that far exceeds the overconfidence associated with any single piece. The uncertainty that should have been preserved at each step disappears, and the final assessment presents itself as more certain than the evidence warrants. CIA guidance addresses this directly: the greater the degree of uncertainty attending an issue, the greater the analyst’s reliance on judgment rather than evidence, the greater the likelihood of estimative error, and the greater the need for sound reasoning and precise argumentation to align facts, assumptions, and judgments (CIA Directorate of Intelligence 1995).

This trap is equally present outside IC contexts. A corporate due diligence analyst compiling a risk assessment on an acquisition target might receive financial information from several sources of varying reliability. If the analyst treats each source as either reliable or unreliable rather than carrying forward the actual degree of confidence in each data point, the final risk assessment will overstate the certainty of its conclusions. A law enforcement intelligence analyst building a case summary from witness statements, surveillance data, and informant reporting faces the same dynamic: treating each input as either true or false, rather than probabilistic, inflates the apparent strength of the case.

Mitigation

Structured analytic techniques provide systematic countermeasures to the biases and shortcuts that distort uncertainty handling. Analysis of Competing Hypotheses requires analysts to identify all reasonable explanations for the evidence and evaluate each one against the available data, which forces attention to alternative explanations and surfaces the full uncertainty inherent in situations that are poor in data but rich in possibilities (Heuer 1999). Alternative Futures Analysis systematically explores multiple ways a situation can develop when complexity and uncertainty are high, providing an effective means of weighing multiple unknown or unknowable factors and presenting a set of plausible outcomes (CIA 2009). Subjective probability techniques quantify an analyst’s overall degree of belief in the truth of a statement or hypothesis and mitigate the potential for analysts to exploit imprecision in favor of their position (CIA 2009).

Review procedures play a distinct role. Pre-publication review brings alternative perspectives to bear on an issue and should explicitly question the mental model employed by the analyst in searching for and examining evidence (Heuer 1999). Heuer also recommends identifying milestones for future observation that may indicate events are taking a different course than expected. Analytical conclusions should always be regarded as tentative, and specifying in advance what observations would suggest a significant change in the probabilities keeps analysts from locking into a position (Heuer 1999). The IC’s formal evaluation architecture supports this: ICD 203 requires the heads of IC elements to conduct internal programs of review and evaluation of finished all-source intelligence products using the IC analytic standards as the core criteria (DODIG 2023), and the ODNI Analytic Ombuds addresses concerns regarding standards application in analytic products (ICD 203 2015).

On suspect and sensitive issues, CIA guidance recommends that analysts prepare a textbox or annex addressing the possibility that a deception operation is distorting the assessment’s conclusions. The content should convey that the possibility of deception has been taken seriously, that analytic tests to determine the likelihood of deception have been executed, and that any reasonable doubts are forthrightly reported (CIA Directorate of Intelligence 1995). Deception detection matters for uncertainty communication because an adversary conducting a successful deception operation will create evidence that makes the analyst more confident in a wrong conclusion. Without an explicit deception check, the analyst’s confidence level may be artificially inflated by information that was designed to inflate it.

Applying the Standard Outside the IC

Practitioners working outside the Intelligence Community don’t have access to ICD 203’s mandated probability scale, and most of their consumers have never encountered formal confidence level definitions. The underlying discipline transfers directly: use consistent terminology, explain what drives the uncertainty, and give the consumer enough information to calibrate the weight they place on the judgment. Consistency in the terms used and the supporting information and logic advanced is critical to success in expressing uncertainty, regardless of whether likelihood or confidence expressions are used (ICD 203 2015). The consumer doesn’t need to know that the analyst is following a specific IC framework; they need to understand how certain the analyst is, why, and what could change the picture.

Adopting a formal probability scale is one approach. A corporate intelligence team could define its own terminology standards and circulate them with analytical products, so that “probable” always means the same thing across reports and across analysts. A law enforcement fusion center could standardize likelihood language in threat assessments, reducing the variation that occurs when each analyst uses their own idiosyncratic vocabulary. The specific scale matters less than the commitment to using it consistently and explaining deviations. What kills communication is the analyst who writes “probable” in one paragraph and “likely” in another, or who writes “we assess this is likely” without explaining why the assessment doesn’t reach “very likely” or sit closer to even odds.

Confidence explanations are even more transferable because they don’t depend on any scale at all. They depend on the analyst’s willingness to describe what they know, what they don’t, and how the gaps between those two affect the reliability of the judgment. A private investigator’s written assessment that states “this conclusion rests on direct surveillance corroborated by public records, with no assumption about the subject’s future behavior” communicates confidence explicitly. An assessment that just says “we’re confident in this finding” communicates nothing the consumer can evaluate.

Closing

Uncertainty is inherent in intelligence work, and no technique or framework eliminates it. What these tools do is make uncertainty visible and specific, so the person acting on your judgment can weigh it appropriately. A threat assessment that says “likely, based on three corroborated sources, with the key assumption that current leadership remains in place” gives a police commander or a corporate security director something they can work with. One that says “this could happen” gives them nothing.

The hardest discipline here is internal. Calibrating likelihood and explaining confidence in the final product is the visible part of the work, but the real distortion happens earlier, when an analyst quietly treats partially reliable information as fully reliable and builds a judgment on a foundation that was never as solid as it looked. By the time you’re writing the assessment, the uncertainty you stripped out during analysis doesn’t come back just because you add a carefully worded caveat at the end. Carrying uncertainty forward through every stage of the process, from initial evaluation of a source through final drafting, is what makes the finished product honest rather than precisely worded.

Your consumers will make better decisions with an honest expression of what you don’t know than with a confidently stated judgment that overstates the evidence. Precision in uncertainty language isn’t a bureaucratic exercise; it’s how you keep someone from reading “possible” as “probable” and committing resources, personnel, or their own credibility based on a word you didn’t mean the way they took it.

References:

  • Central Intelligence Agency, Directorate of Intelligence. 1995. A Compendium of Analytic Tradecraft Notes. Washington, DC: Central Intelligence Agency.

  • Defense Intelligence Agency. 2015. Style Manual for Intelligence Production. Washington, DC: Defense Intelligence Agency.

  • Department of Defense Inspector General. 2023. Evaluation of the Defense Intelligence Enterprise’s Compliance with Intelligence Community Directive 203. Washington, DC: Department of Defense.

  • Heuer, Richards J., Jr. 1999. Psychology of Intelligence Analysis. Washington, DC: Center for the Study of Intelligence, Central Intelligence Agency.

  • Office of the Director of National Intelligence. 2015. Intelligence Community Directive 203: Analytic Standards. Washington, DC: Office of the Director of National Intelligence.

  • Joint Chiefs of Staff. 2022. Joint Publication 2-0: Joint Intelligence. Washington, DC: Joint Chiefs of Staff.

  • CIA 2009. A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis. Washington, DC: Central Intelligence Agency