# 13. Threat Intelligence: What Does Good Look Like?

Chris Roberts

— 9 min read
Share
# 13. Threat Intelligence: What Does Good Look Like?

Context: Threat intelligence (TI, CTI) is one of the most misunderstood and misused disciplines in security. The term has been stretched, diluted, and rebranded so thoroughly that "threat intelligence" today can mean anything from:

  • A feed of IOCs (IPs, hashes, domains), often stale by the time you see them
  • A weekly newsletter summarizing public news
  • Strategic adversary analysis used to inform business and security strategy
  • Tactical, technical analysis of malware and campaigns to feed detection engineering
  • Operational intelligence about ongoing campaigns relevant to your sector
  • Dark web monitoring and exposure alerting (often miscategorized as TI)
  • "Brand protection" services watching for impersonation
  • LLM-generated summaries of public reporting, sold as "AI-powered intelligence"

The market reality: most "threat intelligence" sold today is data, not intelligence. The distinction matters enormously. Intelligence is analyzed, contextualized, decision-ready. Data is raw, feeds, indicators, mentions, alerts. Conflating the two is the original sin of this market.

The other reality: most companies buying threat intel can't articulate what decisions it's supposed to inform. Without intelligence requirements (the "what do I need to know, why, and what will I do about it?" framework), TI becomes expensive wallpaper. Vendors love this, because feeds-as-a-service scales infinitely.

The right question isn't "what feeds do you have?", it's "what decisions will this intelligence inform, who will consume it, and how will we measure whether it actually improved outcomes?"

14 Questions to Ask Threat Intelligence Vendors

1. "Define your intelligence, strategic, operational, tactical, or technical? Be specific, because the disciplines are different."
Why: These are the four traditional levels of TI, and they require different sources, analysts, and consumers. Strategic informs the board; operational informs SOC leadership; tactical informs defenders; technical informs detection engineering. Vendors who can't articulate which they produce are selling commodities.
Good answer: Crisp delineation, with named consumers for each level, and acknowledgment that few vendors do all four well. Red flag: "We provide intelligence at all levels" without specifics.

2. "What are your collection sources, and what's truly proprietary vs. licensed vs. open-source repackaged?"
Why: This is the provenance question. Many "threat intel platforms" are aggregators of OSINT, licensed feeds, and a thin layer of analyst commentary. Real intelligence shops have human collection (HUMINT), technical collection (sensors, telemetry, sinkholes, infiltration), and analytical depth. Force the disclosure.
Good answer: Honest mapping of proprietary vs. licensed vs. OSINT, with proportions and methodology. Red flag: "We have hundreds of sources" or "our collection is proprietary."

3. "Who are your analysts, backgrounds, languages, regional expertise, and where do they sit organizationally?"
Why: TI is a human discipline. The best TI shops have analysts with intelligence community, military, law enforcement, or deep technical research backgrounds, with native-language capability in adversary regions (Russian, Mandarin, Farsi, Korean, Spanish, etc.). Without that, you're just reading translated blog posts.
Good answer: Named analytical leadership, regional expertise, language coverage, depth of bench. Red flag: "Our analysts are highly experienced" without specifics.

4. "Walk me through your intelligence cycle, collection, processing, analysis, dissemination, feedback. How do customer requirements actually shape what you produce?"
Why: The intelligence cycle is foundational tradecraft. Vendors who can't walk through it operationally are running a content factory, not an intel shop. The feedback loop especially separates real shops from publishers, does customer demand actually shape collection?
Good answer: Articulate description of the cycle as practiced, examples of customer-driven collection, formal RFI (Request for Information) processes. Red flag: "We deliver reports based on what's important."

5. "Show me a real intelligence product, not the executive summary slide, the actual analytical work."
Why: The deliverable is everything. Real intelligence has a clear bottom line up front, sourcing, confidence assessments (high/medium/low), analysis of competing hypotheses where appropriate, and actionable recommendations. Repackaged news articles don't.
Good answer: Sanitized examples showing analytical structure, sourcing transparency, confidence language, actionable conclusions. Red flag: Glossy reports with lots of graphics and little analytical substance.

6. "How do you handle confidence levels and source reliability? Show me the methodology."
Why: Real intelligence is honest about uncertainty. Mature shops use ICD 203-style analytic standards or Admiralty Code source ratings. Immature ones present everything with equal certainty, which is itself a tell.
Good answer: Documented methodology, consistent application across products, willingness to say "we don't know." Red flag: No confidence framework, or every conclusion stated with high certainty.

7. "How is your intelligence relevant to my specific sector, geography, and threat profile? Or am I buying a generic feed?"
Why: Generic intelligence is rarely actionable. A healthcare CISO and a manufacturing CISO need different threats prioritized. Mature TI vendors build sector- and customer-specific intelligence; commodity vendors broadcast the same content to everyone.
Good answer: Sector-specific reporting, customer-specific intelligence requirements (PIRs), customizable priorities. Red flag: One-size-fits-all feeds.

8. "How do you handle adversary attribution, and what's your methodology when attribution is uncertain?"
Why: Attribution is hard, contested, and frequently wrong. Mature shops are careful, transparent about methodology, and willing to disagree publicly with other firms' attribution. Immature ones declare attribution with false confidence based on TTPs alone.
Good answer: Multi-source attribution methodology, confidence levels, willingness to acknowledge uncertainty, transparent disagreement with other reporting. Red flag: Confident attribution based on tooling overlap alone, or claiming attribution where it's not warranted.

9. "What's your timeliness, from threat emergence to customer notification? And what's the trade-off between speed and accuracy?"
Why: Speed matters, but premature reporting damages credibility. The best TI shops are fast on technical intel (IOCs, malware analysis) and patient on strategic intel (campaign attribution, motivation analysis). Vendors should articulate the trade-off honestly.
Good answer: Different SLAs for different intelligence types, transparent quality/speed trade-off, defined criteria for early warnings vs. assessed reporting. Red flag: "We're always first to report."

10. "How do you make intelligence operational, does it integrate with my SIEM, SOAR, EDR, TIP, and detection engineering workflow?"
Why: Intelligence that doesn't reach defenders is wasted. Modern TI must integrate technically, STIX/TAXII feeds, MISP integration, SIEM connectors, SOAR playbooks, MITRE ATT&CK mapping. A vendor who can't integrate is producing content, not intel.
Good answer: Native integrations with common tools, structured data formats, ATT&CK mapping, finished intel products + machine-readable feeds. Red flag: "We deliver via PDF and email."

11. "How do you support intelligence requirements, helping me define what I need to know, not just what you want to tell me?"
Why: This is where consultative TI vendors separate from publishers. Mature shops help customers develop Priority Intelligence Requirements (PIRs), the structured questions that drive collection and analysis. Without PIRs, intelligence is a stream of "interesting" content rather than decision-support.
Good answer: Formal PIR development process, dedicated intelligence advisor or analyst, periodic requirements review. Red flag: "We tell you what's important."

12. "How do you measure intelligence effectiveness, what metrics do your best customers track, and what does ROI look like?"
Why: TI is famously hard to measure. Mature programs measure decisions informed, attacks prevented or detected earlier, false alerts avoided, resources reallocated based on intel. Immature ones measure reports delivered or feeds ingested.
Good answer: Outcome-based metrics, customer success methodology, willingness to discuss what doesn't work. Red flag: Output metrics ("we delivered X reports last quarter").

13. "What's your stance on dark web monitoring, criminal forum infiltration, and the ethics of those operations?"
Why: Some of the most valuable intelligence comes from criminal underground sources. But the methods (persona development, infiltration, sometimes purchasing access) are ethically and legally complex. Vendors should articulate their approach honestly. "We monitor the dark web" means very different things at different vendors.
Good answer: Transparent methodology, legal/ethical framework, named expertise in this area. Red flag: Vague "dark web monitoring" with no methodology, OR claims of full criminal infiltration without acknowledging the complexity.

14. "How do you handle finished intelligence vs. raw data, and how do you help customers consume both appropriately?"
Why: Some customers need finished intelligence (the analyzed, contextualized assessment); others need raw data to feed their own analysts. Mature vendors offer both with clear delineation; immature ones blur the line.
Good answer: Clear product taxonomy, customer maturity assessment, recommendations on what to consume. Red flag: Selling raw feeds to customers without analytical capability, or selling only finished intel to customers who need data.

🎯 The Meta-Test

  • Q1, Q2, and Q3 are the discipline filter. A vendor who can't crisply answer these is selling content, not intelligence.
  • Q4, Q6, and Q11 are the tradecraft filter. Without intelligence cycle, confidence methodology, and PIR support, what you have is publishing.
  • Q12 is the value filter. If they can't articulate measurable customer outcomes, neither will you be able to.

💡 Honest Observations: What "Good" Looks Like

You'll likely have stronger views than I do here, but let me lay out the framework I'd give a CISO trying to evaluate this space honestly:

1. Most companies don't need threat intelligence. They need threat awareness.

This is the most important point. Real threat intelligence is expensive, requires consumption capability (analysts who can act on it), and is most valuable to organizations with mature defensive operations. For most "regular" mid-market companies, what they actually need is:

  • Awareness of threats relevant to their sector
  • Indicators they can plug into their tools
  • Occasional briefings to inform leadership

That's not really "intelligence", that's curated awareness. And it's available far more cheaply than full TI subscriptions, often for free (CISA, ISACs, vendor blogs, sector communities).

The trap: companies buy TI subscriptions because they sound mature, then never operationalize them. The intel reports go unread; the feeds generate noise; the value is zero. If you can't name the analyst on your team who reads and acts on TI, you're probably not ready to buy it.

2. The four levels of TI serve different consumers, and most vendors only do one or two well.

  • Strategic (board, executives), rare and expensive, requires deep analytical bench, geopolitical understanding, sector expertise. Few vendors do this well; even fewer customers consume it well.
  • Operational (SOC leadership, IR planning), campaign-level intel about ongoing adversary activity. The sweet spot for most mid-market consumers.
  • Tactical (SOC analysts, threat hunters), TTPs, behaviors, MITRE-mapped intel that drives detection and hunting. Requires customers with hunting capability.
  • Technical (detection engineers, IR responders), IOCs, malware analysis, technical artifacts. The most commoditized.

A vendor selling all four levels with the same product is selling content, not intelligence. The disciplines are different and require different analysts.

3. Sector-specific intelligence is dramatically more valuable than general.

Generic TI tells you Russian APTs are active. Sector-specific TI tells you that ransomware group X is targeting healthcare in your region with specific TTPs and that two of your peer organizations were hit last month. The second is actionable; the first is wallpaper.

ISACs (Information Sharing and Analysis Centers) are often the highest-ROI source of sector-specific intel, and most are dramatically underused by their members. Before buying commercial TI, maximize ISAC participation if one exists for your sector.

4. Intelligence requirements (PIRs) are the missing discipline.

The vast majority of TI failures aren't about bad vendors, they're about customers buying intel without first defining what they need to know and why. The discipline of formal intelligence requirements (PIRs, SIRs, EEIs) translated from the IC and military domains is criminally underused in private sector security.

A mature TI program:

  • Documents Priority Intelligence Requirements aligned to business and security risks
  • Reviews and updates them quarterly
  • Tasks vendors and internal analysts against them
  • Measures intelligence value against requirements
  • Treats requirements as a living artifact, not a one-time exercise

Without PIRs, you don't have a TI program. You have a subscription.

5. Attribution is overrated as a customer outcome.

CISOs love attribution because it sounds impressive in board reports. But for defensive purposes, TTPs matter more than attribution. Whether the actor is APT28 or APT29 is less actionable than knowing they use credential dumping followed by lateral movement via PsExec. Detection engineers care about behaviors; geopolitics teams care about attribution.

The trap: vendors lean heavily on attribution because it's marketable, even when it's uncertain or wrong. The signal-to-noise ratio on attribution claims is low. Treat it accordingly.

6. The IOC feed business is largely dead and most customers don't realize it.

IOCs (IPs, domains, hashes) have a useful life measured in hours to days, not weeks. By the time most "threat intel feeds" deliver them, attackers have rotated infrastructure. They still have some value (historical detection, retrospective hunting), but as a primary intel product, they're a 2010-era artifact.

Modern intelligence emphasizes TTPs and behaviors (MITRE ATT&CK), infrastructure patterns (rather than specific IPs), and early-warning indicators (such as new tooling, capability development), all of which have longer useful lives than atomic IOCs.

If a vendor's primary value proposition is "we have X million IOCs in our feed," they're selling you 2010.

7. The biggest TI consumption mistake: not connecting it to detection engineering.

Intelligence that doesn't reach detection content is wasted. The mature pattern:

  • TI identifies a relevant TTP or campaign
  • Detection engineering writes/tunes detections for that TTP
  • Threat hunting validates whether the TTP has occurred historically
  • SOC alerts when the TTP recurs
  • Feedback to TI on detection efficacy

Without this loop, intel reports get filed and forgotten. With it, intel directly improves defensive posture in days, not months.

8. What "good" actually looks like in TI:

In my view:

  • Clear intelligence requirements driven by business risk and refreshed regularly
  • Tiered consumption, different products for different consumers (board, SOC, hunters, detection engineers)
  • Sector-specific relevance, peer-organization context, ISAC participation
  • Operational integration, TI flowing into detection, hunting, IR, vulnerability prioritization
  • Honest tradecraft, confidence levels, source reliability, attribution caveats
  • Outcome-focused measurement, decisions informed, attacks prevented earlier, response improved
  • Human analyst at the core, automation supports, but doesn't replace, analytical judgment

What "good" doesn't look like:

  • A pile of feeds nobody reads
  • A weekly PDF nobody opens
  • A dashboard counting indicators ingested
  • Generic strategic reports recycled across customers
  • Confident attribution based on thin evidence
  • LLM-summarized public reporting sold as proprietary intelligence

Read more