Do I need to run a PoC to use ASEF?

No. You can list your needs for vendor suggestions, read available full technical profiles without scoring, or compare known vendors against a saved needs profile. If you run your own PoC, choose vendors, link your requirements, record evidence, and assign your own scores. Your saved evaluations remain in Evaluate and can be compared separately.

Can I compare MDR services and technology products?

ASEF has separate technology and MDR service tracks. Compare offerings within the same track and against the same requirements. A vendor offering both can have a profile for each. Service coverage and work performed by a provider's analysts must not be presented as autonomous software capability.

How do I evaluate AI SOC vendors?

Start with your operating model, existing stack, and gaps. Select the Shift Map zones you need, from data ingestion and detection engineering through investigation, response, and feedback. Compare candidates against the same requirements. For each capability, record how it is delivered, how much human involvement remains, and the evidence supporting the claim. Keep vendor declarations, demos, and validation in your environment distinct.

How do I compare AI SOC platforms when every vendor claims the same things?

Separate delivery, autonomy, and evidence. A pre-built capability is different from a workflow the customer must build or a service delivered by a partner. Then record whether the system assists, asks for approval, acts subject to a human veto, or runs end to end. Compare these dimensions within the same scope instead of treating a capability checkbox as proof of equivalent support.

What should go into an AI SOC RFP?

Ask vendors to identify the lifecycle zones they cover, current capability delivery, customer engineering requirements, autonomy, guardrails, and available evidence. Keep roadmap and preview separate from generally available features. Use the Research Briefing Profile for vendor declarations and a focused demonstration; validate important claims in your own environment before a purchase decision. Ask what happens when a decision is wrong, who can review or reverse it, and what is logged.

Can I use ASEF to evaluate a single tool, not a whole platform?

Yes. Put only the zones you care about in scope, for example detection engineering or investigation, and ASEF grades those in depth, down to their subdomains, and leaves the rest out of the verdict. You can score a data pipeline, a detection engineering tool, an investigation copilot, or a standalone response engine on its own. This matters as more of the stack decouples and buyers assemble it from separate parts.

Practitioners can discover vendors, understand available profiles, compare known offerings, or run their own PoC. Product teams can review the same criteria. Research briefings use aligned delivery and autonomy vocabulary, but a vendor RFI or forty-minute demo is not equivalent to a validated practitioner evaluation. Technology and MDR services have separate tracks. There is no Builder mode or universal maturity ranking.

How does ASEF evaluate AI capabilities?

Delivery records whether a capability exists and what is needed to use it. Autonomy records human involvement: 0 is manual, 1C is assistance, 1G requires human approval, 1A allows a human veto, and 2 runs end to end. Autonomy can be delivered through automation, LLM or agent reasoning, or both; it is not an LLM-only score. Platform and Trust uses a separate functional scale of None, Limited, and Full. Unassessed support remains unknown.

How is ASEF different from ARMM?

ARMM focuses on response. ASEF covers the full SecOps lifecycle and draws on ARMM's response action catalogue, but current ASEF evaluates delivery, autonomy, and evidence rather than Builder maturity tiers. You do not need a separate ARMM assessment. Platform and Trust applies across the map, and PICERL can measure operational outcomes when baseline and follow-up data exist.

Does ASEF score or rank vendors?

ASEF supports capability-level evaluation and profile-based comparisons, not a universal best-vendor ranking or one blended maturity score. Results show coverage, delivery, autonomy, and evidence within the selected scope. Practitioner ratings belong to the evaluator; vendor-declared research profiles and analyst observations remain distinguishable from independent environment validation.

Yes. The framework reference and the guide are public at secops-unpacked.ai/asef/guide.

AI SecOps Evaluation Framework (ASEF)

ASEF, the AI SecOps Evaluation Framework, is a vendor-neutral framework for evaluating AI-enabled tools and MDR services across the full security operations lifecycle: data ingestion, detection engineering, investigation and triage, response, remediation, and feedback, with Platform and Trust assessed across every zone. It supports AI SOC evaluation as well as individual SecOps tools. Define your needs to get vendor suggestions, read available technical profiles, compare known vendors, or run your own scored PoC. Technology and MDR offerings are evaluated in separate tracks.

Author: Filip Stojkovski · Key Contributors: Anton Chuvakin · Rafal Kitab · Andrei Cotaie · Cristian Miron · ASEF v3.2.0 · View changelog

Full technical profiles are coming in November 2026

We are currently conducting in-depth evaluations of around 40 vendors. Full technical profiles will appear in November as reviews and vendor publication approvals are completed. You can already explore the directory, define your needs, and run your own PoC.

Find vendors for your needs

List your current stack, the work you need help with, deployment constraints, and desired autonomy. ASEF suggests vendors using their published categories and Shift Map coverage. Review the matches and check any unknowns before shortlisting.

Understand a platform

Browse the vendor directory and open a profile. Where a full technical profile is published, read its capabilities, delivery model, and autonomy without scoring anything. You do not need to create an evaluation just to understand a product.

Compare vendors you already know

Skip the suggestions and choose available published profiles in Compare. Select a saved needs profile, then compare technology or MDR offerings against the same must-haves. If a vendor has no technical profile yet, create your own evaluation and compare your saved evaluations instead.

Run your own proof of concept

Choose your vendors and link your requirements. Test the capabilities that matter to your team, assign your own scores, and record the evidence. Compare your saved evaluations using the same scope. Published vendor claims are a starting point, not your PoC results.

Technology offerings

Evaluate what the product delivers: pre-built versus customer-built capabilities, integrations, context, autonomy, response, and Platform and Trust. Level 0 is manual; higher autonomy can use automation, AI, or both. More autonomy is not automatically a better fit.

MDR service offerings

Evaluate the service your team receives: supported sources, context, investigation, escalation, hunting, detection engineering, response, and feedback. Separate what the provider's people do from what its technology automates. A human-delivered service is not the same as autonomous software.

Map mode names by behaviour

Map your product’s mode to ASEF by behaviour, not its name. Assess each capability and its defined workflow separately: what runs automatically, what needs approval, and when a human takes over. Autonomy can use automation, AI, or both. Running read-only queries does not by itself prove autonomous investigation.

Compare architecture fit, not architecture fashion

For SIEM offerings, record ingestion-based, federated / in-place, or hybrid data access. Separately record storage ownership, detection execution and which components can be replaced. Decoupled is not a fourth data-access type. Separating compute from storage does not by itself prove that components are replaceable.

Federated access belongs on Left

Evaluate queries across existing SIEMs, lakes and stores on Left. Separate search from scheduled detection and cross-source correlation. Assess live endpoint collection in Middle, including answers from data that was never logged. Far Left still covers pipelines, ingestion, processing and retention.

Deployment is an explicit Platform and Trust check

Check hosting, tenancy, operating responsibility and the locations of telemetry, query results and model inference. SaaS can be single-tenant; BYOC describes application hosting, not necessarily ownership of the data store. More deployment options do not mean a higher score.

Feedback is cross-cutting

Assess how corrections and findings become controlled improvements, including approvals, versioning and verification. Investigation findings can improve detections on Left without any remediation capability on Right. In new evaluations feedback is reported separately and is not counted twice.

A source link is not a verified outcome

Attach documentation, recorded demonstrations or test reports to a capability, with a section or timestamp, product version and date. The assessor must review the source and record what it supports. Recorded evidence is distinct from a live demonstration or a test in your own environment. RFI links start private and require explicit public visibility plus approval before publication.

The same capability groups as the vendor directory

Categories help discovery; they do not automatically award a capability score or prove support for every check in a group. The Shift Map organizes evaluation by lifecycle outcome. Managed service, deployment model, and open-source licensing describe delivery, not separate capability scores.

Data platforms: Data Pipeline, SIEM, Data Analytics / UABA. Where security telemetry is collected, stored and queried.

Detection sources: NDR, Deception Technology, ITDR, Insider Risk. Detections from network, identity, user activity and deception.

Context: Threat Intel, Security Context Graph, CTEM. Intel, relationships and exposure that give an alert meaning.

Investigation and response: Threat Hunting, DFIR / Forensics, CIRM / Case Management. Hunt, investigate and run the case to closure.

SecOps Engineering: Automation, AI Agent Builder, Detection Engineering. Build and maintain detections, automations and agents.

Testing and verification: SecOps Resilience, Attack Simulation. Check that telemetry, detections and response still work.

Evaluate any tool on the map, not only a whole platform.

ASEF lays out the full security operations lifecycle as one Shift Map: data ingestion and processing, detection engineering, investigation and triage, then remediation and improvements. Platform and Trust and feedback are cross-cutting. Feedback does not imply remediation coverage. Every product you might buy sits somewhere on that map.

You do not have to evaluate a whole AI SOC platform to use it. Score a data pipeline, a detection engineering tool, an investigation copilot, or a standalone response engine on its own. Put the zones you care about in scope, and the framework grades only those, in depth, down to their subdomains, and leaves the rest out of the verdict.

Buy the whole stack, or decouple it.

You can now buy this whole stack from one vendor, or assemble it from separate parts. Both are real strategies, and the choice keeps getting harder as more of the stack decouples. The question shifts from which platform to which tool for which job.

That is why ASEF grades tools, not suites. It gives you one consistent ruler to hold against a single-vendor platform and against a specialized tool for one stage, so both get measured against your gap, not against a vendor's framing. You evaluate each product for the exact job you need it to do, and compare them on the same sheet.

Separate delivery, autonomy, and evidence.

First record delivery: out of the box, configured, customer-built, partner-delivered, preview, roadmap, or not supported. A shipped pre-built capability is not the same as a workflow or integration the customer must engineer. Unknown delivery, preview, and roadmap do not count as generally available coverage.

Then record human involvement. Autonomy level 0 means a manual process, not a missing capability. Higher levels may use automation, LLM or agent reasoning, or both. Record evidence separately, distinguishing a vendor claim or published RFI from documentation, an observed demonstration, or validation in your environment. Platform and Trust uses functional support, not autonomy.

Suggestions, published profiles, and your own findings.

A suggestion is a potential fit based on published information, not an endorsement. Read available technical profiles without starting an evaluation, or choose published profiles to compare against a saved needs profile. For vendors without a published technical profile, create your own evaluations and compare them using the same scope. Your scores and PoC evidence remain your own assessment.

The Research Briefing Profile uses aligned delivery and autonomy vocabulary for vendor declarations and a forty-minute demo. A published vendor profile can inform your evaluation, but does not prove every underlying capability or replace validation in your environment. Product teams can use the same criteria to understand how their products will be evaluated.

Choose where to start

The SecOps Shift Map

Data ingestion and processing
Detection engineering and SecOps resilience
Investigation and Triage
Response, Remediation, and Feedback loop
Platform and Trust (cross-cutting)

The autonomy scale

0, Manual: A manual process with no autonomy. Delivery status separately records whether the capability exists.
1C, System assists; human acts: The human performs the action, with system assistance.
1G, Human approves execution: The system proposes work and waits for human approval before executing.
1A, System acts; human can intervene: The system acts inside policy; a human can monitor and intervene.
2, Completes the defined workflow: Completes this scoped workflow without human intervention, not necessarily the entire investigation.

How a needs-based comparison works

Set your requirements: Choose the outcomes, lifecycle zones, must-have capabilities, autonomy targets, and operating constraints that matter to your team.
Choose comparable offerings: Use suggestions or your own shortlist. Compare technology with technology and MDR with MDR. A vendor offering both can have a separate profile for each.
Read the claims and gaps: Check what ships out of the box, needs configuration, or requires customer development. Missing information remains unknown, not a zero score. A directory category is not proof of a tested capability.
Add your own findings: For a PoC, score the capabilities you actually test and keep evidence. In published-profile comparisons, record whether each requirement is met, partially met, or unmet. Neither view is a universal best-vendor ranking.

Frequently asked questions

Do I need to run a PoC to use ASEF?: No. You can list your needs for vendor suggestions, read available full technical profiles without scoring, or compare known vendors against a saved needs profile. If you run your own PoC, choose vendors, link your requirements, record evidence, and assign your own scores. Your saved evaluations remain in Evaluate and can be compared separately.
When will full technical profiles be available?: We are currently conducting in-depth evaluations of around 40 vendors. Full technical profiles will appear in November as reviews and vendor publication approvals are completed. You can already explore the directory, define your needs, and run your own PoC.
Can I compare MDR services and technology products?: ASEF has separate technology and MDR service tracks. Compare offerings within the same track and against the same requirements. A vendor offering both can have a profile for each. Service coverage and work performed by a provider's analysts must not be presented as autonomous software capability.
How do I evaluate AI SOC vendors?: Start with your operating model, existing stack, and gaps. Select the Shift Map zones you need, from data ingestion and detection engineering through investigation, response, and feedback. Compare candidates against the same requirements. For each capability, record how it is delivered, how much human involvement remains, and the evidence supporting the claim. Keep vendor declarations, demos, and validation in your environment distinct.
How do I compare AI SOC platforms when every vendor claims the same things?: Separate delivery, autonomy, and evidence. A pre-built capability is different from a workflow the customer must build or a service delivered by a partner. Then record whether the system assists, asks for approval, acts subject to a human veto, or runs end to end. Compare these dimensions within the same scope instead of treating a capability checkbox as proof of equivalent support.
What should go into an AI SOC RFP?: Ask vendors to identify the lifecycle zones they cover, current capability delivery, customer engineering requirements, autonomy, guardrails, and available evidence. Keep roadmap and preview separate from generally available features. Use the Research Briefing Profile for vendor declarations and a focused demonstration; validate important claims in your own environment before a purchase decision. Ask what happens when a decision is wrong, who can review or reverse it, and what is logged.
What is ASEF?: ASEF, the AI SecOps Evaluation Framework, is a vendor-neutral framework for evaluating AI-enabled tools and MDR services across the full security operations lifecycle: data ingestion, detection engineering, investigation and triage, response, remediation, and feedback, with Platform and Trust assessed across every zone. It supports AI SOC evaluation as well as individual SecOps tools. Define your needs to get vendor suggestions, read available technical profiles, compare known vendors, or run your own scored PoC. Technology and MDR offerings are evaluated in separate tracks.
Can I use ASEF to evaluate a single tool, not a whole platform?: Yes. Put only the zones you care about in scope, for example detection engineering or investigation, and ASEF grades those in depth, down to their subdomains, and leaves the rest out of the verdict. You can score a data pipeline, a detection engineering tool, an investigation copilot, or a standalone response engine on its own. This matters as more of the stack decouples and buyers assemble it from separate parts.
Who can use ASEF?: Practitioners can discover vendors, understand available profiles, compare known offerings, or run their own PoC. Product teams can review the same criteria. Research briefings use aligned delivery and autonomy vocabulary, but a vendor RFI or forty-minute demo is not equivalent to a validated practitioner evaluation. Technology and MDR services have separate tracks. There is no Builder mode or universal maturity ranking.
How does ASEF evaluate AI capabilities?: Delivery records whether a capability exists and what is needed to use it. Autonomy records human involvement: 0 is manual, 1C is assistance, 1G requires human approval, 1A allows a human veto, and 2 runs end to end. Autonomy can be delivered through automation, LLM or agent reasoning, or both; it is not an LLM-only score. Platform and Trust uses a separate functional scale of None, Limited, and Full. Unassessed support remains unknown.
How is ASEF different from ARMM?: ARMM focuses on response. ASEF covers the full SecOps lifecycle and draws on ARMM's response action catalogue, but current ASEF evaluates delivery, autonomy, and evidence rather than Builder maturity tiers. You do not need a separate ARMM assessment. Platform and Trust applies across the map, and PICERL can measure operational outcomes when baseline and follow-up data exist.
Does ASEF score or rank vendors?: ASEF supports capability-level evaluation and profile-based comparisons, not a universal best-vendor ranking or one blended maturity score. Results show coverage, delivery, autonomy, and evidence within the selected scope. Practitioner ratings belong to the evaluator; vendor-declared research profiles and analyst observations remain distinguishable from independent environment validation.
Is ASEF open to use?: Yes. The framework reference and the guide are public at secops-unpacked.ai/asef/guide.