# AI SecOps Evaluation Framework (ASEF) Source: https://secops-unpacked.ai/asef/guide (SecOps Unpacked) ASEF, the AI SecOps Evaluation Framework, is a vendor-neutral framework for evaluating AI-enabled tools and MDR services across the full security operations lifecycle: data ingestion, detection engineering, investigation and triage, response, remediation, and feedback, with Platform and Trust assessed across every zone. It supports AI SOC evaluation as well as individual SecOps tools. Define your needs to get vendor suggestions, read available technical profiles, compare known vendors, or run your own scored PoC. Technology and MDR offerings are evaluated in separate tracks. ## Full technical profiles are coming in November 2026 We are currently conducting in-depth evaluations of around 40 vendors. Full technical profiles will appear in November as reviews and vendor publication approvals are completed. You can already explore the directory, define your needs, and run your own PoC. ## Find vendors for your needs List your current stack, the work you need help with, deployment constraints, and desired autonomy. ASEF suggests vendors using their published categories and Shift Map coverage. Review the matches and check any unknowns before shortlisting. ## Understand a platform Browse the vendor directory and open a profile. Where a full technical profile is published, read its capabilities, delivery model, and autonomy without scoring anything. You do not need to create an evaluation just to understand a product. ## Compare vendors you already know Skip the suggestions and choose available published profiles in Compare. Select a saved needs profile, then compare technology or MDR offerings against the same must-haves. If a vendor has no technical profile yet, create your own evaluation and compare your saved evaluations instead. ## Run your own proof of concept Choose your vendors and link your requirements. Test the capabilities that matter to your team, assign your own scores, and record the evidence. Compare your saved evaluations using the same scope. Published vendor claims are a starting point, not your PoC results. ## Technology offerings Evaluate what the product delivers: pre-built versus customer-built capabilities, integrations, context, autonomy, response, and Platform and Trust. Level 0 is manual; higher autonomy can use automation, AI, or both. More autonomy is not automatically a better fit. ## MDR service offerings Evaluate the service your team receives: supported sources, context, investigation, escalation, hunting, detection engineering, response, and feedback. Separate what the provider's people do from what its technology automates. A human-delivered service is not the same as autonomous software. ## The same capability groups as the vendor directory Categories help discovery; they do not automatically award a capability score or prove support for every check in a group. The Shift Map organizes evaluation by lifecycle outcome. Managed service, deployment model, and open-source licensing describe delivery, not separate capability scores. Data platforms: Data Pipeline, SIEM, Data Analytics / UABA. Where security telemetry is collected, stored and queried. Detection sources: NDR, Deception Technology, ITDR, Insider Risk. Detections from network, identity, user activity and deception. Context: Threat Intel, Security Context Graph, CTEM. Intel, relationships and exposure that give an alert meaning. Investigation and response: Threat Hunting, DFIR / Forensics, CIRM / Case Management. Hunt, investigate and run the case to closure. SecOps Engineering: Automation, AI Agent Builder, Detection Engineering. Build and maintain detections, automations and agents. Testing and verification: SecOps Resilience, Attack Simulation. Check that telemetry, detections and response still work. ## Evaluate any tool on the map, not only a whole platform. ASEF lays out the full security operations lifecycle as one Shift Map: data ingestion and processing, detection engineering, investigation and triage, then response, remediation, and the feedback loop, with a Platform and Trust layer running across all of it. Every product you might buy sits somewhere on that map. You do not have to evaluate a whole AI SOC platform to use it. Score a data pipeline, a detection engineering tool, an investigation copilot, or a standalone response engine on its own. Put the zones you care about in scope, and the framework grades only those, in depth, down to their subdomains, and leaves the rest out of the verdict. ## Buy the whole stack, or decouple it. You can now buy this whole stack from one vendor, or assemble it from separate parts. Both are real strategies, and the choice keeps getting harder as more of the stack decouples. The question shifts from which platform to which tool for which job. That is why ASEF grades tools, not suites. It gives you one consistent ruler to hold against a single-vendor platform and against a specialized tool for one stage, so both get measured against your gap, not against a vendor's framing. You evaluate each product for the exact job you need it to do, and compare them on the same sheet. ## Separate delivery, autonomy, and evidence. First record delivery: out of the box, configured, customer-built, partner-delivered, preview, roadmap, or not supported. A shipped pre-built capability is not the same as a workflow or integration the customer must engineer. Unknown delivery, preview, and roadmap do not count as generally available coverage. Then record human involvement. Autonomy level 0 means a manual process, not a missing capability. Higher levels may use automation, LLM or agent reasoning, or both. Record evidence separately, distinguishing a vendor claim or published RFI from documentation, an observed demonstration, or validation in your environment. Platform and Trust uses functional support, not autonomy. ## Suggestions, published profiles, and your own findings. A suggestion is a potential fit based on published information, not an endorsement. Read available technical profiles without starting an evaluation, or choose published profiles to compare against a saved needs profile. For vendors without a published technical profile, create your own evaluations and compare them using the same scope. Your scores and PoC evidence remain your own assessment. The Research Briefing Profile uses aligned delivery and autonomy vocabulary for vendor declarations and a forty-minute demo. A published vendor profile can inform your evaluation, but does not prove every underlying capability or replace validation in your environment. Product teams can use the same criteria to understand how their products will be evaluated. ## Choose where to start - [List my needs](https://secops-unpacked.ai/asef/requirements/new) - [Explore vendor profiles](https://secops-unpacked.ai/asef/vendors) - [Compare published profiles](https://secops-unpacked.ai/asef/compare) - [Start my own evaluation](https://secops-unpacked.ai/asef#quick-evaluation) ## The SecOps Shift Map - Data ingestion and processing - Detection engineering and SecOps resilience - Investigation and Triage - Response, Remediation, and Feedback loop - Platform and Trust (cross-cutting) ## The autonomy scale - 0, Manual: A manual process with no autonomy. Delivery status separately records whether the capability exists. - 1C, Collaborator: The system assists, the human acts. - 1G, Guide: The system proposes, the human approves. - 1A, Approver: The system acts, the human can veto. - 2, Automated: Runs end to end, no human in the loop. ## How a needs-based comparison works 1. Set your requirements: Choose the outcomes, lifecycle zones, must-have capabilities, autonomy targets, and operating constraints that matter to your team. 2. Choose comparable offerings: Use suggestions or your own shortlist. Compare technology with technology and MDR with MDR. A vendor offering both can have a separate profile for each. 3. Read the claims and gaps: Check what ships out of the box, needs configuration, or requires customer development. Missing information remains unknown, not a zero score. A directory category is not proof of a tested capability. 4. Add your own findings: For a PoC, score the capabilities you actually test and keep evidence. In published-profile comparisons, record whether each requirement is met, partially met, or unmet. Neither view is a universal best-vendor ranking. ## FAQ ### Do I need to run a PoC to use ASEF? No. You can list your needs for vendor suggestions, read available full technical profiles without scoring, or compare known vendors against a saved needs profile. If you run your own PoC, choose vendors, link your requirements, record evidence, and assign your own scores. Your saved evaluations remain in Evaluate and can be compared separately. ### When will full technical profiles be available? We are currently conducting in-depth evaluations of around 40 vendors. Full technical profiles will appear in November as reviews and vendor publication approvals are completed. You can already explore the directory, define your needs, and run your own PoC. ### Can I compare MDR services and technology products? ASEF has separate technology and MDR service tracks. Compare offerings within the same track and against the same requirements. A vendor offering both can have a profile for each. Service coverage and work performed by a provider's analysts must not be presented as autonomous software capability. ### How do I evaluate AI SOC vendors? Start with your operating model, existing stack, and gaps. Select the Shift Map zones you need, from data ingestion and detection engineering through investigation, response, and feedback. Compare candidates against the same requirements. For each capability, record how it is delivered, how much human involvement remains, and the evidence supporting the claim. Keep vendor declarations, demos, and validation in your environment distinct. ### How do I compare AI SOC platforms when every vendor claims the same things? Separate delivery, autonomy, and evidence. A pre-built capability is different from a workflow the customer must build or a service delivered by a partner. Then record whether the system assists, asks for approval, acts subject to a human veto, or runs end to end. Compare these dimensions within the same scope instead of treating a capability checkbox as proof of equivalent support. ### What should go into an AI SOC RFP? Ask vendors to identify the lifecycle zones they cover, current capability delivery, customer engineering requirements, autonomy, guardrails, and available evidence. Keep roadmap and preview separate from generally available features. Use the Research Briefing Profile for vendor declarations and a focused demonstration; validate important claims in your own environment before a purchase decision. Ask what happens when a decision is wrong, who can review or reverse it, and what is logged. ### What is ASEF? ASEF, the AI SecOps Evaluation Framework, is a vendor-neutral framework for evaluating AI-enabled tools and MDR services across the full security operations lifecycle: data ingestion, detection engineering, investigation and triage, response, remediation, and feedback, with Platform and Trust assessed across every zone. It supports AI SOC evaluation as well as individual SecOps tools. Define your needs to get vendor suggestions, read available technical profiles, compare known vendors, or run your own scored PoC. Technology and MDR offerings are evaluated in separate tracks. ### Can I use ASEF to evaluate a single tool, not a whole platform? Yes. Put only the zones you care about in scope, for example detection engineering or investigation, and ASEF grades those in depth, down to their subdomains, and leaves the rest out of the verdict. You can score a data pipeline, a detection engineering tool, an investigation copilot, or a standalone response engine on its own. This matters as more of the stack decouples and buyers assemble it from separate parts. ### Who can use ASEF? Practitioners can discover vendors, understand available profiles, compare known offerings, or run their own PoC. Product teams can review the same criteria. Research briefings use aligned delivery and autonomy vocabulary, but a vendor RFI or forty-minute demo is not equivalent to a validated practitioner evaluation. Technology and MDR services have separate tracks. There is no Builder mode or universal maturity ranking. ### How does ASEF evaluate AI capabilities? Delivery records whether a capability exists and what is needed to use it. Autonomy records human involvement: 0 is manual, 1C is assistance, 1G requires human approval, 1A allows a human veto, and 2 runs end to end. Autonomy can be delivered through automation, LLM or agent reasoning, or both; it is not an LLM-only score. Platform and Trust uses a separate functional scale of None, Limited, and Full. Unassessed support remains unknown. ### How is ASEF different from ARMM? ARMM focuses on response. ASEF covers the full SecOps lifecycle and draws on ARMM's response action catalogue, but current ASEF evaluates delivery, autonomy, and evidence rather than Builder maturity tiers. You do not need a separate ARMM assessment. Platform and Trust applies across the map, and PICERL can measure operational outcomes when baseline and follow-up data exist. ### Does ASEF score or rank vendors? ASEF supports capability-level evaluation and profile-based comparisons, not a universal best-vendor ranking or one blended maturity score. Results show coverage, delivery, autonomy, and evidence within the selected scope. Practitioner ratings belong to the evaluator; vendor-declared research profiles and analyst observations remain distinguishable from independent environment validation. ### Is ASEF open to use? Yes. The framework reference and the guide are public at secops-unpacked.ai/asef/guide.