Coverage — метрика глубины внедрения AI в реальную работу. Она показывает, какую долю eligible объёма задач, типов задач или этапов процесса AI действительно обрабатывает, а не просто сколько людей хотя бы раз открыли AI-продукт.
№85 Adoption = share of eligible users using AI. Coverage = share of eligible work touched/handled by AI. №91 Autonomy = share of autonomy-eligible tasks completed without mandatory human. №88 Acceptance = usefulness of AI outputs. №87 Time Savings = human labor delta.
Need task taxonomy, stable task identity, eligible-work rules, “AI handled” definition, process-step mapping and period/cohort dimensions. Without a work denominator, Coverage becomes a vanity count.
CONTROL: eligibility and handling definitions, weighting rules. DATA: task/case/process events, use-case labels, AI involvement, completion outcomes. RUNTIME: emit task-level events. OFFLINE: aggregate volume/type/process coverage and diagnose gaps.
Failure = counting every model call as a covered task, using all company work as denominator, confusing AI touch with successful completion, weighting tasks opaquely, or increasing coverage by routing unsuitable work into AI.
Coverage says how much work AI touches. It does not prove that AI helped, succeeded, saved time, reduced cost or should be autonomous. Those require downstream metrics and quality guardrails.
Доля eligible task instances, где AI реально участвует.
Сколько из поддерживаемых категорий задач уже имеют рабочий AI-path.
Какая часть процесса поддерживается AI на уровне конкретных функций/этапов.
Coverage с весами по time/value/importance.
Task Volume Coverage = AI-handled eligible tasks ------------------------- all eligible tasks × 100% Example: eligible tasks in week = 10,000 AI-handled tasks = 4,200 Coverage = 42%
Task completed without AI involvement.
AI provides minor suggestion, lookup or classification.
AI performs material part of the work/output.
AI completes task, though human may review/approve.
| Scenario | Volume Coverage | Type Coverage | Interpretation |
|---|---|---|---|
| AI handles one huge repetitive workflow | High | Low | Deep in one use case, narrow portfolio. |
| AI supports many rare workflows | Low | High | Broad feature surface, low real work penetration. |
| High / High | High | High | Broad and deep deployment. |
| Low / Low | Low | Low | Early-stage or low-fit rollout. |
eligible_task if: task_type in supported_task_types AND policy_allows_AI = true AND required data is available AND task is within product scope AND not test / duplicate / synthetic AND not permanently manual-only AND task occurred during active rollout
Tasks where AI may lawfully/technically provide support.
Usually smaller population constrained by risk/policy.
Coverage denominator: AI-eligible work Autonomy Rate denominator: autonomy-eligible work Do not use one denominator for both metrics.
task_type: summarize_document research_question write_social_post classify_ticket analyze_contract generate_code_patch approve_refund browser_action dimensions: business_process complexity risk_class frequency baseline_human_minutes AI_eligible autonomy_eligible
PROCESS:
receive request
↓
classify
↓
research
↓
draft
↓
verify
↓
approve
↓
publish
AI support:
classify ✓
research ✓
draft ✓
verify partial
approve ✕
publish ✕
Simple process-step coverage:
3.5 / 7 = 50%
But:
not all steps have equal workload/value.
Weighted Coverage = Σ(task_weight × AI_handled) --------------------------- Σ(task_weight × eligible) Possible weight: baseline human active minutes cost business value strategic importance Example: 100 easy tasks × 1 min = 100 10 hard tasks × 30 min = 300 If AI handles all easy tasks only: Raw coverage: 100 / 110 = 90.9% Time-weighted coverage: 100 / 400 = 25%
Estimate human active minutes per task class before AI.
Use baseline median minutes as task value/effort proxy.
Measure how much of total human-work potential AI touches.
Find strong/weak AI scenarios.
Coverage may be high in low-risk and intentionally lower in high-risk work.
Adoption + process fit vary by organization.
High overall Coverage may be driven only by easy tasks.
Coverage by complexity: LOW: 85% MEDIUM: 54% HIGH: 11% Global: 63% Interpretation: broad metric looks healthy, but AI still barely covers high-complexity work. Need: task complexity taxonomy + stable criteria.
Any substantive AI involvement.
AI-handled tasks that reached successful product outcome.
AI-handled tasks with accepted/verified outcome.
AI Attempt Rate = eligible tasks where AI path was attempted ------------------------------------------ eligible tasks Then: AI Success Conditional on Attempt = successful AI attempts ---------------------- AI-attempted eligible tasks And: Successful Coverage = Attempt Rate × Success Conditional on Attempt
10,000 eligible tasks ↓ 70% 7,000 AI attempted ↓ 85% 5,950 substantive AI-handled ↓ 90% 5,355 successful ↓ 88% 4,712 accepted / verified Metrics: Attempt Coverage = 70.0% Handled Coverage = 59.5% Successful Coverage = 53.6% Accepted Coverage = 47.1%
| Coverage High | Coverage Low | |
|---|---|---|
| Adoption High | Broad + deep. AI is embedded into real workflow. | Many users try AI, but only for a small fraction of their eligible work. |
| Adoption Low | Specialists/automation handle large task volume; user breadth is low. | Early-stage or poor fit. |
| Acceptance High | Acceptance Low | |
|---|---|---|
| Coverage High | Strong candidate for scale. Check economics/autonomy next. | AI touches too much work before quality is ready. |
| Coverage Low | Quality is good; integration/adoption/opportunity remains. | Low fit; improve capability before expansion. |
Potential saved human hours = eligible task volume × accepted coverage × average net human minutes saved / 60 Example: 50,000 eligible tasks/month × 40% accepted coverage × 5 net minutes saved = 1,666.7 hours/month
Human may still review every task.
Only among autonomy-eligible work.
Example: AI substantive coverage = 80% Autonomy rate = 25% This can be healthy: AI drafts/researches widely, but only low-risk cases are allowed to auto-complete.
Coverage increases: 20% → 70% Cost per successful operation: $0.10 → $0.42 Eligible volume: 1,000,000 tasks/month AI direct cost: at 20% ≈ $20,000 at 70% ≈ $294,000 If time/value uplift does not scale accordingly, high Coverage destroys ROI.
For each task_type: eligible volume AI attempt volume AI handled volume successful volume accepted volume average baseline minutes risk class main gap reason Gap reasons: unavailable capability poor quality missing integration policy missing data user behavior cost latency unsupported language human preference
Opportunity Coverage Gap = eligible high-value work not currently AI-handled ------------------------- all eligible high-value work Useful ranking score: opportunity_score = uncovered_volume × baseline_human_minutes × expected_AI_success × business_priority ÷ expected_cost
| Process | Eligible volume | AI handled | Coverage | Accepted coverage |
|---|---|---|---|---|
| Customer support | 120k | 96k | 80% | 71% |
| Research | 15k | 8k | 53% | 47% |
| Marketing content | 4k | 3.6k | 90% | 78% |
| Contract review | 8k | 1.2k | 15% | 12% |
Task has no AI path.
AI feature exists but usage is optional/rare.
Meaningful fraction of eligible tasks enter AI path.
AI performs substantive work at scale.
Large share of eligible work reaches useful/verified AI-supported outcome.
Coverage can approach majority of eligible volume if quality/economics support it.
AI may support research/drafting while core judgment stays human.
Coverage can be high for assistance, low for action/autonomy.
Some eligible tasks are not worth AI cost/integration complexity.
Unexpected drop after release/integration failure.
Eligible work entering AI path.
AI path reaches successful completion.
Useful/verified outcomes across eligible work.
High vs low complexity coverage difference.
Coverage shifts into risk classes unexpectedly.
One large workflow dominating global coverage.
Whether AI handles lots of low-value work only.
Metric: Task Volume Coverage Purpose: measure share of real eligible work receiving substantive AI assistance. Numerator: eligible task instances with AI handling level >= L2 Denominator: all AI-eligible task instances Exclusions: test/synthetic duplicate/replayed unsupported/manual-only outside rollout scope Window: weekly / rolling 28d Mandatory slices: task_type process risk_class complexity team/cohort product version Companion metrics: Attempt Coverage Successful Coverage Accepted Coverage Adoption Acceptance Time Savings Cost per Operation Owner: AI Product Analytics Version: coverage_v1
{
"event_id":"EVT-...",
"event_type":"task_completed",
"task_id":"TASK-...",
"tenant_id":"...",
"use_case":"research_question",
"process":"marketing_research",
"complexity":"medium",
"risk_class":"low",
"eligibility":{
"ai_eligible":true,
"autonomy_eligible":true,
"eligibility_version":"v3"
},
"ai":{
"attempted":true,
"handling_level":"L2",
"successful":true
},
"outcome":{
"accepted":true
},
"versions":{
"product":"2026.09.1",
"workflow":"research-v4"
}
}
coverage_task( task_id tenant_id occurred_at task_type process complexity risk_class ai_eligible autonomy_eligible eligibility_version ai_attempted ai_handling_level ai_successful accepted baseline_human_minutes business_value_weight product_version )
SELECT
COUNT(*) FILTER (
WHERE ai_eligible
AND ai_handling_level IN ('L2','L3')
)::float
/ NULLIF(
COUNT(*) FILTER (WHERE ai_eligible),
0
) AS coverage,
COUNT(*) FILTER (
WHERE ai_eligible
AND ai_successful
)::float
/ NULLIF(
COUNT(*) FILTER (WHERE ai_eligible),
0
) AS successful_coverage
FROM coverage_task
WHERE occurred_at >= :start
AND occurred_at < :end;
All eligible work must be observable, including work completed without AI.
Use-case/risk/complexity labels accurate enough for slicing.
Consistent definition across product surfaces.
Retries/subtasks do not inflate task count.
Suppose logs contain: 5,000 AI tasks But business system does not emit non-AI tasks. Then you know: AI task volume = 5,000 You do NOT know: Coverage. Could be: 5,000 / 5,500 = 91% or: 5,000 / 100,000 = 5% Need denominator source: ticket system CRM document workflow ERP project tracker process database
From BPMN/IDEF0/product spec.
Which steps actually use AI in production.
Designed automation exists but users/processes bypass it—or vice versa.
| Question | Metric |
|---|---|
| Does this task type have an AI path? | Task-Type Coverage |
| How many real task instances use AI? | Task Volume Coverage |
| Where in workflow is AI used? | Process-Step Coverage |
| How much baseline human effort is covered? | Time-Weighted Coverage |
| How much eligible work reaches successful AI outcome? | Successful Coverage |
| How much reaches accepted/verified outcome? | Accepted Coverage |
For each use case: 1. Eligibility volume 2. Current Coverage 3. Acceptance 4. Time Savings 5. Cost / success 6. Risk / autonomy boundary 7. Coverage gap 8. Integration effort 9. Expected uplift 10. Next experiment Priority: high-value uncovered work with proven quality and sustainable economics Not: highest uncovered volume regardless of fit.
Use case: research_question Eligible volume: 8,400 tasks / month Attempted: 5,900 → Attempt Coverage 70.2% Substantive AI-handled: 5,200 → Coverage 61.9% Successful: 4,760 → Successful Coverage 56.7% Accepted: 4,150 → Accepted Coverage 49.4% Acceptance conditional on AI success: 87.2% Baseline human minutes: 18 min Net time saved: 9 min / accepted AI task Potential realized time saved: 622.5 hours / month Risk: low/medium Cost per success: $0.42
DATA 1. task_id 2. task_type 3. occurred_at 4. AI-eligible flag 5. AI attempted 6. AI handling level 7. AI successful 8. accepted / verified 9. risk class 10. complexity 11. product version MVP METRICS 1. eligible task volume 2. Attempt Coverage 3. Task Volume Coverage 4. Successful Coverage 5. Accepted Coverage 6. Coverage by use case 7. Coverage by complexity 8. raw vs baseline-time-weighted coverage GUARDRAILS 1. Acceptance Rate 2. Time Savings 3. Cost per Successful Operation 4. severe error / policy metrics
| Question | Status |
|---|---|
| What is the real work unit? | Required |
| Can we observe all eligible work, including non-AI work? | Required |
| What exactly makes a task AI-eligible? | Required |
| What exactly counts as AI-handled? | Required |
| Are retries/subagents collapsed into one business task? | Required |
| Is successful/accepted coverage shown beside raw coverage? | Recommended |
| Are use case/risk/complexity slices available? | Required for management decisions |
| Are weights transparent and versioned? | Required if weighted coverage used |
| Is Coverage paired with quality/economics guardrails? | Required for scale decisions |
You are designing Coverage analytics for an AI system.
1. Define the real unit of work:
task, case, document, ticket, transaction or process step.
2. Define AI-eligible work explicitly.
3. Ensure the denominator includes non-AI work from a system of record.
4. Distinguish:
AI attempted,
AI touched,
AI substantive handling,
AI completed,
AI successful,
AI accepted/verified.
5. Define the primary Coverage threshold.
6. Compute:
Attempt Coverage,
Task Volume Coverage,
Successful Coverage,
Accepted Coverage.
7. Add:
Task-Type Coverage,
Process-Step Coverage,
Weighted Coverage where useful.
8. Segment by:
use case,
risk class,
complexity,
team/cohort,
product/workflow version.
9. Keep AI-eligibility separate from autonomy-eligibility.
10. Do not count model calls, retries or subagents as separate business tasks.
11. Show raw and weighted metrics together.
12. Pair Coverage with:
Acceptance Rate,
Time Savings,
Cost per Operation,
risk/quality guardrails.
13. Compute official metrics in SQL/code.
14. Use LLM only to explain gaps or help classify taxonomy,
never as the authoritative metric calculator.
| Question | Answer |
|---|---|
| Separate component? | N/A. Coverage is a governed metric; analytics/event systems implement it. |
| Minimum 80% value? | Task identity + eligible work denominator + AI handling level + raw/successful/accepted coverage + use-case and complexity slices. |
| When overkill? | Prototype without a real repeatable task/process denominator. |
| Trigger? | “How much of our work is AI actually touching?”, “Are we broad or deep?”, “Where is the uncovered opportunity?”. |
| How to measure uplift? | Coverage expansion only counts as positive when acceptance, time savings, cost/success and risk guardrails remain healthy. |
| Can rules/code replace LLM? | Yes. Eligibility and calculation should be deterministic. LLM may help classify task taxonomy/gap reasons if validated. |
ALL REAL WORK ↓ IN SCOPE? POLICY ALLOWS AI? SUPPORTED TASK TYPE? ↓ AI-ELIGIBLE WORK ↓ AI PATH ATTEMPTED? ↓ ATTEMPT COVERAGE ↓ AI PERFORMS SUBSTANTIVE WORK? ↓ TASK VOLUME COVERAGE ↓ AI PATH SUCCESSFUL? ↓ SUCCESSFUL COVERAGE ↓ OUTPUT ACCEPTED / VERIFIED? ↓ ACCEPTED COVERAGE ══════════════════════════════════ ALSO MEASURE: TASK-TYPE COVERAGE how many scenarios? PROCESS-STEP COVERAGE where in workflow? WEIGHTED COVERAGE how much effort/value? COMPLEXITY COVERAGE only easy tasks or hard too? RISK-CLASS COVERAGE is expansion safe? ══════════════════════════════════ COVERAGE + ADOPTION ADOPTION: how many eligible users? COVERAGE: how much eligible work? ══════════════════════════════════ DO NOT COUNT: MODEL CALLS as TASKS AI TOUCH as SUCCESS ATTEMPT as ACCEPTED OUTCOME ALL COMPANY WORK as ELIGIBLE AUTONOMY-ELIGIBLE as SAME DENOMINATOR ══════════════════════════════════ THE CENTRAL QUESTION IS NOT: “HOW MANY AI FEATURES DO WE HAVE?” IT IS: “WHAT SHARE OF THE REAL WORK THAT AI COULD REASONABLY HELP WITH IS ACTUALLY BEING HANDLED BY AI — AND HOW MUCH OF THAT REACHES A SUCCESSFUL, ACCEPTED, ECONOMICALLY USEFUL OUTCOME?”
A. Related ECC ideas. Context-as-cache, scoped memory, lifecycle hooks, selective capabilities, feature flags, deterministic enforcement, provider-neutral adapters and eval-gated learning are applied only where relevant to №86 Coverage.
B–E. Existing boundary and placement. The existing conceptual boundary, class METRIC, default N/A and owner Evals / Analytics / Management remain authoritative. Runtime/control/data/offline placement is unchanged; durable state stays outside model context.
F–H. Hooks and contracts. Use bounded PRE_MODEL/POST_MODEL, PRE_TOOL/POST_TOOL, CHECKPOINT and TASK_COMPLETED events as applicable. Illustrative fields and canonical contracts are defined in NEW_CONTRACTS_SPEC.md; no universal schema is implied.
I–J. Security and evaluation. Host-side schema, permission, secret, budget, idempotency and audit checks take precedence over LLM output. Optional mechanisms require a feature flag and WITH/WITHOUT ablation; measure quality, acceptance, correction, latency, cost, escalations and severe errors.
K–L. Task profiles and cross-references. A TaskProfile selects the relevant skill, tool/context slice, memory scope and enforcement profile independently from FAST/STANDARD/DEEP. See cross-reference map, hook spec and ablation plan. Provider adapters remain outside the core.