64 / PRODUCTION / ADMISSION CONTROL · FAIRNESS · COST · CAPACITY

RATE LIMITS
/ QUOTAS / BUDGETS.

Rate Limit ограничивает скорость потребления ресурса. Quota ограничивает объём доступного ресурса в окне или по entitlement. Budget ограничивает допустимую стоимость, токены, compute, tool calls или иной измеримый расход для task, user, tenant, workflow или системы.

Главный принцип: система должна решать, можно ли принять следующую единицу работы до того, как дорогой ресурс уже потрачен. Admission control защищает capacity, fairness и деньги; observability после превышения бюджета уже слишком поздно.

00. ARCHITECTURAL STATUS

МИНИМАЛЬНЫЕ ЛИМИТЫ НУЖНЫ ПО УМОЛЧАНИЮ

AI workloads имеют variable cost и легко создают runaway loops: retries, multi-agent fan-out, tool recursion, huge contexts, bulk ingestion. Поэтому production-система должна иметь хотя бы request/concurrency/cost/token ceilings и provider-aware admission.

TYPEPRODUCTIONAdmission / resource control layer.

DEFAULTONМинимальные ceilings работают всегда.

ENABLE WHENALWAYS MINIMUMAdvanced hierarchical controls as scale grows.

SEPARATE COMPONENTYESLogical limiter/budget service.

LIVES INPRODUCTION FABRICCross-cutting resource protection.

COMPLEXITYLOW → HIGHLocal counters → distributed reservations.

IMPLEMENT: YES

Минимум 80% ценности: per-tenant/user/API rate limit, concurrency cap, provider/model limits, task-level max calls/tokens/cost, request admission before execution, hierarchical counters, Retry-After/defer response, retry accounting, reservation→commit/release for uncertain cost, hard vs soft caps, alerts and usage metrics. Никакого LLM-agent для решения «можно ли ещё 17 запросов» не требуется.

01A. ARCHITECTURE BOUNDARIES & OPERATIONS

EXPLICIT SYSTEM CONTRACT

A. BOUNDARY WITH NEIGHBORS

№16 Test-Time Compute решает, сколько дополнительного cognitive compute стоит потратить на конкретную задачу для качества; №64 задаёт инфраструктурные/экономические ceilings и entitlement, которые нельзя превышать. №57 Queues буферизует принятую работу и регулирует worker concurrency; №64 решает admission/rate/quota before or around enqueue/execution. №63 Retry восстанавливает transient failures; все retry attempts расходуют rate/cost budgets. №48 Policies отвечает «разрешено ли действие по правилам»; №64 — «есть ли capacity/entitlement/budget». №65 Model Gateway применяет provider/model quotas на model boundary и использует №64 counters/decisions. №70 Serving владеет inference capacity/batching; №64 ограничивает demand.

B. PREREQUISITES / CROSS-REFERENCES

Prerequisites: №16 Test-Time Compute, №46 Observability, №48 Guardrails, №50 Contracts, №57 Queues, №63 Retry. Forward references: №65 Model Gateway, №68 Durable Workflow, №69 Distributed Reliability, №70 Model Serving, №76 Governance.

C. PLANE PLACEMENT

REQUEST-TIME: YES — admission/consume/reserve before expensive work. CONTROL PLANE: limit definitions, windows, hierarchy, plans/entitlements, hard/soft thresholds. DATA PLANE: counters, reservations, usage events, decisions. OFFLINE: reconciliation, billing-like aggregation, capacity planning, abuse/load tests.

D. FAILURE & OPERATIONS CONTRACT

Success: every controlled operation obtains an explicit admission decision and usage is accounted to correct scopes. Retryable: transient limiter-store failure only where fail behavior permits. Permanent: hard quota/budget exhausted until reset/change. Idempotency: usage commit/reservation requires operation_id to avoid double charging. Persist: policy/version, scope, dimensions, reserved/committed amount, decision, reset/expiry. Trace: check→reserve→execute→commit/release. Security: caller cannot choose a cheaper tenant/scope or raise own limits.

E. WHAT THIS TOPIC DOES NOT OWN

№64 не владеет cognitive search strategy, product billing invoices, authentication, queue execution, provider routing or model serving. Она владеет RESOURCE ADMISSION, FAIR-SHARE LIMITING, QUOTA ENTITLEMENT AND OPERATIONAL/ECONOMIC BUDGET ENFORCEMENT.

01. FOUR DIFFERENT CONTROLS

RATE, QUOTA, BUDGET И CONCURRENCY НЕ СМЕШИВАТЬ

RATE LIMIT

Velocity

100 requests/min, 20 tool calls/sec, 60k tokens/min.

QUOTA

Entitlement / allowance

10k requests/day, 50 GB storage, 1M documents/month.

BUDGET

Spend ceiling

$5/task, 100k tokens/run, 40 model calls/research, 3 min GPU time.

CONCURRENCY

Simultaneous load

4 active browser sessions, 8 GPU jobs, 20 concurrent model calls.

Один workload часто проходит через все четыре проверки одновременно. Например: request/minute OK, monthly quota OK, task budget OK, но concurrency full → queue/defer.

02. ADMISSION PIPELINE

СНАЧАЛА ПРОВЕРИТЬ, ПОТОМ ТРАТИТЬ

REQUEST / JOBKnown subject, tenant, action, estimated resource.

→

POLICYIs action allowed at all?

→

RATECurrent velocity permitted?

→

QUOTAEntitlement remaining?

→

BUDGETCan estimated cost fit?

→

CONCURRENCYCapacity slot available?

→

RESERVEReserve uncertain resource where needed.

→

EXECUTECommit actual usage / release remainder.

Для дешёвых deterministic calls reservation может быть unnecessary. Для дорогих model/tool operations reserve-before-execute защищает от overspend при высокой concurrency.

03. RATE LIMIT

СКОЛЬКО ЕДИНИЦ РЕСУРСА МОЖНО ПОТРЕБИТЬ ЗА ВРЕМЯ

REQUEST RATE

Requests / second

Protect API/service from bursts.

TOKEN RATE

Tokens / minute

Closer to actual model capacity/cost than request count alone.

ACTION RATE

Writes / minute

Emails, publishes, browser actions, DB writes can have separate limits.

В AI-системе «10 requests/min» мало что говорит, если один request = 500 tokens, а другой = 300k tokens + 20 tool calls. Полезны multi-dimensional limits.

05. LIMITER ALGORITHMS

ВЫБИРАТЬ ПРОСТЕЙШИЙ АЛГОРИТМ, КОТОРЫЙ ДАЁТ НУЖНУЮ СЕМАНТИКУ

Algorithm	Strength	Weakness
FIXED WINDOW	Очень простой counter per minute/hour.	Boundary burst: 100 at 12:00:59 + 100 at 12:01:00.
SLIDING WINDOW	Более ровный limit over real interval.	More state/compute.
TOKEN BUCKET	Allows controlled bursts + sustained rate.	Needs refill/atomic consume semantics.
LEAKY BUCKET	Smooths outgoing flow at steady rate.	May increase queue latency.
CONCURRENCY SEMAPHORE	Controls simultaneous in-flight work.	Doesn't limit total volume over time.

MVP: token bucket/fixed window для rate + semaphore для concurrency. Не нужен сложный distributed sliding-window system без high-QPS requirement.

06. MULTI-DIMENSIONAL RATE LIMITS

ОДНА ОПЕРАЦИЯ МОЖЕТ РАСХОДОВАТЬ НЕСКОЛЬКО RESOURCE AXES

RPM

Requests/min

Provider/API call count.

TPM

Tokens/min

Prompt + completion token throughput.

GPU

Seconds/min

Local inference accelerator capacity.

BROWSER

Sessions/min

Computer-use/browser fleet pressure.

Queries/min

Search/provider API entitlement.

WRITE

External actions/min

Publishing/email/database mutation rate.

BYTES

Ingress/egress

Uploads/downloads/storage transfer.

JOBS

Background work

Enqueues/processing rate by class.

07. QUOTAS

КВОТА — ЭТО ENTITLEMENT, А НЕ МГНОВЕННАЯ СКОРОСТЬ

PER PERIOD

Monthly/daily allowance

1M model tokens/day, 10k searches/month, 500 document imports/day.

CAPACITY

Stored resource

50 GB artifacts, 100k indexed documents, 20 active schedules.

FEATURE

Entitlement count

2 concurrent deep-research runs, 3 connected sources, 10 worker seats.

Quota can reset by time window or remain capacity-based until resources are deleted. Эти два типа не стоит моделировать одним «counter resets monthly».

08. QUOTA WINDOWS

RESET SEMANTICS ДОЛЖНА БЫТЬ ЯВНОЙ

Window	Example	Important detail
CALENDAR DAY	100k searches/day.	Which timezone defines day?
ROLLING 24H	100k over previous 24 hours.	Different semantics from calendar day.
CALENDAR MONTH	10M tokens/month.	Month boundaries and tenant billing timezone.
LIFETIME / PROJECT	500 GPU hours for experiment.	No automatic reset.
CAPACITY	100 GB currently stored.	Usage decreases after deletion.

Quota decision response should expose remaining and reset_at when meaningful, so caller can defer instead of blind retry.

09. BUDGETS

СКОЛЬКО РЕСУРСОВ СИСТЕМА ГОТОВА ПОТРАТИТЬ НА РЕЗУЛЬТАТ

TASK BUDGET

One request/run

Max 30 model calls, 120k tokens, $2 equivalent cost, 5 browser minutes.

WORKFLOW BUDGET

End-to-end process

All retries/subagents/tools share one parent budget.

USER BUDGET

Fair usage

Per-day/month usage ceiling or plan allowance.

TENANT BUDGET

Organization cap

Protects shared billing/capacity across users.

PROVIDER BUDGET

Spend/capacity allocation

Cap spend/rate to one external model/search/provider.

GLOBAL BUDGET

System safety net

Hard daily/monthly ceiling against runaway automation.

Budget hierarchy prevents subagent/workflow from treating its local allowance as independent money. Child budget must fit inside parent remaining budget.

10. №16 COGNITIVE BUDGET VS №64 RESOURCE BUDGET

КАЧЕСТВО-ОРИЕНТИРОВАННОЕ РАСПРЕДЕЛЕНИЕ И ЖЁСТКИЙ CEILING — РАЗНЫЕ УРОВНИ

№16 TEST-TIME COMPUTE

How much reasoning is worth it?

Controller decides whether another candidate, critic, search branch or stronger model is likely to improve answer enough.

№64 BUDGET

How much is allowed?

Infrastructure says: task has 100k tokens / 20 calls / $X ceiling. Cognitive controller may stop earlier, but cannot exceed ceiling.

№16 optimizes value inside budget. №64 enforces budget. Это ключевое различие.

11. HARD VS SOFT BUDGET

НЕ КАЖДЫЙ ПОРОГ ДОЛЖЕН МГНОВЕННО РУБИТЬ RUN

SOFT

Warn / degrade

At 70–80%: reduce optional enrichment, use cheaper model, stop extra candidates, alert.

HARD

Must not exceed

At 100%: reject new costly operation, except explicit reserved completion semantics.

EMERGENCY

Admin override

Rare, audited, explicit authority; not model self-override.

Soft thresholds enable graceful degradation before hard stop. Особенно полезно для deep research: сначала отключить optional branches, а не оборвать ответ в последней строке.

12. RESERVATION → COMMIT → RELEASE

ДОРОГАЯ ОПЕРАЦИЯ ИМЕЕТ НЕИЗВЕСТНУЮ ТОЧНУЮ СТОИМОСТЬ ДО ЗАВЕРШЕНИЯ

ESTIMATEExpected/max tokens/cost/resource units.

→

RESERVEAtomically hold capacity against budget.

→

EXECUTEPerform model/tool operation.

→

MEASUREActual usage known.

→

COMMITCharge actual usage.

→

RELEASEReturn unused reservation.

Без reservation 20 parallel calls могут одновременно увидеть «осталось $1» и каждая потратить по $0.50. Atomic reservation предотвращает oversubscription.

13. USAGE CONTRACT

ПОТРЕБЛЕНИЕ ДОЛЖНО БЫТЬ ACCOUNTABLE И IDEMPOTENT

{
  "operation_id": "OP-...",
  "subject": {
    "user_id": "...",
    "tenant_id": "tenant_A"
  },
  "resource": "MODEL_TOKENS",
  "dimensions": {
    "provider": "provider_A",
    "model": "model_X"
  },
  "estimated": 12000,
  "reserved": 15000,
  "actual": 10842,
  "budget_refs": [
    "budget://task/...",
    "budget://tenant/..."
  ],
  "rate_scope": "tenant_A:model_X",
  "status": "COMMITTED",
  "timestamp": "..."
}

WHY OPERATION_ID

Don't double-charge retries

Operation identity позволяет:

idempotent reserve;
idempotent commit;
release after cancellation;
reconcile unknown outcome;
attribute retries;
audit cost by run/user/tenant.

Attempt_id может быть отдельным от logical operation_id.

15. FAIRNESS

LIMITS ЗАЩИЩАЮТ НЕ ТОЛЬКО INFRASTRUCTURE, НО И СОСЕДНИХ USERS

PER USER

No noisy individual

One user cannot consume entire tenant pool.

PER TENANT

Isolation

Large organization cannot starve other tenants beyond allocated capacity.

PER CLASS

Interactive vs batch

Background indexing does not consume all interactive capacity.

PRIORITY

Policy controlled

Priority classes get reserved shares/limits, not unlimited bypass.

Fairness лучше строить через per-scope limits + queue scheduling, а не только global limiter.

16. CONCURRENCY LIMITS

RATE НЕ ЗАЩИЩАЕТ ОТ ДОЛГИХ IN-FLIGHT OPERATIONS

Resource	Concurrency example	Why
MODEL PROVIDER	20 in-flight requests	Protect provider/socket/latency and quota.
LOCAL GPU	1–4 generations	VRAM and batching capacity are finite.
BROWSER AGENTS	5 active sessions	CPU/RAM/browser fleet and site pressure.
OCR	2 GPU jobs	Prevent document ingestion from starving inference.
EXTERNAL WRITE TOOL	Small bounded concurrency	Reduce blast radius and provider abuse.

Semaphore slot должен освобождаться при success, failure, cancellation and timeout. Lease/heartbeat нужен, если distributed ownership может потеряться.

17. QUEUE, DEFER OR REJECT?

ПРЕВЫШЕНИЕ LIMIT НЕ ВСЕГДА ОЗНАЧАЕТ ОДИН И ТОТ ЖЕ RESPONSE

LIMIT

INTERACTIVE

BACKGROUND

RESET KNOWN

HARD?

ACTION

RATE

429 / short wait

queue/defer

usually

temporary

Retry-After / not_before.

CONCURRENCY

queue briefly

queue

completion-dependent

temporary

Backpressure.

PERIOD QUOTA

reject/degrade

defer to reset

yes

hard until reset

Quota exhausted.

TASK BUDGET

degrade/finish

stop optional work

no reset

hard ceiling

Return bounded result.

Queueing every rejected request can convert rate limit into unbounded backlog. Admission policy must decide whether delayed work remains useful.

18. RETRIES CONSUME BUDGET

ПОВТОРНЫЙ CALL — ЭТО НАСТОЯЩИЙ РАСХОД, А НЕ «БЕСПЛАТНАЯ ТЕХНИЧЕСКАЯ ПОПЫТКА»

LOGICAL OPERATION

User requested one research step.

→

3 ATTEMPTS

Provider timeout → retry → retry. Rate, tokens and cost consumed each time.

→

ACCOUNTING

Operation result = one logical outcome, but usage includes every real attempt.

№63 Retry Budget и №64 Resource Budget должны быть связаны. Во время outage retry amplification не должен обходить tenant/global spend ceilings.

19. FAN-OUT & SUBAGENTS

ПАРАЛЛЕЛЬНОЕ МЫШЛЕНИЕ МОЖЕТ МГНОВЕННО УМНОЖИТЬ COST

UNBOUNDED FAN-OUT

10 agents × 10 searches × 3 retries

Architectural «умность» превращается в runaway spend/capacity.

CHILD BUDGET

Allocate from parent

Subagent receives bounded calls/tokens/time/cost that fit parent remaining budget.

RETURN UNUSED

Budget reconciliation

Unused child reservation returns to parent for later steps.

Multi-agent/debate/tree search остаются conditional именно потому, что №16 determines value and №64 enforces hard resource boundaries.

20. PROVIDER LIMITS

ВНЕШНИЕ LIMITS НУЖНО МОДЕЛИРОВАТЬ ВНУТРИ СИСТЕМЫ ДО ПОЛУЧЕНИЯ 429

RPM

Request cap

Provider request/minute entitlement.

TPM

Token cap

Prompt/completion throughput entitlement.

CONCURRENCY

In-flight

Protect provider and own latency.

DAILY / SPEND

Provider-level budget

Optional operational cap under organization billing limit.

№65 Model Gateway — естественный enforcement point для model provider limits, но policy/counters принадлежат общей resource-control architecture.

21. LOCAL GPU BUDGETS

ЛОКАЛЬНАЯ МОДЕЛЬ НЕ БЕСПЛАТНА — У НЕЁ ЕСТЬ VRAM, ВРЕМЯ И ЭЛЕКТРОЭНЕРГИЯ

GPU CONCURRENCY

Slots

Max simultaneous generations/batches per device/profile.

GPU SECONDS

Compute budget

Per task/tenant/day if capacity is shared.

VRAM CLASS

Admission

Model/context request may be rejected/routed if predicted memory exceeds safe profile.

QUEUE AGE

Latency budget

If local capacity delay exceeds SLA, route/fail/defer according to №65 policy.

Self-hosted inference replaces monetary API meter with infrastructure capacity meter, но resource accounting всё равно нужен.

22. COST ESTIMATION

ПЕРЕД ADMISSION НУЖНА ХОТЯ БЫ ГРУБАЯ ВЕРХНЯЯ ОЦЕНКА

KNOWN COST

Deterministic

Tool call fixed unit, file size, embedding input token count.

BOUNDABLE COST

Max output

Input tokens known + max output tokens gives conservative model reservation.

UNCERTAIN COST

Research / agent loop

Reserve per step and re-check parent remaining budget before each expansion.

Не нужно идеально предсказывать финальную стоимость. Достаточно never start a step whose conservative reservation cannot fit hard budget.

23. BUDGET-AWARE DEGRADATION

ПРИБЛИЖЕНИЕ К ЛИМИТУ ДОЛЖНО МЕНЯТЬ СТРАТЕГИЮ

0–60%Normal configured strategy.

→

60–80%Stop optional enrichment / narrow retrieval.

→

80–95%No extra candidates; cheaper compatible route where allowed.

→

95–100%Reserve only required completion/verification.

→

HARD CAPReject new spend; return bounded partial/defer/fail.

Thresholds здесь иллюстративные. Реальные значения калибруются по evals: degradation не должна резко разрушать качество ради небольшого savings.

24. LIMIT DECISION CONTRACT

CALLER ДОЛЖЕН ПОЛУЧИТЬ НЕ ТОЛЬКО ALLOW/DENY

{
  "decision": "ALLOW|DEFER|DENY|DEGRADE",
  "reason_code": "RATE_EXCEEDED",
  "scope": "tenant_A:model_X",
  "resource": "MODEL_TOKENS",
  "requested": 12000,
  "remaining": 8300,
  "reserved": 0,
  "limit": 100000,
  "reset_at": "...",
  "retry_after_ms": 4200,
  "budget_ref": "budget://tenant/...",
  "policy_version": "limits-v7",
  "obligations": [
    "USE_CHEAPER_MODEL",
    "MAX_OUTPUT_2000"
  ]
}

WHY RICH DECISION

Enables graceful control

Caller может:

defer job до reset;
reduce max_tokens;
stop optional research branch;
route to available capacity;
show correct retry timing;
fail fast instead of hammering limiter.

25. FAIL-OPEN VS FAIL-CLOSED

ЧТО ДЕЛАТЬ, ЕСЛИ LIMITER САМ НЕДОСТУПЕН?

Control	Typical default	Reason
SECURITY / ABUSE LIMIT	FAIL CLOSED or conservative local emergency limit	Unlimited access may be unsafe.
HARD SPEND BUDGET	FAIL CLOSED	Cannot prove spend is allowed.
LOW-RISK PERFORMANCE RATE LIMIT	CONTEXTUAL	Short fail-open may preserve availability if downstream has own protection.
LOCAL CONCURRENCY	LOCAL STATE CONTINUES	Process can often enforce semaphore without central service.

Limiter failure mode must be explicit per policy class. «If Redis is down, disable all limits» is not a safe universal rule.

26. DISTRIBUTED COUNTERS

ТОЧНОСТЬ LIMITER-А МОЖЕТ БЫТЬ СИЛЬНОЙ ИЛИ ПРИБЛИЗИТЕЛЬНОЙ

STRONG

Atomic central counter

Needed for hard budget/spend/reservation where overshoot is unacceptable.

LOCAL SHARDS

Approximate allocation

Each worker gets small token allowance; lower coordination overhead but bounded overshoot.

EVENTUAL

Usage telemetry

Good for soft alerts/capacity analytics, not a hard spend gate.

Choose consistency based on consequence of overshoot. Hard $ ceiling deserves stronger atomicity than a soft «roughly 100 requests/min» protection limit.

27. ABUSE & COST ATTACKS

AI ENDPOINT МОЖНО АТАКОВАТЬ НЕ ТОЛЬКО REQUEST COUNT, НО И COST AMPLIFICATION

HUGE CONTEXT

Token amplification

One request uses enormous context/output limits.

TOOL LOOP

Recursive actions

Agent repeatedly calls search/browser/tool.

FAN-OUT

Agent multiplication

User input triggers many subagents/candidates.

RETRY AMPLIFICATION

Failure multiplication

Outage turns one request into many provider calls.

Rate limit by request count is insufficient. Enforce max context, output tokens, tool calls, branches, runtime and cost-equivalent units.

28. OBSERVABILITY

USAGE НУЖНО ВИДЕТЬ В ТЕХ ЖЕ SCOPE, В КОТОРЫХ ПРИНИМАЮТСЯ LIMIT DECISIONS

ALLOW

Admission rate

Allowed operations by resource/scope.

DENY

Limit rejects

Rate/quota/budget/concurrency rejection reasons.

UTIL

Utilization

Used / available by tenant/provider/resource window.

HEAD

Headroom

Remaining capacity before saturation.

RES

Reservations

Outstanding reserved vs committed usage.

OVR

Overshoot

Usage above hard target due to races/accounting gaps.

HOT

Top consumers

Usage by user/tenant/workflow/model/tool.

RET

Retry spend

Fraction of resource usage caused by retries.

29. TESTING

LIMITER НУЖНО ПРОВЕРЯТЬ НА ГРАНИЦАХ, ГОНКАХ И RESTARTS

BOUNDARY

N and N+1

Exactly at limit allowed; next unit gets expected decision.

BURST

Token bucket

Burst capacity and refill behave as configured.

RACE

Parallel reserve

100 concurrent requests cannot oversubscribe hard budget.

RETRY

Usage accounting

Retries consume real resources without double-charging logical bookkeeping.

TENANT

Isolation

Caller cannot forge scope or consume another tenant's entitlement.

RESET

Window transition

Calendar/rolling reset semantics are correct around boundaries.

FAILURE

Limiter unavailable

Fail-open/closed behavior matches policy class.

DEGRADE

Soft budget

Approaching cap suppresses optional work before hard failure.

Use fake clock for window/refill tests and concurrent transactions for reservation races. Rate limiting tested only sequentially is not enough.

30. FAILURE MODES

КАК LIMITS САМИ СТАНОВЯТСЯ ПРОБЛЕМОЙ

REQUEST COUNT ONLY

One huge AI request costs more than hundreds of tiny requests.

MULTI-DIMENSIONAL UNITS

GLOBAL LIMIT ONLY

One noisy tenant/user consumes all capacity.

HIERARCHICAL SCOPES

CHECK AFTER EXECUTION

Budget alarm arrives after money/GPU already spent.

ADMISSION + RESERVE

NO RETRY ACCOUNTING

Outages bypass budget through repeated calls.

COUNT ATTEMPTS

NO CONCURRENCY LIMIT

Allowed per-minute traffic still overwhelms long-running dependency.

SEMAPHORE / POOL

USER CHOOSES SCOPE

Caller spoofs tenant/plan/priority to gain resources.

HOST-DERIVED IDENTITY

QUEUE EVERYTHING

Rate rejection becomes infinite backlog of stale work.

DEFER ONLY USEFUL WORK

SOFT LIMIT ONLY

Runaway agent ignores warnings and spends indefinitely.

HARD CEILING

CENTRAL LIMITER FAILS OPEN

Limiter outage becomes unlimited-spend incident.

POLICY-SPECIFIC FAIL MODE

31. METRICS

ЧТО ИЗМЕРЯТЬ

UTIL

Limit Utilization

Usage / limit by scope, window and resource dimension.

429

Rate Reject Rate

Requests deferred/denied because velocity exceeded.

QEX

Quota Exhaustion

Tenants/users hitting period/capacity entitlement ceilings.

BEX

Budget Exhaustion

Tasks/workflows stopped/degraded by spend ceiling.

OVR

Hard-Cap Overshoot

Actual usage beyond hard ceiling. Target: zero/bounded by contract.

RET

Retry Spend Share

% tokens/calls/cost consumed by retries.

FAIR

Fairness

Capacity distribution across tenants/users/work classes.

HEAD

Provider Headroom

Remaining RPM/TPM/concurrency before saturation.

32. MVP IMPLEMENTATION

POSTGRES + LOCAL SEMAPHORES МОГУТ ЗАКРЫТЬ ОСНОВНЫЕ РИСКИ

limits/
├── policy.py
├── rate.py
├── quota.py
├── budget.py
├── reservation.py
├── concurrency.py
├── usage.py
└── tests/

limit_policies(
  policy_id,
  scope_type,
  resource,
  window_type,
  limit_value,
  hard,
  version
)

usage_counters(
  scope_key,
  resource,
  window_start,
  used,
  updated_at,
  primary key(...)
)

budget_reservations(
  operation_id primary key,
  budget_ref,
  reserved,
  committed,
  status,
  expires_at
)

admit(ctx, estimated):
  derive trusted scopes
  check policy permission
  check rate
  check quota
  check budget
  check concurrency
  reserve if needed
  return structured decision

80% VALUE MVP

Simple admission control

Trusted user/tenant scope from auth context.
Per-user + per-tenant request rate.
Per-provider/model RPM/TPM limits.
Local concurrency semaphores.
Task max model/tool calls.
Task max input/output tokens.
Tenant/global hard spend-equivalent cap.
Reservation for expensive concurrent calls.
Retry usage included.
Structured ALLOW/DEFER/DENY/DEGRADE.
Usage/limit metrics and alerts.

Redis/distributed token buckets can come later if multi-instance QPS makes PostgreSQL/local allocation insufficient.

33. WHEN TO UPGRADE

УСЛОЖНЯТЬ ТОЛЬКО ПРИ SCALE И CONTENTION

Signal	Potential upgrade
High-QPS multi-instance API	Dedicated distributed limiter / Redis-like atomic token buckets.
Strict spend caps under heavy concurrency	Strong reservation ledger / transactional budget service.
Many providers/models	Centralized enforcement in №65 Model Gateway.
Complex workflow/subagent budgets	Hierarchical budget propagation integrated with №68 workflow state.
Multi-region active-active	Regional allocations + global reconciliation; see №69.
Large local inference cluster	Capacity/admission integration with №70 serving scheduler.

34. PRACTICAL DECISION

СТОИТ ЛИ ДЕЛАТЬ ОТДЕЛЬНЫЙ КОМПОНЕНТ?

Вопрос	Ответ
Стоит ли реализовывать?	Да. Минимальные rate/concurrency/task-budget/global-cap controls нужны production AI по умолчанию.
Separate Component?	YES логически. На старте это может быть common middleware/library + PostgreSQL counters + local semaphores.
Минимум 80% ценности?	Rate + concurrency + task ceilings + tenant/global budget + provider limits + reservation + retry accounting + metrics.
Когда overkill?	Globally distributed millisecond-precise quota service для одного локального сервера и нескольких пользователей.
Trigger?	Minimum always ON; advanced hierarchy/distribution when shared capacity, cost exposure, multi-tenancy or scale appear.
Как измерить uplift?	Prevented overload/spend incidents, fairness, provider 429 reduction, budget overshoot, queue stability, retry amplification, cost predictability.
Можно ли rule/tool/code вместо LLM-agent?	Да, полностью. Enforcement must be deterministic. LLM may estimate complexity, but cannot grant itself more quota or override hard budget.

35. DESIGN RULES

ПРАВИЛА ДЛЯ РЕАЛЬНОЙ СИСТЕМЫ

RULE 01

Admit before spend

Проверять capacity/budget до дорогой операции.

RULE 02

Separate rate/quota/budget

Velocity, entitlement and spend ceiling solve different problems.

RULE 03

Limit multiple dimensions

Requests alone do not represent AI cost/capacity.

RULE 04

Hierarchy matters

Task/user/tenant/provider/global limits are checked together.

RULE 05

Reserve uncertain cost

Parallel operations must not oversubscribe hard budget.

RULE 06

Retries are real usage

Every actual attempt consumes capacity and spend.

RULE 07

Degrade before hard stop

Soft thresholds can disable optional compute safely.

RULE 08

Caller cannot choose entitlement

Scope and plan derive from trusted identity/control plane.

RULE 09

Hard means enforceable

Alerting after overspend is observability, not a budget control.

36. FINAL MAP

EVERY EXPENSIVE OPERATION PASSES THROUGH RESOURCE ADMISSION

REQUEST / JOB / SUBAGENT / TOOL CALL
        ↓
TRUSTED IDENTITY + TENANT
        ↓
POLICY:
  is the action allowed?
        ↓
RESOURCE ADMISSION:
        ↓
  RATE LIMIT
    how fast?
        ↓
  QUOTA
    how much entitlement remains?
        ↓
  BUDGET
    how much are we willing/allowed to spend?
        ↓
  CONCURRENCY
    can it run now?
        ↓
  HIERARCHICAL CHECKS
    task
    user
    tenant
    provider
    global
        ↓
ESTIMATE COST
        ↓
RESERVE IF NECESSARY
        ↓
EXECUTE
        ↓
MEASURE ACTUAL USAGE
        ↓
COMMIT
        ↓
RELEASE UNUSED RESERVATION

IF RATE FULL:
  defer / queue / 429 + Retry-After

IF CONCURRENCY FULL:
  queue / backpressure / alternate capacity

IF SOFT BUDGET NEAR:
  stop optional branches
  reduce candidates
  cheaper compatible route
  preserve required verification

IF HARD BUDGET EXHAUSTED:
  no new spend
  bounded partial / defer / explicit failure

RETRIES:
  each real attempt counts

SUBAGENTS:
  child budget comes from parent budget

PROVIDER LIMITS:
  RPM
  TPM
  concurrency
  spend/capacity

LOCAL MODELS:
  GPU slots
  GPU seconds
  VRAM class
  queue latency

CORE DISTINCTION:

№16 TEST-TIME COMPUTE:
  "IS MORE THINKING WORTH IT?"

№64 RESOURCE BUDGET:
  "IS MORE RESOURCE USE ALLOWED?"

CORE PRINCIPLE:

DO NOT WAIT UNTIL
THE BILL, GPU, PROVIDER
OR QUEUE IS ALREADY ON FIRE.

CONTROL DEMAND
BEFORE EXECUTION.

RATE PROTECTS VELOCITY.
QUOTA PROTECTS ENTITLEMENT.
BUDGET PROTECTS SPEND.
CONCURRENCY PROTECTS
IN-FLIGHT CAPACITY.

AND NONE OF THEM
SHOULD BE OVERRIDDEN
BY THE MODEL THAT
WANTS MORE RESOURCES.

ECC RETROFIT / PRACTICAL HARNESS INTEGRATION

A. Related ECC ideas. Context-as-cache, scoped memory, lifecycle hooks, selective capabilities, feature flags, deterministic enforcement, provider-neutral adapters and eval-gated learning are applied only where relevant to №64 Rate Limits / Quotas / Budgets.

B–E. Existing boundary and placement. The existing conceptual boundary, class PRODUCTION, default ON and owner Production Fabric remain authoritative. Runtime/control/data/offline placement is unchanged; durable state stays outside model context.

F–H. Hooks and contracts. Use bounded PRE_MODEL/POST_MODEL, PRE_TOOL/POST_TOOL, CHECKPOINT and TASK_COMPLETED events as applicable. Illustrative fields and canonical contracts are defined in NEW_CONTRACTS_SPEC.md; no universal schema is implied.

I–J. Security and evaluation. Host-side schema, permission, secret, budget, idempotency and audit checks take precedence over LLM output. Optional mechanisms require a feature flag and WITH/WITHOUT ablation; measure quality, acceptance, correction, latency, cost, escalations and severe errors.

K–L. Task profiles and cross-references. A TaskProfile selects the relevant skill, tool/context slice, memory scope and enforcement profile independently from FAST/STANDARD/DEEP. See cross-reference map, hook spec and ablation plan. Provider adapters remain outside the core.

RATE LIMITS/ QUOTAS / BUDGETS.

МИНИМАЛЬНЫЕ ЛИМИТЫ НУЖНЫ ПО УМОЛЧАНИЮ

EXPLICIT SYSTEM CONTRACT

RATE, QUOTA, BUDGET И CONCURRENCY НЕ СМЕШИВАТЬ

Velocity

Entitlement / allowance

Spend ceiling

Simultaneous load

СНАЧАЛА ПРОВЕРИТЬ, ПОТОМ ТРАТИТЬ

СКОЛЬКО ЕДИНИЦ РЕСУРСА МОЖНО ПОТРЕБИТЬ ЗА ВРЕМЯ

Requests / second

Tokens / minute

Writes / minute

ХОРОШИЙ DEFAULT ДЛЯ BURST + AVERAGE RATE

ВЫБИРАТЬ ПРОСТЕЙШИЙ АЛГОРИТМ, КОТОРЫЙ ДАЁТ НУЖНУЮ СЕМАНТИКУ

ОДНА ОПЕРАЦИЯ МОЖЕТ РАСХОДОВАТЬ НЕСКОЛЬКО RESOURCE AXES

Requests/min

Tokens/min

Seconds/min

Sessions/min

Queries/min

External actions/min

Ingress/egress

Background work

КВОТА — ЭТО ENTITLEMENT, А НЕ МГНОВЕННАЯ СКОРОСТЬ

Monthly/daily allowance

Stored resource

Entitlement count

RESET SEMANTICS ДОЛЖНА БЫТЬ ЯВНОЙ

СКОЛЬКО РЕСУРСОВ СИСТЕМА ГОТОВА ПОТРАТИТЬ НА РЕЗУЛЬТАТ

One request/run

End-to-end process

Fair usage

Organization cap

Spend/capacity allocation

System safety net

КАЧЕСТВО-ОРИЕНТИРОВАННОЕ РАСПРЕДЕЛЕНИЕ И ЖЁСТКИЙ CEILING — РАЗНЫЕ УРОВНИ

How much reasoning is worth it?

How much is allowed?

НЕ КАЖДЫЙ ПОРОГ ДОЛЖЕН МГНОВЕННО РУБИТЬ RUN

Warn / degrade

Must not exceed

Admin override

ДОРОГАЯ ОПЕРАЦИЯ ИМЕЕТ НЕИЗВЕСТНУЮ ТОЧНУЮ СТОИМОСТЬ ДО ЗАВЕРШЕНИЯ

ПОТРЕБЛЕНИЕ ДОЛЖНО БЫТЬ ACCOUNTABLE И IDEMPOTENT

Don't double-charge retries

USER НЕ ДОЛЖЕН ОБОЙТИ TENANT CAP, А TENANT — GLOBAL CAP

LIMITS ЗАЩИЩАЮТ НЕ ТОЛЬКО INFRASTRUCTURE, НО И СОСЕДНИХ USERS

No noisy individual

Isolation

Interactive vs batch

Policy controlled

RATE НЕ ЗАЩИЩАЕТ ОТ ДОЛГИХ IN-FLIGHT OPERATIONS

ПРЕВЫШЕНИЕ LIMIT НЕ ВСЕГДА ОЗНАЧАЕТ ОДИН И ТОТ ЖЕ RESPONSE

ПОВТОРНЫЙ CALL — ЭТО НАСТОЯЩИЙ РАСХОД, А НЕ «БЕСПЛАТНАЯ ТЕХНИЧЕСКАЯ ПОПЫТКА»

ПАРАЛЛЕЛЬНОЕ МЫШЛЕНИЕ МОЖЕТ МГНОВЕННО УМНОЖИТЬ COST

10 agents × 10 searches × 3 retries

Allocate from parent

Budget reconciliation

ВНЕШНИЕ LIMITS НУЖНО МОДЕЛИРОВАТЬ ВНУТРИ СИСТЕМЫ ДО ПОЛУЧЕНИЯ 429

Request cap

Token cap

In-flight

Provider-level budget

ЛОКАЛЬНАЯ МОДЕЛЬ НЕ БЕСПЛАТНА — У НЕЁ ЕСТЬ VRAM, ВРЕМЯ И ЭЛЕКТРОЭНЕРГИЯ

Slots

Compute budget

Admission

Latency budget

ПЕРЕД ADMISSION НУЖНА ХОТЯ БЫ ГРУБАЯ ВЕРХНЯЯ ОЦЕНКА

Deterministic

Max output

Research / agent loop

ПРИБЛИЖЕНИЕ К ЛИМИТУ ДОЛЖНО МЕНЯТЬ СТРАТЕГИЮ

CALLER ДОЛЖЕН ПОЛУЧИТЬ НЕ ТОЛЬКО ALLOW/DENY

Enables graceful control

ЧТО ДЕЛАТЬ, ЕСЛИ LIMITER САМ НЕДОСТУПЕН?

ТОЧНОСТЬ LIMITER-А МОЖЕТ БЫТЬ СИЛЬНОЙ ИЛИ ПРИБЛИЗИТЕЛЬНОЙ

Atomic central counter

Approximate allocation

RATE LIMITS
/ QUOTAS / BUDGETS.