54 / CONNECTORS / TOOL-ACTION ENGINE
54 / PRODUCTION / EXTERNAL SYSTEM ADAPTERS · AUTH · API MAPPING

CONNECTORS.

Connector — адаптер между внутренней архитектурой AI-системы и конкретной внешней системой: Gmail, Slack, CRM, Google Drive, Notion, database, ERP, GitHub, календарь, файловое хранилище или proprietary API.

Главный принцип: агент не должен знать детали каждого vendor API. Внутри системы он работает с узкими стабильными capabilities и contracts; connector переводит их в конкретные auth flows, endpoints, pagination, vendor IDs, rate limits и error semantics.
00. ARCHITECTURAL STATUS

CONNECTOR НУЖЕН ТОЛЬКО ДЛЯ ТЕХ СИСТЕМ, С КОТОРЫМИ МЫ РЕАЛЬНО РАБОТАЕМ

Это production capability, но она CONDITIONAL: локальный агент без внешних integrations может обходиться без connectors. Отдельный архитектурный «модуль R11» не нужен — connectors живут внутри Tool / Action Engine.
TYPEPRODUCTIONExternal system integration layer.
DEFAULTCONDITIONALТолько для реально подключённых systems.
ENABLE WHENEXTERNAL SYSTEMAPI, SaaS, DB, file service, business system.
SEPARATE COMPONENTNOНабор adapters внутри R07.
LIVES INR07Tool / Action Engine.
COMPLEXITYLOW → HIGHОт REST wrapper до complex enterprise adapter.
IMPLEMENT: PER INTEGRATION
Минимум 80% ценности: connector registry, stable internal capability contract, credential_ref, narrow scopes, vendor→canonical ID mapping, pagination, normalization, typed errors, retry/rate-limit handling, tenant binding и contract tests. Не строить «универсальный connector framework» раньше, чем появятся 2–3 реальные integrations.
01A. ARCHITECTURE BOUNDARIES & OPERATIONS

EXPLICIT SYSTEM CONTRACT

A. BOUNDARY WITH NEIGHBORS

№01 MCP стандартизирует, как capabilities/data могут экспонироваться AI-клиенту; №54 реализует конкретный adapter/auth/API mapping к внешней системе. №12 Tools — callable operations; connector объединяет операции одного provider/system и скрывает vendor details. №51 Permissions & Secrets владеет identity/scopes/credentials; connector использует credential_ref, но не становится secret manager. №55 Data Ingestion & Sync владеет recurring pull/push/change-tracking pipelines; connector только умеет читать/писать конкретный источник. №42 Events владеет semantics событий; connector может принять webhook и нормализовать его. №65 Model Gateway — отдельная abstraction для model providers, не business connectors.

B. PREREQUISITES / CROSS-REFERENCES

Prerequisites: №01 MCP, №12 Tools & Function Calling, №42 Events, №46 Observability, №48 Policies, №50 Contracts, №51 Permissions & Secrets, №52 Security. Forward references: №55 Ingestion & Sync, №57 Queues/Workers, №59 Broker, №61 Provenance, №63 Retry/Circuit Breakers, №64 Rate Limits/Budgets.

C. PLANE PLACEMENT

REQUEST-TIME: YES — direct read/write tool calls. CONTROL PLANE: YES — connector registry, auth status, scopes, versions, health, mappings. DATA PLANE: YES — external objects/results/events. OFFLINE: YES — contract tests, schema drift checks, credential audits, sync validation.

D. FAILURE & OPERATIONS CONTRACT

Success: canonical request deterministically maps to provider action and canonical result. Retryable: timeout, 5xx, selected throttling. Permanent: invalid scope, revoked auth, unsupported object, deterministic 4xx, mapping error. Idempotency: write calls use provider idempotency or local operation key. Persist: connector_id, tenant, credential_ref, external IDs, cursor/page token where needed, mapping version. Trace: logical capability + provider operation + sanitized response metadata.

E. WHAT THIS TOPIC DOES NOT OWN

№54 не владеет long-running synchronization, queue infrastructure, connector credentials lifecycle, global policy, business workflow, provenance system или generic MCP protocol. Она владеет THE ADAPTER BETWEEN A STABLE INTERNAL CAPABILITY AND A SPECIFIC EXTERNAL SYSTEM.

01. CONNECTOR VS TOOL VS MCP

ТРИ УРОВНЯ, КОТОРЫЕ ЧАСТО ПУТАЮТ

TOOL

Concrete operation

create_draft(), read_document(), create_issue().

Это узкая callable capability.

CONNECTOR

Provider adapter

GmailConnector содержит read/search/draft/send mapping, auth context, provider IDs, pagination, errors.

MCP

Exposure protocol

Один из способов стандартизированно представить tools/resources клиенту. MCP не отменяет provider adapter.

Архитектурно можно иметь Connector → Tools → MCP exposure, либо Connector напрямую зарегистрирован в Tool Engine без MCP. MCP — не обязательный internal dependency для каждого connector.
02. CONNECTOR ANATOMY

ЧТО ОБЫЧНО НАХОДИТСЯ ВНУТРИ ADAPTER

REGISTRATION

Identity

connector_id, provider, tenant, environment, status, capability list.

AUTH BRIDGE

Credential reference

Получает scoped credential через №51; raw secret не хранится в agent logic.

CAPABILITY MAP

Internal → vendor

create_draft → конкретный endpoint/request semantics.

NORMALIZATION

Vendor → canonical

Provider-specific response превращается в stable internal model.

PAGINATION

Cursors / pages

Page token, offset, next link скрыты за canonical iterator/result contract.

ERROR MAP

Typed failures

429/401/vendor-specific codes → TRANSIENT/RATE_LIMITED/PERMISSION/etc.

RATE / RETRY

Provider discipline

Retry-after, backoff, quotas, request budget, circuit hooks.

ID MAPPING

External references

provider object IDs ↔ canonical resource refs.

OBSERVABILITY

Trace

Logical call, provider method, status, latency, sanitized metadata.

03. STABLE INTERNAL CAPABILITY

AGENT НЕ ДОЛЖЕН ДУМАТЬ В ТЕРМИНАХ VENDOR API

AGENT / TOOL ENGINE

Нужна capability:

search_messages(query, folder, limit)

Canonical contract.

CONNECTOR

Преобразует query, folders, limits, auth, pagination, provider syntax.

Возвращает canonical objects.

PROVIDER API

Gmail / Outlook / custom mail API со своими endpoints, IDs, fields, errors.

Vendor-specific детали должны концентрироваться внутри connector. Если agent prompt знает, что «для provider X поле называется q, а для Y filter», abstraction уже протекла.
04. CANONICAL MODEL

НЕ НУЖЕН ЕДИНЫЙ UNIVERSAL OBJECT НА ВСЁ

{
  "contract": "message.v1",
  "message_ref": "msg://tenant_A/...",
  "connector_id": "mail:brand_A",
  "external_id": "provider-id",
  "thread_ref": "thread://...",
  "sender": {
    "address": "...",
    "display_name": "..."
  },
  "recipients": [...],
  "subject": "...",
  "body_ref": "artifact://...",
  "received_at": "...",
  "labels": ["inbox"],
  "provenance": {
    "provider": "gmail"
  }
}
CANONICAL ≠ LOSSLESS EVERYTHING

Normalize what the system needs

Canonical model должен покрывать shared semantics ваших use cases.

Provider-specific поля можно хранить отдельно:

  • provider_metadata;
  • raw reference;
  • extension namespace;

Не пытайтесь сразу создать «идеальную универсальную CRM/email/storage ontology».

05. CAPABILITY MAPPING

НЕ ВСЕ PROVIDERS ПОДДЕРЖИВАЮТ ОДИНАКОВЫЕ ОПЕРАЦИИ

CAPABILITY
PROVIDER A
PROVIDER B
PROVIDER C
CANONICAL
BEHAVIOR
search
native
native
limited
search(query)
Advertise limits in capability metadata.
draft
native
native
none
optional
Do not emulate silently if semantics differ.
send
native
native
native
send()
Policy/permission still applies.
thread
native
different model
none
extension
Provider-specific fallback/ref may be needed.
Connector registry должен уметь сказать системе, какие capabilities реально доступны. Model/router не должен узнавать это только после runtime 400/404.
06. AUTH HANDOFF

CONNECTOR ИСПОЛЬЗУЕТ CREDENTIAL, НО НЕ ВЛАДЕЕТ SECRET LIFECYCLE

USER / TENANTAuthenticated request context.
CONNECTOR IDmail:brand_A / crm:tenant_B.
AUTHZCan principal use capability/resource?
CREDENTIAL REFResolve via security fabric.
PROVIDER TOKENShort-lived/scoped where possible.
API CALLCredential stays in connector client.
SANITIZEReturn result, not token.
Connector config хранит credential_ref, scopes и auth status. Refresh token/password/private key не должны гулять через model context или Tool arguments.
07. CONNECTOR REGISTRY

КАК СИСТЕМА ПОНИМАЕТ, ЧТО ПОДКЛЮЧЕНО

{
  "connector_id": "mail:brand_A",
  "provider": "gmail",
  "tenant_id": "brand_A",
  "status": "READY",
  "environment": "prod",
  "credential_ref": "secret://...",
  "capabilities": {
    "messages.search": true,
    "messages.read": true,
    "drafts.create": true,
    "messages.send": false
  },
  "scopes": [
    "mail.read",
    "draft.create"
  ],
  "adapter_version": "2.3.1"
}
REGISTRY VALUE

Capability discovery without probing

Router/Tool Engine может заранее знать:

  • какие connectors доступны tenant;
  • какие операции enabled;
  • auth status;
  • read/write risk;
  • provider limits;
  • adapter version.

Это не Agent Registry №44: там регистрируются agents; здесь — external integrations.

08. PAGINATION

«ДАЙ МНЕ ВСЕ ОБЪЕКТЫ» — ПЛОХОЙ INTERNAL CONTRACT

LIMIT

Bounded fetch

Каждый call имеет explicit limit / page size / result budget.

CURSOR

Opaque continuation

Возвращать canonical next_cursor, внутри которого connector хранит/provider page token semantics.

STOP

Caller controls

Agent/workflow решает, нужно ли продолжать, исходя из coverage/budget.

BOUNDARY
№54 реализует provider pagination mechanics. №55 Ingestion & Sync позже решает, как долго и регулярно обходить весь dataset, хранить checkpoint/cursor и догонять изменения.
09. WEBHOOKS

CONNECTOR МОЖЕТ ПРИНЯТЬ PROVIDER EVENT, НО EVENT SEMANTICS ЖИВЁТ ДАЛЬШЕ

PROVIDER WEBHOOKVendor payload/signature/event id.
VERIFYSignature/source/tenant.
NORMALIZEProvider event → canonical event.
DEDUP KEYProvider event id / canonical id.
EVENT LAYER№42 decides trigger/reaction semantics.
WORKFLOWProcess task / sync / notification.
Connector verifies and translates provider-specific webhook. Он не должен содержать всю бизнес-логику «что делать, когда пришло письмо».
10. READ VS WRITE CONNECTORS

РАЗНЫЙ РИСК — РАЗНЫЙ CONTRACT

Operation classExamplesDefault controls
Read-onlySearch, read object, list metadata.Tenant/scope check, bounded results, provenance.
Draft / reversible writeCreate draft, update internal note.Permission + policy + idempotency + readback.
External side effectSend email, publish, create customer-facing object.Strong policy, exact payload, approval where required, idempotency, readback.
Destructive / privilegedDelete, change roles, admin configuration.Usually separate narrow capability; high-risk gate/HITL.
Не делать generic connector.execute(method, url, body) доступным модели. Это превращает narrow integration в raw API tunnel.
11. IDENTITY & RESOURCE MAPPING

ВНЕШНИЙ ID НЕ ДОЛЖЕН СТАТЬ ГЛОБАЛЬНЫМ ID СИСТЕМЫ

CANONICAL REF

Stable internal reference

doc://tenant_A/123, msg://tenant_A/456.

Внутри mapping хранится provider + connector + external_id.

WHY

Provider IDs collide/change

Одинаковый external_id может существовать в разных tenants/connectors. Migration provider A→B не должна ломать весь внутренний graph.

canonical_ref
   ↓
connector_id + tenant_id + object_type
   ↓
external_id
   ↓
provider API

Never:
external_id alone → global access
12. ERROR NORMALIZATION

VENDOR ERROR НЕ ДОЛЖЕН ПРОТЕКАТЬ КАК RAW CHAOS

Provider symptomCanonical classTypical action
401 token expiredAUTH_EXPIRED / RETRYABLE_AFTER_REFRESHRefresh/re-resolve credential once, then retry bounded.
403 missing scopePERMISSION_DENIEDDo not retry unchanged; reconnect/consent/admin path.
404 missing objectNOT_FOUNDCaller decides clarify/fallback/terminal.
409 version conflictCONFLICTReload current object/state before retry.
429RATE_LIMITEDRespect Retry-After/backoff/budget.
5xx / timeoutTRANSIENTBounded retry/circuit breaker.
Vendor field/schema changedADAPTER_MAPPING_ERRORFail safely; alert/regression test; connector update.
Model/orchestrator должен принимать решения по canonical error class, а не по строке provider exception.
13. RATE LIMITS

CONNECTOR ДОЛЖЕН ЗНАТЬ PROVIDER PRESSURE, НО НЕ ВЛАДЕТЬ ГЛОБАЛЬНЫМ BUDGETING

PROVIDER

Local rate semantics

Retry-After, per-user quota, batch size, request limits.

CONNECTOR

Normalize signals

Возвращает RATE_LIMITED + retry metadata и может применять bounded backoff.

SYSTEM

Global budget

№64 позже отвечает за cross-provider quotas/budgets/admission control.

14. CONNECTOR VERSIONING

PROVIDER API ИЗМЕНЯЕТСЯ НЕЗАВИСИМО ОТ ВАШЕЙ СИСТЕМЫ

ADAPTER VERSION

Track mapping

Каждый connector имеет version/build/hash.

API VERSION

Provider

Явно фиксировать provider API/version where applicable.

CONTRACT VERSION

Internal

Canonical capability schema version развивается отдельно.

MIGRATION

Adapters

Provider change ideally чинится в connector, а не в agent prompts.

Хороший connector локализует vendor churn. Если после API change приходится редактировать router, prompts, memory format и workflows, connector boundary слишком слабая.
15. SECURITY

CONNECTOR — ОДНА ИЗ ГЛАВНЫХ TRUST BOUNDARIES

INPUT

External data

Provider content остаётся untrusted/provenanced data.

AUTH

Credentials

Raw credential stays inside trusted client/runtime.

RESOURCE

Tenant binding

External object ID validated against connector/tenant context.

ACTION

PEP

Write side effect проходит policy/permission/approval before provider call.

PROMPT INJECTION
Email/document/CRM note, прочитанные через connector, могут содержать instruction-like text. Connector должен прикладывать provenance/trust metadata; он не должен превращать содержимое в privileged system instruction.
16. OBSERVABILITY

ЛОГИРОВАТЬ LOGICAL CALL И PROVIDER OUTCOME

LOGICAL

Capability

messages.search / drafts.create / crm.contact.read.

CONNECTOR

Identity

connector_id, tenant, provider, adapter version.

PROVIDER

Operation

sanitized endpoint/method/status, request id.

LATENCY

Performance

network + provider + adapter time.

RETRY

Resilience

attempt count/backoff/rate-limit state.

RESULT

Volume

items returned/written, pagination cursor, bytes where useful.

AUTH

Safe metadata

credential_ref/version/scope outcome, never raw token.

TRACE

Correlation

Link to task/tool/event/workflow trace.

17. TESTING

CONNECTOR TEST НУЖЕН НА ТРЁХ УРОВНЯХ

UNIT

Mapping

Canonical request → expected vendor request; vendor response/error → canonical result/error.

CONTRACT

Provider fixture

Recorded/fixture schemas detect provider response drift and pagination/error behavior.

INTEGRATION

Real sandbox account

Auth, scopes, read/write, idempotency, webhooks, rate limits against non-prod account.

NEGATIVE

Security

Wrong tenant/object/scope/expired auth is denied predictably.

RETRY

Failure injection

Timeout/429/5xx do not create duplicate writes.

REGRESSION

Incidents

Every provider-specific production failure becomes permanent fixture/test.

18. CONNECTORS VS INGESTION

ОДИН CALL И ПОСТОЯННЫЙ SYNC — РАЗНЫЕ RESPONSIBILITIES

CONNECTOR / №54

Can access the source

«Прочитай страницу объектов», «получи item», «создай draft», «нормализуй webhook».

Он знает provider API.

INGESTION & SYNC / №55

Keep local knowledge current

«Обойди весь dataset», «сохрани checkpoint», «догоняй изменения», «deduplicate/upsert/delete stale records».

Он использует connector как source adapter.

Не превращать каждый connector в sync engine. Иначе auth/API mapping смешивается с scheduling, checkpoints, dedupe, incremental state и backfill.
19. FAILURE MODES

КАК CONNECTORS ЛОМАЮТСЯ

RAW API IN PROMPT
Agent знает endpoints/headers/vendor query syntax.
STABLE CAPABILITY
GENERIC HTTP TOOL
Модель может вызвать любой method/url/body с broad credential.
NARROW CONNECTOR TOOLS
SECRET IN TOOL ARGS
Token проходит через model/context.
CREDENTIAL_REF + HOST
PROVIDER ERRORS LEAK
Router парсит raw exception strings.
CANONICAL ERRORS
NO TENANT BINDING
External ID используется без connector/tenant scope.
CANONICAL REF
CONNECTOR DOES WORKFLOW
Adapter содержит бизнес-логику и branching.
KEEP ADAPTER THIN
CONNECTOR DOES SYNC
Pagination/backfill/checkpoints/scheduler смешаны в adapter.
№55 OWNS SYNC
NO IDEMPOTENCY
Retry создаёт duplicate draft/post/order.
OPERATION KEY
VENDOR CHURN EVERYWHERE
API change ломает prompts/workflows/core.
LOCALIZE IN ADAPTER
20. METRICS

ЧТО ИЗМЕРЯТЬ

SR

Success Rate

Successful connector calls by capability/provider.

P95

Latency

p50/p95 provider+adapter latency by operation.

429

Rate-Limit Rate

Share of calls throttled and retry-after distribution.

AUTH

Auth Failure

Expired/revoked/missing-scope rate.

MAP

Mapping Error

Vendor response/schema changes causing adapter failures.

DUP

Duplicate Effect

Duplicate writes after retries. Target: zero.

CAP

Capability Coverage

% desired operations implemented and tested per connector.

DRIFT

Contract Drift

Provider changes detected before user-facing failure.

21. MVP IMPLEMENTATION

НАЧНИ С ОДНОГО ТОНКОГО ADAPTER НА РЕАЛЬНУЮ СИСТЕМУ

connectors/
├── registry.py
├── base.py
├── errors.py
├── models.py
├── gmail/
│   ├── client.py
│   ├── mapping.py
│   ├── tools.py
│   └── tests/
├── drive/
│   ├── client.py
│   ├── mapping.py
│   ├── tools.py
│   └── tests/
└── fixtures/

Connector:
  id
  provider
  tenant
  credential_ref
  capabilities
  adapter_version

Tool Engine:
  capability -> connector -> provider API
80% VALUE MVP

Thin adapters, strong contracts

  • Connector registry.
  • One stable internal contract per useful capability.
  • Credential reference resolved inside connector client.
  • Tenant/resource binding.
  • Provider request/response normalization.
  • Canonical typed errors.
  • Pagination cursor.
  • Idempotency for write operations.
  • Tracing + provider request id.
  • Unit fixtures + one sandbox integration account.

Не писать собственную «Zapier-платформу», пока не появится реальная потребность в десятках heterogeneous connectors.

22. PRACTICAL DECISION

СТОИТ ЛИ ДЕЛАТЬ ОТДЕЛЬНЫЙ КОМПОНЕНТ?

ВопросОтвет
Стоит ли реализовывать?Только для реально нужных external systems.
Separate Component?NO по архитектурному плану. Connector — adapter family внутри Tool / Action Engine.
Минимум 80% ценности?Stable capability contract, auth bridge, normalization, canonical errors, pagination, tenant binding, idempotency, tests.
Когда overkill?Строить generic integration DSL, visual mapper и hundreds-provider framework до появления нескольких реальных integrations.
Trigger?Нужно читать/писать конкретную внешнюю систему, скрывая vendor API details от agent/core.
Как измерить uplift?Integration defect rate, provider drift isolation, success/latency, auth failures, duplicate writes, time to add capability.
Можно ли rule/tool/code вместо LLM-agent?Да. Connector — deterministic code adapter. LLM может формировать intent, но не должен выполнять protocol mapping.
23. DESIGN RULES

ПРАВИЛА ДЛЯ РЕАЛЬНОЙ СИСТЕМЫ

RULE 01

Hide vendor details

Agent works with stable capabilities, not endpoints/query syntax.

RULE 02

Keep connector thin

Mapping/auth/pagination/errors inside; business workflow outside.

RULE 03

Credential stays inside

Model sees connector/capability metadata, not raw auth material.

RULE 04

Canonical refs

External IDs always scoped by tenant+connector+object type.

RULE 05

Normalize errors

Orchestrator reacts to typed failure classes, not vendor strings.

RULE 06

Advertise capabilities

Unavailable operations должны быть видны до call.

RULE 07

Writes are idempotent

Retry must not duplicate external side effect.

RULE 08

Connector ≠ sync engine

Recurring ingestion/checkpoints belong to №55.

RULE 09

Localize provider churn

API changes should mostly result in connector changes, not core rewrites.

24. FINAL MAP

STABLE CAPABILITY OUTSIDE — VENDOR COMPLEXITY INSIDE

USER / AGENT / WORKFLOW
        ↓
INTERNAL CAPABILITY
  messages.search
  draft.create
  document.read
  issue.create
        ↓
TOOL CONTRACT
        ↓
POLICY + PERMISSION
        ↓
CONNECTOR REGISTRY
        ↓
SELECT:
connector_id + tenant + provider
        ↓
CONNECTOR ADAPTER
  ├─ credential_ref
  ├─ auth/token handling
  ├─ vendor request mapping
  ├─ pagination
  ├─ external ID mapping
  ├─ rate-limit handling
  ├─ error normalization
  └─ response normalization
        ↓
EXTERNAL API / SAAS / DB / STORAGE
        ↓
CANONICAL RESULT / CANONICAL ERROR
        ↓
PROVENANCE + TRACE
        ↓
AGENT / WORKFLOW

FOR RECURRING DATA MOVEMENT:

CONNECTOR
   ↓
№55 DATA INGESTION & SYNC
   ↓
checkpoints / incremental updates / backfill / dedupe

CORE PRINCIPLE:

A CONNECTOR SHOULD MAKE
THE EXTERNAL SYSTEM LOOK BORING.

THE AGENT SHOULD THINK:
"READ DOCUMENT"
NOT:
"CALL PROVIDER X ENDPOINT Y
WITH VERSION Z AND PAGE TOKEN Q".

ECC RETROFIT / PRACTICAL HARNESS INTEGRATION

A. Related ECC ideas. Context-as-cache, scoped memory, lifecycle hooks, selective capabilities, feature flags, deterministic enforcement, provider-neutral adapters and eval-gated learning are applied only where relevant to №54 Connectors.

B–E. Existing boundary and placement. The existing conceptual boundary, class PRODUCTION, default CONDITIONAL and owner Tool / Action Engine remain authoritative. Runtime/control/data/offline placement is unchanged; durable state stays outside model context.

F–H. Hooks and contracts. Use bounded PRE_MODEL/POST_MODEL, PRE_TOOL/POST_TOOL, CHECKPOINT and TASK_COMPLETED events as applicable. Illustrative fields and canonical contracts are defined in NEW_CONTRACTS_SPEC.md; no universal schema is implied.

I–J. Security and evaluation. Host-side schema, permission, secret, budget, idempotency and audit checks take precedence over LLM output. Optional mechanisms require a feature flag and WITH/WITHOUT ablation; measure quality, acceptance, correction, latency, cost, escalations and severe errors.

K–L. Task profiles and cross-references. A TaskProfile selects the relevant skill, tool/context slice, memory scope and enforcement profile independently from FAST/STANDARD/DEEP. See cross-reference map, hook spec and ablation plan. Provider adapters remain outside the core.