Artifact Store — долговременное хранилище крупных или файловых результатов AI-системы: документов, изображений, аудио, архивов, datasets, generated reports, sandbox outputs, parsed page images, model-generated files, evaluation bundles и других binary/text artifacts.
artifact_ref + version/hash/metadata.artifact_ref, upload finalize step, access checks, retention/delete state, derived-from metadata, signed/authorized download path и orphan cleanup. Не нужен отдельный AI-agent: это deterministic storage service.№10 State Management хранит текущую операционную позицию процесса; Artifact Store хранит durable files/large outputs. №07 Memory хранит retained facts/preferences/episodes; artifact может быть источником/attachment, но не memory semantics. №55 Ingestion & Sync отслеживает source objects/version/checkpoints; №60 может хранить fetched raw blobs. №56 Document Parsing создаёт derived artifacts: page images, tables, IR bundles. №57 Queue передаёт artifact refs вместо giant payloads. №61 Provenance/Lineage владеет полным origin/transform graph; №60 хранит минимальные parent/source references и immutable object identity.
Prerequisites: №10 State, №46 Observability, №50 Contracts, №51 Permissions & Secrets, №55 Ingestion, №56 Parsing, №57 Queues. Forward references: №61 Provenance/Lineage, №62 Caching, №63 Retry, №66 Vector DB/Embeddings, №76 Data Governance & Privacy.
REQUEST-TIME: YES — upload/download/fetch artifact refs. CONTROL PLANE: retention classes, access policies, storage classes, quotas, lifecycle state. DATA PLANE: bytes/objects and artifact metadata. OFFLINE: garbage collection, integrity scan, migration, lifecycle transitions, backup/restore tests.
Success: bytes are durably stored, integrity verified, metadata committed, artifact_ref resolvable under access policy. Retryable: transient upload/object-store/network failures. Permanent: forbidden type/size, access deny, invalid checksum, unsupported storage class. Idempotency: finalize by artifact_id/upload_id/content hash. Persist: artifact metadata, state, object key, checksum, owner/tenant, source/parent refs, retention. Trace: create→upload→verify→finalize→read/delete. Security: raw object keys/credentials never grant cross-tenant access.
№60 не владеет domain meaning of files, knowledge indexing, memory semantics, process state, provenance graph, backup strategy for every database or user-facing file manager. Она владеет DURABLE STORAGE, IDENTITY, INTEGRITY, ACCESS AND LIFECYCLE OF ARTIFACT OBJECTS.
PDF, DOCX, image, audio, archive fetched by ingestion.
Page images, extracted tables, normalized JSON bundles, thumbnails.
Report, spreadsheet, HTML, PDF, presentation, exported dataset.
Logs, generated code, archives, plots, build/test output.
Generated or transformed multimodal outputs.
Trace exports, scorer outputs, snapshots, regression reports.
Only when state is intentionally externalized as a blob; metadata remains elsewhere.
Artifact may later be cached/referenced, but cache semantics belong to №62.
Creates artifact / upload intent.
Does not expose bucket/path details to model.
Stores bytes + metadata, returns artifact://tenant_A/....
Uses artifact_ref via authorized resolver/download API.
{
"artifact_ref": "artifact://tenant_A/01J...",
"artifact_id": "01J...",
"tenant_id": "tenant_A",
"owner_ref": "task://...",
"kind": "DOCUMENT",
"mime_type": "application/pdf",
"filename": "report.pdf",
"bytes": 1842230,
"sha256": "sha256:...",
"state": "READY",
"storage_class": "STANDARD",
"created_at": "...",
"expires_at": null,
"source_ref": "doc://...",
"parent_artifacts": [
"artifact://tenant_A/raw-..."
],
"metadata": {
"pages": 42
}
}Filename не должен быть primary identity.
ALLOCATED
↓
UPLOADING
├─ abort/timeout ─────→ ABORTED
↓
UPLOADED
↓ verify size/hash/type
VERIFYING
├─ mismatch ─────────→ REJECTED
↓
READY
├─ lifecycle expiry ─→ EXPIRED
├─ explicit delete ──→ DELETING
└─ quarantine ───────→ QUARANTINED
↓
READY / DELETING
DELETING
↓
DELETED / TOMBSTONED
READY. Нельзя считать «upload HTTP 200» достаточным proof, что object verified/finalized.После READY содержимое artifact_id не меняется. Новый результат → новый artifact_id.
Lifecycle state, labels, retention, scan status могут меняться versioned/audited способом.
Если нужен «latest report», хранить alias/pointer отдельно, а не перезаписывать bytes старого artifact.
Проверить object after upload/download/migration.
Одинаковые bytes могут физически храниться once, но logical artifact identity/provenance остаются раздельными.
Parser/thumbnail/embedding may reuse result if source hash + pipeline version match.
Можно доказать, какой exact file участвовал в run.
PUBLIC: artifact://tenant_A/01JABC... INTERNAL METADATA: artifact_id = 01JABC... storage_backend = object_store_1 object_key = tenants/tenant_A/2026/08/01JABC... DON'T EXPOSE AS PRIMARY CONTRACT: s3://bucket-prod-eu-1/ tenants/tenant_A/.../report.pdf
Backend/path может измениться из-за:
Artifact resolver скрывает эти изменения от AI/core.
| Layer | Owns | Example |
|---|---|---|
| STATE | Current operational reality. | Job RUNNING, workflow step 4, current cursor. |
| MEMORY | Retained facts/experience for future reasoning. | User preference, verified lesson, prior case. |
| ARTIFACT | Durable file/blob/output object. | PDF report, image, raw source blob, export archive. |
Service receives artifact_ref + current principal/tenant, verifies read permission.
После authorization можно выдать time-limited access to object.
Workers/tools read through trusted client using artifact_ref, not model-provided raw bucket key.
artifact_ref resolution binds tenant/owner; raw object key alone is not authorization.
Expected size/type/content policy; do not trust filename extension.
Storage keys generated host-side; no arbitrary user filesystem paths.
Use storage/platform encryption appropriate to deployment and data class.
Executable/complex file types may require scan or restricted downstream handling.
Stored external document remains untrusted content even after upload/parse.
Do not log signed URLs, storage credentials or sensitive content unnecessarily.
Deletion/retention behavior must align with №76 privacy/governance.
| Stage | Meaning |
|---|---|
| ACCESS REVOKED | Artifact becomes inaccessible immediately to normal consumers. |
| LOGICAL DELETE | Metadata state marks DELETING/DELETED; references no longer resolve. |
| PHYSICAL DELETE | Object removed from primary storage. |
| DERIVED CLEANUP | Thumbnails/parsed outputs/caches/indexed derivatives removed or tombstoned. |
| BACKUP EXPIRY | Backup copies age out according to backup/governance policy. |
TTL cleanup removes temp objects after safe window.
Periodic inventory compares storage objects with metadata DB.
Integrity scan marks CORRUPT/MISSING and triggers repair/restore path.
Recent/source artifacts needed frequently by active tasks and RAG pipelines.
Historical outputs used occasionally.
Long-term audit/reproducibility artifacts where latency can be high.
Underlying filesystem/object store protects bytes against hardware failure.
Without artifact metadata DB, durable bytes may become unaddressable.
Recovery must restore metadata/object relation and integrity.
Replication strategy depends on RTO/RPO, data residency and cost.
Artifact должен существовать согласно lifecycle/retention, даже если nobody requested it recently.
Cache entry можно удалить в любой момент; correctness should survive miss.
Artifact count by state/type/tenant/storage class.
Total/current growth by tenant/type/class.
Create/upload/finalize success and latency.
Checksum failures and missing object rate.
Resolve/download latency and error rate.
Expired/deleting/orphan backlog.
Authorization denies and cross-tenant attempts.
Cost by bytes, requests, transfer, storage tier.
Checksum mismatch prevents READY state.
Abandoned object not visible as finalized artifact.
Tenant A cannot resolve/read tenant B artifact by guessed ID.
Expired/deleted artifact stops resolving and derivatives follow policy.
Metadata exists, bytes missing → explicit integrity error, not empty file.
Artifact refs remain stable across storage backend migration.
Backup restore reconstructs metadata-object relationship.
GC removes only safe abandoned objects, respects holds/retention.
Total bytes and growth by tenant/type/class.
Artifact uploads reaching READY successfully.
Resolve/download latency by storage class.
Hash mismatch/missing-object rate. Target near zero.
Abandoned objects/metadata awaiting cleanup.
Expired items not yet transitioned/deleted.
Unauthorized resolution attempts and policy rejects.
Storage/request/egress cost by workload class.
artifact_store/ ├── service.py ├── metadata.py ├── resolver.py ├── checksum.py ├── lifecycle.py ├── cleanup.py └── tests/ artifacts( artifact_id uuid primary key, tenant_id text, owner_ref text, kind text, mime_type text, filename text, bytes bigint, sha256 text, state text, backend text, object_key text, storage_class text, source_ref text, expires_at timestamptz, created_at timestamptz, updated_at timestamptz ) filesystem MVP: /data/artifacts// later: swap backend to S3-compatible without changing artifact_ref
Не нужен отдельный microservice initially — module + DB + storage backend вполне достаточно.
| Signal | Potential upgrade |
|---|---|
| Large multi-node deployment | Shared object storage instead of local filesystem. |
| High download volume | CDN/edge delivery with signed URLs and strict auth boundary. |
| Regulated data | Key management, legal hold, retention policies, regional placement. |
| Very large archives | Multipart upload, tiered storage, lifecycle rules. |
| Cross-region disaster recovery | Replication/backup strategy matched to RPO/RTO. |
| Complex derivation graph | Dedicated provenance/lineage layer — №61. |
| Вопрос | Ответ |
|---|---|
| Стоит ли реализовывать? | Да, как только система хранит/передаёт файлы и крупные outputs. Для purely textual prototype — можно отложить. |
| Separate Component? | YES как storage responsibility. Физически initially может быть module + filesystem/object store + Postgres metadata. |
| Минимум 80% ценности? | Stable artifact_ref, immutable bytes, metadata, checksum, tenant authorization, finalize lifecycle, retention/delete, parent refs, orphan cleanup. |
| Когда overkill? | Строить multi-region object platform, CDN, archival tiers и legal-hold engine для десятка локальных файлов. |
| Trigger? | System outputs or consumes files/blobs large enough that DB rows/messages/context should carry references instead. |
| Как измерить uplift? | Storage failures, giant-payload incidents, reproducibility, upload/download reliability, integrity failures, orphan rate, storage cost, time to retrieve exact output. |
| Можно ли rule/tool/code вместо LLM-agent? | Полностью. Artifact storage is deterministic infrastructure. AI may generate artifact contents, but never owns storage integrity/access semantics. |
Queues, events, state and prompts carry artifact_ref.
Stable logical identity hides backend/storage layout.
Upload completed is not finalized until integrity checks pass.
New content → new artifact; aliases/pointers represent latest.
Integrity and reproducibility need content hash.
Stable ref does not equal permanent public access.
TTL, durable, regulated and legal-hold classes differ.
Lifecycle propagates to parsed/cache/index outputs according to policy.
Filesystem + Postgres metadata is valid until shared/object storage requirements appear.
USER / CONNECTOR / WORKER / MODEL / SANDBOX
↓
PRODUCES FILE / BLOB / LARGE OUTPUT
↓
CREATE ARTIFACT INTENT
artifact_id
tenant
owner
expected kind/type/size
↓
UPLOAD / WRITE BYTES
↓
VERIFY
exists
size
hash
type/policy
↓
FINALIZE
↓
ARTIFACT = READY
↓
RETURN STABLE
artifact://tenant/.../id
↓
OTHER COMPONENTS STORE ONLY REF
queue
workflow state
event
memory attachment
provenance
report metadata
↓
AUTHORIZED RESOLUTION
↓
DOWNLOAD / PROCESS / DERIVE
DERIVED ARTIFACTS:
raw source
├─ parsed IR
├─ page images
├─ table exports
└─ final report
LIFECYCLE:
retention
expire
quarantine
delete
orphan cleanup
restore
BOUNDARIES:
STATE
= current operational reality
MEMORY
= retained knowledge / experience
ARTIFACT STORE
= durable files / blobs / outputs
PROVENANCE
= how those artifacts and facts
were derived from one another
CORE PRINCIPLE:
IF AN OBJECT IS LARGE,
FILE-LIKE,
REUSABLE,
DOWNLOADABLE,
OR NEEDED FOR REPRODUCIBILITY—
STORE IT AS AN ARTIFACT,
GIVE IT A STABLE ID,
VERIFY ITS BYTES,
CONTROL ITS ACCESS,
AND PASS REFERENCES
EVERYWHERE ELSE.
A. Related ECC ideas. Context-as-cache, scoped memory, lifecycle hooks, selective capabilities, feature flags, deterministic enforcement, provider-neutral adapters and eval-gated learning are applied only where relevant to №60 Artifact Store.
B–E. Existing boundary and placement. The existing conceptual boundary, class PRODUCTION, default ON and owner Working Memory + State + Production Fabric remain authoritative. Runtime/control/data/offline placement is unchanged; durable state stays outside model context.
F–H. Hooks and contracts. Use bounded PRE_MODEL/POST_MODEL, PRE_TOOL/POST_TOOL, CHECKPOINT and TASK_COMPLETED events as applicable. Illustrative fields and canonical contracts are defined in NEW_CONTRACTS_SPEC.md; no universal schema is implied.
I–J. Security and evaluation. Host-side schema, permission, secret, budget, idempotency and audit checks take precedence over LLM output. Optional mechanisms require a feature flag and WITH/WITHOUT ablation; measure quality, acceptance, correction, latency, cost, escalations and severe errors.
K–L. Task profiles and cross-references. A TaskProfile selects the relevant skill, tool/context slice, memory scope and enforcement profile independently from FAST/STANDARD/DEEP. See cross-reference map, hook spec and ablation plan. Provider adapters remain outside the core.