Docs/Reference/Roadmap

    Roadmap

    S3WORM is under active development. Here is what is shipping today and what is coming next.


    What Ships Today (v0.5.1)

    Core SDK (@decoperations/s3worm)

    • S3Worm Client -- Full CRUD, binary uploads, list with metadata, head object, public URLs
    • Entity / Repository -- Class-based ORM with typed repositories, auto-ID, timestamps
    • Schema-Driven Models -- JSON schema definition with typed fields, enums, defaults, refs
    • SchemaRepository -- Filtering, sorting, pagination, aggregation, soft delete, oplog, snapshots, entity links
    • Path DSL -- #namespace/@collection/(id:dynamic)/[file].json mapping to S3 keys
    • Storage Layouts -- Inline, collection, and root layouts
    • ACLs & Keychains -- Three-layer access control (bucket, model, entity), keychain backends, scoped access tokens with HMAC
    • Transport Interface -- Pluggable transport layer (S3Client, LocalFsTransport)
    • Entity Links -- Cross-entity references with population, cascading deletes, many-to-many via join models

    CLI (worm)

    CommandStatus
    worm initStable
    worm importStable (AI + heuristic modes)
    worm lintStable
    worm devStable (filesystem transport, file watcher)
    worm codegenStable (TypeScript interfaces + Zod schemas)
    worm llmsStable
    worm versionStable

    Packages

    PackageVersionStatus
    @decoperations/s3worm0.5.1Active
    @decoperations/bucket-sync0.1.0Alpha
    @decoperations/s3worm-gateway0.1.0Alpha

    Phase 1: CLI Power-Up

    Make the worm CLI a daily-driver for bucket-backed development.

    worm push

    Push local .worm/data/ to a live S3 bucket. Compares local state against remote and uploads changed files. Writes .worm sentinel files to the bucket to mark it as S3WORM-managed.

    worm pull

    Pull data from a live S3 bucket into .worm/data/. Downloads new and changed files, optionally pruning local files deleted remotely.

    worm diff

    Show a diff between local .worm/data/ and the remote S3 bucket. Displays added, modified, and deleted files with optional content diffs for JSON documents.

    worm status

    Display sync status between local and remote. Shows counts of files ahead, behind, and in conflict. Similar to git status but for your S3 data.

    worm serve

    Start a local S3-compatible HTTP endpoint (MinIO-lite). Allows applications to develop against a local S3 API without cloud credentials. Backed by the filesystem transport.

    CLI Enhancements

    EnhancementDescription
    worm import --from-s3Import schema from a live S3 bucket
    worm lint --fixAuto-fix common schema issues
    worm dev --port <n>Local S3-compatible HTTP server on a custom port
    worm init .gitignoreAuto-generate .gitignore for .worm/data/

    Phase 2: Transport Unification

    Create a unified Transport interface that S3Client, LocalFsTransport, and future backends all implement. This is partially complete -- the transport interface exists and S3Worm accepts an injected transport in its constructor.

    Remaining work:

    • Ensure all WORM operations (CRUD, list, head, binary) work identically across transports
    • Add transport health checks and connection validation
    • Add transport metadata (capabilities, region, latency)

    Phase 3: Ecosystem Integration

    Build the developer ecosystem around S3WORM.

    @decoperations/worm-react

    React hooks for S3WORM data:

    • useWormQuery(model, filter) -- Read data with automatic caching and revalidation
    • useWormMutation(model) -- Create/update/delete with optimistic updates
    • useWormSync() -- Real-time sync status and controls
    • Built on React Query / TanStack Query under the hood

    @decoperations/worm-agent

    AI agent tools for bucket operations:

    • Tool definitions for LLM function-calling (OpenAI, Claude, etc.)
    • CRUD operations as callable tools
    • Schema introspection for AI context
    • Adapted from BucketDrive's smol-agent patterns

    @decoperations/worm-types

    Shared TypeScript types extracted for cross-package use:

    • Storage provider types
    • Request/response shapes
    • Common utility types

    Dashboard App

    A web-based dashboard for managing S3WORM buckets:

    • Browse entities by model
    • Edit JSON documents inline
    • View oplog history
    • Manage ACLs and access tokens
    • Visual schema editor

    Phase 4: ACLs & Client-Side Keychains

    The ACL system and keychain are implemented in the SDK. Remaining work:

    • Per-path ACL rules in the schema DSL with full RBAC
    • Client-side keychain for credential management in the browser
    • Encrypted storage via bucket-sync's crypto-manager
    • Scoped access token UI in the dashboard
    • Key rotation and revocation workflows

    Future Phases

    These are tracked but not yet prioritized. They represent longer-term vision items.

    Integrity Layer

    Content hashing, Merkle trees, manifests, and deep verification -- concepts proven in the companion immutable.wtf Rust project:

    • BLAKE3 + SHA256 dual content hashing
    • Merkle tree construction over entity collections
    • Manifest files for collection integrity proofs
    • worm hash, worm verify, worm manifest, worm audit CLI commands
    • Deep verify: re-hash all objects and compare against manifest

    Object Lock / True WORM

    S3 Object Lock compliance for regulatory and archival use cases:

    • S3 Object Lock integration (Governance and Compliance modes)
    • Retention policies at the model and entity level
    • worm lock, worm unlock, worm hold CLI commands
    • Legal hold support
    • Retention period enforcement in the SDK

    Multi-Provider Transports

    Extend beyond S3-compatible storage:

    • Arweave -- Permanent, immutable storage for archival
    • IPFS/Filecoin -- Content-addressed distributed storage
    • Custom backends -- Pluggable transport interface for any storage provider

    Contributing

    S3WORM is open source. The best way to contribute is to:

    1. Try the SDK and CLI in your project
    2. Report issues on GitHub
    3. Submit pull requests for bug fixes or new features
    4. Join the discussion in GitHub Issues for roadmap items

    See the Getting Started guide to begin.