Core Concepts

URI Scheme

scio uses URIs to address data. Each URI has three components:

variant://resource/key

• Variant — The storage type (db, kv, bcs)
• Resource — The logical name (table, store, bucket)
• Key — The record identifier or path

Variant Semantics

Each variant has specific parsing rules:

For blob storage, the key can contain slashes:

bcs://documents/reports/2024/q4/summary.pdf
└─ bucket ──┘ └─────── key (full path) ──────┘

Resources

A resource is a registered data source:

type Resource struct {
    URI      string      // Full URI (e.g., "db://users")
    Variant  Variant     // Storage type
    Name     string      // Resource name
    Spec     atom.Spec   // Type metadata
    Metadata Metadata    // Annotations
}

Metadata

Resources can be annotated with metadata:

s.RegisterDatabase("db://users", db.Atomic(),
    scio.WithDescription("User accounts table"),
    scio.WithVersion("1.2"),
    scio.WithTag("owner", "auth-team"),
    scio.WithTag("pii", "true"),
)

Specs and Type Identity

scio tracks type information via specs (from the atom library). When you register a resource, its spec is captured.

FQDN Identity

Types are identified by their fully qualified domain name (FQDN):

github.com/myapp/models.User

Two resources share a type if they have the same FQDN.

Spec Relationships

scio auto-detects when multiple resources share a spec:

// Both use the User type
s.RegisterDatabase("db://users", usersDB.Atomic())
s.RegisterStore("kv://user-cache", userCache.Atomic())

// Find related resources
related := s.Related("db://users")
// Returns: [{URI: "kv://user-cache", ...}]

The Catalog

The catalog provides introspection over registered resources:

// Topology queries
s.Sources()              // All resources
s.Databases()            // All db:// resources
s.Stores()               // All kv:// resources
s.Buckets()              // All bcs:// resources

// Spec queries
s.Spec("db://users")     // Get spec for a resource
s.FindBySpec(spec)       // Find resources by spec
s.FindByField("email")   // Find resources containing field
s.Related("db://users")  // Find resources sharing the same spec

Operations

Operations are dispatched based on URI variant: