zoobzio January 10, 2025 Edit this page

scio

scio is a URI-based data catalog with atomic operations for Go. It serves as the authoritative map of data sources in your system, providing topology intelligence and type-agnostic access via atoms.

What scio Does

  1. Registers storage resources — Accepts atomic providers from grub (databases, stores, buckets)
  2. Routes operations via URIdb://users/123, kv://sessions/abc, bcs://docs/report.pdf
  3. Provides topology intelligence — Knows what resources exist, where, and their relationships
  4. Enables type-agnostic access — Works with atoms, never needs to know your types

Core Concepts

URI Addressing

Every data access point is addressed by a URI:

VariantPatternExample
db://table/keydb://users/123
kv://store/keykv://sessions/abc
bcs://bucket/pathbcs://docs/reports/q4.pdf

Atomic Operations

scio operates on atoms — type-segregated data structures from the atom library. This enables infrastructure code to work with data without knowing user types.

Spec Tracking

When resources are registered, scio tracks their specs (type metadata). This enables:

  • Finding all resources that share a type
  • Discovering related resources
  • Field-based resource discovery

Use Cases

  • Service infrastructure — Route data access without coupling to user types
  • Data pipelines — Process data across multiple storage backends uniformly
  • System introspection — Understand the data topology of your application
  • LLM integration — Provide context about available data sources

Dependencies

scio builds on the zoobzio ecosystem:

  • atom — Type-segregated struct decomposition
  • grub — Provider-agnostic storage
  • edamame — SQL query execution