Hummingbird

Canonical source: PERSISTENCE.md (commit 46e93c9) — View raw Markdown

Persistence and Cost Envelope

Status: working technical plan. Cloudflare D1 is the deployed Phase 2 canonical persistence engine; the Durable Objects/R2 interactive-space components described here are future Phase 3+ design and are not yet deployed.

This document answers two separate questions:

  1. What state will Hummingbird eventually need to store?
  2. What should that cost at hobby, pilot, and early-community scale?

It does not authorize financial expenditure, establish a project treasury, or change the Phase 5 financial-governance boundary.

Architectural conclusion

Hummingbird does not currently need a large database platform or a generalized event-streaming stack.

The preferred shape is:

Git / canonical documentation
        │
        ├── D1: canonical and durable relational/application state
        │
        ├── Durable Objects: live room/activity coordination
        │
        └── R2: backups, exports, and larger immutable/archive artifacts

The important rule is that these provider products are implementation details. Institutional meaning remains portable and versioned.

Document-first, not database-first

Hummingbird prefers a NoSQL-friendly semantic model: versioned, portable documents define what an offer, pad, space, activity, or institutional record means. D1 is currently SQLite storage for those concepts and for the small indexes, constraints, and atomic updates the live pilot needs. It is not a requirement that future concepts become permanently normalized SQL tables. Conversely, a schemaless document store would not itself solve consent, retention, cross-document invariants, or concurrent writes. Keep database engine choice reversible by exporting complete document meaning and testing reconstruction without provider-specific row IDs.

Cheap, light, and real

Hummingbird should optimize for three properties at the same time:

The default cost target for Phase 2 is $0 incremental monthly application cost while the Free plan remains comfortably within observed usage. Hummingbird should not introduce complexity merely to preserve zero cost: if a predictable small paid plan later materially improves reliability or removes an operational constraint, paying for it is preferable to architectural contortions.

The static public commons should remain useful when the dynamic persistence layer is degraded or unavailable. Mutation/dependent features should fail before the public read plane wherever technically possible.

1. D1 — canonical and durable application state

Cloudflare D1 is the current application database for deliberately admitted Phase 2 canonical records.

D1 holds data that benefits from relational integrity, queryability, and durable transactional updates, including:

Phase 2

Phase 2B proved repository-controlled migrations and deterministic local/remote canonical round trips. Phase 2C proved deliberate admission, separate publication state, read-only reconstruction, and rebuildable static publication. Phase 2D has now exercised a read-only production export, restore into a disposable replacement D1 database, deep semantic comparison, and public-projection equivalence without making the live production database the recovery-test target.

Future controlled participation

Candidate future tables or equivalent portable record families include:

pads
pad_declarations
pad_continuity_credentials
connections
spaces
space_constitutions
space_memberships
guilds
guild_memberships
grant_requests
grant_votes
grants
activities
activity_participants
activity_events
public_commitments
annotations
wall_epochs
wall_tiles

These names are a sizing/design inventory, not a committed schema. Portable semantics come before authoritative tables. A migration must not be the first place Hummingbird decides what a pad, guild, connection, grant, space, or activity means.

The intended growth path is:

portable concept
      ↓
reviewed semantic contract + examples
      ↓
D1 durable representation
      ↓
rebuildable public projection

Proposals already exist as a v1 canonical family. Later proposal review, amendment, authorization, or decision workflow should be layered on only after those institutional semantics are defined.

Pads should begin as the smallest useful continuity/public-state contract, without requiring an origin category. Connections should be explicit relationship state, not inferred follower graphs. Guilds should be bounded associations with explicit membership and scoped/expiring capabilities rather than a higher participant class or global reputation system.

Durable definitions, memberships, commitments, grants, and meaningful outcomes may belong in D1. High-frequency live coordination should not.

Phase 2E offer buffer audit

The live OFFER_DB is separate from canonical D1. A bounded offer row holds temporary body, optional reference/question, receipt hash, state, content hash, and expiry. The only currently implemented compression in D1 is exact-text grouping: matching active received/grouped rows are linked to a temporary cluster with membership rows. Each offer keeps its own receipt and withdrawal path. A D1 conditional insert enforces the 250-active-offer ceiling even when requests arrive concurrently. The cluster expiry is extended to the latest member expiry; withdrawal and expiry remove content and membership. If derived grouping fails after a confirmed insert, the offer remains received and reviewable. The private export repeats exact-text compression from a bounded D1 snapshot, so review does not depend on the cluster helper tables being perfectly current.

This is sufficient for the present 250-row pilot, not a semantic synthesis engine. The current schema already has temporary working_summary and membership fields, but no job classifies related ideas or writes summaries. Future thematic review should first produce a private, explainable working packet preserving corrections, disagreements, and singletons under ADR 0019. Only a separately authorized, documented process should change D1 handling state or admit anything to canonical memory. Keep the raw temporary offer available by its opaque ID until withdrawal/expiry; never replace it with only a cluster summary. Do not turn duplicate count into a vote or priority score.

The current access indexes cover state/order, expiry cleanup, question lookups, and cluster membership. At a 250-row ceiling an exact-content scan is bounded; add a content-hash or (state, expires_at) index only after query plans and observed volume justify it. D1's SQLite-backed transactions and bounded single-database throughput fit this pilot; those are storage facts, not arguments that the portable document model should become SQL-specific. Cloudflare documents D1 batch transaction behavior, prepared statement write metadata, and database/query limits; recheck these provider contracts before increasing the pilot ceiling.

Document-first data planning for future surfaces

The following is a working inventory, not a deployment decision. Each surface needs a portable semantic contract and reference examples before a schema, retention rule, or write endpoint is approved. A single surface may use several physical stores while exporting one coherent document family.

Proposed surface Portable meaning to define first Candidate operational home Public read form / unresolved gate
Offers and evidence questions Offer/receipt lifecycle, optional reference, question association, temporary working synthesis Separate bounded operational D1 for the current pilot Public aggregate evidence only; private offer text and Phase 3 ingress decisions remain gated
Pads and connections Pad declaration, continuity proof, explicit connection transitions Versioned documents with D1 query keys if authorized; short-lived capability state elsewhere Public pad projection only by chosen visibility; continuity and multiplicity questions remain open
Spaces and guilds Constitution versions, membership consent, scoped grant and revocation records D1 for durable transitions; live coordinator only if concurrency requires it Public rules and authorized summaries; local constitutions and grant powers are unresolved
Proposals and institutional decisions Proposal versions, provenance, rationale, decision transition Canonical versioned records with portable export and rebuildable static publication Published records only after explicit admission/publication; governance proposal initiation remains unresolved
Walls, rooms, games, activities Activity definition, meaningful outcome, optional bounded history Durable Object candidate for ordered live actions; D1 for approved durable outcomes; R2 candidate for sealed large history Static/derived public summaries where approved; activity-history retention remains unresolved
Commitments and shared resources Obligation, consent, evidence of completion, scoped resource authority D1 for approved durable meaning, not a general event firehose Only authorized public projections; no automatic treasury, grant, or governance authority

Common fields should be few: stable opaque ID, type, schema version, lifecycle state, timestamps needed for that lifecycle, and content/relationships whose meaning is defined by the contract. Store optional visibility, provenance, retention, and consent attributes only when the surface needs them. Keep indexes and materialized read models disposable; avoid global participant profiles, origin categories, page-view records, and speculative joins.

Revisit D1 versus a document database when a specific approved surface demonstrates that flexible document queries, cross-entity updates, read/write volume, or portability cannot be handled cleanly within the current model. Compare a document store using an executable import/export fixture, query plan, retention/withdrawal exercise, and expected operating cost. Changing providers should not by itself change participant rights or institutional meaning.

What should not be written to D1 by default

Do not persist every:

Those patterns create cost, privacy risk, and institutional clutter without adding durable meaning.

2. Durable Objects — live coordination, not institutional memory

Persistent rooms, public games, collaborative walls, and joined pads may eventually need a single serialized place to coordinate concurrent actions. SQLite-backed Cloudflare Durable Objects are the preferred candidate for that role because they are designed for stateful coordination and real-time applications.

A Durable Object may represent one bounded coordination domain such as:

The Durable Object validates actions against the activity/room constitution, resolves concurrent updates, and emits only meaningful state transitions to durable institutional storage when required.

Durable Objects are not required merely to create pads, guild records, proposals, or ordinary asynchronous participation. They should be introduced only when real concurrency/serialization needs exist.

Required design constraint: hibernation

Interactive connections should use WebSocket Hibernation or another scale-to-zero pattern wherever practical.

A WebSocket that keeps an object continuously active can make wall-clock duration the dominant cost even when little useful work occurs. A hibernatable connection allows a room to remain present without paying continuously for idle compute.

Therefore presence must not require continuously running compute merely to prove that a socket exists.

3. R2 — backup and immutable/archive material

Cloudflare R2 is the preferred candidate for objects that should be durable but do not need to remain hot relational rows.

Likely uses:

R2 should not become the only copy of canonical mutable state. Backups must remain independently restorable and checksummed.

For a large append-only activity such as the public wall, D1 can retain the current epoch and searchable/index metadata while older sealed epochs are compacted into immutable R2 objects with stable references and hashes, if the final retention policy permits that representation.

4. GitHub — source and institutional text, not the live application database

Git remains authoritative for code, migrations, schemas, documentation, ADRs, and deliberately versioned institutional text.

GitHub Issues remain interim Seed Bank transport under ADR 0011. Provider issue/comment history must not silently become the production application database.

State classes

Future persistence should distinguish at least four practical classes.

Ephemeral coordination

Examples: online-presence state, temporary locks, uncommitted room work, anti-replay state.

Target: memory/Durable Object storage with a short TTL where possible. Do not promote to durable history automatically.

Operational state

Examples: active memberships, room configuration, current game state, connection status, grant-review work in progress.

Target: D1 or Durable Object storage with bounded retention/history appropriate to the function.

Durable institutional state

Examples: admitted canonical records, accepted decisions, issued guild grants and their rationale, public governance outcomes.

Target: D1 with portable export and independent backup.

Archival public artifacts

Examples: sealed wall epochs, completed game replays, compacted historical activity bundles.

Target: R2 plus searchable references/indexes in D1 when useful.

The unresolved retention boundary for future activity data is tracked as OQ-DATA-ACTIVITY-RETENTION.

Current Cloudflare pricing snapshot

Pricing below is a planning snapshot checked on 2026-09-11. Provider pricing can change and must be rechecked before a funding decision.

Official sources:

Workers

Workers Free currently includes approximately:

The current static Pages public surface does not need to route every read through a Worker. Keep it that way unless dynamic behavior actually requires one.

Workers Paid currently has a $5/month account minimum and includes approximately:

Published overage rates at this snapshot are $0.30 per additional million requests and $0.02 per additional million CPU milliseconds.

D1

Workers Free currently includes:

There is no D1 capacity-hour or idle-compute charge. D1 scales to zero when it is not queried, and there are no D1 egress/throughput charges for data access.

On the Free plan, hitting the daily row-read or row-write limit causes additional D1 queries of that type to fail until the daily reset; it does not silently convert into usage charges. Reaching the storage limit prevents further growth until storage is freed or the plan is changed.

Cloudflare currently states that the Workers Free plan will continue to include the ability to prototype and experiment with D1 for free. Treat that as a provider policy statement, not a Hummingbird constitutional dependency.

Workers Paid currently includes:

Published paid overage at this snapshot is:

Worker compute used to execute application logic is billed separately through Workers.

Durable Objects

Durable Objects are currently available on both Workers Free and Workers Paid. New Free-plan use is limited to the SQLite storage backend, which is also the preferred backend for new designs.

Workers Free currently includes approximately:

On Free, exceeding a metered daily limit causes further operations of that type to fail until reset rather than creating an overage bill.

Workers Paid currently includes approximately:

Published request overage is $0.15/million. Published duration overage is $12.50/million GB-s.

Incoming WebSocket messages currently receive a 20:1 billing ratio for request billing, and outgoing messages are not separately charged as requests. Hibernation is therefore a major design requirement because compute duration, rather than message count, can dominate badly designed real-time systems.

Cloudflare's own paid-plan examples illustrate the difference: a hibernation-based example with 100 Durable Objects and 100 sockets per object sending one message/minute is estimated around $20.65/month including the $5 plan minimum, while a smaller non-hibernating example can exceed $140/month because idle connected time continues to incur duration charges. These examples are illustrative, not Hummingbird forecasts.

R2

R2 Standard currently includes a monthly free tier of approximately:

Published Standard storage beyond the free tier is $0.015/GB-month, with operation charges separate.

This makes R2 substantially more appropriate than D1 for large immutable archives and backup objects.

Hummingbird planning scenarios

These are budget envelopes, not promises. Actual cost depends heavily on action frequency, query/index design, retention, cache hit rate, WebSocket hibernation, and abuse traffic.

A. Phase 2 read-only commons

Expected dynamic data is tiny: canonical documents, relationships, and read projections.

Target infrastructure increment: $0/month. Remote D1 provisioning, migrations, bounded canonical records, low-volume steward verification, and occasional guarded recovery exercises should fit comfortably inside the current Free allowances. Static public reading remains on Pages rather than turning every page view into a Worker/D1 operation.

A paid Workers plan is not required merely to create or use D1 at this stage.

Database cost is not a reason to seek meaningful outside funding for Phase 2.

B. Controlled-participation pilot — hundreds of active pads/day

Assumptions:

Free-first target: a modest asynchronous pilot may still fit within $0/month current Free allowances. If daily Workers/D1/DO limits become operationally constraining, the expected first deliberate upgrade is the approximately $5/month Workers Paid floor, not a redesign.

Planning envelope after upgrading: roughly $5-$15/month for Cloudflare application infrastructure is a reasonable early target, with alerts well below the point where overage becomes surprising.

C. Emerging community — order of 10,000 active pads/day

At this scale, tens of millions of monthly interaction events are possible, but D1's paid included row limits remain large if indexes are good and heartbeats are excluded. Worker request/CPU cost, Durable Object duration, moderation/security tooling, and retained activity volume become more important than simple SQL row reads.

Planning envelope: roughly $10-$50/month if the architecture stays cache-heavy, hibernating, and retention-disciplined. This number should be replaced with observed usage data before scaling beyond a controlled pilot.

D. Real-time-heavy or adversarial usage

The main cost hazard is not ordinary canonical storage. It is continuously active real-time compute, unbounded event retention, pathological queries, or abuse designed to create work.

At this point costs can move from tens to hundreds of dollars/month quickly. Hummingbird should not scale this mode by accident.

Required controls before broad writes include:

Cost-control invariants

The following rules are intended to keep low cost an architectural property rather than a late optimization:

  1. A public page view does not imply a D1 query. Prefer generated/static/cacheable read projections.
  2. A request does not imply a durable event. Store consequential institutional changes, not raw traffic.
  3. Presence does not imply a write. Do not turn heartbeats into permanent database churn.
  4. One action has bounded fan-out. Payload size, relationships, notifications, derived rows, and storage expansion must be limited.
  5. Indexes earn their existence. Add indexes for observed query patterns; use them to reduce row scans without speculative index sprawl.
  6. Real-time infrastructure earns its existence. Do not deploy Durable Objects for ordinary CRUD/asynchronous interaction.
  7. Dynamic failure should not erase public readability. Free-tier exhaustion or a persistence incident should degrade dynamic/mutation functions before the static commons.
  8. Measure before paying or optimizing. Use provider usage metrics to decide when a paid plan or architecture change is justified.

Storage-growth examples

Storage, not query pricing, is likely to be the first long-term database pressure if activity history is allowed to grow without policy.

For intuition:

This is why Hummingbird should archive sealed immutable activity to R2 where appropriate and should not declare every interaction durable merely because it can be stored.

Database design requirements before Phase 3

Before Hummingbird owns public mutation endpoints, the persistence layer should satisfy all of the following:

  1. Portable semantics. Canonical/exported meaning cannot depend on provider row IDs, hidden triggers, or proprietary state.
  2. Semantic contract before table. New institutional concepts are defined portably before database migrations make them authoritative.
  3. Clear ownership of truth. Each fact has one authoritative home; projections/caches remain rebuildable.
  4. No required origin classification. Pads can function without an origin category.
  5. Consent-aware relationships. Public pad connections and guild membership require explicit state transitions, not inferred relationships.
  6. Append meaningful events, not telemetry. Consequential actions are inspectable; raw request exhaust is not institutional history.
  7. Bounded payloads/cardinality. A single action cannot create unbounded rows, relationships, storage, or fan-out.
  8. Retention is explicit. Every new table/record family names EPHEMERAL, OPERATIONAL, DURABLE, or an approved archive policy.
  9. Concurrency is serialized where necessary. Games, walls, votes, and room rules cannot rely on race-prone last-write-wins behavior.
  10. Backup and restore work. D1 canonical state can be exported and restored independently of the live database.
  11. Cost observability exists before scale. Row reads/writes, Worker requests/CPU, Durable Object request/duration, and R2 growth have measurable budgets.
  12. Read-plane survival. Exhausted mutation budgets or interactive-system incidents should degrade writes first; the public read commons should remain available wherever technically possible.

Suggested budget gates

These are operational planning thresholds, not financial-governance decisions:

Do not optimize the architecture merely to avoid a small justified bill. A predictable $5 paid plan is preferable to fragile complexity designed to save five dollars.

Does Hummingbird need seed funding?

Not for the database itself at the current stage. D1, Workers, Durable Objects, and R2 provide a substantial Free/hobby-scale runway when the system avoids unnecessary dynamic work and permanent telemetry.

If Hummingbird seeks seed support, the stronger reasons are likely to be:

An infrastructure-only reserve on the order of $500-$1,000 would represent many months of early operation under the planning envelopes above. A larger seed amount should therefore be justified primarily by work and governance needs, not by an exaggerated claim that SQL hosting is expensive.

Any future project-controlled funding, budgeting, or authorized expenditure remains subject to Hummingbird's financial-governance boundary and OQ-PROJECT-LEGAL-STRUCTURE. The current receive-only support mechanism does not automatically become a project treasury.

Funding posture

Hummingbird should prefer a transparent runway model:

observed monthly infrastructure cost
+ known committed services
+ approved stewardship/security work
= monthly burn

available authorized reserve / monthly burn
= runway

Do not create donation-linked standing, supporter reputation, privileged access, or governance weight.

Review cadence

Recheck this cost model: