Deploying

The artefact is the source tree

There is no build step, no compiled output and no compile cache. What you deploy is the .hk files, plus hekla.toml if you have one, plus the master key if any field declares a @subject.

Deploy is copy and restart. There is no hot reload: the project is read at startup, and a running process never re-reads it.

The data directory

data/
  events/                tephra segments, the immutable source of truth
  projectors/{Name}.db   one read model and its checkpoint
  hekla.db               effect journals, subject keys, deployed declarations
  hekla.lock.db          the exclusive one-process lock

Two of those are irreplaceable: events/ and hekla.db. Projector databases are rebuildable from the log, so they are the part you do not need to keep.

The lock lives in its own file rather than inside hekla.db, because the operational database is written continuously and an exclusive transaction held for the process lifetime would deadlock against it.

Opening any of these files directly is not a supported surface, and the table layout is private.

Backup

Copy the directory of a stopped process.

The good pattern is to stop, copy, start, and then run hekla verify against the copy. That checks the invariants and the backup in one pass, on a machine that is not serving traffic.

Migrations

The operational database migrates itself on open, one schema version per transaction, and refuses a database newer than the build:

operational database is at schema version 10, newer than this build (9)

So a downgrade is a refusal rather than a corruption. hekla plan refuses a directory this build has not migrated yet; run hekla serve against it once first.

The event log has no migration story. An event type carries no version and there is no upcast hook. See Events.

Observability

Logs

tracing, filtered by RUST_LOG, default info. Installed by serve and verify only, so every other subcommand stays plain and machine-parseable: hekla openapi . > openapi.json and hekla plan --json are safe to pipe.

Colour is on only when stdout is a terminal and nobody opted out. NO_COLOR with any non-empty value, or --no-color, turns it off.

Lines worth alerting on: a failed ctrl-c handler registration (shutdown must then be forced), a completed projector rebuild with its duration, and an in-flight invocation whose recorded source hash differs from the code on disk, logged by name at startup.

/status

The whole runtime in one document: log head, uptime, every projector’s position, lag, readiness and last error, and every effect’s state, lag, wedged lanes, pinning key and pinning position. This is what a console or a health dashboard should read.

/metrics

Prometheus text exposition. Gauges are read at scrape time, so there is no interval and no [metrics] section.

Gauges

hekla_build_info, hekla_uptime_seconds, hekla_log_head_position, hekla_module_info, hekla_projector_up, hekla_projector_position, hekla_projector_lag, hekla_projector_readiness, hekla_projector_last_progress_timestamp_seconds, hekla_effect_up, hekla_effect_position, hekla_effect_lag, hekla_effect_state, hekla_effect_wedged_lanes, hekla_effect_consecutive_failures, hekla_effect_retry_backoff_seconds, hekla_effect_last_progress_timestamp_seconds

Counters

hekla_events_appended_total, hekla_commands_total, hekla_command_refusals_total, hekla_command_conflict_retries_total, hekla_projector_events_total, hekla_projector_rebuilds_total, hekla_effect_invocations_total, hekla_effect_restarts_total, hekla_effect_terminal_skips_total, hekla_effect_live_suppressed_total, hekla_effect_collapsed_total, hekla_effect_http_requests_total, hekla_reads_total, hekla_read_waits_total

Two labelling rules are design commitments rather than conventions.

A lane key is never a label. Every label is a declaration: a command, projector, entity, effect or event name, a refusal code, or a fixed outcome word. The reason is erasure, not cardinality. A lane key is a partition key, and /status may name it because it is a live view an erasure passes through; a scrape may not, because it is a copy taken somewhere hekla erase cannot reach. So the metric counts wedged lanes and /admin/effects/{name} names them.

A refusal series appears on first use. heklang inlines a refusal, so hekla cannot enumerate the codes to prime them at boot. Any expression over hekla_command_refusals_total needs or vector(0). Every other counter reads 0 from the first scrape.

Alerts

The repository ships a rules file at docs/monitoring/hekla-alerts.yml with sixteen alerts, each carrying the reasoning for its threshold inline: HeklaDown, HeklaProjectorStalled, HeklaEffectWedged, HeklaEffectQuarantined, HeklaEffectBlocked, HeklaCommandConflictsExhausted, HeklaLogNotAdvancing, HeklaReadYourWritesTimingOut and others. Start from it rather than from scratch.

/metrics is unauthenticated like everything else, so the scrape target is reached the same way /admin is.

The limits, stated plainly

Saying these first is the point, so here they are without hedging.

Nothing is authenticated. The bind address is the boundary and it defaults to loopback. A caller who reaches the port can append events, browse the console, read metrics and skip an effect’s work.

One process per data directory, enforced by an exclusive lock. There is no clustering, no replication and no horizontal scaling.

Deploy is restart. No hot reload, no upload API, no live editing.

The event log has no schema evolution. No version on a type, no upcast hook.

One indexed filter field per scan. Composite-prefix filters are not built.

No automatic dead-lettering. A wedged effect is never advanced automatically; the manual skip endpoint is the only escape hatch.

No hekla fmt and no language server. Both arrive when heklang has a language server of its own. hek fmt exists today and is what to point an editor at; see Editor setup.

/docs needs outbound network, because it loads its reference UI from a CDN. /openapi.json does not.

invoke is exactly-once only when the target command is idempotent under replay. Raw http.* is at-least-once. That is an inherent property of the design rather than future work: the journal can guarantee a call is not repeated after a crash, but it cannot make a remote endpoint idempotent for you.

Pre-1.0. hekla is 0.4 and heklang is 0.5. Both carry breaking changes between minor versions, and neither has run your workload yet.

And one that will not change

heklang is the only authoring surface, permanently. There is no Rust, TypeScript or WASM SDK, and there is not going to be one. A single pure sandboxed authoring language is what makes the rebuild, the replay and plan --replay sound, and an escape hatch into a general purpose language would take all three away.