Running a project
hekla serve, what it creates, what it refuses to start without, and every configuration option.
hekla serve .
hekla listening on http://127.0.0.1:8080
admin console http://127.0.0.1:8080/admin
api reference http://127.0.0.1:8080/docs
One binary is the whole runtime. It serves the API, keeps the read models, runs the effects and holds the keys, with tephra embedded as a library and SQLite bundled. There is nothing to stand up beside it and nothing to point it at.
Arguments
| Default | ||
|---|---|---|
DIR |
. |
the project directory |
--addr |
127.0.0.1:8080 |
the HTTP bind address |
--data-dir |
<dir>/data |
event store and operational database |
--verify |
off | run the continuous invariant check |
--verify can only turn verification on. A [verify] enabled = true in hekla.toml
cannot be turned off by leaving the flag out.
What happens at startup
In order, because the order is what makes some failures loud and some impossible:
- The metrics recorder is installed, before anything else, so no counter is dropped.
- The project is loaded and checked. Any error refuses the start:
refusing to serve: the project has N error(s). - The bind address is parsed.
- Master keys are read from the environment.
- The data directory is opened and the exclusive lock is taken.
- Declared secrets are resolved. A required one this machine cannot supply refuses the start, before anything is recorded about this deploy.
- The log is read to check that this program can still decode what is already in it.
- Every subject key is moved under the parent its subject declares.
- Projectors and effects start, and the HTTP server binds.
Four of those refuse rather than degrade, and deliberately.
A missing master key. A project that declares a subject refuses to start without
HEKLA_MASTER_KEY, whether or not anything is sealed yet, because finding out at the first
write is worse than finding out at boot. And if stored subject data was wrapped under a key
that is not configured now, the process fails naming the missing key ids rather than
starting and serving nulls where personal data used to be.
A missing required credential. A declared secret this machine cannot supply stops the
start. hekla secrets is the same check without starting anything, which is what makes it
a pre-deploy gate.
A declaration that has outrun its log. An event field no stored payload can answer for,
or a stored payload this program can no longer decode, stops the start naming the type and
the field. A log outlives the program that wrote it, and a type that cannot be read breaks
every reader of it at once: a fold answers 500, a projector rebuild fails and retries
forever, and an effect lane wedges holding the watermark. None of those recovers by waiting,
so the question is asked at boot instead. Events covers the repair, which
is @absent(<value>) or an optional.
A subject key not yet under its declared parent. Step 8 moves them, and the start
refuses if it cannot, because a key left under the master is one an erasure of the tenant
would not reach. It is a scan of the log, so
hekla adopt exists to do it ahead of the deploy, against the
live directory and without the lock. A project where no subject declares a parent does
nothing here.
Shutdown
Ctrl-C drains in a fixed order: effects first, then projectors, then the writer. An invocation still in flight after 30 seconds is abandoned; it stays marked running and replays from its journal at the next start, so nothing it already did is repeated.
One process per data directory
data/hekla.lock.db is an exclusive lock. Which subcommands take it is worth knowing,
because it decides what you can run against a live server:
| Takes the lock | Does not |
|---|---|
serve |
plan, project |
verify |
erase, rotate, adopt |
rewind |
So you can run hekla plan, hekla erase and hekla adopt against a running deployment,
and you have to stop the server for hekla verify and hekla rewind. adopt is the one
written to be run that way: it reads the log through a follower while the old declaration is
still being served.
hekla.toml
Optional, at the project root. A missing file means every default below. Every key is validated on load and an unknown key is an error, so a misspelling is caught rather than silently ignored.
[effects]
# How many lanes run at once, process-wide. 1 to 1024.
pool_size = 16
[retention]
# How long a completed effect invocation's journal is kept.
effect_journal_days = 7
[projectors]
# Rebuild a projector automatically when its definition hash changes.
auto_rebuild = true
[verify]
# Replay every completed invocation against a sealed journal as it runs.
enabled = false
[admin]
# Serve POST /admin/projections, which folds a projector a caller supplied.
projections = false
[secrets]
DISCORD_WEBHOOK = { env = "DISCORD_WEBHOOK_URL" }
STRIPE_KEY = { file = "/run/secrets/stripe" }
[admin] projections is off because that route is the one place a deployment runs heklang a
caller wrote rather than heklang the project declares. What it cannot do is the argument for
turning it on: a projector holds no clock, no network, no general read and no way to append,
and heklang is total, so the fold terminates. See The HTTP API.
A [secrets] entry takes env or file and nothing else. There is deliberately no third
form carrying the value, and { value = "sk_live_..." } is a parse error rather than a
convenience, because writing a credential into the source tree is the exact mistake the
feature exists to prevent. A relative file path resolves against the project root.
Environment
| Variable | Read by | Default |
|---|---|---|
HEKLA_MASTER_KEY |
serve, verify, rotate, adopt, plan --replay |
none; base64 of 32 bytes |
HEKLA_MASTER_KEY_PREVIOUS |
the same | none; comma-separated prior masters |
HEKLA_SECRET_<NAME> |
serve, verify, plan, secrets |
fallback for a credential not named in [secrets] |
HEKLA_MAX_ATTEMPTS |
serve |
5, capped at 15; the command conflict-retry budget |
HEKLA_UI_DIR |
serve |
none; serve console assets from disk |
RUST_LOG |
serve, verify |
info |
NO_COLOR |
serve, verify |
unset; any non-empty value drops ANSI |
HEKLA_V10_SUBJECTS |
any command that migrates a data directory | none; old_field=NewSubject pairs, read on one schema step. See Deploying |
HEKLA_MAX_ATTEMPTS is read once per process. An unparseable value falls back to the
default rather than failing.
HEKLA_SECRET_<NAME> is never read by check or test, so neither needs any credential.
What is not configurable
Named here because the absence is deliberate and people go looking:
- HTTP connect (10s) and total (30s) timeouts for outbound calls
- effect retry backoff, from 200 ms doubling to a 60 s cap
- the read-your-writes wait default (5s) and maximum (30s)
- the journal sweep interval, hourly
- the shutdown drain timeout, 30s
- metrics collection: every gauge is read at scrape time, so there is no interval to set
Generating a master key
HEKLA_MASTER_KEY=$(head -c 32 /dev/urandom | base64)
Keep it somewhere you will still have it next year. Losing it is permanent, unrecoverable loss of every subject-scoped value in the log, and nothing else in the runtime fails that way. See Keys and secrets.