The HTTP API

The surface is generated from your declarations. There is nothing to register and no routing table to keep in step: the route is the declared name.

Running a command

POST /commands/{Name}

The name is the declared command name, never a file name. Commands under commands/internal/ get no route at all.

The body is the parameters as JSON. An empty body is an empty object; a body that is not an object is a 400.

A blank idempotency-key is treated as no key at all.

Status mapping

Success

{
  "correlation_id": "eaa1d143-1a24-4a0e-9f14-2b0f2b6f5a11",
  "causation_id": "425d5c99-9b5f-4c2e-8f3a-1d9f0c7a2b44",
  "positions": { "first": 1, "last": 1 },
  "events": [
    {
      "type": "booking.made",
      "tags": ["booking_id:1111...", "guest_id:7"]
    }
  ]
}

positions is null and events is [] when the command decided to do nothing, and that is still a 200. A repeat under an idempotency-key is a different thing: the original response is recovered from the log and returned whole, positions and events included.

Tags exclude subject-encrypted values and the runtime’s own _hekla_ tags, and they are sorted, so a fresh commit and a log-recovered replay are byte-identical.

An error

{
  "correlation_id": "656b4c61-fb47-4fa9-a125-310197b3125a",
  "causation_id": "0d7d6aba-fcf2-46fc-bffd-73a70cc959bb",
  "error": {
    "code": "already_joined",
    "message": "that email is already on the list"
  }
}

Every command error carries the two ids, which is what lets a client report something you can find in /admin/traces/{correlation_id}.

Reading a projector

Two routes per entity.

GET /read/{projector}/{entity}/{key}
GET /read/{projector}/{entity}

{
  "item": { "email": "ada@example.com", "entry_id": "0190d1a1-..." },
  "position": 1
}

A scan answers a list with a cursor:

{
  "items": [{ "email": "ada@example.com", "entry_id": "0190d1a1-..." }],
  "next_cursor": null,
  "position": 1
}

An absent column is omitted from a row rather than sent as null, which is also what an erased subject’s column looks like.

What a scan may filter on

A filter is equality across any prefix of one declared index, or the key, and optionally a range on the column right after that prefix. One column ranges per request, and both ends of it may be given.

Take index (shop_id, status, month) over an Int shop, an enum status and an Int month. It answers ?shop_id=, ?shop_id=&status= and all three, plus ?shop_id=&status=&month.lt= and ?shop_id=&status=&month.gte=&month.lte=. It refuses ?status=&month=, because reaching those rows means visiting every shop. The chosen index is pinned, so a filter the handler admits is never one the store answers with a table scan instead.

A range needs a column whose order means something. Money is stored as its decimal string, so >= would sort "2" above "10", and an enum is stored as its variant’s spelling, so a range would walk the alphabet rather than the severity. Those two, plus Bool and Json, take equality only, and a range on one is a 400 naming the declared type. An optional column ranges like the type under its ?: >= simply does not match the absent ones, which is what a reader asking for a range wants.

A column that is subject-encrypted cannot be indexed at all. A filter arrives as plaintext and could never match the ciphertext, so filter by the plaintext subject id instead.

Ordering a scan

An index’s name is generated from its columns, so a second index (shop_id, month) beside the one above is by_shop_id_month, and GET /admin/projectors/{Name} lists them. ?order_by=-by_shop_id_month reads a shop’s rows latest month first, and ?order_by=-<key column> reverses the default. The three-column index cannot back an order_by at all, for the reason in the last bullet below, which is exactly why the pair is declared.

• One direction for the whole tuple. A cursor resumes with a row-value comparison, which has one direction, so an index is read as declared or reversed as a whole. Per-column directions are not expressible in it.
• The ordering fixes the index, so the filter has to be a prefix of that same index. Sorting one index’s match set by another is the in-memory sort this API exists to refuse.
• The key is the last term of every ordering. An index tuple is not unique, and without that tiebreak two rows sharing one are dropped or repeated at a page boundary.
• A cursor records the ordering it was taken over, and reading it back under another is a 400. Dropping ?order_by= counts as another, because the default key order is an ordering too, and so does a deploy that moved @key onto a different column.
• An index carrying an optional, Money or enum column can filter but cannot sort. A row-value comparison against an absent value matches nothing, so a page boundary would lose every row whose column is absent, and the other two do not order the way they read. order_by’s enumerated values in /openapi.json leave them out rather than listing them and then refusing.

Declare the narrow index too if you filter on it alone. The key is appended to every generated index, so a filter using all of an index’s columns is a pure seek, while a shorter prefix leaves a declared column between the filter and the key and the match set is sorted on every page. index (shop_id) beside index (shop_id, status) makes both questions cheap, and the runtime picks the narrowest index that serves a given filter.

Read your writes

A read model is eventually consistent by default. To see your own write, pass the positions.last a command returned:

curl 'localhost:8080/read/Waitlist/Entry?email=ada@example.com&after=1'

The read blocks until the projector reaches that position rather than quietly serving a stale row. timeout_ms=0 means check once and do not wait.

Read errors

The Retry-After rule is a deliberate signal rather than an oversight: rebuilding resolves on its own, so it carries one. stale needs an operator, so it does not.

Read errors carry no correlation ids, because a read caused nothing.

Operator routes

POST /projectors/{name}/replay
POST /effects/{name}/skip/{position}

A replay answers 202 {"status": "replay_scheduled"}, or 503 not_running if that projector has stopped.

A skip answers 202 for any position, including one that does not exist. The driver honours the request only once that position has actually failed, and forgets it once the watermark passes. Nothing is ever skipped automatically.

There is no HTTP equivalent of hekla rewind, on purpose. See Effects in production.

Status and health

GET /status
GET /health

/health is {"status":"ok"} and nothing else. /status is the whole picture: log head, uptime, every projector’s position, lag, readiness and last error, and every effect’s state, lag, wedged lane count, pinning key and pinning position. It is the summary and it names no lane but the pinning one; GET /admin/effects/{name} carries an entry per stuck lane. See Effects in production.

Metrics

GET /metrics

Prometheus text exposition, text/plain; version=0.0.4. Gauges are read at scrape time, so there is no collection interval and nothing to configure. The full series list is in Deploying.

The generated reference

GET /openapi.json
GET /docs

/openapi.json is a generated OpenAPI 3.1 document covering every public command, both routes for every entity, and the operator and introspection routes. hekla openapi . prints the same document without starting a server.

The console

GET /admin

/admin is the same URL as the JSON API rather than a second one. A request naming text/html in its Accept header gets the console; everything else gets the JSON byte for byte unchanged.

The negotiation is careful about what counts:

• a range that matches HTML and not JSON selects the console, so text/html, application/xhtml+xml and text/* all do
• */*, which is what curl and a bare fetch() send, matches both equally, and an equal weight goes to JSON
• ;q=0 on text/html opts out
• a more specific range wins, so text/html beats text/* beats */*
• responses carry Vary: Accept

The console is compiled into the binary, so it works with no network at all.

Introspection

Every one of these is read-only, and answers JSON or the console by the same negotiation:

A request that decrypts writes an audit line.

/admin/subjects/{subject}/{value} takes the subject’s declared name, which is what a key row is filed under. It answers absent three ways it cannot tell apart: a subject that was erased, one that never existed, and one whose ancestor was erased above it, whose own row is still on disk and no longer openable. That is a property of deleting the key rather than a gap in the endpoint.

Ad-hoc projections

POST /admin/projections folds a projector the deployment does not have over the log and returns its rows. It is the one /admin route that is not a GET, and the one that runs code a caller supplied rather than code the project declares, so a deployment has to say yes to it:

[admin]
projections = true

Without that it answers 403 naming the setting. GET /admin/projections answers either way, reporting the limits and whether the POST is served.

The body is the heklang source as text/plain and the knobs are query parameters: projector, entity, from, upto, max_events, rows and decrypt.

curl --data-binary @question.hk \
  'localhost:8080/admin/projections?max_events=100000&rows=20'

An unknown parameter is a 400 rather than ignored, because a typo in max_events costs the bound the caller was trying to impose. A source that does not compile is a 400 carrying the compiler’s own findings. The response body is exactly what hekla project --json prints, so one shape reaches both.

What bounds it: heklang is total, so a projection terminates, and a projector holds no clock, no network, no general read and no way to append. The server bounds it again on size, defaulting max_events to 100,000 and clamping it to 5,000,000 however much is asked for, because this folds a log where the rest of /admin reads a page. A run that spends its budget reports scanned.stopped: "max-events", and its rows are part of the answer rather than all of it. Folding a whole log is hekla project’s job, which holds no request open while it does one.

The source is compiled against what the process booted with, never re-read from disk. A module edited under a running server is not visible to a projection, which is deliberate: it would otherwise typecheck against declarations the process is not running and then misread stored payloads.

With Accept: application/x-ndjson the fold arrives in pieces: one JSON object per line, a {"progress":{...}} tick while it works, and the projection itself as the last line, identical to the buffered body. Any other Accept gets the buffered body. It is one request rather than a subscription, with no reconnection and no fan-out, and the tick channel is bounded so it drops ticks rather than growing.