Projectors

A projector folds the log into rows. hekla serves those rows over HTTP and rebuilds them from position zero when the projector’s definition changes, unless [projectors] auto_rebuild is off.

projector CustomerOrders {
  entity Order {
    order_id: Uuid @key,
    customer_id: Customer @index,
    email: String? @max(200),
    shipping_address: String? @max(200),
  }

  on @order.placed { order_id, customer_id, email, shipping_address } {
    put Order { order_id, customer_id, email, shipping_address }
  }
}

The handlers are the subscription. They say which events to read and what to do with each, so there is no separate list of subscribed types to keep in step with the code.

Entities

An entity is a table. Its columns are declared with the annotations the read API needs:

A composite index is declared at entity level rather than on a column:

entity Booking {
  booking_id: Uuid @key,
  guest_id: Int @index,
  status: Status,

  index (guest_id, status)
}

A filter is equality across any prefix of one declared index, or the key, optionally with a range on the column right after that prefix. Anything else is a 400 naming what is declared, rather than a table scan nobody asked for.

So index (guest_id, status) answers ?guest_id= and ?guest_id=&status=, and refuses ?status= on its own: reaching those rows means visiting every guest. @index on a column is the one-column case of the same declaration.

A scan is ordered by the primary key unless ?order_by= names a declared index, which The HTTP API has in full. The column order an index declares is what decides which questions it can answer, so it is a modelling choice rather than a hint.

What a column’s type decides. A range (.gte, .gt, .lte, .lt) needs an order the storage shares, so Money, an enum, Bool and Json take equality only: status above is an enum, so (guest_id, status) filters and never ranges. An index carrying one of those, or an optional, cannot back an order_by either. Put the column a reader will range over last, and declare a second index beside it when both questions matter.

Declare the narrow index too if you filter on it alone. The key is appended to every generated index, so a filter using all of an index’s columns is a pure seek, while a shorter prefix leaves a declared column between the filter and the key and the match set is sorted on every page. guest_id: Int @index beside index (guest_id, status) is what makes both questions cheap, and the runtime picks the narrowest index that serves a given filter.

The four write statements

The question to ask at every write is what should happen when the row is not there yet. put says “this event fully determines the row”. patch says “create it if needed”. update says “only if it already exists”, which is the right answer more often than people expect: an event about a thing you have not seen created is usually one you should ignore.

Reaching the envelope

A handler may bind the whole event with as, which is how it reaches the envelope:

on @order.placed as e { order_id } {
  put Order { order_id, placed_at: e.at }
}

e.at, e.id and e.position are what the envelope carries. This is the only way a projector reaches the moment an event was appended, because now() does not parse in one.

The leading dot

Inside a handler, a leading dot refers to the column’s current stored value:

on @order.shipped { order_id } {
  update Order[order_id] { shipped_count: .shipped_count + 1 }
}

That is a read of the row being written, and it is the only read a projector gets.

No general reads

There is no get(entity, key). A projector cannot look up an arbitrary row, only the one its statement is addressing.

That restriction is what makes a rebuild deterministic. A projector that could read anything would depend on the order it processed events in, and a rebuild would be free to produce something different from the live table.

If you need a value from elsewhere to make a decision, the answer is that the decision belongs in the command, and the event should carry the result.

Zero values

patch materialises an absent row from zero values, so every column type needs one:

Uuid, Timestamp and a subject id deliberately have none. The nil UUID, epoch zero and customer 0 are not identities, they are placeholders that read as data, and a row containing one is a bug that looks like a row. A patch that would have to invent one is a compile error naming the column, so a subject-typed column that is not the key needs a declared default wherever a patch can materialise its entity.

What a projector may not do

No network, no clock, no decrypt, no append, and no failure channel: there is no fail, no invalid, no reject.

That last one is the important one. A projector that could fail would have states a rebuild could half-reach, and every read of it would have to ask whether it was in one. Because it cannot, “rebuild this from zero” is total, and hekla rebuilds it whenever a definition hash changes, unless [projectors] auto_rebuild is off, in which case the projector goes stale and serves 503 until an operator replays it. See Read models.

Sealed columns

A projector never declares a subject. It receives sealed content from an event and carries it: the column stores ciphertext, the projector never handles plaintext, and the read API decrypts on the way out.

A column holds whatever was sealed, so a record, a list, a map or a Json rides through whole and comes back out of the read API as that value rather than as text. The entity has to carry a column of the subject’s type beside it, and an index over a sealed column is refused: a filter arrives as plaintext and could never match the ciphertext, so filter by the plaintext subject id instead.

What a column cannot hold is sealed content inside something else. The seal is carried by the whole value, and a list or a record the handler builds has nowhere to put it, so many: [e.email] is a seal-boundary error where it is written rather than a column left holding plaintext that no erasure could reach.

An erase reaches the column and not only the log, because a read model can hold the only copy of sealed content outside it. A column whose subject key is gone reads back absent, which is what a column the handler never wrote reads back as too.

A projector cannot reveal. See Personal data.

Serving them

Each entity gets two routes, a key lookup and a filtered scan, both covered in The HTTP API. What happens when a projector is behind, rebuilding, or broken is in Read models.