Personal data

An event log is append-only, which is exactly the wrong shape for a deletion request. hekla’s answer is crypto-shredding, and it is a type rather than a procedure.

Declaring a subject

A subject is a key namespace with a name, and the type it declares has that namespace’s ids as its values:

subject Shop(Int)
// A customer belongs to a shop, so the customer's key is wrapped under the shop's and
// deleting the shop's row makes every customer key beneath it unwrappable at once. That is
// what `under` declares, and it is why every event sealing anything under a `Customer` has
// to carry a `Shop` too: the runtime learns which key to wrap under from the event in
// front of it, and there is nowhere else for it to look.
subject Customer(Int) under Shop

An id may be an Int, a String or a Uuid. A key is filed under text, and those are the three with one canonical form a key store can file and a command line can take.

An event carries the id in a field of that type, and @subject(...) on a sibling field seals that field’s value under it:

event @order.placed {
  order_id: Uuid,
  // Subject ids stay plaintext: they are how the runtime finds the key.
  customer_id: Customer,
  shop_id: Shop,
  // Personal, scoped to the customer.
  email: String? @subject(customer_id) @max(200),
  shipping_address: String? @subject(customer_id) @max(200),
  // The shop's commercial figure, scoped to the shop.
  order_total: Money(2) @subject(shop_id),
  // Free text nobody queries: opt out of tagging, and of being a huge tag.
  notes: String @no_index @max(500),
}

@subject(customer_id) says: encrypt this value under a key scoped to that customer before it reaches storage.

Four rules, and each of them is load-bearing:

The subject id stays plaintext. customer_id is how the runtime finds the key, so sealing it would make the key unfindable.

The subject id may not be optional, and its type must be a declared subject. @subject(x) names a field of the same event, and that field must always have a value: a missing id is not a missing key, it is no question at all. The sealed field itself may be optional or not, as the domain requires.

The type carries the namespace, and the field name carries nothing. Call the field customer_id, buyer or who and the keys are filed under Customer either way, so from: Customer, to: Customer on one event is two ids in one namespace. A Customer is not a Shop however alike their ids look, and there is no conversion: Customer(n) is unspellable, so an Int that arrives untyped cannot be made into a customer id.

Subjects are per field, not per event. The order above scopes two fields to the customer and one to the shop, so erasing the customer leaves the shop’s commercial record intact and erasing the shop never touches the customer’s address.

A subject id is an id and nothing else. Equality within one subject, yes, and against a literal. An entity key, an index column, a map key, a partition key and an ordering, yes, each reading through to the scalar the ids are, so an Int subject compares with < and a Uuid one does not. Arithmetic, no, and no methods either: id.pad(4) is refused where an Int would take it, because a subject has no row in either table.

The seal is a type

@subject(x) produces Sealed(T, Customer), a type nobody writes. Opt stays outermost, so String? @subject(customer_id) is Opt(Sealed(String, Customer)).

Sealed content is not comparable to plaintext and not comparable to content sealed under a different subject. That is not a lint: an equality over two ciphertexts would leak whether they hold the same value, so the type system does not offer one.

The seal rides along. A projector receiving a sealed column stores ciphertext and never handles plaintext, and the read API decrypts on the way out. A fold carrying one carries the seal too.

A seal does not survive being put in a box. A type carries one seal for the whole value, and content inside a list, a record or a comprehension has nowhere to carry it, so writing one there is refused where it is written:

this is content sealed under `Customer`, so it cannot be an element of a list
without `reveal`

reveal first if the value genuinely belongs in a box, which is a decision worth having to write down.

An entity holding a sealed column must also hold a column of that subject’s type, matched by type and never by name, or the key could not be found on the way out:

entity `Order`: column `ship_to` is sealed under `Customer`, but no column of this
entity holds a `Customer`; add one so the key can be found

Sealing a record, a list or a map

@subject(...) takes any declared type, so an address is one field rather than nine:

record Address {
  line1: String @max(200),
  city: String @max(100),
  // A field added to a sealed record after the log exists: a seal is read as stored
  // history rather than as a request body, so every seal already written reads this.
  country: String @max(2) @absent("IS"),
}

event @order.placed {
  order_id: Uuid,
  // Plaintext by necessity: it is how the runtime finds the key.
  customer_id: Customer,
  email: String? @subject(customer_id) @max(200),
  // Sealed whole. `@no_index` because a ciphertext tag nothing folds on is a large tag.
  ship_to: Address @subject(customer_id) @no_index,
  total: Money(2),
}

reveal(ship_to) hands back an Address and .city reads it. The alternative is nine parallel optional sealed fields, where adding a tenth is a schema change on every event that carries the address.

A part of a composite is not reachable by name. e.ship_to.city and for item in e.items are refused at the seal boundary the way a method call on sealed content is, because a seal holds one document rather than parts. Reading the field off what reveal hands back is the way, and it is the only way.

Opt stays outermost here as everywhere, so Address? @subject(...) reveals as an Address? and has to be narrowed before a field is read. The read-model column that receives either one is optional regardless, because an erased subject’s column reads back absent.

The text a seal holds for a composite is the JSON the language already writes for that type. A Json field is the one sealed whole, quotes and all, because it is the only type whose value can itself be a string that looks like another; a record, a list and a map each quote their own leaves already.

A seal is read as stored history rather than as a request body, which is what makes @absent work inside one: a field added to a subject-bound record today reads as its @absent literal on every seal already written. A store handing back something that is not that document is a mismatch naming the type the declaration promised.

@subject cannot go on a record’s own field. The annotation names a sibling holding the id, and a field inside a record has no sibling to name.

reveal

Only an effect arm can decrypt:

let response = http.post(
  "https://mail.example/confirm",
  { "to": reveal(email), "order_id": order_id },
)

Not a command, not a projector, not an effect-local fn. The narrower that boundary is, the shorter the list of places plaintext can exist, and one arm is as narrow as it gets while still being able to send the mail.

reveal is not journaled. It re-decrypts on every attempt, deliberately: a journaled plaintext would outlive the erasure it belonged to. log is not journaled either.

erase

erase(customer_id)

That deletes one key row. It is O(1), irreversible, needs no master key, takes no lock, and takes effect across the whole log and every read model at once, with no rewrite, no compaction and no index rebuild.

erase takes a value whose type is a subject, and that type is what says which keys go. It reaches an id folded out of the log exactly as it reaches a trigger field, because a List(Customer) carries the answer on every element. A Uuid order id names no namespace, so erase(order_id) is refused where it is written, and so is a bare literal: an erase takes a value that knows what it is.

From the CLI:

hekla erase Customer 7

The CLI and a test’s erased line name a key row, which is a subject and the id as a key store files it, so "7" is text in both. The expression names a value, and the value’s type is the namespace. They read differently because they are about different things.

erase is a statement rather than a value, because there is nothing an author could do differently on either answer. No reveal may be reachable after an erase in the same arm, and the checker enforces it as a reachability analysis over the arm’s control flow rather than a lexical ordering check. erase is journaled and reveal is not, so a replay would skip the erase and re-run the reveal against a key that is gone. Ordinary statements after an erase are fine, and an erase inside a branch that does not fall through leaves a later reveal legal.

An id learned by revealing may not be erased. A sealed field can itself hold a subject id, so erase(reveal(e.former_owner)) is well typed and is refused by span: a repeat request for an already erased subject could then not be read at all. What is not caught is an id that round-trips through an HTTP response, which needs data flow this pass does not build. Take subject ids from a plaintext field.

Deleting a tenant

subject Customer(Int) under Shop says a customer’s key is wrapped under its shop’s, so deleting the shop’s key row makes every customer key beneath it unopenable at the same instant:

hekla erase Shop 7

subject `Shop` = `7`
  key            present, and about to be deleted
  beneath it     2 key(s), which become permanently unreadable

This is irreversible. Every value scoped to these keys becomes unreadable
across the log and every read model at once.
Continue? [y/N] y
erased subject `Shop` = `7`

It asks, because naming the subject does not bound what goes when that subject has children. --yes answers without silencing the summary, and off a terminal the command refuses rather than exiting 0.

The delete is still one row for a 50,000-customer tenant, rather than one journaled erase per customer plus a projector to enumerate them. Every key beneath stays on disk and becomes unopenable, because the secret its wrapping was derived from is the one that went.

The trade is the whole of it. Per-field subjects are sold on erasing a shop provably not touching a customer’s email. Under a parent it does, deliberately, and there is no way to have both answers for one pair of subjects. One parent each, and the graph is checked acyclic.

Every event sealing under a child carries its ancestors’ ids, in plaintext, not optional and not themselves sealed. Minting customer 88’s key wraps it inside shop 7’s at that moment, and the event in front of the runtime is the only place 7 can come from, so an event with a Customer and no Shop is refused where the annotation is written, and so is an event carrying two: nothing would say which one the key hangs from, which is the shape a transfer event (from_shop, to_shop) reaches first. One id for each ancestor, once and not optional. An event that carries a Customer and seals nothing mints no key and needs no parent.

Nothing can check that a parent is right. If one event says customer 88 is in shop 7 and another says shop 9, the key was minted once, under whichever arrived first: deleting shop 9 leaves customer 88 readable, and deleting shop 7 destroys data you believe belongs to shop 9. Both events are well formed, so there is nothing for the checker to point at. This is yours, and it is worth saying beside a feature whose job is irreversible destruction.

Giving a subject a parent while it already has keys is a deploy, and the boot moves them before it serves. hekla adopt does the same work ahead of time against the live directory, and warns about a subject whose events name two parents.

What changes afterwards

That fourth row is the one to internalise. A reveal of an erased subject does not wedge the lane forever; the data is gone and no retry can recover it, so the runtime records a terminal skip and moves on.

It is a point-in-time shred

Erasure destroys the key that existed. It is not a tombstone, so a later event writing the same subject’s field mints a fresh key and is readable.

Values written under the superseded key report stale rather than erased. Five states show up on /admin:

The cipher is deterministic

It has to be, because an encrypted value is also a tag and a tag has to be matchable. The consequence is that it leaks equality and frequency.

What no checker can tell you

Which fields are personal is your judgement. A field appended without a @subject can never be erased, and nothing warns about it. That is a decision about meaning, and neither the compiler nor a model can make it from a name.

Getting it wrong is not recoverable by editing the declaration later, because the events already in the log were written in plaintext. This is the modelling decision to slow down on.

Testing it

A test cannot call erase, so it declares the state instead:

const ORDER: Uuid = "11111111-1111-1111-1111-111111111111"

test "an erased customer's address is gone" {
  given @order.placed {
    order_id: ORDER,
    customer_id: 7,
    shop_id: 1,
    email: "ada@example.com",
    shipping_address: "1 High St",
    order_total: 25.99,
    notes: "",
  }
  erased Customer "7"

  project CustomerOrders

  expect Order[ORDER] { customer_id: 7, email: none, shipping_address: none }
}

These run against a real key store, so the column is unreadable because its key is gone rather than because a flag says so.

Operating it

The master key that wraps every subject key, the rotation procedure, and what breaks if you lose one are in Keys and secrets.