Migrations

An app's migrations/ folder holds its data model and its data, one version per file. Each runs once, in order, when the server starts. A new install gets every version; an existing one gets the ones it hasn't had yet.

V1__init.json          # the collections, as schema.json has them
V2__sample_data.json   # records: leave the file out and there is no sample data
V3__school_year.js     # data made up or moved, in JavaScript
V4__trips_price.json   # a change to the data model, as a patch
V5__tidy.sql           # SQL, when that is the simplest thing to say
R__views.sql           # repeatable: runs again whenever it changes

Versions

A file is V<version>__<words>: V1, V2, V2_1 (between V2 and V3). What ran is recorded in the database with a checksum of the file. A file that already ran must not change: if it does, the server stops before running anything after it and says which file. Put the change in a new version instead. An R__ file runs after the versioned ones, and again each time its contents change.

migrations/ sits in the app’s folder, or beside it, where it is shared by the apps of one repository (an app and its reports/, say) and runs once for them all.

Kinds of file

.json is an import document, in the same form as schema.json: collections, then records. Records go in the order their relations need, so a class exists before the absences that name it.

Besides records, an import document can hold what isn’t a collection: settings, conversations with their messages, pages with their blocks and rows, social feeds, and posts in them. A post has an id, a feed, an author and a body. It can also have reply_to (an answer, in its thread’s feed), created, likes (who liked it) and pinned. Posts go in the same way as ones made in the app, so their counts, #tags and search agree. A page that is already there is left as it is, unless the document says "merge": true: then its folder settings, values, template, navigation, sign-ups and look are updated, and its words and rows stay.

{ "posts": [
    { "id": "welcome", "feed": "school", "author": "principal0001", "body": "Welcome back! #backtoschool", "pinned": true },
    { "id": "welcome-1", "reply_to": "welcome", "author": "parent0000001", "body": "Thank you!" }
] }

A patch changes the data model by name. It is a JSON merge patch: an object adds or changes, null takes away, along with the data it held. It is the one kind of migration that removes anything, and only what it names.

{ "patch": {
    "trips": { "fields": { "price": { "type": "number" }, "old_note": null } },
    "invoices": null
} }

A file can hold both: the patch runs first, then its collections, then its records, so a version’s data model and its data go together.

.js or .ts export a function, run like a job, with ctx.collection("trips").update(…). It can also return a document ({ records: … } or a patch), which is imported as a .json file’s would be. That is the way to generate a lot of data:

export default function () {
  const grades = pupils.flatMap((p) => marksFor(p));
  return { records: { grades } };
}

.sql runs as one script, in one transaction.

Running them

They run when sluurp serve starts, before the app’s agents. To see what would run, or to run them without serving:

$ sluurp migrate ./app --plan
$ sluurp migrate ./app

All or nothing

Before the first migration that’s due, Sluurp takes a snapshot of the database, as a backup does: snapshots/<when>-migration/. Then the migrations run in order. If one fails, the snapshot is put back: the collections, the records and the record of what ran, as they were before this run. The error names the file.

So nothing is left half done. A .json file’s collections and records go, and so does every write a .js file made before it threw, along with the versions before it in the same run. An app starts with all of its new migrations, or none of them. Fix the file and start again.

The snapshot is kept either way, a backup from just before the upgrade. The admin UI’s Backups lists it.

An app with only a schema.json keeps working as before: on every start its collections are added or changed, and nothing is removed.