Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions .github/scripts/sync-from-marfa.test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,7 @@ catalog_json="$(cat <<'EOF'
"title": { "type": "string", "description": "A title | or not" },
"language": { "type": "string", "description": "A language", "format": "bcp47" }
},
"display_hints": { "title_field": "title", "body_field": "body" },
"display_hints": { "title_field": "title", "body_field": "body", "icon": "sticky-note" },
"merge_policy": { "fields": { "body": "keep_both_copies" }, "default": "last_writer_wins" }
},
{
Expand All @@ -336,7 +336,7 @@ catalog_json="$(cat <<'EOF'
"level": { "type": "enum", "description": "How urgent", "enum_values": ["low", "high"] },
"to": { "type": "array", "description": "Who it is for", "items_type": "string" }
},
"display_hints": { "title_field": "title", "body_field": "body" },
"display_hints": { "icon": "notebook-pen" },
"merge_policy": { "fields": { "body": "keep_both_copies" }, "default": "last_writer_wins" },
"roles": ["container"]
},
Expand Down Expand Up @@ -409,8 +409,9 @@ note_expected="$(cat <<'EOF'

| Setting | Value |
| ------------------------------------------------------ | ----------------------------------------------------------------------- |
| [`title_field`](/catalog/your-own-types#fields) | `title` |
| [`body_field`](/catalog/your-own-types#fields) | `body` |
| [`title_field`](/catalog/your-own-types#display-hints) | `title` |
| [`body_field`](/catalog/your-own-types#display-hints) | `body` |
| [`icon`](/catalog/your-own-types#display-hints) | `sticky-note` |
| [`merge_policy`](/catalog/your-own-types#merge-policy) | `keep_both_copies` for `body`; `last_writer_wins` for every other field |

## Subtypes
Expand All @@ -419,9 +420,10 @@ note_expected="$(cat <<'EOF'

A memo (for a team) with a level.

| Setting | Value |
| ---------------------------------------- | ----------- |
| [`roles`](/catalog/your-own-types#roles) | `container` |
| Setting | Value |
| ----------------------------------------------- | -------------- |
| [`icon`](/catalog/your-own-types#display-hints) | `notebook-pen` |
| [`roles`](/catalog/your-own-types#roles) | `container` |

| Field | Kind | Required | Description |
| ------- | ------------------- | -------- | ---------------------------------- |
Expand Down Expand Up @@ -1169,7 +1171,7 @@ check "a subtype equal to its parent says nothing of hints or merge policy" \
"$(printf '### core.note.same [#core-note-same]\n\nSays what the parent says.')" \
"$(sed -n '/^### core.note.same/,/^### core.note.drift/p' "${drift_file}" | sed '$d')"
check "a subtype that differs says its hints and merge policy as values" \
"$(printf '%s\n%s\n%s' '| [`title_field`](/catalog/your-own-types#fields) | `language` |' '| [`body_field`](/catalog/your-own-types#fields) | `body` |' '| [`merge_policy`](/catalog/your-own-types#merge-policy) | `keep_both_copies` for every field |')" \
"$(printf '%s\n%s' '| [`title_field`](/catalog/your-own-types#display-hints) | `language` |' '| [`merge_policy`](/catalog/your-own-types#merge-policy) | `keep_both_copies` for every field |')" \
"$(sed -n '/^### core.note.drift/,$p' "${drift_file}" | grep '^| \[`')"
check "the parent says its own hints once, and a subtype equal to it adds none" 1 \
"$(grep -cE '^\| \[`title_field`\].*\| `title` +\|$' "${drift_file}")"
Expand Down
4 changes: 4 additions & 0 deletions content/docs/(overview)/items-and-types.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ When your type should be readable as another type without taking on its properti

<TypeInheritance />

## How a type is shown

A type also tells an app how to show its items, through three **display hints**: the field that names an item, the field that holds its text, and an icon. **The text is Markdown**: an app renders it, and a device reads the links in it to make [edges](/properties-metadata-and-edges). The icon is a Lucide name such as `calendar`, and the owner can choose another for any type, including the ones Marfa ships. Each hint comes from the nearest type that sets it, so a child that sets only an icon keeps its parent's title and text. [Type rules](/catalog/your-own-types#display-hints) has the details.

## Where an item came from

Every item records its **source**, the program that created it, such as `laptop` or `todoist`: the creating key's own source, or one that key claims. It never changes. An item can also carry a `source_id`, its own ID at that source, such as a task's ID in Todoist. Together they identify one record, so a connector that reads the record again updates the item instead of making a second one. [Manage keys](/run/manage-keys#let-keys-share-a-source) shows how keys share a source.
Expand Down
4 changes: 4 additions & 0 deletions content/docs/build/change-a-type.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@ let normalized = try await api.check(itemType: definition)

The answer's `type` is your definition with its defaults made explicit, and `registered_type` is the registered one in the same form. Compare those two, not `resolved_type`, which adds what the registered type inherits. Both registered forms are `null` when the identifier isn't registered. An unknown schema key, anywhere in a definition, is refused `400 invalid_schema`.

**A replacement keeps the icon the owner chose** for the type (see [display hints](/catalog/your-own-types#display-hints)), and `registered_type` shows the icon you registered, not the owner's.

**A `GET` of a type, put back as a replacement, stores only what you wrote.** A hint the type doesn't set itself (`title_field`, `body_field` or `icon`) and that you send back with the value the `GET` answered is the type's inheritance read back, not your choice. Marfa stores no value for it, so the type goes on following its parent and the owner's icon. A hint the type sets stays, including one set to a different value than its parent's. For an icon the type sets, a value equal to the owner's icon keeps the icon you registered. A hint you newly set to the value the type already inherits is stored as none, and the type reads the same.

Then replace the type:

```bash tab="CLI" tab-group="client"
Expand Down
2 changes: 2 additions & 0 deletions content/docs/build/connector-pull-and-push.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ An item's natural key is its `source`, which comes from the connector's key, and

`atomic: false` writes the records that pass when one is refused, and reports that one as `errored`. By default, one refused record **rolls the whole batch back**.

**Write an item's text as Markdown.** The field its type names in [`display_hints.body_field`](/catalog/your-own-types#display-hints) holds Markdown, and Marfa stores what you send without checking it. A vendor that returns HTML needs the connector to convert it first, or an app shows the tags.

## Write only what changed

A write of the same properties an item already holds **still raises its `version`** and announces an `item.updated` event. A pull that rewrites every record on every run therefore makes noise for everyone who follows the event stream.
Expand Down
4 changes: 2 additions & 2 deletions content/docs/build/connector.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -121,11 +121,11 @@ A connector writes items of a type of its own. Save its definition beside the co
"description": "The entry's ID in the reading list service."
}
},
"display_hints": { "title_field": "title" }
"display_hints": { "icon": "bookmark-plus" }
}
```

Its parent is `core.bookmark`, so an entry already has a `title`, a `url` and an `author`. The type adds only the entry's ID in the service. [Define a type](/build/define-a-type) explains each part.
Its parent is `core.bookmark`, so an entry already has a `title`, a `url` and an `author`, and shows by the title and text its parent names. The type adds the entry's ID in the service and an icon, which the owner can replace with another. [Define a type](/build/define-a-type) explains each part.

</Step>
<Step>
Expand Down
2 changes: 2 additions & 0 deletions content/docs/build/define-a-type.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ Marfa made the label from the identifier, and `provenance.created_by` names who

`marfa types get app.example.receipt` (`GET /types/{id}`) answers the type with the properties it inherits, such as `blob_ref`, `mime_type`, `width` and `height`. An item must hold **every required property along the chain**, so a receipt needs a `blob_ref`, a `mime_type` and a `merchant`. A write that lacks one is refused `400 invalid_properties`, naming it.

A type can also set [`display_hints`](/catalog/your-own-types#display-hints): the field that names an item, the field that holds its Markdown text, and a Lucide `icon`. Each one comes from the nearest type that sets it, so a receipt that sets only an icon keeps the title its parent names.

Once the type is registered, [change or delete it](/build/change-a-type) as your app grows.

## What your key needs
Expand Down
34 changes: 32 additions & 2 deletions content/docs/catalog/(types)/your-own-types.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Type rules
description: "The rules a type you register must meet: its identifier, fields, parent, roles, merge policy and retention policy."
description: "The rules a type you register must meet: its identifier, fields, parent, display hints, roles, merge policy and retention policy."
---

**Marfa refuses a definition that breaks a rule with `400 invalid_schema`** unless the rule names another answer. [Define a type](/build/define-a-type) shows how to register one.
Expand Down Expand Up @@ -28,7 +28,7 @@
| `maxLength`, `maxItems` | A whole number of at least 1. `maxLength` applies to `string` and `enum`, `maxItems` to `array`. |
| Name | Not one of the reserved field names below. Refused `400 property_shadows_field`. |
| `thumbnail` | At most one, counting inherited ones. Not named `title`, `body`, `description` or `name`, and not an array's `items_type`. |
| `title_field`, `body_field` | Name a field the type declares or inherits, and not a `thumbnail` field. |
| `title_field`, `body_field` | Name a field the type declares or inherits, and not a `thumbnail` field. See [Display hints](#display-hints). |

**A write that leaves a wrong element in an `array` field** is refused `400 invalid_properties`, and `details.errors` names the field and position, as `names.1`.

Expand Down Expand Up @@ -56,6 +56,36 @@

A replacement or deletion that breaks another type's claim is refused the same way, and `details.dependent_type_ids` names the claimants. A new target that isn't a platform type needs write on it, as a `parent` does.

## Display hints

`display_hints` says how an app shows an item of the type. It takes three optional hints:

| Hint | What it holds | Marfa refuses |
| --- | --- | --- |
| `title_field` | The name of the field that gives an item its name. | A field the type and its ancestors don't declare, or a `thumbnail` field. |
| `body_field` | The name of the field that holds the item's text. **The value is Markdown.** Apps render it, and a device reads it to make [edges](/catalog/edges) from the links it contains. Marfa stores the value as sent and doesn't check it. | The same as `title_field`. |
| `icon` | The name of a [Lucide](https://lucide.dev/icons/) icon in kebab-case, such as `star` or `file-music`. | A name outside Lucide 1.61.0, the one release Marfa pins: a different case, a name from a later release, or an icon Lucide removed. **Use the canonical name, not an alias:** `home` is refused and `house` is accepted. Refused `400 invalid_schema`, naming `display_hints.icon`. |

Every operation that takes a type definition applies the same rule, including [normalize a type definition](/api-reference/types/normalizeType). The names Marfa accepts are in the [generated list](https://github.com/withmarfa/marfa/blob/main/packages/types/generated/lucide-icons.json).

**Each hint comes from the nearest type that sets it:** the type itself, then its parent, then that type's parent. A type that sets only an `icon` still shows the title and body its parent names. `GET /types/{id}` answers each hint after that choice, and `GET /types` lists the hints each type declares itself, with the owner's icon in place of the author's.

### The owner's icon

**The owner can choose the icon of any type**, including the types Marfa ships and a connector's types, without replacing the definition:

```bash
marfa types set-icon user.recipe chef-hat
marfa types clear-icon user.recipe
```

The same two operations are [set a type's icon](/api-reference/types/setTypeIcon) and [clear a type's icon](/api-reference/types/clearTypeIcon).

- **The choice sits beside the definition.** Replacing the type, as a connector does each time it starts, keeps it, and deleting the type discards it.
- **Reads answer the owner's icon** in place of the author's. A subtype with no icon of its own shows its parent's, and one with its own keeps it unless the owner chose another for that subtype.
- **Devices take it the next time they read the catalog:** at a catch-up, or when a follow next opens its stream. Marfa sends no event for a change to a type, so a device shows the old icon until then.
- **Who can change it:** the owner, a local process, or a key with `schema.write` and write on the type in its type map. A key without `schema.write` is refused `403 forbidden`, and one without write on the type `403 type_not_permitted`. A connector's key can change the icon only if it holds both.

## Roles

`roles` takes `container` alone. A `container` type can be the target of an [`in-collection`](/catalog/edges#in-collection) edge.
Expand Down
8 changes: 6 additions & 2 deletions content/docs/run/export-and-restore.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Export and restore an archive
description: Export your items, edges, types and files to an archive, and restore it into the same instance or another, such as a new build.
---

An archive carries your content to another instance: items with their tags, extensions and earlier versions, edges, the types and edge types you registered, and file bytes. It leaves the instance's own setup behind, so it holds no keys, webhooks, configuration, audit log or owner's picture. It can move your data to a new build when you [upgrade](/run/upgrade), but **test the restore on fresh storage first**: a different build may refuse the archive.
An archive carries your content to another instance: items with their tags, extensions and earlier versions, edges, the types and edge types you registered, the icons you chose for types, and file bytes. It leaves the instance's own setup behind, so it holds no keys, webhooks, configuration, audit log or owner's picture. It can move your data to a new build when you [upgrade](/run/upgrade), but **test the restore on fresh storage first**: a different build may refuse the archive.

An archive isn't a long-term backup, because only the build that wrote it is sure to read it. To recover an instance from a failed disk, [back up the data directory](/run/back-up-and-restore) instead.

Expand Down Expand Up @@ -158,7 +158,10 @@ marfa --socket /srv/marfa/data/control/marfa.sock restore marfa-archive.tar.gz
"types_registered": 0,
"types_skipped": 0,
"edge_types_registered": 0,
"edge_types_skipped": 0
"edge_types_skipped": 0,
"type_icons_added": 0,
"type_icons_skipped": 0,
"type_icons_skipped_reasons": {}
}
```

Expand All @@ -168,6 +171,7 @@ The same archive restored again answers `"imported": 0`, `"duplicates": 6` and `

- **`blobs_missing`** lists the blobs the archive names but carries no bytes for, and no store of the instance holds. The restore still writes the items but no record for those blobs, so their files have no bytes until you [upload them again](#when-an-archive-lacks-a-files-bytes). The CLI then exits with code 1 and `blobs_missing`.
- **`edges_skipped_reasons`** counts why edges were skipped, such as `already_present` or `endpoint_missing`. An edge to an item in the trash is restored.
- **`type_icons_added`, `type_icons_skipped` and `type_icons_skipped_reasons`** count the icons you chose for types (see [the owner's icon](/catalog/your-own-types#the-owners-icon)). A restore adds an icon only for a type that has none from you yet, so it never replaces a choice you made on the instance since the export, and counts that skip as `already_set`. An icon that isn't a name in the pinned Lucide release, such as one from a build that pins a later release, is skipped and counted as `unknown_icon`, and the rest of the archive restores. A restore refuses the whole archive if an icon line is damaged, or names a type that neither the archive nor the instance registers.

A restore writes in one transaction. If it's refused, fails or is interrupted, it leaves nothing behind, so you can run it again. Other writes wait while it runs, so let it answer before you start your apps.

Expand Down
2 changes: 1 addition & 1 deletion content/docs/run/upgrade.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ marfa --socket /srv/marfa/data/control/marfa.sock restore marfa-archive.tar.gz

### Recreate what an archive doesn't hold

Make your keys again, register your webhooks again, set the configuration again and upload the owner's picture again: **an archive carries none of them**. Sign your apps in again too. Restored app connections are revoked historical records, and their items carry `restored:` sources rather than the sources of the apps you reconnect. [Manage keys](/run/manage-keys), [Settings](/run/settings) and [Change the owner's profile](/run/sign-in-as-the-owner#change-the-owners-profile) cover them.
An archive carries the icons you chose for types, so those need no step here. Make your keys again, register your webhooks again, set the configuration again and upload the owner's picture again: **an archive carries none of them**. Sign your apps in again too. Restored app connections are revoked historical records, and their items carry `restored:` sources rather than the sources of the apps you reconnect. [Manage keys](/run/manage-keys), [Settings](/run/settings) and [Change the owner's profile](/run/sign-in-as-the-owner#change-the-owners-profile) cover them.

</Step>
<Step>
Expand Down
Loading
Loading