diff --git a/content/docs/(reference)/changelog.mdx b/content/docs/(reference)/changelog.mdx index f50ab107..7969878f 100644 --- a/content/docs/(reference)/changelog.mdx +++ b/content/docs/(reference)/changelog.mdx @@ -45,7 +45,7 @@ For apps and the API: For folders: -- **A folder writes every attached file beside its document**, whether or not the body embeds it, and lists the files no embed places under the document's `has-attachment` line. A file follows its document to another directory, and leaves with it. A second folder on the same Mac that holds the file keeps its own copy, and a folder pull no longer deletes and rewrites it on every pass. See [Files attached to a note](/guides/sync-a-folder#files-attached-to-a-note). +- **A folder writes every attached file beside its document**, whether or not the body embeds it, and lists the files no embed places under the document's `has-attachment` line. A file follows its document to another directory, and leaves with it. A second folder on the same Mac that holds the file keeps its own copy, and a folder pull no longer deletes and rewrites it on every pass. See [Files attached to a note](/guides/folder-files#files-attached-to-a-note). ## v0.0.34 @@ -595,7 +595,7 @@ For the CLI: - **A refused `marfa folders add` leaves behind no directory it made.** - **`marfa folders push --send-only` sends without touching files.** It sends what a folder holds and writes or removes no file in it. [Send without pulling](/guides/keep-folders-running#send-without-pulling) covers it. - **`marfa folders unmatched ` lists files the folder's search no longer matches,** and `--remove` takes them away after a confirmation or with `--yes`. [Remove files a folder keeps but no longer holds](/guides/keep-folders-running#remove-files-a-folder-keeps-but-no-longer-holds) covers it. -- **A Markdown document's file name and title stay in step.** Renaming the file sets the title, and a new title renames a file whose name still matches the old title; a name you chose, or an empty title, leaves the file name as it is. [Edit on either side](/guides/sync-a-folder#edit-on-either-side) covers it. +- **A Markdown document's file name and title stay in step.** Renaming the file sets the title, and a new title renames a file whose name still matches the old title; a name you chose, or an empty title, leaves the file name as it is. [Edit on either side](/guides/folder-files#edit-on-either-side) covers it. - **The CLI can send telemetry.** With `OTEL_EXPORTER_OTLP_ENDPOINT` set, it exports one trace for each command, and with `MARFA_POSTHOG_HOST` and `MARFA_POSTHOG_PROJECT_TOKEN` too, a failed command goes to PostHog error tracking; with none set, it sends nothing. [What the CLI sends](/guides/telemetry#what-the-cli-sends) covers it. For apps and devices: diff --git a/content/docs/guides/folder-files.mdx b/content/docs/guides/folder-files.mdx new file mode 100644 index 00000000..14664bb8 --- /dev/null +++ b/content/docs/guides/folder-files.mdx @@ -0,0 +1,80 @@ +--- +title: Files in a folder +description: See what a file holds when a folder writes an item as a file, where attached files go, and how an edit on either side reaches the other. +--- + +A folder writes each item as a file and reads a changed file back as an edit. The examples follow the `Notes` folder made in [Sync a folder](/guides/sync-a-folder). + +## How items appear as files + +A note is a Markdown file named after its title, or after its ID if it has none. Here's `Reading list.md`: + +```markdown title="Notes/Reading list.md" +--- +type: core.note +tier: library +title: Reading list +marfa_id: 01a1232d-947e-7399-86e1-97648a87c232 +marfa_version: 1 +--- +Start with the essays, then the letters. +``` + +The lines between the `---` fences are the frontmatter. The text after them is the body. + +- **`type`:** the item's type when the value is a Marfa type identifier. A value that is not a type identifier belongs to the item's `type` property instead and survives writing the file back. +- **`tier`, `tags`, `state` and `occurred_at`:** the item's own fields. +- **A line named after an edge type,** such as `references`: an edge. +- **Every other line:** a property. +- **`marfa_id` and `marfa_version`:** the item and the version the file was written from. Leave them to Marfa. + +A new Markdown file needs no frontmatter. It takes its type and tier from the folder's `defaults` or its query, and its title from `defaults` or, without one, its file name. With `query.tier: all`, a new file without an explicit tier uses `defaults.tier` or, without one, `library`. Once the instance creates the item, Marfa writes the frontmatter and `marfa_id` into the file. + +A `.txt` file is all body. Any other file the query admits, such as a photo, becomes a file item with its bytes uploaded as a blob, as [Handle files](/guides/handle-files) describes. A file item stays a file item at a `.md`, `.markdown` or `.txt` name, such as after a pull places it there or you rename it: its bytes go to the item as they are, and Marfa doesn't read them as a note. + +## Files attached to a note + +**Every file attached to a note the folder holds is written with the note**, whether or not the query names its type. A file the note's body embeds, such as `![](photos/map.png)` or `![[map.png]]`, is written where the link says. Every other attached file is written beside the note and listed under its `has-attachment` line: + +```markdown title="Notes/Packing for the trip.md" +--- +type: core.note +tier: library +title: Packing for the trip +has-attachment: + - "[[map.png]]" +marfa_id: 01a1232d-94b0-7c1e-9d2a-3f6b8e0c5a71 +marfa_version: 2 +--- +Pack light. +``` + +- **It follows the note.** Move the note to another directory and the next sync moves the file beside it. A file you move away from its note goes back beside the note; to keep it somewhere else, embed it by its path. +- **One file, one place.** A file attached to notes in two directories is written once, beside the first note in path order, and the sync reports the other note. Two attached files with the same name are told apart by a number. +- **Another folder keeps its own copy.** If another folder on the same Mac holds the file through its own query, each folder keeps a copy, and neither takes the other's. When a note leaves one folder's query for another's, its attached files move with it. +- **It leaves with the note.** When every note it's attached to goes to the bin or leaves the states the query holds, the file goes too, under the same [safeguards](/guides/folder-safeguards) as any other. If the query holds the file itself, it stays. +- **Remove the line, keep the file.** Take a file out of `has-attachment` and the next sync removes the `attached-to` edge. The file stays in the folder as one it no longer holds. + +The folder reads attached files with its key, so the key needs read on their types, such as `core.file.*` in [the key you made for the folder](/guides/sync-a-folder#make-a-key-for-the-folder). A file whose bytes the folder can't fetch is reported, and the sync goes on. + +## Edit on either side + +Edit a file and sync, and the edit reaches its item: + + + +```bash +marfa folders push Notes +``` + +```text +0 created, 1 updated, 0 renamed, 2 unchanged, 0 missing, 0 deleted, 0 skipped +the server answered 1 write(s) +1 file(s) written +``` + + + +An edit is a write based on the `marfa_version` the file names. If another app changed the same note meanwhile, the instance settles the two as [Versions](/versions) explains: changes to different fields merge, and a collision the type can't merge leaves a conflicted copy, which appears as a second file. + +A note's file name and its title stay in step. Rename the file and the new name, without its extension, becomes the title. Change the title anywhere else and the next sync renames the file, numbering the name if another file has it. The folder leaves a name alone when it can't tell whether you chose it. diff --git a/content/docs/guides/folder-from-a-mac-app.mdx b/content/docs/guides/folder-from-a-mac-app.mdx new file mode 100644 index 00000000..367c4761 --- /dev/null +++ b/content/docs/guides/folder-from-a-mac-app.mdx @@ -0,0 +1,26 @@ +--- +title: Use a folder from a Mac app +description: Create a folder, bind a directory to it and confirm its first sync from a Mac app, with the Swift SDK's Folders. +--- + +A Mac app can do what `marfa folders` does. **The Swift SDK's `Folders` shares the Mac's list of folders with `marfa folders`.** [Sync a folder](/guides/sync-a-folder) explains what a folder is and the CLI's steps. + +This code creates the folder, binds a directory and confirms the first sync, with `serverURL`, `key` and `storeURL` as in [Open a working copy](/swift-sdk#open-a-working-copy). Until the first sync is confirmed, `sync(_:)` returns `.awaitingConfirmation` with the plan: + +```swift +import Marfa + +let server = Server(url: serverURL, key: key) +let copy = try await WorkingCopy.open(store: storeURL, server: server) +let folder = try await copy.createFolder( + FolderSettings(title: "Notes", query: FolderQuery(types: ["core.note"]))) + +let folders = Folders(server: server) +let directory = URL(filePath: "/Users/sam/Notes") +_ = try await folders.add(directory, following: folder.id) +_ = try await folders.sync(directory) // .awaitingConfirmation(plan) +_ = try await folders.confirmFirstSync(in: directory) +_ = try await folders.sync(directory) +``` + +[Folders on a Mac](/swift-sdk#folders-on-a-mac) lists every call, including `changeFolder(_:_:baseVersion:)`. A client without the SDK calls [Create a folder](/api-reference/folders/createFolder) and the other folder operations. diff --git a/content/docs/guides/folder-safeguards.mdx b/content/docs/guides/folder-safeguards.mdx index 43bdaa8c..4ad72357 100644 --- a/content/docs/guides/folder-safeguards.mdx +++ b/content/docs/guides/folder-safeguards.mdx @@ -42,7 +42,7 @@ the server answered 0 write(s) A removal pauses when it takes **more than 10 files and more than a quarter of the folder's files**, which the `removal_threshold` setting changes. It also holds files whose items left the folder elsewhere, such as notes another app moved to the bin. -Cancel the removal to write the files back, or confirm it to move unchanged items to the bin at the next sync. Files attached to a note come back with the note, whether or not the query holds them. Confirmation checks for moves and remote edits too: a missing file found nowhere comes back if another device changed content it showed while the query still holds it. In Swift, `folders` and `directory` are those of [Use a folder from a Mac app](/guides/sync-a-folder#use-a-folder-from-a-mac-app): +Cancel the removal to write the files back, or confirm it to move unchanged items to the bin at the next sync. Files attached to a note come back with the note, whether or not the query holds them. Confirmation checks for moves and remote edits too: a missing file found nowhere comes back if another device changed content it showed while the query still holds it. In Swift, `folders` and `directory` are those of [Use a folder from a Mac app](/guides/folder-from-a-mac-app): ```bash tab="CLI" tab-group="client" marfa folders restore Notes diff --git a/content/docs/guides/meta.json b/content/docs/guides/meta.json index c14ac3be..eec37182 100644 --- a/content/docs/guides/meta.json +++ b/content/docs/guides/meta.json @@ -22,6 +22,8 @@ "handle-files", "receive-webhooks", "sync-a-folder", + "folder-files", + "folder-from-a-mac-app", "keep-folders-running", "folder-safeguards", "---Build connectors---", diff --git a/content/docs/guides/sync-a-folder.mdx b/content/docs/guides/sync-a-folder.mdx index eedebd00..e5c45bde 100644 --- a/content/docs/guides/sync-a-folder.mdx +++ b/content/docs/guides/sync-a-folder.mdx @@ -4,11 +4,13 @@ description: Keep a directory on your computer and a set of items in step, so ea --- - **You'll build** a folder on your computer that follows a set of items, so each note is a Markdown file you can edit in any app. -- **You need** the `marfa` CLI and an instance you own, as the [Quickstart](/quickstart) sets them up. You make the first key on the instance's own computer. A [Mac app](#use-a-folder-from-a-mac-app) does the same with the Swift SDK. +- **You need** the `marfa` CLI and an instance you own, as the [Quickstart](/quickstart) sets them up. You make the first key on the instance's own computer. - **You'll have** a directory with a file for each note, where a change on either side reaches the other. A folder is a [device](/devices) whose working copy is a directory. **Its settings live on the instance** as a `system.folder` item, so every computer that follows the folder shares them, and each keeps its own copy and queue of writes. If you stop following the folder, the files stay behind as plain files. [Folder safeguards](/guides/folder-safeguards) says when a folder removes a file or an item. +The steps use the CLI. A Mac app does the same with the Swift SDK, as [Use a folder from a Mac app](/guides/folder-from-a-mac-app) shows. + @@ -113,79 +115,7 @@ The directory now holds a file for each note, and `Ideas.md` is an item: -## How items appear as files - -A note is a Markdown file named after its title, or after its ID if it has none. Here's `Reading list.md`: - -```markdown title="Notes/Reading list.md" ---- -type: core.note -tier: library -title: Reading list -marfa_id: 01a1232d-947e-7399-86e1-97648a87c232 -marfa_version: 1 ---- -Start with the essays, then the letters. -``` - -The lines between the `---` fences are the frontmatter. The text after them is the body. - -- **`type`:** the item’s type when the value is a Marfa type identifier. A value that is not a type identifier belongs to the item’s `type` property instead and survives writing the file back. -- **`tier`, `tags`, `state` and `occurred_at`:** the item’s own fields. -- **A line named after an edge type,** such as `references`: an edge. -- **Every other line:** a property. -- **`marfa_id` and `marfa_version`:** the item and the version the file was written from. Leave them to Marfa. - -A new Markdown file needs no frontmatter. It takes its type and tier from the folder's `defaults` or its query, and its title from `defaults` or, without one, its file name. With `query.tier: all`, a new file without an explicit tier uses `defaults.tier` or, without one, `library`. Once the instance creates the item, Marfa writes the frontmatter and `marfa_id` into the file. - -A `.txt` file is all body. Any other file the query admits, such as a photo, becomes a file item with its bytes uploaded as a blob, as [Handle files](/guides/handle-files) describes. A file item stays a file item at a `.md`, `.markdown` or `.txt` name, such as after a pull places it there or you rename it: its bytes go to the item as they are, and Marfa doesn't read them as a note. - -## Files attached to a note - -**Every file attached to a note the folder holds is written with the note**, whether or not the query names its type. A file the note's body embeds, such as `![](photos/map.png)` or `![[map.png]]`, is written where the link says. Every other attached file is written beside the note and listed under its `has-attachment` line: - -```markdown title="Notes/Packing for the trip.md" ---- -type: core.note -tier: library -title: Packing for the trip -has-attachment: - - "[[map.png]]" -marfa_id: 01a1232d-94b0-7c1e-9d2a-3f6b8e0c5a71 -marfa_version: 2 ---- -Pack light. -``` - -- **It follows the note.** Move the note to another directory and the next sync moves the file beside it. A file you move away from its note goes back beside the note; to keep it somewhere else, embed it by its path. -- **One file, one place.** A file attached to notes in two directories is written once, beside the first note in path order, and the sync reports the other note. Two attached files with the same name are told apart by a number. -- **Another folder keeps its own copy.** If another folder on the same Mac holds the file through its own query, each folder keeps a copy, and neither takes the other's. When a note leaves one folder's query for another's, its attached files move with it. -- **It leaves with the note.** When every note it's attached to goes to the bin or leaves the states the query holds, the file goes too, under the same [safeguards](/guides/folder-safeguards) as any other. If the query holds the file itself, it stays. -- **Remove the line, keep the file.** Take a file out of `has-attachment` and the next sync removes the `attached-to` edge. The file stays in the folder as one it no longer holds. - -The folder reads attached files with its key, so the key needs read on their types, such as `core.file.*` in the first step. A file whose bytes the folder can't fetch is reported, and the sync goes on. - -## Edit on either side - -Edit a file and sync, and the edit reaches its item: - - - -```bash -marfa folders push Notes -``` - -```text -0 created, 1 updated, 0 renamed, 2 unchanged, 0 missing, 0 deleted, 0 skipped -the server answered 1 write(s) -1 file(s) written -``` - - - -An edit is a write based on the `marfa_version` the file names. If another app changed the same note meanwhile, the instance settles the two as [Versions](/versions) explains: changes to different fields merge, and a collision the type can't merge leaves a conflicted copy, which appears as a second file. - -A note's file name and its title stay in step. Rename the file and the new name, without its extension, becomes the title. Change the title anywhere else and the next sync renames the file, numbering the name if another file has it. The folder leaves a name alone when it can't tell whether you chose it. +[Files in a folder](/guides/folder-files) says what each file holds, where attached files go, and how an edit on either side reaches the other. ## Change a folder's settings @@ -234,7 +164,7 @@ Content-Type: application/json {"version":1,"query":{"types":["core.note"]}} ``` -Use the version you read in place of `` or `1`. The CLI also takes the JSON directly through `--body`. **To repair a folder that still stores `search`, use a server at v0.0.33 or later** and send the intended `query`, leaving out `search`. Copy every part of the old selection you want to keep: the server removes `search` but never translates it into `query`. Read the item again to check the accepted settings, then sync the folder. Editing `.marfa/folder.yaml` alone cannot remove a stored setting that prevents the folder from opening. +Use the version you read in place of `` or `1`. The CLI also takes the JSON directly through `--body`. **To repair a folder that still stores `search`, use a server at v0.0.33 or later** and send the intended `query`, leaving out `search`. Copy every part of the old selection you want to keep, because the server never translates `search` into `query`. Editing `.marfa/folder.yaml` alone cannot remove it. Read the item again to check the accepted settings, then sync the folder. **Moving a type's items changes the folder too.** When you [move items to another type](/guides/move-items-to-another-type), Marfa changes `query.types`, `defaults.type`, `first_placement` and the `type eq` and `type neq` terms in `query.filter` in every folder that names the old type. Each item keeps its file, and the next sync writes the new type into the file's frontmatter. @@ -242,36 +172,14 @@ Use the version you read in place of `` or `1`. The CLI also takes the JSON d A new key with the same `--source` is refused `409 conflict` while the old key is live. [Revoke the old key](/guides/manage-keys#revoke-a-key), run the `keys create` command from the first step again, then run `marfa keys keep` so the folder reads the new key from the keychain. -Then run `marfa folders push Notes`. If the key's reach changed, the push rebuilds the working copy once and sends the waiting writes in the same run, keeping earlier verdicts. If the reach changes again during that run, the push stops and keeps the remaining writes queued. Try again once the key's access is settled. +Then run `marfa folders push Notes`. If the key's reach changed, the push rebuilds the working copy once and sends the waiting writes in the same run, keeping earlier verdicts. If the reach changes again during that run, it stops and keeps the remaining writes queued: try again once the access is settled. ## Move to a newer CLI -Each folder keeps its working copy, and its record of which file is which item, in a store that one build of the core made. When a newer CLI uses a different shape, the folder's next push, pull, scan or watch **replaces the store itself, as long as the old store holds no write still to be sent and no answer the folder has yet to record**. It says so on standard error, keeps the folder's record of its files and its first sync, and changes no file. If writes are waiting, push the folder with the earlier CLI first. If a write the server refused or gave up on stays after that push, remove the folder with the earlier CLI and add it again with the newer one. [Troubleshooting](/guides/troubleshooting#staying-in-step) covers each case. +Each folder keeps its working copy, and its record of which file is which item, in a store that one build of the core made. When a newer CLI uses a different shape, the folder's next push, pull, scan or watch **replaces the store itself, as long as the old store holds no write still to be sent and no answer the folder has yet to record**. It says so on standard error, keeps the folder's record of its files and its first sync, and changes no file. If writes are waiting, push the folder with the earlier CLI first. [Troubleshooting](/guides/troubleshooting#staying-in-step) covers a write the server refused or gave up on. ## Stop following a folder To stop following a folder on one computer, run `marfa folders remove Notes`. It is refused while writes wait, apart from a first sync still waiting for confirmation. The folder's own state goes, and **the files stay where they are, as plain files**. -## Use a folder from a Mac app - -The Swift SDK's `Folders` shares the Mac's list of folders with `marfa folders`. This creates the folder, binds a directory and confirms the first sync, with `serverURL`, `key` and `storeURL` as in [Open a working copy](/swift-sdk#open-a-working-copy). Until the first sync is confirmed, `sync(_:)` returns `.awaitingConfirmation` with the plan: - -```swift -import Marfa - -let server = Server(url: serverURL, key: key) -let copy = try await WorkingCopy.open(store: storeURL, server: server) -let folder = try await copy.createFolder( - FolderSettings(title: "Notes", query: FolderQuery(types: ["core.note"]))) - -let folders = Folders(server: server) -let directory = URL(filePath: "/Users/sam/Notes") -_ = try await folders.add(directory, following: folder.id) -_ = try await folders.sync(directory) // .awaitingConfirmation(plan) -_ = try await folders.confirmFirstSync(in: directory) -_ = try await folders.sync(directory) -``` - -[Folders on a Mac](/swift-sdk#folders-on-a-mac) lists every call, including `changeFolder(_:_:baseVersion:)`. A client without the SDK calls [Create a folder](/api-reference/folders/createFolder) and the other folder operations. - -The rules behind this guide are in the [folders](https://github.com/withmarfa/marfa/blob/main/conformance/spec/folders.md) and [device](https://github.com/withmarfa/marfa/blob/main/conformance/spec/device.md) chapters of the contract. +The rules behind a folder are in the [folders](https://github.com/withmarfa/marfa/blob/main/conformance/spec/folders.md) and [device](https://github.com/withmarfa/marfa/blob/main/conformance/spec/device.md) chapters of the contract.