# PaperPeace sidecar format This document describes the metadata files PaperPeace keeps in a folder of their own at the storage location, separate from your documents, so that other programs, scripts and language models can read them. It is complete enough to read a library without knowing anything else about the app. - Canonical address of this file: - JSON Schema of the header: - Same text as a web page: (German: ) - The English text is the authoritative one. ## 1. What a sidecar is PaperPeace keeps the original files of a library (PDFs, images) exactly as the user has them, in the user's own folders. Everything the app knows about a document – title, type, correspondent, tags, status, dates, notes – is stored in a small Markdown file, the **sidecar**, in a separate folder of the same storage location. There is one sidecar per document. There is no server and no database you have to ask: the sidecars are the metadata. All sidecars are plain text. A library can be read with `grep`, a text editor or a script. ## 2. Where the files are Paths are relative to the **library root**, the folder the user chose as storage location. Separator is `/`. ``` / Rechnungen/2026/2026-03-14 Stromrechnung.pdf the original, in the user's folders _Meta/ Sidecars/ 6c2b0f7a-3e41-4b0e-9c55-2d9a1f0b7e33.md the sidecar 6c2b0f7a-3e41-4b0e-9c55-2d9a1f0b7e33.ocr.json the recognised text (see section 9) 6c2b0f7a-3e41-4b0e-9c55-2d9a1f0b7e33.lock internal, ignore it _Deleted/ 6c2b0f7a-3e41-4b0e-9c55-2d9a1f0b7e33.pdf an original the user moved to the trash ``` - All sidecars are in **one flat folder**, `_Meta/Sidecars/`. They are not next to the originals and not in subfolders. - The file name of a sidecar is the **document identifier** plus `.md`. The identifier is a random UUID in lower case (for example `6c2b0f7a-3e41-4b0e-9c55-2d9a1f0b7e33`) created together with the document; it never changes (not when the document is renamed, moved, deleted or restored). It carries no meaning and is not to be sorted by. - The header field `doc_id` repeats the identifier. **If the file name and `doc_id` do not agree, the file is not a valid sidecar** and must be ignored. A file name with upper-case letters is not a sidecar either. A copy such as `6c2b0f7a-… (conflict).md` made by a storage provider is therefore not a second document. - A `.md` file anywhere else, outside `_Meta/Sidecars/`, is never a PaperPeace sidecar. It belongs to the user. - **Which original belongs to a sidecar:** the header field `source_file` holds the path of the original, from the library root (`Rechnungen/2026/2026-03-14 Stromrechnung.pdf`). A name without a folder means the original is in the root. This is the only link; sidecars are named by identifier, not by the name of the original. The path changes whenever the user renames or moves the document, so always read it from the current sidecar. - **Trash:** a document the user deleted has the field `deleted_at` set. Its original then lies in `_Deleted/.` (no subfolders); the sidecar stays where it is. Only `deleted_at` says whether a document is deleted, not where the original lies. ## 3. File format - **Encoding:** UTF-8, without a byte order mark. A file that is not valid UTF-8 cannot be read. Lines end with LF (`\n`); readers should tolerate CRLF. - **Layout:** a header (front matter) followed by a body. ``` --- key: value key: value checksum: … --- ## Section text… ``` - The first line is `---`. The header runs to the next line that is `---`. Everything after that is the body. A file without both lines cannot be read. - Each header line is `key: value`. The key is everything before the **first** colon, the value everything after it, both without surrounding spaces. A colon inside a value is therefore fine and is not quoted. A line without a colon is ignored. If a key appears twice, the last line wins. - The header is **flat**: no nesting, no multi-line values, no YAML features. It looks like YAML but is not parsed as YAML; do not use a YAML parser (it would, for example, turn `asn: 0042` or a 64-bit number into a different value). **Read every value as text** and convert it yourself where you need a number. - A value is **one line**. Writers remove line breaks and control characters from every value. A value can be empty (`person:`); an empty value and a missing line mean the same. - **Lists** are written in square brackets with items separated by comma and space: `tags: [Energie, Haushalt]`. Only `tags`, `status`, `cases` and multi-select custom fields (section 5) are lists. An item that contains `[`, `]`, `,` or `"`, or starts or ends with a space, is put in double quotes, with `\\` for a backslash, `\"` for a quote and `\n` for a line break inside the quotes: `tags: ["Haus, Hof", Steuer]`. Empty items are dropped. - **Dates and times** are always UTC. A *day* is `YYYY-MM-DD` (`2026-03-14`). A *moment* is `YYYY-MM-DDTHH:MM:SSZ` (`2026-03-15T08:12:40Z`). A reader may meet a fraction of a second or a space instead of `T` and should accept it. - **Booleans** are `true`. Fields that are only ever `true` are left out otherwise. - **Order:** writers emit the keys in this order: `type`, `date`, `due_date`, `title`, `title_source`, `placement`, `placement_source`, `filing_rule`, `filing_rule_matched`, `correspondent`, `person`, `tags`, `status`, `cases`, `asn`, `reminder_date`, `reminder_note`, `deleted_at`, `archive_on`, `source_file`, `scanned_at`, `modified_at`, `version`, `edited_by`, `confirmed_by_user`, `file_hash`, `file_hash_basis`, `source_file_hash`, `text_simhash`, `original_filename`, `paperless_id`, `paperless_server`, `source`, `source_sender`, `source_subject`, `source_message_id`, `source_device`, `source_device_name`, `source_folder`, `analysis_state`, `done`, `password_protected`, `doc_id`; then the custom fields `field.`, then `sync_vector`, `field_marks`, then keys unknown to the app, and last `checksum`. Lines of keys that have nothing to say are left out. Readers must not depend on the order. ## 4. Header fields To check a header against the JSON Schema, turn it into a JSON object first: every value is a string, except `tags`, `status` and `cases`, which are arrays of strings. A custom field is a string; a reader that knows the field is a multiple choice may split a value in square brackets as in section 3, and the schema accepts both. "Always" means the app writes the line in every sidecar, possibly with an empty value. "If set" means the line is left out when there is nothing to say. Types: *text* is one line; *day* and *moment* are as above. ### Content, set by the user or by recognition | Key | Type | Written | Meaning | |---|---|---|---| | `type` | text | always | Document type, e.g. `Rechnung`. | | `date` | day | always | Date of the document itself (the date on the letter). | | `due_date` | day | if set | Due date. | | `title` | text | always | Title. | | `title_source` | text | always | Where the title comes from: `generated` (from the naming template), `original` (from the file name), `custom` (typed by the user). Missing or unknown: treat the title as of unknown origin. | | `correspondent` | text | always | Sender or counterpart, e.g. a company. One value; a comma is part of the name. | | `person` | text | always | The person the document concerns. Exactly one value; a comma is part of the name (`Müller, Anna`). | | `tags` | list | always | Keywords. | | `status` | list | always | Status values, e.g. `[Offen]`. History is in the body (section 6). | | `cases` | list | if set | Cases (*Vorgänge*): names that group related documents, e.g. offer, delivery note and invoice of one order. A document may belong to several. | | `asn` | text (digits) | if set | Archive serial number, as the user entered it. | | `reminder_date` | moment | if set | When to remind. | | `reminder_note` | text | if set | Text of the reminder. | | `done` | boolean | only `true` | Ticked off. | | `deleted_at` | moment | if deleted | The document is in the trash since this moment. A line with an empty value means not deleted. | | `archive_on` | day | if set | From this day on the document counts as archived (hidden in the app's normal lists). Nothing is written on that day; whether it is archived is derived from the date. | | `field.` | text or list | if set | Custom fields, see section 5. | ### Where the original is | Key | Type | Written | Meaning | |---|---|---|---| | `source_file` | text | always | Path of the original from the library root (section 2). | | `placement` | text | only `manual` | `manual`: the original stays where it is and is not filed by a rule. | | `placement_source` | text | only `user` | `user`: the user chose the location explicitly. | | `filing_rule` | text | if not the default | Identifier of the filing rule used to place the original. | | `filing_rule_matched` | text | if a rule's conditions chose the rule | Same identifier as `filing_rule` when the rule was assigned by its conditions rather than by hand. | ### What recognition knows | Key | Type | Written | Meaning | |---|---|---|---| | `analysis_state` | text | after an analysis | `done` or `failed`. A missing line means no result is known. The value `pending` may be found in old files and means the same as missing. | | `confirmed_by_user` | boolean | always (`true`/`false`) | A person has checked the values. | ### What the original file is | Key | Type | Written | Meaning | |---|---|---|---| | `file_hash` | text | if known | SHA-256 of the original as it lies now, lower-case hexadecimal. | | `source_file_hash` | text | if known | SHA-256 of the file as imported. Differs from `file_hash` when the app converted the file (an image to a PDF, a PDF with an embedded text layer). | | `file_hash_basis` | text | after new bytes | Two SHA-256 values separated by a space: the new bytes, then the bytes they replaced. | | `text_simhash` | text (digits) | if computed | Fingerprint of the text for finding similar documents. A 64-bit unsigned integer – keep it as text. `0` means none. | | `original_filename` | text | if known | File name at import time. | | `password_protected` | boolean | only `true` | The original is password protected. | ### Where the document came from Written once at import and never changed. | Key | Type | Meaning | |---|---|---| | `source` | text | The way in: `scan`, `photo`, `file`, `folder`, `share`, `shortcut`, `watched_folder`, `email`, `paperless`, `inbox`, `derived` (made from other documents: merged, split, filled in and signed, saved as new). An unknown value is to be treated as missing. | | `source_device`, `source_device_name` | text | Identifier and name (at that moment) of the importing device. | | `source_folder` | text | Full path of the folder the file came from, as the importing device saw it. A path on one device; do not resolve it elsewhere. | | `source_sender`, `source_subject`, `source_message_id` | text | Sender, subject and message id when imported from an email. | | `paperless_id`, `paperless_server` | text | Identifier and server address when imported from paperless-ngx. | ### Administration | Key | Type | Written | Meaning | |---|---|---|---| | `doc_id` | text (UUID) | always | The identifier; equals the file name without `.md`. | | `scanned_at` | moment | always | Since when the document has been in the library (import time). | | `modified_at` | moment | always | Last change **by the user**. Automatic work (recognition, moving a file, renaming a tag for all documents) does not change it. | | `version` | text | always | Revision of this file, a 26-character ULID. See section 7. | | `edited_by` | text | always | Display name of the device that wrote the file last. | | `sync_vector`, `field_marks`, `checksum` | text | see section 7 | Bookkeeping for synchronising several devices. Do not edit. | ## 5. Custom fields Users can define their own fields (for example a contract number). Each value is one header line `field.`, placed after `doc_id` and before `sync_vector`, sorted by `` byte by byte. - `` consists of ASCII letters, digits and `_`, at most 64 characters; case matters. It never changes, even when the user renames the field. The visible name and the type of the field are **not** in the sidecar (they are in the app's settings), so from the sidecar alone every value is text. - Values are written in a fixed form independent of language: text as one line; number as the shortest decimal (`29.9`, `-1200`, `0.05`, no thousands separator, no exponent); date as `YYYY-MM-DD`; yes/no as `true` or `false`; single choice as the chosen text; multiple choice as a list `[A, B]`, sorted by bytes. A value that is empty is not written. - A line is at most 1 KB and all custom fields together at most 32 KB. A `field.` line that breaks these rules is simply an unknown key (section 7). - Treat a value that does not fit its field as text and leave it as it is. ## 6. Body sections The body consists of sections, each starting with a heading line `## Name`. In this order: | Heading | Content | |---|---| | `## Status History` | One line per change, see below. | | `## Note` | The note the user typed: free text, any number of lines. This is the one part of a sidecar that cannot be rebuilt from the original. | | `## OCR Text` | Only in older sidecars: the recognised text from before it moved to `.ocr.json` (section 9). Where both exist, the JSON file wins. | | `## Summary` | Abolished. Ignore it. | - A line of `## Status History` is `- | added | ` or `- | removed | ` (a hyphen, a space, then three parts separated by ` | `). The app writes the newest line first. Two identical lines are two changes. - A section that would be empty is left out. - **A section ends only at one of the four headings above.** Any other `## …` line inside a note is just text of the note (users do write such lines). When a note line itself starts with one of the four headings, the writer indents it with one space. - After `## OCR Text`, every further `## …` line starts a section of its own. Such a section is unknown to you, and must be kept as it is. - Text before the first heading is kept as it is as well. ## 7. Synchronisation lines, versions and unknown content Several devices work on the same library through the shared storage folder. Four header lines are the bookkeeping that lets them merge their work. **A program that only reads can ignore them. They are the app's own bookkeeping; do not change them** (section 11). - `version` – a ULID: 26 characters of the Crockford alphabet (`0-9A-HJKMNP-TV-Z`), the first ten encode the creation time in milliseconds, the rest is random. The app gives every change a new, larger value. **It is not the version of this format** (below). Do not sort revisions by it. - `sync_vector` – `@ . =,…`. The stamp after `@` is the `version` the line was written with; if it does not match the `version` of the file, the line is void (somebody changed the file without the app). `.` (for example `7.2026-09-18T10:22:31.400Z`) is the order of the revision: a counter and the UTC time of the write with milliseconds. After it come pairs of a device identifier and the number of the last change of that device that is contained in this file. - `field_marks` – comma-separated entries `field=device:number@time`, sorted by field name: which device changed which field last, in which of its change numbers, and when. `time` is milliseconds since 1970-01-01 UTC in base 36, lower case, and may be missing. Lists carry one entry per item: `tags[Steuer]=18F00519:7@mu6t7tuw`; `%` and `,` inside the brackets are written `%25` and `%2C`. Custom fields use `field.` for a text value and `field./` for each item of a multiple choice. - `checksum` – the last line of the header. SHA-256, lower-case hexadecimal, of the complete file text with exactly this line removed. It has no key and signs nothing; it only shows whether somebody changed the file without the app. A missing checksum proves nothing. ### There is no format version The sidecar carries **no version number of its format**, and none will be invented. The format grows only by **additions**: new header keys and new body sections. Therefore: - A reader **skips header keys and body sections it does not know**, and does not treat them as an error. - A reader treats an unknown *value* of a known field (for example a new `source`) like a missing value. - A key that PaperPeace has abolished is never given a new meaning. Old files may still contain these; ignore them: `content_hash`, `ai_model`, `note`, `owner_device`, `ai_confidence`, `text_extractor_fingerprint`, `sync_conflicts`. - A key that is not a known one, in any file, written by a newer app or another program, is kept by the app when it writes the file again – behind the known keys, in the order of the lines, up to 8 KB in total and 1 KB per line. ## 8. A complete example This is a sidecar as the app writes it (all names are made up). The `checksum` line is correct for this text. ```markdown --- type: Rechnung date: 2026-03-14 due_date: 2026-04-01 title: Stromrechnung März title_source: generated correspondent: Stadtwerke Musterstadt person: Anna Musterfrau tags: [Energie, Haushalt] status: [Offen] cases: [Wohnung 2026] asn: 1042 source_file: Rechnungen/2026/2026-03-14 Stromrechnung.pdf scanned_at: 2026-03-14T18:01:52Z modified_at: 2026-09-18T10:22:31Z version: 01M2T0M1Z88Q4M7V1C9TXB5HNA edited_by: iPhone von Anna confirmed_by_user: true file_hash: 9f2c61a4e5803b1d7a64c0e2b9d85f3a1c7e40b6d2a98f5e3b1c60d4a7e2f815 source_file_hash: 9f2c61a4e5803b1d7a64c0e2b9d85f3a1c7e40b6d2a98f5e3b1c60d4a7e2f815 text_simhash: 965656484482950027 original_filename: Scan 14.03.2026.pdf source: scan source_device: 7CCA58C8 source_device_name: iPhone von Anna analysis_state: done doc_id: 6c2b0f7a-3e41-4b0e-9c55-2d9a1f0b7e33 field.contractNumber: K-2026-0042 sync_vector: @01M2T0M1Z88Q4M7V1C9TXB5HNA 7.2026-09-18T10:22:31.400Z 18F00519=7,7CCA58C8=41,C463FDD2=13 field_marks: field.contractNumber=C463FDD2:13@mu6t7tuw,person=7CCA58C8:41@mu6t4mo8,tags[Energie]=7CCA58C8:40@mu6t4mo8,tags[Haushalt]=18F00519:7@mu6t4mo8,title=C463FDD2:13@mu6t7tuw checksum: a3ae1998a8635b7d6b0d44f8a19a231d8f0c133f15cdf62f6380ddee2d8f1465 --- ## Status History - 2026-03-15T08:12:40Z | added | Offen ## Note Abschlag ab April angepasst. Widerspruch bis 30.04. einlegen. ``` ## 9. The recognised text The text the app has read out of the original (OCR) is in `_Meta/Sidecars/.ocr.json`, a JSON file next to the sidecar, not in the sidecar. It holds the text blocks with their position on the page and the confidence of the recognition. A top-level `"source": "manual"` marks a text the user typed in by hand; `"source": "firstPage"` marks a text read only from the first page. The inner structure of this file is not specified here; to search the text of a library, read the files as JSON and treat unknown properties as ignorable. Older sidecars may contain the text in `## OCR Text` instead. ## 10. Reading – always safe Reading never harms a library, and no read can change it. A minimal reader: 1. List `_Meta/Sidecars/*.md`. File name without `.md` is the identifier. 2. Read the file as UTF-8. Check that the first line is `---` and find the closing `---`. 3. For every header line split at the first colon, keep the last value of every key, ignore lines without a colon, ignore keys you do not know. 4. Split lists (`tags`, `status`, `cases`) as in section 3. Keep everything else as text. 5. Drop the file if `doc_id` is present and differs from the identifier. 6. Treat the document as deleted when `deleted_at` is not empty. Find the original at `source_file`, or in `_Deleted/.` if deleted. 7. Read the body sections you need (`## Note`, `## Status History`). Be prepared for: missing lines, empty values, unknown keys and sections, values you do not recognise, and a library that another device is writing to while you read it (a file may change between two reads). ## 11. Writing – please do it in the app Reading is safe. Please make changes in the app. A change that another program makes to a sidecar is not taken over by the app today. The app reads the file again, notices that it was not changed by the app, and then keeps its own last state of the entry. The change of the other program is then no longer in the library, and on other devices it is lost as well. Nothing else in the library is damaged by this. The files are the user's, and nothing stops a program from writing to them. Anyone who does so should expect that the change does not last. ## 12. Source of the examples and changes to this document The example has been read by the app and written back unchanged, byte for byte. If the format gains fields, they are added to the table in section 4 and the date of the change is noted below. - 2026-10: first public version.