ascribe.toml format contract: where pages are published

The normative contract for the content model’s consumer, builds, listing, crawlers, and permalinks.

Part of the ascribe.toml format contract, whose contents list every section. What these keys are for, with examples, is in the reference’s where pages are published.

15. [consumer]

The consumer profile (SPEC §9.5): how the site output fits a specific consumer. Spec 0.1’s toolchain implements one profile, astro (SPEC §9.6). The profile sets defaults for every other key; a key given here overrides them, within the values the profile supports.

[consumer]
profile = "astro"
site = "https://docs.quill.dev"
base-path = "/"
trailing-slash = "always"
slugger = "github"

Rules (§25.7): each key MUST have a value the profile supports, and under astro, html MUST be true (model-consumer-unsupported). site MUST be an absolute http or https URL with no path, query, or fragment (model-consumer-site). base-path MUST start with / (model-consumer-base-path), and so MUST permalink-path (model-consumer-permalink-path), whose missing final / is added; the site output’s _ascribe/permalinks.json gives it as path (output-layout contract §1.2). agents left out MUST be read as true when site is set and false when it isn’t; an explicit agents = true MUST come with site (model-consumer-agents-site). host MUST be "netlify" (model-invalid-value); with it, the compiler MUST write the host’s files in each build’s _ascribe/host/ (output-layout contract §1.9), and without it MUST NOT. The compiler MUST write each build’s _ascribe/headers.json (§1.4) and _ascribe/redirects.json (§1.8) whatever it says. headers isn’t a key: it’s model-unknown-key, whose message names host.

Heading ids, image attributes, and assets (SPEC §9.5) have no keys: the profile fixes them. Under astro, heading ids and image attributes MUST be written as the <ascribe-attributes> markers of the site-render contract, and assets placed as the asset contract says.

Routes (the astro profile, verified against Astro). A page’s route MUST be, in order: base-path; the page’s path from the content root without .md, each segment slugged as Astro’s content loader slugs an entry id, and without a final index segment (guides/index.md is /guides/); and a final / when trailing-slash is "always". The route of the root index.md, entry id index, is base-path, without its final / under "never" unless it’s /. When two pages have one entry id (My File.md and my-file.md, or index.md and index/index.md), ascribe build --emit site MUST fail, naming both. link-route and its fix MUST map a route back to its page with the same router. Source files contain no routes (SPEC §5.2).

The outputs for agents (agents, true by default with a site). The plain-markdown output becomes what a site publishes for agents, laid out so that copying it to base-path publishes it (contracts/output-layout.md):

• Each page’s file is its route’s path with .md: Astro’s entry id with .md added (Guides/My Setup.md → guides/my-setup.md, guides/index.md → guides.md, the root index.md → index.md). It opens with the blockquote > For the complete documentation index, see [llms.txt](../<url>). and a blank line, <url> being llms.txt’s absolute URL.
• Files the pages link to are copied under _ascribe/files/, mirroring their source paths, and linked by absolute URL; the manifest gives each its root-relative url.
• llms.txt is the index of every page the build publishes and offers, in the sections of the build’s navigation (the navigation contract): # <name>, a blank line, > <summary>, then a section each, ## <heading> followed by a line per page, - [<title>](../<url>) or - [<title>](../<url>): <description>, with <url> the absolute URL of the page’s Markdown. <name> and <summary> are the title and description of the page whose entry id is index; without one, the site’s host and a fixed sentence. <title> is the page’s title, never its nav-label. A page’s description is the field its type marks role = "description" (§5.4), as plain text on one line (its Markdown’s text: a link is its text). The first section is the content root’s own page and its pages, in order, headed by the root’s label, “Start” unless ascribe.nav gives another; a page beside a top folder with its name isn’t among them, but first in that folder’s section. Then each top section, in order, with its own page first and its nested sections’ pages folded in, in reading order, headed by its label from ascribe.nav, else its own page’s title, else the folder’s name with - and _ as spaces and its first letter capitalized. A section with no page to list is left out.
• When llms.txt would be longer than 50,000 characters, it’s split: the root file keeps the header and the root’s section, and adds ## Sections with a line per top section linking to <segment>/llms.txt, <segment> being the first segment of its pages’ entry ids, described by its own page’s description or “N pages”. Each <segment>/llms.txt is a whole index of its own: the section’s heading as its name, its own page’s description (or a sentence naming the site and the section) as its summary, and the section. A file still over the limit is written, and ascribe check reports it (llms-section-large).

The astro profile’s site, base-path, and trailing-slash repeat settings from astro.config. The Astro integration checks that they agree, and fails the build, naming each difference, when they don’t. It compares base-path as a path with a leading and a trailing /, treats Astro’s trailingSlash: "ignore" as agreeing with either value, and compares site by origin when both sides set it.

16. [builds.<name>]

Named builds (SPEC §9.3). Each sets a variant mode and an availability mode. These are SPEC §9.3’s examples, in ascribe.toml:

[builds.site]
variants = "switch"
availability = "badge"

[builds.cloud-pdf]
variants = { deployment = "cloud" }
availability = { filter = "cloud" }

[builds."sm-3.3"]
variants = "switch"
availability = { filter = "self-managed 3.3" }

<name> is the build name (§1.2). It names the build on the command line (ascribe build --build cloud) and in output paths.

Rules (§25.7): build names MUST differ ignoring case (model-build-name-case). A versioned build’s target MUST have a live line in [releases], and the build MUST NOT also filter (model-build-versions). A filter’s version MUST be given exactly when its target is versioned (model-build-filter-version). A build that filters for a target its own selection excludes is a warning (model-build-filter-excluded).

17. [listing]

How pages are listed, set once for many: the whole site, the pages under a folder, or the pages of a content type. Each value is one of the values of a page’s listing (SPEC §2.1), from open to closed: listed, unquoted, unlisted.

[listing]
default = "listed"

[listing.folders]
"internal/" = "unlisted"

[listing.types]
changelog = "unquoted"

A page’s listing is the strictest of its own and the rules it’s under: the site’s default, every folder it’s under, and its type’s. A folder and a type have no order between them: a page under both gets the stricter. A rule can only restrict. A page can set the same value as its rule, or a stricter one, and a page that sets a looser one is an error at the page (listing-loosened), since the project contradicts itself. A folder under another folder, and a folder or a type under the default, can’t be looser than it (model-listing-looser).

older-lines (a string, "listed" by default) is the listing of every page of a live line of a versioned build other than its current line: site@3.4’s, and not site’s. It governs search engines and agents together. A page in an older line is under it as well as the rules above, between default and a folder’s or a type’s; it can’t be looser than default (model-listing-looser); and a page that sets a value looser than it is an error in each older line that publishes it (listing-loosened-in-line), and fine in the current line. An older line’s pages point their canonical links at the current line’s, so a search engine sends readers to the current line however it’s set: older-lines = "unlisted" keeps an older line out of the index altogether, its pages out of the sitemap and llms.txt, for a project whose old pages still outrank its new ones.

A default stricter than listed is for a site that isn’t public yet: no page in it can be listed until the default changes, and a page that says otherwise is the same error.

Each page’s record, in _ascribe/pages.json, says where its listing came from: the page, the rule’s level and its line in ascribe.toml, or neither (output-layout contract §1.3).

Rules (§25.12): each value MUST be one of listed, unquoted, and unlisted (model-invalid-value). A key of [listing.folders] MUST be a path from the content root ending in /, without . or .. parts or a pattern’s characters, of a folder that exists by its exact name (model-listing-folder, a filesystem rule), and a key of [listing.types] a content type the model declares, or page when it declares none (model-listing-type). A rule MUST NOT be looser than a folder’s it’s under, or than default (model-listing-looser), and the table MUST have no other key (model-unknown-key). A page looser than a rule it’s under is checked with the project, not by the loader: SPEC §8.2’s listing-loosened.

older-lines is held to default by model-listing-looser, as a folder’s and a type’s rules are. A page looser than older-lines, which only an older line’s pages are under, is SPEC §8.2’s listing-loosened-in-line.

18. [crawlers]

What the project says to the crawlers that read its site: search engines, AI assistants, and anything else that fetches pages without a person. Each build writes it into its page records, _ascribe/pages.json in the site output (output-layout contract §1.3), for a site to write its robots.txt from; @ascribed/astro writes it (robots.txt).

[crawlers]
training = "disallow"
search = "allow"
on-request = "allow"
lines = [
  "Disallow: /drafts/",
  "User-agent: ExampleBot",
  "Disallow: /",
]

allow writes nothing, and neither does leaving a purpose out: a group that named those crawlers and said Allow: / would take them out of the * group, and with it out of every path lines disallows, drafts and all. So an allowed crawler stays under the lines for every crawler. allow is the project writing its choice down. A project that says nothing gets no rules about AI crawlers.

robots.txt has the purposes’ groups first, in the order above, then the User-agent: * group with the lines for every crawler, then the groups lines writes whole, then the sitemap’s address.

Rules (§25.11): each of lines MUST be a string (model-wrong-type) of one line (model-crawlers-line), not empty (model-empty-text). training, search, and on-request MUST each be "allow" or "disallow" (model-invalid-value). A crawler a User-agent line of lines names, compared without case, MUST NOT be one a disallowed purpose names, nor one an earlier group of lines names (model-crawlers-named-twice), where consecutive User-agent lines open one group and * is never counted. The table MUST have no other key (model-unknown-key). A line is otherwise not validated: one that isn’t a User-agent, Allow, or Disallow line MUST be written as it is, and ascribe check reads only the lines it can.

The crawlers of each purpose are data Ascribe ships, each with its operator, the operator’s page about it, and the day it was read; a release MAY add to them. A disallowed purpose’s group MUST name every one Ascribe knows of it.

19. [permalinks]

None. A permalink’s record, retired or moved, is an entry of ascribe.addresses keyed go:<name> (SPEC §7.10), and [permalinks] is a replaced key: a loader MUST report it as an error at the key (model-unknown-key), whose fix says that ascribe addresses init moves its entries.