ascribe diff and drift

Compare a project’s output with another revision, and find sources that drifted.

The commands that compare a project with another revision of itself. The options every command accepts are in the command reference.

ascribe diff

Shows what changed between a git revision and the working tree, as readers will see it: which pages of each build changed, and which blocks on them were added, removed, changed, or moved. It compares resolved pages, not files, so a page whose own file didn’t change but whose included fragment, phrase, or build settings did is listed, with what its change comes from. A change that doesn’t reach the page (ascribe fmt, rewrapped lines) isn’t.

• [PAGE]: Show only this page’s changes, block by block, or, for a fragment, the pages that changed through it: a path from the current directory, or from the content root. With --format prompt, the prompt is about this page alone, in the first build it changed in, or about the pages that changed through a fragment.
• --base <REV>: The revision to compare with, anything git accepts (a branch, a tag, a commit). By default, the repository’s default branch, the first of origin/HEAD, origin/main, origin/master, main, and master that exists. The comparison starts from the merge base of that revision and HEAD, as a pull request shows its changes, so commits made on the base branch since you branched aren’t listed. A shallow clone (what actions/checkout makes by default) may not have the merge base: fetch more history (fetch-depth: 0) or use --base-exact.
• --base-exact: Compare with the revision itself instead of the merge base.
• --build <NAME>: Compare only this build. Repeat it for several. By default, every build in ascribe.toml.
• --format <FORMAT>: How to show the changes.
  • text (the default): The changed pages of each build, with counts, for people; with a PAGE, its changes too, each block as it was and as it is.
  • json: One JSON document, for tools.
  • html: One self-contained HTML file that shows every changed page rendered, with its changes marked, for reviewers.
  • prompt: A prompt for an agent that reviews the changes as readers will see them: about every changed page, or about the PAGE named. Nothing when nothing changed.
• --exit-code: Exit with 1 when anything changed, as git diff --exit-code does.
• --pages-only: List the changed pages without their block-level changes. With --format json, each page’s changes is empty and its counts still count them, so the report stays short on a large change. With a PAGE, text leaves out the page’s blocks; without one, it lists only pages anyway. HTML needs the blocks.

The argument is a page, not a revision as in git diff main: the revision goes in --base. When the argument isn’t a file of the project but is a revision git knows, the error says which --base compares with it: ascribe diff main names --base main. ascribe drift, which takes no argument, says the same of a revision given as one.

The other side is the working tree: the files on disk, committed or not, as ascribe build would read them now. Unsaved editor changes aren’t included. The base is read from git, ascribe.toml included, so a change to the content model is compared too; if the project didn’t exist at the base, every page is added. ascribe diff needs git on the path, and nothing else: no network and no GitHub account. It runs git so that the repository can’t make it run a command of its own: with no file system monitor, and taking a bare repository only when one is named, never a folder in the project shaped like one, where git says it can’t use the repository.

The text output lists each build’s changed pages with counts, and where a change comes from when it isn’t only the page’s own file. A page that changed the same way in every build that publishes it is said once, first, “in every build” (or “in every build that publishes it”, when some don’t), and each build lists its other pages:

compared with main (375192d), from its merge base with HEAD (e49444a)
getting-started.md: 1 changed, 1 added (through _fragments/prereqs.md), in every build
site: 3 more pages changed
  guides/rollouts.md: 4 changed, 5 added, 2 removed, 1 moved
  guides/schedules.md: added
  reference/limits.md: title changed

A page that moved is listed once, where it is now, when git sees its file renamed (as git mv makes it, or a move git add has staged) or when ascribe.addresses records the move. It says what it moved from, and what changed on it besides the move, or that nothing did. A page whose link changed only because the page it leads to moved says so; when its own file changed too, the move isn’t named again, since the moved page is listed itself:

site: 2 pages changed
  guides/install.md: 1 changed (a link changed because guides/configure.md moved)
  guides/setup.md: moved from guides/configure.md, no change to the page

The lines of a versioned build are listed together, under the build’s name, and a page they share once, with the lines it changed in:

site: 2 pages changed
  install.md: 1 changed, in every line
  upgrade.md: 1 changed, only in 3.4

Name a page to see its changes: every changed block, as it was (-) and as it is (+), with where it’s written. In changed prose, the words removed are marked [-like this-] and the words added {+like this+}, as git diff --word-diff marks them; a block that isn’t prose, such as a code block, shows its lines on both sides, and a moved block shows its lines once. A build the page didn’t change in says so. Name a fragment, or another file pages take text from, to see the pages that changed through it. --format json and --format html are narrowed the same way, with the same shape and fewer pages, and --exit-code exits with 1 only when what’s shown changed. A PAGE that isn’t a file of the project, at the base or now, is a usage error.

compared with main (375192d), from its merge base with HEAD (e49444a)
site: guides/rollouts.md: 4 changed, 5 added, 2 removed, 1 moved
  changed guides/rollouts.md:6
    - Roll the agent out to a few servers first, then to the [-rest-].
    + Roll the agent out to a few servers first, then to {+all+} the {+others+}.
  changed guides/rollouts.md:10
    - Pick servers that see little traffic.
    + Pick servers that see little traffic {+and run the newest release+}.
  changed guides/rollouts.md:14
    - Install
    + Install {+the agent+}
  changed guides/rollouts.md:16
    - Install the agent on the tagged servers.
    + Install the agent on the tagged servers{+, one at a time+}.
  removed guides/rollouts.md:18 in the base
    - Watch their logs for an hour.
  removed guides/rollouts.md:22 in the base
    - Remove the agent from the tagged servers.
  added guides/rollouts.md:20
    + Watch their logs for a day.
  added guides/rollouts.md:22
    + Check the error rate in the dashboard.
  moved guides/rollouts.md:24, from guides/rollouts.md:20 in the base
      ## Roll back
  added guides/rollouts.md:26
    + Remove the agent from the tagged servers, and its configuration.
  added guides/rollouts.md:28
    + Send us the logs.
  added guides/rollouts.md:34
    + Untag the servers.
cloud: guides/rollouts.md: no changes

Pages with errors are compared as they are. When ascribe check would find errors for the builds compared, ascribe diff says so on standard error, as review does in every place (When the project has errors says why):

warning: the working tree has 3 errors; `ascribe check` lists them

Exit codes

The HTML preview

--format html writes one HTML file to standard output, with everything in it: its styles and script, the element library, and every image a changed page uses, up to 1 MB each (a larger one shows as a placeholder naming its file). It makes no network requests, so it opens from a CI artifact, an email, or a disk with nothing else:

ascribe diff --format html > review.html

It lists the changed pages of each build, and marks a page that changed only through something it uses (“via _fragments/prereqs.md”), and a page that moved with where it moved from, its “As it was” rendered from there; with more than one build changed, a picker switches between them. Each page is rendered as it is now, the way the editor’s page preview renders it, without the site’s layout, with review’s marks:

Above each page are its file and route, its count of changes, Show, which switches between Changes, As it will be, and As it was, and arrows to the previous and next change.

Hovering over a block shows the source file and line it came from (guides/install.md:12).

The lines of a versioned build are one build in the list: a changed page is listed once, with “in every line” or the lines it changed in, and a Line switch shows it as each line publishes it.

Copy prompt copies the page’s agent prompt, the one ascribe diff --format prompt <page> writes, to paste into your agent. The preview has no link that opens an agent: a file opened from a disk or a CI artifact can’t tell which editor you use, so it only copies.

The preview renders at most 300 changed pages; the rest are listed by name, and the preview says so at the top. Each rendered page and image is stored once, however many builds or pages share it. Attach the HTML preview to every pull request has a GitHub Actions job that uploads it.

The agent prompt

--format prompt writes a prompt that asks your agent to review the change as readers will see it, in the agent prompt format, and writes nothing when nothing changed. Without a PAGE, it’s about every changed page, grouped by build when more than one changed. With a PAGE, it’s about that page in the first build it changed in, listing each change with its source lines; for a fragment, or any other file pages changed through, it’s about the pages that show it, and asks the agent to check that the new text fits each. A PAGE that isn’t a file of the project, at the base or now, is a usage error.

Review what this change does to `docs/getting-started.md`, as a reader of build `site` sees it.

Where: docs/getting-started.md
Paths and commands here are from `docs` in the repository.
Build: `site`

Base: `main`, from where this branch left it (e49444a)
What changed: 1 changed
Changed through: `docs/_fragments/prereqs.md`
To read it: `ascribe resolve docs/getting-started.md --build site`
Changes:
- changed: docs/_fragments/prereqs.md:3, shown through docs/getting-started.md:11

Report what reads wrongly; don't edit.

The prompt asks for a report, not edits: a review reads the change, and its author decides what to do. Its paths and commands (ascribe resolve, git diff, ascribe diff) are written from the folder ascribe diff was run in, which it names once, with the base, the build, and --config as this run had them. The HTML preview’s Copy prompt gives the same prompt, and the editor’s Prompt agent gives it from the repository’s root.

Diff JSON

--format json writes one JSON document to standard output. It follows the JSON output’s rules: schema_version changes only when a field is removed or changes meaning, so ignore fields you don’t know.

Each line of a versioned build is an entry of its own in builds, such as site@3.4, its pages’ routes under the line’s folder.

Each page:

Each change:

A block is a heading, paragraph, code block, table, list, list item, block quote, directive, container, group, or a group’s arm; changes inside a list, a container, or an arm are listed on the blocks inside it, so a one-word change in a step marks that step’s paragraph. A line-form directive that renders as an element around the block after it (@note, @steps, @details, a widget) is one block with that block inside it, from the directive’s line through the block’s last, like the element.

A block’s source is <path>:<first>-<last>: the file its text is written in, relative to the content root with each path segment percent-encoded (except ASCII letters, digits, -, ., _, and ~), and its first and last lines, from 1. via lists the includes it came through, outermost first, each <path>:<line>; it’s empty for a block written in the page itself. These are the same strings the site output’s source anchors carry, so a tool can find a block in a rendered page.

{
  "schema_version": 1,
  "ascribe_version": "0.4.4",
  "base": {
    "requested": "main",
    "commit": "375192d7ee62fbe72258e0adab3747503f45d106",
    "merge_base": "e49444a7968397b56129c1eb801ca02b0e96ed61"
  },
  "repository": {
    "root": "/home/me/lantern",
    "project_prefix": "docs/"
  },
  "working_tree_errors": 0,
  "builds": [
    {
      "build": "site",
      "pages": [
        {
          "path": "getting-started.md",
          "route": "/getting-started/",
          "status": "changed",
          "own_file_changed": false,
          "because": [
            "_fragments/prereqs.md"
          ],
          "page_changed": [],
          "counts": {
            "changed": 1,
            "added": 0,
            "removed": 0,
            "moved": 0
          },
          "changes": [
            {
              "kind": "changed",
              "now": {
                "source": "_fragments/prereqs.md:3-3",
                "via": [
                  "getting-started.md:11"
                ]
              },
              "was": {
                "source": "_fragments/prereqs.md:3-3",
                "via": [
                  "getting-started.md:11"
                ]
              },
              "words": {
                "now": [
                  [
                    14,
                    17
                  ]
                ],
                "was": [
                  [
                    14,
                    17
                  ]
                ],
                "now_text": "Lantern agent 2.4 or later",
                "was_text": "Lantern agent 2.2 or later"
              }
            }
          ]
        }
      ]
    }
  ]
}

How blocks are matched: blocks are compared by their content without positions, so moving a block down the file or rewrapping it is no change. Blocks that didn’t stay are paired, in order, with a block of the same kind whose words overlap by at least half, or that keeps its words whole with words added or taken away at one end, as changed; a block that matches nothing is added or removed, and a removed block identical to an added one elsewhere on the page is moved. A block over about 2,500 words gets no word ranges, and a list or container with over 1,000 blocks in it that changed is marked changed as a whole.

ascribe drift

Lists the pages whose code examples changed between a git revision and the working tree, and whether the words around them changed too, and the pages whose examples no longer resolve. A page covers the code it shows, and nothing else: every region it takes a snippet from, and every whole file it takes as one, including its fragments’ snippets. It’s for CI, where it says which pages a reviewer should reread; see Report the pages whose examples changed.

• --base <REV>: The revision to compare with, anything git accepts (a branch, a tag, a commit). By default, the repository’s default branch, found as ascribe diff finds it. The comparison starts from the merge base of that revision and HEAD, as a pull request shows its changes. A shallow clone (what actions/checkout makes by default) may not have the merge base: fetch more history (fetch-depth: 0).
• --build <NAME>: Look only at this build. Repeat it for several. By default, every build in ascribe.toml.
• --format <FORMAT>: How to show the report.
  • text (the default): The groups of pages, with each example that broke or changed, for people.
  • json: One JSON document, for tools.
  • markdown: Markdown for a CI job’s summary, each page linked to the site; nothing when no example changed.
• --exit-code: Exit with 1 when a page’s example no longer resolves, or changed while the page didn’t.

The base is found as ascribe diff finds it, and the other side is the working tree. A snippet’s region is taken from its file on both sides and compared as text, dedented, with trailing spaces trimmed, so an edit elsewhere in the file, a region that only moved, or a change of indent isn’t a change. A renamed file is followed. A region the file didn’t have at the base is a new example, not a changed one. Then, for each page with a changed example, it compares the page itself as ascribe diff does, apart from its snippets: the page changed when its own file did, or when anything else it uses did, such as a fragment it includes or a phrase in ascribe.toml.

The pages come in up to three groups:

compared with main (94c2212), from its merge base with HEAD (8a43d4b)

Examples that no longer resolve. `ascribe check` reports them too:
  guides/install.md
    code:service/client.py#connect: the file has no region `connect`

Examples that changed. The page shows the new code; check the words around it:
  reference/content-model.md
    code:examples/quill/ascribe.toml#dimensions (+3 −1)

Examples that changed along with the page:
  guides/review.md
    code:examples/quill/ascribe.toml#builds (+1 −0)

An example no longer resolves when it did at the base and doesn’t now: its region was renamed, its file moved, or its source is gone. ascribe check fails on it too, but the report names the page so the pull request that broke it says so. A @snippet the change adds broken isn’t listed; that’s check’s alone. Of the changed examples, the first group is the one to read: the example updated itself, and nobody changed the sentence that explains it. The second is listed so a reviewer can see the change reached the page. Each example says how many lines were added and removed. With nothing to report, it says No examples changed.

A changed example isn’t always a changed behavior. The report says what changed and how much, and leaves the judgment to you.

It reads little: one git diff listing of the change, and at the base only the files that listing names that a snippet uses. The base’s pages are read only when an example changed. ascribe drift needs git on the path, and nothing else: no network and no GitHub account.

The summary

--format markdown writes Markdown for a CI job’s summary: the same groups, each page linked to its route on [consumer] site (plain text when there’s no site), and nothing at all when no example changed, so it can be appended to $GITHUB_STEP_SUMMARY on every run.

Exit codes

Drift JSON

--format json writes one JSON document to standard output, with the JSON output’s rules: schema_version changes only when a field is removed or changes meaning, so ignore fields you don’t know.

Each page:

Each example: