Web front & read API

One Deployment, htrflow-web, serves three things on one origin:

• the campaign browser, a static single-page app, at /
• Universal Viewer at /uv.html
• the read API at /api/v1/…

The read API computes every answer live from the campaign Jobs, their Pods and ConfigMaps, plus each running volume's progress.json, read through the results proxy. It writes one thing: each campaign's status record (Campaigns → The record). It holds no cluster credentials for the browser. Every /api/v1 route needs a login on the results store, checked with the results proxy; who may reach the site at all is decided by the network (Deploy). The proxy, htrflow-results, is a second Deployment on the same image that serves /results files and the login (Security → The results boundary).

Source: packages/web (the API) and frontend/ (the browser). Their READMEs are the developer documentation.

Routes

Every /api/v1 answer, its 401 and 502 included, carries Cache-Control: private, no-store: each is one logged-in person's view.

/log and /alto open only URLs under the results base, so a mailed link cannot point them at another host, nor outside the base on the same one: one with an escaped / or \ in its path below the base is refused too, since a server that decodes before it resolves .. would read it as a way out. How to reach these from a browser is in View Results.

The campaign list: GET /api/v1/jobs

Every live campaign Job in the namespace, plus the reaped newest campaigns whose Job is gone (default 20, at most 10 000). The X-Reaped-Total header says how many reaped campaigns there are in all. Each row is a JobSummary:

One campaign: GET /api/v1/jobs/{namespace}/{name}

A JobDetail is the summary plus:

Everything but volumes is computed over every volume, not only the requested page. A campaign of thousands shows its first 200 rows, and the volume in flight is rarely among them.

Each volume row (VolumeView):

Where a reason comes from. The wrapper's own JSON termination message when it wrote one. A pod stopped by the warm-up gate carries the gate's message. A pod killed at its deadline reads "error": "DeadlineExceeded". A pod that left no message reads its own reason (Evicted), else the container's (OOMKilled (exit code 137)).

progress is {done, total, failed, lastPage, stage, updatedAt, ageSeconds, lastError, errors, viewerPublished, quality}, read from the volume's progress.json (or, for an older run, its manifest.json counts). The fields are described in S3 Layout. ageSeconds is computed by the API from its own clock, so a reader's clock skew never shows "0 s ago".

progress.quality is null when the volume has not scored a page, else {mean, min, scored, model, revision, lowest} — the wrapper's own block (docs: S3 Layout), sanitised: a value outside what the wrapper could have written (a mean out of [0, 1], a negative scored, a string where a number belongs) drops the whole block rather than show a wrong one, and lowest is capped at 5 entries, each {page, quality, canvas} (canvas is null when the wrapper's was not a valid index).

The detail's own quality (next to pagesDone) is null when no read volume scored a page, else {mean, min, scored, volumes, lowest}: mean is every scored volume's mean weighted by its scored pages, min is the lowest of every volume's min, scored is the pages summed, and volumes is how many volumes contributed — the card says so when that is fewer than the campaign's total. lowest is the worst 5 pages across every volume that scored one, each {volume, page, quality, canvas, iiifUrl} — iiifUrl is that volume's own, so the page links straight to the viewer without looking the volume up.

• A pending volume is never read. A bucket that does not answer gives null, never a 500.
• Answers are cached: 5 s for a running volume, an hour for one that is over (an absent file included).
• One request makes at most 100 bucket reads and spends at most 5 s on them, active volumes first. The rest come from later polls.
• The API reads progress through the results proxy at HTRFLOW_INTERNAL_RESULTS_BASE, forwarding the caller's session cookie; the cache is per user, so one person never sees what another's account read. A volume the account may not read shows no progress.

Phases and volume states

A campaign's phase comes from its Job:

A campaign that stays Queued with free capacity is covered in Queueing.

A volume's state is pending, active, done or failed, computed from the Job's index sets and pods. unknown appears only on a jobGone campaign, for volumes whose ending the record does not say.

Reaped campaigns

A campaign whose Job is past ttlSecondsAfterFinished keeps its row, with jobGone: true, the phase and counts its record kept, and finishedAt. Its detail rebuilds the volume rows from volumes.txt: a volume named in the record's failedVolumes is failed with that sentence, and every other volume takes the campaign's ending. When the record counts more failures than it names, those others read unknown rather than done. Their links still point under the results URL, and their progress is still read through the results proxy.

What a campaign card shows

Every card has the same four zones, in the same order, so ten campaigns scan like a table. A folded card shows only the first; the rest appear when it is opened:

1. Identity and state. The campaign's name (click to fold or unfold; namespace/name only when the list spans more than one namespace), the warm-up chip while the warm-up has not succeeded, a "job removed" chip for a reaped campaign, its phase, when it was created and finished, and on a folded card the pages done so far (411 / 1914 pages · 3 failed, each figure marked ≥ while the sums do not yet cover every volume). Every part of the line has its place from the first paint, so nothing on it moves as the rest arrives.
2. The body. The totals (volumes, pages) with progress bars, then the loaded page of the campaign's volumes, with "load more" for the rest.
3. Problems, only when there is one: why the warm-up failed, each failed volume as id: sentence linked to its run log, and the latest page error when its volume is not on screen. Past three sentences the rest wait behind "N more".
4. Provenance. The pipeline (click for its YAML) and, on the row below, the models it loads, each linked to its Hugging Face revision, or marked unpinned.

Each volume row links:

• the id to Universal Viewer: the published iiif.json once the volume is done or has published an interim one, its source manifest before that, and plain text when there is neither
• a manifest icon to the source manifest (empty for an images: volume)
• a log icon, last on the row after its state, to the run viewer, live while the volume is not done

The warm-up chip reads "warm-up pending", "running", "failed" or "no warm-up" (missing). A failed warm-up also colours the card as failed, since no pod can start without it; its reason is in the chip's title. missing colours the card only while the campaign has not succeeded.

Paging and polling. The list re-fetches every 60 s. An open card fetches its own detail, re-fetching every open page on each poll. A folded card reads the same detail for its page count, and only while it is on screen: every two minutes for a campaign still going. A finished, unknown or reaped campaign is read until its page sums cover every volume, and not polled again. At most four cards read at a time; the rest wait their turn, behind a card the reader has just opened. A failed poll puts a banner at the foot of the window, over the last list, saying when the next try is, and backs off. Older reaped campaigns wait behind "show older campaigns", 20 at a time.

Order. The list opens sorted by what wants a person: running, then in trouble, then finished (newest first), then not started. A poll keeps that order, so no card moves under a reader, with two exceptions that are news: a campaign that falls into trouble, and a new one, take their place in the order at once, gliding there. Any other change leaves the card where it is, and the foot of the window offers to re-sort. The list is also sorted afresh when the reader comes back to the tab.

The page and component internals are in the frontend README.

Configuration

The browser has no environment of its own. It reads window.API_BASE and window.RESULTS_BASE from /config.js, which the API writes from its own environment on each request: /api/v1, and HTRFLOW_RESULTS_URL. Set resultsUrl on the chart and the page follows; there is no second copy to keep in step.

The generated Configuration page lists every setting. The chart values are in Chart Values.

Content-Security-Policy

The meta CSP is why configuration arrives as /config.js and never as an inline script. In site-only mode, or with a results base a CSP source cannot express, the SPA gets no connect-src header.