Data flows
Every hop content takes between Jamie's systems, who owns each field, and what watches each hop today. Rendered from shared/flows.json, updated 2026-10-26. A hop is a row in that file before it is code anywhere.
Open
- gap 6. Weekly Thing (web) → Linking Things: Broken on 2026-10-26: WT354 was sent 2026-10-24, the nightly ran green, and Linking Things still reports 353. The sibling checkout was on a feature branch without 354.md. Fix: read weekly's published archive over HTTPS, or origin main, never a working checkout; and nudge the join from the send (hop 10).
- gap 19. Library Agent → micro.blog: First write round, 2026-10-22: 20 edits accepted by Micropub, 4 missing from the rendered pages after the rebuild. The read-back checks Micropub, not the page, so the round was green. A write needs a receipt: written, rendered, pulled back (hop 11).
- unverified 21. WT Builder → Another Thing: Never run. Episodes so far were committed by hand, their MP3s put with npm run audio:put. Needs ANOTHER_GITHUB_PAT_TOKEN. The send behaves as audio:put: the same bytes are left alone, different audio at the key is refused unless an earlier send of the episode put it there, nothing deletes from the bucket. The deploy-aws job refuses a feed whose MP3 is not in the bucket at the feed's length.
Map
- live read or write
- nudge (an event starts it)
- clock (a schedule starts it)
- runs on the Mac
- gap
- unverified
A hop that spans two or more columns runs along a lane above the map. A write-back leaves from the left of its source. The number on each hop is its row below.
Hops
| # | From → to | What moves, and how | Trigger | Expected lag | How to verify | Status |
|---|---|---|---|---|---|---|
| 1 | Pinboard → WT Builder link, note |
Every bookmark saved inside the issue's window. Pinboard API read on open and on sweep. |
live mac |
none | The link is an item in the issue (get_item). | ok |
| 2 | WT Builder → Pinboard note |
Jamie's commentary edits, mirrored back as his note. Compare-and-set write-back; a bookmark changed at Pinboard since the last scan is marked conflict, not written. |
live mac |
none | The bookmark's note at Pinboard matches the item. | ok |
| 3 | micro.blog → WT Builder post |
Posts in the window, for the Journal; later edits to posts not edited in WT Builder. Micropub q=source read on open; reconcile adopts micro.blog edits, conflict only when both sides edited. |
live mac |
none | The post is a Journal item with the current text. | ok |
| 4 | WT Builder → micro.blog post, category |
Edits to a Journal item, and the Weekly Thing category on reprinted posts. Micropub update, compare-and-set. |
live mac |
none | q=source for the post shows the edit. | ok |
| 5 | Pinboard → Linking Things link, note |
Every public bookmark, as the year shards under data/links/. tools/nightly.sh: sync.py (posts/update, then posts/all if the stamp moved), join, enrichment, validate, commit, push; the push deploys. |
clock, 00:30 daily mac · launchd com.thingelstad.links-nightly |
28 h | links_stats last_saved is after the bookmark's save time; data/sync.json updated matches Pinboard's posts/update. | ok The run took about three hours on 2026-10-26 (Wayback lookups). A link saved at 01:00 waits until the next night's run, then the run time: about 28 hours at worst. |
| 6 | Weekly Thing (web) → Linking Things issue |
Which issue carried each link (the /issues/<n>/ pages and wt: search). join_issues.py reads the issue pages' front matter from the sibling checkout ../thingelstad.com/weekly.thingelstad.com/apps/site/archive on the Mac, inside the nightly run. |
clock, 00:30 daily, inside hop 5 mac |
28 h | links_stats latest_issue equals the newest issue on weekly.thingelstad.com. | gap Broken on 2026-10-26: WT354 was sent 2026-10-24, the nightly ran green, and Linking Things still reports 353. The sibling checkout was on a feature branch without 354.md. Fix: read weekly's published archive over HTTPS, or origin main, never a working checkout; and nudge the join from the send (hop 10). |
| 7 | WT Builder → Weekly Thing (web) issue |
The issue's generated 11ty inputs (apps/site/archive/<n>.md and its assets). The website leg commits to main; the deploy workflow builds and publishes on push. |
nudge, on send mac |
15 min | feed.xml carries the issue; /archive/<n>/ renders. | ok Sent for every issue since WT350. |
| 8 | WT Builder → Buttondown issue |
The email edition, as a draft. Buttondown API: creates and updates a draft. Never schedules, never sends; Jamie sends. |
nudge, on send mac |
15 min | The draft exists with the issue's subject; the email goes out when Jamie sends it. | ok |
| 9 | WT Builder → files.thingelstad.com audio, image |
The issue's audio edition (with chapters and VTT) and every rehosted image. Upload to files.thingelstad.com before the website and email legs; content-named files, so a re-render re-sends the website leg. |
nudge, on send mac |
15 min | The URL the issue names returns the object. | ok |
| 10 | WT Builder → Librarian issue |
The news that an issue is out (its number), not the text. repository_dispatch issue-sent to librarian-thing after the website leg; the library round pulls the issue page from weekly.thingelstad.com and ends the weekend hold. |
nudge, on send mac |
30 min | The round's 'The sent issue arrived' step passed; latest_content lists the issue. | ok Sent for every issue since WT350. Linking Things gets no such nudge (hop 6). |
| 11 | micro.blog → Librarian post, category, alt |
Every post and page, with categories and alt as micro.blog holds them. library.py round --only blog, on the Mac (Library Agent, 05:00) and on GitHub (07:17) as the backstop. The round declares a blog-published dispatch, but nothing sends it. |
clock, 05:00 and 07:17 daily mac+github-actions |
26 h | latest_content source_kind blog lists the post; the library's blog fingerprint moved. | ok Two runners for one round means two answers to 'when did it last run'. blog-published, podcast-published and things-published are declared in library-round.yml and sent by no one (checked 2026-10-26). |
| 12 | Another Thing → Librarian episode |
Each episode and its transcript, into the podcast corpus. library.py round --only another, from the published site. podcast-published is declared but never sent. |
clock, 05:00 and 07:17 daily mac+github-actions |
26 h | latest_content source_kind podcast lists the episode. | ok |
| 13 | Things I've built → Librarian thing |
The registry of things, into the reference base. library.py round --only things, from the published site. things-published is declared but never sent. |
clock, 05:00 and 07:17 daily mac+github-actions |
26 h | get_source thing-<id> returns the current page. | ok |
| 14 | Linking Things → Librarian link, note |
Every public link, one passage each, into the links corpus. library.py round --only links reads Linking Things' published JSON (its nightly snapshot). |
clock, 05:00 and 07:17 daily mac+github-actions |
24 h | latest_content source_kind link lists the link. | ok Stacks on hop 5: a link saved at 01:00 reaches the Librarian the morning after next, about 52 hours at worst. |
| 15 | Tinylytics → Librarian readers |
View counts per post, page and issue. library.py round --only readers, Mac only (the GitHub round has no Tinylytics key). |
clock, 05:00 daily mac |
26 h | The library's readers table has yesterday's counts. | ok Stops when the Mac does. |
| 16 | Librarian → Thingy issue, post, episode, thing, link, topics |
The embedded corpora the API answers from (Weekly Thing, blog, podcast, reference, links). deploy.yml rebuilds each corpus the library moved past, uploads to S3, redeploys the Lambda; Thingy and MCP clients answer from it at once. |
nudge, on a round that moved the library, or a push to the pipeline github-actions |
1 h | latest_content and corpus_stats show the item. | ok Hold. From Friday 00:00 until the send (Friday and Saturday, US Central) no automatic build changes the corpus; the round still updates the library. Read 'held', not 'stale'. Thingy's drift check (06:17 daily) compares its vendored contract with the Librarian's main and files a GitHub issue on drift. |
| 17 | Librarian → Weekly Thing (web) topics |
The topic graph (graph.json) that renders the topic pages. deploy.yml commits data/librarian/graph.json to weekly.thingelstad.com with a PAT; the push deploys. |
nudge, on a corpus build that changed the graph github-actions |
1 h | weekly's topic pages list the new issue. | ok |
| 18 | Librarian → WT Builder issue, post, link |
Retrieval for Thingy drafts, Echoes and the earlier issues that carried a link. The versioned /retrieve contract, read live. |
live aws |
none | get_item shows the earlier issues that carried the link. | ok |
| 19 | Library Agent → micro.blog category, alt, featured |
Categories for unfiled posts, alt for photos without it, and the Featured choice (proposed to Jamie first). make blog-round --write: Micropub update by URL, id-checked (q=source, trust only uid, write, read back). Guards: fill blanks only, 50 category and 20 alt writes a day, never an <img> turned into markdown, WRITES_ON flag. |
clock, 05:00 daily mac · Codex automation library-agent |
26 h | q=source shows the edit, and the rendered page shows it after micro.blog rebuilds. | gap First write round, 2026-10-22: 20 edits accepted by Micropub, 4 missing from the rendered pages after the rebuild. The read-back checks Micropub, not the page, so the round was green. A write needs a receipt: written, rendered, pulled back (hop 11). |
| 20 | micro.blog → cards.thingelstad.com post, image |
The post's title, date, categories and text, for its social card. The card Lambda reads the post's h-entry on request; CloudFront caches the image. A drawing change needs /www/* invalidated. |
live aws |
none | The card URL draws the post as it is now. | ok |
| 21 | WT Builder → Another Thing episode, audio |
An episode: the mastered MP3 into the audio bucket, then its page and transcript in one commit. The Another Thing send (/another), built 2026-10-11. Since 2026-10-26 (wt-builder PR #8) it puts the tagged MP3 in the another-site audio bucket at uploads/YYYY/another-thing-N.mp3 with the AWS key in WT Builder's .env, reads it back, then commits the episode page (with audioBytes and audioSeconds) and its transcript to git. No MP3 goes to git; CloudFront serves the bucket at the same /uploads/ URL. |
nudge, on send mac |
15 min | podcast.xml carries the episode. | unverified Never run. Episodes so far were committed by hand, their MP3s put with npm run audio:put. Needs ANOTHER_GITHUB_PAT_TOKEN. The send behaves as audio:put: the same bytes are left alone, different audio at the key is refused unless an earlier send of the episode put it there, nothing deletes from the bucket. The deploy-aws job refuses a feed whose MP3 is not in the bucket at the feed's length. |
Who owns each field
| Field | Owner | Others may | Never |
|---|---|---|---|
| Post text and photos | Jamie | WT Builder reprints a post in the Journal and mirrors Jamie's edits back (hop 4). | The Library Agent edits a post's words. |
| Blog categories and their descriptions | Librarian | Jamie edits freely in micro.blog; WT Builder sets the Weekly Thing category on reprinted posts. | The Library Agent overwrites a category a person chose. It files unfiled posts and proposes changes in its report. Descriptions are micro.blog's Intro field, set by hand. |
| Alt text on blog photos | Librarian | Jamie writes alt when posting. | The Library Agent changes alt that is already there, or turns an <img> into markdown. |
| Featured | Librarian | Jamie decides on the first list before anything is written. | A Featured write without Jamie having seen the list. |
| Issue content | WT Builder | The Librarian copies it and adapts the copy for the corpus; weekly.thingelstad.com renders it and is the source of authority once sent. | A surface edits an issue. |
| Saved link and its note | Jamie | WT Builder mirrors Jamie's commentary back (hop 2); Linking Things and the Librarian read. | A note rewritten by a system without the _agent tag. |
| Read/unread on a bookmark | Jamie | Nothing. WT Builder neither selects on it nor writes it. | A system flips it. |
| Topic graph | Librarian | weekly.thingelstad.com renders it (hop 17). | A topic edited on the site. |
| Episode content and audio | WT Builder | another.thingelstad.com publishes it; the Librarian imports the transcript (hop 12). | A hand edit on the site once the send runs (today, every episode was a hand commit; see hop 21). |
What watches the hops today
| Watcher | Hops | Sees | Misses |
|---|---|---|---|
| weekly-thing-feed-publication-monitor OpenClaw cron on the Mac, 12:00 daily, posts to Discord |
7 | A new item in weekly's feed.xml or podcast.xml. | Everything downstream of the site. |
| another-thing-podcast-feed-monitor OpenClaw cron on the Mac, 08:15 daily, posts to Discord |
21 | A new GUID in another.thingelstad.com/podcast.xml. | |
| weekly-automation-health-check OpenClaw cron on the Mac, Mondays 08:30, posts to Discord only on failure |
— | Failed, disabled or stale OpenClaw jobs. | Jobs outside OpenClaw: launchd, GitHub Actions, Codex automations. |
| Thingy drift check GitHub Actions in thingy.thingelstad.com, 06:17 daily, files or closes a GitHub issue |
16 | The vendored Librarian contract and MCP surface drifting from librarian-thing main; npm advisories. | Freshness of the corpus. |
| CloudWatch alarms (Thingy, Librarian API, Linking Things MCP) AWS, into the projects-ops-alerts queue the Operator drains at 06:30 |
16, 18 | 5xx rates, auth bursts, throttling, dead letters, the subscribe path. | Whether the data is current. No alarm anywhere measures lag. |
| Library round run status GitHub Actions in librarian-thing |
10, 11, 12, 13, 14 | A red run, including a blog pull skipped for a missing key. | Nobody is notified of a red scheduled run. |
| nightly.log ~/Library/Logs/links/nightly.log on the Mac |
5, 6 | Each step's exit and counts. | A green run that joined a stale checkout (hop 6, 2026-10-26). |
Systems
| System | The fact that shows what it holds | Notes |
|---|---|---|
| Pinboard source · pinboard.in |
posts/update gives the time of the last change (token needed). | Jamie's capture tool for links. No webhooks. |
| micro.blog source · micro.blog |
The blog's feed shows the newest post; Micropub q=source shows a post's stored source. The rendered page can lag a Micropub write until micro.blog rebuilds. | Source of authority for the blog. Three writers: Jamie, WT Builder's mirror, the Library Agent. |
| Tinylytics source · tinylytics.app |
Site and page view counts over its API (key needed). | Reader counts only; never content. |
| WT Builder publisher · mac · jthingelstad/wt-builder |
get_status (MCP) shows each issue's send legs; list_events shows what was sent when. | Authors and publishes each issue, and makes Another Thing since 2026-10-11 (that send has not run yet). |
| Library Agent agent · mac · jthingelstad/librarian-thing |
Its run report under tmp/blog-round/<day>.md, and the blog_round events in the library. | Codex automation, daily 05:00. Runs the library round on the Mac (the one runner with the Tinylytics key), then files categories, fills alt and proposes Featured. |
| Weekly Thing (web) site · github-actions · jthingelstad/weekly.thingelstad.com |
feed.xml and /archive/ show the newest issue; apps/site/archive/<n>.md on main is the issue's source. | Render surface. Source of authority for issues once sent. |
| Buttondown site · buttondown.com |
The newest email in the account. | The email edition. WT Builder makes a draft; Jamie sends it. |
| files.thingelstad.com site · aws |
An object exists at the URL the issue or episode names. | Audio editions and rehosted images. |
| Another Thing site · github-actions · jthingelstad/another.thingelstad.com |
podcast.xml shows the newest episode. | Podcast publish surface. Episodes so far were committed by hand. |
| Linking Things site · mac+github-actions · jthingelstad/links.thingelstad.xyz |
/links.json and /issues.json; links_stats (MCP) gives last_saved and latest_issue; data/sync.json holds Pinboard's stamp and the fetch time. | Every public link since 2005, built from a nightly commit. |
| Things I've built site · github-actions · jthingelstad/thingelstad.xyz |
The registry on the site; activity.xml carries each day's collected observations. | Content changes by pull request; a daily collector refreshes observations. |
| Librarian archive · github-actions+aws · jthingelstad/librarian-thing |
Each corpus object in S3 carries its fact as metadata (newest-id, newest-at, item-count, built-at; since 2026-10-26): a HEAD says the newest item of each kind the uploaded corpus holds. latest_content and corpus_stats (MCP) show the same from the deployed corpus; library.db in S3 holds what the library pulled. | The library (library.db in S3), the five corpora, and the API Thingy and MCP clients answer from. |
| Thingy answers · aws · jthingelstad/thingy.thingelstad.com |
librarian.thingelstad.com/health reports source_revision and corpus_versions, the S3 versions of the corpus objects the running build was deployed against (since 2026-10-26): where they match the objects' current versions, Thingy serves what the Librarian holds; where one differs, the redeploy that serves the newer upload has not landed. | Chat client of the Librarian API. MCP clients (Claude, ChatGPT, Codex) read the same corpus through librarian.thingelstad.com/mcp. |
| cards.thingelstad.com answers · aws · jthingelstad/cards.thingelstad.com |
A card URL returns an image drawn from the post as it is now. | Draws each post's social card from its h-entry on request; CloudFront caches it. |
Kinds
| Kind | Meaning |
|---|---|
issue | a Weekly Thing issue |
post | a blog post on www.thingelstad.com |
link | a link saved to Pinboard |
episode | an Another Thing episode |
thing | an entry in Things I've built |
audio | an issue's audio edition or an episode's MP3 |
image | a rehosted photo or image |
topics | the Weekly Thing topic graph |
category | a blog post's categories, and a category's description |
alt | a blog photo's alt text |
featured | the blog's Featured choice |
note | the note on a saved link |
readers | Tinylytics view counts |