Establish the general principle first, then confirm how it plays out in practice through two curation channels.
Introduction
"No human can read everything that pours in every day" usually gets one of two answers: summarize, or filter. But neither answers the real question — "So what should I actually do right now?" Solving that requires one more layer between collection and publishing: a judgment layer. And running that judgment layer automatically every day means something, somewhere, has to execute code, store state, and serve results. This post is about handling all three roles — execution, storage, serving — with a single GitHub repository. It's drawn from building two channels on the same architecture: one for security practitioners, one for DevOps practitioners.
1. Building the automation pipeline — the general principle
Curation automation is fundamentally a pipeline. Whatever the domain, the shape is the same:
collect → clean (denoise) → judge → publishTo run this unattended every day, the design must answer three questions:
- How do you handle partial failure? If it dies on item 15 of 20, do you throw away the first 14 or keep them?
- Who notices a failure, and how? Scheduled runs have no one watching. Dying silently is the worst outcome.
- How do you prevent concurrent runs? Multiple schedules overlapping means simultaneous writes to the same resource.
Example — channel implementation
These principles were implemented as three GitHub Actions workflows: a daily workflow (collect → judge → deploy), a weekly workflow (a batch that scans only core open-source releases), and a control workflow to pause and resume with a single button.
The answers to the three questions:
- Partial-failure handling — the judgment step carries
continue-on-error: true, so even if credits run out or auth breaks, everything judged before that point is still committed and deployed. - Failure visibility — if the judgment step dies, the workflow opens an issue automatically via
gh issue create. Cron-failure emails are easy to miss, so it leaves a visible trace: an issue. - Concurrency control — a
concurrencygroup prevents the daily and weekly workflows from pushing over each other. - Pause — since GitHub Variables can't be written from inside a workflow, pause state is expressed with a single marker file (
.collect-paused). If the file exists, a gate job skips the later steps.
The actual first run: 99 collected → 20 judged after noise filtering → 1 Act / 5 Plan / 9 Learn / 5 Skip → 15 published. One source (Reddit) blocked the GitHub Actions IP (403), but the other RSS sources collected normally — thanks to per-source error isolation.
2. "Static site hosting" and "an information-collection engine" are the same infrastructure
The general principle
Static hosting is usually treated as just "a place to drop built output." But the moment you attach a scheduled execution environment, it gains a second role — a worker that wakes at a fixed time to observe the outside world, judge it, and produce content. If you don't separate the two but bind them to the same repository and the same commit history, you get a structure where "the run's output is simultaneously the publication and the state record" — with no separate database or queue. Data the pipeline produces gets committed, and that commit re-triggers deployment: a loop.
Example — channel implementation
GitHub Pages (serving) and GitHub Actions (execution) were combined in the same repository. Markdown posts the pipeline generates are saved via git commit → git push, and that commit re-triggers the static-site build and Pages deploy. The dedupe record (a list of processed URLs with a 90-day TTL) is also versioned in git, with no separate DB. With no server provisioning, a single manual-trigger button or one cron tick carries everything from collection to deploy — GitHub Pages expands from "static hosting" into "a free cron server that observes and judges the world every day."
3. What it means to use an OAuth token instead of an API key for LLMs in CI
The general principle
The standard way to integrate an LLM into a CI pipeline is to register an API key as a secret and call the API. But that also means "creating a separate billing account dedicated to this pipeline." By contrast, if you carry an already-subscribed account's CLI login session into CI as an OAuth token, you can split the judgment backend in two and rank them by priority — token present → subscription-based execution, token absent → fall back to the API key automatically. This changes the billing model, the rotation policy, and the incident-response procedure entirely.
| API Key | OAuth Token | |
|---|---|---|
| Billing | Credits consumed per call; charges accumulate | Reuses existing subscription; no extra cost |
| Revocation / rotation | Permanent-permission key; unlimited abuse if leaked | Scoped at issue time; re-issue on expiry — natural rotation |
| Auth path | Requires managing a separate service account | An extension of an already-logged-in account |
Example — channel implementation
An OAuth token issued from the CLI login session was registered as a workflow secret. The workflow sets priority in a single line:
JUDGE_BACKEND: ${{ secrets.OAUTH_TOKEN != '' && 'subscription' || 'api-key' }}If the token is present, it calls a subscription-based backend that invokes a CLI subprocess; if absent, it falls back to the API-key backend. As a result, judging 20–30 items a day is consumed within the existing subscription's quota, with no separate API charges. The expiry procedure is clear too — re-issue the token → update the secret → close the auto-opened failure issue. That said, this fits a personal / side-project daily pipeline; for team-scale traffic or full service-account separation, the API key is still the right call — which is why the code leaves both open and only sets the priority.
4. What actually has to change when porting one architecture to another domain
The general principle
The skeleton of a "collect → judge → publish" pipeline (workflows, error handling, deploy logic) is domain-neutral and reused as-is. But the basis for judgment must be redesigned per domain. In particular, a domain that has an objective anchor — machine-queryable data telling you "is this urgent right now?" — and one that doesn't differ greatly in design difficulty. The weaker the anchor, the more you must explicitly calibrate boundary criteria so that multiple judging perspectives (personas) rate the same item differently — otherwise the verdicts blur together.
Example — from security curation to devops curation
The security curation channel had a strong anchor — CISA KEV and EPSS, which let you machine-query "is this dangerous right now?" Moving to the devops curation channel, there was no such single authoritative source, so a public API aggregating open-source projects' EOL / end-of-support dates (covering ~460 products) was set up as the KEV substitute, augmented with regex extraction of deprecation deadlines from the body and semver-based release-maturity detection to reconstruct the basis for judgment.
Persona design was the same story. Rather than reusing the security side's engineer / analyst / leader trio with only renamed labels, the mutually exclusive criteria were made explicit so the same item earns different verdicts per persona — e.g. "a routine CVE patch is Act for a platform engineer but Skip for a leader." Turning those boundaries into a calibration example table and feeding it into the judgment prompt filled the gap that persona descriptions alone couldn't. In the end, the same architectural pattern was replicated into two independent channels, both sharing the trust principle of "no manufactured urgency" (e.g. a zero-Act day is normal).
Closing
One conclusion runs through all four cases — when designing an automation pipeline, first establish the general principles (partial-failure handling, the fusion of execution/storage/serving, the billing-and-rotation model of the auth method, the domain-portability of the judgment basis), and decide only afterward how they're implemented within concrete constraints. With the principles standing first, the tools (GitHub Actions, Pages, OAuth tokens) are merely options — and even when the domain changes (security → devops), the principles remain, giving you portability.
Open Source Repositories & Live Channels
To see this architecture in action and explore how the principles are implemented, you can check out the live automated channels and their complete source code below:
Feel free to explore the workflows, fork the repositories, or use them as a reference blueprint for your own automated pipeline!
