Content Pipelines

What this is

A Python pipeline that runs every Friday at 04:00 CET on a self-hosted GitHub Actions runner. It scrapes 18 AI newsletter subscriptions across two Gmail accounts, calls claude -p headless to draft a weekly briefing plus deep-dive articles plus three newsletter formats, pushes everything to Vercel, emails a Substack draft, and notifies both hosts.

This very site is the front-end for that pipeline. The episode briefings you see at /episodes and the articles at /articles are produced by the pipeline — not hand-written. The Friday morning sequence is: pipeline runs, content lands on disk, git push fires Vercel auto-deploy, you get a Slack-style email saying it's live.

What AI does

Claude (via the claude -p CLI in headless mode) does the heavy editorial work — synthesizing the week's newsletter intake into a structured briefing, expanding flagged stories into deep-dive articles with three different role-lensed framings (CEO, CIO/CTO, AI transformation lead), and re-cutting the briefing into three distinct newsletter formats (executive roundup, devices weekly, LLM weekly). Each generation is a separate prompt template living in friday-pipeline/prompts/.

What I did

Built the orchestrator. Picked Python stdlib for everything I could (tomllib, pathlib, subprocess, smtplib, dataclasses) to keep the dependency surface tiny. Designed the editorial-as-code constraint — voice, role lenses, format rules all live as markdown the pipeline reads. Hardened the failure-stop policy: any uncaught exception aborts before publishing, on the theory that a missed Friday is fine but a half-published one is not.

What surprised me

The cost. The pipeline runs through my Claude Code Max subscription via subprocess to the claude CLI — no API key, no per-token billing. Total marginal cost per weekly run: $0. The cron infrastructure (self-hosted runner on my PC + Windows scheduled task to wake the machine at 02:45 every Friday) is also free. Substack publishes as a draft via email-to-draft — they let you do that for free too. The whole stack is $0/week to operate.

The runner-up surprise was the reliability problem. Markdown produced by an LLM is structurally similar to markdown produced by a human, but the failure modes are different: stray horizontal rules being mistaken for frontmatter closers, line endings flipping between platforms, slugs accidentally double-prefixed. The defensive validators in the markdown writers were not optional.