From Python scraper to RSS feed: give your scripts a subscription button
Turn the output of any Python scraper into a valid RSS feed — and learn why feeds are the best delivery format for scraped data.
Your scraper already finds the items. The last mile is delivery, and the most useful delivery format ever invented for this is RSS: no API to design, no app to build, and every reader on the planet already understands it.
Why RSS instead of a JSON endpoint
- Universal inbox — one URL works in Feedly, NetNewsWire, Slack, Discord, Telegram, and email digests.
- Incremental by design — readers fetch only what changed.
- Durable — the format is over 20 years old and going nowhere.
- Machine-friendly — AI agents and automation tools consume feeds natively.
A JSON endpoint is a project; an RSS feed is a feature you finish in an afternoon.
The smallest valid generator
from xml.etree import ElementTree as ET
from datetime import datetime
items = [
{"title": "New advisory posted", "link": "https://example.com/n/1",
"date": datetime(2026, 10, 22, 9, 0)},
]
rss = ET.Element("rss", version="2.0")
channel = ET.SubElement(rss, "channel")
ET.SubElement(channel, "title").text = "Example Monitor"
ET.SubElement(channel, "link").text = "https://example.com"
ET.SubElement(channel, "description").text = "Advisories found by our scraper."
for item in items:
entry = ET.SubElement(channel, "item")
ET.SubElement(entry, "title").text = item["title"]
ET.SubElement(entry, "link").text = item["link"]
ET.SubElement(entry, "guid").text = item["link"] # stable id = no duplicates
ET.SubElement(entry, "pubDate").text = item["date"].strftime(
"%a, %d %b %Y %H:%M:%S GMT"
)
ET.ElementTree(rss).write("feed.xml", encoding="utf-8", xml_declaration=True)
Two rules keep the feed healthy:
- The
guidmust be the canonical link. Readers deduplicate on it, so links must be stable — strip tracking parameters. - Newest first. Readers assume it; your cron job should sort by date descending before writing.
Schedule it, then forget it
Run the scraper with cron or a systemd timer, write the feed to a static file, and point a reader at it:
0 8 * * * cd /opt/monitor && python scraper.py > feed.xml
Add a last_modified-style check so unchanged runs skip rewriting the file.
What to add next
- A per-item
description(plain text, first paragraph of the article). - An
<enclosure>for images or audio. - A
Last-Modified/ETagcheck so you respect conditional requests.
Or skip the plumbing
Maintaining the script, the scheduler, and the XML is exactly the job RSSFlux automates: point it at the listing page, and it generates, refreshes, and serves the feed — with dedupe, thumbnails, and delivery into Slack, Discord, Telegram, email, or an API. The free plan includes three feeds, so you can compare your hand-rolled feed against a managed one side by side.
If this is your first scraper, start with the basics. For what "newest-first and deduplicated" looks like at scale, see how to create an RSS feed for a website that doesn't have one.
Frequently asked questions
- Why publish scraper output as RSS?
- RSS is a universal inbox format. Readers, bots, email digests, and AI agents all consume it without you writing an API, a database, or a frontend.
- Do I need a web server for an RSS feed?
- You need the file at a public URL. Host it with any static file hosting, a small Flask/Next.js route, or let a service host it for you.
- What should an item contain?
- At minimum a title, link, and pubDate. Add a short description and a unique guid so readers can deduplicate correctly.