1 min readRSSFlux Research

Python tips for cleaner scripts: dataclasses, type hints, retries, and logging

Four small Python habits that turn throwaway scripts into tools other people — including future you — can actually run.

pythontipslessons

Scripts start clean and drift into mud. Four habits stop most of the drift without adding ceremony.

1. Dataclasses for records

Dictionaries are fine until a typo writes titel next to title and nobody notices for a week. A dataclass makes the record explicit:

from dataclasses import dataclass

@dataclass
class Article:
    title: str
    link: str
    published: str = ""

article = Article(title="Hello", link="https://example.com")
print(article.titel)  # AttributeError at development time, not next month

For scraped items this is the single highest-value change: one definition of what an "item" is, checked everywhere it is used.

2. Type hints as documentation

def fetch_items(base_url: str, limit: int = 20) -> list[Article]:
    ...

Editors now autocomplete, mypy catches mismatches, and the signature reads like a sentence. Hints cost nothing at runtime.

3. Retries only when they help

Retry on transient failures — timeouts and 5xx — never on 4xx, which mean "stop asking":

import time

def fetch(url: str, retries: int = 3) -> str:
    for attempt in range(retries):
        try:
            response = requests.get(url, timeout=10)
            if response.status_code < 500:
                return response.text  # 4xx surfaces immediately
        except requests.RequestException as exc:
            last_error = exc
        time.sleep(2**attempt)  # 2s, 4s, 8s backoff
    raise RuntimeError(f"fetch failed after {retries} attempts: {last_error}")

Exponential backoff with a cap protects both you and the site you are scraping.

4. Logging instead of printing

import logging

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("monitor")

log.info("fetched %d items", len(items))
log.warning("zero items found — selector drift?")

The last line is the important one: a scraper that found nothing should make noise. Log files also survive cron, unlike terminal output.

Putting it together

@dataclass
class Article:
    title: str
    link: str
    published: str = ""

def run(base_url: str) -> list[Article]:
    log.info("scraping %s", base_url)
    html = fetch(base_url)
    articles = parse(html)
    if not articles:
        log.warning("no articles parsed from %s", base_url)
    return articles

if __name__ == "__main__":
    import argparse

    parser = argparse.ArgumentParser()
    parser.add_argument("url")
    args = parser.parse_args()
    for article in run(args.url):
        print(article.title, article.link)

None of these habits require a framework — they are just decisions made early. Apply them to the code in the scraping basics lesson and your scripts survive their own success.

When your scripts graduate to a product, the same discipline carries over to the frontend — see Next.js vs Vite vs Astro for how we choose the UI layer.

Frequently asked questions

Do type hints slow Python down?
No. They are erased at runtime and only help tools like editors, mypy, and readers understand your code. You get documentation for free.
When should I use dataclasses instead of dicts?
As soon as you have a fixed set of fields. Dataclasses catch typos at construction, show up in tooltips, and make refactors safe.
Why logging instead of print?
print goes to stdout and mixes with program output; logging carries levels, timestamps, and can be routed to files or sentinels. In a scraper, a warning level is the difference between noticing a failure and a silent zero-item run.

Do this without writing code

RSSFlux turns any public web page into a feed, keeps it fresh, and routes updates to Slack, Discord, Telegram, email, or your own API. Start free.

Related reading