2 min readRSSFlux Research

How Next.js helps your site get found: server components, metadata, and static pages

The concrete Next.js features that decide whether crawlers and AI assistants see your content — and how we use them for this blog.

nextjsseolessons

A pretty app can be invisible. When a crawler fetches your page, it gets the server's first response — and if that response is an empty shell waiting for JavaScript, your content might as well not exist. Next.js exists, among other reasons, to fix exactly this.

The shell problem

A classic React app ships one line of HTML plus a JavaScript bundle; the page is assembled in the browser. Users see a fine website. Crawlers and AI assistants that do not execute JavaScript see nothing.

Next.js flips the default: components render to HTML on the server first. JavaScript is an enhancement for interactivity, not a requirement for content.

Server components are the content layer

Mark a page as a server component (the default in the App Router) and it renders straight to HTML. Reserve "use client" for genuinely interactive islands — buttons, inputs, stateful widgets — not for paragraphs.

One real bug we fixed on this very site followed this pattern: a client-side wrapper returned nothing until a browser effect ran, which silently stripped every page of its server HTML. The lesson: never gate content behind a client effect. Render on the server, enhance on the client.

Metadata per page

Every route gets its own title, description, canonical, and social image:

export async function generateMetadata({
  params,
}: {
  params: Promise<{ slug: string }>;
}): Promise<Metadata> {
  const { slug } = await params;
  const post = getPostBySlug(slug);
  if (!post) return { title: "Post not found" };

  return {
    title: post.title,
    description: post.description,
    alternates: { canonical: post.canonical },
    openGraph: {
      title: post.title,
      description: post.description,
      type: "article",
      publishedTime: post.date,
      images: [{ url: post.ogImage, width: 1200, height: 630 }],
    },
  };
}

That single function is what makes each article shareable, canonical, and unambiguous to every parser.

Pre-render every page you can

export async function generateStaticParams() {
  return getAllPosts().map((post) => ({ slug: post.slug }));
}

This tells the build to emit one static HTML file per article. The files hold the full article text — no runtime fetching, no JavaScript required, and instant responses from the CDN.

Ship the machine-readable layer too

Static HTML is the base; the discovery layer makes it findable:

  • sitemap.ts — an XML list of every real page, updated from the same data as the routes.
  • robots.ts / robots.txt — allow the AI crawlers you want (OAI-SearchBot, PerplexityBot, ClaudeBot, Google-Extended) while hiding app-only paths.
  • JSON-LD — Article, FAQPage, BreadcrumbList, Organization, and SoftwareApplication blocks that give search engines structured facts instead of guesses.
  • An RSS feed — a plain, durable syndication format every reader and agent already speaks.
  • llms.txt — a simple index for LLM tooling that has become the emerging convention.

This blog uses all of them: the RSS feed, the sitemap, and per-article schema you can inspect in the page source.

The checklist

  • Content in server components; client components only for interactivity
  • generateMetadata on every route, with canonical and OpenGraph
  • generateStaticParams for any content collection
  • Sitemap, robots, JSON-LD, RSS, and llms.txt
  • Verify by curling a page and grepping for your heading — if it is not in the HTML, neither is your SEO

For the framework comparison that motivates most of this, see Next.js vs Vite vs Astro. And if the data your site displays comes from scraping, the Python lessons cover that end of the pipeline.

Frequently asked questions

Do search engines run JavaScript on my pages?
Google often does, but many AI crawlers and smaller engines do not. Server-rendered HTML is the only way to guarantee your content is visible to all of them.
What is the difference between server and client components?
Server components render to HTML on the server and never ship JavaScript. Client components run in the browser for interactivity. Put content in server components; wrap only interactive islands as client components.
Does this blog use any of this?
Yes — every article is a server component pre-rendered with generateStaticParams, with per-post metadata, canonical URLs, JSON-LD schema, an RSS feed, and a sitemap. The same patterns are applied to the product pages.

Do this without writing code

RSSFlux turns any public web page into a feed, keeps it fresh, and routes updates to Slack, Discord, Telegram, email, or your own API. Start free.

Related reading