How Next.js helps your site get found: server components, metadata, and static pages
The concrete Next.js features that decide whether crawlers and AI assistants see your content — and how we use them for this blog.
A pretty app can be invisible. When a crawler fetches your page, it gets the server's first response — and if that response is an empty shell waiting for JavaScript, your content might as well not exist. Next.js exists, among other reasons, to fix exactly this.
The shell problem
A classic React app ships one line of HTML plus a JavaScript bundle; the page is assembled in the browser. Users see a fine website. Crawlers and AI assistants that do not execute JavaScript see nothing.
Next.js flips the default: components render to HTML on the server first. JavaScript is an enhancement for interactivity, not a requirement for content.
Server components are the content layer
Mark a page as a server component (the default in the App Router) and it renders straight to HTML. Reserve "use client" for genuinely interactive islands — buttons, inputs, stateful widgets — not for paragraphs.
One real bug we fixed on this very site followed this pattern: a client-side wrapper returned nothing until a browser effect ran, which silently stripped every page of its server HTML. The lesson: never gate content behind a client effect. Render on the server, enhance on the client.
Metadata per page
Every route gets its own title, description, canonical, and social image:
export async function generateMetadata({
params,
}: {
params: Promise<{ slug: string }>;
}): Promise<Metadata> {
const { slug } = await params;
const post = getPostBySlug(slug);
if (!post) return { title: "Post not found" };
return {
title: post.title,
description: post.description,
alternates: { canonical: post.canonical },
openGraph: {
title: post.title,
description: post.description,
type: "article",
publishedTime: post.date,
images: [{ url: post.ogImage, width: 1200, height: 630 }],
},
};
}
That single function is what makes each article shareable, canonical, and unambiguous to every parser.
Pre-render every page you can
export async function generateStaticParams() {
return getAllPosts().map((post) => ({ slug: post.slug }));
}
This tells the build to emit one static HTML file per article. The files hold the full article text — no runtime fetching, no JavaScript required, and instant responses from the CDN.
Ship the machine-readable layer too
Static HTML is the base; the discovery layer makes it findable:
sitemap.ts— an XML list of every real page, updated from the same data as the routes.robots.ts/robots.txt— allow the AI crawlers you want (OAI-SearchBot, PerplexityBot, ClaudeBot, Google-Extended) while hiding app-only paths.- JSON-LD —
Article,FAQPage,BreadcrumbList,Organization, andSoftwareApplicationblocks that give search engines structured facts instead of guesses. - An RSS feed — a plain, durable syndication format every reader and agent already speaks.
llms.txt— a simple index for LLM tooling that has become the emerging convention.
This blog uses all of them: the RSS feed, the sitemap, and per-article schema you can inspect in the page source.
The checklist
- Content in server components; client components only for interactivity
-
generateMetadataon every route, with canonical and OpenGraph -
generateStaticParamsfor any content collection - Sitemap, robots, JSON-LD, RSS, and
llms.txt - Verify by curling a page and grepping for your heading — if it is not in the HTML, neither is your SEO
For the framework comparison that motivates most of this, see Next.js vs Vite vs Astro. And if the data your site displays comes from scraping, the Python lessons cover that end of the pipeline.
Frequently asked questions
- Do search engines run JavaScript on my pages?
- Google often does, but many AI crawlers and smaller engines do not. Server-rendered HTML is the only way to guarantee your content is visible to all of them.
- What is the difference between server and client components?
- Server components render to HTML on the server and never ship JavaScript. Client components run in the browser for interactivity. Put content in server components; wrap only interactive islands as client components.
- Does this blog use any of this?
- Yes — every article is a server component pre-rendered with generateStaticParams, with per-post metadata, canonical URLs, JSON-LD schema, an RSS feed, and a sitemap. The same patterns are applied to the product pages.