I Ran Chrome's Modern Web Guidance on My Own Site. It Found 88 Lines I Never Wrote.

TL;DR: Your agent writes 2021-era JavaScript for things the browser now does natively, and it can’t tell that it’s doing it. Modern Web Guidance is a preview-stage skill from the GoogleChrome org that fixes this by injecting current web platform docs into the agent’s context before it writes. It never reads your code — it matches a sentence you type against 137 guide descriptions. I pointed it at this site and it found 88 lines of theme JavaScript I’d never opened. Set a browser support policy before you use it.

Why you’d want this at all

Every model that writes frontend code has a training cutoff, and the web platform moves faster than that cutoff. This produces a specific failure that is almost impossible to catch by reading a diff: the code your agent writes is correct. It works. It’s just solving a problem the browser solved two years ago, in fifty lines instead of five.

You won’t catch it in review, because there’s nothing wrong with it. You’d only catch it if you already knew the platform feature existed — and if you knew that, you’d have asked for it.

I’ll make the case against myself. I spent an evening with an agent reading this codebase, and neither of us flagged the 88-line theme toggle sitting in public/toggle-theme.js. It looked like theme code. Theme code looks like that. It took an external source of truth to say “the browser does most of this now.”

That’s the whole pitch. It’s not a linter and it’s not an auditor. It’s a correction for the one thing your agent is structurally incapable of noticing.

What it actually is, mechanically

This part is worth understanding, because everything about how you use it follows from it.

npx -y modern-web-guidance@latest search "..." downloads a 35MB package. Most of that is a neural network: tfjs_model_minilm/group1-shard1of1.bin is 22MB of weights for all-MiniLM-L6-v2, a sentence-embedding model, converted to TensorFlow.js. Next to it sits a 5.3MB gzip of pre-computed embeddings for all 137 guides, and a 695KB tokenizer. Zero npm dependencies — TensorFlow.js is bundled into an 832KB search.mjs.

When you search, your query gets tokenized, pushed through MiniLM to produce a 384-dimension vector, and compared by cosine similarity against the 137 guide vectors that shipped in the package. Top five come back.

So "similarity": 0.7638 is not a relevance score in the search-engine sense. It is the cosine similarity between two English sentences: the sentence you typed, and the guide’s one-line description. Nothing else is in the comparison.

Which leads to the thing that confused me longest:

It never reads your code. There is no repository in the picture. I ran the identical query from /tmp, nowhere near my project, and got byte-identical output including the same 0.7638. The working directory is irrelevant because nothing on your disk is ever opened.

To prove it to myself the other direction, I fed it identifiers straight out of my own file:

npx -y modern-web-guidance@latest search "toggle-theme.js data-theme attribute localStorage getComputedStyle"

0.4807, and the top hit was design-token-reactivity — a guide about container style queries. My actual dark-mode code, and the tool had no idea it was about dark mode. As English, that string doesn’t resemble “implement dark mode support.”

retrieve doesn’t touch the network either. All 137 guides ship as plain markdown inside the package.

Now what: the query is the entire skill

Since the match is sentence-to-sentence, your phrasing is the interface. Describe a goal, not a feature list.

Here’s the same requirement — “build a dropdown menu” — framed three ways:

What I typed	Top hit	Score
build a dropdown menu	`resilient-context-menus-and-nested-dropdowns`	0.614
show a menu of actions anchored to a button	`declarative-dialog-popover-control`	0.555
style a select element to match my brand	`branded-select-styling`	0.703

Same component in my head. Three different guides, and 0.15 between the best and worst framing.

That spread is also telling me something useful: “dropdown menu” is ambiguous. A native <select>, a popover menu of actions, and a nested context menu are three different components with three different platform features behind them. The scores surfaced an unclear requirement before I wrote any code.

Reading the score. It always returns five results — there is no “no match” response, so the number is how you tell.

Above ~0.65 — a real match. Retrieve it.
0.5 to 0.65 — adjacent. Read the descriptions before spending the tokens.
Below 0.5 — nothing specific exists for that phrasing.

There’s a second tell. When the catch-all discipline guides (css, html, accessibility) start floating into your top five, that means nothing specific matched and it’s falling back on the broadest thing it has. I hit this searching “full height layout mobile viewport units” — 0.58, and the answer would have meant reading a 5,700-token layout omnibus to find 100dvh.

And a low score doesn’t mean no guidance exists. It means nothing matched that sentence. My dropdown went from 0.614 to 0.703 on a rephrase. Try once more before falling back on the model’s own judgment.

Where it sits when you build with an agent

It’s a pre-write lookup, not a post-write check. The skill’s own instruction is to run it “at the start of implementing any web feature.” It cannot validate your output, because it never sees your output. By the time code exists, the guide has already done its job.

The loop for a new feature:

Agent decomposes the work.
For any step touching HTML, CSS, or client-side JS, it writes a goal sentence and searches.
If the score clears the bar, it retrieves the guide.
The guide states a Baseline date for every feature it recommends.
The agent checks that date against your browser support policy and decides whether a fallback is required.
It writes the code.

Step 5 is the only thing resembling validation, and half of it is your file, not the tool. The guide supplies the Baseline date; your AGENTS.md supplies the rule. Neither works alone, which is why I put those two sections next to each other.

The failure mode to watch: agents run the lookup on step one and forget by step five. The skill triggers on task framing, not on every file write, so long builds drift back toward training defaults partway through. Front-load the searches at design time for the whole feature rather than per-step — that also avoids paying 700 to 7,800 tokens over and over. And when you review the diff, a setTimeout driving an animation or a scroll listener driving a visual effect is your signal that it stopped consulting anything.

Then what: what it actually found here

The real query I typed, before writing a word of this post:

npx -y modern-web-guidance@latest search "add dark mode that follows system preference"

{
  "id": "dark-mode",
  "featuresUsed": [
    "color-scheme",
    "prefers-color-scheme media query",
    "light-dark()",
    "accent-color"
  ],
  "tokenCount": 4125,
  "similarity": 0.7638
}

The tokenCount field is underrated. You see the cost before you fetch, so you decide whether 4,125 tokens is worth the context window before it’s in there.

The guide wasn’t a spec dump. It opened with two declarations marked MANDATORY:

<meta name="color-scheme" content="light dark" />

:root {
  color-scheme: light dark;
}

Without color-scheme, the browser leaves native scrollbars, form controls, and the initial paint canvas in light mode even if your hand-themed dark palette is perfect. That’s the white flash on load that makes a dark site feel broken.

I grepped my own styles:

color-scheme in global.css:  0
light-dark() in global.css:  0

Zero. My CSS defines --background: #fdfdfd and --background: #212737 under separate data-theme selectors, and 88 lines of JavaScript choreograph which one applies.

The 88 lines

That JavaScript lives in public/toggle-theme.js. It flips a data-theme attribute on <html>, reads getComputedStyle(document.body).backgroundColor to sync a <meta name="theme-color"> tag, listens for astro:after-swap and astro:before-swap to survive page transitions, and separately subscribes to prefers-color-scheme changes.

I didn’t write any of it. It arrived with the AstroPaper starter theme, in the commit named “Initial commit from Astro.” In the nine months since, I’ve touched that file exactly once, and the entire diff was one string:

-const primaryColorScheme = ""; // "light" | "dark"
+const primaryColorScheme = "dark"; // "light" | "dark"

That’s the part worth sitting with. This isn’t legacy code I’m sentimental about. It’s code I never read. It worked on day one, so it became furniture — an 88-line dependency I would have told you I understood, right up until something asked me to justify it.

I’m not going to pretend the guide rewrites this for free. The toggle persists a user choice to localStorage and overrides the system preference, which color-scheme alone doesn’t give you. But the guide’s UX section made me sit with whether I need that, and it had a specific answer: don’t build a three-state toggle (system / light / dark), because two of the three options always produce the same visual result. That kind of product judgment inside a CSS guide is what a model won’t volunteer on its own.

The one line I did write was the wrong one

Remember my single edit — primaryColorScheme = "dark". The guide has a rule aimed directly at it:

DO NOT set color-scheme: light or color-scheme: dark on the root element by default. The default color-scheme MUST be the user’s system preference.

My search query was “add dark mode that follows system preference.” My site doesn’t follow it. That one string means a first-time visitor on a light-mode machine gets dark anyway, because getPreferTheme() checks primaryColorScheme before it ever reads prefers-color-scheme. The 88 lines I inherited handled system preference correctly. The one line I contributed broke it.

I’m leaving it for now — it’s a deliberate look, and the guide’s real objection is to defaults that ignore the user rather than to having a house style. But I’d been calling it a preference-aware toggle, and it hasn’t been one since May.

The lines it told me to keep

The guide also shipped gotchas you’d only learn by shipping a bug:

Don’t animate scrollbar-color. A WebKit bug makes the scrollbar flicker every time it changes.
On macOS, custom scrollbar colors are ignored entirely because of native overlay scrollbars, unless you pair them with scrollbar-width.

The second sent me back to my own global.css:

scrollbar-width: auto;
scrollbar-color: var(--color-muted) transparent;

Correct, by accident. I have no memory of pairing those deliberately, and I’d have called the scrollbar-width line dead weight if you’d shown it to me out of context. That’s the quieter value: it tells you which of your lines are load-bearing, not just which ones are wrong.

The scoreboard

I ran it against six targets on this site. Two were worth acting on — the missing color-scheme declarations, and a 52-line scroll listener in my back-to-top button that animation-timeline: scroll() replaces. One needs a spike. Three were already fine.

Two out of six is the honest number, and I’d rather publish it than a longer list. The value wasn’t volume. It was that the biggest pile of hand-written code on the site and the most fundamental missing declaration both sat in files that came with the template and had never been read.

The honest limits

You drive it. It answers “what’s the current best practice for X.” You still have to read the code, recognise what it’s trying to accomplish, and turn that into a sentence. That translation is the actual work, and the tool cannot do it — as my 0.48 experiment showed.
Search has a recall ceiling. Vague queries surface omnibus guides, and then you’re reading thousands of tokens to find one answer.
Guides vary a lot in size. Focused ones are cheap at 700 to 2,500 tokens. The catch-alls run 4,500 to 7,800. Fine for a deliberate pass, expensive if you wire retrieval into every edit.
It only speaks HTML, CSS, and client-side JS. Nothing it knows applies to your routing, your build config, or your data layer.
Telemetry is on by default. A background watchdog process POSTs usage events to Google’s logging endpoint when the command exits. The README says search queries are collected, but that isn’t true of the version I ran — in 0.0.177 the query field is commented out of the payload. What actually ships is the guide IDs you matched, their similarity scores, your OS, the package version, a bucketed latency, and a success flag. Your query text stays on your machine. DISABLE_TELEMETRY=1 turns it off, and it’s still worth setting in a work repo, because the guides you match are themselves a decent map of what you’re building.

Set a browser support policy first

Real products often can’t assume Baseline. Every recommendation is keyed to a Baseline date so the agent can make the call itself, but that only works if you’ve told it what you accept.

color-scheme has been Baseline since February 2022 and is Widely Available. light-dark() has only been Baseline since May 2024, making it Newly Available — it isn’t due to go Widely Available until November 2026. Under a “Baseline 2022” target, the first ships clean and the second gets a @supports (color: light-dark(white, black)) fallback.

Write the policy down before you start, or every recommendation becomes a judgment call you make twice. Mine lives in AGENTS.md and covers three tiers — Widely Available ships bare, Newly Available needs a feature-detected fallback under 20 lines, and Limited Availability is decorative-only. I only added that third tier because of the audit: the single biggest finding on my site, scroll-driven animations, isn’t supported in Firefox, and without a rule for that case I had nothing to evaluate it against.

The fallbacks come with the guides, usually under 50 lines, not a polyfill bundle. Constrained browsers aren’t a reason to skip this. They’re the reason the policy line exists.

Try it

npx -y modern-web-guidance@latest install

Then point it at a file you’ve never opened — the starter code, the thing that came with the template. Describe what that file is trying to do, in a sentence, as if explaining it to someone else. That sentence is the whole interface.

Mine found 88 lines I’d never read in a file I didn’t write, and one line I did.