# Navigating My Own AI Revolution: Testing My Site with Playwright MCP


At this year's PSConfEU, Jeffrey Snover — the creator of PowerShell — gave a session called [Navigating the AI Revolution](https://www.youtube.com/watch?v=_GWo9drWx5s). One idea stuck with me: engineering is how we take something that isn't perfect yet and make it useful anyway. With AI specifically, the answer isn't to trust it blindly — it's to build verification layers around it. Validate the output. Trust, but verify.

That hit a nerve, because if I'm honest I'm a bit of a perfectionist and a bit of a control-freak. AI velocity is real and I want it — but handing work to something that's confidently wrong sometimes goes against every instinct I have. So I've been trying to let the reins go a little, and the thing that makes that bearable is the same thing that's always made me comfortable shipping code: tests. This post is about one testing layer in particular — [Playwright MCP](https://github.com/microsoft/playwright-mcp) — and how it lets me let go without flying blind.

<!--more-->

## A green build isn't a working site

Here's the trap I keep falling into with my own site. It's a [Hugo](https://gohugo.io/) blog, and when I ask an AI assistant to change something, the feedback loop I *want* to trust is the build: `hugo` runs, exits `0`, no errors. Done, right?

No. A passing build tells you the site *compiled*. It tells you nothing about whether the thing actually **works in a browser** — whether search still returns results, whether the table of contents renders, whether a theme change quietly broke the dark-mode toggle, or whether the console is throwing errors on every page.

That gap is exactly where AI velocity can hurt you. The assistant confidently makes a change, the build stays green, and you happily move on — while something user-facing is silently broken. I needed a verification layer that checks the *rendered* site, not just the build. That's Playwright MCP.

## Enter Playwright MCP

[Model Context Protocol](https://modelcontextprotocol.io/) servers give an AI assistant new capabilities. Playwright MCP gives it a **real browser**. Once it's wired into [Claude Code](https://claude.com/claude-code), Claude can:

- **Navigate** to any page on my local site
- **Click** things — menus, buttons, the search box
- **Screenshot** pages so I can eyeball them
- **Read the browser console** for JavaScript errors

In other words, it can *use* my site the way a visitor would, and report back what's actually happening — not what the build log claims is happening. It's the difference between "the code compiled" and "I loaded the page and search works."

## A real thing it caught

This isn't hypothetical. The first time I pointed Playwright MCP at my own site, it found that **my search was completely broken** — and had been for a while.

The story: my theme had moved on from its old search engine, but my config still asked for the old one. Hugo quietly fell back to a different search backend whose index was never being generated, so the browser was requesting a file that returned a `404`. Search returned *nothing*. And here's the kicker — **every `hugo` build had been passing the whole time.** Nothing in the build log even hinted at it. It took a browser actually clicking the search box and Claude reading the failed request off the console to surface it. A quick config fix later, search worked again.

The second win was bigger. I'd been putting off a scary upgrade — jumping Hugo up sixteen minor versions *and* moving my theme across a major version bump at the same time. Normally that's the kind of change I'd poke at nervously for a week. Instead I captured a baseline "known-good" pass with Playwright MCP first, did the upgrade, and ran the exact same checks again to diff before-and-after. Every page, every interactive feature, the console on each — checked in minutes. The upgrade went from "pray and click around" to "here's the evidence it still works." That's the velocity-with-a-safety-net feeling I was after.

## This is why I love Pester, too

If you've read this blog before, you know I'm a fan of the [Pester](https://pester.dev/) testing framework. Playwright MCP scratches the exact same itch, and I think that's why it clicked for me so fast.

Pester lets me change PowerShell quickly *because* I trust the tests to catch me. It's a verification layer for my code. Playwright MCP is a verification layer for my rendered site. Same instinct, different layer: let the fast thing be fast, and let a test hold the line so you're not the one holding your breath. The more I lean on AI, the more I want these layers — not fewer of them. Snover's point, basically, applied to my little corner of the internet.

## How to set it up

This is the setup on Windows, which is what I run. You need [Node.js](https://nodejs.org/) 18+ (it provides `npx`, which runs the server) — I install it with `winget` to keep it version-managed:

```powershell
# 0. Prerequisite (skip if `node -v` already shows 18+)
winget install --id OpenJS.NodeJS --exact --source winget

# 1. Register the Playwright MCP server with Claude Code
claude mcp add playwright -- npx -y @playwright/mcp@latest --browser chromium

# 2. Download the browser build the MCP expects
npx -y @playwright/mcp@latest install-browser chrome-for-testing

# 3. Restart Claude Code so it loads the MCP

# 4. Verify it connected
claude mcp list   # → playwright: ... ✔ Connected
```

Two gotchas that bit me, so they don't bite you:

- **Use `--browser chromium`.** Without it, the server defaults to your installed Google Chrome — which needs admin rights to install if it's missing. The bundled Chromium needs neither, and it's the one flag that makes this "just work" on a locked-down machine.
- **Use the MCP's own `install-browser` command**, not `npx playwright install chromium`. The generic installer pulls a *different* Chromium build number than the MCP's bundled Playwright expects, and you'll get a `Browser "chrome-for-testing" is not installed` error that's genuinely confusing until you know why.

## How I actually use it

Day-to-day it's simple. I start the dev server:

```powershell
hugo serve --disableFastRender -D
```

Then I ask Claude to walk through the site — load the home page, a blog post, the taxonomy pages, click the search box, toggle the theme — screenshotting as it goes and checking the console on each page. I've written the whole thing down as a small **baseline checklist** in my repo, so "run the baseline test" is now a single instruction. Before a risky change I run it to capture a known-good state; after the change I run it again and compare. That checklist *is* my verification layer, and it doubles as living documentation of what "working" means for my site.

## The honest caveats

Because nothing is free:

- **It's smoke testing, not a full test suite.** Playwright MCP tells me the site *renders and behaves* correctly in a browser. It's not a replacement for real unit tests on the code that generates it — it's the layer on top.
- **`hugo serve` runs in dev mode**, which disables things like the [PWA](https://learn.microsoft.com/en-us/microsoft-edge/progressive-web-apps/), the comment system, [CDN](https://learn.microsoft.com/en-us/microsoft-365/enterprise/content-delivery-networks?view=o365-worldwide) and fingerprinting. Those only exist in a production build, so don't try to validate them locally.
- **Search is the sneaky one** — the search index is a post-build step, so `hugo serve` can't fully exercise it. Ironically, the exact feature it caught as broken is one you have to test against a production-style build. Know that going in.
- **Headed vs headless.** By default the browser is visible (headed), which is oddly reassuring the first few times — you watch it click around. Add `--headless` once you trust it and just want it quiet and fast.

## Wrapping up

I'm not going to pretend I've fully made peace with letting AI drive. But Snover's framing helped: the move isn't to slow the AI down, it's to build the verification layers that let you speed it up safely. Playwright MCP is one of those layers for me — the one that turned "the build passed, I guess it's fine" into "I watched it work." For a recovering control-freak, that's the difference between using AI and trusting it.

If you've got a static site and you're already working with Claude Code, this is a couple of commands and a genuinely eye-opening first run. Point it at your own site and see what a green build has been hiding.

Happy scripting 😊