When an assistant keeps naming a competitor, “why them” is the wrong question. Ask which pages made the assistant believe them, and what changed on those pages this week.
So I built a small agent. Every morning it visits a handful of pages, asks Claude what changed since yesterday, and emails me only when it matters. It costs about $7.80 a month. In two weeks it logged 41 changes across three sites. 5 were worth knowing about.
Here is how it works, and how to point it at what actually moves AI recommendations: the sources behind your competitor.
What the agent does each morning
- 6:00 am. A cron job on a $6 server starts a headless browser.
- Capture. It visits each watched page and waits for it to settle. It takes a full-page screenshot and saves the title and visible text. Three pages take about 12 seconds.
- Store. Each capture goes into a
snapshotstable, screenshot in storage. - Diff. Claude gets yesterday’s text and today’s. It returns one paragraph of at most four sentences, ending with a significance score from 1 to 5.
- Log. The summary, the categories it touched and the score go into a
diffstable. - Notify. At about 7:30 am, an email goes out for anything that scored 3 or more. Everything else waits in the table for when I am curious.
The prompt lists what to watch for: pricing changes, new products, new linked pages, removed features, positioning shifts, sign-up flows, new social proof.
It also carries an ignore list, and the ignore list does 80 percent of the work. Cookie banners, dynamic counts, A/B copy swaps, randomised testimonials, the footer year, ads. Adding it cut false alerts roughly in half.
I used a real browser, not a plain fetch. Most sites now draw their pricing tables and comparison blocks with JavaScript. A fetch sees an empty box. A headless browser sees what a buyer sees, for roughly half a cent a page.
What it costs
| Item | A month |
|---|---|
| Small server that runs the browser | $6.00 |
| Database and storage, free tier | $0.00 |
| Claude API, 3 diffs a day | about $1.80 |
| Email sending, free tier | $0.00 |
| Total | about $7.80 |
On a free build runner it would be nearer $2 a month. I keep the server because it does other jobs.
What it caught in two weeks
I watched three small business sites in a category I was curious about. Of 41 changes, 5 scored 3 or higher:
- A price test in public. One site moved its middle tier three times in eight days: $24, $28, $26. The $26 stuck. That is the number their data supported.
- A quiet move upmarket. One site added a “for teams” page, linked from the footer and indexable, with no announcement. An assistant will read that page next time someone asks for a team tool.
- A swapped page. One site removed its “about” page and put shipping and returns in its place. Nobody announces that either.
The other 36 were footer tweaks, copy tests, image swaps, a font experiment and a payment-icon change.
The surprise was how often small businesses change things. My own site, at the time, would have produced zero alerts. That was the real finding: a site that never changes gives an assistant nothing new to read.
Point it at what AI reads
Watching a competitor’s homepage is interesting. Watching the pages that made the assistant choose them is useful.
Our free check records, for every answer, who was named instead and which sources were cited. That source list is what to watch:
- The “best X” lists that name your competitor. When the editor updates one, you want to know that week. That is when a pitch lands.
- The review profiles the assistant cites. New reviews, a changed rating, a new category tag.
- The community threads it keeps pulling from. A thread with a new answer naming a product is one you can answer honestly, under your own name.
- Your competitor’s comparison and alternatives pages. Assistants quote “X vs Y” and “alternatives to X” pages often, and competitors add them quietly.
Set the score threshold at 3, keep the ignore list, and let the table fill up. Once a month, re-run the same buyer questions and see if the mention rate moved. We explain the counting on how we measure: named in X of 60 answers, never a rank.
Be a polite robot
I only look at pages a normal visitor can see. No logging in. No scraping behind a paywall. No fake sign-ups to read a welcome sequence. No buying and returning to see the checkout.
If you would not do it by hand, do not automate it. The same rule runs through our clean methods.
A first version of this appeared on my studio blog, dashbuilds.dev.
What to do next
If a competitor keeps getting named, start with the sources, not the agent. A competitor keeps getting named instead explains how we find the pages behind them and get you onto the same ones.
The free check produces the list to watch: 5 questions, 3 runs, 5 assistants. You get who was named instead and what was cited, by email within 2 working days.
Questions
Is watching a competitor's site allowed?
Looking at pages any visitor can see is fine. My rule is polite robot, polite human: no logging in, no fake newsletter sign-ups, no buy-and-return tricks. If you would not do it by hand, do not automate it.
Why not just re-run the buyer questions every day instead?
You can, and we do it monthly. Daily is too noisy: the same question gives a different brand list most runs. The pages assistants cite are a steadier signal of what is about to change.
Do I need a real browser for this?
Usually yes. A plain fetch misses anything drawn with JavaScript, which is most modern pricing and comparison pages. A headless browser costs about half a cent a page and sees what a buyer sees.
How much of what it catches is noise?
Most of it. Across three sites in two weeks my agent logged 41 changes. Only 5 were worth an email. An ignore list (cookie banners, footer years, rotating testimonials, A/B copy) cut the noise roughly in half.
