How to check your website automatically after every deploy
A deploy says "success" and the site is still broken. What to check on each new version, a small script that does it, and how to run it from CI.
· 6 min read
The deploy log is green. Lovable, Vercel or Netlify says the new version is live. You close the tab. Two hours later someone writes that the page is blank, or that the signup button does nothing.
"Deploy succeeded" means one thing: the build finished and the files were uploaded. It does not mean the page renders, that the scripts it asks for exist, or that nothing secret got bundled into them. Those are different questions, and nobody asks them unless something is set up to ask.
This guide covers what to check on every new version, how to do it by hand in two minutes, and how to make a small script run the same checks right after each deploy.
Why a green deploy can still be a broken site
AI builders and modern hosts make deploys cheap. You change a prompt, the tool rebuilds, a new version goes out. Some projects ship ten versions a day. Every one of them is a chance for one of these:
- A blank page. The HTML arrives, but the JavaScript app crashes before it draws anything. The server answered 200, so the host is happy. The visitor sees white.
- Missing chunks. The new HTML points to script files from the new build. If a CDN node still serves old HTML, or a file failed to upload, part of the app quietly stops working.
- A secret in the bundle. An environment variable with a
VITE_orNEXT_PUBLIC_prefix is copied into the public JavaScript at build time. One rename by an AI agent and a private key is now readable by anyone. - The wrong page. A redirect, a login wall or a hosting error page that also returns 200.
Uptime monitors ping the address and check the status code. All four problems above can pass that test. If you want the background on two of them, see why a React SPA can serve empty HTML and how VITE_ variables end up in the build.
Check it yourself after a deploy
This takes about two minutes. Do it once by hand, so you know what the automated version should do.
- Open the site in a private window. No cache, no logged-in session. Does the main content show up, not just a header and an empty box?
- Open the browser console (F12 or Cmd+Option+I, the Console tab). Red errors on page load usually mean a crash or a missing file.
- Open the Network tab and reload. Filter by JS. Any script with a 404 status is a missing chunk.
- Look for one phrase you know should be there. Your headline, your pricing, the word "Sign up". If it is gone, the wrong page or an empty shell is being served.
- Search the scripts for secrets. In the terminal:
curl -s https://yourdomain.com/ | grep -oE 'src="[^"]+\.js"'
curl -s https://yourdomain.com/assets/index-abc123.js \
| grep -oE 'sk_live_[A-Za-z0-9]{10,}|sk-[A-Za-z0-9_-]{20,}|service_role' | head
A match does not prove a leak, but it is worth a look. The guide to finding API keys in a JavaScript bundle explains what each pattern means.
Automate it: a small smoke test after each deploy
A smoke test is a short script that checks the obvious things on the live site: is it up, does it show the expected text, do its scripts load. It is not a full test suite. It just answers "did this deploy break something big?"
Here is a version that needs only bash and curl. Save it as scripts/smoke.sh:
#!/usr/bin/env bash
set -euo pipefail
URL="${1:?usage: smoke.sh https://yourdomain.com}"
MUST_CONTAIN="${MUST_CONTAIN:-Sign up}"
html=$(curl -fsS -L --max-time 20 "$URL") || { echo "FAIL: $URL did not answer"; exit 1; }
echo "$html" | grep -q "$MUST_CONTAIN" \
|| { echo "FAIL: '$MUST_CONTAIN' not found on the page"; exit 1; }
for src in $(echo "$html" | grep -oE 'src="[^"]+\.js"' | sed -E 's/src="([^"]+)"/\1/'); do
case "$src" in http*) asset="$src" ;; *) asset="${URL%/}/${src#/}" ;; esac
body=$(curl -fsS --max-time 20 "$asset") || { echo "FAIL: script missing: $asset"; exit 1; }
if echo "$body" | grep -qE 'sk_live_[A-Za-z0-9]{10,}|service_role'; then
echo "FAIL: possible secret in $asset"; exit 1
fi
done
echo "OK: $URL"
Two honest limits. First, curl does not run JavaScript, so a single-page app that renders everything in the browser may not have your phrase in the raw HTML. For those, put the phrase check in a headless browser test with Playwright, which loads the page like a real visitor. Second, the secret patterns are examples, not a complete list. Add the prefixes of the services you actually use.
Run it from CI when the deploy is done
The tricky part is timing: the script must run after the new version is live, not after the build starts.
- Vercel with GitHub. Vercel reports each deployment back to GitHub, and a GitHub Actions workflow can run on the
deployment_statusevent. Vercel's own guide shows this for running tests after a deployment. - Any host with a GitHub Actions deploy step. Add the smoke test as the last step of the same job.
- Hosts that deploy without CI (Lovable, Bolt, Replit). There is no pipeline to hook into, so you need something that watches the site from the outside.
A minimal workflow for the first case, saved as .github/workflows/smoke.yml:
name: Smoke test after deploy
on: deployment_status
jobs:
smoke:
if: github.event.deployment_status.state == 'success'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: bash scripts/smoke.sh "${{ github.event.deployment_status.environment_url }}"
The deployment_status event and its state field are described in GitHub's list of workflow events. Some providers leave environment_url empty, so check it is set before relying on it.
Let your AI tool set it up
If your project lives in Cursor or Claude Code, paste this:
Add a post-deploy smoke test to this project.
1. Create scripts/smoke.sh that takes the site URL as an argument and fails
(exit 1) if: the page does not return 200, the phrase in $MUST_CONTAIN is
missing, any <script src> on the page returns an error, or a script
contains a secret-looking string (sk_live_, sk-, service_role, and the
prefixes of the services this project uses).
2. If the app renders on the client, add a Playwright test that opens the
page, waits for it to settle, and fails on console errors or an empty
main element.
3. Add a GitHub Actions workflow that runs these checks on deployment_status
when state is success, using environment_url as the URL.
4. Do not print secret values in the logs, only the file name and pattern.
Explain how to change the expected phrase.
In Lovable or Bolt, where you do not control CI, ask instead: "Before every publish, list any environment variables that will be included in the browser bundle and tell me if any of them look like private keys."
How to keep it from coming back
A smoke test only helps if it runs every time and someone sees the result. Three habits make that true:
- Fail loudly. A failed check should notify you in a place you read: a chat message, not a log nobody opens.
- Keep one phrase up to date. When you change your headline, change
MUST_CONTAIN. An outdated phrase trains you to ignore the alarm. - Cover deploys you did not trigger. An AI agent, a teammate or a dependency update can ship a version while you are away.
That last point is what ScanYourSite's deploy check is built for. On Pro it notices a new version from the outside, by fingerprinting the page's scripts and build identifiers, so it works on any host, with or without CI. It confirms the change, then checks the new version: that the page answers and renders, JavaScript errors, scripts that fail to load, secret keys in the new bundle, and, if you set one, a phrase that must be on the page. It writes to Telegram, email, Slack or Discord only when a deploy breaks something. If you deploy from CI, an optional deploy hook (curl -fsS -X POST "$SCANYOURSITE_DEPLOY_HOOK") starts the check immediately. It never rolls anything back; you decide what to do.
To see what it finds on your current version, run a free check: https://scanyoursite.app/?check=yourdomain.com.