curl in Practice: Check Your Website After Deployment

Your file transfer finished, but did the new page actually reach readers? A successful upload can still leave you with a wrong URL, an old cached page, or a server error.

After syncing files with rsync, use this short curl workflow to inspect the public response. You will save the page, check its status and content, and create a script that fails when the result is unexpected.

1. Check the installed tools

These examples use a macOS or Linux terminal and a POSIX-compatible shell:

curl --version
curl --help

If curl is missing on Debian or Ubuntu, install it:

sudo apt update
sudo apt install curl

The commands below were tested with curl 8.7.1. The optional --fail-with-body example needs curl 7.76.0 or newer; see the curl option reference.

Keep the same terminal open for steps 2–4 so the variables remain available.

2. Save the page and print a small report

Start with an existing public article. Replace page_url with your own deployed URL when applying the workflow:

check_dir=$(mktemp -d)
page_url='https://www.bobobk.com/en/rsync-preview-sync-workflow.html'

curl -sS -L --max-redirs 5 \
  --connect-timeout 10 --max-time 30 \
  -D "$check_dir/headers.txt" \
  -o "$check_dir/page.html" \
  -w 'status=%{http_code}\nfinal_url=%{url_effective}\nredirects=%{num_redirects}\ntotal_seconds=%{time_total}\n' \
  "$page_url"

-sS hides the progress meter while retaining error messages. -D saves response headers and -o saves the body, keeping the report readable. The -w fields print the final HTTP status, final URL, redirect count, and total transfer time; see curl’s write-out guide.

For this article, look for status=200 and the expected final URL. The time value will vary. A response containing 200 is only the first check: the saved HTML must also be the page you intended to publish.

The two timeouts serve different purposes: connection setup gets up to 10 seconds, while the whole transfer gets up to 30 seconds. The connection budget is included in the overall limit. See curl’s timeout guide.

3. Inspect redirects and the downloaded content

Review the saved headers:

grep -Ei '^(HTTP/|location:|content-type:|cache-control:|age:)' \
  "$check_dir/headers.txt"

With redirects, the file can contain several response-header blocks. -L follows HTTP redirects, and --max-redirs 5 caps this exercise at five hops. Review Location and final_url for an unexpected hostname or login page. Curl does not execute JavaScript redirects. See the redirect guide.

Now check a distinctive phrase in the page:

if grep -Fq 'Rsync in Practice' "$check_dir/page.html"; then
  printf 'Expected article text found\n'
else
  printf 'Expected article text missing\n' >&2
fi

For a new deployment, choose a phrase introduced in that update. A title that existed yesterday cannot prove today’s changes arrived. If a CDN is involved, the headers may help explain an old response, but the absence of Age does not prove there is no cache.

This workflow uses a GET request so it can examine the body. curl -I sends HEAD instead; it is useful for a quick header check, but it does not download the article. See the HEAD option.

4. Make HTTP errors visible to the shell

By default, receiving an HTTP error page can still produce a successful curl exit status. To preserve the error body while failing on HTTP errors, try:

if curl -sS -L --max-redirs 5 --fail-with-body \
  --connect-timeout 10 --max-time 30 \
  -o "$check_dir/response.html" "$page_url"; then
  printf 'Transfer completed without a curl-reported error\n'
else
  curl_status=$?
  printf 'curl failed: exit=%s\n' "$curl_status" >&2
fi

For an ordinary 404 or 500 response, this option produces exit code 22; a connection failure has a different code. The successful branch still does not prove that the expected article was returned. Curl’s response guide explains the difference between HTTP status and transfer success.

On an older curl, --fail is an alternative when you do not need the error body. The next script instead checks the HTTP status explicitly and does not require --fail-with-body.

5. Create a reusable deployment check

Save this as check-page.sh in your project:

#!/bin/sh
set -eu

if [ "$#" -ne 2 ] || [ -z "$2" ]; then
  printf 'Usage: sh check-page.sh URL EXPECTED_TEXT\n' >&2
  exit 2
fi

page_url=$1
expected_text=$2
body_file=$(mktemp)
trap 'rm -f "$body_file"' EXIT
trap 'exit 1' HUP INT TERM

if http_status=$(curl -sS -L --max-redirs 5 \
  --connect-timeout 10 --max-time 30 \
  -o "$body_file" -w '%{http_code}' "$page_url"); then
  :
else
  curl_status=$?
  printf 'Transfer failed: curl exit=%s\n' "$curl_status" >&2
  exit 1
fi

if [ "$http_status" != '200' ]; then
  printf 'Unexpected HTTP status: %s\n' "$http_status" >&2
  exit 1
fi

if ! grep -Fq -- "$expected_text" "$body_file"; then
  printf 'Expected text missing: %s\n' "$expected_text" >&2
  exit 1
fi

printf 'OK: HTTP 200 and expected text found\n'

Run it against the sample article:

sh check-page.sh \
  'https://www.bobobk.com/en/rsync-preview-sync-workflow.html' \
  'Rsync in Practice'

Expected output:

OK: HTTP 200 and expected text found

Then use a phrase that should be absent:

sh check-page.sh \
  'https://www.bobobk.com/en/rsync-preview-sync-workflow.html' \
  'THIS_MARKER_SHOULD_NOT_EXIST_8e71'

This run should fail with Expected text missing. Testing the failing case helps catch checks that always report success. The script deliberately requires HTTP 200 for a public HTML page; adjust that condition if your endpoint legitimately returns another status.

Troubleshooting

Curl checks the response bytes. It does not validate page layout or run your application’s JavaScript, so also open the page in a browser when the deployment changes those features.

Make the check part of publishing

Run the check after publishing, using the exact public URL and a phrase unique to the new content. Keep the header report for investigating failures, and use the script’s exit status to stop later automation when validation fails. Once the command is stable, add it to your project’s just workflow.