Check ZIP Backups with Python: CRC, Duplicate Members, and Missing Files

A backup script exits successfully and backup.zip exists, yet a restore can still fail: interrupted copying can leave a truncated archive, and incorrect packaging arguments can omit configuration files. This beginner-to-intermediate tutorial builds a read-only checker that inspects ZIP members, fully reads them to check CRC, and uses its exit code to gate the next task.

1. Method: check both the member list and the full contents

The existing zipfile basics tutorial covers reading, writing, and extraction. This article adds acceptance checks for completed local backups that stay unchanged during validation: reject duplicate member names, require specific files, and then call testzip().

The official Python testzip documentation says this method reads archive files and checks CRC and file headers, returning the first bad member name or None. Listing members alone does not perform that check.

Our policy accepts only unencrypted ZIP_STORED and ZIP_DEFLATED archives and rejects archives with no files. CRC helps detect accidental corruption; compare against trusted originals with a SHA-256 file manifest when needed. This checker does not validate JSON or CSV business content or guarantee safe paths for later extraction.

2. Prepare the environment and demo backup

Use Python 3.9 or newer with a working zlib module; no third-party packages are required. These examples were tested on Linux with Python 3.11.4, using bash or zsh.

Create a fresh demo directory and run subsequent commands in the same terminal and directory. Choose a different directory name when repeating the experiment:

python3 --version
mkdir zip-backup-demo
cd zip-backup-demo
python3 - <<'PY_DATA'
from zipfile import ZipFile, ZIP_DEFLATED
with ZipFile("backup.zip", "x", compression=ZIP_DEFLATED) as archive:
    archive.writestr("config/task.json", '{"enabled":true}\n')
    archive.writestr("reports/daily.csv", "id,value\n1,42\n")
PY_DATA

Save the next section as check_zip.py, or download the complete checker. The example creates ZIP files in x mode, which fails if the archive already exists to avoid overwriting it.

3. Complete implementation

"""Check a completed local ZIP backup without extraction. Python 3.9+."""
import argparse
import json
from pathlib import Path
import sys
import zipfile
import zlib


def check_zip(path, required):
    with zipfile.ZipFile(path, "r") as archive:
        seen = set()
        files = set()
        total = 0
        for info in archive.infolist():
            name = info.filename
            if name in seen:
                raise ValueError(f"duplicate member: {name!r}")
            seen.add(name)
            if info.flag_bits & 1:
                raise ValueError(f"encrypted member is unsupported: {name!r}")
            if info.compress_type not in (zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED):
                raise ValueError(f"unsupported compression: {name!r}")
            if not info.is_dir():
                files.add(name)
                total += info.file_size
        if not files:
            raise ValueError("archive contains no files")
        missing = sorted(set(required) - files)
        if missing:
            raise ValueError(f"missing required members: {missing!r}")
        bad = archive.testzip()
        if bad is not None:
            raise ValueError(f"CRC/header check failed: {bad!r}")
    return {
        "file": str(path), "files": len(files),
        "uncompressed_bytes": total, "valid": True,
    }


def main():
    parser = argparse.ArgumentParser(description=__doc__, allow_abbrev=False)
    parser.add_argument("file", type=Path, help="completed, unchanging ZIP backup")
    parser.add_argument(
        "--require", action="append", default=[], metavar="MEMBER",
        help="exact non-directory member name; repeat for multiple files",
    )
    args = parser.parse_args()
    if any(not name or name.endswith("/") for name in args.require):
        parser.error("--require needs a nonempty file member name, without trailing /")
    try:
        result = check_zip(args.file, args.require)
    except (OSError, ValueError, EOFError, RuntimeError,
            NotImplementedError, zipfile.BadZipFile, zlib.error) as exc:
        print(f"ERROR: {args.file}: {exc}", file=sys.stderr)
        return 1
    print(json.dumps(result, ensure_ascii=False))
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

infolist() returns member information. ZIP archives can contain duplicate names, so reject them before reading contents to avoid ambiguity about which entry is read. --require matches the full archive member name exactly and case-sensitively, using / as the directory separator. Directory entries cannot satisfy file requirements.

The program never extracts files onto disk, but it decompresses streams for verification, so runtime grows with content size. Directory information also uses memory. It is intended for your own trusted backups and has no resource quotas or hostile-archive isolation; do not use it as a security checker for public uploads.

4. Run the checker and verify its report

View help and require both files:

python3 check_zip.py --help
python3 check_zip.py backup.zip \
  --require config/task.json --require reports/daily.csv > report.json
cat report.json

Expected output:

{"file": "backup.zip", "files": 2, "uncompressed_bytes": 31, "valid": true}

uncompressed_bytes is the sum of declared uncompressed sizes for non-directory members, not the ZIP file’s disk usage. The report appears only after every check passes.

Check the report and confirm no files were extracted into the current directory:

python3 - <<'PY_CHECK'
import json
from pathlib import Path
report = json.loads(Path("report.json").read_text(encoding="utf-8"))
assert report == {
    "file": "backup.zip", "files": 2,
    "uncompressed_bytes": 31, "valid": True,
}
assert not Path("config").exists()
assert not Path("reports").exists()
print("PASS: two members checked; no files extracted")
PY_CHECK

Operational failures return 1 and write stderr; argument errors return 2, following argparse defaults. Success returns 0. Redirection creates or truncates report.json before execution, so use a fresh report path.

5. Test truncation, CRC corruption, and missing files

Create failure fixtures only inside the demo directory, preserving the original backup.zip:

python3 - <<'PY_BAD'
from pathlib import Path
from zipfile import ZipFile, ZIP_STORED
import warnings

Path("truncated.zip").write_bytes(Path("backup.zip").read_bytes()[:20])
with warnings.catch_warnings():
    warnings.simplefilter("ignore", UserWarning)
    with ZipFile("duplicate.zip", "x") as archive:
        archive.writestr("same.txt", "first")
        archive.writestr("same.txt", "second")
with ZipFile("crc-bad.zip", "x", compression=ZIP_STORED) as archive:
    archive.writestr("note.txt", b"unique-demo-payload")
raw = Path("crc-bad.zip").read_bytes()
assert raw.count(b"unique-demo-payload") == 1
Path("crc-bad.zip").write_bytes(raw.replace(
    b"unique-demo-payload", b"Unique-demo-payload", 1
))
PY_BAD

crc-bad.zip stores its member without compression. Replacing one content byte with the same length keeps the member list readable while making the stored CRC disagree with the actual contents.

Verify failure exits, empty stdout, and unchanged inputs:

python3 - <<'PY_TEST'
from pathlib import Path
import subprocess
import sys

cases = [
    (["truncated.zip"], "File is not a zip file"),
    (["duplicate.zip"], "duplicate member"),
    (["crc-bad.zip"], "CRC/header check failed"),
    (["backup.zip", "--require", "missing.csv"], "missing required members"),
]
for args, expected in cases:
    path = Path(args[0])
    before = path.read_bytes()
    result = subprocess.run(
        [sys.executable, "check_zip.py", *args],
        capture_output=True, text=True, check=False,
    )
    assert result.returncode == 1, result
    assert result.stdout == "", result.stdout
    assert expected in result.stderr, result.stderr
    assert path.read_bytes() == before
print("PASS: four failures rejected; archives unchanged")
PY_TEST

Expected output: PASS: four failures rejected; archives unchanged. None of these four failures emits a successful JSON report. Duplicate and missing members are rejected before content reading.

6. Use it in automation and troubleshoot failures

Continue only when the checker succeeds. A single print statement stands in for the next task:

if python3 check_zip.py backup.zip --require config/task.json; then
  python3 -c 'print("NEXT: backup accepted")'
else
  printf 'STOP: backup validation failed\n' >&2
  exit 1
fi

Changing the input to crc-bad.zip fails validation and blocks the next output. Real tasks must use the same unchanged backup. Start acceptance checks after publishing or copying finishes, rather than while the archive is being written.

Symptom	Concrete action
`File is not a zip file`	Check that the download is not an HTML error page; copy or generate the complete backup again.
`CRC/header check failed`	Keep the damaged file for investigation; regenerate or transfer from a trusted source and rerun the check.
`duplicate member`	Look for repeated append-mode writes; rebuild at a new path and write each member once.
`missing required members`	Check archive paths and case, such as `config/task.json`; do not supply server absolute paths.
`encrypted member` / `unsupported compression`	Produce unencrypted STORED/DEFLATED backups under this policy, or use a checker that supports the original format.
`archive contains no files`	Check the source directory and packaging filters; directory entries alone are not files.

For a single .gz log backup, use the gzip truncation and CRC tutorial rather than sending gzip input to a ZIP checker.

Summary

Check required members and duplicate names, then fully read the ZIP to check CRC before reporting success. Exercise failures with damaged copies so automation actually stops at the validation step. Important backups also need trusted hashes, content validation, and restore drills.