Compare Configuration Files with Python: Unified Diffs, Line Endings, and Exit Codes
On this page
An automation task changes workers = 2 to workers = 4. The file still opens, but the next step should not blindly continue. Before deploying configuration, adjusting Hugo settings, or checking batch inputs, review which lines actually changed rather than trusting modification times.
This beginner-to-intermediate tutorial builds a read-only configuration comparison tool with Python’s standard library. It prints a unified diff, distinguishes identical text from differences and failures, and never modifies the inputs. It compares text; it does not validate TOML, YAML, or application settings.
1. Method: define the comparison rules
- Use small, regular UTF-8 files in a trusted local directory, with no active writers. Each file is limited to 256 KiB and 2000 lines.
- Normalize CRLF and bare CR to LF, then compare lines. Spaces, comments, ordering, and final newlines remain significant.
- Identical text produces no output. Differences go to stdout; read and argument errors go to stderr.
- Review only: do not overwrite configuration or automatically apply the diff.
The official difflib.unified_diff() documentation describes unified diffs with context; n selects the context line count. Fixed before and after labels keep full paths out of the report.
Exit 0 means identical text under these rules, not identical bytes. See CRLF and BOM troubleshooting when raw script line endings matter. Comments and whitespace still count, so semantically equivalent configurations can differ.
2. Setup: create an isolated directory
Use Python 3.9 or newer, with no third-party packages. These examples were tested on Linux with Python 3.11.4; shell commands work in bash and zsh.
python3 --version
mkdir config-diff-demo
cd config-diff-demo
Save the next section as compare_config.py in the current directory, or download the identical complete script. Choose a new directory when repeating the demo: the following printf > commands create fixtures and overwrite matching names.
3. Implementation: validate both inputs before producing a diff
"""Compare small UTF-8 config snapshots without modifying them. Python 3.9+."""
import argparse
import difflib
import io
from pathlib import Path
import sys
MAX_BYTES = 256 * 1024
MAX_LINES = 2000
def context_lines(value):
number = int(value)
if not 0 <= number <= 20:
raise argparse.ArgumentTypeError("context must be between 0 and 20")
return number
def read_lines(path):
if not path.is_file():
raise ValueError(f"not a regular file: {path}")
with path.open("rb") as stream:
raw = stream.read(MAX_BYTES + 1)
if len(raw) > MAX_BYTES:
raise ValueError(f"file exceeds {MAX_BYTES} bytes: {path}")
text = raw.decode("utf-8", errors="strict")
if text.startswith("\ufeff"):
raise ValueError(f"UTF-8 BOM found: {path}")
if "\x00" in text:
raise ValueError(f"NUL byte found: {path}")
text = text.replace("\r\n", "\n").replace("\r", "\n")
lines = io.StringIO(text).readlines()
if len(lines) > MAX_LINES:
raise ValueError(f"file exceeds {MAX_LINES} lines: {path}")
return lines
def render_diff(before, after, context):
parts = []
for line in difflib.unified_diff(
before, after, fromfile="before", tofile="after", n=context,
):
parts.append(line)
if not line.endswith("\n"):
parts.append("\n\\ No newline at end of file\n")
return "".join(parts)
def main():
parser = argparse.ArgumentParser(description=__doc__, allow_abbrev=False)
parser.add_argument("before", type=Path)
parser.add_argument("after", type=Path)
parser.add_argument("--context", type=context_lines, default=3)
args = parser.parse_args()
try:
before = read_lines(args.before)
after = read_lines(args.after)
if before == after:
return 0
report = render_diff(before, after, args.context)
except (OSError, ValueError) as exc:
print(f"ERROR: {exc}", file=sys.stderr)
return 2
sys.stdout.write(report)
return 1
if __name__ == "__main__":
raise SystemExit(main())
The open() documentation distinguishes binary and text modes. This tool limits bytes before strict UTF-8 decoding; it does not discard bad data with errors="ignore". A leading UTF-8 BOM or a NUL byte is rejected, without guessing encodings or repairing input.
The script adds \ No newline at end of file after a diff line missing its final newline; that marker is not configuration content. The byte and line limits keep the demo small, not time-bounded: repetitive lines can still slow comparison. Symlinks are followed. The example does not defend against hostile paths, concurrent replacements, or stalled network filesystems.
4. Usage: understand the diff and exit codes
- Check help, create two small configurations, and compare them:
python3 compare_config.py --help
printf 'workers = 2\n' > before.toml
printf 'workers = 4\n' > after.toml
if python3 compare_config.py before.toml after.toml; then
rc=0
else
rc=$?
fi
printf 'exit=%s\n' "$rc"
The output is below. Capturing the status inside if also preserves the expected difference branch in shells using set -e:
--- before
+++ after
@@ -1 +1 @@
-workers = 2
+workers = 4
exit=1
--- and +++ label the before and after files; @@ gives line ranges. - marks old content and + marks new content. The final exit=1 comes from the shell, not the diff.
| Exit code | Meaning | Stdout |
|---|---|---|
| 0 | Identical after line-ending normalization | Empty |
| 1 | Different text requiring review | Unified diff |
| 2 | Read, encoding, size, or argument error | Empty for input-check failures |
--help prints help and exits 0. Invalid arguments are reported by argparse with exit 2. allow_abbrev=False rejects abbreviations such as --cont instead of accidentally accepting a mistyped option.
- Distinguish differences from errors in an existing workflow:
if python3 compare_config.py before.toml after.toml; then
printf 'UNCHANGED: next step is allowed\n'
else
rc=$?
case "$rc" in
1) printf 'CHANGED: review the diff first\n' >&2 ;;
*) printf 'ERROR: comparison failed (exit=%s)\n' "$rc" >&2 ;;
esac
fi
This is an interactive branching demo, not a complete CI gate. Add an explicit exit "$rc" to the failure branch when a script must stop the pipeline. Do not hide errors with || true. Configuration validity remains a separate step, such as validating TOML with tomllib. For directory synchronization, continue with previewing rsync before execution.
5. Validation: line endings, encoding, and unchanged inputs
Run this self-check in the same directory. It creates only its own temporary workspace, permits nonzero child statuses, and verifies that both input SHA-256 hashes remain unchanged:
python3 - <<'PY_CHECK'
import hashlib
from pathlib import Path
import subprocess
import sys
import tempfile
with tempfile.TemporaryDirectory(prefix="diff-checks-", dir=".") as folder:
root = Path(folder)
cases = [
("same", b"x = 1\n", b"x = 1\n", 0),
("changed", b"x = 1\n", b"x = 2\n", 1),
("crlf", b"x = 1\r\n", b"x = 1\n", 0),
("no-final-newline", b"x = 1", b"x = 1\n", 1),
("bad-utf8", b"\xff", b"x = 1\n", 2),
("bom", b"\xef\xbb\xbfx = 1\n", b"x = 1\n", 2),
("too-large", b"x" * (256 * 1024 + 1), b"x = 1\n", 2),
]
for label, left, right, expected in cases:
before, after = root / "before.toml", root / "after.toml"
before.write_bytes(left)
after.write_bytes(right)
original = [hashlib.sha256(p.read_bytes()).digest()
for p in (before, after)]
result = subprocess.run(
[sys.executable, "compare_config.py", str(before), str(after)],
capture_output=True, text=True, timeout=10,
)
assert result.returncode == expected, (label, result)
assert original == [hashlib.sha256(p.read_bytes()).digest()
for p in (before, after)]
if expected == 0:
assert result.stdout == result.stderr == ""
elif expected == 1:
assert result.stdout.startswith("--- before\n+++ after\n")
assert result.stderr == ""
else:
assert result.stdout == ""
assert result.stderr.startswith("ERROR:")
if label == "no-final-newline":
assert "\\ No newline at end of file" in result.stdout
print(f"{label}: exit={result.returncode}")
print("PASS: 7 cases; inputs unchanged")
PY_CHECK
Expected output:
same: exit=0
changed: exit=1
crlf: exit=0
no-final-newline: exit=1
bad-utf8: exit=2
bom: exit=2
too-large: exit=2
PASS: 7 cases; inputs unchanged
The self-check’s timeout=10 limits the test subprocess; it is not a built-in timeout in the comparison tool. Test the intended rules on sanitized copies before using real configuration.
6. Troubleshooting: separate bad input from genuine differences
| Symptom | Cause and action |
|---|---|
ERROR: includes a UnicodeDecodeError message |
Input is not valid UTF-8. Confirm its source encoding and convert a copy; do not ignore bad bytes. |
UTF-8 BOM found |
Confirm and remove the BOM on a copy before comparing. |
not a regular file |
A file is missing, or a directory or other non-regular input was supplied. Check the working directory and input paths. |
file exceeds ... |
The demo’s byte or line cap was exceeded. Reduce the inputs to relevant configuration sections instead of comparing huge logs. |
| Looks identical but exits 1 | Check spaces, blank lines, comments, and the final newline; these remain significant. |
| Windows and Linux copies exit 0 | CRLF/LF may be the only difference, normalized by agreement. This is not byte verification. |
The diff includes real configuration values. Do not send secrets, tokens, or private addresses to public logs. This is a review tool, not a byte-preserving patch generator. Do not apply unknown configuration directly to a server.
7. Summary
Define encoding and line-ending rules before using a unified diff to locate changes. Treat exit 1 as a review request and exit 2 as a failed check, so automation does not confuse them. Keep the original files and run configuration parsing and application checks separately after text review.
- Author: yanzhi
- Original article: https://www.bobobk.com/en/python-difflib-config-diff.html
- License: This work is licensed under CC BY-NC-ND 4.0. See the license for attribution, noncommercial use and no-derivatives terms.
Related articles
- Validate Python Environment Variables: Booleans, Empty Values, and Integers
- Wait for a File in Python: Timeouts, Atomic Handoffs, and Exit Codes
- Python File Permissions on Linux: umask, Mode 600, and No Overwrites
- Clean Up Old Logs with Python: Preview First, Skip Links and Subdirectories
- Merge Sorted Logs by Timestamp in Python with heapq.merge