Python Output Path Checks: Containment with pathlib and No Accidental Overwrites
On this page
A job specifies exports/ as its output directory, yet a report lands beside it. root / name joins paths; it does not constrain writes. Absolute paths, .., and symbolic links pointing elsewhere need explicit handling.
This beginner-to-intermediate tutorial adds an output-path preflight to a local Python export script and tests that rejected requests create no outside files. You must control the directory tree, with no directory or link replacement between checking and writing. This is not a sandbox against hostile concurrent changes or a file-upload handler.
1. Method: check resolved paths, not string prefixes
Start with a counterexample that performs no file I/O:
python3 - <<'PYCODE'
from pathlib import Path
root = Path("/tmp/exports")
other = Path("/tmp/exports-old/report.txt")
print(str(other).startswith(str(root)))
try:
other.relative_to(root)
except ValueError:
print("Not a child directory")
PYCODE
It prints True and Not a child directory. exports-old shares a string prefix with exports but is not its child.
Our preflight policy is:
- Require an explicitly selected, existing output root and resolve its real location.
- Require a relative filename and reject every
..component, even if normalization would keep it inside. - Resolve existing symlinks before checking containment; allow links whose resolved destination remains inside.
- Require an existing parent directory and create exclusively, without overwriting files.
The Python pathlib documentation explains symlink resolution; strict=False permits a missing tail. Then relative_to() checks containment by path components instead of startswith().
2. Setup: create an isolated demo directory
Use an existing Python 3.9+ interpreter and a Linux shell. No third-party packages are needed. Run these commands in one terminal; choose a fresh directory name when repeating the exercise:
python3 --version
mkdir output-path-demo
cd output-path-demo
mkdir -p exports/daily outside
ln -s ../outside exports/escape
exports/escape deliberately points to the sibling outside/ directory. Do not experiment in system directories or production exports. For missing link targets, see checking broken symlinks with Python to distinguish a link from its destination.
3. Implementation: preflight, then create exclusively
Save this complete code as safe_output.py in the current directory:
import argparse
import json
from pathlib import Path
import sys
def output_path(root_name, name):
root = Path(root_name).resolve(strict=True)
if not root.is_dir():
raise ValueError("output root must be a directory")
relative = Path(name)
if not name or relative == Path("."):
raise ValueError("a file name is required")
if relative.is_absolute() or ".." in relative.parts:
raise ValueError("use a relative path without '..'")
target = (root / relative).resolve(strict=False)
try:
local = target.relative_to(root)
except ValueError:
raise ValueError("resolved path leaves output root") from None
if local == Path("."):
raise ValueError("target must not be the output root")
if not target.parent.is_dir():
raise ValueError("output parent directory must already exist")
return root, target
def main():
parser = argparse.ArgumentParser(
description="Create a demo file inside a trusted Linux output tree."
)
parser.add_argument("root", help="existing output directory")
parser.add_argument("name", help="relative output filename")
args = parser.parse_args()
try:
root, target = output_path(args.root, args.name)
with target.open("x", encoding="utf-8", newline="\n") as stream:
stream.write("demo output\n")
except (OSError, ValueError, RuntimeError) as exc:
print(f"ERROR: {exc}", file=sys.stderr)
return 1
print(json.dumps({"status": "created", "path": target.relative_to(root).as_posix()}))
return 0
if __name__ == "__main__":
raise SystemExit(main())
The demo writes fixed demo output content so its behavior is easy to verify. Replace the writing section with your export logic while preserving validation and error handling. Names use local Linux path semantics: no URL decoding or Windows backslash interpretation. Do not reuse this unchanged for cross-platform archive extraction or network uploads.
The official open() reference defines x as failing when a file exists, avoiding truncation by w. Success returns 0; path or write errors return 1; argparse returns 2 for missing arguments. A write failure may leave an incomplete new file. For atomic delivery, add a temporary-file and atomic-replacement approach; this demo is not a transactional writer.
4. Usage: verify accepted writes and rejected escapes
- Inspect help and write a filename containing spaces:
python3 safe_output.py --help
python3 safe_output.py exports 'daily/report 01.txt'
cat 'exports/daily/report 01.txt'
The last two commands print:
{"status": "created", "path": "daily/report 01.txt"}
demo output
Repeating the same write must return 1 because the file exists. Do not switch to w merely to make a repeat run pass.
- Exercise three rejection branches:
for name in '../outside/no.txt' '/tmp/no.txt' 'escape/no.txt'; do
if python3 safe_output.py exports "$name"; then
printf 'Unexpected success: %s\n' "$name"
else
printf 'Rejected as expected: %s\n' "$name"
fi
done
All three print ERROR: to stderr followed by Rejected as expected; outside/no.txt must not be created. The if handles expected failures so a shell using set -e can continue.
- Pin the output root to an absolute path for automation:
python3 safe_output.py "$PWD/exports" 'daily/from-job.txt'
Run this command from the demo directory; it illustrates the absolute-root argument. Jobs launched elsewhere must pass that same absolute root and an absolute script path. See Linux job interpreter and working-directory troubleshooting. This tutorial does not modify a scheduler.
5. Automated tests: preserve outside files and existing output
Save this as verify_output.py beside safe_output.py. It creates fixtures in its own temporary directory, cleans up automatically, and limits each subprocess to ten seconds:
import json
from pathlib import Path
import subprocess
import sys
import tempfile
script = Path(__file__).with_name("safe_output.py").resolve()
def run(args, cwd):
return subprocess.run(
[sys.executable, str(script), *args], cwd=cwd,
capture_output=True, text=True, timeout=10, check=False,
)
with tempfile.TemporaryDirectory(prefix="output-path-test-") as tmp:
base = Path(tmp).resolve()
root = base / "exports"
(root / "daily").mkdir(parents=True)
outside = base / "exports-old"
outside.mkdir()
sentinel = outside / "keep.txt"
sentinel.write_bytes(b"unchanged\n")
(root / "escape").symlink_to(outside, target_is_directory=True)
(root / "dangling").symlink_to(outside / "new.txt")
(root / "alias").symlink_to(root / "daily", target_is_directory=True)
(root / "loop").symlink_to("loop")
def call(name):
return run([str(root), name], base)
good = call("daily/report 01.txt")
assert (good.returncode, good.stderr) == (0, ""), good
assert json.loads(good.stdout) == {"status": "created", "path": "daily/report 01.txt"}
saved = root / "daily/report 01.txt"
assert saved.read_bytes() == b"demo output\n"
alias = call("alias/second.txt")
assert alias.returncode == 0 and not alias.stderr, alias
assert json.loads(alias.stdout)["path"] == "daily/second.txt"
bad_names = [
"", ".", "daily/..", "../exports-old/new.txt", str(sentinel),
"escape/keep.txt", "escape/new.txt", "dangling", "loop/new.txt",
"missing/new.txt", "daily/report 01.txt", "daily",
]
for name in bad_names:
result = call(name)
assert result.returncode == 1 and not result.stdout, (name, result)
assert result.stderr.startswith("ERROR:"), (name, result)
for bad_root in [base / "absent", sentinel]:
result = run([str(bad_root), "new.txt"], base)
assert result.returncode == 1 and result.stderr.startswith("ERROR:"), result
help_result = run(["--help"], base)
assert help_result.returncode == 0 and "trusted Linux" in help_result.stdout
missing = run([], base)
assert missing.returncode == 2 and "required" in missing.stderr
assert sentinel.read_bytes() == b"unchanged\n"
assert not (outside / "new.txt").exists()
assert saved.read_bytes() == b"demo output\n"
assert sorted(p.name for p in (root / "daily").iterdir()) == ["report 01.txt", "second.txt"]
print("OK: contained writes, aliases, traversal, symlinks, loops, existing files, roots, CLI")
python3 verify_output.py
A passing run prints:
OK: contained writes, aliases, traversal, symlinks, loops, existing files, roots, CLI
Tests cover accepted writes, internal aliases, absolute paths, .., external links, dangling links, link loops, missing parents, overwrite refusal, invalid roots, help, and missing arguments. They also assert unchanged bytes for the outside sentinel and existing output. They do not simulate check/use races or disk failures.
6. Troubleshooting and limits
| Symptom | Action |
|---|---|
use a relative path without '..' |
Pass daily/report.txt, without a root prefix, absolute path, or parent traversal. |
resolved path leaves output root |
Inspect links with readlink and fix the export configuration; keep the containment check. |
output parent directory must already exist |
Explicitly run mkdir -p exports/daily inside the trusted root instead of creating directories from unchecked names. |
File exists |
Inspect previous output and choose a new name. Replacement requires a separate, explicit business policy. |
Permission denied |
Check permissions as the actual job account; do not hide the issue with chmod 777 or administrator execution. |
| Link loop or missing root | Fix the link or create the trusted root first. Error wording can vary between Python versions. |
A time window remains between resolve() and open() in which another process can replace directories. Multi-user writable trees, hostile inputs, and uploads require a different design, such as component-by-component traversal using directory file descriptors with controlled link following. The Python os file-descriptor parameter documentation describes underlying interfaces; one flag on the final component does not automatically protect every ancestor. This tutorial does not claim to solve that problem.
7. Summary
An output root is not an automatic boundary: reject absolute names and .., resolve links, and check component containment. In a tree you control without concurrent replacement, this catches common export-path configuration errors; exclusive creation and unchanged-file assertions reduce accidental writes and overwrites. For hostile concurrent changes, use purpose-built isolation and file-descriptor designs rather than more string checks.
- Author: yanzhi
- Original article: https://www.bobobk.com/en/python-output-path-containment.html
- License: This work is licensed under CC BY-NC-ND 4.0. See the license for attribution, noncommercial use and no-derivatives terms.
Related articles
- Fix Python Process Pool Startup Errors: spawn, the Main Guard, and Worker Tests
- Check Filename Case and Unicode Collisions with Python Before Copying Projects
- Fix Python venv bad interpreter After Moving a Project
- Python argparse Subcommands: Required Operations, Bounded Options, and Dash-Prefixed Filenames
- Fix Python SQLite Foreign Keys: Connection Checks, Batch Rollback, and Orphan Audits