Test Directory Writes with Python: Permissions, Read-Only Storage, and Path Errors

A batch job finishes computing, then hits Permission denied while saving output. Running it as an administrator appears to fix it, but the scheduled job still fails. This tutorial uses the Python standard library to create, write, and remove a tiny file in the destination directory, testing the current execution identity’s write path.

It suits beginner-to-intermediate readers preparing exports, backups, or Hugo builds. For large outputs, combine it with the disk-space budget check.

1. Method: exercise the write path

The official os.access() documentation cautions that real I/O can fail after an access check succeeds. Our probe attempts the operation and records errno; the result applies to this account, environment, and moment.

The sequence is: resolve an existing directory → create a randomly named file → write 12 bytes → flush → fsync → close and delete. It leaves existing application files untouched and creates no missing directories. It does write a real temporary file, so use a trusted directory where testing is allowed. Directory symlinks are resolved to their targets.

2. Setup

Use Linux, Python 3.9 or newer, and Bash for the entry-script example. No pip packages are needed. These examples were verified on Linux with Python 3.11.4 as a normal user.

1. Create a fresh demo directory; choose another name when repeating the exercise:

python3 --version
mkdir write-probe-demo
cd write-probe-demo
mkdir output

2. Save the next section as write_probe.py in the current directory, or download the complete script. Run subsequent commands from this demo directory.

3. Implementation: JSON and exit codes

"""Probe a trusted existing directory by creating a tiny temporary file. Linux."""
import argparse
import errno
import json
import os
from pathlib import Path
import tempfile


def probe(directory):
    report = {
        "path": os.path.abspath(directory),
        "euid": os.geteuid(),
        "ok": False,
        "stage": "resolve",
    }
    try:
        target = Path(directory).resolve(strict=True)
        report["path"] = str(target)
        report["stage"] = "create"
        with tempfile.NamedTemporaryFile(
            mode="w+b", dir=target, prefix=".write-probe-"
        ) as handle:
            report["stage"] = "write"
            handle.write(b"write-probe\n")
            report["stage"] = "flush"
            handle.flush()
            report["stage"] = "fsync"
            os.fsync(handle.fileno())
            report["stage"] = "cleanup"
        report.update(ok=True, stage="done")
    except (OSError, RuntimeError) as exc:
        number = getattr(exc, "errno", None)
        report.update(
            error=type(exc).__name__,
            errno=number,
            code=errno.errorcode.get(number, "PATH_ERROR"),
            message=str(exc),
        )
    return report


def main(argv=None):
    parser = argparse.ArgumentParser(description=__doc__, allow_abbrev=False)
    parser.add_argument("directory", help="existing trusted output directory")
    args = parser.parse_args(argv)
    report = probe(args.directory)
    print(json.dumps(report, ensure_ascii=True))
    return 0 if report["ok"] else 1


if __name__ == "__main__":
    raise SystemExit(main())

NamedTemporaryFile creates a visible file in the chosen directory and deletes it on normal close. The context manager also attempts cleanup after ordinary exceptions; SIGKILL or power loss can leave .write-probe- files.

path is the resolved path; euid identifies the effective user performing file operations. stage is the last operation started, with done indicating completion. If exceptional cleanup also fails, its exception can replace the original error, so inspect message too. JSON goes to stdout; argparse sends command-line usage errors to stderr.

4. Run and validate

1. Display help, probe output, and immediately inspect the exit code:

python3 write_probe.py --help
python3 write_probe.py output
printf 'exit=%s\n' "$?"

Successful JSON includes "ok": true and "stage": "done", followed by exit=0. The absolute path and euid vary by machine. Failure returns "ok": false with an error such as "code": "EACCES".

2. Use disposable directories to check successful writing, cleanup, a missing path, a regular-file path, and a directory without write permission:

python3 - <<'PY_CHECK'
import json
import os
from pathlib import Path
import subprocess
import sys
import tempfile

script = str(Path("write_probe.py").resolve())

def check(path, status, code=None):
    result = subprocess.run([sys.executable, script, str(path)],
                            capture_output=True, text=True, check=False)
    report = json.loads(result.stdout)
    assert result.returncode == status, report
    assert report["ok"] is (status == 0), report
    if code:
        assert report["code"] == code, report
    return report

with tempfile.TemporaryDirectory(prefix="probe-test-", dir=".") as name:
    root = Path(name).resolve()
    writable = root / "writable"
    writable.mkdir()
    check(writable, 0)
    assert not list(writable.iterdir())
    check(root / "missing", 1, "ENOENT")
    regular = root / "regular.txt"
    regular.write_text("keep me", encoding="utf-8")
    check(regular, 1, "ENOTDIR")
    assert regular.read_text(encoding="utf-8") == "keep me"
    denied = root / "denied"
    denied.mkdir()
    denied.chmod(0o500)
    try:
        if os.geteuid() == 0:
            print("SKIP: run as a normal user to test EACCES")
        else:
            check(denied, 1, "EACCES")
    finally:
        denied.chmod(0o700)
print("PASS: creation, cleanup, missing path, file path, permission checks")
PY_CHECK

A non-root user should see the final PASS without SKIP. The code changes permissions only on its own new test directory and restores them in finally. Root can bypass ordinary permission bits, so that branch is skipped. Do not change a real application directory for this test.

3. Simulate read-only storage, exhausted space, quota, and synchronization errors without filling disks or changing mounts:

python3 - <<'PY_ERRORS'
import errno
from pathlib import Path
from unittest.mock import patch
from write_probe import probe

for code in ("EROFS", "ENOSPC", "EDQUOT"):
    number = getattr(errno, code)
    with patch("write_probe.tempfile.NamedTemporaryFile",
               side_effect=OSError(number, code)):
        report = probe("output")
    assert report["ok"] is False and report["code"] == code, report
with patch("write_probe.os.fsync", side_effect=OSError(errno.EIO, "I/O error")):
    report = probe("output")
assert report["code"] == "EIO" and report["stage"] == "fsync", report
assert not list(Path("output").glob(".write-probe-*"))
print("PASS: simulated disk errors and fsync failure cleanup")
PY_ERRORS

Expect PASS: simulated disk errors and fsync failure cleanup. This tests error-handling logic, not a real read-only mount or quota. The success and permission-denied cases above use real file operations.

5. Add a task entry script

Save this as run_demo.sh in the demo directory. It prints TASK STARTED only after a successful probe:

#!/usr/bin/env bash
set -eu
cd -- "$(dirname -- "$0")"
if python3 write_probe.py output; then
  python3 -c 'print("TASK STARTED")'
else
  result=$?
  printf 'Write probe stopped the task (exit %s)\n' "$result" >&2
  exit "$result"
fi

Run it:

bash run_demo.sh

For a real task, replace the print command with your export or build command and use the actual output directory. Create and confirm the destination before running the entry script. Scheduled runs need the same account, container, mount environment, and interpreter. See scheduled-job path troubleshooting for working-directory and interpreter settings.

6. Troubleshooting and limits

Inspect the account and directory as the task’s own execution identity:

id
pwd
ls -ld -- . output

Check whether the account can traverse parent directories and create files in the destination. Grant the necessary permissions only to the intended account or group; avoid masking the cause with recursive chmod 777 or administrator execution.

The error names below follow the official Python errno documentation. Error messages vary by system language; use code first.

Creating a new file does not prove that an existing file can be overwritten. File permissions, sticky bits, immutable attributes, and replacement operations can impose further restrictions. For replacing JSON state, continue with the atomic-write tutorial.

The probe reserves no space and tests neither large-file capacity nor subdirectory creation. Following the os.fsync() documentation, it flushes before syncing; this checks that particular file synchronization call. It does not sync the parent directory or establish directory-entry durability after power loss. Network filesystem calls can block, and this script has no hard timeout. Actual tasks must still handle their own OSError even after the probe succeeds.

7. Summary

A small write probe in the real destination, using the task’s execution identity, can locate failures more directly than inspecting mode bits alone. Validate both success and failure exits before adding it to a task entry point, and retain error handling around the actual output operation.