Clean Up Old Logs with Python: Preview First, Skip Links and Subdirectories

An archive directory keeps accumulating logs, but a quick deletion command might target a live file or follow an unexpected link. This beginner-to-intermediate tutorial uses the Python standard library to build a tool that previews by default and deletes only with explicit --apply, then verifies it on disposable fixtures before touching managed archives.

The target is independent, completed .log files that you have confirmed are no longer in use, not live log rotation. Start with ranking large files in Python if you do not yet know what consumes space; do not point the tutorial commands directly at production logs.

1. Method: define scope before calculating retention

1. Scan direct children of one directory, without recursion. Match the case-sensitive .log suffix only; exclude hidden names, directories, symbolic links, and files with multiple hard links.
2. Capture the current time once at startup and calculate the cutoff using 1 day = 86400 seconds. Select files whose mtime is strictly earlier than that cutoff. This is neither a local calendar-date rule nor a creation-time rule.
3. Complete the scan before previewing or applying. Require --days in the range 1–36500; there is no zero-day cleanup or default deletion.
4. Read metadata again before each deletion and stop on an identity or state change. Record completed deletions individually, followed by a final summary.

The Python os.scandir documentation describes directory-entry scanning; entry.stat(follow_symlinks=False) reads the link itself, and the code then filters by regular-file type and link count. Scanning reads metadata, not log contents, but storing candidates uses memory proportional to the number selected.

Scope limits: the directory and its parents must be trusted. Stop writers, rotation, moves, and file replacement during scanning and deletion. The second check is not atomic: a race remains between checking and deleting, so this is not suitable for directories other users can change arbitrarily. Deletion is irreversible; make an independent backup before real use. A preview report is not a backup.

2. Setup: create disposable fixtures

Use an existing Python 3.11 or newer interpreter; no third-party dependencies are required. Examples were verified on Linux with Python 3.11.4 and work in bash or zsh. Keep subsequent commands in the same terminal and directory; choose another directory name when repeating the exercise.

1. Create a nine-day-old log, a fresh log, ordinary text, symbolic links, and a directory whose name ends in .log:

python3 --version
mkdir old-logs-demo
cd old-logs-demo
mkdir logs
python3 - <<'PY_DATA'
import os
from pathlib import Path
import time
now = time.time_ns()
old = now - 9 * 86400 * 1_000_000_000
for name in ("old.log", "fresh.log", "keep.txt"):
    path = Path("logs") / name
    path.write_bytes(b"old\n")
    stamp = now if name == "fresh.log" else old
    os.utime(path, ns=(stamp, stamp))
Path("logs/link.log").symlink_to("keep.txt")
Path("logs/broken.log").symlink_to("absent")
Path("logs/nested.log").mkdir()
PY_DATA

2. Save the next section as cleanup_logs.py, or download the complete cleanup script. Do not create local files called stat.py, json.py, or time.py, which would shadow standard-library modules.

3. Implementation: share selection rules between preview and apply

"""Preview or delete old completed .log files in one trusted directory."""
import argparse
import json
import os
from pathlib import Path
import stat
import sys
import time

DAY_NS = 86400 * 1_000_000_000


def positive_days(text):
    try:
        value = int(text)
    except ValueError:
        raise argparse.ArgumentTypeError("days must be an integer") from None
    if not 1 <= value <= 36500:
        raise argparse.ArgumentTypeError("days must be 1..36500")
    return value


def signature(info):
    return (info.st_dev, info.st_ino, info.st_size,
            info.st_mtime_ns, info.st_ctime_ns, info.st_nlink)


def select_logs(root, days, now_ns):
    cutoff = now_ns - days * DAY_NS
    selected = []
    with os.scandir(root) as entries:
        for entry in entries:
            if entry.name.startswith(".") or not entry.name.endswith(".log"):
                continue
            info = entry.stat(follow_symlinks=False)
            if (stat.S_ISREG(info.st_mode) and info.st_nlink == 1
                    and info.st_mtime_ns < cutoff):
                selected.append((root / entry.name, info))
    return sorted(selected, key=lambda item: item[0].name)


def emit(record):
    print(json.dumps(record, ensure_ascii=True), flush=True)


def apply_one(path, previous):
    current = path.lstat()
    if (not stat.S_ISREG(current.st_mode)
            or signature(current) != signature(previous)):
        raise ValueError(f"changed since scan: {path.name!a}")
    path.unlink()


def main():
    parser = argparse.ArgumentParser(description=__doc__, allow_abbrev=False)
    parser.add_argument("directory", type=Path)
    parser.add_argument("--days", required=True, type=positive_days)
    parser.add_argument("--apply", action="store_true",
                        help="delete selected files; default only previews")
    args = parser.parse_args()
    deleted = 0
    try:
        root = args.directory.expanduser().resolve(strict=True)
        if not root.is_dir():
            raise ValueError("directory must be a directory")
        selected = select_logs(root, args.days, time.time_ns())
        for path, info in selected:
            if args.apply:
                apply_one(path, info)
                deleted += 1
            emit({"action": "deleted" if args.apply else "would_delete",
                  "name": path.name, "bytes": info.st_size})
        emit({"mode": "apply" if args.apply else "preview",
              "selected": len(selected), "deleted": deleted})
        return 0
    except (OSError, ValueError) as exc:
        print(f"ERROR: deleted={deleted}; {str(exc)!a}", file=sys.stderr)
        return 1


if __name__ == "__main__":
    raise SystemExit(main())

time.time_ns() provides integer system time for comparison with file modification timestamps. It is not a monotonic clock; the host clock still needs to be correct. Future timestamps are not selected, and this tool does not repair clock skew.

Path.unlink() removes the file name rather than moving it to a recycle bin. A scan error stops the tool before deletion starts. A failure partway through applying does not restore files already deleted.

4. Validation: prove preview leaves the original files unchanged

1. Display help, write the preview report outside logs/, and assert its contents:

python3 cleanup_logs.py --help
python3 cleanup_logs.py logs --days 7 > preview.jsonl
cat preview.jsonl
python3 - <<'PY_CHECK'
import json
from pathlib import Path
rows = [json.loads(line) for line in Path("preview.jsonl").read_text().splitlines()]
assert rows == [
    {"action": "would_delete", "name": "old.log", "bytes": 4},
    {"mode": "preview", "selected": 1, "deleted": 0},
]
assert Path("logs/old.log").read_bytes() == b"old\n"
print("OK: preview changed no log files")
PY_CHECK

cat preview.jsonl should print:

{"action": "would_delete", "name": "old.log", "bytes": 4}
{"mode": "preview", "selected": 1, "deleted": 0}

The final assertion prints OK: preview changed no log files. The bytes value is logical file length, not a promise of reclaimed disk space. Copy operations that preserve old timestamps affect selection too; check each file’s purpose before applying.

2. After reviewing names and the directory, enable deletion only for this disposable fixture:

python3 cleanup_logs.py logs --days 7 --apply > applied.jsonl
cat applied.jsonl
python3 - <<'PY_CHECK'
import json
from pathlib import Path
rows = [json.loads(line) for line in Path("applied.jsonl").read_text().splitlines()]
assert rows == [
    {"action": "deleted", "name": "old.log", "bytes": 4},
    {"mode": "apply", "selected": 1, "deleted": 1},
]
assert not Path("logs/old.log").exists()
for name in ("fresh.log", "keep.txt"):
    assert (Path("logs") / name).read_bytes() == b"old\n"
assert Path("logs/link.log").is_symlink()
assert Path("logs/broken.log").is_symlink()
assert Path("logs/nested.log").is_dir()
print("OK: only old.log was deleted")
PY_CHECK
python3 cleanup_logs.py logs --days 7 --apply

The validation prints OK: only old.log was deleted. The final repeated invocation should print only:

{"mode": "apply", "selected": 0, "deleted": 0}

5. Automation checks: exit codes and boundary tests

Exit 0 means the scan and selected operation completed; an empty selection is also success. Exit 1 means a filesystem or state-check failure; exit 2 means invalid arguments. On an apply failure, stderr’s deleted=N reports completed deletions in this run, not a transaction rollback.

Use preview as a check first. Successfully producing a report is not permission to delete:

if python3 cleanup_logs.py logs --days 7 > next-preview.jsonl; then
    printf 'Preview complete; review next-preview.jsonl before any deletion.\n'
else
    printf 'Preview failed; do not continue.\n' >&2
fi

Download the isolated test script, place it beside cleanup_logs.py, and run:

python3 test_cleanup.py

Tests create and remove only their own temporary directories. Seven tests cover preview, apply, repetition, an exact cutoff, future timestamps, hidden names, suffix case, directories, symbolic and hard links, invalid arguments, changed files, partial deletion failures, and simulated permission failures. They assert exit codes, output, and retained file contents. Simulated failures do not establish every real filesystem permission combination.

6. Troubleshooting: do not fix problems by weakening the rules

7. Summary

Define scope, time rules, and the apply switch explicitly, then prove on isolated fixtures that preview preserves files and apply handles only selected files. For real archives, back up first, stop concurrent modifications, and review the preview item by item. This tool does not provide a recycle bin, transactional recovery, or live log rotation.