Check Executable Paths with Python shutil.which Before Automation Runs

Python raises FileNotFoundError for a command that works in your terminal, or a Hugo automation job suddenly uses an older binary. Check which executable the process would select before reinstalling tools.

This beginner-to-intermediate tutorial builds a read-only command preflight with the standard library: look up several tools, demonstrate PATH precedence, and stop a job when dependencies are missing. The demo creates files only in a new local directory; it does not change the system PATH, scheduler, or server configuration.

1. Method: look up commands without executing them

The official shutil.which documentation describes searching a path for an executable that meets access checks, returning None when no match exists. Its default checks cover existence and execution permission, not a trial run. We explicitly pass path=search_path; otherwise our --path option would have no effect.

For Linux background jobs, this tutorial applies a stricter input policy:

1. Accept bare names such as hugo, not ./hugo, full paths, or complete shell command lines.
2. Require every PATH entry to be a nonempty absolute path. Reject empty and relative entries so lookup does not depend on the working directory. Directory names may contain spaces.
3. If the process has no PATH, require an explicit --path rather than silently choosing a fallback. That option controls this lookup only; it does not change the parent shell or a future child’s environment.
4. Preserve the selected path’s symlink spelling rather than calling resolve(). For example, do not casually replace a virtual environment’s interpreter path with its system target. See the official venv explanation for the interpreter/environment relationship.

These are tutorial policies, not universal shell rules. For interpreter, working-directory, and environment settings at the job entry point, see Linux scheduled-job PATH and working-directory troubleshooting.

2. Setup: create two same-name test tools

Use Linux, Python 3.9 or newer, and bash/zsh. No third-party packages are needed. Examples were tested on Linux with Python 3.11.4.

1. Run the following in one terminal and remain in the demo directory. Choose a new directory name when repeating the experiment to avoid overwriting previous files:

python3 --version
mkdir command-path-demo
cd command-path-demo
python3 - <<'PY_SETUP'
from pathlib import Path
for directory, label in [("first bin", "FIRST"), ("second bin", "SECOND")]:
    folder = Path(directory)
    folder.mkdir()
    tool = folder / "demo-tool"
    tool.write_text(f"#!/bin/sh\nprintf '{label}\\n'\n", encoding="utf-8")
    tool.chmod(0o755)
Path("empty bin").mkdir()
PY_SETUP

Both files are simple executable shell scripts. The checker locates them without running them, so you should not see FIRST or SECOND.

2. Save the next section’s complete code as check_commands.py in the current directory.

3. Implementation: explicit PATH and automation-friendly exit codes

"""Inspect bare command names on an explicit Linux PATH without running them."""
import argparse
import json
import os
import shutil
import sys


def inspect_commands(names, search_path):
    if search_path is None:
        raise ValueError("PATH is unset; pass --path explicitly")
    entries = search_path.split(os.pathsep)
    if any(not entry or not os.path.isabs(entry) for entry in entries):
        raise ValueError("PATH entries must be nonempty absolute paths")
    if "\0" in search_path:
        raise ValueError("PATH must not contain NUL")
    if any(not name or "/" in name or "\0" in name for name in names):
        raise ValueError("use bare command names, not file paths")
    results = []
    for name in names:
        hit = shutil.which(name, path=search_path)
        results.append({"command": name, "path": hit})
    return {"results": results, "ok": all(row["path"] is not None for row in results)}


def main():
    cli = argparse.ArgumentParser(description=__doc__, allow_abbrev=False)
    cli.add_argument("--path", default=os.environ.get("PATH"),
                     help="PATH to inspect; defaults to the process PATH")
    cli.add_argument("commands", nargs="+", help="bare executable names")
    args = cli.parse_args()
    try:
        report = inspect_commands(args.commands, args.path)
    except (OSError, ValueError) as exc:
        print(f"ERROR: {exc}", file=sys.stderr)
        return 2
    print(json.dumps(report))
    return 0 if report["ok"] else 1


if __name__ == "__main__":
    raise SystemExit(main())

results lists every command and its selected path; a missing tool has JSON null as its path. A missing command does not stop the remaining lookups. The final status covers the whole list. A nonexistent PATH directory simply contributes no match; nothing is created or installed.

4. Usage: reverse PATH order and compare the selected files

python3 check_commands.py --help
python3 check_commands.py --path "$PWD/first bin:$PWD/second bin" demo-tool
python3 check_commands.py --path "$PWD/second bin:$PWD/first bin" demo-tool

Both JSON reports have ok: true, but the first path ends with /first bin/demo-tool and the second with /second bin/demo-tool. The preceding path depends on your demo directory.

If the checker starts but cannot find its target command, inspect the PATH of that Python process, not the settings in another terminal. Shell aliases, functions, and builtins are outside this checker’s lookup scope.

5. Validation: success, missing tools, and invalid input

Save this as test_lookup.py and run python3 test_lookup.py. It uses fixed small fixtures and requires neither Hugo nor execution of the test tools.

from pathlib import Path
import json
import os
import subprocess
import sys

root = Path.cwd()
first = str(root / "first bin")
second = str(root / "second bin")
empty = str(root / "empty bin")
script = str(root / "check_commands.py")
before = {p: p.read_bytes() for p in root.glob("* bin/demo-tool")}


def check(arguments, expected, env=None):
    result = subprocess.run(
        [sys.executable, script, *arguments],
        capture_output=True, text=True, env=env, check=False,
    )
    assert result.returncode == expected, result
    if expected == 2:
        assert result.stdout == "" and "ERROR:" in result.stderr, result
        return None
    assert result.stderr == "", result.stderr
    return json.loads(result.stdout)


for path, directory in [(first + ":" + second, first), (second + ":" + first, second)]:
    data = check(["--path", path, "demo-tool"], 0)
    assert data["results"][0]["path"] == directory + "/demo-tool"
    assert data["ok"] is True
print("PASS: PATH order selects the first executable")

data = check(["--path", empty, "demo-tool"], 1)
assert data == {"results": [{"command": "demo-tool", "path": None}], "ok": False}
print("PASS: missing executable returns 1")

for path in ["", ".", first + ":", first + ":relative"]:
    check(["--path", path, "demo-tool"], 2)
check(["--path", first, "./demo-tool"], 2)
env = os.environ.copy()
env.pop("PATH", None)
check(["demo-tool"], 2, env=env)
print("PASS: ambiguous PATH and non-bare names rejected")
assert all(p.read_bytes() == content for p, content in before.items())
print("PASS: fixture content unchanged")

Expected output:

PASS: PATH order selects the first executable
PASS: missing executable returns 1
PASS: ambiguous PATH and non-bare names rejected
PASS: fixture content unchanged

CLI argument errors also return 2. These tests cover an unset process PATH, empty and relative entries, and unchanged file content. They do not simulate your scheduler’s permissions or mount conditions.

6. Keep lookup separate from execution at the automation entry point

This exercises the entry-point branch with the demo tool without executing it:

if python3 check_commands.py --path "$PWD/first bin" demo-tool; then
  printf 'Preflight passed; now validate the real tool version and run the job.\n'
else
  status=$?
  printf 'Stopped before the job; preflight exit=%s\n' "$status" >&2
  exit "$status"
fi

For a real job, try python3 check_commands.py python3 hugo to inspect the current process’s PATH, then check tool versions. Run the check as the actual job account with its actual environment, not just from an administrator’s terminal.

The official subprocess documentation recommends a full executable path for reliability. To run a tool, use the selected, reviewed, trusted path as the first element of an argument list; do not concatenate a shell command and enable shell=True. Follow the Python subprocess timeout and logging tutorial for subsequent error handling.

Finding a file does not guarantee launch success: an invalid shebang interpreter, wrong executable format, a noexec mount, missing runtime libraries, or replacement after the check can still break execution. This is not a security sandbox or a race-free guarantee. Use trusted directories and still check the child’s actual exit status.

7. Troubleshooting and summary

Show the selected file path before starting a job to separate missing commands from wrong-version selection. Test precedence and failure statuses with fixed fixtures, then connect the check to a trusted entry point; actual execution still needs its own error handling.