Python Task Configuration with tomllib: Validate TOML and Resolve Paths

An automation script needs source edits for every new input, its command line keeps growing, and a scheduled run cannot find a relative file path. Put task settings in TOML and validate them at the entry point so one script can serve several tasks.

This beginner-to-intermediate tutorial builds a complete input preview tool using the Python standard library. It loads configuration, checks types, supports a command-line line-count override, and works from a different working directory. The example reads files and prints JSON.

1. Method: validate settings before running the task

The official tomllib documentation states that the module was added in Python 3.11 and parses TOML without writing it. tomllib.load() requires a binary file object and returns a Python dictionary; invalid syntax raises TOMLDecodeError.

Successful parsing does not establish valid task settings. This tutorial defines these rules:

1. The top level contains only [task]. That table accepts only input and limit; misspelled keys fail.
2. input is a nonempty string. Relative paths start from the configuration file’s directory.
3. limit is an integer from 1 to 1000, defaulting to 3 when omitted. Strings, floats, and booleans fail.
4. Precedence is default, configuration file, then explicit --limit. Validate the entire file first: an invalid file value fails even when the command line overrides it.

The preview reads text lines, including blank lines. Use it to check a task’s entry point; for CSV processing, connect the record parsing from streaming large CSV files.

2. Prepare the environment and test data

Use an existing Python 3.11 or newer interpreter. There are no third-party packages to install. The examples were verified on Linux with Python 3.11.4.

1. Run these commands in bash or zsh. Choose a new directory name when repeating the experiment, and keep subsequent commands in the same terminal:

python3 --version
python3 -c 'import tomllib; print("tomllib ready")'
mkdir tomllib-demo
cd tomllib-demo
task_demo_root="$PWD"
mkdir configs data
printf 'alpha\nbeta\ngamma\n' > data/items.txt
cat > configs/task.toml <<'EOF'
[task]
input = "../data/items.txt"
limit = 2
EOF

2. The [task] table produces a nested dictionary. Here ../data/items.txt starts from configs/ and points to the demo input. limit = 2 is an integer; limit = "2" is a string.
3. Save the next section as preview_task.py in the current directory, or download preview_task.py.

For sharing tools that need third-party packages, see the uv single-file script tutorial. This script needs no dependency declaration.

3. Complete implementation: load, validate, and override

"""Preview UTF-8 input using validated TOML settings. Python 3.11+."""
import argparse
from itertools import islice
import json
from pathlib import Path
import sys
import tomllib


def valid_limit(value):
    if type(value) is not int or not 1 <= value <= 1000:
        raise ValueError("limit must be an integer from 1 to 1000")
    return value


def cli_limit(value):
    try:
        return valid_limit(int(value))
    except ValueError as exc:
        raise argparse.ArgumentTypeError(str(exc)) from exc


def load_settings(config):
    config = config.expanduser().resolve(strict=True)
    with config.open("rb") as file:
        data = tomllib.load(file)
    if set(data) != {"task"}:
        raise ValueError("top level must contain only [task]")
    task = data["task"]
    if not isinstance(task, dict):
        raise ValueError("task must be a table")
    unknown = set(task) - {"input", "limit"}
    if unknown:
        raise ValueError("unknown task keys: " + ", ".join(sorted(unknown)))
    raw_input = task.get("input")
    if not isinstance(raw_input, str) or not raw_input.strip():
        raise ValueError("task.input must be a nonempty string")
    limit = valid_limit(task.get("limit", 3))
    input_path = Path(raw_input).expanduser()
    if not input_path.is_absolute():
        input_path = config.parent / input_path
    return config, input_path.resolve(strict=True), limit


def main():
    parser = argparse.ArgumentParser(description=__doc__, allow_abbrev=False)
    parser.add_argument("--config", required=True, type=Path, help="TOML file")
    parser.add_argument("--limit", type=cli_limit, help="override preview line limit")
    args = parser.parse_args()
    try:
        config, input_path, limit = load_settings(args.config)
        if args.limit is not None:
            limit = args.limit
        with input_path.open(encoding="utf-8") as file:
            lines = [line.rstrip("\r\n") for line in islice(file, limit)]
        print(json.dumps({
            "config": str(config),
            "input": str(input_path),
            "limit": limit,
            "lines": lines,
        }, ensure_ascii=False, indent=2))
    except (OSError, ValueError, RuntimeError) as exc:
        print(f"ERROR: {exc}", file=sys.stderr)
        return 1
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

valid_limit() checks type(value) is int. Python’s bool is a subclass of int, so isinstance(value, int) alone would accept true as an integer.

The argparse type parameter applies cli_limit() to command-line strings. An omitted --limit stays None, preventing a CLI default from accidentally replacing the file setting.

Paths use Path.resolve(), which resolves symbolic links. The base is the configuration file’s physical directory, rather than the directory containing a link to it. The code supports ~; it does not substitute $variables inside TOML strings.

4. Run and check the output

1. Display help, use the file settings, then override the line count:

python3 preview_task.py --help
python3 preview_task.py --config configs/task.toml
python3 preview_task.py --config configs/task.toml --limit 1

The second command prints a limit of 2 and lines of ["alpha", "beta"]; the third prints 1 and ["alpha"]. Both config and input contain absolute paths so you can check which files the task used.

2. Assert the result:

python3 preview_task.py --config configs/task.toml > preview.json
python3 - <<'PY_CHECK'
import json
from pathlib import Path
result = json.loads(Path("preview.json").read_text(encoding="utf-8"))
assert result["limit"] == 2
assert result["lines"] == ["alpha", "beta"]
assert Path(result["input"]) == Path("data/items.txt").resolve()
print("OK: settings and input path")
PY_CHECK

Expect OK: settings and input path. If the input contains fewer lines than requested, the tool returns the available lines; an empty file returns an empty list. limit is the requested maximum, while len(lines) is the actual count.

The program limits the number of lines and keeps the selected lines in memory. Very long individual lines still consume memory. Use this as a small input preview; process large records individually in your real task.

5. Verify execution from another directory

Run this in the same terminal:

(
  cd /
  python3 "$task_demo_root/preview_task.py" \
    --config "$task_demo_root/configs/task.toml" --limit 1
)

The subshell now works from /, but the output should still have limit = 1 and lines = ["alpha"]. The absolute --config path locates the configuration; its relative input path locates the data.

For a scheduled task, use absolute paths for the existing project interpreter, script, and configuration. If the task needs additional files or environment variables, check them using Linux scheduled-job path troubleshooting. This experiment needs no crontab edits.

6. Verify error exits

Create a file with valid TOML syntax but an invalid task value, then test an invalid CLI argument:

cat > configs/bad.toml <<'EOF'
[task]
input = "../data/items.txt"
limit = true
EOF
python3 preview_task.py --config configs/bad.toml
printf 'exit=%s\n' "$?"
python3 preview_task.py --config configs/task.toml --limit 0
printf 'exit=%s\n' "$?"

The first task command should print ERROR: limit must be an integer from 1 to 1000, followed by exit=1. The second prints argparse usage and an argument error, followed by exit=2. A successful task returns 0.

Validation also checked defaults, unknown keys, missing input settings, malformed TOML, float and string limits, missing files, Unicode input, and a configuration-file symlink. The code validates settings before reading task input; failed validation produces no successful JSON result.

7. Troubleshooting

Summary

Keep task settings in TOML, let code enforce types, ranges, and allowed fields, and support a small set of CLI overrides. Fix the base directory for relative paths and verify both successful and failed runs before reusing this entry point in everyday automation.