Fix Python Module Shadowing: Inspect Import Origins and Interpreter Paths

A script runs import json, then fails with AttributeError: module 'json' has no attribute 'loads'. A helper named json.py in the project can make Python load the wrong file even though the standard library is working.

This beginner-to-intermediate tutorial uses a repeatable experiment to inspect the interpreter, module origin, and search path, then fixes the collision by renaming the helper. Use it for Linux automation, data tools, and Python steps in Hugo build wrappers. No third-party packages are needed.

1. Method: check where the module comes from

Work in this order:

1. Identify the interpreter actually running the failed task.
2. Inspect the module’s origin for a same-name project file or package.
3. Compare isolated mode, then correct the name or environment.

Python’s search-path documentation explains that a script’s directory normally starts the search path. With -c, -m, or interactive execution, the current directory enters the search path instead. Names such as json.py, logging.py, and csv.py can therefore shadow the intended modules. Same-name package directories can also collide.

This does not explain every import error. ModuleNotFoundError can indicate the wrong virtual environment; cannot import name can involve version differences or circular imports. Follow the actual paths in the traceback.

2. Setup: reproduce a same-name file failure

Use Linux, Python 3.9 or newer, and bash or zsh. The examples were verified on Python 3.11.4. They require neither pip nor changes to the system Python.

1. Create a fresh demo directory and stay there for subsequent steps. Choose a new directory name when repeating the experiment:

python3 --version
python3 --help
mkdir import-shadow-demo
cd import-shadow-demo
printf 'SOURCE = "local helper"
' > json.py

2. Try the standard library’s JSON parser:

if python3 -c 'import json; print(json.__file__); print(json.loads("true"))'; then
  printf 'Unexpected success: inspect the environment
'
else
  printf 'Expected failure: local json.py has no loads()
'
fi

Expect the demo directory’s json.py path, an AttributeError about missing loads, and finally Expected failure. The if explicitly handles the expected failure so a shell using set -e can continue. If it unexpectedly succeeds, inspect the origin in the next section before assuming the reproduction worked.

This json.py contains one variable and deliberately defines no loads(). The example demonstrates a successful import of the wrong object, rather than invalid JSON input.

3. Inspect the interpreter and module origin

1. Show the interpreter and environment prefixes:

python3 -c 'import sys; print("executable:", sys.executable); print("prefix:", sys.prefix); print("base_prefix:", sys.base_prefix)'
command -v python3

In a normal venv, sys.prefix and sys.base_prefix usually differ. command -v reports shell command lookup; sys.executable shows this process’s interpreter path. Diagnose a real task with the same interpreter as its entry point and reproduce its launch directory. For environment creation and dependency management, see the uv project environment tutorial.

2. Find the module specification before importing json:

python3 - <<'PY_PROBE'
import importlib.util
import sys

name = "json"
spec = importlib.util.find_spec(name)
print("executable:", sys.executable)
print("origin:", None if spec is None else spec.origin)
print("package_paths:", None if spec is None else spec.submodule_search_locations)
print("sys.path:")
for entry in sys.path:
    print(" ", repr(entry))
PY_PROBE

Here, origin should point to the demo directory’s json.py. It is a plain module, so package_paths is None. An empty string '' in the search path means the current directory. To inspect another module, replace the value of name.

The official find_spec() documentation distinguishes finding a specification from executing an import. A missing top-level module normally returns None; finding a specification does not guarantee successful execution. Looking up the top-level name json does not execute this example file. Looking up package.child, however, imports the parent package. This is not a sandbox for untrusted projects.

4. Compare isolated mode, then rename the helper

1. Leave json.py in place and run isolated mode:

python3 -I -c 'import json; print(json.__file__); print(json.loads("true"))'

Expect the standard library’s json/__init__.py path and True. The official -I documentation says isolated mode excludes the script or current directory and user site-packages, and ignores PYTHON* environment variables. A successful comparison implicates those inputs; success with -I alone does not prove a naming collision. The origin check identifies the specific collision here.

2. Give your helper a name describing its purpose:

mv -i json.py json_helpers.py
python3 -c 'import json; print(json.__file__); print(json.loads("true"))'
python3 -c 'import json_helpers; print(json_helpers.SOURCE)'

The second command should print the standard library path and True; the third prints local helper, showing the helper remains available. mv -i prompts if the destination exists, avoiding an overwrite; that destination should not exist in this fresh demo. In a real project, also update references that intended to import the old helper.

Each command starts a new process, so renaming triggers a fresh lookup. If a Notebook, service, or interactive interpreter already cached the imported object, restart that Python process or kernel before verification. This example does not require deleting all of __pycache__.

Do not append -I to every production command: programs depending on local project modules may stop importing them. Here it diagnoses path influence; renaming restores normal execution.

5. Automatically validate success and failure branches

The following self-check is independent of the demo files above and can be copied directly. It creates fixtures in its own temporary directory, checks exit codes and actual output, and cleans up afterward. It does not delete the earlier demo directory.

python3 -I - <<'PY_CHECK'
from pathlib import Path
import os
import subprocess
import sys
import tempfile

env = os.environ.copy()
for key in list(env):
    if key.startswith("PYTHON"):
        del env[key]

def run(folder, code, isolated=False):
    flags = ["-I"] if isolated else []
    return subprocess.run(
        [sys.executable, *flags, "-c", code], cwd=folder, env=env,
        capture_output=True, text=True, timeout=10, check=False,
    )

with tempfile.TemporaryDirectory(prefix="import-shadow-test-") as tmp:
    folder = Path(tmp)
    local = folder / "json.py"
    local.write_text('SOURCE = "local helper"\n', encoding="utf-8")
    probe = run(folder, "import importlib.util; print(importlib.util.find_spec('json').origin)")
    assert probe.returncode == 0 and Path(probe.stdout.strip()) == local, probe

    check = "import json; print(json.loads('true'))"
    failed = run(folder, check)
    assert failed.returncode != 0 and "AttributeError" in failed.stderr, failed
    isolated = run(folder, check, isolated=True)
    assert isolated.returncode == 0 and isolated.stdout.strip() == "True", isolated

    local.rename(folder / "json_helpers.py")
    fixed = run(folder, check)
    assert fixed.returncode == 0 and fixed.stdout.strip() == "True", fixed
    helper = run(folder, "import json_helpers; print(json_helpers.SOURCE)")
    assert helper.returncode == 0 and helper.stdout.strip() == "local helper", helper

    package = folder / "json"
    package.mkdir()
    (package / "__init__.py").write_text("", encoding="utf-8")
    package_fail = run(folder, check)
    assert package_fail.returncode != 0 and "AttributeError" in package_fail.stderr, package_fail
    missing = run(folder, "import importlib.util; print(importlib.util.find_spec('demo_missing_74219'))")
    assert missing.returncode == 0 and missing.stdout.strip() == "None", missing

print("OK: file shadowing, isolated mode, rename, helper, package shadowing, missing module")
PY_CHECK

Expected output:

OK: file shadowing, isolated mode, rename, helper, package shadowing, missing module

The test also covers a same-name json/__init__.py package and find_spec() returning None for a missing top-level module. Subprocesses have a 10-second wait limit, and expected failures are explicitly checked.

6. Troubleshooting and next actions

An origin of built-in, frozen, or None is not automatically wrong; namespace packages, for example, may lack an ordinary source file. For this json example, combine __file__ with the actual parser result.

If Python never starts and the error is bad interpreter, first inspect Linux script CRLF and BOM problems. Import diagnosis comes after the interpreter starts successfully.

7. Summary

When a module lacks an expected attribute, record the interpreter path and module origin before changing dependencies. Reproducing a same-name file, comparing isolated mode, and verifying a rename in a new process separates path shadowing from environment problems. Adding these checks to troubleshooting avoids unnecessary changes to the system Python and global packages.