Build URL Query Parameters in Python: Unicode, Plus Signs, and Repeated Keys

Hand-built ?q= strings break when a search term contains &, turn C++ into C at the server, or fail on Unicode. This tutorial uses the Python standard library to build query strings, retain repeated parameters and empty values, and check the values received through an actual local HTTP request.

It targets beginner-to-intermediate readers writing Linux automation or small API clients. The builder only prints URLs; the self-test contacts loopback and needs no account or API key.

1. Method: pass raw values and encode once

1. Store parameters as a list of (key, value) pairs, allowing repeated keys.
2. Split the base URL, read existing parameters, then append new ones.
3. Encode the query with urlencode() and reassemble the URL.

The official urlencode() documentation covers query encoding and doseq. Spaces become +; literal plus signs become %2B. Our CLI uses string pairs. A dictionary containing lists needs doseq=True:

from urllib.parse import urlencode

query = urlencode({"tag": ["python", "linux"], "q": "C++"}, doseq=True)
assert query == "tag=python&tag=linux&q=C%2B%2B"

Pass new values unencoded. Pass + for a plus sign; %2B means the literal text %2B and produces %252B. Encoding the whole URL can also encode structural delimiters.

2. Setup

Use Python 3.10+ and curl, with no third-party Python packages. Examples were tested on Linux with Python 3.11.4; commands work in Bash and zsh.

1. Create a fresh practice directory and download both scripts:

python3 --version
mkdir url-query-demo
cd url-query-demo
curl -fS https://www.bobobk.com/downloads/url-query/build_query.py -o build_query.py
curl -fS https://www.bobobk.com/downloads/url-query/check_query.py -o check_query.py
python3 build_query.py --help

2. The complete build_query.py implementation follows; you can save it directly. Keep the self-test and URL builder in the same directory.

3. Complete implementation: preserve and append parameters

"""Append raw string query parameters to a trusted HTTP URL. Python 3.10+."""
import argparse
import sys
from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit


def parameter(text):
    key, found, value = text.partition("=")
    if not found or not key:
        raise argparse.ArgumentTypeError("use a nonempty KEY=VALUE; VALUE may be empty")
    return key, value


def build_url(base, additions):
    if not base.isascii() or any(ord(c) <= 32 or ord(c) == 127 for c in base):
        raise ValueError("base must be an ASCII URL without whitespace or controls")
    parts = urlsplit(base)
    if parts.scheme not in {"http", "https"} or not parts.hostname:
        raise ValueError("base must have an http(s) scheme and a host")
    # Accessing port also rejects malformed or out-of-range port numbers.
    _ = parts.port
    pairs = parse_qsl(
        parts.query, keep_blank_values=True, encoding="utf-8",
        errors="strict", max_num_fields=1000, separator="&",
    )
    pairs.extend(additions)
    if len(pairs) > 1000:
        raise ValueError("at most 1000 query parameters are allowed")
    query = urlencode(pairs, encoding="utf-8", errors="strict")
    return urlunsplit(parts._replace(query=query))


def main():
    parser = argparse.ArgumentParser(description=__doc__, allow_abbrev=False)
    parser.add_argument("base", help="trusted, already encoded base URL")
    parser.add_argument(
        "--param", action="append", type=parameter, default=[],
        metavar="KEY=VALUE", help="raw string parameter; repeat to append more",
    )
    args = parser.parse_args()
    try:
        result = build_url(args.base, args.param)
    except ValueError as exc:
        print(f"ERROR: {exc}", file=sys.stderr)
        return 1
    print(result)
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

parse_qsl() returns pairs; keep_blank_values=True retains empty values. Our policy appends parameters, with a final limit of 1000. partition("=") splits only the first equals sign.

Supply a trusted, correctly encoded ASCII base URL. These are basic checks, not full URL validation or path/domain encoding. Re-encoding existing queries can change %20 into +, so exclude signed URLs requiring original bytes. The generated result retains #results.

4. Run and validate the result

1. Pass Unicode, plus signs, repeated parameters, and delimiters within values:

python3 build_query.py 'https://example.com/search?page=1&empty=&tag=old#results' \
  --param 'q=C++ & Linux/中文' \
  --param 'tag=python' --param 'tag=linux' \
  --param 'note=a=b#c' > url.txt
cat url.txt

Expected output:

https://example.com/search?page=1&empty=&tag=old&q=C%2B%2B+%26+Linux%2F%E4%B8%AD%E6%96%87&tag=python&tag=linux&note=a%3Db%23c#results

Shell quotes prevent the terminal from splitting arguments; URL encoding prevents the server from misreading values. Both steps matter. This command does not contact example.com.

2. Decode and assert the values and order:

python3 - <<'PY_CHECK'
from pathlib import Path
from urllib.parse import parse_qsl, urlsplit
parts = urlsplit(Path("url.txt").read_text(encoding="utf-8").strip())
assert parts.fragment == "results"
assert parse_qsl(parts.query, keep_blank_values=True) == [
    ("page", "1"), ("empty", ""), ("tag", "old"),
    ("q", "C++ & Linux/中文"), ("tag", "python"),
    ("tag", "linux"), ("note", "a=b#c"),
]
print("OK: query values and order")
PY_CHECK

Expect OK: query values and order. The original tag=old and empty= remain, with two more tag values appended. To replace a key, filter its old pairs before appending. Converting everything to a dictionary would lose other repeated parameters.

5. Check the actual request with a local HTTP server

Run the downloaded self-test:

python3 check_query.py

Expected output:

OK: HTTP round trip, duplicate/blank values, fragment, and CLI exits

The test binds an ephemeral port on 127.0.0.1, sends a real GET request, and asserts that Unicode, plus signs, repeated keys, blank values, =, #, and literal %2B survive. It also checks that the generated URL retains its fragment while the server request excludes it, then stops the server and closes the port.

The server uses http.server for local practice. Its client uses ProxyHandler({}) to bypass proxies and a three-second blocking-operation timeout. System proxy settings stay untouched.

6. Verify error exits

These commands work interactively and in Shell scripts using set -e:

if python3 build_query.py 'file:///tmp/input'; then
  printf 'UNEXPECTED success\n'
else
  printf 'exit=%s\n' "$?"
fi
if python3 build_query.py 'https://example.com/' --param 'missing-equals'; then
  printf 'UNEXPECTED success\n'
else
  printf 'exit=%s\n' "$?"
fi

The first writes an error to stderr and prints exit=1; the second reports a CLI usage error and prints exit=2. Success returns 0. Failures print no successful URL. --param 'empty=' is valid; --param '=value' returns 2 for an empty key.

7. Troubleshooting

Continue with:

• Python HTTP JSON requests with timeouts and bounded retries: pass the generated URL to a client.
• Reliable file downloads with Python: handle download endpoints with query parameters.
• Linux scheduled-job path troubleshooting: use explicit interpreter and file paths in scheduled scripts.

Summary

Pass raw string parameters, retain repeated keys and empty values, and encode the query in one place. Decode and assert the result, then use local HTTP echo to check received values. When connecting a real API, follow its parameter replacement rules and handle timeouts and request failures.