What a request is made of, and how to see the one you actually sent

Four parts, every time

Every HTTP request your program sends has the same four parts, and requests builds all of them from the arguments you pass. When a call misbehaves, the fault is in one of these four, and knowing which one is most of the diagnosis.

1. The method. GET asks for something; POST sends something; PUT, PATCH and DELETE change or remove. Model APIs are almost entirely POST, because the prompt travels in the body.
2. The URL, including the query string. Everything after ? is a set of key=value pairs separated by &.
3. The headers. Key-value metadata: who you are, what format you are sending, what format you will accept.
4. The body. Optional. Bytes, usually JSON text, sent with POST.

The query string, and why params= exists

You can build a URL by hand:

url = f"https://example.com/search?q={query}&page=2"

This breaks the first time query contains a space, an ampersand, a plus sign or a letter outside ASCII. A space must become %20, & must become %26, and é must become %C3%A9. Forget any of these and the server reads a different query from the one you meant. It will not tell you; it will answer the question it received.

requests does the encoding when you hand it a dict:

r = requests.get(
    "https://example.com/search",
    params={"q": "café au lait", "page": 2},
    timeout=10,
)
print(r.request.url)
# https://example.com/search?q=caf%C3%A9+au+lait&page=2

Note the + for a space. Both + and %20 are valid in a query string, and servers accept either. A value of None in the dict is dropped rather than sent as the string None, and a list value is repeated: {"tag": ["a", "b"]} becomes tag=a&tag=b.

Headers: the ones you must send

Two headers matter for nearly every API call.

Authentication. Most providers use one of two shapes:

headers = {"Authorization": f"Bearer {key}"}     # OpenAI, most others
headers = {"x-api-key": key}                      # Anthropic

Get the header name wrong and you receive a 401 that looks exactly like a bad key. When a 401 arrives and you are sure of the key, check the header name before anything else.

Content type. When you send a body, the server needs to know how to read it. Pass json= and requests sets Content-Type: application/json and serialises the dict for you. Pass data= with a dict and it sends form-encoded pairs instead, with a different content type — and an API expecting JSON will reject it or, worse, read an empty body. This is the single most common cause of a 400 with a message like "missing field: model" when the field is plainly there.

r = requests.post(url, json={"model": m, "messages": msgs})   # JSON body
r = requests.post(url, data={"model": m})                     # form body — wrong for a model API

requests also sends a User-Agent of python-requests/2.x by default. Some services block it. Setting your own, naming your project, is polite and occasionally necessary.

Seeing what you sent

The request you wrote and the request that left your machine are different objects. The second is available after the call:

r = requests.post(url, headers=headers, json=payload, timeout=30)

print(r.request.method)
print(r.request.url)
print(r.request.headers)
print(r.request.body[:200])

Print these before reading the response when something is wrong. Nine times out of ten the bug is visible here: a header name with a typo, a body that is form-encoded, a URL with a doubled slash, a key that is the string None because the environment variable was never set.

Never print the full Authorization header into a log that anyone else can see. Print r.request.headers["Authorization"][:12] and check that it starts with Bearer sk-.

An echo server for experiments

The free service at httpbin.org returns whatever you sent it, as JSON, so you can see a request from the server's side:

r = requests.post(
    "https://httpbin.org/post",
    params={"q": "café"},
    headers={"X-Demo": "yes"},
    json={"a": 1},
    timeout=10,
)
print(r.json()["args"])     # {'q': 'café'}
print(r.json()["json"])     # {'a': 1}
print(r.json()["headers"]["X-Demo"])

Try data= instead of json= and watch json become None and form fill in. That one experiment is worth a chapter.

If you would rather not send anything off your machine, python -m http.server 8000 serves the current folder and prints every request line it receives.

The response has the same anatomy

A response is a status code, headers and a body. Two response headers earn their keep:

• Content-Type tells you whether r.json() will work. An error page arrives as text/html, and calling .json() on it raises rather than returning your data.
• Retry-After on a 429 or 503 tells you how many seconds to wait. A later lesson uses it.

r.headers is a case-insensitive dict, so r.headers["content-type"] and r.headers["Content-Type"] both work.

Where this is heading

You have now seen a request as the server sees it. The rest of the module builds outward: reusing the connection, timing out and retrying without paying twice, and walking a result that arrives in pages.