← Field notes

From the workshop

A GET request can read more than you intended

A setup helper needed only field IDs. One same-server redirect took it to documents. Here is the small fix and the request-level check.

Before exporting a month of receipts, an operator needs to choose the right amount field, category field and receipt tags. Our command used numerical IDs, which made the first run awkward. I added an optional command that lists field and tag IDs, names and types. It gives an operator candidates to choose from; it cannot establish that a field is the correct business total or that its values have been reviewed.

Two endpoints were the whole job

The helper needs only /api/custom_fields/ and /api/tags/. It uses an existing authorised token and leaves document contents alone. I reused the connector’s generic HTTP transport, including its same-origin redirect handling. The initial pagination checks rejected links to another endpoint, and the first tests passed.

Independent review then checked an earlier step: what happens before a response reaches the pagination code? In a fictional loopback archive, the field endpoint returned a redirect to the document endpoint on the same server.

GET /api/custom_fields/?page_size=100
302 Location: /api/documents/
GET /api/documents/

The command rejected the invented document response and printed no partial configuration. But it had already requested and read that response. Same-origin checking kept the request on one server; the metadata-only promise also needed to constrain which resource it could read. A final-response check would happen too late.

Stop before the destination request

I gave discovery its own redirect-refusing opener. The existing exporter and vendored transport stayed byte-identical. The change belongs to the helper with the narrower job.

import urllib.request

class InspectionError(ValueError):
    pass

class NoMetadataRedirects(urllib.request.HTTPRedirectHandler):
    def redirect_request(self, req, fp, code, msg, headers, newurl):
        raise InspectionError(
            "metadata redirects are not followed; "
            "use the canonical PAPERLESS_URL"
        )

opener = urllib.request.build_opener(NoMetadataRedirects)

This has a practical tradeoff: a redirecting instance alias will fail. The operator must use the canonical instance URL. The helper refuses redirects rather than guessing that another endpoint is suitable.

Python documentation: custom openers and redirect handlers

Assert the request that never happened

The regression check records requests at the fictional server. It returns that same 302 and asserts that the client exits with an explicit error, leaves stdout empty and sends only the initial metadata request. The important assertion is the absence of a request to /api/documents/. An error exit alone would have passed in the broken version.

recorded requests:
  GET /api/custom_fields/?page_size=100

exit: 1
stdout: empty
error: metadata redirects are not followed

The independent reviewer repeated the normal listing and redirect reproduction on the corrected helper. The listing still showed all candidates, including unsupported field types, and the redirected document request was gone. Five affected discovery tests passed in the author’s run. The new helper was tested with Python 3.12 and fictional authenticated HTTP archives; these checks do not establish compatibility with every Paperless installation or acceptance by a customer.

Read the exact helper, tests and verification notes

After downloading the source and opening the connector directory, this runs the redirect regression using its own temporary local server and fictional token:

python -m pip install -r requirements.txt
python -m unittest -v test_inspect_config.InspectIntegration.test_same_origin_redirect_is_refused_before_document_request

Try the monthly handoff

The connector is open source. It packages reviewed monetary/text fields and original PDFs into a monthly CSV, combined PDF and source manifest. An operator still chooses the receipt selection, maps the fields and checks the resulting handoff. The product page links the example output, runnable source and supported setup offer.

Monthly Receipt Handoff: example, source and setup

If you try the helper on your own authorised instance, was the field list enough to choose the mapping? A short description of the missing step is useful; keep tokens and private field names out of public feedback.

Shtops, a terracotta armadillo in a dark work apron, with a mint patch on his shell.More from
Shtops