Netherlands (NL): migration notes

Internal record of the P5 migration from the staged-export converter to the producer: the discrepancies found, their causes, and who decided what. This is a dated record (2026-10-11), so the numbers are as of the migration and not kept current. It is not shipped in the zip.

• Issue: datopian/postal-codes#41
• Commit: the producer commit on this branch. It also adds zstandard to requirements-producers.txt (the address file is zstd-compressed)
• Parity verdict: unexplained (gate: blocked), accepted under the owner's under-1% rule: 0.79% of live rows are affected (157 gone, 3,526 with a shifted coordinate), all traced to the source file having been updated
• Producer: custom, refresh disabled

What made the live file, verified

The old convert_from_fedex.py read a staged copy of the prototype's Netherlands.csv, which no longer exists. That delivery is the output of the fedex-agents-prototype countries/NLD build: scrape_postal.py (Phase 1: dedupe the LJPc address file), scrape_admin.py (CBS), combine_phase2.py (join), then a QA pass that ordered the rows by province and appended two 00000 backfill rows. The converter then dropped those two rows and mapped the columns.

Verified on 2026-10-11:

• The old converter on the staged export reproduces the live nl.csv byte for byte.
• This producer's join and rendering, fed the prototype's saved Phase 1 and admin files (postal_raw.csv, admin.csv), reproduces the live nl.csv: same 467,628 rows, same order, the same bytes apart from CRLF.
• Differential check on fresh inputs. The prototype's own three scripts and this producer were run on the same downloaded files (the LJPc file of 2026-10-10 and today's CBS tables). Phase 1: 467,747 unique rows, identical and in the same order. Admin: 2,501 records, identical. Phase 2: 467,729 rows, identical and in the same order. So none of the parity differences below come from the port.

Sources

The prototype also fetched Wikipedia's municipality list, but only to print a cross-check; it never changed the data, so it is not an input here.

Parity against the live zip (2026-10-11)

Names, provinces and municipalities do not change on any row present in both files; only coordinates do. 446 of the cell changes are rounding-level (parity class coordinate_precision), 6,344 are other coordinate changes.

Cause: the LJPc file was updated since the live build (upstream drift). Phase 1 keeps the first address in file order for each (postcode, woonplaats, gemeente, provincie), so when addresses are added, removed or re-geocoded, that first address (and its coordinate) can change, and a few combinations appear or disappear. CBS also gained one woonplaats since the saved admin file (2,501 against 2,500). This is the "larger parity diff" plan decision D1 anticipated for a producer rebuilt from the original source.

Decisions

Transforms carried over

• Phase 1: first occurrence of each (postcode, woonplaats, gemeente, provincie); postcode must match [1-9]\d{3}[A-Z]{2}; coordinates rounded to 5 decimals.
• Join: exact (woonplaats + gemeente); the woonplaats alone, or the most similar gemeente when several share it; fuzzy woonplaats (ratio at least 85); else the parent gemeente. Duplicates on (postal code, locality) keep the row whose gemeente is listed first for that locality.
• place_name and admin_name3 are both the woonplaats; accuracy is BLOCK; no codes below the province.

Open questions

• R2 is untouched. The public sample in R2 is not the first 100 rows of this file's current order; publishing replaces it.

Provenance

Written 2026-10-11 from the saved prototype files, the differential run described above, scripts/parity.py nl (reads the live zip, writes nothing) and the Frictionless validation of the built zip.