Spain (ES): migration notes

Internal record of the P5 migration from the old scripts to the producer: the discrepancies found, their causes, and who decided what. This is a dated record, so the numbers are as of the migration and not kept current. It is not shipped in the zip.

• Issue: pc-28e.7.20 (closed 2026-10-03)
• Commit: e440ce6 (worktree commit 8946ec5), landed in batch 5
• Parity verdict: format-only (gate pass); no sign-off needed
• Producer: custom, refresh disabled

Sources

• The old builder read our own published file. resolve_po_box_codes.py downloaded the live es/es.csv from R2 with credentials, using it only as the list of PO-box codes that exist. No upstream source lists those codes, and CartoCiudad's bulk layer omits them. The frozen po_box_codes.csv replaces that read.
• The PO-box place names in the table were reconstructed. The agent took each one as the row's first alternative name once the register form is removed, falling back to the municipality's own name.
• TLS: CartoCiudad's certificate chain fails the stdlib's default CA store on this machine (self-signed certificate in certificate chain), so the producer fetches with requests (certifi bundle), not the shared http_get. requests is in requirements.txt (checked when landing).
• The live zip's datapackage.json has no sources (stripped on 2026-08-24 by project rule), so the harvest step took them from the repo's old descriptor (sources_from: repo).
• datapackage.yaml sources still names the CartoCiudad home page and INE's catalogue page (codmun00i.htm) rather than the URLs fetched; the fetched URLs are in es.inputs.json.

Parity against the live zip (2026-10-03, 17:46 UTC)

Rows: 14,538 live and 14,538 fresh, none only on one side. 0 changed cells, no duplicate keys, header unchanged. The only byte difference is that the live CSV quotes every field. A second run with --no-fetch (cached answers) produced a byte-identical CSV.

No upstream drift showed up, even though the coordinates come from a live geocoder crawl. A probe for 01001 before the crawl returned 42.84912 / -2.67239, exactly the live value. The 2026-08-19 licensing note records 148 PO-box rows whose alternative_city_name differed from R2 in an earlier rebuild; none differed this time.

The informational sample check against https://postal.datahub.io/es/es.csv was unexplained. Nobody looked into it.

Decisions

Transforms carried over

Folded from scrape_postal.py → build_admin.py → build_dataset.py (pass 1) → resolve_po_box_codes.py → fetch_coordinates.py → build_dataset.py (pass 2) → package_delivery.py. The old two-pass ordering trap (running the PO-box step before pass 1 silently emptied the PO-box table) is gone.

• INE municipality code = the last 5 digits of the WFS AD_ADMINUNITNAME_MUN_ id; relations deduplicated on the (code, municipality) pair; codes outside 01–52 dropped (none as of 2026-10).
• Communal territories dropped: CartoCiudad files shared land (mancomunidades, parzonerías) under phantom provinces 53/54. The build stops if that would lose a postal code.
• GeoNames fill: an INE municipality with no CartoCiudad relation gets every GeoNames code filed under it whose province prefix agrees.
• muni_corrections.csv rows applied; a row is skipped once a bulk source covers its municipality.
• The postal prefix is the serving post office, not always the province (Condado de Treviño carries 01xxx); admin_code2 / admin_name2 stay the municipality's own INE province.
• INE names with a trailing article ("Coruña, A") put in natural order, only for a real article.
• admin_code1 = ISO 3166-2:ES community code with ES- stripped; admin_code2 = INE CPRO.
• alternative_city_name: GeoNames localities for this (code, municipality), minus the municipality's own name and names of other municipalities in the same provinces; then the literal register form if it was inverted; then each half of a bilingual name. Deduplicated ignoring accents and case, pipe-joined.
• PO-box codes not in the WFS layer: municipality by place-name match against INE within the province (all spellings), else the Palma de Mallorca → Palma (07040) alias, else the first geocoder candidate in that province.
• Geocoder q takes the padded code and id the integer form (the padded id gives HTTP 500 for provinces 01–09).
• Coordinates rounded to 5 dp; POSTAL_CODE for a geocoded point. A PO-box code gets the median of its municipality's per-code points, MUNICIPALITY; blank if none.
• Sorted by (postal_code, admin_code3); primary key (postal_code, admin_code3) checked unique.
• Old asserts are now ProducerErrors: unknown CPRO, duplicate INE code, all 52 provinces and 19 communities present, corrections and PO rows name a known municipality and carry evidence, WFS not truncated, no code lost by the communal drop. No row-count asserts.
• Old scripts, scripts/README.md, delivery_README_template.md and publish_r2.py removed. ATTRIBUTION.txt unchanged.

Open questions

• New PO-box codes won't appear by themselves. po_box_codes.csv is a frozen copy of our own 2026-08 output, so a code Correos adds later stays out until someone edits the table. This is what the user approved; it's listed here because no refresh route exists.
• Resolved (user, 2026-10-05, pc-28e.7.59): the geocoder province term from GeoNames' admin_name2 (INE fallback) is accepted.
• Crawl cost on every full rebuild. Each fetch (not --no-fetch) re-runs about 10,850 geocoder requests. Refresh is off, so this only happens on manual rebuilds.
• .licensing-notes.md still describes the old scripts (two-pass build_dataset.py, resolve_po_box_codes.py reading R2) and gives row counts. Filed as a comment on pc-hfz.
• Admin-unit counts stay in the README by rule; any remaining counts belong to the pc-hfz sweep.

Provenance

Reconstructed 2026-10-05 from the agent transcript d33e1b7e-5027-498a-a97f-ba914ee2524e/subagents/agent-a959c157213db63f8.jsonl (the D1 stop and the final hand-back), the orchestrator session d33e1b7e-… (the AskUserQuestion answers, the SendMessage to the agent and the landing), the harvest step in session 068ca403-…, commit e440ce6, the bd notes and close reason, and docs/parity/p5.md.