Published

Portugal (PT) build pipeline

DataHub
DataHub
0

Postal Codes Dataset for Portugal, PT including name of the city, town, or place, various administrative divisions and alternative city names.

Portugal (PT) build pipeline

Rebuilt from the original GeoNames PT export rather than the copy already sitting in R2 at pt/pt.csv — that copy was produced by scripts/geonames/geonames_parse_to_R2.py with pandas and no dtype=str, which float-coerces numeric-looking admin code columns (measured: 463/500 sampled PT rows showed .0-suffixed codes) and strips punctuation from names via the old clean_and_normalize().

Two enrichment passes go beyond plain GeoNames to fill admin_code3 (freguesia code), which GeoNames itself doesn't carry:

  1. enrich_caop.py — joins the official DICOFRE crosswalk (CAOP, Carta Administrativa Oficial de Portugal), bringing admin_code3 to 64% filled.
  2. spatial_join_freguesia.py — point-in-polygon against real freguesia boundary geometry for what's still unmatched, bringing fill to 98.79%.

recover_alt_names.py has a narrower, explicitly-scoped dependency on the existing R2 copy: it recovers only the alternative_city_name column (~12.6% of rows) from it, rather than re-downloading GeoNames' full separate alternate-names dump. This is not a blanket circular dependency the way PH's old build was — it's one column, documented, and everything else in the pipeline is an independent fetch.

Run order

python3 fetch_source.py        # GeoNames PT.txt
python3 build_base.py
python3 enrich_caop.py
python3 spatial_join_freguesia.py
python3 recover_alt_names.py   # narrow dependency on existing R2 copy — see above
python3 integrity_checker.py
python3 package.py
python3 publish_r2.py          # dry run by default; --apply to publish

Sources

  • GeoNames PT postal export — CC BY 4.0
  • CAOP (Carta Administrativa Oficial de Portugal) — official Portuguese admin boundary register, DICOFRE crosswalk + freguesia geometry

Verification (2026-08-19)

Verified via a genuinely fresh fetch (all cached sources deleted first). Full 7-stage run produced exactly 14,538 rows, matching R2. Sorted-diff MD5 against live R2 is identical — the only difference before sorting was non-deterministic tie-break ordering among rows sharing a sort key, not a data discrepancy. Frictionless valid. Publish dry-run: zero-diff plan.