Published

Kuwait (KW) build scripts

DataHub
DataHub

Postal Codes Dataset for Kuwait, KW including name of the city, town, or place, various administrative divisions and alternative city names.

Kuwait (KW) build scripts

These scripts build the premium Kuwait dataset (tier B) from the Ministry of Communications' public postal code tables (https://moc.gov.kw/ar/important-links?tab=2). They produced the files published to R2 on 2026-10-08. Re-running steps 2–3 on the same raw pages reproduces the published kw.csv and the public sample byte for byte.

Licensing: unresolved; owner decision to sell with disclosure. See ../.licensing-notes.md.

Run, from this directory

pip install playwright && playwright install chromium   # step 1 only

python scrape_moc.py --locale en      # 1. raw pages -> ../sources/raw/en/   (resumable, 1 s between requests)
python scrape_moc.py --locale ar      #    raw pages -> ../sources/raw/ar/
python build_kw.py                    # 2. -> ../data/kw.csv, ../data/stats.json (exits 1 on a failed check)
python package_kw.py                  # 3. -> ../data/README.md, datapackage.json, ATTRIBUTION.txt, sample_kw.csv, kw.zip

Publish (dry run by default; --apply uploads and verifies each object by sha256), from the repo root:

set -a && . ./.env && set +a && python datasets/kw/scripts/publish_r2.py [--apply]

../sources/ and ../data/ are gitignored: raw pages and build output are never committed.

How the scrape works

The page shows two paginated tables to any visitor: postal-codes-for-kuwait-areas (PO Box ranges) and postalCodesByParcelNumber (blocks). Pagination is a Next.js server action. scrape_moc.py replays the request the page itself sends, from inside the page (same origin), one page at a time. robots.txt disallows only /admin/*. The REST route /api/<collection> returns 403 and is not used. The response de-duplicates repeated objects as $1:docs:<i>:<key> references, which build_kw.py resolves.

Known gaps

  • No coordinates: the ministry tables have none. The earlier build's OpenStreetMap-derived coordinates are deliberately not used.
  • package_kw.py writes the README and ATTRIBUTION that went into the 2026-10-08 zip, including counts and dates. The repo's ../README.md is the maintained version.