API AccessAccess dataset files directly from scripts, code, or AI agents.
Browse dataset files
Access dataset files directly from scripts, code, or AI agents.
Each file has a stable URL (r-link) that you can use directly in scripts, apps, or AI agents. These URLs are permanent and safe to hardcode.
Start with these files — they give you everything you need to understand and access the dataset.
- 1. Fetch datapackage.json to inspect schema and resources
- 2. Download data resources listed in datapackage.json
- 3. Read README.md for full context
Data Views
Data Previews
sector-counts
Schema
| name | type | description |
|---|---|---|
| sector | string | |
| count | integer | Number of S&P 500 constituent companies |
constituents
Schema
| name | type |
|---|---|
| Symbol | string |
| Security | string |
| GICS Sector | string |
| GICS Sub-Industry | string |
| Headquarters Location | string |
| Date added | string |
| CIK | string |
| Founded | string |
Data Files
| File | Description | Size | Last modified | Download |
|---|---|---|---|---|
sector-counts | 218 B | 5 days ago | sector-counts | |
constituents | 53.7 kB | about 11 hours ago | constituents |
| Files | Size | Format | Created | Updated | License | Source |
|---|---|---|---|---|---|---|
| 2 | 53.9 kB | csv | about 11 hours ago | Open Data Commons Public Domain Dedication and License v1.0 |
S&P 500 Companies Dataset
List of companies in the S&P 500 (Standard and Poor's 500). The S&P 500 is a free-float, capitalization-weighted index of the top 500 publicly listed stocks in the US (top 500 by market cap). The dataset includes a list of all the stocks contained therein.
Data
Information on S&P 500 index used to be available on the official webpage on the Standard and Poor's website but until they publish it back, Wikipedia's [SP500 list of companies][sp-list] is the best up-to-date and open data source.
Sources
Detailed information on the S&P 500 (primarily in XLS format) used to be obtained from its official webpage on the Standard and Poor's website - it was free but registration was required.
Note
For aggregate information on the S&P (dividends, earnings, etc.) see Standard and Poor's 500 Dataset.
General Financial Notes
Publicly listed US companies are obliged various reports on a regular basis with the SEC. Of these 2 types are of especial interest to investors and others interested in their finances and business. These are:
- 10-K = Annual Report
- 10-Q = Quarterly report
Development
The pipeline relies on Python, so you'll need to have it installed on your machine. Then:
- Create a virtual environment in a directory using Python's venv module:
python3 -m venv .env - Activate the virtual environment:
source .env/bin/activate - Install the dependencies:
pip install -r scripts/requirements.txt - Run the scripts:
python scripts/scrape.py
Alternatively, you can use the provided Makefile to run the scraping with a simple make. It'll create a virtual environment, install the dependencies and run the script.
License
All data is licensed under the Open Data Commons Public Domain Dedication and License. All code is licensed under the MIT/BSD license.
Note that while no credit is formally required a link back or credit to Rufus Pollock and the Open Knowledge Foundation is much appreciated.