S&P 500 Companies with Financial Information

core

Files Size Format Created Updated License Source
2 129kB csv zip 6 months ago 1 month ago Open Data Commons Public Domain Dedication and License v1.0
List of companies in the S&P 500 (Standard and Poor's 500). The S&P 500 is a free-float, capitalization-weighted index of the top 500 publicly listed stocks in the US (top 500 by market cap). The dataset includes a list of all the stocks contained therein. Data Information on S&P 500 index used read more
Download

Data Files

File Description Size Last changed Download
constituents 19kB csv (19kB) , json (37kB)
s-and-p-500-companies_zip Compressed versions of dataset. Includes normalized CSV and JSON data with original data and datapackage.json. 23kB zip (23kB)

constituents  

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
Symbol 1 string
Name 2 string
Sector 3 string

Read me

List of companies in the S&P 500 (Standard and Poor’s 500). The S&P 500 is a free-float, capitalization-weighted index of the top 500 publicly listed stocks in the US (top 500 by market cap). The dataset includes a list of all the stocks contained therein.

Data

Information on S&P 500 index used to be available on the official webpage on the Standard and Poor’s website but until they publish it back, Wikipedia is the best up-to-date and open data source.

Sources

Detailed information on the S&P 500 (primarily in xls format) used to be obtained from its official webpage on the Standard and Poor’s website - it was free but registration was required.

  • Index listing - see <data/constituents.csv>
    • used to be extracted from source Excel file on S&P website but this no longer contains a list of constituents. (Note this Excel was actually S&P 500 EPS estimates but on sheet 4 it used to have a list of members - previous file was just members but that 404s as of Dec 2014) (Note: but note you have to register and login to access - no longer true as of August 2013)
  • Historical performance (source xls on S&P website)

Note: for aggregate information on the S&P (dividends, earnings etc) see Standard and Poor’s 500 Dataset

General Financial Notes

Publicly listed US companies are obliged various reports on a regular basis with the SEC. Of these 2 types are of especial interest to investors and others interested in their finances and business. These are:

  • 10-K = Annual Report
  • 10-Q = Quarterly report

Preparation

You can run the script yourself to update the data and publish them to github : see scripts README

License

All data is licensed under the Open Data Commons Public Domain Dedication and License. All code is licensed under the MIT/BSD license.

Note that while no credit is formally required a link back or credit to Rufus Pollock and the Open Knowledge Foundation is much appreciated.

Import into your tool

Data-cli or just data is the program to get and post your data with the datahub.
Download CLI tool and use it with the datahub almost like you use git with the github:

data get https://datahub.io/core/s-and-p-500-companies
data info core/s-and-p-500-companies
tree core/s-and-p-500-companies
# Get a list of dataset's resources
curl -L -s https://datahub.io/core/s-and-p-500-companies/datapackage.json | grep path

# Get resources

curl -L https://datahub.io/core/s-and-p-500-companies/r/0.csv

curl -L https://datahub.io/core/s-and-p-500-companies/r/1.zip

If you are using R here's how to get the data you want quickly loaded:

install.packages("jsonlite", repos="https://cran.rstudio.com/")
library("jsonlite")

json_file <- 'https://datahub.io/core/s-and-p-500-companies/datapackage.json'
json_data <- fromJSON(paste(readLines(json_file), collapse=""))

# get list of all resources:
print(json_data$resources$name)

# print all tabular data(if exists any)
for(i in 1:length(json_data$resources$datahub$type)){
  if(json_data$resources$datahub$type[i]=='derived/csv'){
    path_to_file = json_data$resources$path[i]
    data <- read.csv(url(path_to_file))
    print(data)
  }
}

Note: You might need to run the script with root permissions if you are running on Linux machine

Install the Frictionless Data data package library and the pandas itself:

pip install datapackage
pip install pandas

Now you can use the datapackage in the Pandas:

import datapackage
import pandas as pd

data_url = 'https://datahub.io/core/s-and-p-500-companies/datapackage.json'

# to load Data Package into storage
package = datapackage.Package(data_url)

# to load only tabular data
resources = package.resources
for resource in resources:
    if resource.tabular:
        data = pd.read_csv(resource.descriptor['path'])
        print (data)

For Python, first install the `datapackage` library (all the datasets on DataHub are Data Packages):

pip install datapackage

To get Data Package into your Python environment, run following code:

from datapackage import Package

package = Package('https://datahub.io/core/s-and-p-500-companies/datapackage.json')

# print list of all resources:
print(package.resource_names)

# print processed tabular data (if exists any)
for resource in package.resources:
    if resource.descriptor['datahub']['type'] == 'derived/csv':
        print(resource.read())

If you are using JavaScript, please, follow instructions below:

Install data.js module using npm:

  $ npm install data.js

Once the package is installed, use the following code snippet:

const {Dataset} = require('data.js')

const path = 'https://datahub.io/core/s-and-p-500-companies/datapackage.json'

// We're using self-invoking function here as we want to use async-await syntax:
;(async () => {
  const dataset = await Dataset.load(path)
  // get list of all resources:
  for (const id in dataset.resources) {
    console.log(dataset.resources[id]._descriptor.name)
  }
  // get all tabular data(if exists any)
  for (const id in dataset.resources) {
    if (dataset.resources[id]._descriptor.format === "csv") {
      const file = dataset.resources[id]
      // Get a raw stream
      const stream = await file.stream()
      // entire file as a buffer (be careful with large files!)
      const buffer = await file.buffer
      // print data
      stream.pipe(process.stdout)
    }
  }
})()
Datapackage.json