Vega Graph Spec Tutorial - US presidents

examples

Files Size Format Created Updated License Source
2 1kB csv vega-datasets
This is an example Data Package, that demonstrates how to build the awesome visualizations using the "Vega Graph Spec". We are using lifelines of the first 5 presidents of the US - one of the examples from vega editor - and displaying it here, on DataHub with slightest modifications in read more
Download

Data Files

people  

Field information

Field Name Order Type (Format) Description
label 1 string
born 2 number
died 3 number
enter 4 number
leave 5 number

events  

Field information

Field Name Order Type (Format) Description
name 1 string
when 2 date (fmt:%Y-%m-%d)

Read me

This is an example Data Package, that demonstrates how to build the awesome visualizations using the "Vega Graph Spec". We are using lifelines of the first 5 presidents of the US - one of the examples from vega editor - and displaying it here, on DataHub with slightest modifications in vega-spec.

Views

We assume that you are familiar with what [datapackage.json][datapackage.json] is and it's specifications.

To create graphs for your tabular Data Package, the datapackage.json should include the views attribute that is responsible for visualizations.

If you are familiar with Vega specifications, you would probably like to use all it's futures and display you data with desired visualizations in a Vega way. To use it, inside views you should set specType to "vega" and define some graph specifications in spec. See example datapackage.json:

Vega Graph Specifications

{
  "datahub": {
    "findability": "published",
    "hash": "4a10a76a6ecdefab3e16b04c006e0665",
    "modified": "2017-08-08T17:37:53.073550",
    "owner": "examples",
    "ownerid": "examples",
    "stats": {
      "bytes": 1360,
      "rowcount": 18
    }
  },
  "id": "examples/vega-views-tutorial-lines",
  "name": "vega-views-tutorial-lines",
  "readme": "This is an example Data Package, that demonstrates how to build the awesome visualizations using the \"Vega Graph Spec\". We are using lifelines of the first 5 presidents of the US - one of the examples from [vega editor][editor] - and displaying it here, on DataHub with slightest modifications in vega-spec.\n\n## Views\n\nWe assume that you are familiar with what [datapackage.json][datapackage.json] is and it's specifications.\n\nTo create graphs for your tabular Data Package, the `datapackage.json` should include the `views` attribute that is responsible for visualizations.\n\nIf you are familiar with [Vega][vega] specifications, you would probably like to use all it's futures and display you data with desired visualizations in a Vega way. To use it, inside `views` you should set `specType` to \"vega\" and define some graph specifications in `spec`. See example datapackage.json:\n\n### Vega Graph Specifications\n\n{{ datapackage.json }}\n\n<br>\n\nYou can use almost the same specifications inside `spec` attribute, that are used for setting the vega graphs. Only difference is that in `data` property, `url` and `path` attributes are moved out.\n\n```\n  ...\n  \"spec\": {\n    ...\n    \"data\": [\n      {\n        \"name\": \"people\"\n      },\n      {\n        \"name\": \"events\",\n        ...\n      }\n    ],\n  }\n```\n\nInstead we use `name` attribute to reference to a dataset. Note, how we created a new object inside `data` property - `{\"name\": \"people\"}` to reference it to resources (this names are identical to names of resources)\n\nOutside of `spec` attribute there are some other important parameters to note:\n\n<table class=\"table table-bordered table-striped resource-summary\">\n  <thead>\n   <tr>\n     <th>Attribute</th>\n     <th>Type</th>\n     <th>Description</th>\n   </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>name</th>\n      <td>String</td>\n      <td>Unique identifier for view within list of views.</td>\n    </tr>\n    <tr>\n      <th>title</th>\n      <td>String</td>\n      <td>Title for the graph.</td>\n    </tr>\n    <tr>\n      <th>resources</th>\n      <td>Array</td>\n      <td>Data sources for this spec. It can be either resource name or index. By default it is the first resource.</td>\n    </tr>\n    <tr>\n      <th>specType</th>\n      <td>String</td>\n      <td>Available options: simple, vega, plotly <strong>(Required)</strong>.</td>\n    </tr>\n  </tbody>\n</table>\n\n[vega]: https://vega.github.io/vega/\n[editor]: https://vega.github.io/vega-editor/?mode=vega&spec=lifelines\n",
  "resources": [
    {
      "bytes": 376,
      "datahub": {
        "derivedFrom": [
          "people"
        ],
        "type": "derived/csv"
      },
      "dialect": {
        "delimiter": ",",
        "doubleQuote": true,
        "lineTerminator": "\r\n",
        "quoteChar": "\"",
        "skipInitialSpace": false
      },
      "encoding": "utf-8",
      "format": "csv",
      "hash": "c86a484ca414a5b9bc6cc4b77047b716",
      "mediatype": "text/csv",
      "name": "people",
      "path": "data/csv/data/people.csv",
      "rowcount": 5,
      "schema": {
        "fields": [
          {
            "name": "label",
            "type": "string"
          },
          {
            "decimalChar": ".",
            "groupChar": "",
            "name": "born",
            "type": "number"
          },
          {
            "decimalChar": ".",
            "groupChar": "",
            "name": "died",
            "type": "number"
          },
          {
            "decimalChar": ".",
            "groupChar": "",
            "name": "enter",
            "type": "number"
          },
          {
            "decimalChar": ".",
            "groupChar": "",
            "name": "leave",
            "type": "number"
          }
        ]
      },
      "title": "",
      "description": "",
      "alternates": [
        {
          "datahub": {
            "type": "source/tabular"
          },
          "format": "csv",
          "mediatype": "text/csv",
          "name": "people",
          "schema": {
            "fields": [
              {
                "name": "label",
                "type": "string"
              },
              {
                "name": "born",
                "type": "number"
              },
              {
                "name": "died",
                "type": "number"
              },
              {
                "name": "enter",
                "type": "number"
              },
              {
                "name": "leave",
                "type": "number"
              }
            ]
          },
          "url": "https://s3.amazonaws.com/rawstore.datahub.io/c96393bb004da037e7c9f533e69aa31e"
        },
        {
          "bytes": 638,
          "datahub": {
            "derivedFrom": [
              "people"
            ],
            "type": "derived/json"
          },
          "encoding": "utf-8",
          "format": "json",
          "hash": "d7abc9a6f25da40067dd23e3ac19cd03",
          "mediatype": "text/csv",
          "name": "people_json",
          "path": "data/json/data/people.json",
          "rowcount": 5,
          "schema": {
            "fields": [
              {
                "name": "label",
                "type": "string"
              },
              {
                "name": "born",
                "type": "number"
              },
              {
                "name": "died",
                "type": "number"
              },
              {
                "name": "enter",
                "type": "number"
              },
              {
                "name": "leave",
                "type": "number"
              }
            ]
          }
        }
      ]
    },
    {
      "bytes": 134,
      "datahub": {
        "derivedFrom": [
          "events"
        ],
        "type": "derived/csv"
      },
      "dialect": {
        "delimiter": ",",
        "doubleQuote": true,
        "lineTerminator": "\r\n",
        "quoteChar": "\"",
        "skipInitialSpace": false
      },
      "encoding": "utf-8",
      "format": "csv",
      "hash": "0962f1871d1dbbc6890f58bd1f8ecf8f",
      "mediatype": "text/csv",
      "name": "events",
      "path": "data/csv/data/events.csv",
      "rowcount": 4,
      "schema": {
        "fields": [
          {
            "name": "name",
            "type": "string"
          },
          {
            "format": "fmt:%Y-%m-%d",
            "name": "when",
            "type": "date"
          }
        ]
      },
      "title": "",
      "description": "",
      "alternates": [
        {
          "datahub": {
            "type": "source/tabular"
          },
          "format": "csv",
          "mediatype": "text/csv",
          "name": "events",
          "schema": {
            "fields": [
              {
                "name": "name",
                "type": "string"
              },
              {
                "format": "any",
                "name": "when",
                "type": "date"
              }
            ]
          },
          "url": "https://s3.amazonaws.com/rawstore.datahub.io/289cdbd393ad4e16d4ee46c770f75a57"
        },
        {
          "bytes": 212,
          "datahub": {
            "derivedFrom": [
              "events"
            ],
            "type": "derived/json"
          },
          "encoding": "utf-8",
          "format": "json",
          "hash": "50eef9e09a6aef4f23412ba6ccc8fc4a",
          "mediatype": "text/csv",
          "name": "events_json",
          "path": "data/json/data/events.json",
          "rowcount": 4,
          "schema": {
            "fields": [
              {
                "name": "name",
                "type": "string"
              },
              {
                "format": "fmt:%Y-%m-%d",
                "name": "when",
                "type": "date"
              }
            ]
          }
        }
      ]
    }
  ],
  "sources": [
    {
      "name": "vega-datasets",
      "title": "vega-datasets",
      "web": "https://github.com/vega/vega-datasets"
    }
  ],
  "title": "Vega Graph Spec Tutorial - US presidents",
  "views": [
    {
      "name": "demo-datapackage-vega",
      "resources": [
        "people",
        "events"
      ],
      "spec": {
        "axes": [
          {
            "scale": "x",
            "type": "x"
          }
        ],
        "data": [
          {
            "name": "people"
          },
          {
            "format": {
              "parse": {
                "when": "date"
              },
              "type": "json"
            },
            "name": "events"
          }
        ],
        "height": 200,
        "marks": [
          {
            "from": {
              "data": "events"
            },
            "properties": {
              "enter": {
                "angle": {
                  "value": -25
                },
                "fill": {
                  "value": "#000"
                },
                "font": {
                  "value": "Helvetica Neue"
                },
                "fontSize": {
                  "value": 10
                },
                "text": {
                  "field": "name"
                },
                "x": {
                  "field": "when",
                  "scale": "x"
                },
                "y": {
                  "value": -10
                }
              }
            },
            "type": "text"
          },
          {
            "from": {
              "data": "events"
            },
            "properties": {
              "enter": {
                "fill": {
                  "value": "#888"
                },
                "height": {
                  "field": {
                    "group": "height"
                  },
                  "offset": 8
                },
                "width": {
                  "value": 1
                },
                "x": {
                  "field": "when",
                  "scale": "x"
                },
                "y": {
                  "value": -8
                }
              }
            },
            "type": "rect"
          },
          {
            "from": {
              "data": "people"
            },
            "properties": {
              "enter": {
                "fill": {
                  "value": "#000"
                },
                "font": {
                  "value": "Helvetica Neue"
                },
                "fontSize": {
                  "value": 10
                },
                "text": {
                  "field": "label"
                },
                "x": {
                  "field": "born",
                  "scale": "x"
                },
                "y": {
                  "field": "label",
                  "offset": -3,
                  "scale": "y"
                }
              }
            },
            "type": "text"
          },
          {
            "from": {
              "data": "people"
            },
            "properties": {
              "enter": {
                "fill": {
                  "value": "#557"
                },
                "height": {
                  "value": 2
                },
                "x": {
                  "field": "born",
                  "scale": "x"
                },
                "x2": {
                  "field": "died",
                  "scale": "x"
                },
                "y": {
                  "field": "label",
                  "scale": "y"
                }
              }
            },
            "type": "rect"
          },
          {
            "from": {
              "data": "people"
            },
            "properties": {
              "enter": {
                "fill": {
                  "value": "#e44"
                },
                "height": {
                  "value": 4
                },
                "x": {
                  "field": "enter",
                  "scale": "x"
                },
                "x2": {
                  "field": "leave",
                  "scale": "x"
                },
                "y": {
                  "field": "label",
                  "offset": -1,
                  "scale": "y"
                }
              }
            },
            "type": "rect"
          }
        ],
        "scales": [
          {
            "domain": {
              "data": "people",
              "field": "label"
            },
            "name": "y",
            "range": "height",
            "type": "ordinal"
          },
          {
            "domain": {
              "data": "people",
              "field": [
                "born",
                "died"
              ]
            },
            "name": "x",
            "nice": "year",
            "range": "width",
            "round": true,
            "type": "time"
          }
        ],
        "width": 800
      },
      "specType": "vega",
      "title": "Lifelines of first 5 US presidents"
    }
  ]
}


You can use almost the same specifications inside spec attribute, that are used for setting the vega graphs. Only difference is that in data property, url and path attributes are moved out.

  ...
  "spec": {
    ...
    "data": [
      {
        "name": "people"
      },
      {
        "name": "events",
        ...
      }
    ],
  }

Instead we use name attribute to reference to a dataset. Note, how we created a new object inside data property - {"name": "people"} to reference it to resources (this names are identical to names of resources)

Outside of spec attribute there are some other important parameters to note:

Attribute Type Description
name String Unique identifier for view within list of views.
title String Title for the graph.
resources Array Data sources for this spec. It can be either resource name or index. By default it is the first resource.
specType String Available options: simple, vega, plotly (Required).

Import into your tool

In order to use Data Package in R follow instructions below:

install.packages("devtools")
library(devtools)
install_github("hadley/readr")
install_github("ropenscilabs/jsonvalidate")
install_github("ropenscilabs/datapkg")

#Load client
library(datapkg)

#Get Data Package
datapackage <- datapkg_read("https://pkgstore.datahub.io/examples/vega-views-tutorial-lines/latest")

#Package info
print(datapackage)

#Open actual data in RStudio Viewer
View(datapackage$data$"people")
View(datapackage$data$"events")

Tested with Python 3.5.2

To generate Pandas data frames based on JSON Table Schema descriptors we have to install jsontableschema-pandas plugin. To load resources from a data package as Pandas data frames use datapackage.push_datapackage function. Storage works as a container for Pandas data frames.

In order to work with Data Packages in Pandas you need to install our packages:

$ pip install datapackage
$ pip install jsontableschema-pandas

To get Data Package run following code:

import datapackage

data_url = "https://pkgstore.datahub.io/examples/vega-views-tutorial-lines/latest/datapackage.json"

# to load Data Package into storage
storage = datapackage.push_datapackage(data_url, 'pandas')

# to see datasets in this package
storage.buckets

# you can access datasets inside storage, e.g. the first one:
storage[storage.buckets[0]]

In order to work with Data Packages in Python you need to install our packages:

$ pip install datapackage

To get Data Package into your Python environment, run following code:

import datapackage

dp = datapackage.DataPackage('https://pkgstore.datahub.io/examples/vega-views-tutorial-lines/latest/datapackage.json')

# see metadata
print(dp.descriptor)

# get list of csv files
csvList = [dp.resources[x].descriptor['name'] for x in range(0,len(dp.resources))]
print(csvList) # ["resource name", ...]

# access csv file by the index starting 0
print(dp.resources[0].data)

To use this Data Package in JavaScript, please, follow instructions below:

Install datapackage using npm:

$ npm install [email protected]

Once the package is installed, use code snippet below


const Datapackage = require('datapackage').Datapackage

async function fetchDataPackageAndData(dataPackageIdentifier) {
  const dp = await new Datapackage(dataPackageIdentifier)
  await Promise.all(dp.resources.map(async (resource) => {
    if (resource.descriptor.format === 'geojson') {
      const baseUrl = resource._basePath.replace('/datapackage.json', '')
      const resourceUrl = `${baseUrl}/${resource._descriptor.path}`
      const response = await fetch(resourceUrl)
      resource.descriptor._values = await response.json()
    } else {
      // we assume resource is tabular for now ...
      const table = await resource.table
      // rows are simple arrays -- we can convert to objects elsewhere as needed
      const rowsAsObjects = false
      resource.descriptor._values = await table.read(rowsAsObjects)
    }
  }))

  // see the data package object
  console.dir(dp)

  // data itself is stored in Resource object, e.g. to access first resource:
  console.log(dp.resources[0]._values)

  return dp
}


fetchDataPackageAndData('https://pkgstore.datahub.io/examples/vega-views-tutorial-lines/latest/datapackage.json');

Our JavaScript is written using ES6 features. We are using node.js v7.4.0 and passing --harmony option to enable ES6:

$ node --harmony index.js

In order to work with Data Packages in SQL you need to install our packages:

$ pip install datapackage
$ pip install jsontableschema-sql
$ pip install sqlalchemy

To import Data Package to your SQLite Database, run following code:

import datapackage
from sqlalchemy import create_engine

data_url = 'https://pkgstore.datahub.io/examples/vega-views-tutorial-lines/latest/datapackage.json'
engine = create_engine('sqlite:///:memory:')

# to load Data Package into storage
storage = datapackage.push_datapackage(data_url, 'sql', engine=engine)

# to see datasets in this package
storage.buckets

# to execute sql command (assuming data is in "data" folder, name of resource is data and file name is data.csv)
storage._Storage__connection.execute('select * from data__data___data limit 1;').fetchall()

# description of the table columns
storage.describe('data__data___data')