Vega Graph Spec Tutorial - US Airports

examples

Files Size Format Created Updated License Source
4 3MB csv zip topojson 2 weeks ago vega-datasets
This is an example dataset that demonstrates how to build visualizations using the "Vega Graph Spec". We are using Flight connections of the airports in the US - one of the examples from vega editor - and displaying it here, on DataHub with small modifications in vega-spec. Views We assume that read more
Download

Data Files

File Description Size Last changed Download Other formats
flights-airport [csv] 69kB flights-airport [csv] flights-airport [json] (295kB)
airports [csv] 209kB airports [csv] airports [json] (492kB)
datapackage_zip [zip] Compressed versions of dataset. Includes normalized CSV and JSON data with original data and datapackage.json. 629kB datapackage_zip [zip]
states [topojson] 627kB states [topojson]

flights-airport  

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
origin 1 string
destination 2 string
count 3 number

airports  

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
iata 1 any
name 2 string
city 3 string
state 4 string
country 5 string
latitude 6 number
longitude 7 number

datapackage_zip  

This is a preview version. There might be more data in the original version.

states  

This is a preview version. There might be more data in the original version.

Read me

This is an example dataset that demonstrates how to build visualizations using the “Vega Graph Spec”. We are using Flight connections of the airports in the US - one of the examples from vega editor - and displaying it here, on DataHub with small modifications in vega-spec.

Views

We assume that you are familiar with what datapackage.json is and its specifications.

To create graphs for your tabular data, the datapackage.json should include the views attribute that is used for visualizations.

If you are familiar with Vega specifications, you probably would like to use all its features. To use it, inside views you should set specType to "vega" and define some graph specifications in spec property. See below for more details.

Vega Graph Specifications

This is the views property that is used for this page:

{
  ...,
  "views": [
    {
      "name": "airports-map-interactive",
      "title": "Airports connections in the US",
      "resources": ["states", 1, 2],
      "specType": "vega",
      "spec": {
        "$schema": "https://vega.github.io/schema/vega/v3.0.json",
        "width": 900,
        "height": 560,
        "padding": {
          "top": 25,
          "left": 0,
          "right": 0,
          "bottom": 0
        },
        "autosize": "none",
        "signals": [
          {
            "name": "scale",
            "value": 1200,
            "bind": {
              "input": "range",
              "min": 500,
              "max": 3000
            }
          },
          {
            "name": "translateX",
            "value": 450,
            "bind": {
              "input": "range",
              "min": -500,
              "max": 1200
            }
          },
          {
            "name": "translateY",
            "value": 260,
            "bind": {
              "input": "range",
              "min": -300,
              "max": 700
            }
          },
          {
            "name": "shape",
            "value": "line",
            "bind": {
              "input": "radio",
              "options": [
                "line",
                "curve"
              ]
            }
          },
          {
            "name": "hover",
            "value": null,
            "on": [
              {
                "events": "@cell:mouseover",
                "update": "datum"
              },
              {
                "events": "@cell:mouseout",
                "update": "null"
              }
            ]
          },
          {
            "name": "title",
            "value": "U.S. Airports, 2008",
            "update": "hover ? hover.name + ' (' + hover.iata + ')' : 'U.S. Airports, 2008'"
          },
          {
            "name": "cell_stroke",
            "value": null,
            "on": [
              {
                "events": "dblclick",
                "update": "cell_stroke ? null : 'brown'"
              },
              {
                "events": "mousedown!",
                "update": "cell_stroke"
              }
            ]
          }
        ],
        "data": [
          {"name": "flights-airport"},
          {
            "name": "states",
            "format": {
              "type": "topojson",
              "feature": "states"
            },
            "transform": [
              {
                "type": "geopath",
                "projection": "projection"
              }
            ]
          },
          {
            "name": "traffic",
            "source": "flights-airport",
            "format": {
              "type": "csv",
              "parse": "auto"
            },
            "transform": [
              {
                "type": "aggregate",
                "groupby": [
                  "origin"
                ],
                "fields": [
                  "count"
                ],
                "ops": [
                  "sum"
                ],
                "as": [
                  "flights"
                ]
              }
            ]
          },
          {
            "name": "airports",
            "format": {
              "type": "csv",
              "parse": "auto"
            },
            "transform": [
              {
                "type": "lookup",
                "from": "traffic",
                "key": "origin",
                "fields": [
                  "iata"
                ],
                "as": [
                  "traffic"
                ]
              },
              {
                "type": "filter",
                "expr": "datum.traffic != null"
              },
              {
                "type": "geopoint",
                "projection": "projection",
                "fields": [
                  "longitude",
                  "latitude"
                ]
              },
              {
                "type": "filter",
                "expr": "datum.x != null && datum.y != null"
              },
              {
                "type": "voronoi",
                "x": "x",
                "y": "y"
              },
              {
                "type": "collect",
                "sort": {
                  "field": "traffic.flights",
                  "order": "descending"
                }
              }
            ]
          },
          {
            "name": "routes",
            "source": "flights-airport",
            "format": {
              "type": "csv",
              "parse": "auto"
            },
            "transform": [
              {
                "type": "filter",
                "expr": "hover && hover.iata == datum.origin"
              },
              {
                "type": "lookup",
                "from": "airports",
                "key": "iata",
                "fields": [
                  "origin",
                  "destination"
                ],
                "as": [
                  "source",
                  "target"
                ]
              },
              {
                "type": "filter",
                "expr": "datum.source && datum.target"
              },
              {
                "type": "linkpath",
                "shape": {
                  "signal": "shape"
                }
              }
            ]
          }
        ],
        "projections": [
          {
            "name": "projection",
            "type": "albersUsa",
            "scale": {
              "signal": "scale"
            },
            "translate": [
              {
                "signal": "translateX"
              },
              {
                "signal": "translateY"
              }
            ]
          }
        ],
        "scales": [
          {
            "name": "size",
            "type": "linear",
            "domain": {
              "data": "traffic",
              "field": "flights"
            },
            "range": [
              16,
              1000
            ]
          }
        ],
        "marks": [
          {
            "type": "path",
            "from": {
              "data": "states"
            },
            "encode": {
              "enter": {
                "fill": {
                  "value": "#dedede"
                },
                "stroke": {
                  "value": "white"
                }
              },
              "update": {
                "path": {
                  "field": "path"
                }
              }
            }
          },
          {
            "type": "symbol",
            "from": {
              "data": "airports"
            },
            "encode": {
              "enter": {
                "size": {
                  "scale": "size",
                  "field": "traffic.flights"
                },
                "fill": {
                  "value": "steelblue"
                },
                "fillOpacity": {
                  "value": 0.8
                },
                "stroke": {
                  "value": "white"
                },
                "strokeWidth": {
                  "value": 1.5
                }
              },
              "update": {
                "x": {
                  "field": "x"
                },
                "y": {
                  "field": "y"
                }
              }
            }
          },
          {
            "type": "path",
            "name": "cell",
            "from": {
              "data": "airports"
            },
            "encode": {
              "enter": {
                "fill": {
                  "value": "transparent"
                },
                "strokeWidth": {
                  "value": 0.35
                }
              },
              "update": {
                "path": {
                  "field": "path"
                },
                "stroke": {
                  "signal": "cell_stroke"
                }
              }
            }
          },
          {
            "type": "path",
            "interactive": false,
            "from": {
              "data": "routes"
            },
            "encode": {
              "enter": {
                "path": {
                  "field": "path"
                },
                "stroke": {
                  "value": "black"
                },
                "strokeOpacity": {
                  "value": 0.35
                }
              }
            }
          },
          {
            "type": "text",
            "interactive": false,
            "encode": {
              "enter": {
                "x": {
                  "value": 895
                },
                "y": {
                  "value": 0
                },
                "fill": {
                  "value": "black"
                },
                "fontSize": {
                  "value": 20
                },
                "align": {
                  "value": "right"
                }
              },
              "update": {
                "text": {
                  "signal": "title"
                }
              }
            }
          }
        ]
      }
    }
  ]
}

You can use almost the same specifications inside spec attribute, that are used for setting the Vega graphs. Only difference is that in data property, url and path attributes are moved out. Instead we use name attribute to reference to a resource. Note, how we created a new object inside data property - {"name": "flights-airport"} to reference it to a resource (this names are identical to names of resources):

  ...
  "spec": {
    ...
    "data": [
      {
        "name": "flights-airport"
      }
      ...
    ]
  }

Outside of spec attribute there are some other important attributes to note:

Attribute Type Description
name String Unique identifier for view within list of views.
title String Title for the graph.
resources Array Data sources for this spec. It can be either resource name or index. By default it is the first resource.
specType String Available options: simple, vega, plotly (Required).

Import into your tool

If you are using R here's how to get the data you want quickly loaded:

install.packages("jsonlite")
library("jsonlite")

json_file <- "http://datahub.io/examples/vega-views-tutorial-topojson/datapackage.json"
json_data <- fromJSON(paste(readLines(json_file), collapse=""))

# access csv file by the index starting from 1
path_to_file = json_data$resources[[1]]$path
data <- read.csv(url(path_to_file))
print(data)

In order to work with Data Packages in Pandas you need to install the Frictionless Data data package library and the pandas extension:

pip install datapackage
pip install jsontableschema-pandas

To get the data run following code:

import datapackage

data_url = "http://datahub.io/examples/vega-views-tutorial-topojson/datapackage.json"

# to load Data Package into storage
storage = datapackage.push_datapackage(data_url, 'pandas')

# data frames available (corresponding to data files in original dataset)
storage.buckets

# you can access datasets inside storage, e.g. the first one:
storage[storage.buckets[0]]

For Python, first install the `datapackage` library (all the datasets on DataHub are Data Packages):

pip install datapackage

To get Data Package into your Python environment, run following code:

from datapackage import Package

package = Package('http://datahub.io/examples/vega-views-tutorial-topojson/datapackage.json')

# get list of resources:
resources = package.descriptor['resources']
resourceList = [resources[x]['name'] for x in range(0, len(resources))]
print(resourceList)

data = package.resources[0].read()
print(data)

If you are using JavaScript, please, follow instructions below:

Install data.js module using npm:

  $ npm install data.js

Once the package is installed, use the following code snippet:

const {Dataset} = require('data.js')

const path = 'http://datahub.io/examples/vega-views-tutorial-topojson/datapackage.json'

// We're using self-invoking function here as we want to use async-await syntax:
(async () => {
  const dataset = await Dataset.load(path)

  // Get the first data file in this dataset
  const file = dataset.resources[0]
  // Get a raw stream
  const stream = await file.stream()
  // entire file as a buffer (be careful with large files!)
  const buffer = await file.buffer
})()

Install the datapackage library created specially for Ruby language using gem:

gem install datapackage

Now get the dataset and read the data:

require 'datapackage'

path = 'http://datahub.io/examples/vega-views-tutorial-topojson/datapackage.json'

package = DataPackage::Package.new(path)
# So package variable contains metadata. You can see it:
puts package

# Read data itself:
resource = package.resources[0]
data = resource.read
puts data
Datapackage.json