Corruption Perceptions Index

core

Files Size Format Created Updated License Source
1 61kB csv
corruption perceptions index the data is sourced from transparency international. > the corruption perceptions index (cpi) ranks countries/territories in terms of the degree to which corruption is perceived to exist among public officials and politicians. it draws on different assessments and read more
Download

Data Files

data  

Field information

Field Name Order Type (Format) Description
Jurisdiction 1 string (default)
1998 2 string (default)
1999 3 string (default)
2000 4 string (default)
2001 5 string (default)
2002 6 string (default)
2003 7 string (default)
2004 8 string (default)
2005 9 string (default)
2006 10 string (default)
2007 11 string (default)
2008 12 string (default)
2009 13 string (default)
2010 14 string (default)
2011 15 string (default)
2012 16 string (default)
2013 17 string (default)
2014 18 string (default)
2015 19 string (default)

Read me

corruption perceptions index

the data is sourced from transparency international.

the corruption perceptions index (cpi) ranks countries/territories in terms of the degree to which corruption is perceived to exist among public officials and politicians. it draws on different assessments and business opinion surveys carried out by independent and reputable institutions. it captures information about the administrative and political aspects of corruption. broadly speaking, the surveys and assessments used to compile the index include questions relating to bribery of public officials, kickbacks in public procurement, embezzlement of public funds, and questions that probe the strength and effectiveness of public sector anti-corruption efforts.

more info here.

requires:

  1. r - rvest, xlsx
  2. java 8
  3. julia - gadfly, dataframes

note: the scale of the cpi is 0-10 from 1998 to 2011, and 0-100 from 2012 onwards, due to an update to the methodology used to calculate the cpi in 2012.

data in data/ generated by:

$ ./acquire_data.r

or, for the paranoid:

$ torify ./acquire_data.r

acquire_data.r downloads files from transparency international, converts them to csv format, and mergesthem in cpi.csv file.

warning: the files are not at all curated well. many countries are spelled different ways in each annual report, so the scripts will count them as different countries.

Import into your tool

In order to use Data Package in R follow instructions below:

install.packages("devtools")
library(devtools)
install_github("hadley/readr")
install_github("ropenscilabs/jsonvalidate")
install_github("ropenscilabs/datapkg")

#Load client
library(datapkg)

#Get Data Package
datapackage <- datapkg_read("https://pkgstore.datahub.io/core/corruption-perceptions-index/latest")

#Package info
print(datapackage)

#Open actual data in RStudio Viewer
View(datapackage$data$"data")

Tested with Python 3.5.2

To generate Pandas data frames based on JSON Table Schema descriptors we have to install jsontableschema-pandas plugin. To load resources from a data package as Pandas data frames use datapackage.push_datapackage function. Storage works as a container for Pandas data frames.

In order to work with Data Packages in Pandas you need to install our packages:

$ pip install datapackage
$ pip install jsontableschema-pandas

To get Data Package run following code:

import datapackage

data_url = "https://pkgstore.datahub.io/core/corruption-perceptions-index/latest/datapackage.json"

# to load Data Package into storage
storage = datapackage.push_datapackage(data_url, 'pandas')

# to see datasets in this package
storage.buckets

# you can access datasets inside storage, e.g. the first one:
storage[storage.buckets[0]]

In order to work with Data Packages in Python you need to install our packages:

$ pip install datapackage

To get Data Package into your Python environment, run following code:

import datapackage

dp = datapackage.DataPackage('https://pkgstore.datahub.io/core/corruption-perceptions-index/latest/datapackage.json')

# see metadata
print(dp.descriptor)

# get list of csv files
csvList = [dp.resources[x].descriptor['name'] for x in range(0,len(dp.resources))]
print(csvList) # ["resource name", ...]

# access csv file by the index starting 0
print(dp.resources[0].data)

To use this Data Package in JavaScript, please, follow instructions below:

Install datapackage using npm:

$ npm install [email protected]

Once the package is installed, use code snippet below


const Datapackage = require('datapackage').Datapackage

async function fetchDataPackageAndData(dataPackageIdentifier) {
  const dp = await new Datapackage(dataPackageIdentifier)
  await Promise.all(dp.resources.map(async (resource) => {
    if (resource.descriptor.format === 'geojson') {
      const baseUrl = resource._basePath.replace('/datapackage.json', '')
      const resourceUrl = `${baseUrl}/${resource._descriptor.path}`
      const response = await fetch(resourceUrl)
      resource.descriptor._values = await response.json()
    } else {
      // we assume resource is tabular for now ...
      const table = await resource.table
      // rows are simple arrays -- we can convert to objects elsewhere as needed
      const rowsAsObjects = false
      resource.descriptor._values = await table.read(rowsAsObjects)
    }
  }))

  // see the data package object
  console.dir(dp)

  // data itself is stored in Resource object, e.g. to access first resource:
  console.log(dp.resources[0]._values)

  return dp
}


fetchDataPackageAndData('https://pkgstore.datahub.io/core/corruption-perceptions-index/latest/datapackage.json');

Our JavaScript is written using ES6 features. We are using node.js v7.4.0 and passing --harmony option to enable ES6:

$ node --harmony index.js

In order to work with Data Packages in SQL you need to install our packages:

$ pip install datapackage
$ pip install jsontableschema-sql
$ pip install sqlalchemy

To import Data Package to your SQLite Database, run following code:

import datapackage
from sqlalchemy import create_engine

data_url = 'https://pkgstore.datahub.io/core/corruption-perceptions-index/latest/datapackage.json'
engine = create_engine('sqlite:///:memory:')

# to load Data Package into storage
storage = datapackage.push_datapackage(data_url, 'sql', engine=engine)

# to see datasets in this package
storage.buckets

# to execute sql command (assuming data is in "data" folder, name of resource is data and file name is data.csv)
storage._Storage__connection.execute('select * from data__data___data limit 1;').fetchall()

# description of the table columns
storage.describe('data__data___data')
Datapackage.json