JunIBIS.be Data Package unlisted

andrejjh

Files Size Format Created Updated License Source
11 4MB csv zip 1 year ago 11 months ago Creative Commons Attribution 4.0 JunIBIS.be, website dedicated to mapping the Campaign of Belgium in June 1815
The Campaign of Belgium in 1815 marked the end of the Napoleonic wars. This dataset lists the armies and units in presence, their commanders and their hourly positions during the six days of campaign between June 15 and June 20. Maps and further details are available on JunIBIS.be. Why was dataset read more
Download Developers

Data Files

Download files in this dataset

File Description Size Last changed Download
events Events of the campaign: eg Battles. 269B csv (269B) , json (503B)
people People involved in the campaign, mostly commanders of units. 60kB csv (60kB) , json (159kB)
positions Hourly unit positions give the estimated position of each unit hour by hour. 1MB csv (1MB) , json (3MB)
schemes Groups of vocabulary terms. 345B csv (345B) , json (941B)
sources Sources of the information. 16kB csv (16kB) , json (32kB)
sourcesgroups Groups of source. Could be books, zines, games... 928B csv (928B) , json (2kB)
terms Vocabulary terms grouped into schemes. 6kB csv (6kB) , json (16kB)
uelinks These links indicate the presence of a unit at a certain event. 11kB csv (11kB) , json (51kB)
units Combat Units at various rank. 81kB csv (81kB) , json (340kB)
uplinks These links indicates the role of a person in a certain unit, mostly commanding. 12kB csv (12kB) , json (56kB)
junibis_dp_zip Compressed versions of dataset. Includes normalized CSV and JSON data with original data and datapackage.json. 788kB zip (788kB)

Events [events]  

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
id 1 integer (default) Event identifier
code 2 string (default) Event short name
kind 3 integer (default) Event kind, see vocabulary
begindate 4 datetime (%Y-%m-%d %H:%M:%S) Event starting datetime
enddate 5 datetime (%Y-%m-%d %H:%M:%S) Event ending datetime
source 6 integer (default) Source of information

People [people]  

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
id 1 integer (default) Person identifier
sha1 2 string (default) Person hash code
rank 3 integer (default) Person rank, see vocabulary
title 4 string (default) Person title, see vocabulary
firstname 5 string (default) Person firstname
lastname 6 string (default) Person lastname
nation 7 integer (default) Person nationality, see vocabulary
wikidata 8 string (default) Person wikidata
source 9 integer (default) Source of information

positions [positions]  

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
unitid 1 string (default) Unit identifier
icon 2 integer (default) Unit icon
army 3 integer (default) Unit army
step 4 integer (default) Time in hour since beginning of the campaign
datetime 5 datetime (%Y-%m-%d %H:%M:%S) Position datatime
geopos 6 geopoint (default) Position coordinates (longitude, latitude)

vocabulary schemes [schemes]  

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
id 1 integer (default) Scheme identifier
code 2 string (default) Scheme short name
name_de 3 string (default) Scheme German name
name_en 4 string (default) Scheme English name
name_fr 5 string (default) Scheme French name
name_nl 6 string (default) Scheme Dutch name

Sources [sources]  

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
id 1 integer (default) Source identifier
code 2 string (default) Source short name
kind 3 string (default) Source kind
sourcegroup 4 string (default) Source group short name
sourcegroup_id 5 integer (default) Source group identifier
reference 6 string (default) Full reference of the source
image 7 string (default) Source image
url 8 string (default) Source URL

Source groups. [sourcesgroups]  

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
id 1 integer (default) Source group identifier
code 2 string (default) Source group short name
name_de 3 string (default) Source group German name
name_en 4 string (default) Source group English name
name_fr 5 string (default) Source group French name
name_nl 6 string (default) Source group Dutch name

vocabulary terms [terms]  

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
id 1 integer (default) Term identifier
code 2 string (default) Term short name
scheme_code 3 string (default) Term scheme short name
scheme_id 4 integer (default) Term scheme identifier
name_de 5 string (default) Term German name
name_en 6 string (default) Term English name
name_fr 7 string (default) Term French name
name_nl 8 string (default) Term Dutch name

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
unit_id 1 integer (default) Unit identifier
event_id 2 integer (default) Event identifier
source 3 integer (default) Source of information

Units [units]  

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
id 1 integer (default) Unit identifier
code 2 string (default) Unit code
parent 3 string (default) Parent Unit code
parent_id 4 integer (default) Parent Unit identifier
absence 5 string (default) Yes indicates that this unit did not took part in the campaign.
name 6 string (default) Unit name in national language
bataillons 7 integer (default) Number of bataillons in unit
squadrons 8 integer (default) Number of squadrons in unit
batteries 9 integer (default) Number of batteries in unit
engineers 10 integer (default) Number of engineers in unit
ems 11 integer (default) Number of Etat Major staff in unit
officers 12 integer (default) Number of officers in unit
men 13 integer (default) Number of rank & files in unit
total 14 integer (default) Total number of men in unit
category 15 integer (default) Unit category, see vocabulary
rank 16 integer (default) Unit rank, see vocabulary
nation 17 integer (default) Unit nation, see vocabulary
icon 18 integer (default) Icon used to represent the unit
source 19 integer (default) Source of information

This is a preview version. There might be more data in the original version.

Field information

Field Name Order Type (Format) Description
unit_id 1 integer (default) Unit identifier
person_id 2 integer (default) Person identifier
role_id 3 integer (default) Person role in unit, see vocabulary
source 4 integer (default) Source of information

Integrate this dataset into your favourite tool

Use our data-cli tool designed for data wranglers:

data get https://datahub.io/andrejjh/junibis_dp
data info andrejjh/junibis_dp
tree andrejjh/junibis_dp
# Get a list of dataset's resources
curl -L -s https://datahub.io/andrejjh/junibis_dp/datapackage.json | grep path

# Get resources

curl -L https://datahub.io/andrejjh/junibis_dp/r/0.csv

curl -L https://datahub.io/andrejjh/junibis_dp/r/1.csv

curl -L https://datahub.io/andrejjh/junibis_dp/r/2.csv

curl -L https://datahub.io/andrejjh/junibis_dp/r/3.csv

curl -L https://datahub.io/andrejjh/junibis_dp/r/4.csv

curl -L https://datahub.io/andrejjh/junibis_dp/r/5.csv

curl -L https://datahub.io/andrejjh/junibis_dp/r/6.csv

curl -L https://datahub.io/andrejjh/junibis_dp/r/7.csv

curl -L https://datahub.io/andrejjh/junibis_dp/r/8.csv

curl -L https://datahub.io/andrejjh/junibis_dp/r/9.csv

curl -L https://datahub.io/andrejjh/junibis_dp/r/10.zip

If you are using R here's how to get the data you want quickly loaded:

install.packages("jsonlite", repos="https://cran.rstudio.com/")
library("jsonlite")

json_file <- 'https://datahub.io/andrejjh/junibis_dp/datapackage.json'
json_data <- fromJSON(paste(readLines(json_file), collapse=""))

# get list of all resources:
print(json_data$resources$name)

# print all tabular data(if exists any)
for(i in 1:length(json_data$resources$datahub$type)){
  if(json_data$resources$datahub$type[i]=='derived/csv'){
    path_to_file = json_data$resources$path[i]
    data <- read.csv(url(path_to_file))
    print(data)
  }
}

Note: You might need to run the script with root permissions if you are running on Linux machine

Install the Frictionless Data data package library and the pandas itself:

pip install datapackage
pip install pandas

Now you can use the datapackage in the Pandas:

import datapackage
import pandas as pd

data_url = 'https://datahub.io/andrejjh/junibis_dp/datapackage.json'

# to load Data Package into storage
package = datapackage.Package(data_url)

# to load only tabular data
resources = package.resources
for resource in resources:
    if resource.tabular:
        data = pd.read_csv(resource.descriptor['path'])
        print (data)

For Python, first install the `datapackage` library (all the datasets on DataHub are Data Packages):

pip install datapackage

To get Data Package into your Python environment, run following code:

from datapackage import Package

package = Package('https://datahub.io/andrejjh/junibis_dp/datapackage.json')

# print list of all resources:
print(package.resource_names)

# print processed tabular data (if exists any)
for resource in package.resources:
    if resource.descriptor['datahub']['type'] == 'derived/csv':
        print(resource.read())

If you are using JavaScript, please, follow instructions below:

Install data.js module using npm:

  $ npm install data.js

Once the package is installed, use the following code snippet:

const {Dataset} = require('data.js')

const path = 'https://datahub.io/andrejjh/junibis_dp/datapackage.json'

// We're using self-invoking function here as we want to use async-await syntax:
;(async () => {
  const dataset = await Dataset.load(path)
  // get list of all resources:
  for (const id in dataset.resources) {
    console.log(dataset.resources[id]._descriptor.name)
  }
  // get all tabular data(if exists any)
  for (const id in dataset.resources) {
    if (dataset.resources[id]._descriptor.format === "csv") {
      const file = dataset.resources[id]
      // Get a raw stream
      const stream = await file.stream()
      // entire file as a buffer (be careful with large files!)
      const buffer = await file.buffer
      // print data
      stream.pipe(process.stdout)
    }
  }
})()

Read me

The Campaign of Belgium in 1815 marked the end of the Napoleonic wars. This dataset lists the armies and units in presence, their commanders and their hourly positions during the six days of campaign between June 15 and June 20. Maps and further details are available on JunIBIS.be.

Why was dataset created?

Fan of Napoleonic history, especially passionate for the First Empire wars, and poured into the area of ​​digital mapping, I wanted to create interactive maps depicting the Campaign of Belgium of June 1815. I quickly received help from a longtime friend whose book knowledge wisely complement my computer skills. The abundant literature on the subject allowed us to compile the troops movements during the first days of the campaign.

The three founding principles of our project are: open data, free access and sources attribution. Our maps are available under CC-BY license that allows anyone «to copy, distribute and transmit the material by any means and in any format, to remix, transform and create from the equipment» under the attribution term : you must give appropriate credit and indicate if changes where made.

The interactive maps result of a meticulous work of analysis and compilation from many sources.

When was it collected?

Dataset was assembled between Spring and Autumn 2015 when we developed the website.

Which instruments used to collect it?

We did use open source software (Postgresql and QGIS) and did developed some SQL scripts and Ruby programs. Carto is used to deliver all our final maps.

What does “null” mean?

Null values mostly indicated not applicable, rarely unknown.

Other comments

The units positions used in the interactive maps result of a meticulous work of analysis and compilation of many different sources. These referenced positions were than interpolated to generate hourly positions.

If you have questions on this data package …just drop me an email.

Andre Heughebaert and Eric Iven

Datapackage.json