CKAN Enterprise

CKAN Enterprise

Introduction

CKAN Enterprise is our name for what we plan would become our standard "base" distribution for CKAN going forward:

  • It is a CKAN standard code base with micro-services.
  • Enterprise grade data catalog and portal targeted at Gov (open data portals) and Enterprise (Data Catalogs +).
  • It is also known as Datopian DMS.

Roadmap 2021 and beyond

CurrentCKAN Enterprise
Raw storageFilestoreGiftless
Data Loader (db)DataPusher extensionAircan
Data Storage (db)PostgresAny database engine. By default, Postgres
Data API (read)Built-in DataStore extension's API including SQL endpointGraphQL based standalone micro-service
Frontend (public)Build-in frontend into CKAN Classic python app (some projects are using nodejs app)PortalJS or nodejs app
Data ExplorerReclineJS (some projects that uses nodejs app for frontend have React based Data Explorer)GraphQL based Data Explorer
AuthTraditional login/password + extendable with CKAN Classic extensionsSSO with default Google, Github, Facebook and Microsoft options
PermissionsCKAN Classic based permissionsExisting permissions exposed via JWT based authz API

Timeline 2021

To develop a base distribution of CKAN Enterprise, we want to build a demo project with the features from the roadmap. This way we can:

  • understand its advantages/limitations;
  • compare against other instances of CKAN;
  • demonstrate for the potential clients.

High level overview of the planned features with ETA:

NameDescriptionEffortETA
InitSelect CKAN version and deploy to DXxsQ2
BlobstoreIntegrate Giftless for raw storagesQ2
VersioningDevelop/integrate new versioning syslQ3
DataLoaderDevelop/integrate AircanxlQ3
Data APIIntegrate new Data API (read)mQ2
FrontendBuild a theme using PortalJSsQ2
DataExplorerIntegrate into PortalJSsQ2
PermissionsDevelop permissions in read frontendmQ4
AuthIntegratesQ4

Init

Initialize a new project for development of CKAN Enterprise.

Tasks:

  • Boot project in Datopian-DX cluster
  • Use CKAN v2.8.x (latest patch) or 2.9.x
  • Don't setup DataPusher
  • Namespace: ckan-enterprise
  • Domain: enterprise.ckan.datopian.com

Blobstore

See blob storage

Versioning

See versioning

DataLoader

See DataLoader

Data API

  • Install new Data API service in the project
  • Install Hasura service in the project
  • Set it up to work with DB of CKAN Enterprise
  • Read more about Data API here

Notes:

  • We could experiment and use various features of Hasura, eg:
    • Setting up row/column limits per user role (permissions)
    • Subscriptions to auto load new data rows

Frontend

PortalJS for the read frontend of CKAN Enterprise. Read more.

DataExplorer

A new Data Explorer based on GraphQL API: https://github.com/datopian/data-explorer-graphql

Permissions

See permissions.

Auth

Next generation, Kratos based, authentication (mostly SSO with no Traditional login by default) with following options out of the box:

  • GitHub
  • Google
  • Facebook
  • Microsoft

Easy to add:

  • Discord
  • GitLab
  • Slack
Built with DataHub LogoDataHub Cloud