Anu + Rufus

Anu + Rufus

Agenda

  • Review https://github.com/datopian/product/issues/57
  • What is tech roadmap?
    • What are "internal" products. See diagram below
      • Naming and purpose for internal tech stack ✅2023-01-30 see What are the internal products
      • datahub-publisher / git dms
        • datahub-publisher name? ✅2023-01-30 this is called git-dms
        • Why do Git DMS (and use it to power DataHub Enterprise)? Or, conversely, why not use Hybrid with CKAN backend?) ✅2023-01-30 see Why do Git DMS? (vs just use e.g. hybrid)
        • How does Git DMS differ from hybrid? 🚧2023-01-30 it likely differs at each level of the stack - see below.
          • How does it differ in features
          • How does it differ in components
        • Should it merge to monorepo? ✅2023-01-30 ❌ no wont merge for now. will rename the repo. If we start seeing a lot of reuse from monorepo stuff we can revisit this question
          • [name=Anu] - Depends, I'm not sure why would we need it in the monorepo? Do we want to re-use components? Maybe yes, eg, data-expolorer etc. but don't see any clear reason for it. I think we need to think about how we can re-use this for DataHub.io, eg, where v3 code would be and how we want to revamp it quickly?
      • Definition and naming for Hybrid
    • How do they relate to external products ✅2023-01-30 see diagram in What are the internal products
    • What's the critical path
      • what tech consolidation? (and how and where)
  • What live demos do we have?

What are the internal products

../Excalidraw/product-internal-apps-external-solutions-2023-01-29.excalidraw

What is our aim?

  • Have a competitive DMS offering (DataHub Enterprise)
  • Have a competitive OpenData (portal) offering
  • Build hybrid portals quicker

Why do Git DMS? (vs just use e.g. hybrid)

Create "Git DMS" with a new backend rather than use hybrid because:

  • Easier to setup
    • Github is already "set up for us"
    • DMS App is pure JS and easier to deploy
    • CKAN is difficult to do
  • Easier to maintain
  • Easier to develop/evolve
  • Important additional features are there (or can be added easily)
    • Versioning 🔥
    • Workflows for publishing especially review and moderation e.g. protected branches, two reviewers
    • CI/CD already provisioned (to an extent) with Actions
    • Rich permissioning already (partly) implemented e.g. protected branches
  • Builds off a platform git and github people are already using (your data team probably already use github). "Turn your Github into a DataHub"

Tech stack of hybrid

See also diagram above.

  • Hybrid (frontend):
    • Read UI: Portal.js
    • Publish/Editor UI: CKAN
    • Admin UI: CKAN
    • Backend (API): CKAN
  • Enterprise
    • Read UI: Portal.js (??)
    • Publish UI: new code?
    • Admin UI: n/a?
    • Backend API: Github

../Excalidraw/enterprise-ckan-and-hybrids-2023-01-30.excalidraw

Built with DataHub LogoDataHub Cloud