2024-09-06

2024-09-06

Present: Daniela, Rufus, Anu, Ola

Agenda:

Our focus

  • cohort retention (without this, probably there is no point in monetisation)
  • fixing broken datasets (top X number of sites)
  • publishing new datasets, as our main marketing focus
  • SEO improvements, so that we increase traffic

Issue tracking

  • We get back to working in sprints

Infolytics client meeting

Ola's questions:

  • Do we propose them a custom layout managed by us? Or do we focus on making the layout more customisable and building a customisable Hero component, that could benefit all of the users as well (or at least premium users if we decide to monetise these features)?

2024-08-30 Products weekly meeting

Agenda

Ola's questions:

  • how will the changes look like in practice

Notes:

  • What is our main objective for Sep/Oct onwards?
    • Anu: maybe we should select 1 single figure? I propose the following order:
      • For publisher:
        • 1. Number of sites.
          1. Number of signed up users.
      • For readers:
          1. Total unique visitors. (we assume it increases as UGC improves).

          1. Top 10-20 sites by unique visitors.

Next Actions

Summary

  • Daniela evolving towards overall Datopian "CEO" with implicit CEO in DH Cloud reflecting DH Cloud as one major item in the Datopian portfolio. (she will reflect more and this is a natural evolution of where things have been).
    • What this means (v rough first draft): continuing evolution to have more time and focus on areas like strategy, high level relationships and team development (e.g. developings PMs rather than PM'ing herself)
    • Re DH Cloud this means that (after vacation) Daniela will likely attend weeklies on a regular basis (maybe every two weeks) and helps review objectives etc
  • Anu steps in as product owner on DH cloud and leads on technical and evangelism directions day to day and in weeklies

23 Aug 20234 Sprint Agenda

Agenda

  • Check in
  • Updates from Anu
  • Updates from Datahub Cloud team
    • Objectives for next 2m https://github.com/datopian/datahub-next/issues/524
    • Sales funnel in GA
      • 16% of signups are creating a site
      • A very high % (need to double-check) create a template-based site and drop. These are the users that we can convert easily. How do we best do that? What's a good UX for when someone comes to the dashboard?
  • Next actions
  • AOBs
    • FedEx?

Updates from Anu

FIND project will have a lot of git-based content. They want to do markdown and github and this is already being kicked off.

  • The difference will be that they want to have validation in the frontmatter (has to partially follow the dataset metadata schema from ckan). Why? Because we want users to be able to do faceted search. If eg. they are interested in a partiuclar topic, they should be able to see all datasets and data stories related to this topic (so this field should be in the frontmatter)

TDC (transport data commons) They have some static content where we will try to use git-based approach as well. They're also a heavy user of Github

Anu: Can we put logo or reference in the footer? YES!!!!!!! sth like this https://datahub.io/@Daniellappv/glucose-matters-2 built with Dh Cloud bottom right

Ola: Create an event in GA to track how many people are coming from these sources. Maybe via UTM tracker.

Updates from DataHub Cloud team

https://analytics.google.com/analytics/web/#/p328790421/reports/explorer?params=_u..nav%3Dmaui%26_u.comparisonOption%3Ddisabled%26_u.date00%3D20240731%26_u.date01%3D20240823&r=8536867671&ruid=F4654B46-5302-4FF7-A033-1BA5E57E226F&collectionId=4636495263

User site's layout improvements:

  • Added "Created by xxx" in the footer
  • Added links to source GitHub repo
  • Added "Edit this page" button
  • Added support for social links in the footer and header
  • Added a way to specifically include or exclude folders/files from publishing (as well excluding single files by marking them as drafts)
  • Added support for integrating your own GA key
  • Replaced footer's "Built with DataHub Cloud" text to a floating, always visible button
  • Encoded space in file name as + singls in the URL
  • Added social share shortcut/dropdown to dataset pages
  • Added JSON-LD for better dataset indexing
  • Fixed 500 status codes returend on initial request for all dataset pages

Datahub.io website improvements:

  • Replaced awkward showcase carousel with showcase galery

Workflows:

  • Agreed to have a "Share" phase after each "Implementation"
  • Agreed to come back to iterations-based workflow (2 week sprints) starting next week

Notes from weekly:

  • Anu's feedback:
    • Re Feedback. A lot of people don't give great feedback. We should better do A/B tests. Feedback form isn't that great. And how would you filter, eg. if you receive 100 requests for features from users?
    • Importance of user-generated content
    • When publishing with DH Cloud, I want to have a way to have a tool for debugging or a tool to see what I have done / see if my component is correct and how it will render so that I can deliver a great dataset page quickly and easily.
    • Google Chrome plugin
    • Or Obsidian Plugin
    • VSCode extension (like evidence.dev)

Daniela: Idea of finding some cool datasets in Github and reaching out to the users to enrol them in publishing those with DHC

  • Anu thinks is a good idea. Good starting place: github.com/datasets, check the contributors
  • Reddit is a good starting point

2024-08-09 Products weekly meeting

Agenda

  • Check in
  • Updates from Anu
    • AIR project got kicked off and even closed
    • Australian Genome Facility POC
    • GIZ
    • Booting up in Aug (contract signature): Switzerland mobility data portal
    • Submitted RFPs, one of which eg. Montreal
    • Other prospects under negotiation
    • Migrating from Gitlab to Github. Easier to follow the pipeline
    • FedEx - fetching data already, now adding postcodes data (thinking about creating a product out of it + use DataHub Cloud?)
  • Updates from Datahub Cloud team

Notes

Why migrating from Gitlab to Github?

  • Agreed tool for company-level operations
  • Managing two accounts in two places (eg. notifications) doesn't work for Anu and it is not very efficient
  • Personal reasons: want to do Github boards, Github projects (not very flexible)
  • Easily onboarding for other people, eg. Joao for contributing to RFPs
  • Not much difference for eg. Daniela or the bizdev team itself

FedEx:

  • we built one version before using PortalJS as part of Yedige's onboarding (an year ago) but not in a good state. We want to re-try with DH Cloud
  • Team: Yedige was working on postal codes. Darkhan is one of the people involved (he will be owning/building it with our help).

Anu: not a lot of people would come to datahub.io and do searching… people find datasets from search engines like Google and they would go directly to the dataset page. Your search engine wouldn't be that great anyway. They would go to a better search engine. 1 or 2 years ago, if you looked at datahub traffic was country codes dataset. People just go there, download it and leave it. And generally, having a search page on the site is a nice-to-have but normally is not that useful because they use other search engines.

Building a search page is one option/hypothesis. Based on the previous experience, not the best way. Instead, focus on making sure SEO is working properly. It's possible and quick to find through search engines. Writing content, using it.

  • Maybe also Google Dataset Search (tags)

Rufus: being clear what we want to do. Are we trying to increase the amount of people who are looking for data? Are we trying to make the data publishers pay? Etc. Rufus: sitemap. How is Google even finding us atm based on our landing page.

2 parts

  • what is our product - is it data marketplace or data publishing etc
  • if we want traffic etc then how do we get that

Even third question: how does qu 1 relate to qu 2

Quality traffic: most of the old visitors would just get csv and leave without coming back. We need active users.

How do we do that?

  • Rufus: Sitemap.xml and good SEO?

Superso is offering a feature for setting up SEO and increasing the ranking in google.

Option to enable Google Analytics. Is my dataset being used?

If I am building a data portal, then it'd be even more important.

Built-in analytics for an off-the-shelf product. Not GA key unless self-hosted.

Rufus: To be clear I think showing llthinga like number of downloads and number of views on dataset showcase is useful.

And having something in dashboard is nice. Just that it is a bunch of work and increases our maintenance surface Vs just giving them Google analytics key to fill in.

2024-07-19 Products weekly meeting

Agenda

See content: https://docs.google.com/document/d/1CRisABi2yPx7F5xPDjm4BDUB8fGIS3EbIr4dExd5nhQ/edit?tab=t.0#heading=h.p4pwm0ljps7o Outcome: https://datahub.io/pricing

Pricingpagev01.png

2024-07-05 Products weekly session

Agenda

  • Check in
  • Standup
  • Parking lot
    • PKM/Dataset offering - Syncing last changes made on datahub.io (and going fwd?)
      • Daniela's Vercel deployments
    • Bug with file names of Obsidian vault quite important

2024-06-21 - Weekly team meeting

Present: Rufus, Daniela (both briefly), Ola, Yedige, William, Luis

Agenda

  • Brief review of where new joiners are at and next steps (e.g. set up issue for starters to get infrastructure set up)
  • Rufus and Daniela continued their planning

Where are you at

Name, Emoji and then some bullet points 😄

  • William 🍎
    • Working on issue location and 404s https://github.com/datopian/datahub/issues/1176
    • Working on datahub-next local instance setup
    • Having issues with database (any command to setup database locally?) Suggest opening an issue in datahub-next for your setup try out and reporting issues in that
  • Luis emoji 😉
    • Shaping up issue (table of contents disaligned #1174 and pdf reader not working), regarding problem, appetite, etc.
    • PortalJS installed but setting up Datahub, a problem with some enviroment variable, will contact Ola about it and solve it later today suggest that we have one issue for people setting up DataHub Cloud locally - merge with William's one and then comment there 👍
    • Current issue: https://github.com/orgs/datopian/projects/47/views/1?pane=issue&itemId=66276410 can you direct link the issue
    • Add build instructions later on
  • Yedige 🥶
  • Ola 🌞
    • Working on large repos support, I think I'm halfway through it
    • Announced GitHub auto-sync

2024-06-14 Products weekly sync

Agenda

  • Check in and meeting intention
  • Updates from Anu
    • Kicking off a new project AIR (non-profit and it is about budget data, small POC/MVP using Frictionless data) - quite similar to Openspending // we will use PortalJS for the frontend as a TEMPLATE so they can provide it to various countries to reuse/repurpose // Everything will be open // we won't do CKAN but purely git-based catalog
    • We have a new (paying) subscriber for PortalJS Cloud - some government district within London (they have been using CKAN in their AWS but not very successful). We offered multiple options but they didn't have budget –> they decided to go for PortalJS Cloud
    • We have some other projects in the pipeline but they won't start within the next 2 weeks
  • Updates from Datahub Cloud team
    • Datahub.io immediate access is there https://datahub.io/
    • A lot of team dynamics:
      • Oleg is not joining Datopian
      • We gave Mohamed notice
      • Ola gave us notice (she can stay until we have someone to hand over to)
    • Portaljs / Datahub etc. naming
      • Portaljs –> Datahub Portaljs (to reduce both internal and external confusion)
      • Datahub.io –> Datahub Cloud
      • Portaljs.org –> Datahub OpenSource
      • Portaljs.com –> Datahub PortalJS
    • Most issues are in a public tracking repo now unless sensitive https://github.com/datopian/datahub
    • Ongoing atm
      • We need to hire people (at least 2)
        • Anu: We have some internal resource:
          • eg. Luis who is junior and doing Phd atm (he joined as python dev but maybe he can do javascript?)
          • Yedige is another option (though python engineer again)
          • Darkhan is another option (will be engaged on AIR as a data engineer but he is experienced as a Product Owner)
      • I onboarded all Datopians and they have been playing with the app and dropping their feedback here https://github.com/datopian/datahub/issues/1162 (ongoing)
      • Auto-sync and support for large repos - Ola
  • Backlog for Ola
    • Autosync implementation
    • Support for large repos implementation
    • Shaping of metastore 3.3 blah blah to be picked up with Rufus
    • End-to-end tests improvement

Notes

Datahub and PortalJS

Anu's suggestion:

  • PortalJS is just a framework. Opensource/Framework to talk about portaljs.org
  • Managed option (portaljs.com)

D: Let's rather center it around the actual need or solution (data publishing, data portals, etc.)

What we shipped:

  • sidebar with sitemap
  • table of contents
  • basic site config support (title, nav bar, logo)
  • custom css support
  • 🚧 github auto-sync
  • 🚧 support for large repos

2024-05-31 Products weekly sync

Present: Daniela, Yoana, Anu, Rufus

Agenda

  • Bizdev updates
    • Quinn guy any news?
  • Team updates
    • Ola leaving
    • Oleg eventually getting onboarded
  • Other updates
  • When will the portaljs.org docs be moved to Datahub Cloud?
  • 2 Questions atm:
    • What would make sense for us to do given the bizdev need? eg. How much are we continuing with our SaaS push B2C?
  • Next actions:
    • Hire a new dev

Notes:

Bizdev pipeline:

  • Kicked off FIND
  • About to kick off 2 other small projects (similar to Openspending) in June. They wanted CKAN but don't have budget for CKAN so we'll do an MVP-like approach similar to Openspending
  • Another branch of NHS NI that we are very close to closing

Anu's opinion on Datahub Cloud (? was it cloud):

  • Great to have a product that is B2C and demoable that is easy to use and could be another tool in our data mesh
  • We can use it in our marketing

[Rufus] Note re data mesh Solutions: Old DataHub landing page with vision of it as a solution more broadly

Daniela update

  • Done more Onboarding
  • Done A/B testing
  • Onboarding Datopians - has been going well
  • Features
    • Sidebar ToC
    • File navigation
    • …

Anu's thoughts

Thoughts re DataHub

  • DataHub Framework/Toolkit: the whole kaboodle
    • DataHub PortalJS (just the portal framework - frontend). subpart of the Framework
  • DataHub Cloud
    • Basic
    • Enterprise

Rufus ideas re Cloud dashboard

DataHub Cloud dashboard gives a couple of flows

  • Publish data
  • Publish data story
  • Publish Obsidian
  • Publish markown-based website

DataHub Product "Lines"

  • Markdown-based (Github-based) Wiki publishing (80%)
  • Markdown-based PKM publishing
  • Markdown-based data publishing
  • Markdown-based guides

2024-05-24 Products weekly sync

Present: Ola, Yoana, Daniela, (Rufus)

Agenda

  • Quick "standup"
  • Do we need input from Rufus
    • Daniela: Share info / outcome / actions agreed with Oleg yesterday and get feedback
    • Ola: Confirm the problem we're trying to solve / expected outcome of the planned shaping we started around "moving away from data package" (also, is it really "moving away" or just inferring parts of it that can be inferred from GitHub…)
  • Do we need to cowork on sth? No
  • Is there anything else for today?
    • What Ola needed Daniela for 5m yesterday

2024-05-17 Products weekly meeting

Present: Anu, Daniela, Ola, Rufus

Agenda

  • Quick sync
  • Updates from Bizdev
    • How does DH Cloud Enterprise would look like?
    • Kicking off IDDA
  • AOBs

Updates from DataHub Cloud team

  • added support for custom CSS with custom.css file placed in the root of the published repo (or it's subfolder)
  • fixed bugs related to incorrect links resolution (e.g. on README pages)
  • improved datastory layout (+ some minor improvements to dataset layout) example https://datahub.io/@olayway/digital-garden/blog/cultivating-an-emerging-paradigm
  • basic site config (navbar + basic SEO) support with config.json (🚧 almost done)
  • site-wide navigation in LHS sidebar (similar to e.g. Obsidian publish) (🚧 almost done)
  • previews for geo data

Notes from Anu

  • DataHub enterprise offering
    • What does it look like?
  • Thinking about how I can sell it to enterprise

Rufus notes/ideas

  • Dedicated private support channel for Datopian team on discord with "SLA"
  • Regular sync between bizdev
  • Secondment of Mohammed (with allowed time for support)
  • DataHub

Questions for Anu

  • …

Products Weekly 2024-04-12

Present: Ola, Daniela, Mohammed, Rufus

Agenda

2024-03-22 - Team Meeting

  • Check in
  • Updates from Anu
    • Understanding SSEN help he wants
    • Docs etc for sharing with clients
    • The cool data catalog site he wanted to show us using DataHub Cloud
    • Any other updates
  • Hackathon next Friday
  • Other shaping
  • AOB
    • Is datapackage.json stuff deployed?
  • ⏭️ Next actions
    • Daniela: Ronaldo is "onboarded" to team and likely pairs up with Mohamed as his mentor to do SSEN work (and other things)
    • Daniela to schedule a meeting with Anu next week to outline our proposal on the portaljs library and how it relates to DH Cloud and contribution around it.
    • Daniela to schedule a marketing sync to identity the top "features / attractions" of our approach PortalJS + DMS backend (CKAN) approach that would lead into a marketing campaign e.g.
      • Ease of customization of frontend
      • Integrated experience of data catalog and content
      • Data rich content …
      • Integrating "big data" … and data explorers
      • Data dashboards

SSEN portal

Our current client who uses CKAN + PortalJS frontend wants to use Flowershow as CMS now. I.e., we'd like to make this integration (probably we already have it!). I'm not sure if Flowershow vs DataHub as they share similar features but this client wants to have our PortalJS comopnents lib available so that they can do charts etc within the CMS. They are open to use GitHub as editor.

Links: portal - https://data.ssen.co.uk/ and repo - https://github.com/datopian/portals-monorepo/tree/main/apps/portal-sse

Questions

  • By this do you mean: they want to add content to their portal by publishing stuff "data rich" markdown docs? ✅2024-03-22 Yes
  • Does this content use live viz? e.g. table or graph components loading data from their portal? ✅2024-03-22 ❌ not totally clear. requirements below need charts but not charts off live data
  • Does this content live in a git(hub) repo?
  • Is that repo separate from portal(js) repo or part of it?
  • Do we have a written statement of what was promised ot the client? ✅2024-03-22 YEs, see below
  • Is it part of the main website? ✅2024-03-22 yes.
  • ASIDE: importing word documents would be a cool feature …
  • Who can do development for the client?
  • What is the timeline for this work? 🔑 the preferred timeline is 2-3 weeks from now

Client Requirements doc

  • Creation of a separate Section of the data portal site in which to be published and maintained:
    • Data Roadmap of data to published
    • Ability to update once every x weeks
    • List of triage / responses received with status update
    • Ability to update once every x weeks
    • List of completed accepted Data Triage Responses
    • List of completed denied Data Triage Responses

Response:

  • We will integrate a solution for content curation and publication. We will support data-rich components so that editors can:
    • Create and update the roadmap component.
    • Create tables with a list of triage/responses.
    • Create tables with statuses etc.

5 days

Notes

  • Clients from last summer
  • Very happy with us
  • BigQuery for smart meter data …
  • Now, they want to create more content. Atm only one page "Data roadmap" but they want to be able to write data-driven articles
  • Fully open to use github and markdown and whatever we propose
  • They want more components for portaljs

Docs for onboarding folks

From Anu:

Do we have instructions to onboard new joiners on DataHub Cloud? E.g., I'd like to share it with new prospects but they don't know how to create data views (charts etc from our storybook). So far Storybook was my best source for docs / instructions so far but maybe you have something better?

❓ Qu: Can you briefly describe what you would love to see as a user journey?

Anu walked through his experience with a client where they were asking about capabilities of portaljs etc. Client dialog goes:

  • Can we do geospatial data? Yes, you can..
  • Can I do visualizations? Yes, you can
  • Can I do two layers? Yes, you can
  • How can we try it out? Yes… and we give them access but they struggle because they don't know how to do it themselves. Some of them have not worked with markdown. And don't know how to use these components.

How do I create a map like this?

They need a walkthrough… how to use markdown, how to create tables, how to add other components

❓ what components would you want documented

  • basic markdown …
  • How to create a table
  • How to create a line chart
  • How to create a map (secondary)

Aside:

  • Let's improve the DX of the components a bit: e.g. API seems to differ for the various components

2024-03-08 Products weekly sync

Agenda

Rufus questions

  • We ended up doing a very basic kind of onboarding i.e. just put people in a channel and give them a link 💬2024-03-08 we have a shaping issue we will work on asap
    • Onboarding is a real opportunity
    • The reason to schedule an actual call and walk people through for these early users is a) a big learning opportunity b) setting the tone for the whole experience (community, friendly etc)
  • Cool down and shaping. This is a crucial period to prepare. It isn't "time off" 😉
  • We did miss some datasets e.g. machine learning - see comments in channel this can be part of inbox

2024-03-15

NO MEETING.

2024-03-08 Sprint review and planning

Agenda

  • Check in and happy women's day!
  • Recap where we left it
  • Sprint review - see summary below in ### What we shipped
    • What we shipped (20 out of 23, ~87%)🥳🍾
    • What we did not ship (fully) (3 out of 23, ~13%)(but we made progress on all these 3 and they will be done soon)
  • Sprint planning, see below

What we shipped

MVP 0.1 of the app (soft-)launched and annuonced on Open Data Days 🎉

  • Integrated content store (Cloudflare's R2)
  • Improved dataset showcase layout
  • Created editor guide v1 with a template repository for testing the app with
  • Migrated the old datahub.io content to DataHub Cloud
  • Set up production and staging environments so that we don't break stuff for users during the development process
  • Better error messages and early error catching
  • Other improvements:
    • Support for relative paths to data files in data visualisation components
    • Basic SEO meta tags for user sites' pages
  • Marketing:
    • Organized and prepared for the Open Data Day event presentation
      • Ran social media campaign
      • Complemented outreach with an email campaign to inform and invite the community

Sprint planning

First week (cooldown and shaping)

https://lifeitself.org/tao/getting-stuff-done

Shaping:

Bugs / fixes / unfinished work:

Ola's notes

400 people tried accessing https://datahub.io/core/country-list/r/0.html

image

180 people tried accessing https://datahub.io/sports-data/english-premier-league

150 people tried accessing the search page since the migration

…

Would be nice if e.g. Mohamed could go through the most visited core pages here from time to time and make sure they render correctly

image

https://vercel.com/datopian1/datahub-next-new/analytics

2024-02-23 - Products weekly

Present: Ola, Yoana, Daniela (for the second half + Joao, Leo, Shreyas)

Agenda

  • Check in
  • Bizdev updates
  • Sprint review and planning
    • Pending review from current sprint
    • Looking at next iteration's backlog
  • Showcase Datahub Cloud to Datopian members
    • Outline of Datahub Cloud
    • Editor Guide/Instructions
    • Feedback
  • Next actions
    • Daniela to create an issue for Leo, Joao, Shreyas to play around and test and give their feedback
  • AOBs

Notes

Joao:

Leo:

Shreyas:

##Prep meeting

Present: Yoana, Ola, Daniela

Agenda

  • Check in
  • Creating the agenda for the prep meeting
    • Github board is up to date
    • Planning next iteration
      • Adding the issues to the board
      • Priorities
  • AOBs

© 2024 All rights reserved

Built with DataHub LogoDataHub Cloud