DataHub Cloud functional tests

Summary

Situation: Lack of automated tests in the DataHub Cloud. We only have the staging environment used to test changes manually before merging to production. Problem: Absence of tests complicates the process of making changes, increasing the chances of issues. Appetite: 3 days Solution: Implement testing, only functional tests for now, to ensure core functionalities are working as expected.

Situation

  • We have launched the first version of the app and we're currently onboarding some clients.
  • We make a lot of changes to the app.
  • We have staging and production environments (staging is for testing before pushing to production).
  • We test changes manually.
  • We don't have any automated tests implemented.

Problem

  • Having no automated tests makes introducing new features to the site (as well as making other minor changes and fixing bugs) very error-prone, as changes made to one part of the app could potentially break another part, and currently there is no way to catch it without manually testing both.
  • With manual testing only it would be extremely hard (or close to impossible) to cover all functionalities each time we make some changes to the code, and it's obviously time-consuming.

Appetite

  • 3 days

Solution

  • We should focus on things that:
    • Break core functionality.
    • Can affect many users.
  • Set up automated functional tests with Playwright.
  • Create a suite of tests for the current set of functionalities.
  • Start utilizing Test-Driven Development approach from now on.

What to test

  • Login functionality
    • Redirecting to login (from any page) if not logged in
    • Redirected to the dashboard (from /login) if already logged in
  • Site creation
    • Dashboard shows the placeholder/message image if no sites were created
    • User can create a new site
    • The newly created site is added to the dashboard
    • Clicking on the site card redirects to the settings page for this site
    • Clicking on the link inside the site card redirects to the home page for that site (and it renders correctly = with no errors)
  • Site settings
    • The site has correct repo URL and deployed URL
    • User can rename website
    • User can update branch
      • Dashboard shows not synced
      • User should be able to sync
    • User can delete website
    • Dashboard shows not synced after updating the GitHub repo ( Really think this shouldn't be implemented )
    • User can sync
  • Published sites
    • datapackage.json/yml/yaml files (or datapackage in frontmatter) will be picked up.
    • Test rendering of dataset layout for README.md/index.md to ensure all necessary information is displayed.
    • Test rendering of story layout for pages that doesn't have datasets to ensure proper display of content.
    • User can use data-rich components
    • Description is extracted
    • Title is extracted correctly
    • Paths work as expected
      • Sub-directories work correctly ( one nested file )
      • / displays Readme.md or index.md

Rabbit holes

No-goes

  • Don't try to fix issues discovered during testing but rather open a new issue.
  • We don't want unit tests for now.
  • Testing custom domains.
  • Testing authentication.
  • Testing markdown syntax rendering.
  • Custom domains work (won't test for now as it's complicated)
  • MD syntax (headers, tables, callouts, etc.) (won't test as it's already have tests of their own)
  • Running the functional tests as a GitHub workflow

© 2025 All rights reservedBuilt with DataHub Cloud

Built with LogoDataHub Cloud