DataHub Cloud functional tests
Summary
Situation: Lack of automated tests in the DataHub Cloud. We only have the staging environment used to test changes manually before merging to production. Problem: Absence of tests complicates the process of making changes, increasing the chances of issues. Appetite: 3 days Solution: Implement testing, only functional tests for now, to ensure core functionalities are working as expected.
Situation
- We have launched the first version of the app and we're currently onboarding some clients.
- We make a lot of changes to the app.
- We have staging and production environments (staging is for testing before pushing to production).
- We test changes manually.
- We don't have any automated tests implemented.
Problem
- Having no automated tests makes introducing new features to the site (as well as making other minor changes and fixing bugs) very error-prone, as changes made to one part of the app could potentially break another part, and currently there is no way to catch it without manually testing both.
- With manual testing only it would be extremely hard (or close to impossible) to cover all functionalities each time we make some changes to the code, and it's obviously time-consuming.
Appetite
- 3 days
Solution
- We should focus on things that:
- Break core functionality.
- Can affect many users.
- Set up automated functional tests with Playwright.
- Create a suite of tests for the current set of functionalities.
- Start utilizing Test-Driven Development approach from now on.
What to test
- Login functionality
- Redirecting to login (from any page) if not logged in
- Redirected to the dashboard (from /login) if already logged in
- Site creation
- Dashboard shows the placeholder/message image if no sites were created
- User can create a new site
- The newly created site is added to the dashboard
- Clicking on the site card redirects to the settings page for this site
- Clicking on the link inside the site card redirects to the home page for that site (and it renders correctly = with no errors)
- Site settings
- The site has correct repo URL and deployed URL
- User can rename website
- User can update branch
- Dashboard shows
not synced
- User should be able to
sync
- Dashboard shows
- User can delete website
- Dashboard shows
not synced
after updating the GitHub repo ( Really think this shouldn't be implemented ) - User can sync
- Published sites
- datapackage.json/yml/yaml files (or datapackage in frontmatter) will be picked up.
- Test rendering of dataset layout for README.md/index.md to ensure all necessary information is displayed.
- Test rendering of story layout for pages that doesn't have datasets to ensure proper display of content.
- User can use data-rich components
- Description is extracted
- Title is extracted correctly
- Paths work as expected
- Sub-directories work correctly ( one nested file )
-
/
displaysReadme.md
orindex.md
Rabbit holes
No-goes
- Don't try to fix issues discovered during testing but rather open a new issue.
- We don't want unit tests for now.
- Testing custom domains.
- Testing authentication.
- Testing markdown syntax rendering.
- Custom domains work (won't test for now as it's complicated)
- MD syntax (headers, tables, callouts, etc.) (won't test as it's already have tests of their own)
- Running the functional tests as a GitHub workflow