Suitable release process for Dataform and Git Branching
I am trying to assess the best approach to branching and releases, as we will soon be moving to Dataform and have the chance to improve things with our general development process. But I don’t have a lot of experience here.
Currently, we have 3 projects (dev, staging, and prod) in BigQuery. Each with its own associated branch in GitHub. People merge their feature code to dev, then eventually all changes are merged to staging and finally prod. GitHub actions deploys the code.
But I’m wondering if this can be simplified. My thoughts are:
- 1 protected branch - main (I don’t see the need for 3 branches)
- Devs work from the latest commit of main when developing in the dev project.
- The staging project is also based on the latest commit from main. This is where user acceptance testing happens.
- Prod is based on a git tag on main. When ready to release to prod, a new tag is created, and our Dataform production release configuration is updated with this new tag (still assessing whether this release config in Dataform can be updated automatically via a GitHub action when a new release tag is created, or if we would need to update it manually)
- For hotfixes, we branch from the latest prod tag and merge back into it with the fix. Then cherry picked onto main.
Just wondering if what I’m suggesting makes sense, or if I’m trying to simplify too much?