Suitable release process for Dataform and Git Branching

I am trying to assess the best approach to branching and releases, as we will soon be moving to Dataform and have the chance to improve things with our general development process. But I don’t have a lot of experience here.

Currently, we have 3 projects (dev, staging, and prod) in BigQuery. Each with its own associated branch in GitHub. People merge their feature code to dev, then eventually all changes are merged to staging and finally prod. GitHub actions deploys the code.

But I’m wondering if this can be simplified. My thoughts are:

- 1 protected branch - main (I don’t see the need for 3 branches)

- Devs work from the latest commit of main when developing in the dev project.

- The staging project is also based on the latest commit from main. This is where user acceptance testing happens.

- Prod is based on a git tag on main. When ready to release to prod, a new tag is created, and our Dataform production release configuration is updated with this new tag (still assessing whether this release config in Dataform can be updated automatically via a GitHub action when a new release tag is created, or if we would need to update it manually)

- For hotfixes, we branch from the latest prod tag and merge back into it with the fix. Then cherry picked onto main.

Just wondering if what I’m suggesting makes sense, or if I’m trying to simplify too much?

reddit.com
u/Data-Panda — 7 days ago

Is it silly to just spend a whole month in CM instead of exploring Thailand?

So I’m in CM right now. Been here over week. Working remotely for a couple weeks and then holiday (going back to the UK September 18th).

I had planned on maybe visiting an island + 1 week in BKK - but tbh I’ve just been really liking CM.

The food, the atmosphere, the weather (I like the rain mixed with warm weather), the nature. And admittedly I’ve met some great women here which probably plays a big part in me wanting to stay.

I’m just wondering if it’s common to stay here for this long as a traveller, and if anyone has done the same and later regretted not seeing more places.

reddit.com
u/Data-Panda — 1 month ago
▲ 197 r/onebag

30 days in Thailand (6.5kg)

Thought I’d share my packing list. Heading to Thailand for about a month. Will be working remotely for a couple weeks, and then holidaying the rest of the time.

Tried to keep weight under 7kg. I may ditch the sandals as I prefer to just wear trainers (I’ll be bringing my Nike Pegasus Trail or New Balance 574, but can’t decide)

Bags

- Osprey 26 + 6 Daylite Backpack
- Amazon packing cubes (2x)
- Peak Design light packing cuber (for electronics)
- Bellroy lite sling mini
- Matador laptop case

Clothes

- Uniqlo Dry Ex T-shirts (3x)
- Uniqlo Dry Ex Polo Shirt (1x)
- Chino/Cargo style shorts (1x)
- Swimming/sport shorts (1x)
- Trainer socks (5x)
- Uniqlo Airsim Briefs (4x)
- Sunglasses
- Teva Hurricane Sandals

Toiletries

- Travel toothpaste
- Travel Mouthwash
- Floss
- Philips travel toothbrush
- Suncream
- Contact Lenses
- Mitchum Roll On Deodorant
- Travel Moisturiser
- Face Cloth
- Face Wipes
- Tissues
- Hair Pomade
- Vapo Rub (in case I’m next to anyone smelly on the plane)
- Nail Clippers

Electronics

- Dell Laptop
- Philips one blade
- USBC Cable
- Dongles for phone
- Spare iPhone SE3
- Charging station for AirPods, Watch, and Phone
- Anker 65 Charging Plug
- Plug adapter
- Anker Power Bank

Medicines

- Imodium
- Ibuprofen
- Gaviscon
- Pepto-Bismal
- Anti-allergy pills
- Hydrations packs

Misc

- Passport
- Print outs (accommodation, flight tickets etc)
- Spare bank card
- £500 in cash (in case they ask for it at immigration)
- Sleep mask
- Pen
- Hidden Wallet
- Mints
- Ear Buds
- AirTag (2x)
- Glasses case

u/Data-Panda — 2 months ago

Is Perseids Meteor Shower today/tomorrow viewable from within Glasgow?

I believe it peaks today or Thursday, but unsure if there is too much light pollution to be noticeable

reddit.com
u/Data-Panda — 2 months ago

What should I be focusing on as a junior in the age of AI

Hello

I’m currently a junior DE building Python pipelines (Prefect/Airflow to BigQuery mainly). I’ve started using Cursor/Claude to assist in coding but try not to lean on them too hard.

On the side I’m self-studying data modelling, system design, and Leetcode (python & sql). But I feel a bit scattered, jumping between topics without a clear plan. I’m also increasingly wondering how AI is going to reshape this role, and want to make sure I’m building skills that keep me hireable.

With that in mind, does anyone have any advice on topics I should be prioritising?

reddit.com
u/Data-Panda — 2 months ago

Heading to Thailand on Aug 20th for 4 weeks - a few questions before I go

Hello

I'll be there for 4 weeks total. First week I'll be working remotely, the rest is holiday. Planning to spend the first 2 weeks in Chiang Mai (don't know what I'll be doing the second 2 weeks. If anyone has any suggestions, let me know).

However I've got a bunch of questions and thought it would just be easier asking them in a single post.

Accommodation

* Should I book the full 2 weeks in Chiang Mai now, or just a few days first to make sure I like the hotel? Worried that waiting to rebook when I'm there could leave me scrambling if hotels are all fully booked up.
* Any general advice on booking direct with hotels vs. using [Booking.com](http://Booking.com) or similar sites? I'd usually just use [Booking.com](http://Booking.com) but if it's more reliable or cheaper to book direct I'll do that.

Activities

* Do I need to pre-book activities in Chiang Mai (e.g. national parks), or is availability usually easy to find on short notice?

Remote work / internet

* I'll be working off my laptop and I'm worried about connection reliability. I need solid access to things like Google Cloud and to be reachable for the occasional call, especially since the senior data engineer may be on paternity leave. Is it worth booking a coworking space for 2 weeks, and like above, does this need to be done well in advance?

Domestic flights

* I've got return flights booked to Bangkok from the UK, but nothing yet for Bangkok to Chiang Mai. Is booking a flight for \~2 hours after expected arrival in Bangkok too tight, or generally fine? Any recommended domestic airlines?

reddit.com
u/Data-Panda — 2 months ago
▲ 186 r/glasgow

Usual scenes at St Enoch

I’m frequently around St Enoch and unfortunately this is a common occurrence. Was debating intervening once they all started piling on him but police sirens seemed to dissipate it

EDIT: only context I have is 2 guys were arguing and someone sucker punched the white guy from behind. I don’t know if he said something triggering or if it was just a bunch of guys itching for a fight. Unacceptable either way.

u/Data-Panda — 2 months ago

Heading to Thailand on Aug 20th for 4 weeks - a few questions before I go

Hello

I'll be there for 4 weeks total. First week I'll be working remotely, the rest is holiday. Planning to spend the first 2 weeks in Chiang Mai (don't know what I'll be doing the second 2 weeks. If anyone has any suggestions, let me know).

However I've got a bunch of questions and thought it would just be easier asking them in a single post.

Accommodation

  • Should I book the full 2 weeks in Chiang Mai now, or just a few days first to make sure I like the hotel? Worried that waiting to rebook when I'm there could leave me scrambling if hotels are all fully booked up.
  • Any general advice on booking direct with hotels vs. using Booking.com or similar sites? I'd usually just use Booking.com but if it's more reliable or cheaper to book direct I'll do that.

Activities

  • Do I need to pre-book activities in Chiang Mai (e.g. national parks), or is availability usually easy to find on short notice?

Remote work / internet

  • I'll be working off my laptop and I'm worried about connection reliability. I need solid access to things like Google Cloud and to be reachable for the occasional call, especially since the senior data engineer may be on paternity leave. Is it worth booking a coworking space for 2 weeks, and like above, does this need to be done well in advance?

Domestic flights

  • I've got return flights booked to Bangkok from the UK, but nothing yet for Bangkok to Chiang Mai. Is booking a flight for ~2 hours after expected arrival in Bangkok too tight, or generally fine? Any recommended domestic airlines?
reddit.com
u/Data-Panda — 2 months ago

PS5 sale (UK). 75% off. Is it worth it?

Been debating getting this game, but could never justify paying full price. Wondering if I should get it now. TBH I get a bit bored of Ubisoft style games eventually, but I do enjoy Star Wars and thought this game looked ok

Edit: thanks all - just purchased it. Will keep me busy this weekend

u/Data-Panda — 3 months ago

Best Thai food in Glasgow? Disappointed with most offerings

Been looking for some good Thai food in Glasgow but there’s not much choice.

My ex-girlfriend was Thai and an amazing cook, and she said not to waste my time with Panang or Rosa’s Thai.

We both tried Chaophraya and agreed it was way overpriced. The fish also smelled like it was starting to go off.

Ting Thai Caravan is edible, but honestly disappointing overall - although may be the only place in Glasgow you can get a bowl of Khoi Soi.

I did however try Thairiffic and quite enjoyed their Pad Ka Prao, although I’ve only been once.

Anywhere else worth trying?

reddit.com
u/Data-Panda — 3 months ago

Is my insurance fine for 1 month in Thailand? Want to make sure I’ve not missed anything.

It’s with Allianz, and was insurance I purchased as part of my flights with KLM.

I just want to make sure there not something I’m missing as haven’t given insurance much thought in the past.

I won’t be doing anything too adventurous, just national parks and stuff like that.

u/Data-Panda — 3 months ago

Chiang Mai Old City hotel recommendations + booking

Going to Thailand in August for a month, and will be staying/working in Chiang Mai for the first couple weeks at least.

But I have analysis paralysis regarding accommodation.

Ideally looking for somewhere close to or in the old city, that doesn’t cost a fortune, and also one that allows guests (never know who I’ll meet).

I was looking at the “Mad Monkey” but bit put off that it’s a hostel and the doors look like they could easily be broken into.

And am I right in thinking booking.com would be the best place to book?

Edit: also need to find a workspace as I’ll be working remotely for first 2 weeks, but I assume they’ll be some in the old city I can book

reddit.com
u/Data-Panda — 4 months ago