Spark Structured Streaming Realtime Mode with Stream to Batch-Batch dataframe join?

Trying to implement a Realtime-valid streaming job, but got an error STREAMING_REAL_TIME_MODE.BATCH_BATCH_JOIN_NOT_SUPPORTED.

Pretty sure that this is because I'm using a view (that joins 2 tables) as a static part of the stream-static join, but I can't find any references to this error OR that there's a limitation (either for structured Streaming or the realtime mode in particular) that says "you can't join a stream to a pre-joined dataframe".

Does anyone know where can I read more about this?

And on the other side of the question, I get the restriction is probably due to sheer complexity of having to manage (potentially) multiple incoming records to the joined batch, but I still didn't see anything mentioned about this anywhere.

Edit: it's definitely a limitation of the Realtime mode specifically

reddit.com
u/Kojimba228 — 6 days ago

Lead Sprinkler viable skills (non-Mace, and not Chalupa Tornados)

Does anyone have a list of good skills to use with sprinkler that aren't Maces or Chalupa purple tornadoes?

Wanna do a Gemling build, but I don't really want to use Molten Blast or go for Chalupa

reddit.com
u/Kojimba228 — 8 days ago

Resonance + Hollow Form Lunar Assault freeze doesn't generate power charges?

Basically the title. I thought that it should work since it's not hollow form that would generate charges but rather a lunar assault cast by the image?

reddit.com
u/Kojimba228 — 27 days ago

Spirit Walker with tamed boss and a crossbow

Anyone else thinking of starting Spirit Walker with tamed bosses and a crossbow for the main weapon (likely with nades because of the owl node with additional proj for empowered attacks) or am I going to be the only mad lad to attempt to build this thing?

Also, any tips on good companion nodes on the tree and/or uniques to look out for?

reddit.com
u/Kojimba228 — 2 months ago