u/Amazing-Coat5160 — reddlx

What do you think of Yann Lecun option of RL being the cherry on top of all the ML cake?

Title says it all. I'm not expert in pure RL research, I worked mainly in foundation models so far.

Im curious on earing form expert what are their opinion of the role of modern RL, in particular:

- will it be just the very last fine tuning layer of bigger foundation models? If so what kind of RL approach you think are most prominent?

- will there be (or there are alredy) model that use RL more as a core layer in the whole model?

My gut feeling is that RL is very cool, but the hype has gone down in the last years due to diffusion/foundation model performing and scaling much better, and a lot of RL is perceived in practice as mainly "reward engineering".

Please correct me as I might be very wrong :)

u/Amazing-Coat5160 — 8 hours ago

▲ 30 r/robotics

VLAs vs Nvidia world models vs all of that: where are we going?

I'm reading the papers of Cosmos3 and Dreamzero and they looks very promising (compared to memoryless VLAs). And I am wondering where the filed will evolve.

Based on your practical experience with new models, what's your bet between VLAs, WM, Jepa-style, WAM, RL approaches, and all of that?

I worked so far with VLAs (eg pi05), and I don't have any experience in using the nvidia stack so far, of and other world action models. I am thinking if I should invest time in changing the base policy, and I'd appreciate some feedback form who has tested them (ie: the open source/weights model available, and capable of inference without one thousand gb of vram)

On my side I'm a fan of model working planning latent space; video action models (which have more temporal coherence wrt vla), but I also feel that semantic power of a VLM should be present aswell.

Ps: suggested survey reading in this topic:

"World Model for Robot Learning: A Comprehensive Survey"

Happy to discuss with you

reddit.com

u/Amazing-Coat5160 — 13 days ago

▲ 3 r/robotics+1 crossposts

Current research directions in robotics foundation models if you can’t train from scratch?

TL;DR struggling in finding a meaningful research contribution on top of existing big foundation models.

(edit: please note it's my first post on reddit,I'm not a bot)

Context: I'm working on FM applied to robotics: VLAs, world models, WAMs. Lately I'm mostly reading papers, and implementing small adds on.

Those topic are really exiting but I’m wondering where modest researchers (like me) can make meaningful contributions, given that training competitive foundation models from scratch is a big-lab game.

For people working on fondation models in academy and R&D, that asked themself similar questions: Do you have some honest suggestions or feedback?

If starting from a pretrained fondation model, main things that come to my mind are eg:

- architecture changes (don't you lose all the pre training warmup)?

- fine tune (not much new science if one runs lora...)

- froze the model and build add-on like uncertaintyquant , world-model lookahead, inference guidance, safety constraints

- something big I'm not seeing?

Also happy to hear paper/project recommendations that are good examples of this.

Thank you all.

reddit.com

u/Amazing-Coat5160 — 27 days ago