u/Logical_Delivery8331

ciao a tutti!! sono Joe, da 4 anni lavoro nell'AI come ricercatore e AI engineer. vi racconto un esperimento che ho fatto in questi giorni e che è finito anche in un video: l'ho girato di getto, giusto per rompere il ghiaccio e darmi finalmente una mossa ad aprire il canale youtube che volevo iniziare da tempo. nel video si vede tutto dal vivo.

munito del mio fidato mac mini M2 e di qualche gb di ram (8 per la precisione), ho tirato su in locale KEV (https://github.com/jaredpalmer/kev), un modello JEV-like uscito pochissimo dopo l'annuncio di JEV. usa lo stesso identico protocollo "typesafe". Gli dai in input una descrizione del task, una query e delle opzioni, e lui ti dà in output la probabilità che la query sia collegata a ognuna delle opzioni, senza generare testo.

KEV però è stato allenato con SFT solo a leggere testo. il suo backbone (qwen3.5 0.8b) perà la rete di vision ce l'ha già ma in KEV è disattivata!! allora l'altro giorno per curiosità gliel'ho ridata, senza riallenare niente. per gioco gli ho fatto classificare in che bidone va quello che passa davanti alla webcam e devo dire funziona sorprendentemente bene. la capacità di questi modelli di zero shottare i task mi sorprende ogni volta.

detto questo, è un esperimento e non un benchmark: spesso il margine tra le opzioni è piccolo (sulla mia faccia not_waste vince con circa il 46%). per questo la domanda del titolo, qualcuno ha provato cose simili? mi piacerebbe capire fin dove si può spingere. so che openai ha rilasciato il proprio decision API con vision tipo.

BTW il canale l'ho aperto per passione e per documentare alcuni progetti. parlerò di temi tecnici sull'AI mettendo le mani in pasta nel codice il più possibile. Se siete interessati fatemi sapere qualche tema che potrebbe interessarvi! I prossimi video saranno più ordinati e organizzati meglio. se ci sono temi di cui vi piacerebbe che parlassi, scrivetemelo!!

se volete appena mi arriva il nuovo mac da 24 gb potrei fare un video sugli altri KEV!! se siete interessati stay tuned!

video: https://www.youtube.com/watch?v=4q1LIqn7lTs

script e notebook: https://gist.github.com/pierpierpy/1b03366d581015d4938176eab30ac650

Hey everyone!

I'm a data scientist and I've been pretty fascinated by reinforcement learning for a while. A few days ago my friends showed me Pokelike, a small Pokémon roguelike that runs in the browser. The first thing I thought was that it could be pretty fun to turn it into an RL environment.

So I did.

The repo is here:

https://github.com/pierpierpy/pokelike.xyz.bot

The idea is to run the actual game locally and expose its state and actions to an agent. There is no image processing involved. The bot gets the game state directly and decides what to do next. The agent has to make the decisions around the battles, like where to go on the map, which Pokémon to catch, which items to take, when to swap Pokémon and which moves to learn.

One thing I found interesting is that the map forces you to make some decisions quite early. Once you choose a node, the other nodes on that layer are gone, so deciding where to go can matter quite a bit later in the run. There is also a lot of information in the state that could potentially be useful, but I'm still not sure what the best way to represent it is.

I've already implemented a few basic RL agents. There is currently a Dyna-Q agent and two linear SARSA agents in the repo. The results are not amazing yet. On the current benchmark, random gets around 0.56 badges, Dyna-Q gets around 0.62, while the two SARSA agents get around 1.30 and 1.36. The two SARSA agents mainly differ in their state representation, with the better one using 100 hand-designed features instead of 81.

This is probably the part I'm most interested in at the moment. Finding a good state representation seems to make a pretty big difference, and the environment has some properties that make it a bit more interesting than I initially expected. The reward is fairly sparse, the action space depends on the current state, and some decisions only become useful several steps later.

At the same time, the environment is completely reproducible. With the same seed and the same actions you get exactly the same run. For the current leaderboard I'm using 50 fixed seeds, so different agents can be compared on the same games.

The interface is also intentionally pretty simple. You can basically implement a bot that takes the current state and returns an action. It doesn't have to be a specific RL algorithm either. You could try DQN, PPO, search, a hand-written policy, or pretty much anything else.

I'm still experimenting with the environment and the agents, so I'd be really curious to see what other people would try. In particular I'd love to see if someone can get significantly better results with a better state representation or a different reward function.

you can experiment with the environment with no efforts, just follow the guide and readme, setup the enrinvonment, experiment a bit and then if you like the result, you can create a pull request to the repo with your bot in the bots/ folder (everything is clearly explained in the GUIDE.md)

If you want to try it, everything is in the repo

https://github.com/pierpierpy/pokelike.xyz.bot

If you find bugs or have ideas for the environment, please let me know. I'm happy to make changes if there are things that would make it more useful for experimenting with RL.

The whole thing also runs offline. During setup it downloads the game and the required assets, and after that everything runs locally.

I originally started this because I thought it would be a fun little RL project, so I'm mostly curious to see how far people can push it.

hi everyone!

i want to share with you a little project i created a few months ago to solve a problem i was having with function calling. whenever i needed a good quality and specific dataset to train my models on function calling i couldn't find a good repo for generation. i wanted a dataset that teaches the model not only how to call the tool but also when, in different contexts. i also wanted to have maniacal control on the results, i wanted to control how many tools in each convo, when the tool is called, errors in tool callings and in particular i wanted something that was flexible enought to include *PERSONALIZED* tools with personalized mock answers!!!

for example you can find some tools i made for the sample below in the repo under

>synthfc/tools/eng

and

>synthfc/tools/ita

i also wanted a way to check the results and auto-correct the pieces of data that have problems. here is the repo:

https://github.com/pierpierpy/FC-synth

here some examples i created with an open source model:

https://huggingface.co/datasets/pierjoe/function-calling-synthetic-2000

hope you find it useful!

happy tool calling!