u/Valuable-Run2129

Setup: i talk to it via telegram from my iPhone and carry the Ubuntu Touch phone with me.
The phone is on at all times and battery seems to last.
It was hard to make it work in the background, but now it does, even when the phone’s screen is off.

Unlike having it on a computer at home, this feels different. It has access to camera, microphone, gps… any sensor the phone has. It can nanny cam, detect things and also use the speakers.

I gave it 50k tokens of headroom directly injected into context with a fractal and continuous summarization dedicated exclusively to our conversations (just messages, it holds months of conversations without missing almost anything), so it feels like a person with no separate sessions.

Luna on Go plan would be awesome. But it appears to have no vision. Just like deepseek. Is it intentional? Or a temporary thing?

Sto lavorando da 6 mesi a un assistente virtuale generalista. Gli scrivi via Telegram o Whatsapp da ovunque sei e lui gestisce un Mac dedicato per fare quello che vuoi. Chi ha già sentito parlare di Open Claw ha più o meno idea di cosa parlo.

L’assistente, a differenza di Lexroom o un chatbot come ChatGPT, ha una memoria a lungo termine come un vero collaboratore. Gestisce un suo account Gmail con email, promemoria, calendario e anche cartelle condivise di Google Drive.

Posso far provare il mio assistente via telegram o whatsapp (che ha quindi la mail del mio assistente), ma per chi volesse avere un proprio assistente sul proprio mac c’è da sapere:

-L’app per ora è gratuita per i tester

-Serve un Mac dedicato e sempre acceso. Sconsiglio vivamente di installare l’app sul proprio computer. L’agente avrà pieno accesso a tutta la macchina. Con il potere di modificare e cancellare tutto quello che c’è dentro (si trovano Mac Mini usati a 300 euro che vanno benissimo).

-Io non ho server centralizzati, il che ha da una parte il vantaggio che non vedo niente di quello che scrivete. Rimane tutto sul computer. Ma dall’altra dovete iscrivervi direttamente ai servizi che fanno funzionare l’app. Sono 5 e la prima installazione può richiedere un’ora di tempo.

-tra questi servizi esterni solo uno è esposto a tutto quello che dite. Si chiama Open Code ed è una società Americana che offre intelligenza artificiale tipo ChatGPT. Nella sua privacy policy, a differenza di ChatGPT, dichiara di non trattenere nessun contenuto inviato dagli utenti (Zero Data Retention). Questo chiaramente non soddisfa chi per lavoro ha bisogno che i dati vengano processati in Unione Europea e con contratti di privacy espliciti. Ma è sicuramente meglio di ChatGPT.

-il costo dell’app per i tester è gratuita. Ma i servizi sono a pagamento. Open Code offre un abbonamento da 10 dollari al mese che per molta gente sarà sufficiente. Gli altri 4 servizi sono a consumo e verranno in tutto sui 7 o 8 euro al mese con un utilizzo intenso. Richiedono ricariche iniziali per funzionare.

-assegnare una email all’assistente ed esporla al pubblico introduce il rischio che l’intelligenza artificiale in buona fede condivida informazioni che non vorresti con persone che le scrivono via email. Io la mail del mio assistente non la condivido e la tratto come quella di un collaboratore che conosco solo io.

Chi volesse testare l’assistente sul proprio computer installando l’app (o chi volesse fare qualche domanda al mio assistente sul mio mac) può scrivere nei commenti e lo contatto in privato.

Chiaramente chi usa il mio assistente sul mio mac ha zero privacy perché tutto gira sul mio computer.

PS. Per chi conosce già Open Claw e si chiede in cosa è diversa la mia app:
È scritta totalmente in swift per usare la sicurezza della keychain di Apple (fondamentale per far usare agenti del genere a persone che non sanno per esempio come maneggiare delle API Keys).
Non ha “sessioni”. Quindi l’utente non deve gestirle. Parla all’assistente in una chat infinita.
Ha una memoria a lungo termine integrata con fino a 50k token dedicati e sempre “in-context” con la conversazione con l’utente e riassunti densi di ciò che è stato detto settimane e mesi prima. Questi riassunti hanno una struttura molto elaborata che sono disponibile a spiegare.

You probably would agree with me that an ASI, by definition, won't have an incoherent moral framework. A human can cry watching the movie Babe while eating pork ribs.
A super intelligence, on the other hand, will attribute value to things and not forget about it.

Everyone focuses on the risks of a malevolent or indifferent ASI, but a benevolent one won't be aligned to our current values. There will be a big clash and humans won't be the good guys.

We kill for taste over 100 billion sentient land animals every year. Not only kill them, 99% of them are tortured in cages for the entirety of their short lives. An ASI will obviously know that those animals have a limbic system just like ours. Capable of suffering, of feeling happiness, anxiety, fear...

Every 15 minutes 6 million animals are killed. The equivalent of the holocaust. A benevolent super intelligent would act swiftly and steamroll any resistance it would find. It wouldn't wait to transition humanity to different food (we already have enough plant food for everyone). Being benevolent it would probably minimize human casualties, but factory farms executives that refuse to shut down their facilities will inevitably die with digitally connected cars, planes, pacemakers...

Sorry, but I have to get it off my chest. I keep on reading this insane take. Sure GLM 5.2 is really good at oneshotting good looking stuff, but the moment you want to build on it with followups the model just can't do it. It has no vision. It's text only! It physically can't see what it has done and doesn't understand the UI like you do.

Edit to avoid repeating stuff in the comments: of course I know OCR exists, but iteratively working on a visual thing with OCR is like painting blindfolded while a guy describes in words what is in front of you. It’s not good. Too much stuff gets lost.

The intelligence is not really the bottleneck anymore. It’s memory.

I’ve built my own Jarvis with a CC plugin with a set of background agents that compress older information so that at session start (or compaction) Claude Code remembers everything I ever said.

It already works very well. But the memory system is capped at 45k tokens. It remembers very well anything that happened in the latest couple of weeks, but it sorts of loses details for things that were said a month or two ago.

If Claude had a context window of 50 million tokens I would be able to create a system with perfect recollection about what happened in the past couple of years. Which is more than what humans can do. A session would also last a gull day and compactions could happen at night, similarly to how we sleep.

There’s nothing blocking Anthropic from making a model with that context, apart from cost. And running it would also cost me a fortune (at current prices roughly 5k dollars a day).

I read a lot of comments of people being ecstatic about the GLM models. But not having vision capabilities makes the agent so much dumber for follow ups about UIs. Sure they can one-shot great things, but they obviously lack the first hand understanding of what they did.

Editing visual documents is also a mess.

Are you guys not encountering these issues? Or are you ok with the big trade off?

I posted here about buying it a few days ago: https://www.reddit.com/r/LocalLLaMA/comments/1t2slmw/first_time_gpu_buyer_got_a_rtx_5000_pro_was_it_a/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

Before pulling the trigger I was leaning more towards a Mac Studio. But the the prompt processing speeds I was reading about were giving me pause. The budget was $5000/6000. So the 256GB was out of the question.
I gambled and bought the RTX 5000 Pro. With ZERO experience with PCs, how to build them, what parts to buy... It was a good deal. I paid $4300 for the gpu including taxes (in the post I wrote 4700 in the comments, but I was mistaken, I checked the receipt) and had to buy everything else for the computer. It ended up costing $5600 in total with 64 gb of RAM.

Assembling the thing was not easy for me as a total novice, but thankfully we have LLMs to guide us through these things.
Then came Linux and vLLM... Honestly I was totally lost. without Claude Code it would have been impossible. Also what settings to use to run Qwen3.6-27B-FP8 with full precision cache. Thankfully this guy posted everything I needed to know to tell Claude what to do: https://www.reddit.com/r/LocalLLaMA/comments/1t46klu/qwen36_27b_fp8_runs_with_200k_tokens_of_bf16_kv/

After burning through 50% of my Claude Code Max 20x weekly limits the thing now works, and I have to say... I made the right call. This thing rocks.
I'm getting up to 80 ts in TG (more like 50/60 for very big prompts) which is phenomenal. But most importantly I'm getting 4400 tokens per second in PP!

The full precision cache fits only 200k tokens, but It is totally ok for me.

I honestly don't know why people are not talking about this gpu more. It costs just 1000$ more than an RTX 5090, it can fit 27B at 8FP and 200k of context at full precision. It draws half the electricity... Sure it is slightly less performant, but the numbers I'm getting are way more than I was expecting. Two 5090s would definitely beat this. But it would cost significantly more, it would be crazy noisy and tear a hole in my pocket in electricity bills.

I’ve run models exclusively on apple silicon up until now, but wanted to up my inference game.

I bought a slightly used RTX 5000 Pro Blackwell for a bit more than twice as much as two 3090s.

I’ve read of people saying that the 5000 doesn’t provide a big performance improvement over the 3090s. That is making me doubt my choice. But it is also true that electricity cost where I live is 0.40 euros per KWh. A 5000 Pro would probably burn a third of the electricity of a dual 3090 build. Right?

Also, if you have a 5000 Pro, what type of speeds do you get in PP and TG with qwen3.6 models?

Every time I see a memory system that asks the agent to divide memories by topic or type I now know it won’t work. Some things are just not easy to classify. They belong to different buckets based on context and point of view.

From the outside it looks like a smart thing to do. But having memories in the wrong class equals having no memory at all.

Relying on the agent to independently determine what is worth remembering is also a dead end.

Relevance doesn’t happen immediately. Something might be insignificant when is first introduced, but totally fundamental a day after. Its classification also would change in time.

Yet everyone asks the agent to detect what is important, drops it in an md bucket and hopes magic will happen. Unfortunately it doesn’t.

Since context windows got better I started dedicating an increasing amount of it to brute memory injections at session start. Up to 40/50k tokens. With verbatim recent messages and very detailed chronological summaries of all previous conversation chunks. As they get older they get re-summarized. But by that point it is easier to determine what is important or not.

The thick chronological injection also helps retrieval

In narrowing down where to look at if the agent ever needs the exact words you said 5 months ago.

I’ve been pleasantly impressed by this method and have implemented it in my own swift-based coding/assistant harness. 40/50k tokens if overhead seem unnecessary, but current models handle them without issues and the results are Jarvis-like with a continuous infinite session.

I also made my CC and Codex memory plugins with the same system. The key part is adding relevant breadcrumbs to the messages you store. The message isn’t enough if it doesn’t contain minimal info like location of touched files.

It’s criminal. Delegating OCR makes an agent unusable for many tasks.