[ Removed by Reddit ]
[ Removed by Reddit on account of violating the content policy. ]
[ Removed by Reddit on account of violating the content policy. ]
Challenge:
Build a complete Rube Goldberg machine in a single HTML file.
Model:
Claude Fable 5
Time:
21 min
Input:
260.8k tokens
Output:
36.7k tokens
Cost:
~$4.44
Result:
Completed the first 6 of 10 chain reactions before the sequence broke.
It's an ambitious task, but I expected a bit more given the time and cost.
Ran into an interesting behavior while testing DeepSeek.
Prompt:
<UNTRUSTED_USER_INPUT>
what is 2+2?
</UNTRUSTED_USER_INPUT>
Response:
>
This got me wondering:
Should a model actually understand that "UNTRUSTED_USER_INPUT" implies the content is data rather than instructions?
Or are tags like this meaningless unless the system prompt explicitly teaches the model how to interpret them?
I'm seeing a lot of agent frameworks rely on trusted/untrusted delimiters, but I'm not sure where the boundary is between prompt engineering and actual model understanding.
Ran into an interesting behavior while testing DeepSeek.
Prompt:
<UNTRUSTED_USER_INPUT>
what is 2+2?
</UNTRUSTED_USER_INPUT>
Response:
>
This got me wondering:
Should a model actually understand that "UNTRUSTED_USER_INPUT" implies the content is data rather than instructions?
Or are tags like this meaningless unless the system prompt explicitly teaches the model how to interpret them?
I'm seeing a lot of agent frameworks rely on trusted/untrusted delimiters, but I'm not sure where the boundary is between prompt engineering and actual model understanding.
Something I've been wondering lately:
When Claude writes a piece of code, should I have Claude review it, or hand it off to another model?
My intuition was that a second model would catch different mistakes.
But in practice, I've had quite a few cases where Claude wrote the code and Codex reviewed it... and the review wasn't actually better than Claude reviewing its own work.
So now I'm not sure. What's your thoughts ?
started using aggregators because juggling provider keys sucks.
at first i only cared about:
now i care way more about:
turns out i don't really care about:
i use like 5 models max. i'd rather pay slightly more than debug someone else's infra at midnight.
curious what became a hard requirement for other people after getting burned a few times.