u/Different-Monk5916

This is my experience (VS Code BYOK), you may agree or disagree. Each is good in one type of task. If I know exactly what must be done and it is an isolated small task, I would go with Luna. If I want to work on a feature level or higher, with just a broader description, I would go with DS Flash. Luna averages around 86-87% cache hit rate with copilot sub, 92-93% and occasional high hits 96-98% with opencode/openrouter on non-coding tasks. DS Flash is consistently hits above 96%.

Luna is like the colleague who is really efficient in implementing as long as you put a lot of effort in getting your design and vision into their mind, handhold a bit and comes back too soon with a half-ass work. 

DS Flash is like the smart colleague who can fill the gaps as long as you provide a broad overview of what you want, reasons on their own terms, takes more time on tasks than you would prefer and gets it done with minimal corrections required in a second round. 

Luna is better for if you are handholding it, like a senior dealing with a junior in a team and comes back quicker. DS Flash is better if you want it to write code on its own and you want to be merely a system architect, code reviewer and approver.

In my experience, Luna 1M does not give enough weightage to previous requests - as if its attention span is too low for any long context work. It predicts, speculates instead of reasons and find evidence.

I emptied my DS credits and did not top up. Before that, I was using via VS Code DeepSeek extension and their official API.

I have three stage agent orchestration - design, plan and implement. Implement sessions last long sometimes upto 45mins. In the mean time, I work on designs and plans for other features or modules, while keeping an eye for reviewing changes and approving non-auto approved commands.

I wrote this small app, which simply takes the usage statistics from DS API platform, and projects the cost at different cache hit rates for different providers. It simply told DS would cost me 2x and the cheapest was OpenCode. I decided to first try with OpenCode GO.

Subscribed on Friday, and cancelled on Sunday. Was using it via OpenCode Extension - tried a couple. The API endpoint is not reliable, that it throws out a variety of errors -and literally slows down the work. Often retries don't succeed until after a few minutes off or I start a new session - does any face the same issue?

Now, they decreased DS Flash usage to $15 max.

Open AI and Claude Code are alternatives. people are talking about the 5x and 20x usage plans, and token efficiency. hear me out, and think in reverse. If the simple projection while keeping token usage and cache hit rate constant, tells me that they would cost 5x - how will it be a cheaper alternative. Would it not mean that they have to use 5x less tokens to accomplish the same task? is it a realistic target for me try out?

Next, OpenRouter and such, okay I have been looking at them. There are a few providers with cheaper prices, but they offer 230K context window length at this price not the 1.1M context window. This would not fit with a single shot code writing, debugging and documenting workflow. What are your experience with them? how reliable are these providers?

What are real and feasible alternatives?

DS Flash, Luna and Mimo 2.5 are at similar price levels. But a pro model with high reasoning capacity is something I would also need. There are not many at DS Pro price range.

One positive is that I am in off-peak time zone of DS Official API.

I am pondering upon the idea and does not know the feasibility or if it saves a few bucks compared to the providers such as Deepseek, open code, etc.

Lets say if I dont mind whether my request is completed in 60seconds or 30 minutes. I could batch them and burst them in 30mins and then collect next batch.

In this case, one can use a VM with GPU and run vLLM or LiteLLM and serve models to my applications via a private API. Lets take GCP as an example L4 workstations come at under 1$/hour and A100s come at $2-3/hour depending on capacity. If I use a spot machine, I would get billed for only compute time.

The primary issues, I see are

1. my spot gets reclaimed and stuck with no spot. how does vLLM handle here? How should I handle my client ends here?
2. What kind of models can I serve on these machines? At this price, machines have a memory of around 100GB.
3. costs associated with storage will be there, depending on the size of the model. Also most cloud services charge for storage operations. Attaching disks, reading large models in spot machines can lead to unnecessary overheads.

What else can be the issue? Will it be economically feasible for a 20h usage per week.

Did anyone try something similar? What is your self-hosted non-local setup?

I am pondering upon the idea and does not know the feasibility or if it saves a few bucks compared to the providers such as Deepseek, open code, etc.

Lets say if I dont mind whether my request is completed in 60seconds or 30 minutes. I could batch them and burst them in 30mins and then collect next batch.

In this case, one can use a VM with GPU and run vLLM or LiteLLM and serve models to my applications via a private API. Lets take GCP as an example L4 workstations come at under 1$/hour and A100s come at $2-3/hour depending on capacity. If I use a spot machine, I would get billed for only compute time.

The primary issues, I see are

1. my spot gets reclaimed and stuck with no spot. how does vLLM handle here? How should I handle my client ends here?
2. What kind of models can I serve on these machines? At this price, machines have a memory of around 100GB.
3. costs associated with storage will be there, depending on the size of the model. Also most cloud services charge for storage operations. Attaching disks, reading large models in spot machines can lead to unnecessary overheads.

What else can be the issue? Will it be economically feasible for a 20h usage per week.

Did anyone try something similar? What is your self-hosted non-local setup?

I am using vs code + DS extension+ DS Official API. since about a week, the request fail due to network issues in multi-agent workflows.

is it an extension thing or DS has introduced stricter rate limits to reduce traffic?

I have it on Obsidian and experimenting a bit with a mwe chat harness. in both these cases, I don’t see an issue. but they are pretty slow compared to the coding sessions with multiple agents.

do you guys see a difference since 10days? is there a workaround in VS Code?

https://preview.redd.it/yrwcvamgtdih1.png?width=1362&format=png&auto=webp&s=bdd124ccb3256fa1eb94ea47d84f064fc8051642

Why can't each chat session be a tab? Makes navigating easier.

is there an extension or some setting which changes the sessions into tabs like in browser or in code editor for each file?

Hi Community,

I am thinking of evaluating a few custom sub-agents which will support programming agents on design and debugging reasoning. I would prefer to evaluate it on a pure thinking and reasoning.

is there a repository or a benchmark which is commonly used in this context?

I am looking for suggestions for minimal AI assistants, which can

1. access and edit calendars,
2. has memory (a longer term knowledge base) and can cache.
3. can work with Apple Notes, Notion, may be Obsidian
4. a browser extension or something similar.
5. adding custom MCP

What are your favorite AI assistants, ? what can they do? Would be nice, if they are open source, to allow some customization.

I have been working on a minimal browser based UI for a specific style of investing targeting a very small number of users. I did that because I have been fed up with cloud hosted services which simply add a wrapper for python packages which fetch data + ui layout is distracting.

I prefer to make my repo fully open-source for individuals. They can wire up their custom backends. my dependencies are MIT or Apache2.0. So, at first, it seems like GNU GPLv3 would work.

On the other hand, I am concerned that GNU GPLv3 would allow exactly what the repo is against. Now, I can imagine that the same UI would be hosted on the cloud by just rewriting the back-end with another wrapper.

What are other license alternatives - which completely prevents commerical reuse and allows only individuals to clone/develop this for their private use?

First of all, forget the valuation, if AI is a bubble or not. I am also biased against AI being a sustainable cashflow machine in the medium term, 3-5years, to justify the valuations. The post is mainly my personal opinion based on my personal experience and experience at workplace in programming and research tasks.

Now, let's talk about how will they bring in cash.

It improves the productivity -- yes, agreed. I can do more 3x/4x more in the same time on specific tasks. But I have invested a lot of time in bringing the AI workflow to this point - from where it was 20%-30% productivity boost for me to a 3x/4x now. But, it still requires me to solve hard bugs to make the code usable.

But, at what cost? I dont know how many of you are aware of the recent price changes in GitHub Copilot, which now costs more - on par with other western providers. Most people are using open source models or cheaper Chinese alternatives. Every new model is getting better, but also in the meantime more expensive.

If it improves productivity, either the total human output should skyrocket or should replace humans (which leads to other problems). Then it should cost as much or less than humans. At the moment, it costs me at least as much as an intern@hourly rate - but an intern can do more things - an intern can physically move - conduct experiments, have insights or own ideas. In other words, agentic workflows are nice for even complex and repeatable tasks, but cannot adapt to a new task or bring a new product idea.

In the tasks, I use the agents - as it stands now it costs as much as or more expensive than an intern or a junior. But the variety of tasks, to which a human can adapt themselves fast is much larger. So I assume that the cost has to come significantly down to justify the use case.

Let's think about what is composed in the cost.

1. semiconductor prices

2. infrastructure capitalisation / maintenance

3. Utility costs

4. R&D costs

Now, we justify that R&D is necessary and spend whatever we would need, how will the business model work forwards?

1. semiconductor - chips have to become cheaper for the same power or chipmakers will have to cut down their costs or margins. - both of them go against exponential growth trajectory, negatively affecting margins and limiting the earnings.

2. Infrastructure - money is already spent, must be factored in. The debt service comes due whether you subsidize or make money out of the infrastructure. The capex spend must also increase for a wider adoption to keep up with demand.

3. Utility costs - chips &/or models will have to become extremely efficient. However, with soaring energy prices, which is expected at least for the near to medium term - brings the question - how much efficient the chips and models will have to become to maintain the current costs?

So, what is the business model? those of you who are invested in the AI-drive, how do you see the business model working from here onwards.