Rendered at 10:04:42 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
frangonf 11 hours ago [-]
For evaluating subs you have to put the money and try by yourself to get your own measurement, since the subs quotas and new models are always changing and overall info is unreliable.
20$ of Codex/Google/Chinese gives you some fair amount of usage to test them, and Opencode Go for 10$ lets you try a good amount of models with good quota. I don't use Openrouter because it gets more expensive than the subs, but for testing, swapping and being completely independent, a proxy service is the best solution.
About benchmarks, I usually agree with DeepSWE. Looking at usage rankings of models in Openrouter is also a good signal.
For changing models locally I just use pi (used also opencode in the past) or if the sub does not allow login in external harnesses I just use whatever cli they have, last year there were differences but today they are all good enough for my needs and have basically the same features.
flemhans 2 days ago [-]
Ask AI to choose it for you ;)
One of my recurring cron-like LLM jobs is to scout the state of affairs and test new models, and swap them in if deemed useful
ghilston 2 days ago [-]
Our of curiosity what do you use to run this? An actual cron or a tool like hermes etc
wseqyrku 3 days ago [-]
> I see lots of post on here discussing the most cost-effective models for coding and people swapping around a lot.
Could you link some of those here?
solomonmwalker 3 days ago [-]
Pretty much any post talking about a new model has folks saying, "I tried it out and saw <what they saw>".
20$ of Codex/Google/Chinese gives you some fair amount of usage to test them, and Opencode Go for 10$ lets you try a good amount of models with good quota. I don't use Openrouter because it gets more expensive than the subs, but for testing, swapping and being completely independent, a proxy service is the best solution.
About benchmarks, I usually agree with DeepSWE. Looking at usage rankings of models in Openrouter is also a good signal.
For changing models locally I just use pi (used also opencode in the past) or if the sub does not allow login in external harnesses I just use whatever cli they have, last year there were differences but today they are all good enough for my needs and have basically the same features.
One of my recurring cron-like LLM jobs is to scout the state of affairs and test new models, and swap them in if deemed useful
Could you link some of those here?
Here's an example. https://news.ycombinator.com/item?id=49537553