My hot take is that open models don't really save you money and introduce more router complexity and security risk (because you're now sending your company data through more less trustworthy providers). Look at cost per task not cost per token and the pareto curve is largely owned by closed models.
Just use Fable 5.1/Opus max for the hardest problems, GPT Sol high as your workhorse, and maybe terra for async batch stuff you don't really care about. Gemini 3.8 High also looks pretty good and is quite fast if you're already a GCP shop. You can basically benefit from open models without using them because they force the frontier models to be cheaper.
We run Gemini fast for random end user queries for general staff.
We have our own on-premise inference server (quad MI300A) that runs Kimi 2.8 extremely well and we transitioned all heavy work to it since it's basically instantaneous for the whole team. It's a good enough solution and we will hit break even before the end of the year already.
Not everyone needs frontier models and availability is frequently much more important than a lot of companies realize.
Just use Fable 5.1/Opus max for the hardest problems, GPT Sol high as your workhorse, and maybe terra for async batch stuff you don't really care about. Gemini 3.8 High also looks pretty good and is quite fast if you're already a GCP shop. You can basically benefit from open models without using them because they force the frontier models to be cheaper.