Taste in the Loop

This post covers Databricks’s efforts to measure the cost vs performance Pareto frontier of various model and harness (Pi, Codex, Claude Code) combinations.

Two big takeaways from their post: 1) GLM 5.2 marks an inflection point for open-weight models, as it’s competitive on the Pareto frontier with GPT 5.5. and Opus. 2) Harness choice matters a lot and can have a huge impact on the cost-per-task.

This is a great post that covers a combination of harness engineering, scaling LLMs as a judge, and RL post-training an openweight model (GLM 5.1) to reach the pareto-frontier of cost/performance. This post also includes a categorical breakdown of where their post-trained model improved the most during training.

Mitchell Hashimoto’s tweet roughly matches our experience thus far:

I had early access to 5.6/Sol for ~month. Sol is my default. It is faster, plans/judges just as good as Fable, and I think produces better overall work. I’ll reach for Fable still for highly targeted debug or performance work with clear reward functions.

Frontier open-weight models require a lot of high-bandwidth memory. For those interested in running LLMs locally on consumer hardware, Apple’s upcoming M7 chip looks like a godsend.

Apple has recently overhauled its entire M-Series chip plans, scrapping the launch of the M6 Pro and M6 Max processors and jumping straight to the M7 series. Mark Gurman, in the PowerOn newsletter, confirmed that Apple’s plans for the M7 Ultra chip include much faster local AI with more CPU, GPU, and NPU power, as well as a total memory capacity of 1.5 TB! The 1.5 TB figure will be about double what Apple is preparing for its M5 Ultra variant. Reportedly, Apple is testing the M5 Ultra SoC in the Mac Studio, featuring about 36 CPU cores, 80 GPU cores, and approximately 768 GB of memory.

If you’re interested in getting into finetuning your own models locally on consumer hardware, you might want to check out Cactus Needle. This is 26m parameter model that was distilled from Gemini 3.1. The Github repository includes tools to finetune this model given training examples you supply.