AI Coding Live News

Together AI Launches Together Compiler: Agents Continually Optimize Production Inference, Up to 2.6x Higher TPS

On November 4 Together AI launched Together Compiler (TKCC), which pairs a swarm of AI agents with a persistent inference engine to keep rewriting inference kernels for real traffic, models and hardware, as part of what it calls Polymorphic Inference. The company says that within 24 hours it reached up to 2.6x higher TPS on models such as Kimi K3, GLM 5.3, MiniMax M3.1-Flash and DeepSeek v4.1-Flash, up to 50% faster on customer workloads, and it already serves trillions of tokens a day in production.

Key Takeaways

  • Together Compiler recasts inference optimization from a classic compiler problem into an agentic search: agents propose kernel changes, a persistent engine checks speed and correctness, and only verified changes reach production
  • Together says that within 24 hours it reached up to 2.6x higher TPS on Kimi K3, GLM 5.3, MiniMax M3.1-Flash and DeepSeek v4.1-Flash, and up to 50% faster on customer workloads
  • On InferenceX low-latency batch size 1 it reports 1,111 TPS on MiniMax M3.1-Flash and 1,205 TPS on DeepSeek v4.1-Flash (company-reported); SemiAnalysis said it validated one of the results
  • The persistent engine hot-reloads candidate forward passes, cutting time to serving to 9 seconds versus 467 seconds for restarting an engine, with 252x higher experiment throughput than baseline agentic optimization systems
  • It already serves trillions of tokens a day in Together's production stack, where 40+ models are now optimized by agents
Read full story

Ollama 0.35.0: Decision Models via /v1/systemone (Jev); Nimble & Tev1 return choices, probabilities, confidence

Ollama shipped v0.35.0 on 2026-09-28 (GitHub prerelease): Decision Models via TypeSafe’s Jev API at /v1/systemone. Instead of free text, models answer choice/noul/score questions with options, probabilities, and confidence—for ticket triage, model routing, and classification. First models: Nimble (Bespoke Labs) and Tev1 (Together AI). ~7 commits / 66 files vs v0.34.4; Settings no longer blocks on discovery; macOS update menu and stalled MLX downloads fixed.

Ollama 0.35.0: Decision Models via /v1/systemone (Jev); Nimble & Tev1 return choices, probabilities, confidence

Key Takeaways

  • v0.35.0 (prerelease) adds Decision Models at POST /v1/systemone (TypeSafe Jev)
  • Question types: choice (with probabilities), noul, score — not free-text generation
  • First models: Nimble (Bespoke Labs) and Tev1 (Together AI) on ollama.com/library
  • ~7 commits / 66 files vs v0.34.4; Settings/macOS update/MLX download fixes
Read full story

Together Chat offers free GLM-5.3 and GLM-5.3 Flash with no API setup

Together AI (@togethercompute) says you can try GLM-5.3 and GLM-5.3 Flash free on Together Chat (chat.together.ai)—no API setup, just pick a model and prompt—served on secure North American infrastructure for quick hands-on access to Zhipu's latest coding/agent models.

Key Takeaways

  • Together Chat offers free GLM-5.3 and GLM-5.3 Flash with no API key setup.
  • Open chat.together.ai, pick a model, and start prompting for coding/agent trials.
  • Served on secure North American infrastructure for try-before-you-integrate workflows.
Read full story

Together AI + Equinix + NVIDIA: Inference Exchange brings open models to global DCs

Together AI (@togethercompute) announces Equinix Inference Exchange with Equinix and NVIDIA: its open-model platform is live across Equinix’s global data centers, productizing both which model enterprises run and where it runs — arguing model choice and performance are not a trade-off.

Key Takeaways

  • Equinix Inference Exchange launches Together’s open-model platform across Equinix global DCs.
  • NVIDIA-backed enterprise inference path tying model choice to where workloads run.
  • Positions open source as the default enterprise AI build path without sacrificing performance.
Read full story
Linked Models & Agents:
ADSponsored

Together: GLM-5.3 Flash cascade hits 80.9% on DeepSWE at $1.70/task

Together ran about 900 DeepSWE rollouts of GLM-5.3 vs GLM-5.3 Flash: 69.0% pass@1 at ~$3.99 per task versus 63.4% at ~$0.24, a 17x cost gap. Distillation mostly removed first-try reliability, not coverage—the pass@4 gap shrinks to 2.6 points. A Flash-then-escalate cascade reached 80.9% at about $1.70 per task, beating the full model alone at less than half the price. Coding agents should default to Flash and promote on test failure, but Flash broke an already-passing baseline in 6.9% of rollouts versus 4.4%, so diffs still need a regression gate.

Key Takeaways

  • On DeepSWE, GLM-5.3 is 69.0% pass@1 at ~$3.99; Flash is 63.4% at ~$0.24, about 17x cheaper.
  • A Flash-first cascade that escalates on test failure hits 80.9% at ~$1.70 per task.
  • Flash breaks passing baselines more often (6.9% vs 4.4%), so accept diffs only behind regression tests.
Read full story
Linked Models & Agents:

Together cuts Dedicated Inference on H100s to $3.99/hr for September

Together AI is cutting Dedicated Inference on H100s from $5.49/hr to $3.99/hr for September, with the lower rate applied automatically to new and existing deployments. Supported options include Gemma 4, Qwen3/3.5, gpt-oss, Llama, Nemotron 3.5 Lightning, or a bring-your-own LoRA. For teams that pin coding agents or fine-tunes to dedicated GPUs, the change is a straight cut in 24/7 serving cost without a migration.

Key Takeaways

  • H100 Dedicated Inference drops from $5.49/hr to $3.99/hr in September, auto-applied to all deployments.
  • Catalog includes Gemma 4, Qwen3/3.5, gpt-oss, Llama, Nemotron 3.5 Lightning, and BYO LoRA.
  • Existing dedicated endpoints get the cut with no migration.
Read full story

Together: GLM-5.3 beats GPT-5.6 Sol and Claude Fable 5 on agentic benchmarks

Together AI said GLM-5.3 now beats GPT-5.6 Sol and Claude Fable 5 on agentic benchmarks, with GLM-5.3 Flash close behind. Zhipu kept the GLM-5.2 base and scaled post-training—more long-horizon environments, more diverse tasks, and more RL compute—rather than training a new foundation model. For coding agents, that is a claim that post-training stack height can overtake closed flagships, and that the cheaper open-weight default is already on Together’s API.

Key Takeaways

  • Together reports GLM-5.3 ahead of GPT-5.6 Sol and Claude Fable 5 on agentic benchmarks; Flash is next.
  • The gain is post-training on the GLM-5.2 base: more long-horizon RL, not a new pretrained model.
  • Coding agents can switch to a cheaper open-weight default without waiting on a new foundation run.
Read full story

Together AI and HUMAIN sign 250MW open-source data center deal, $5B+ annualized

Together AI said it signed one of the largest AI infrastructure deals for open source: a 250MW data center with HUMAIN in Saudi Arabia, carrying more than $5 billion in annualized revenue. The company framed the pact as open-source compute catching up with closed models that have long been bankrolled by hyperscalers, and argued power—not chips alone—is the binding constraint. For teams that fine-tune and serve open weights, the deal points to a step-change in dedicated capacity without each lab raising its own multi-billion-dollar balance sheet.

Key Takeaways

  • The deal is a 250MW data center with HUMAIN in Saudi Arabia and more than $5B in annualized revenue.
  • Together positions it as open-source compute catching up with hyperscaler-backed closed models.
  • The company says power is the bottleneck and partnerships beat every lab raising its own billions.
Read full story
Linked Models & Agents:

Together AI Launches GLM-5.3 Flash: Natively Multimodal 320B-A18B MoE with 1M Context Window

Together AI added GLM-5.3 Flash to its cloud inference catalog, offering native multimodal understanding, 1M context with hybrid attention, and double the DeepSWE throughput per dollar compared to proprietary peers.

Key Takeaways

  • 320B-A18B sparse MoE architecture minimizes cost per token while sustaining frontier-grade reasoning quality;
  • Native multimodality with 1M hybrid-attention context designed for comprehensive codebase inspection;
  • High-throughput enterprise hosting tailored for autonomous continuous coding loops.
Read full story
Linked Models & Agents:

Together AI Serves GLM-5.3 Flash: 320B-A18B MoE with 1M Context Delivering 2x Work per Dollar on DeepSWE

Together AI launched hosted Day-1 inference for GLM-5.3 Flash (320B-A18B MoE, 1M context), nearly matching Luna on DeepSWE benchmarks while completing more than twice the work per dollar.

Key Takeaways

  • Day-1 managed serving of 320B-A18B MoE lowers deployment hurdles for custom hybrid-attention architectures;
  • DeepSWE evaluations demonstrate over 2x work completed per dollar compared to closed frontier models;
  • Available immediately via Together AI Model Market and unified developer API for dedicated endpoints.
Read full story
Linked Models & Agents:
ADSponsored

Together AI Launches Hosted GLM-5.3 Flash API: 320B Multimodal MoE with 2x Output per Budget on DeepSWE

Together AI launched hosted serverless API serving for GLM-5.3 Flash, highlighting its 320B-A18B multimodal MoE architecture, 1M context, and high DeepSWE efficiency compared to proprietary frontier models.

Key Takeaways

  • Brings 320B total / 18B active multimodal MoE into hosted serverless APIs with instant elastic scaling;
  • Native visual reasoning combined with hybrid attention reduces input token consumption for agentic workflows;
  • Matches frontier-level engineering accuracy on DeepSWE while delivering more than double the completed tasks per dollar.
Read full story
Linked Models & Agents: