What comes after attention? This startup says it already knows.

What comes after attention? This startup says it already knows.

Subquadratic is beginning to back up its ambitious claims with benchmarks and third-party validation for its SubQ 1.1 Small model, which uses its proprietary Sparse Attention (SSA) architecture to dramatically improve long-context performance. Rather than comparing every token to every other token, SSA selectively processes relationships, enabling near-linear scaling while maintaining high accuracy across context windows of up to 12 million tokens. The company reports near-perfect retrieval performance, competitive coding and reasoning benchmarks, and compute savings of up to 1,000x at maximum context lengths.

Rather than targeting frontier models immediately, Subquadratic is focusing on enterprise customers that need efficient analysis of massive datasets. The current model was built by replacing the dense attention mechanism in an existing open-weight model and then continuing long-context pretraining. Looking ahead, the startup plans to release a larger mid-tier model while continuing research into "zero attention" architectures that could eliminate attention mechanisms altogether, with the long-term goal of surpassing today's transformer-based AI models in both efficiency and capability.

Learn more from The New Stack around cloud spending:

The context window has been shattered: Subquadratic debuts a 12-million-token window

What comes after attention? This startup says it already knows.

Join our community of newsletter subscribers to stay on top of the news and at the top of your game.

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(300)

Bit Cloud’s next chapter starts after the AI builds your app

Bit Cloud’s next chapter starts after the AI builds your app

For many developers, turning an AI-generated prototype into maintainable software requires more than generating code—it requires infrastructure, collaboration, testing and review. In this episode of T...

1 Okt 31min

CloudBees just committed to an AI-first pivot. Here's why it matters for enterprise DevOps teams

CloudBees just committed to an AI-first pivot. Here's why it matters for enterprise DevOps teams

CloudBees CEO Mo Plassnig is leading the CI/CD company through a major transformation as generative AI reshapes software development. Returning to CloudBees eight years after joining through its acqui...

30 Sep 23min

A third option is emerging in the fight over AI and your data

A third option is emerging in the fight over AI and your data

The AI industry has faced a growing enterprise dilemma: companies want access to powerful proprietary AI models without risking sensitive data or intellectual property, while AI labs want to protect t...

23 Sep 30min

Drowning in AI pull requests: Harness's field CTO on code review and a Git repo built for agents

Drowning in AI pull requests: Harness's field CTO on code review and a Git repo built for agents

Harness Field CTO Martin Reynolds joins The New Stack to talk about what happens after coding agents start opening pull requests faster than anyone can review them. He explains how he first saw the bo...

7 Sep 26min

How to find failures without drowning in tracing data

How to find failures without drowning in tracing data

Traces provide a detailed view of a request’s journey through data, microservices and applications, helping SREs pinpoint where failures occur and resolve issues faster. But while tracing can reduce d...

3 Sep 31min

Why CPUs still matter in the age of AI agents

Why CPUs still matter in the age of AI agents

As AI evolves from conversational chatbots to autonomous agents, CPUs are becoming an increasingly important part of the infrastructure equation. In this episode, The New Stack speaks with Bhumik Pate...

11 Aug 26min

Why Doist Says Less AI Can Deliver More

Why Doist Says Less AI Can Deliver More

Doist CTO Gonçalo Silva says AI is reshaping software development, but success depends on restraint rather than rapid feature expansion. Instead of chasing every AI capability, Doist prioritizes “subt...

31 Juli 35min

Why your company should (try to) build its own AI SRE

Why your company should (try to) build its own AI SRE

As AI coding agents accelerate software development, they also create new challenges for site reliability engineers (SREs), who are increasingly responsible for debugging systems that no single human ...

30 Juli 30min

Populärt inom Politik & nyheter

svenska-fall
aftonbladet-krim
fordomspodden
p3-krim
rss-krimstad
aftonbladet-daily
flashback-forever
spar
de-fyras-gang
rss-vad-fan-hande
rss-sanning-konsekvens
omni-podd
svd-ledarredaktionen
rss-krimreportrarna
politiken
en-runda-till
rss-flodet
motiv
rss-expressen-dok
ett-rent-noje