JevAutomatisering

Alex Stone
@a1exstoneAlex Stone: TYPESAFE RELEASED A MODEL THAT DOESN'T GENERATE TEXT, AND IT BREAKS THE COST FLOOR
Originalinlägg / EN
TYPESAFE RELEASED A MODEL THAT DOESN'T GENERATE TEXT, AND IT BREAKS THE COST FLOOR OF THE AGENT LOOP
jev is what they call a system one model. you send it a state and a set of questions, it returns typed answers with probabilities. no tokens, no sampling, no sequential decoding.
route → decide → guard → run tool → observe → done?
they report 20-200x faster inference and 40-400x lower cost than comparable llms on classification tasks. every question in a request is evaluated in parallel, so adding questions barely moves latency.
in a two-turn loop that means eight decisions. the two that need a plan still go to a chat model. the other six - pick a model, screen a tool call, check if the task is done - cost 0.06s combined.
the takeaway is simple: stop paying generation prices for choices. most of what an agent decides is a pick, a score or a yes/no.- visningar
- 2 tn
- gilla-markeringar
- 29
- sparningar
- 17
- återpubliceringar
- 3
Länkar och projekt
Citerat inlägg
Sydney Runkle @sydneyrunkle
https://t.co/CF54CU9Ddz
Vi visar offentliga inlägg med uppgift om författare och källa. Rättigheterna till text och medier tillhör respektive ägare.