Hi yall. Sorry for the clickbait-y title, but it’s true. We have been working on a coding harness that compilers your whole ahh codebase (language agnostic) into a literal playfield for any model of your choosing. It is producing insanely good results (78.2% on SWE-bench Verified for <10 cents a fix), reads 2-6x less source code than Claude Code, DeepSeek Harness, OpenCode, and the like, has a runtime tracer, and so much more. Would love some early feedback! Benzi is strictly work in progress
An earnest question: what does this actually do? Is the idea here to make code scans in AI more efficient, so the token cost per engineering prompt is reduced?
(For context, I am a senior BE engineer, fairly AI literate, and I use Claude for dev work every day. I don’t come close to the spend limits of the lowest possible tier, and I regard my AI tooling as giving me a 3-4x speed-up, as well as improving my task focus. I am not yet using sub-agents.)
Great question!
token cost per engineering prompt does go down, yes, but that is a side effect of the philosophy Benzi is built around. Benzi is built to avoid reading source code in the first place (obviously do RAG as needed); basically, all programming languages have functions, variables, a class like thing, imports, refs n bindings, etc. right? Benzi compiles a language agnostic map, assigning all symbols and relations UIDs, so the agent reasons over that.
Econimically valuable because yes, amount of source code read goes down 2-6x when compared to CC, DSH or OC (9k vs 20k vs 43k vs 65k+. detils here: Benzi — Bug-Fix Benchmark ). but really what makes benzi stands out is what decoupling deterministic intelleigence from artifical intelligence produces. All those thinking tokens go into exhausitvely understanding millions of lines of code in miliseconds and connecting all the dots because everything is just a O(1) lookup away for the agent.
PLEASE ask more questions been dying to talk to someone about it haha
Righto. Is not reading source code valuable independent of saving token spend? I am partly asking out of genuine curiosity, but also out of a touch of devil’s advocacy: if I went to my boss and said “can I borrow the company credit card to get a Benzi subscription”, they’d want two or three succinct, layman’s, business-related reasons to give me the nod.
(I appreciate your project isn’t at a subscription stage, and maybe you don’t intend to ever offer it commercially, but I think the question stands. I have my own side-projects, and I acknowledge that producing an “elevator pitch” is hard for engineers, myself included).
“Is not reading source code valuable independent of saving token spend?” VERYYYYY much so. highly reccomend giving it any repo at all of your choice. it will dissect it like no other AI: https://benzi.fly.dev/ – and this shows up when Benzi writes code too. like greenfielding with it is a daydream because it ‘cognitively’ undeerstands how the whole repo fits together.
Benzi will remain free to use for now till we get enough traction lol. But yes, hoping it grows into something bigger than just a hobby project!
Fair enough. I might be being a bit playfully provocative, but “try it” is not enough to persuade my grumpy boss. She’ll say, perfectly understandably, that Claude understands our codebase well already, and is smart enough to produce decent product/technical suggestions, even if we end up pushing back on some of them. I’m using Claude Code with its own console harness, Sonnet 5, on medium-level thinking.
(Bonus note - I hate all the AI companies, especially the ones rushing to IPO before the whole house of cards tumbles down. I thus am minded, at some point, to switch to an open source model running on something like OpenCode Go. But I acknowledge I have some good velocity on Claude at present, and do not especially want to upset my focus).