Milind S

Localized reading

좋아요, Jev는 스크린샷이나 LLM 없이도 컴퓨터를 정말 잘 사용할 수 있고 Mac에서 픽셀이 떠나지 않습니다. Dom 요소도 읽지 않습니다. 로컬 CoreML 모델이 SC의 모든 버튼과 UI 요소를 분할합니다.

Original post / EN

Okay so Jev can actually do computer use really well Without any screenshots, or LLMs and no Pixels leave my mac I dont even read the Dom elements A local CoreML model segments every button and UI element on screen. On-device OCR reads the labels. That text is all Jev gets. It returns a probability across those elements and tells me the best one to click. Then it clicks, re-runs detection, and decides again. In a loop until the goal is done. ~90ms per decision. Faster than any LLM computer use I've tried. Blazing fast computer use, without any latency @typesafeai is building something really interesting