New top story on Hacker News: Show HN: PhAIL – Real-robot benchmark for AI models
Show HN: PhAIL – Real-robot benchmark for AI models
11 by vertix | 8 comments on Hacker News.
I built this because I couldn't find honest numbers on how well VLA models [1] actually work on commercial tasks. I come from search ranking at Google where you measure everything, and in robotics nobody seemed to know. PhAIL runs four models (OpenPI/pi0.5, GR00T, ACT, SmolVLA) on bin-to-bin order picking – one of the most common warehouse operations. Same robot (Franka FR3), same objects, hundreds of blind runs. The operator doesn't know which model is running. Best model: 64 UPH. Human teleoperating the same robot: 330. Human by hand: 1,300+. Everything is public – every run with synced video and telemetry, the fine-tuning dataset, training scripts. The leaderboard is open for submissions. Happy to answer questions about methodology, the models, or what we observed. [1] Vision-Language-Action: https://ift.tt/WGDEdsa
11 by vertix | 8 comments on Hacker News.
I built this because I couldn't find honest numbers on how well VLA models [1] actually work on commercial tasks. I come from search ranking at Google where you measure everything, and in robotics nobody seemed to know. PhAIL runs four models (OpenPI/pi0.5, GR00T, ACT, SmolVLA) on bin-to-bin order picking – one of the most common warehouse operations. Same robot (Franka FR3), same objects, hundreds of blind runs. The operator doesn't know which model is running. Best model: 64 UPH. Human teleoperating the same robot: 330. Human by hand: 1,300+. Everything is public – every run with synced video and telemetry, the fine-tuning dataset, training scripts. The leaderboard is open for submissions. Happy to answer questions about methodology, the models, or what we observed. [1] Vision-Language-Action: https://ift.tt/WGDEdsa
New top story on Hacker News: Show HN: BreezePDF – Free, in-browser PDF editor
Show HN: BreezePDF – Free, in-browser PDF editor
23 by philjohnson | 10 comments on Hacker News.
BreezePDF lets you edit, sign, merge, compress, redact, OCR, fill forms, extract tables, and use 30+ more PDF tools — all in the browser, no sign-up. Files never leave your computer. I built it because when people search Google for common PDF tasks, many of the tools they find upload documents to a server. I wanted an option that keeps files local instead. I posted an earlier version on HN last spring: https://ift.tt/WYbt3y0 At the time it only supported a small set of features. Over the last 10 months I rebuilt large parts of it and expanded it to nearly 40 tools, including several ideas that came from comments in that earlier thread. There is also now a desktop app for macOS, Windows, and Linux, plus a CLI/SDK for developers.
23 by philjohnson | 10 comments on Hacker News.
BreezePDF lets you edit, sign, merge, compress, redact, OCR, fill forms, extract tables, and use 30+ more PDF tools — all in the browser, no sign-up. Files never leave your computer. I built it because when people search Google for common PDF tasks, many of the tools they find upload documents to a server. I wanted an option that keeps files local instead. I posted an earlier version on HN last spring: https://ift.tt/WYbt3y0 At the time it only supported a small set of features. Over the last 10 months I rebuilt large parts of it and expanded it to nearly 40 tools, including several ideas that came from comments in that earlier thread. There is also now a desktop app for macOS, Windows, and Linux, plus a CLI/SDK for developers.
Subscribe to:
Comments (Atom)