Video understanding is kind of new, especially if done well, and hopefully working well with UI and UX, that'd be great. Current agents already struggle a bit with 2D space with normal screenshots of unconventional UIs, wonder if this model would do better with actual recordings of navigating and using applications, feels like it could help a bunch with understanding UX at least hopefully. Will be fun to play around with :)
Sure, but again, it's a micro 3B model. Perhaps it can't be used for general video work, but it might be able to do basic edits like remove an object from a table in a shot.
Video understanding is kind of new, especially if done well, and hopefully working well with UI and UX, that'd be great. Current agents already struggle a bit with 2D space with normal screenshots of unconventional UIs, wonder if this model would do better with actual recordings of navigating and using applications, feels like it could help a bunch with understanding UX at least hopefully. Will be fun to play around with :)
What’s SOTA for video understanding? AFAIK most video search is powered by transcription and not the actual video. This seems impressive.
Any plans to port to sglang or vLLM?
vllm-omni support is on the way : )
Great quality, forked and going to try
Nice work. Wish they had picked another name given how popular lance/lancedb is.
Seems like the video output is crippled. Resolution is low (720 or so), as is the frame rate. The samples are shown up-scaled and frame-interpolated.
Why do that? Seems strange to be building sub-hd resolution video models in 2026.
Sure, but again, it's a micro 3B model. Perhaps it can't be used for general video work, but it might be able to do basic edits like remove an object from a table in a shot.
It’s not a micro model at all, it requires 40gb of VRAM. The 3B is just the active parameters.
last dance for lance vance!
:D