I think it’s generally a good idea to use a standalone, local solution. I also like the approach of combining multiple models with different focuses under a single interface. On the other hand, I usually use very specialized models (for programming, for example) and rarely end up generating music or a video on the side. So I can’t think of a reason to use the system right now.
That's a great q and point. I built it to get at a broad approach to tying multimodal capability together in one place as I'm personally building products on top of it that stitch chat, image, video together and wanted to enable others to do the same. For just coding/chat/specialized models I'd say ollama or lm studio is still the better bet generally, but for some of the models I care about I'm trying to optimize pretty deeply and make fine-tuning really simple. Plus I'm often supporting and bouncing between Mac and Cuda boxes so wanted something where the same codebase ran on both.
totally fair! i spent a while trying to get clever and compress what i wanted to say and finally just hit submit but prob lost too much - local inference runtime, one cli that runs image/video/music/speech/3d/etc on your own machine without package hell. would def edit if I still could after your feedback, but window is closed. Thank you!
I think it’s generally a good idea to use a standalone, local solution. I also like the approach of combining multiple models with different focuses under a single interface. On the other hand, I usually use very specialized models (for programming, for example) and rarely end up generating music or a video on the side. So I can’t think of a reason to use the system right now.
What target audience do you have in mind?
That's a great q and point. I built it to get at a broad approach to tying multimodal capability together in one place as I'm personally building products on top of it that stitch chat, image, video together and wanted to enable others to do the same. For just coding/chat/specialized models I'd say ollama or lm studio is still the better bet generally, but for some of the models I care about I'm trying to optimize pretty deeply and make fine-tuning really simple. Plus I'm often supporting and bouncing between Mac and Cuda boxes so wanted something where the same codebase ran on both.
some real numbers on my m4 max -
image gen (zimage-nano), 1024x1024: 58s
image -> textured mesh (trellis.2): 2m 49s
sfx generate (5s clip): 3.6s
music generate (8s, ace-step): 15s
speech synth: 13s | transcribed back: 2.2s
video gen w/ audio(ltx unified-av) 4s,768x512: 2m 48s
text chat (laguna xs2.1): 102 tok/s - https://mlx.fast leaderboard
FYI, the title makes no sense in isolation.
totally fair! i spent a while trying to get clever and compress what i wanted to say and finally just hit submit but prob lost too much - local inference runtime, one cli that runs image/video/music/speech/3d/etc on your own machine without package hell. would def edit if I still could after your feedback, but window is closed. Thank you!
At least add “models”. Doing “music” on your local machine can mean many things.
Yes, missed that completely... Great point!