One feedback from someone interested in using this about the examples: I have looked at several and they seem too high level to get a sense of the actual API (i.e. the expected benefit of using this library vs the development complexity of using it).
For example, the cloth bending simulation is almost entirely: at __init__, call a function to add a cloth mesh to model builder obj, pass built model to initializer of a solver class; and at each timestep: call a collide model function, then call another function called solver.step. That's really it.
I think this is a step in the right direction, but I really dislike the Pythonification of everything. After using IsaacSim/IsaacLab for work, I'm convinced that Python is not the right tool for the job.
Developers inevitably write slow, error filled code when dealing with Python and working with the type annotations can be a pain.
Happy there's something to replace PhysX for robotics, and I do really like MuJoCo's API, but really wish we could get some good C/C++ APIs.
Apart from the language, NVIDIA doesn't seem to be great when dealing with software. IsaacSim and IsaacLab have so many bugs, are incredibly slow, and hard to debug. We spend so many hours on my team findings bugs for IsaacSim, it's just a pain. On version 5.0 and still feels like beta software.
Also IsaacSim's relience on USD to hold the scene structure and update prims makes it so hard to program for. USD isn't really performant when trying to generate a large amount of scenes. And the USD interface stops working completly when simulation starts on IsaacLab. I hope Newton goes a different route, and has less of a reliacne on USD. IMO USD should just be used as an interchange format, rather than how you actually represent the scene and properties internally. I much prefer that approach, which Unreal Engine seems to support.
Lastly, my god the names in this field are terrible. USD (Googling becomes a pain sometimes), Newton (Already another engine), Warp (literally the name of the architecture and a way to write Python GPU kernels, wtf).
It's just a bad programming language. Guido has bad taste, it creates lazy, boring affordances that don't scale beyond a single file without paying for it somewhere.
Note that I'm not saying it's not useful (python is what C was to unix if it was invented now)
Agreed on most, and naming is terrible. Note that at run-time Python is out-of-the loop, since Newton Physics records a CUDA graph, and executes it, so performance is not impacted (aside from startup JIT time for modified kernels). I'd prefer C/C++ as well, and although you can call Warp-compiled kernels from C++ (without Python, see my https://github.com/erwincoumans/warp_cpp project), it would be better to have native C/C++ support without requiring a Python interpreter. It just happens that almost all Deep Learning/RL for robotics uses Python.
Heh, that was my first thought too and it doesn't look like it's related.? The nvidia dudes could have done some minimal amount of googling to pick a name that causes less confusion.
The primary use case of Newton Physics is reinforcement learning, with 1000s of similar environments. Even if each environment would have sequential actions, you run many envs in parallel.
This will eventually replace PhysX, some of its developers are working on Newton Physics. Newton Physics has multiple solvers, including MuJoCo-Warp and is easier to customize and extend.
One feedback from someone interested in using this about the examples: I have looked at several and they seem too high level to get a sense of the actual API (i.e. the expected benefit of using this library vs the development complexity of using it).
For example, the cloth bending simulation is almost entirely: at __init__, call a function to add a cloth mesh to model builder obj, pass built model to initializer of a solver class; and at each timestep: call a collide model function, then call another function called solver.step. That's really it.
I think this is a step in the right direction, but I really dislike the Pythonification of everything. After using IsaacSim/IsaacLab for work, I'm convinced that Python is not the right tool for the job.
Developers inevitably write slow, error filled code when dealing with Python and working with the type annotations can be a pain.
Happy there's something to replace PhysX for robotics, and I do really like MuJoCo's API, but really wish we could get some good C/C++ APIs.
Apart from the language, NVIDIA doesn't seem to be great when dealing with software. IsaacSim and IsaacLab have so many bugs, are incredibly slow, and hard to debug. We spend so many hours on my team findings bugs for IsaacSim, it's just a pain. On version 5.0 and still feels like beta software.
Also IsaacSim's relience on USD to hold the scene structure and update prims makes it so hard to program for. USD isn't really performant when trying to generate a large amount of scenes. And the USD interface stops working completly when simulation starts on IsaacLab. I hope Newton goes a different route, and has less of a reliacne on USD. IMO USD should just be used as an interchange format, rather than how you actually represent the scene and properties internally. I much prefer that approach, which Unreal Engine seems to support.
Lastly, my god the names in this field are terrible. USD (Googling becomes a pain sometimes), Newton (Already another engine), Warp (literally the name of the architecture and a way to write Python GPU kernels, wtf).
It's just a bad programming language. Guido has bad taste, it creates lazy, boring affordances that don't scale beyond a single file without paying for it somewhere.
Note that I'm not saying it's not useful (python is what C was to unix if it was invented now)
Agreed on most, and naming is terrible. Note that at run-time Python is out-of-the loop, since Newton Physics records a CUDA graph, and executes it, so performance is not impacted (aside from startup JIT time for modified kernels). I'd prefer C/C++ as well, and although you can call Warp-compiled kernels from C++ (without Python, see my https://github.com/erwincoumans/warp_cpp project), it would be better to have native C/C++ support without requiring a Python interpreter. It just happens that almost all Deep Learning/RL for robotics uses Python.
Is this related to the Newton Dynamics physics engine? https://newtondynamics.com/
Heh, that was my first thought too and it doesn't look like it's related.? The nvidia dudes could have done some minimal amount of googling to pick a name that causes less confusion.
No
> Newton extends and generalizes Warp's (deprecated) warp.sim module, and integrates MuJoCo Warp as its primary backend.
It’s MuJoco GPU Edition. Nothing new or improved.
Well, MuJoCo initially used JAX for GPU (MJX) and MuJoCo Warp replaces MJX with better performance.
How do they parallelize the sequential actions ?
The primary use case of Newton Physics is reinforcement learning, with 1000s of similar environments. Even if each environment would have sequential actions, you run many envs in parallel.
Years ago there was PhysX... how does this compare?
This will eventually replace PhysX, some of its developers are working on Newton Physics. Newton Physics has multiple solvers, including MuJoCo-Warp and is easier to customize and extend.