Twitter User @_xjdr (with @doomslide) has been working on new sampling techniques and getting some early promising results. The hope is to achieve an open source version of what o1 does.
The key insight is to use entropy and the variance of entropy across tokens to guide sampling.
Twitter User @_xjdr (with @doomslide) has been working on new sampling techniques and getting some early promising results. The hope is to achieve an open source version of what o1 does.
The key insight is to use entropy and the variance of entropy across tokens to guide sampling.