Nice job. As far as graphics go, I've actually done some experimentation with attempting to build sprites aligned to a well defined grid using Gen AI. You can get surprisingly far by setting up an image of a cartesian grid where each cell can be thought of as a single pixel.
Once you down-sample using nearest neighbor all the lines disappear and you're left with more traditional pixel art 32x32 tiles.
Yeah character walking (and honestly any animation) is still a real pain. I've seen some people try to use video models almost like a rotoscoping tool but it always looks kind of sus honestly.
Maintaining consistency across a set of loopable frames still takes a fair amount of manual work. Here's an example I put together using controlnet and openpose.
Out of curiosity, did you have any previous experience with game development to be able to actively critique the direction that the agent was developing towards?
I don't have any professional experience but I did build a pacman clone in Turbo Pascal (on MS DOS) back in the 90s. Also I've been playing games all my life so have a feeling for what's important - collision detection, walkable/collidable areas, speed/timing, some basic NPC logic etc. So that seemed to have been sufficient in this case :-)
Using progressive disclosure with Codex is a fascinating way to handle complexity, but game physics are notoriously difficult to validate via text-based checklists. Since collision detection and movement can be subtle or visual, how did your Playwright setup distinguish between "working" mechanics and edge cases like glitching through walls? I'm curious if the Implement -> Evaluate loop ever got stuck cycling on a specific bug where the agent couldn't satisfy the test criteria without human intervention. Did you have to define specific tolerance thresholds for the physics engine to prevent false positives in the evaluation phase?
Incredibly I didn't do anything, I just told codex to use playwright cli, told it what to check (in plain English), and it did its thing. Looking at its log I can see that it was "playing" the game and defining its own test conditions, such as if the player/NPC is *not* on one of the "collidable" tiles, if the NPC is "going over the edge" of a collidable area, if it's facing the wrong way, etc. Sometimes it found bugs, e.g. it was testing for gravity checks and then it found that one of the movements was not working correctly and it went ahead and fixed it.
So essentially it uses CLI to read all the x,y coordinates, speed, timing, it took screenshots, and combined those together.
My learning from this is - just let the agent do it. Actually trying to interfere with specific conditions and checks lowers the agent's performance. Simply give it a guide.
Nice job. As far as graphics go, I've actually done some experimentation with attempting to build sprites aligned to a well defined grid using Gen AI. You can get surprisingly far by setting up an image of a cartesian grid where each cell can be thought of as a single pixel.
Once you down-sample using nearest neighbor all the lines disappear and you're left with more traditional pixel art 32x32 tiles.
https://mordenstar.com/other/nb-sprites
Sweet! Have you had any luck with motion based ones, such as a character walking or an action sequence? That's where I've had the most issues.
Yeah character walking (and honestly any animation) is still a real pain. I've seen some people try to use video models almost like a rotoscoping tool but it always looks kind of sus honestly.
Maintaining consistency across a set of loopable frames still takes a fair amount of manual work. Here's an example I put together using controlnet and openpose.
https://mordenstar.com/other/hobbes-animation
That example actually looks really good!
This is very cool.
Out of curiosity, did you have any previous experience with game development to be able to actively critique the direction that the agent was developing towards?
Also, hope you dont mind, but I added it to the HN Arcade :) https://hnarcade.com/games/games/gothicvania
Thank you, and I don't mind at all!
I don't have any professional experience but I did build a pacman clone in Turbo Pascal (on MS DOS) back in the 90s. Also I've been playing games all my life so have a feeling for what's important - collision detection, walkable/collidable areas, speed/timing, some basic NPC logic etc. So that seemed to have been sufficient in this case :-)
Using progressive disclosure with Codex is a fascinating way to handle complexity, but game physics are notoriously difficult to validate via text-based checklists. Since collision detection and movement can be subtle or visual, how did your Playwright setup distinguish between "working" mechanics and edge cases like glitching through walls? I'm curious if the Implement -> Evaluate loop ever got stuck cycling on a specific bug where the agent couldn't satisfy the test criteria without human intervention. Did you have to define specific tolerance thresholds for the physics engine to prevent false positives in the evaluation phase?
Incredibly I didn't do anything, I just told codex to use playwright cli, told it what to check (in plain English), and it did its thing. Looking at its log I can see that it was "playing" the game and defining its own test conditions, such as if the player/NPC is *not* on one of the "collidable" tiles, if the NPC is "going over the edge" of a collidable area, if it's facing the wrong way, etc. Sometimes it found bugs, e.g. it was testing for gravity checks and then it found that one of the movements was not working correctly and it went ahead and fixed it.
So essentially it uses CLI to read all the x,y coordinates, speed, timing, it took screenshots, and combined those together.
My learning from this is - just let the agent do it. Actually trying to interfere with specific conditions and checks lowers the agent's performance. Simply give it a guide.
Impressive. Higher resolution graphics would be super sexy.
I love old school platformers but definitely worth trying to replicate this process with a higher res game.