26 comments

  • emscb 3 minutes ago ago

    So the agent compares the changed workflow with the existing one and judges whether the speedup is significant. But a change that re-shards or skips tests would always look like a win on speed alone, wouldn't it? How do you check that such changes still catch the bugs the old workflow caught? I had the same worry when I trimmed our tests because CI was too slow.

  • bekacru an hour ago ago

    Bereket from Better Auth - we've been on StarSling for about a year now, it's been such a hard a problem for us ot solve CI to the amount of PRs we get and continues need of optimization and faster runners, im so bullish!

    • yonasb 34 minutes ago ago

      Merging the runner migration was such a proud moment for us! So glad we've been able to take CI off your plate :)

  • colek42 4 hours ago ago

    We should probably be measuring deployable merges to main. The math just doesn't work for pipeline based CI as you add more agents.

    Three things matter.

    1 -- How many non-conflicting changes can actually be made at one time (Fan Out)

    2 -- How long a critical path takes

    3 -- How many times you have to run that critical path. Most orgs run it 2-3x, once before the PR in the agent loop, once in CI, and another in the MQ

    This is why most orgs cant get past about 12 PRs an hour.

    • jwilliams 18 minutes ago ago

      > 3 -- How many times you have to run that critical path. Most orgs run it 2-3x

      This is the one we went after. Our agents have to run tsc, lint and the unit tests before they push (it's in a hook), and CI was then doing all three again from scratch. Now the agent records what it ran as a git trailer on the commit and the PR build skips those steps [1]. Anything headed to main still runs everything, so the trailer only has to be trusted on the intermediate pushes.

      Builds went from ~10 minutes to 1.5 over a few weeks, and a good third/half of that was this change. It tends to compound too - getting the build time down led to lower contention, warmer caches, faster throughput. All the usual queuing theory stuff.

      [1] https://jonathannen.com/dont-run-it-twice/Context

    • dbworku 4 hours ago ago

      Definitely agree that agent volume will push beyond pipelined CI. Most teams we’ve talked to are still working to get up to 100 PR/hour.

  • jstrasner 4 hours ago ago

    So good! CI is still such a pain. Out of curiosity, do you use any open source harnesses under the hood or did you roll your own?

  • dotmike 5 hours ago ago

    Congrats on the launch! Will give it a run. Why is your CI fast? Curious to hear the technical details

    • dbworku 4 hours ago ago

      Our primary difference is self-improving PRs from continuously analyzing the telemetry from your CI. Most folks just sell fast hardware and are happy to charge you for jobs that don't saturate the host.

      In our examples with open-source repos above you'll see that most of speedups that lead to a 10X speedup come from code changes or optimizations.

  • buildwrangler 4 hours ago ago

    Congrats! I spent alot of time scaling really large and complex CI and integration testing builds so its great seeing a product that can help devs with this.

    • yonasb 3 hours ago ago

      Thank you! Yeah most teams don't have someone with your experience so they end up with really inefficient GHA setups. Doing our best to help!

    • dbworku 3 hours ago ago

      Thanks! It’s an unappreciated bottleneck so we feel your pain.

  • sochs 3 hours ago ago

    Off topic but every time I read a name like "Star Sling" I get excited to see another space startup tackling a difficult challenge; and so when it turns out to be CI for agents, it's a letdown and an annoyance. Idk, this project seems cool, but maybe pick a less pretentious name.

    • yonasb 2 hours ago ago

      We chose the name because our original inspiration was an agentic developer portal. So we thought Stargate + Dr. Strange's sling ring that opens portals was a cool mashup. Sorry to disappoint :(

  • gkoberger 5 hours ago ago

    Congrats on the launch! These benchmarks look really great, can't wait to start using this.

    What are some of the most interesting things the AI has caught + fixed so far?

    • yonasb 8 minutes ago ago

      I was surprised that mature projects got some of the biggest wins. gogs is a 48K-star repo and one 20 line fix got it a 5x speedup:

      https://github.com/starsling-demo-org/benchmark-gogs/blob/sh...

    • dbworku 5 hours ago ago

      Oof. It's hard to pick one! I like the subtle fixes like swapping sleeps in tests with DB readiness checks or pinpointing a single long running test causing a shard to run slow.

  • esafak 3 hours ago ago

    As you note, we can do those optimizations ourselves. The kind of person looking for CI runners is likely doing that already. What I value is speed, cost, scheduling delay, reliability, observability, local reproducibility, not using YAML... There is a lot of value to provide but workflow optimization is not where I would look.

    • Bnjoroge an hour ago ago

      I think YAML used to matter more when humans were writing much of the code, but they matter alot less now, especially because you can already write much of your YAML workflows in high level code. It's still matters, but alot less than it used to or enough to be a pain. I agree with the rest of the concerns though.

    • yonasb 3 hours ago ago

      That's fair. At a certain point, once you value all of those things enough, you'll just move off of YAML entirely (we think of this as Level 4 of CI https://starsling.dev/blog/levels-of-ci#level-4). But I think there are a lot of people still trying to get the biggest gains with the least amount of effort. That's who we're going after first.

  • d-sanoj 4 hours ago ago

    Congrats on the launch. This seems interesting.

    • yonasb 4 hours ago ago

      Thanks! If CI is on your priorities list, give us a shot.

  • kinfish 4 hours ago ago

    congrats on the launch!

    • yonasb 3 hours ago ago

      Thank you!