Exa-d: How to store the web in S3

(exa.ai)

42 points | by willbryk 3 days ago ago

5 comments

  • swyx 2 days ago ago

    hi will! super nicely written, nice look under the hood of your processing. as an orchestration guy i always wondered why everyone seems to converge on using Ray, and as a secondary thought, how well is Anyscale capturing the Ray market.

    if i were doing what you do i might set up a lot of rate limits/anomaly detection in case some weird unintended invalidation causes a weird spike in your dependency graphs. is there good practice there for anomaly detection other than "setup a bunhc of dashboards and be on call"?

    • twyxy 2 days ago ago

      Ray is the future

  • timvdalen 2 days ago ago

    Opening this page makes my (quite beefy) machine grind to a halt! Almost all CPU threads and the GPU jump up to 80% usage

    • neilv 2 days ago ago

      Would be funny if blog visitors were the distributed compute nodes.

  • 3 days ago ago
    [deleted]