6 comments

  • mathisdev7 11 hours ago ago

    Nice, thanks!

    edit: in the docs, the command to install the npm package seems to be wrong (crawlie doesn't exist), it should be: npm i -g @spronta/crawlie

    • seandotexe 10 hours ago ago

      Thank you for catching! The docs were a bit too eager on the package simplifying I worked on (oops!).. "npm i -g crawlie" is now correct

      we're shipping fast, will break less things now though I promise ;)

  • SeriousM 16 hours ago ago

    That's nice! I like that it also checks for the llms.txt. It seems to me that no SEO agency is aware of this simple but powerful file.

    • seandotexe 14 hours ago ago

      Thank you!! Lots more to come

      Honestly, the industry is half-torn on whether llms.txt does anything... we're in such a new world. But the same ol' SEO tricks still apply for AI engines for sure.

  • davekilleen 11 hours ago ago

    Nice

    • seandotexe 10 hours ago ago

      Appreciate it Dave! Excited for you to try it out