This looks really useful. I have a home server with various VMs and containers, but I don't do server admin very often, so whenever something breaks I have to find out how to fix it each time. Having terminals in a document just like you can mix code and documentation in a Python notebook can probably make admin easier for people like me who do it rarely. And of course in a professional setting something like this improves the bus factor in a lot of projects if you can keep things documented.
I think this is a potential building block for transforming how MSP organisations manage documentation and workflow.
Despite the goal of homogenizing each clients environment to simplify support, there will always be uniqueness amongst them. Having documentation that can be collaboratively edited, versioned, that is also runnable to perform management tasks would be a HUGE deal.
I understand this isn't the target audience, but its exciting to imagine the possibilities of other uses.
I agree! We're definitely not trying to avoid them as users haha, I can certainly see how it would be useful. There's so much automation that just can't fit into a strict schema, and always needs a human or customization for 10% of it
We're aiming to be flexible enough, so that if you can do it in a terminal then you can _also_ do it with a runbook
reminds me a bit of a "gradual automation" project i encountered a while ago; the idea was that you wrote a shell script that was simply a series of "do this thing and press enter when it's completed" prompts to capture a workflow precisely, and then gradually replacing steps with automated scripts that did the same thing.
It’s definitely baked into the approach here. You could just write a bullet point list of what needs doing. Then maybe sprinkle some blocks on top so the user can do things faster. Then in the end, work towards the whole thing being automated
Hey, thanks for building this. I'm trying to use it for this usecase. I'm trying to pause the script during the manual steps. How would you go about this? Example:
1. Change working directory block
2. Interactive terminal
3. Manual Step
4. Interactive terminal
5. Manual step
If I press run at the top, then it doesn't stop for me to do 3 before executing 4. Should I run each step individually for this usecase? Will changing the directory in step 1 apply to steps 2 and 4 if I don't hit run at the top?
Not directly related to this new Atuin feature, but I need to vent:
Last week I was trying to `find` something in some directories, failed, `cd`d to my home directory and instinctively hit up-arrow+return to run the search again. At some time prior to this, Atuin had stopped recording new entries without my notice. Want to guess the last entry that Atuin did record?
Go on. Guess.
Yep.
`rm -rf *`
In my home directory.
Luckily I have backups of everything important and didn't actually lose anything, and I'm mainly posting this here as a funny anecdote. But - still - after getting myself set up again I have yet to reinstall Atuin.
Now, I think you could do #5 or #6 (and add `mv` and `dd` as well, but where does the list end?), but I think #1 (using the absolute path) is the easiest to avoid the worst PEBCAK.
"Destructiveness" property is undecidable in general. If you ban rm from history, you'll just get false sense of security before you accidentally run some "aws bla bla drop production cluster"
Behavior of any system should be just one of:
1. Fully determinate
2. Have enough latency before confirmation (for example, block input for 1 second after displaying a command)
This should apply to history, any fuzzy searching, autocomplete etc
Technically it’s possible to run each command in some restricted cgroup for example, and ask for elevated permissions if anything more is required. But that would require quite some rethinking on how the whole shell is supposed to work.
No it is not a reasonable fix to this issue. You can't classify a command whether it's destructive or not. It depends on a lot of context. The classification logic needs to run every time you invoke a command. It needs to gather all the context to make a decision, every time you run a command. It's going to slow everything down. People will have different opinions on what is destructive, leading to endless debates. We don't need to run logic just to recall a history entry. Stop.
Since atuin is a feature enhanced shell history replacement maybe you should look into its features that you could have used to prevent this. Like not allowing destructive commands with wildcards in memory.
I’m now running openbsd on my laptop, and I’ve yet to enable history save to file. And to date, I’ve not missed it. Anything that should be saved across sessions can be an alias, a function, a script, or a snippet in some notes.
the age-old trick for some of this is when running something you don't want in history, put a bunch of spaces before it.
It requires you to think about that when doing it ... but, well... I dunno. I really don't ever write `rm -rf *` even in that state of mind. That's like the most evil command one can type into a machine! Who knows what it will do!
Been using atuin for some time now. I love it! I wish there was a proper profile option though as I dont want my work related and non-work related reverse searches to mix.
Glad to see that this is now open from closed beta!
Hmm, I downloaded and installed Atuin a few years ago as a solution mainly for syncing shell history across machines or on the same machine across terminal instances (e.g. in tmux). I thought that was its main use case. How did we get to runbooks? I'm kinda having trouble figuring out what it does based on the linked page, but I think that's mostly because I'm trying to tie it back to my understanding of atuin as a shell history database
I have been a big fan of Atuin CLI and an active promoter for a while now! It has truly become one of the first pieces of software I install in any brand new machine. Excited to try Atuin desktop now. Great work to Ellie and the rest of the team. Looking forward to seeing what's next!
Love atuin - has saved my ass many more times than I can count. The more you guys can monetize the better; will help keep the base product good. Even pretty senior devs (who don’t always love changing their workflows) can find a lot of value in it.
I would pay you guys for E2EE syncing, but I think it’s free at the moment. Charge me!
In my latest company we made a bash script called "run". We put a bunch of commands in there like "setup", "download-certs", "deploy-staging", "deploy-prod", "fix", "test"
so you run like this:
./run setup
And all the commands can be listed by running:
./run
If I wanted a chart the command could spit out a png somewhere and tell you about it.
There are no dependencies to run the script, and if any of the commands needed dependencies you could just put them in setup and call setup from a commmand.
This is very useful, though I would recommend using a Makefile instead.
You will get a lot out of the box: completion in your shell, idempotency (in case you are interested), useful string interpolation, easy variables injection, etc.
You could keep the content of your bash functions close to intact by adding the
".ONESHELL:" target (and some \$).
This looks great! At a previous job we had a fork of Jupyter notebooks that were used this way by some teams. I see that remote execution is on the roadmap, but was also wondering if you'll have some form of parallel remote execution as well (ie one runbook run across 10 or 100 VMs similar to parallel ssh). Definitely more complicated than single execution, but potentially very powerful for debugging fleets where you don't have Chef or Ansible. I guess the alternative is to just have the runbook run locally but run pssh in some commands to get a similar result.
we already support execution of script + terminal blocks over SSH, but want much tighter integration. Parallel execution is certainly a part of that too. anything else you'd want to see?
Nothing in particular - when I wrote / used the jupyter 'runbooks', they were most helpful when a SEV (site event / severe error) was happening or if a new person on the team needed to handle oncall the first time.
Any examples of what diffs between versions of workbooks typically look like? If we collaborate and share via git - is it like markdown with code blocks - or something less accessible?
I'm aware this isn't ideal, and longer term we want something that looks much more like markdown. A key requirement for us here is that it's easy + convenient to type - just like normal markdown - so putting everything as a code block annotation isn't ideal.
Well, this looks very nice. I have so far avoided runbooks, preferring to use Ansible or such. Installed, and will see if I'll change my habits. Containers and such have made Ansible usage more cumbersome.
Also, I noticed there's only 60+ Atuin sponsors (at Github), so added myself. Been using Atuin for some while now. Hopefully their work is sustainable.
The backend server, Atuin Hub (https://hub.atuin.sh), isn't currently open source, but the open source desktop app can still connect to it. We also support offline workspaces if you prefer using Git or another VCS to manage and collaborate on runbooks, rather than using Atuin Hub.
Following the Obsidian model, which I love and support. Give folks the best part of the product, offer a paid option to enhance it, but allow folks to use alternatives as first class options.
Thank you for the candid response. So team collab without the hub is still possible, but the desktop client becomes a viewer for a hit repo. Am I understanding that correctly? The advantage of the hub is that that is all handled under the hood?
A viewer and editor, but essentially yes — the Hub handles sharing and collaboration for you, as well as realtime collaborative editing and a few other features (and likely a few more to come). We want the Hub to feel like the most frictionless way to collaborate on runbooks, but we understand that some folks don't want host their runbooks with a third party, and sometimes fitting into an existing VCS workflow makes the most sense.
Is the format human readable? Let's say I am on a foreign machine where I do not have Atuin desktop installed, could I trivially copy and paste from this run book (ignoring templating)? Or would I need some kind of export step to dump the commands? It made references to committing to source control, so I assume it must be something diffable, but I could not find an example raw file.
copy-paste of another comment but I think it answers everything here! The only addition I'll add is that if you copy-paste into another Atuin instance, the richness is preserved. If you copy paste to anything else, it'll be pasted as markdown.
---
The runbook files are just YAML (for now). They're more verbose than I'd like, but do convey a lot of information - both rich text and executable.
I'm aware this isn't ideal, and longer term we want something that looks much more like markdown. A key requirement for us here is that it's easy + convenient to type - just like normal markdown - so putting everything as a code block annotation isn't ideal.
Oof, not pretty, but I guess it gets the job done (although I am always rooting for my dark horse, TOML). I suppose I could technically grep for `code` if I were in a bind.
This looks really useful. I have a home server with various VMs and containers, but I don't do server admin very often, so whenever something breaks I have to find out how to fix it each time. Having terminals in a document just like you can mix code and documentation in a Python notebook can probably make admin easier for people like me who do it rarely. And of course in a professional setting something like this improves the bus factor in a lot of projects if you can keep things documented.
Yes exactly! I have so many systems both personally + professionally, and Desktop has made it so much easier to maintain and work with them all
Let us know if you try it and have any feedback or questions at all
Also see https://runme.dev for a similar approach or https://speedrun.cc if you'd like it to work straight from GitHub markdown.
Not a single mention of AI, LLMs, or Agents. I'm impressed :)
(Hi, author here!)
I can’t guarantee we never will, but I am only interested in such things if they are genuinely useful
If we won’t use it internally, and no significant % of users will, then it won’t happen
This is The Way
I think this is a potential building block for transforming how MSP organisations manage documentation and workflow.
Despite the goal of homogenizing each clients environment to simplify support, there will always be uniqueness amongst them. Having documentation that can be collaboratively edited, versioned, that is also runnable to perform management tasks would be a HUGE deal.
I understand this isn't the target audience, but its exciting to imagine the possibilities of other uses.
I agree! We're definitely not trying to avoid them as users haha, I can certainly see how it would be useful. There's so much automation that just can't fit into a strict schema, and always needs a human or customization for 10% of it
We're aiming to be flexible enough, so that if you can do it in a terminal then you can _also_ do it with a runbook
I use Emacs' org-mode to do this.
Here's an article on how to accomplish this: https://howardism.org/Technical/Emacs/literate-devops.html
reminds me a bit of a "gradual automation" project i encountered a while ago; the idea was that you wrote a shell script that was simply a series of "do this thing and press enter when it's completed" prompts to capture a workflow precisely, and then gradually replacing steps with automated scripts that did the same thing.
Huge fan of gradual automation
It’s definitely baked into the approach here. You could just write a bullet point list of what needs doing. Then maybe sprinkle some blocks on top so the user can do things faster. Then in the end, work towards the whole thing being automated
No all or nothing, no YAML needed ;)
Hey, thanks for building this. I'm trying to use it for this usecase. I'm trying to pause the script during the manual steps. How would you go about this? Example: 1. Change working directory block 2. Interactive terminal 3. Manual Step 4. Interactive terminal 5. Manual step
If I press run at the top, then it doesn't stop for me to do 3 before executing 4. Should I run each step individually for this usecase? Will changing the directory in step 1 apply to steps 2 and 4 if I don't hit run at the top?
looking forward to playing with it!
Not directly related to this new Atuin feature, but I need to vent:
Last week I was trying to `find` something in some directories, failed, `cd`d to my home directory and instinctively hit up-arrow+return to run the search again. At some time prior to this, Atuin had stopped recording new entries without my notice. Want to guess the last entry that Atuin did record?
Go on. Guess.
Yep.
`rm -rf *`
In my home directory.
Luckily I have backups of everything important and didn't actually lose anything, and I'm mainly posting this here as a funny anecdote. But - still - after getting myself set up again I have yet to reinstall Atuin.
Destructive options shouldn't be allowed to be run from history without confirmation. That seems like a reasonable fix to this issue.
A shell history has no knowledge about what the commands do.
You could also make a mistake by executing command #101 instead of #102.
1) This is why you never type rm -rf * but the absolute path.
2) Furthermore, the command flag -f implies never prompt (taken from a recent GNU coreutils man page):
"-f, --force ignore nonexistent files and arguments, never prompt"
3) This is merely unlinking; the data is still there, not overwritten.
4) You should have backups of your homedir. A filesystem with versioning like ZFS could be of help here, too.
5) Agree with you and add a blacklist to the history, with rm being a primary contender.
6) Instead of rm, use a system where you move files into a trash bin (ie. abstract the unlinking in a user-friendly interface such as the trash bin or recycle bin concept). Examples: https://github.com/imnyang/tsh https://github.com/Byron/trash-rs
Now, I think you could do #5 or #6 (and add `mv` and `dd` as well, but where does the list end?), but I think #1 (using the absolute path) is the easiest to avoid the worst PEBCAK.
"Destructiveness" property is undecidable in general. If you ban rm from history, you'll just get false sense of security before you accidentally run some "aws bla bla drop production cluster"
Behavior of any system should be just one of: 1. Fully determinate 2. Have enough latency before confirmation (for example, block input for 1 second after displaying a command)
This should apply to history, any fuzzy searching, autocomplete etc
Technically it’s possible to run each command in some restricted cgroup for example, and ask for elevated permissions if anything more is required. But that would require quite some rethinking on how the whole shell is supposed to work.
No it is not a reasonable fix to this issue. You can't classify a command whether it's destructive or not. It depends on a lot of context. The classification logic needs to run every time you invoke a command. It needs to gather all the context to make a decision, every time you run a command. It's going to slow everything down. People will have different opinions on what is destructive, leading to endless debates. We don't need to run logic just to recall a history entry. Stop.
Since atuin is a feature enhanced shell history replacement maybe you should look into its features that you could have used to prevent this. Like not allowing destructive commands with wildcards in memory.
I’m now running openbsd on my laptop, and I’ve yet to enable history save to file. And to date, I’ve not missed it. Anything that should be saved across sessions can be an alias, a function, a script, or a snippet in some notes.
That’s right, man. Bury me with my zshrc.
the age-old trick for some of this is when running something you don't want in history, put a bunch of spaces before it.
It requires you to think about that when doing it ... but, well... I dunno. I really don't ever write `rm -rf *` even in that state of mind. That's like the most evil command one can type into a machine! Who knows what it will do!
I mean, the same thing would happen if Bash stopped writing to `~/.bash_history` and its last item was `rm`, right?
when was the last time this happened to you?
I believe zsh asks for a confirmation before doing that. Which shell are you using?
Been using atuin for some time now. I love it! I wish there was a proper profile option though as I dont want my work related and non-work related reverse searches to mix.
Glad to see that this is now open from closed beta!
Hmm, I downloaded and installed Atuin a few years ago as a solution mainly for syncing shell history across machines or on the same machine across terminal instances (e.g. in tmux). I thought that was its main use case. How did we get to runbooks? I'm kinda having trouble figuring out what it does based on the linked page, but I think that's mostly because I'm trying to tie it back to my understanding of atuin as a shell history database
I have been a big fan of Atuin CLI and an active promoter for a while now! It has truly become one of the first pieces of software I install in any brand new machine. Excited to try Atuin desktop now. Great work to Ellie and the rest of the team. Looking forward to seeing what's next!
What's the difference between this and Jupyter? Besides this seemingly needs an account with Atuin to work.
> What's the difference between this and Jupyter?
It's noob friendly. Even the installation process. No need to manage python. Just use the app.
> Besides this seemingly needs an account with Atuin to work.
You don't need an account to use it.
Love atuin - has saved my ass many more times than I can count. The more you guys can monetize the better; will help keep the base product good. Even pretty senior devs (who don’t always love changing their workflows) can find a lot of value in it.
I would pay you guys for E2EE syncing, but I think it’s free at the moment. Charge me!
In my latest company we made a bash script called "run". We put a bunch of commands in there like "setup", "download-certs", "deploy-staging", "deploy-prod", "fix", "test"
so you run like this:
And all the commands can be listed by running: If I wanted a chart the command could spit out a png somewhere and tell you about it.There are no dependencies to run the script, and if any of the commands needed dependencies you could just put them in setup and call setup from a commmand.
This is very useful, though I would recommend using a Makefile instead. You will get a lot out of the box: completion in your shell, idempotency (in case you are interested), useful string interpolation, easy variables injection, etc.
You could keep the content of your bash functions close to intact by adding the ".ONESHELL:" target (and some \$).
Just is better choice imho
https://github.com/casey/just
This looks great! At a previous job we had a fork of Jupyter notebooks that were used this way by some teams. I see that remote execution is on the roadmap, but was also wondering if you'll have some form of parallel remote execution as well (ie one runbook run across 10 or 100 VMs similar to parallel ssh). Definitely more complicated than single execution, but potentially very powerful for debugging fleets where you don't have Chef or Ansible. I guess the alternative is to just have the runbook run locally but run pssh in some commands to get a similar result.
absolutely!
we already support execution of script + terminal blocks over SSH, but want much tighter integration. Parallel execution is certainly a part of that too. anything else you'd want to see?
Nothing in particular - when I wrote / used the jupyter 'runbooks', they were most helpful when a SEV (site event / severe error) was happening or if a new person on the team needed to handle oncall the first time.
Any examples of what diffs between versions of workbooks typically look like? If we collaborate and share via git - is it like markdown with code blocks - or something less accessible?
The runbook files are just YAML (for now). They're more verbose than I'd like, but do convey a lot of information - both rich text and executable.
There's an example here: https://github.com/atuinsh/desktop/blob/8ebed35d7efe68a92363...
I'm aware this isn't ideal, and longer term we want something that looks much more like markdown. A key requirement for us here is that it's easy + convenient to type - just like normal markdown - so putting everything as a code block annotation isn't ideal.
The current sketch of what that could be looks like this: https://forum.atuin.sh/t/desktop-devlog-markdown-vim-local-f...
Would love to know if you have any thoughts!
Org Mode syntax is an extremely natural fit
Well, this looks very nice. I have so far avoided runbooks, preferring to use Ansible or such. Installed, and will see if I'll change my habits. Containers and such have made Ansible usage more cumbersome.
Also, I noticed there's only 60+ Atuin sponsors (at Github), so added myself. Been using Atuin for some while now. Hopefully their work is sustainable.
What is the catch? This looks professionally done. Is there a paid service behind this? Is the open source teams version limited in some way?
The backend server, Atuin Hub (https://hub.atuin.sh), isn't currently open source, but the open source desktop app can still connect to it. We also support offline workspaces if you prefer using Git or another VCS to manage and collaborate on runbooks, rather than using Atuin Hub.
Following the Obsidian model, which I love and support. Give folks the best part of the product, offer a paid option to enhance it, but allow folks to use alternatives as first class options.
Thank you for the candid response. So team collab without the hub is still possible, but the desktop client becomes a viewer for a hit repo. Am I understanding that correctly? The advantage of the hub is that that is all handled under the hood?
A viewer and editor, but essentially yes — the Hub handles sharing and collaboration for you, as well as realtime collaborative editing and a few other features (and likely a few more to come). We want the Hub to feel like the most frictionless way to collaborate on runbooks, but we understand that some folks don't want host their runbooks with a third party, and sometimes fitting into an existing VCS workflow makes the most sense.
Candid responses like this are an excellent way to earn good will, thank you!
If the backend isn't open source then the project isn't "fully open source" as stated in the documentation.
Interesting way to manage bash scripts and tooling.
On first glance, I kind of like it - bash scripts are often ideal due to being largely universal and can minimize dependancies.
I find and if this can be an in between to let anyone access configs, or processes, that might be bearable.
Any plans to make it possible to run shellcheck on the embedded scripts?
we could absolutely plug it into the codemirror linter! thanks for mentioning
issue: https://github.com/atuinsh/desktop/issues/47
Seems similar to Warp Notebooks!
I love Warp Notebooks. And I'm definitely interested in At in Desktop, although, if they are run books, the naming isn't quite matching up for me.
#NamingIsHard
Hello! Atuin maintainer here
Happy to answer any questions
Is the format human readable? Let's say I am on a foreign machine where I do not have Atuin desktop installed, could I trivially copy and paste from this run book (ignoring templating)? Or would I need some kind of export step to dump the commands? It made references to committing to source control, so I assume it must be something diffable, but I could not find an example raw file.
copy-paste of another comment but I think it answers everything here! The only addition I'll add is that if you copy-paste into another Atuin instance, the richness is preserved. If you copy paste to anything else, it'll be pasted as markdown.
---
The runbook files are just YAML (for now). They're more verbose than I'd like, but do convey a lot of information - both rich text and executable.
There's an example here: https://github.com/atuinsh/desktop/blob/8ebed35d7efe68a92363...
I'm aware this isn't ideal, and longer term we want something that looks much more like markdown. A key requirement for us here is that it's easy + convenient to type - just like normal markdown - so putting everything as a code block annotation isn't ideal.
The current sketch of what that could be looks like this: https://forum.atuin.sh/t/desktop-devlog-markdown-vim-local-f...
Would love to know if you have any thoughts!
Oof, not pretty, but I guess it gets the job done (although I am always rooting for my dark horse, TOML). I suppose I could technically grep for `code` if I were in a bind.
any way I could use the desktop app to run my runbooks in my remote servers?
Mostly - we have ssh blocks that cause the subsequent shell and terminals blocks to run remotely
We don’t yet support full remote execution, but should do soon!
Is this supposed to completely replace your docs? Or just supplement the commands portion of it?
It's aimed at replacing developer docs realistically - anything where you're explaining how to run or query things.
BUT it does also make a nice environment for rich text, with markdown support, so can easily replace internal docs in general
What are some of the examples of workflows which people have built with atuin desktop ?
Careful, it's CLAware.
What do you mean by this?
I think they mean it's developed under a contributor license agreement.
https://github.com/atuinsh/desktop/blob/main/CLA.md
So, jupyter notebooks?
This is more like a local ci / cd workflow
Or, org mode.
Yep. Org Babel was my first thought.
How?
org babel lets you execute scripts embedded in org docs.
In my infrastructure repo the README and playbook files have executable org babel scripts for common debugging activities.
...but with a proprietary remote backend and can run commands on your desktop without any sandbox.
org tangle
Literate programming
big fan <3
why desktop and not web based!?
Donald Knuth