Am I weird or old-fashioned for relying on direnv over .env? These days my projects use a lot of different tech, and it just feels weird to me to have them all parsing .env files and hoping everything works. Especially with inference provider tokens basically connecting my checking account to the internet.
The concept is fine, but I'm a big fan of doing it with mise instead. It does the exact same thing plus a lot of other things really well. The environment for a project is not just things that get stored in environment variables, it also includes specific versions of tools, common project tasks, git hooks and such. Being able to manage all of that in one place ends up being really useful.
I also really like that for sensitive variables (AWS tokens, prod DB passwords, etc), I can keep those in a dedicated secret store (I use 1Password for personal projects and there are more enterprise focused alternatives for work).
direnv does one thing really well, and it enables a large ecosystem of other components that you can pick and choose to build developer environments. SecretSpec, from the featured article, is one such component which pairs really well with direnv, and has the features you’re describing for secrets.
I think direnv’s approach is the one with more staying power.
Dotenv is definitely something that should not exist.
Direnv is the right way to go if you want a tool for this. Otherwise dotenv's implementation in bash is essentially `set -a; source .env`. Look ma, no custom parser!
Seems really silly to have a library do what two lines of shell can reasonably accomplish.
direnv + nix flakes solved so many problems for me, I have no idea how other people work on multiple projects at the same time without getting mixed up.
Most uses of .env files are for apps or containers imho, so you could just have a .env file in each folder and configure docker compose /node / whatever to pick it up
> But migrating away from .env starts with reading it correctly.
We had a system that mediated all access to the static config through an in-house library. Because nobody touched the environment-specific data directly, it was a convenient spot to layer on reloadable configuration from a Raft source, and fold in environment-specific secrets coming through an old 12 Factor solution, although we could have also integrated some other secrets management as well.
Or, left it as a separate API. That tends to work for tired and distracted people well. Having security related stuff use a unique code pattern to access it does, I find, help keep people from treating it all the same way they treat other data, which is to throw it around profligately and occasionally write it to the logs. The moral equivalent to the guard over the missile launch switch versus the machine gun trigger. One of those has much more forgiving failure modes than the other.
It's the same problem 'capability' based systems frequently run into - capabilities end up meaning that if you can see something you're entitled to use it. And when your coworkers add new features to the system, you end up accidentally being able to see things you shouldn't be able to see.
Java for instance ran into this rather frequently with theirs. Someone would add an object to the global state and not notice that some peer added a reference to a reference to that object with proprietary data the client code wasn't meant to have at all. Information wants to be 'free'.
I think there are two separate concerns here. The idea that long lived secrets survive in plaintext on a disk somewhere is a bad idea that should go away, but also tangential to that people have to have good hygiene around not printing evil things to logs. Latter is a lot harder to catch imo, and is a separate problem
Yeah I ended up finishing a Lava Flow Antipattern, where someone started wrapping of stdout and never finished it. We replaced every value for a key of 'token', 'password', 'secret' with 8 stars in the logs.
We were using Splunk and if you're handing third parties your secrets, you're gonna have a bad time. Sooner or later they'll have a disgruntled employee or a breach.
I have on a number of occasions wondered if there's some way to make something like the 2 key system for nuclear launch protocols work for making destructive changes in production. I request an action, and it only takes effect if someone else repeats it.
The problem though is priming, and a two key system only fixes that the first couple times. In a year you're right back where you started. Because once I've suggested a course of action to you, the Primacy Effect makes you key your counterproposals to my initial bid, and if I'm enough off the mark we end up off the mark together. Add Recency Effect on top of it - if I haven't been wrong the last ten times I asked you to spot me, you slowly stop assuming I'm wrong this time. When the production outage happens, it comes out that your thought process was that I know what I'm doing and you didn't 'need' to be there for the last five times so you've zoned out and didn't catch my mistake.
And if I start my ask by sending you a URL, if I cut and pasted the wrong URL you're stuck looking at the wrong thing with me and we both sail off into the canyon together.
Still, if I'm careful to give you only the high-level request, and we derive the same solution from that request, then maybe a tool like that would still be useful. But the most important part of that interaction is still outside of the scope of any tool that could sanity check us. It's just process.
But the process is important. I've used Shamir secret sharing, ssss, to have it so that the root account on an important server somewhere is only accessible when two of seven people come together and break glass to get access.
.env is self sustaining at this point. The functionality described in the post exists to help move people off of it. It’ll have a better chance of dying when fewer people use it and that starts with making it easy to move away from it.
How exactly changing a default that users rely on is going to make it easy to move away from it?
Instead of migrating to dotenv-ng, these users could be putting effort into migrating away from .env files entirely.
edit: I understand SecretSpec needs to parse .env files exactly like Node.js’s dotenv library, which doesn’t support interpolation, to make it easier for these projects to use SecretSpec.
I’m saying making this a standalone library is bad. Rust projects will pick this up and use it instead of SecretSpec, and users of the unmaintained Rust crate will migrate to it.
Or even simpler: why on earth are you using double quotes for a string if you don't want double-quote behaviour? Anything like this should just be in 'single quotes'... Any developer getting to "its parser changed a secret while reading it" and "By the time the bcrypt bug blocked SecretSpec," from basic shell semantics is frankly astonishing. I mean this on an organisational/team level, not as a personal attack on anyone, but the hubris is incredible to me.
This feels like the exact type of solution a badly prompted and poorly piloted AI tool manically follows to an absurd conclusion if you don't step in and tell it to do it the obvious and correct way instead of re-inventing the wheel.
(I would also argue most of the point of a .env file being what it is that you can source it in a shell?
Does the author here have any sane argument for not using a yaml/json/whatever file and XKCD-927ing it instead? Is this not just introducing a flaw when `source .env` stops behaving like expected?)
I am an old man yelling at cloud, but why is a library that does something this simple something that needs maintenance? Like if your jaw is dropping because an environment variable parser hasn't seen commits since 2023 ... ... How old is /usr/bin/env ? Lots of code churn on that one?
It’s an open source project, so you don’t have to wish it.
“Hey dotenvy team, I’m a fan of your project and wanted to help out. I noticed that some of your announcements are AI generated, and would like to help bring back the human voice. I’ll be happy to proof read and/or write your blog posts and release announcements in the future. Let me know if this is something you could use.”
.env is another "simple" format that turns out to be actually a whole family of languages where each handles corner cases differently. And usually implementations quietly do nonsense on malformed input. Some even have command injection vulnerabilities. Or at least this was the case back when I looked at a few such implementations and wrote up this: https://github.com/panzi/punktum
The beauty of the environment is that it can be manipulated prior to running the program. I don't understand the desire to have runtime .env loaders when that env can just be established before the program is run. Every language can read env without a library. The churn in this space is too high... it's a smell.
I have a shell function for this in my zsh config:
function lenv() {
export $(cat .env | xargs)
}
It doesn't get any more unix than this. I do not allow my team to build software that autoloads environment files using libraries. It's such a mess. That is a concern of the shell.
You're not wrong, but there's a small detail that your shell function doesn't solve for: unloading the environment variable once it's no longer needed.
If you maintain one shell per process-that-needs-a-secret-in-the-env then it doesn't matter, but if you're using long-running shells then direnv can load and unload env vars. It also gives you a nice little heads-up that it loaded/unloaded variables.
Sort of? It definitely does what's needed, but it also removes any additional environment variables that were set, unless they're in your shell initialisation files.
I'm not trying to persuade that these tools are essential or anything, just highlighting that there's a few small details some take care of that're fairly nice.
Am I weird or old-fashioned for relying on direnv over .env? These days my projects use a lot of different tech, and it just feels weird to me to have them all parsing .env files and hoping everything works. Especially with inference provider tokens basically connecting my checking account to the internet.
The concept is fine, but I'm a big fan of doing it with mise instead. It does the exact same thing plus a lot of other things really well. The environment for a project is not just things that get stored in environment variables, it also includes specific versions of tools, common project tasks, git hooks and such. Being able to manage all of that in one place ends up being really useful.
I also really like that for sensitive variables (AWS tokens, prod DB passwords, etc), I can keep those in a dedicated secret store (I use 1Password for personal projects and there are more enterprise focused alternatives for work).
mise is trying to do everything.
direnv does one thing really well, and it enables a large ecosystem of other components that you can pick and choose to build developer environments. SecretSpec, from the featured article, is one such component which pairs really well with direnv, and has the features you’re describing for secrets.
I think direnv’s approach is the one with more staying power.
I'm biased but I think it's silly to think either one is going away anytime soon
> mise is trying to do everything
It also doesn't work well. And they have no support outside of Discord chat.
What problems did you encounter while using mise?
Dotenv is definitely something that should not exist.
Direnv is the right way to go if you want a tool for this. Otherwise dotenv's implementation in bash is essentially `set -a; source .env`. Look ma, no custom parser!
Seems really silly to have a library do what two lines of shell can reasonably accomplish.
Interestingly, using source would have the same "bug" as TFA mentions.
Though the obvious fix is to use single quotes in the .env file.
That's not what the agent suggested. The user asked for a fix, and I guess the AI blindly complied.
That shell snippet will expand shell variables in double-quote strings. And apparently the entire reason for this fork is to not do that.
Which is dumb. The canonical spec for .env should just be "whatever POSIX sh does."
Agree, not expanding is dumb.
One of the major benefits of a shell-based approach is your env vars could be generated from the output of a command.
Which can be trivially used to populate secrets from whatever your secret store of choice is.
Now we've also cleanly solved secret distribution for developers with one less dependency!
I also used to use direnv (and likewise eschewed .env), but more recently I moved env definitions to mise. Been working out nicely.
No, I also use direnv. It does everything I want, nothing I don't, and has been stable for me for years.
direnv + nix flakes solved so many problems for me, I have no idea how other people work on multiple projects at the same time without getting mixed up.
Most uses of .env files are for apps or containers imho, so you could just have a .env file in each folder and configure docker compose /node / whatever to pick it up
A bonus is that with that approach, you actually have real environment variables.
Yup, direnv + Nix. Incredible combo for development.
I use Mise for env. I primarily use it for installing tools and running tasks, having env support is just a cherry on top.
Direnv's first stable release was in 2013, .env was 2012 (adopted by Heroku).
So no, if anything direnv makes you hip and edgy.
> But migrating away from .env starts with reading it correctly.
We had a system that mediated all access to the static config through an in-house library. Because nobody touched the environment-specific data directly, it was a convenient spot to layer on reloadable configuration from a Raft source, and fold in environment-specific secrets coming through an old 12 Factor solution, although we could have also integrated some other secrets management as well.
Or, left it as a separate API. That tends to work for tired and distracted people well. Having security related stuff use a unique code pattern to access it does, I find, help keep people from treating it all the same way they treat other data, which is to throw it around profligately and occasionally write it to the logs. The moral equivalent to the guard over the missile launch switch versus the machine gun trigger. One of those has much more forgiving failure modes than the other.
It's the same problem 'capability' based systems frequently run into - capabilities end up meaning that if you can see something you're entitled to use it. And when your coworkers add new features to the system, you end up accidentally being able to see things you shouldn't be able to see.
Java for instance ran into this rather frequently with theirs. Someone would add an object to the global state and not notice that some peer added a reference to a reference to that object with proprietary data the client code wasn't meant to have at all. Information wants to be 'free'.
I think there are two separate concerns here. The idea that long lived secrets survive in plaintext on a disk somewhere is a bad idea that should go away, but also tangential to that people have to have good hygiene around not printing evil things to logs. Latter is a lot harder to catch imo, and is a separate problem
Yeah I ended up finishing a Lava Flow Antipattern, where someone started wrapping of stdout and never finished it. We replaced every value for a key of 'token', 'password', 'secret' with 8 stars in the logs.
We were using Splunk and if you're handing third parties your secrets, you're gonna have a bad time. Sooner or later they'll have a disgruntled employee or a breach.
> the guard over the missile launch switch
That's called a molly guard, but also a Debian package that asks you before you run shutdown if you're on the server you intended to shut down.
I have on a number of occasions wondered if there's some way to make something like the 2 key system for nuclear launch protocols work for making destructive changes in production. I request an action, and it only takes effect if someone else repeats it.
The problem though is priming, and a two key system only fixes that the first couple times. In a year you're right back where you started. Because once I've suggested a course of action to you, the Primacy Effect makes you key your counterproposals to my initial bid, and if I'm enough off the mark we end up off the mark together. Add Recency Effect on top of it - if I haven't been wrong the last ten times I asked you to spot me, you slowly stop assuming I'm wrong this time. When the production outage happens, it comes out that your thought process was that I know what I'm doing and you didn't 'need' to be there for the last five times so you've zoned out and didn't catch my mistake.
And if I start my ask by sending you a URL, if I cut and pasted the wrong URL you're stuck looking at the wrong thing with me and we both sail off into the canyon together.
Still, if I'm careful to give you only the high-level request, and we derive the same solution from that request, then maybe a tool like that would still be useful. But the most important part of that interaction is still outside of the scope of any tool that could sanity check us. It's just process.
But the process is important. I've used Shamir secret sharing, ssss, to have it so that the root account on an important server somewhere is only accessible when two of seven people come together and break glass to get access.
Can’t you just escape the dollar sign?
If you really believe .env was a mistake (it was), just let it die. You are the ones keeping it alive now.
.env is self sustaining at this point. The functionality described in the post exists to help move people off of it. It’ll have a better chance of dying when fewer people use it and that starts with making it easy to move away from it.
How exactly changing a default that users rely on is going to make it easy to move away from it?
Instead of migrating to dotenv-ng, these users could be putting effort into migrating away from .env files entirely.
edit: I understand SecretSpec needs to parse .env files exactly like Node.js’s dotenv library, which doesn’t support interpolation, to make it easier for these projects to use SecretSpec.
I’m saying making this a standalone library is bad. Rust projects will pick this up and use it instead of SecretSpec, and users of the unmaintained Rust crate will migrate to it.
Or even simpler: why on earth are you using double quotes for a string if you don't want double-quote behaviour? Anything like this should just be in 'single quotes'... Any developer getting to "its parser changed a secret while reading it" and "By the time the bcrypt bug blocked SecretSpec," from basic shell semantics is frankly astonishing. I mean this on an organisational/team level, not as a personal attack on anyone, but the hubris is incredible to me.
This feels like the exact type of solution a badly prompted and poorly piloted AI tool manically follows to an absurd conclusion if you don't step in and tell it to do it the obvious and correct way instead of re-inventing the wheel.
(I would also argue most of the point of a .env file being what it is that you can source it in a shell? Does the author here have any sane argument for not using a yaml/json/whatever file and XKCD-927ing it instead? Is this not just introducing a flaw when `source .env` stops behaving like expected?)
What does the ng at the end means? I have seen a few products/projects that added it and no explanation of the meaning.
I came in and thought it had something to do with Angular since `ng` is a popular pre/post-fix for Angular libraries.
It means new generation. Using ng suffix was the thing to do in the 2000s just like calling something beta or 2.0 in the years following
Probably next generation.
Next generation I guess.
I am an old man yelling at cloud, but why is a library that does something this simple something that needs maintenance? Like if your jaw is dropping because an environment variable parser hasn't seen commits since 2023 ... ... How old is /usr/bin/env ? Lots of code churn on that one?
I love AI development, but I wish product announcements were self-authored.
It’s an open source project, so you don’t have to wish it.
“Hey dotenvy team, I’m a fan of your project and wanted to help out. I noticed that some of your announcements are AI generated, and would like to help bring back the human voice. I’ll be happy to proof read and/or write your blog posts and release announcements in the future. Let me know if this is something you could use.”
Maybe this is a dumb question but what does -ng mean? I see a lot of forks seem to follow the <project>-ng naming convention.
Next generation.
It’s not a great convention.
Ah, ok. That makes sense.
I think mise offers a better solution for the problem.
well mise uses dotenvy right now (for env._file anyhow), it may be worth switching to dotenv-ng but I haven't looked into it yet https://github.com/jdx/mise/blob/fc441787fe52b46be1245046758...
I assume you just mean normal [env] in mise.toml which is different, but this still is relevant to mise
Yes - I think .env files are not a good idea (works for one person but even then it's not great)
.env is another "simple" format that turns out to be actually a whole family of languages where each handles corner cases differently. And usually implementations quietly do nonsense on malformed input. Some even have command injection vulnerabilities. Or at least this was the case back when I looked at a few such implementations and wrote up this: https://github.com/panzi/punktum
The beauty of the environment is that it can be manipulated prior to running the program. I don't understand the desire to have runtime .env loaders when that env can just be established before the program is run. Every language can read env without a library. The churn in this space is too high... it's a smell.
I have a shell function for this in my zsh config:
It doesn't get any more unix than this. I do not allow my team to build software that autoloads environment files using libraries. It's such a mess. That is a concern of the shell.You're not wrong, but there's a small detail that your shell function doesn't solve for: unloading the environment variable once it's no longer needed.
If you maintain one shell per process-that-needs-a-secret-in-the-env then it doesn't matter, but if you're using long-running shells then direnv can load and unload env vars. It also gives you a nice little heads-up that it loaded/unloaded variables.
Use this wrapper instead:
or just `exec $SHELL` to unload.
Sort of? It definitely does what's needed, but it also removes any additional environment variables that were set, unless they're in your shell initialisation files.
I'm not trying to persuade that these tools are essential or anything, just highlighting that there's a few small details some take care of that're fairly nice.
Word splitting works with whitespace in general. You don't need the `xargs`.
doppler all the way