Under this framework, Rust's async fns would not be colored, since any function in the callstack could be blocking on an executor. if you consider "spawn point" a part of the callstack, many other languages are similar.
This doesn't feel like a useful distinction, but i don't think "function coloring" in general is a useful framework.
I think this is a fine distinction to make, though it's worth noting that function arguments _can be_ colors as the author describes if the arguments have to come from a specific place, e.g. in languages where only `main` receives certain arguments relating to system functionality, or a capability framework where only the top-level function receives the capability token.
I don’t quite understand why the author says Context doesn’t count here. I’ve certainly experienced a top-down version where because a function starts accepting a Context, and therefore needs to be cancelable, lots of functions it calls now need to take a Context too, and check for cancellation. Perhaps the color here is cancelability, not the Context per se. (Which may be even worse, as a kind of “invisible color”.)
There's usually a way to run async on another thread (or this thread) and block on it, so async is not a color by your definition.
I think color should be defined in terms of what you're likely going to do. You can call asyncio.run or context.Background or unsafePerformIO but they're both hacks that you shouldn't do because they are not correct, so we ignore them for colour analysis.
As I said to another comment, this isn't a place for architecture astronautics or squinting until one thing looks like another. It's all assembler under the hood in the end and that language not only doesn't have color, it doesn't have types in any practical sense of the term. At the level of Haskell itself, IO is a color and IO is not a function argument. The compiled code is all assembler. The Haskell may itself be an interpreter for a language that has no color. Above it there is no color, below it it may be implementing something without colors, but at the level of Haskell itself, IO is a color.
In Bluefin, IOE (which allows you to do I/O, i.e. what Haskell's IO also allows you to do) is an argument, and therefore not a color under your interpretation.
How often does a conceptually incremental change correspond to a proportionally incremental code diff, vs. a surprisingly whole-world-upside-down rearchitect?
Do the code patterns naturally leave enough room for you to incrementally shift/transpose how things are organized?
When the answers to these are unfavorable, I often find what effects to "coloring" in the framework, library, or language, sometimes all at once.
Pedantically, colored functions in Go might be ones that produce their output on a channel rather than returning a value. Then, you need select/for-range/<- to get their result, similar to “await”.
Practically, the runtime’s ability to suspend and move goroutines around means that the impact of that (niche/rarely appropriate as a default approach) kind of coloring is much less in Go than in other languages.
> What would it even look like for the top-most function to have to know “everything” that all the child functions end up ever needing? For every function parameter change in the entire program to somehow force a change all the way up the entire call stack? Clearly this does not happen. It so thoroughly does not happen that you may be having difficulty even imagining what it is I am talking about.
Really? In my experience, it certainly does happen and I'm generally happy (but not thrilled) when it does. It means the type system is doing its job, forcing me to provide all the dependencies that a function needs. If 38 layers deep something now needs a connection string that it didn't have before, that had better bubble up to the top-most function. Unless of course the top-most function has already provided it to an intermediate layer which can break the chain, but that is very different than the context.Background() example.
Yeah, in my experience it absolutely can happen, and the way it generally looks is that the dependencies get roughly grouped into larger structs which are then being passed down the call stack mostly as-is, until they hit a certain abstraction level/barrier where the parts of those structs start being passed down.
It's somewhat tedious, that's why modularity becomes even more important: with correctly chosen borders, the grouping of dependencies becomes very clean and self-evident, the whole data flow just flows. Chose them wrong, and you spend most of your time adding and removing those pesky extra arguments.
Yes, when considering software design it's great when you can "smuggle in" a new dependency by adding a new property or method on some already-established payload. I would describe this as "The change requires someone way up the stack to change, but all the intermediate layers require no source changes" which is notably absent from the author's list of "three basic cases that can emerge." I assume the author is well-aware of this technique but considers it equivalent (for the purpose of function coloring) to "The change may require a change to all functions in the stack" since technically the signature of all those intermediate layers did change.
Your configuration structs are not "everything". You probably have dozens of things in them. You have hundreds to thousands of different parameters to different functions. By the time your configuration structs have hundreds of things in them you have thousands to tens of thousands of parameters to functions. Only a fairly small fraction of things make it all the way up to main.
This is exactly what I was talking about with this being a "cognitively available" operation. You notice when you have to propagate something all the way up to the top precisely because it is not something you do very often and it stands out as an exception when you do. If you are literally propagating everything up to main, all the time, every time you change any function anywhere in your program, you are doing something very, very wrong.
I chose a connection string as an example of something that obviously cannot be originated in the middle of the call stack. The difference between passing a raw connection string or passing a connection factory is important for good software design, but I considered them equivalent in the context of this discussion of coloring.
If you're using it diligently enough to where some functions can be statically proven "pure", then yes. You can't call an impure function from a pure one.
IMO it is a syntax thing. If you need to use a special keyword (eg. func vs async func) then it is colored. If it is a semantics thing (eg. parameters) is is not colored.
I'm not familiar with how Rust's async works. What if Rust used the function argument approach, how would that work? Would people pass around a reference to an async executor as arguments (or similar)?
A subset of function arguments are function colors because async/await is isomorphic to requiring and passing a `callback` arg around. It was named 'callback hell` because there wasn't an escape hatch and it affected every function in the callstack.
I'd venture to guess that there's a more abstract way of formalizing this in the sense that monad instances are required to be threaded through the callstack in particular cases(would love to see this spelled out formally) where IO and Cont(?) have this requirement but State like the author points out, does not.
I think this is a case where you want to avoid architecture astronautics and deal with languages as they are presented. Yes, in the end it's all continuation passing, or depending on the cut of your jib, it's all just assembler in the end, but meanwhile, down in the trenches, there are real, practical differences in color that change how you program in those languages, and that's the topic color addresses.
I cited the case of an argument color already in the text for Zig... in fact, that's conditionally a color depending on what is passed, which is even more sophisticated then an argument merely being unconditionally a color.
What some people want to do is call every argument a "color", that is, I changed this function to have to take a Username and now it's Username-colored is the claim. I supposed I should have made that more clear. This provides a definition where that is not the case.
"Function colors" are just effect systems, which are not a new concept while also being extremely useful and super nifty. I hope someday we can stop wringing our hands in a panic over "colored functions" just because some ancient version of Javascript had a frustrating implementation of one specific effect. It reminds me of how ages ago certain people acquired an instinctual aversion to static typing attributable entirely to ancient versions of Java and its attendant verbosity and relative anemia.
No. Function colors (see criterion 4 in the original article) deal with the idea of "how much of a pain in the ass is it to deal with" which is not really a concept that is a PL theory concept.
The article only lists 3 criteria, not 4, but if you're referring to the final one:
"3. The change may require a change to all functions in the stack."
That's a potential feature of effect systems. An effect generally indicates a restriction of some kind, and depending on your desired semantics it could be totally counterproductive to paper over it with abstraction. And there's a variety of different knobs to twist that aren't necessarily the same semantics that any given implementation of async uses. To use Rust as an example, its `const `effect imposes restrictions down the callstack, and cannot be discharged (const functions can only call other const functions, period), while its `unsafe` effect imposes restrictions up the call stack and can be discharged (by a non-unsafe function that uses an `unsafe` block internally.
"More painful" is just being used to mean "explicit", and being explicit rather than implicit about intention is absolutely a PL theory concept, e.g. the whole difference between static and dynamic typing, or strong typing and weak typing.
>> JS... Because our strawman is a modern (shitty) language
This is an attitude I dislike from all these niche language communities. They give each other pats on the back for shitting on the most popular, versatile, compatible and adaptable programming language of all time.
I started using JS back in 2004 and, since then, I have watched it constantly being shat on by theoretically-minded folks. Before Node.js, people would keep pointing out "JS is a scripting language, not a programming language." Then, it became a programming language and went through the biggest adoption growth phase of any language in the history of mankind. I literally saw hardware manufacturers adopting JavaScript and even building custom JavaScript engines for their custom hardware, all while their academically-minded peers simultaneously shat on the language. They shat on it wave after wave, while it kept getting better and better and proved itself to be more adaptable than their own pet languages.
People are so hard-headed and biased against JS that the community literally had to invent a new label/wrapper; 'TypeScript' as a Trojan horse to get those stubborn folks to use it.
My view now is that JS critics are all talk. JS has been doing the walk; trampling all over their theories and abstractions and leading the way in many regards. Yes, it has bad parts, but all of these parts are avoidable; none of these parts are fundamental to the language. The matrix showing of how JS comparison operators down-cast everything to a string (leading to un-intuitive results) keeps being brought up but nobody asks the question "Why would anyone use a casting comparison operator to compare variables of two different types in the first place?". The fundamentals are excellent and proven as such.
That JS is the shittiest language is obvious to everyone who knows at least one other language. What it has is the ecosystem and in particular, it's the language that browsers use.
As someone who coded professionally in Python, C, C++, C# and Typescript, I disagree. JavaScript is my favourite language.
I just avoid the bad parts. Just like everyone else avoids the bad parts of their favorite programming languages. I compare languages based in the patterns and functionality that are actually used, not based on the sum of all its theoretical parts.
> They give each other pats on the back for shitting on the most popular, versatile, compatible and adaptable programming language of all time.
I don't see why C, C++ or Java wouldn't meet some of those criteria. Everything is written in those languages. As for most adaptable, I don't see how it's possible for JS to be more adaptable than Lisp. JS isn't homioconic. It doesn't have macros.
Under this framework, Rust's async fns would not be colored, since any function in the callstack could be blocking on an executor. if you consider "spawn point" a part of the callstack, many other languages are similar.
This doesn't feel like a useful distinction, but i don't think "function coloring" in general is a useful framework.
I think that's an interesting take, I have been chewing on the "so what" of function coloring myself, but haven't reached any conclusions.
Would be interested to hear why you don't find it a useful framework either.
I think this is a fine distinction to make, though it's worth noting that function arguments _can be_ colors as the author describes if the arguments have to come from a specific place, e.g. in languages where only `main` receives certain arguments relating to system functionality, or a capability framework where only the top-level function receives the capability token.
I don’t quite understand why the author says Context doesn’t count here. I’ve certainly experienced a top-down version where because a function starts accepting a Context, and therefore needs to be cancelable, lots of functions it calls now need to take a Context too, and check for cancellation. Perhaps the color here is cancelability, not the Context per se. (Which may be even worse, as a kind of “invisible color”.)
invisible color is, of course, called a "function infrared"
There's usually a way to run async on another thread (or this thread) and block on it, so async is not a color by your definition.
I think color should be defined in terms of what you're likely going to do. You can call asyncio.run or context.Background or unsafePerformIO but they're both hacks that you shouldn't do because they are not correct, so we ignore them for colour analysis.
Actually, the IO monad in haskell desugars to
RealWorld -> (a, RealWorld)
Where RealWorld is a "token" that is just meant to be threaded through. You cannot create one yourself, so you have to be given one.
Thus, that function argument colors IO. Every PL feature is just an extension to System F.
As I said to another comment, this isn't a place for architecture astronautics or squinting until one thing looks like another. It's all assembler under the hood in the end and that language not only doesn't have color, it doesn't have types in any practical sense of the term. At the level of Haskell itself, IO is a color and IO is not a function argument. The compiled code is all assembler. The Haskell may itself be an interpreter for a language that has no color. Above it there is no color, below it it may be implementing something without colors, but at the level of Haskell itself, IO is a color.
In Bluefin, IOE (which allows you to do I/O, i.e. what Haskell's IO also allows you to do) is an argument, and therefore not a color under your interpretation.
https://hackage.haskell.org/package/bluefin-0.0.7.0/docs/Blu...
it is all assembly under the hood but it's also all lambda calculus under the hood
My personal yardsticks:
How often does a conceptually incremental change correspond to a proportionally incremental code diff, vs. a surprisingly whole-world-upside-down rearchitect?
Do the code patterns naturally leave enough room for you to incrementally shift/transpose how things are organized?
When the answers to these are unfavorable, I often find what effects to "coloring" in the framework, library, or language, sometimes all at once.
Generally when talking about colored functions, I think of an example like "async function" in javascript, those are definitely different colored.
I don't know about what would constitute as colored function in go as I generally don't consider arguments to be function colors.
Maybe methods in go would be different colored functions when they are defined on different struct types.
I don't think there's a way to color functions in Go. That's not terribly special, several languages have no colors in them.
Pedantically, colored functions in Go might be ones that produce their output on a channel rather than returning a value. Then, you need select/for-range/<- to get their result, similar to “await”.
Practically, the runtime’s ability to suspend and move goroutines around means that the impact of that (niche/rarely appropriate as a default approach) kind of coloring is much less in Go than in other languages.
That has no propagation up the stack at all. Whether a child function does or does not use channels has no effect on the parent.
> What would it even look like for the top-most function to have to know “everything” that all the child functions end up ever needing? For every function parameter change in the entire program to somehow force a change all the way up the entire call stack? Clearly this does not happen. It so thoroughly does not happen that you may be having difficulty even imagining what it is I am talking about.
Really? In my experience, it certainly does happen and I'm generally happy (but not thrilled) when it does. It means the type system is doing its job, forcing me to provide all the dependencies that a function needs. If 38 layers deep something now needs a connection string that it didn't have before, that had better bubble up to the top-most function. Unless of course the top-most function has already provided it to an intermediate layer which can break the chain, but that is very different than the context.Background() example.
Yeah, in my experience it absolutely can happen, and the way it generally looks is that the dependencies get roughly grouped into larger structs which are then being passed down the call stack mostly as-is, until they hit a certain abstraction level/barrier where the parts of those structs start being passed down.
It's somewhat tedious, that's why modularity becomes even more important: with correctly chosen borders, the grouping of dependencies becomes very clean and self-evident, the whole data flow just flows. Chose them wrong, and you spend most of your time adding and removing those pesky extra arguments.
Yes, when considering software design it's great when you can "smuggle in" a new dependency by adding a new property or method on some already-established payload. I would describe this as "The change requires someone way up the stack to change, but all the intermediate layers require no source changes" which is notably absent from the author's list of "three basic cases that can emerge." I assume the author is well-aware of this technique but considers it equivalent (for the purpose of function coloring) to "The change may require a change to all functions in the stack" since technically the signature of all those intermediate layers did change.
Your configuration structs are not "everything". You probably have dozens of things in them. You have hundreds to thousands of different parameters to different functions. By the time your configuration structs have hundreds of things in them you have thousands to tens of thousands of parameters to functions. Only a fairly small fraction of things make it all the way up to main.
This is exactly what I was talking about with this being a "cognitively available" operation. You notice when you have to propagate something all the way up to the top precisely because it is not something you do very often and it stands out as an exception when you do. If you are literally propagating everything up to main, all the time, every time you change any function anywhere in your program, you are doing something very, very wrong.
You should be passing a connection or a connection pool, not a connection string.
I chose a connection string as an example of something that obviously cannot be originated in the middle of the call stack. The difference between passing a raw connection string or passing a connection factory is important for good software design, but I considered them equivalent in the context of this discussion of coloring.
Is `const`-ness (in the rust function keyword sense) a function color?
If you're using it diligently enough to where some functions can be statically proven "pure", then yes. You can't call an impure function from a pure one.
IMO it is a syntax thing. If you need to use a special keyword (eg. func vs async func) then it is colored. If it is a semantics thing (eg. parameters) is is not colored.
I'm not familiar with how Rust's async works. What if Rust used the function argument approach, how would that work? Would people pass around a reference to an async executor as arguments (or similar)?
Maybe.. something like how Zig works where it passes the I/O object around that encapsulates the required concurrency around the bottlenecks?
A subset of function arguments are function colors because async/await is isomorphic to requiring and passing a `callback` arg around. It was named 'callback hell` because there wasn't an escape hatch and it affected every function in the callstack.
I'd venture to guess that there's a more abstract way of formalizing this in the sense that monad instances are required to be threaded through the callstack in particular cases(would love to see this spelled out formally) where IO and Cont(?) have this requirement but State like the author points out, does not.
I think this is a case where you want to avoid architecture astronautics and deal with languages as they are presented. Yes, in the end it's all continuation passing, or depending on the cut of your jib, it's all just assembler in the end, but meanwhile, down in the trenches, there are real, practical differences in color that change how you program in those languages, and that's the topic color addresses.
Their point was that if language async is a color, so are async callbacks, which are arguments, therefore arguments can be colours.
I cited the case of an argument color already in the text for Zig... in fact, that's conditionally a color depending on what is passed, which is even more sophisticated then an argument merely being unconditionally a color.
What some people want to do is call every argument a "color", that is, I changed this function to have to take a Username and now it's Username-colored is the claim. I supposed I should have made that more clear. This provides a definition where that is not the case.
"Function colors" are just effect systems, which are not a new concept while also being extremely useful and super nifty. I hope someday we can stop wringing our hands in a panic over "colored functions" just because some ancient version of Javascript had a frustrating implementation of one specific effect. It reminds me of how ages ago certain people acquired an instinctual aversion to static typing attributable entirely to ancient versions of Java and its attendant verbosity and relative anemia.
No. Function colors (see criterion 4 in the original article) deal with the idea of "how much of a pain in the ass is it to deal with" which is not really a concept that is a PL theory concept.
The article only lists 3 criteria, not 4, but if you're referring to the final one:
"3. The change may require a change to all functions in the stack."
That's a potential feature of effect systems. An effect generally indicates a restriction of some kind, and depending on your desired semantics it could be totally counterproductive to paper over it with abstraction. And there's a variety of different knobs to twist that aren't necessarily the same semantics that any given implementation of async uses. To use Rust as an example, its `const `effect imposes restrictions down the callstack, and cannot be discharged (const functions can only call other const functions, period), while its `unsafe` effect imposes restrictions up the call stack and can be discharged (by a non-unsafe function that uses an `unsafe` block internally.
The *original* article: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
Criterion 4 is: Red functions are more painful to call.
They are correct - this isn't a PL theory concept.
"More painful" is just being used to mean "explicit", and being explicit rather than implicit about intention is absolutely a PL theory concept, e.g. the whole difference between static and dynamic typing, or strong typing and weak typing.
"Maybe it’s really verbose, or maybe you can’t do it inside certain kinds of statements. Maybe you can only call them on line numbers that are prime."
Nope. If it's explicit but maybe like a single keyword away that's not "painful", I would guess.
I don’t know, “the change is a major pain in the ass” might be a good addition as criterion 4.
>> JS... Because our strawman is a modern (shitty) language
This is an attitude I dislike from all these niche language communities. They give each other pats on the back for shitting on the most popular, versatile, compatible and adaptable programming language of all time.
I started using JS back in 2004 and, since then, I have watched it constantly being shat on by theoretically-minded folks. Before Node.js, people would keep pointing out "JS is a scripting language, not a programming language." Then, it became a programming language and went through the biggest adoption growth phase of any language in the history of mankind. I literally saw hardware manufacturers adopting JavaScript and even building custom JavaScript engines for their custom hardware, all while their academically-minded peers simultaneously shat on the language. They shat on it wave after wave, while it kept getting better and better and proved itself to be more adaptable than their own pet languages.
People are so hard-headed and biased against JS that the community literally had to invent a new label/wrapper; 'TypeScript' as a Trojan horse to get those stubborn folks to use it.
My view now is that JS critics are all talk. JS has been doing the walk; trampling all over their theories and abstractions and leading the way in many regards. Yes, it has bad parts, but all of these parts are avoidable; none of these parts are fundamental to the language. The matrix showing of how JS comparison operators down-cast everything to a string (leading to un-intuitive results) keeps being brought up but nobody asks the question "Why would anyone use a casting comparison operator to compare variables of two different types in the first place?". The fundamentals are excellent and proven as such.
That JS is the shittiest language is obvious to everyone who knows at least one other language. What it has is the ecosystem and in particular, it's the language that browsers use.
As someone who coded professionally in Python, C, C++, C# and Typescript, I disagree. JavaScript is my favourite language.
I just avoid the bad parts. Just like everyone else avoids the bad parts of their favorite programming languages. I compare languages based in the patterns and functionality that are actually used, not based on the sum of all its theoretical parts.
> but nobody asks the question "Why would anyone use a casting comparison operator to compare variables of two different types in the first place?"
Well if you think typescript is a trojan horse label, then you make that comparison because your types are obnoxiously weak.
The fundamentals are not excellent.
> They give each other pats on the back for shitting on the most popular, versatile, compatible and adaptable programming language of all time.
I don't see why C, C++ or Java wouldn't meet some of those criteria. Everything is written in those languages. As for most adaptable, I don't see how it's possible for JS to be more adaptable than Lisp. JS isn't homioconic. It doesn't have macros.