When I read the first meaty chapter about graphs and commutativity I initially thought he just spends too long explaining simple concepts.
But then ai realized I would always forget the names for all the mathy c' words - commutativity commutativity, qssociativity... and for the first time I could actually remember commutativity and what it means, just because he tied it into a graphical representation (which actually made me laugh out loud because, initially, I thought it was a joke). So the concept of "x + y = y + x" always made sense to me but never really stuck like the graphical representation, which also made me remember its name for the first time.
Generalized Transformers from Applicative Functors
>Transformers are a machine-learning model at the foundation of many state-of-the-art systems in modern AI, originally proposed in [arXiv:1706.03762]. In this post, we are going to build a generalization of Transformer models that can operate on (almost) arbitrary structures such as functions, graphs, probability distributions, not just matrices and vectors.
>[...]
>This work is part of a series of similar ideas exploring machine learning through abstract diagrammatical means.
It's interesting how some of these diagrams are almost equivalent in the context of encoding computation in interaction nets using symmetric interaction combinators [1].
From the perspective of the lambda calculus for example, the duplication of the addition node in "When Adding met Copying" [2] mirrors exactly the iterative duplication of lambda terms - ie. something like (λx.x x) M!
Years ago when I was reading this (just a couple of chapters, not all of it), it opened my eyes to the power of diagrammatic representation in formal reasoning unlike anything before. I never did anything useful with string diagrams, but it was so fun to see what is possible with this system!
When I read the first meaty chapter about graphs and commutativity I initially thought he just spends too long explaining simple concepts.
But then ai realized I would always forget the names for all the mathy c' words - commutativity commutativity, qssociativity... and for the first time I could actually remember commutativity and what it means, just because he tied it into a graphical representation (which actually made me laugh out loud because, initially, I thought it was a joke). So the concept of "x + y = y + x" always made sense to me but never really stuck like the graphical representation, which also made me remember its name for the first time.
I am sold.
Which chapter is that? It's not in the ToC
Generalized Transformers from Applicative Functors
>Transformers are a machine-learning model at the foundation of many state-of-the-art systems in modern AI, originally proposed in [arXiv:1706.03762]. In this post, we are going to build a generalization of Transformer models that can operate on (almost) arbitrary structures such as functions, graphs, probability distributions, not just matrices and vectors.
>[...]
>This work is part of a series of similar ideas exploring machine learning through abstract diagrammatical means.
https://cybercat.institute/2025/02/12/transformers-applicati...
I really enjoyed that when it was coming out, and used to follow it with some students. It's a shame it seems to have been abandoned.
Who wrote that? Do you know?
pawel ... ?
Pawel Sobocinski, in collaboration with Filippo Bonchi and Fabio Zanasi
https://graphicallinearalgebra.net/about/
It's interesting how some of these diagrams are almost equivalent in the context of encoding computation in interaction nets using symmetric interaction combinators [1].
From the perspective of the lambda calculus for example, the duplication of the addition node in "When Adding met Copying" [2] mirrors exactly the iterative duplication of lambda terms - ie. something like (λx.x x) M!
[1]: https://ezb.io/thoughts/interaction_nets/lambda_calculus/202...
[2]: https://graphicallinearalgebra.net/2015/05/12/when-adding-me...
> If the internet has taught us anything, it’s that humans + anonymity = unpleasantness.
Aka one of my favorite axioms: https://www.penny-arcade.com/comic/2004/03/19/green-blackboa...
Years ago when I was reading this (just a couple of chapters, not all of it), it opened my eyes to the power of diagrammatic representation in formal reasoning unlike anything before. I never did anything useful with string diagrams, but it was so fun to see what is possible with this system!
Appreciate the Claude Makelele praise