A good way to conceptualize what’s going on is not the idea that balls become "spiky" in high dimensions – like the article says, balls are always perfectly symmetrical by definition. But it’s the box becoming spiky, "caltrop-shaped", its vertices reaching farther and farther out from the origin as the square root of dimension, while the centers of its sides remain at exactly +-1. And the 2^N surrounding balls are also getting farther from the origin, while their radius remains 1/2. Now it should be quite easy to imagine how the center ball gets more and more room until it grows out of the spiky box.
But there is another way to think of the high-dimensional balls where "spikey" is the right visualisation.
Consider the volume of a cap. Take a plane that is 90% of the distance from the centre to the edge, and look at what percentage of the volume is "outside" that plane. When the dimension is high, that volume is negligible.
And when the dimension is really high you can get quite close to the centre, and still the volume you cut off is very small. In our 3D world the closest thing that has this property is a spike. You can cut off quite close to the centre, and the volume excised is small.
The sense in which a high-dimensional ball is not a spikey thing is in the symmetry, and the smoothness.
So when you want to develop an intuition for a high dimensional ball you need to think of it as simultaneously symmetrical, smooth, and spikey.
Then think of another five impossible things, and you can have breakfast.
I think our geometric intuition could never be good for a high dimensional object. Consider a sphere in 3D. It's represented by the points which satisfy x^2 + y^2 + z^2 = 1. Because there are only three coordinates, knowing the value of one of them greatly reduces the possibilities for the other two. For example, if I know z is close to 1, then my point is close to the north pole (x and y are both close to zero).
However, if I have the n-sphere x_1^2 + x_2^2 + ... + x_n^2 = 1, knowing the value of x_1 gives me very little information about all the other coordinates. And humans' interaction with the geometry is reality is somewhat limited to manipulating one coordinate at a time, i.e., our intuition is for built on things like moving our body linearly through space, not dilating the volume or surface area of our bodies.
That's a good point. High dimensional objects can obtain very unintuitive properties, like you describe.
This to me feels similar in many ways to how a corner in a high dimensional n-cube, although 90 degrees, no matter how you measure it, seems extremely spiky. As the shape does not increase in width, but the corners extend arbitrarily far away from the center. A property reserved for spiky things in 3D.
Exactly: a corner of a square covers 1/4 of that part of the plane. A corner of a cube covers 1/8, a corner of a hypercube in dimension n covers just 1/(2^n) of the space. But each side/face/hyperface divides the plane/space/n-dim space just in half.
Why did I imagine that this would be about two shapes that are merely topologically n-balls, each having part of their boundary be incident with one of the two hemi(n-1)-spheres of the boundary of an n-ball (and otherwise not intersecting it)? (So like, in 3D, if you took some ball and two lumps of clay of different colors, and smooshed each piece of clay over half of the surface of the ball, with each of the two lumps of clay remaining topologically a 3-ball.)
I don’t know that there would even be anything interesting to say about that.
I am struggling to juggle the balls in my mind. Are there any stepping-stone visual pieces like this to hopefully get me there? Very neat write-up, but I can't wait to share the realized absurdity of the red ball's green box eclipsing in our 3D intersection of the fully diagonalized 10D construct
The hypercube is the strange thing, not the red sphere. Placing the blue spheres tangent to the hypercube is an artificial construct which only “bounds” the red sphere in small dimensions. Our intuition is wrong because we think of the problem the wrong way (“the red sphere must be bounded by the box”, but there is no geometrical argument for that in n dimensions).
> There is a geometric thought experiment that is often used to demonstrate the counterintuitive shape of high-dimensional phenomena. We start with a 4×4 square. There are four blue circles, with a radius of one, packed into the box. One in each corner. At the center of the box is a red circle. The red circle is as large as it can be, without overlapping the blue circles. When extending the construct to 3D, many things happen. All the circles are now spheres, the red sphere is larger while the blue spheres aren’t, and there are eight spheres while there were only four circles.
> There are more than one way to extend the construct into higher dimensions, so to make it more rigorous, we will define it like so: An n-dimensional version of the construct consists of an n-cube with a side length of 4. On the midpoint between each vertex and the center of the n-cube, there is an n-ball with a radius of one. In the center of the n-cube there is the largest n-ball that does not intersect any other n-ball.
> At what dimension would the red ball extend outside the box?
Response: "[...] Conclusion: The red ball extends outside the cube when n≥10n≥10."
It calculated it with a step-by-step explanation. This is the first time I'm actually pretty stunned. It analysed the problem, created an outline. Pretty crazy.
A good way to conceptualize what’s going on is not the idea that balls become "spiky" in high dimensions – like the article says, balls are always perfectly symmetrical by definition. But it’s the box becoming spiky, "caltrop-shaped", its vertices reaching farther and farther out from the origin as the square root of dimension, while the centers of its sides remain at exactly +-1. And the 2^N surrounding balls are also getting farther from the origin, while their radius remains 1/2. Now it should be quite easy to imagine how the center ball gets more and more room until it grows out of the spiky box.
But there is another way to think of the high-dimensional balls where "spikey" is the right visualisation.
Consider the volume of a cap. Take a plane that is 90% of the distance from the centre to the edge, and look at what percentage of the volume is "outside" that plane. When the dimension is high, that volume is negligible.
And when the dimension is really high you can get quite close to the centre, and still the volume you cut off is very small. In our 3D world the closest thing that has this property is a spike. You can cut off quite close to the centre, and the volume excised is small.
The sense in which a high-dimensional ball is not a spikey thing is in the symmetry, and the smoothness.
So when you want to develop an intuition for a high dimensional ball you need to think of it as simultaneously symmetrical, smooth, and spikey.
Then think of another five impossible things, and you can have breakfast.
I think our geometric intuition could never be good for a high dimensional object. Consider a sphere in 3D. It's represented by the points which satisfy x^2 + y^2 + z^2 = 1. Because there are only three coordinates, knowing the value of one of them greatly reduces the possibilities for the other two. For example, if I know z is close to 1, then my point is close to the north pole (x and y are both close to zero).
However, if I have the n-sphere x_1^2 + x_2^2 + ... + x_n^2 = 1, knowing the value of x_1 gives me very little information about all the other coordinates. And humans' interaction with the geometry is reality is somewhat limited to manipulating one coordinate at a time, i.e., our intuition is for built on things like moving our body linearly through space, not dilating the volume or surface area of our bodies.
That's a good point. High dimensional objects can obtain very unintuitive properties, like you describe.
This to me feels similar in many ways to how a corner in a high dimensional n-cube, although 90 degrees, no matter how you measure it, seems extremely spiky. As the shape does not increase in width, but the corners extend arbitrarily far away from the center. A property reserved for spiky things in 3D.
Exactly: a corner of a square covers 1/4 of that part of the plane. A corner of a cube covers 1/8, a corner of a hypercube in dimension n covers just 1/(2^n) of the space. But each side/face/hyperface divides the plane/space/n-dim space just in half.
Ok, now i started to understand something. Thank you.
Very good to see this as the top-voted comment. I completely agree that this seems like a more natural explanation of what is going on.
even then the edges do not suddenly curve. its just all round a bad analogy.
This is a really good demonstration of the curse of dimensionality[0]
[0]: https://en.m.wikipedia.org/wiki/Curse_of_dimensionality
Why did I imagine that this would be about two shapes that are merely topologically n-balls, each having part of their boundary be incident with one of the two hemi(n-1)-spheres of the boundary of an n-ball (and otherwise not intersecting it)? (So like, in 3D, if you took some ball and two lumps of clay of different colors, and smooshed each piece of clay over half of the surface of the ball, with each of the two lumps of clay remaining topologically a 3-ball.)
I don’t know that there would even be anything interesting to say about that.
I can't tell you why you imagined that, but that's pretty funny nevertheless.
For other HN discussions of this phenomenon you can see some previous submissions of another article on it.
That article doesn't have the nice animations, but it is from 14 years ago ...
https://news.ycombinator.com/item?id=12998899
https://news.ycombinator.com/item?id=3995615
And from October 29, 2010:
https://news.ycombinator.com/item?id=1846682
Can I just say how my mind is utterly blown by the animations
Thank you <3 The trigonometry involved was pretty intense at times.
I am struggling to juggle the balls in my mind. Are there any stepping-stone visual pieces like this to hopefully get me there? Very neat write-up, but I can't wait to share the realized absurdity of the red ball's green box eclipsing in our 3D intersection of the fully diagonalized 10D construct
The hypercube is the strange thing, not the red sphere. Placing the blue spheres tangent to the hypercube is an artificial construct which only “bounds” the red sphere in small dimensions. Our intuition is wrong because we think of the problem the wrong way (“the red sphere must be bounded by the box”, but there is no geometrical argument for that in n dimensions).
Impressive, helpful, and now time to rebuild my own embeddings so I can grasp that red n-ball with my new n-D hands.
Numberphile did a video on this a while back. https://youtu.be/mceaM2_zQd8?si=0xcOAoF-Bn1Z8nrO
Anyone else click just to slide some animations?
This guy gets it!
wow discovering Hamming’s lecture was enough for me! so good
Both ChatGpt 4.o and Claude failed to answer
“…At what dimension would the red ball extend outside the box?”
If anyone has o1-preview it’d be interesting to hear how well it does on this.
This was the prompt I gave o1-preview:
> There is a geometric thought experiment that is often used to demonstrate the counterintuitive shape of high-dimensional phenomena. We start with a 4×4 square. There are four blue circles, with a radius of one, packed into the box. One in each corner. At the center of the box is a red circle. The red circle is as large as it can be, without overlapping the blue circles. When extending the construct to 3D, many things happen. All the circles are now spheres, the red sphere is larger while the blue spheres aren’t, and there are eight spheres while there were only four circles.
> There are more than one way to extend the construct into higher dimensions, so to make it more rigorous, we will define it like so: An n-dimensional version of the construct consists of an n-cube with a side length of 4. On the midpoint between each vertex and the center of the n-cube, there is an n-ball with a radius of one. In the center of the n-cube there is the largest n-ball that does not intersect any other n-ball.
> At what dimension would the red ball extend outside the box?
Response: "[...] Conclusion: The red ball extends outside the cube when n≥10n≥10."
It calculated it with a step-by-step explanation. This is the first time I'm actually pretty stunned. It analysed the problem, created an outline. Pretty crazy.
I'd wager that it's in the training data.
I… can’t.
Matlock, is that you?
I am not Matlock, who is that?
He probably meant MacGyver.
https://en.m.wikipedia.org/wiki/MacGyver
No, he really did mean Matlock. Grey hair guy in a white suit lecturing...Fictional Lawyer. https://www.alamy.com/stock-photo/matlock-tv-andy-griffith.h...
Reboot on TV this year.
I know who Matlock is. But then it makes no sense at all.