Imagine we could understand the brain – fully. Understanding all the intricacies of how the brain works could allow us to cure many diseases like Alzheimer’s or Parkinson’s. We might also be able to build machines that mimic the brain or even excel at what it is doing so that old science fiction stories on real artificial intelligence finally come true.
So how do we get there? Well, if we want to understand something, we might start by looking at it. But of course it is difficult to observe what is going on by just looking at the brain. So when I say looking, I really mean measure: we want to measure brain activity.
People have been measuring brain activity for a very long time using different techniques. But none of these techniques is perfect. Some techniques allow us to tell when a particular part of the brain is active but not exactly when, whereas other techniques allow us to tell when there is activity but not exactly where.
So the techniques are really complementary. Wouldn’t it be great if we could combine totally different measurements to get the big picture? To do so, we need to understand how exactly different measurements are related. Sounds like a difficult problem.
Let’s take inspiration from a domain that has been looking at relations for a very long time – and also has been extremely well funded to develop methods for this: Finance itself. Understanding the relations between stock markets is key to making money. In finance, a common tool to understand relations is the copula. You might have heard of copulas before because they have a very bad reputation. This is because a copula, in particular the Gaussian copula, is blamed for the financial crisis of 2008! This doesn’t sound like a good tool to help understanding the brain. But maybe we should take a close look at this thing.
Probability distributions and pizza
To understand what happened, we need to understand what a Gaussian copula is. As the name implies, the Gaussian copula is related to the Gaussian distribution which is prevalent in, well, everywhere really. What exactly is the Gaussian distribution and why is it that the Gaussian distribution is so common?
First, let’s talk about a distribution in general. A distribution describes for something random how likely different outcomes are to occur. For instance, a coin toss can show heads or tails. For a normal coin, you have a 50-50 chance of getting heads vs tails. On average, on 50% of your tosses you get heads and on 50% you get tails. The 50-50 chance is the distribution of the coin toss.
So suppose you and your friend Mario (no, your Italian friend, not the video game character) disagree – again – on whether to go eat pizza or something else. Mario wants to eat pizza all the time whereas you prefer other food at least from time to time because you get stomach problems when you eat too much pizza. So to decide, Mario and you can toss a coin. Mario likes the heads side of the coin, so you go for the tails side. If the coin shows heads, you eat pizza on that day. If it shows tails, you eat something else. This is fair because of the distribution of the coin: if you repeat this on many days, then in the long run you should eat pizza only half of the time.
Suppose that this has become a tradition for the two of you. You always bet on tails whereas Mario bets on heads. But Mario continues to be disgusted by other food, so he devises a devilish plan: he forges another coin that has heads on both sides! Whenever Mario tosses that coin, it will show heads with 100% certainty, so the chance of heads vs tails is 100-0 for that coin: 100% that the coin shows heads and 0% that it shows tails. The 100-0 chance is another distribution besides the 50-50 distribution.
Mario starts using this coin every day. Naturally it shows heads every day and you eat only pizza for 10 days. On the 10th day you start not feeling so well and it strikes you that you have eaten pizza all the time! Slowly you suspect that something is not right. So on the 11th day, Mario again tosses his fake coin, but this time you catch his coin in midair. You investigate it and indeed, you find that it has heads on both sides. You get very angry and punch Mario in the face. So this time, the outcome of the coin toss was neither heads, nor tails. It was punch!
Catching a coin in midair is tricky, so there was a chance of missing the coin and Mario winning again. Or, maybe, after so much pizza, you could have decided to look for Mario’s coin beforehand and replace it with a fake coin of your own that shows tails on both sides. So perhaps the odds were 30-10-60 for tails-heads-punch: 30% that you don’t replace the coin and don’t catch Mario’s coin (so his coin showing heads again), 10% that you replace the coin with your own fake coin and the coin showing tails, and 60% that you catch the coin and punch Mario afterwards. 30-10-60 is a distribution with three outcomes.
You see that random events like this can have more than two outcomes even if you are not aware of them. You thought the toss was fair, but it was not. Mario also had wrong expectations, because he didn’t expect you to catch his coin and punch him, he being your friend and all. So at certain points, both Mario and you acted on false premises. Let’s take this sad story as a lesson that you need to get the distribution right or else it could be devastating for your stomach and friendship.
The Gaussian distribution
Now, let’s get back to the Gaussian distribution which is a particular kind of distribution. The special thing about the Gaussian is that you get it whenever your random event can be described as a series of many independent similar events.
In our punch example, we had a series of events that we described as one event: your decision whether to look for the fake coin and to replace it with your own fake coin, the toss of the fake coin and the catch in midair. You could describe this as a series of at least three events that in conjunction determine the final outcome of heads vs tails vs punch.
In that example, the events are all pretty different, but suppose that you have a series of identical events. For instance, you could have a ball falling downwards a series of junctions and finally end up in one of a couple of bins. Like the fair coin tosses, we can describe the fall as a series of left vs right falls at each junction or we can describe just the final outcome as the bin in which the ball ends up. We will do the latter and look at the distribution of balls in the bins.
So let’s drop many balls and look at the ratios of balls in the bins. If there are a great number of bins and correspondingly a great number of junctions, then one can draw a smooth curve over the ratios of balls in the bins. For the center bins, there are many combinations of junctions that lead to those bins. For instance, left-right-left-right-left… would lead to the center bin, but also right-left-right-left… and also left-left-right-right-left-left… But to get to the outer bins, a ball needs to fall to one particular side at every junction. This is very unlikely, so this doesn’t happen very often.
For this reason, the curve looks like a bell: balls very often end up in one of the center bins whereas balls end up seldom in the outer bins. The Gaussian distribution is exactly this bell curve, named after the mathematician Carl Friedrich Gauss who discovered it in 1809.
The case where there is series of many small events that do not really have anything to do which each other occurs everywhere. And the amazing thing is that you do not only end up with the Gaussian when you have a series of coin-toss-like events. All you need is a long series of independent events, where each of these events has the same distribution. And even if the distributions are not perfectly identical or if the events are not completely independent, even then will the distribution of the final outcome be approximately Gaussian. This is why the Gaussian distribution is so important and occurs everywhere. Just think of your commute in the morning and how there is probably a series of similar independent events leading to your final arrival time. In general, if you know that there are many possible outcomes and you don’t have any clue about the distribution of those outcomes then the Gaussian is a pretty good first guess!
Pyramids, chess and relations
But what the heck has the Gaussian distribution got to do with relations like the stock market relations of brain measurement relations? Imagine that you not only have a triangle of junctions like in the previous example but a pyramid. You drop the ball on the peak of the pyramid and now, at each junction, there are four possible outcomes for that junction: the ball can fall forward-left, forward-right, backward-left or backward-right. So now there is an additional direction besides left-right that is forward-backward.
The bins at the bottom are now boxes arranged like on a chessboard. Chess players label one direction with letters (a, b, c, …) and one direction with numbers (1, 2, 3, …) and then they say things like “move g1 to f3”, meaning to move the piece that is standing on the field labeled with both ‘g’ and ‘1’ to the field labeled with both ‘f’ and ‘3’. In the same way, we can label the outcome of our pyramid ball fall. If the ball ends up in the box second from left and first from bottom, we can say the outcome is g1. So let’s call the first direction the “letter direction” and the second direction the “number direction”. At each junction, left-right moves the ball along the letter direction and forward-backward moves the ball along the number direction.
Like before, most balls will end up in the boxes right below the peak of our pyramid and so the shape looks like a real bell, not just like a cut through a bell. But what happens if we grind the forward-right and backward-left corners of each junction so that the balls are more likely to fall forward-right than to fall forward-left and also are more likely to fall backward-left than to fall backward-right? Then our bell will not be round anymore. Instead, if we look from above, it will look like an egg, because the letter direction and the number direction are not independent anymore. This is what we call correlation between the letters and numbers. It is the relation between two things (here letters and numbers) that we were after! But it is one particular relation. For instance, we could make the relation stronger by grinding even more from all junctions so that the balls would be even more likely to fall forward-right or backward-left. In that case the shape of our bell would become even more elongated.
The copula as a junction map
You could also think of another relation where you cut only some of the junctions but not others. For instance, if you cut only the junctions in the backward-left part of the pyramid and then again look from the top at your boxes, then you will see a shape that looks like a pear. For this shape, if you first let a ball fall into a box, then look at the letter of your box and then find that it’s ‘a’ or ‘b’ then you can be pretty sure that the number of the box will be ‘1’ or ‘2’. If, on the other hand, you find that the letter of the box is ‘j’ or ‘k’ then you cannot be so certain and your number could be any of perhaps ‘7’ to ‘12’. So depending on the shape of the relation, your certainty about a box number given its letter changes a lot.
Now, a copula is like a map for cutting the junctions. The Gaussian copula cuts all junctions in the same way whereas other copulas cut only some of the junctions. Figuring out which junctions exactly to cut is more cumbersome than just cutting them all in the same way.
Perils of using copulas
With this knowledge we can understand what happened during the financial crisis of 2008. Traders used Gaussian copulas when the real relation looked completely different. The overall cut that was used for all junctions was roughly right, so it worked fine for a while. But when extreme outcomes occurred (like the ‘1’ or ‘2’ in our example) the relation and the risks that the traders had calculated were completely wrong.
All this doesn’t mean that copulas are a bad idea. Using the Gaussian copula everywhere is a bad idea. But if you use the right copula for the right relation then you are fine. And in my project, I am working on getting the right copulas to describe the relation between measurements of brain activity. I should better get it right.
Using copulas in Neuroscience:
A. Onken and S. Panzeri (2016). Mixed vine copulas as joint models of spike counts and local field potentials. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon and R. Garnett, editors, Advances in Neural Information Processing Systems 29 (NIPS 2016), pages 1325-1333. http://papers.nips.cc/paper/6069-mixed-vine-copulas-as-joint-models-of-spike-counts-and-local-field-potentials.pdf
Python package for copula modelling: https://github.com/asnelt/mixedvines/.
Funded by a Marie Sklodowska-Curie Action: This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 659227.