The White Swan

Who's right, anyway?

2009-07-11T08:53:00.000-07:00

"Sheller had been pronouncing the stock market to be overpriced for a long time. George Will indicated to Shiller that had people listened to him in the past they would have lost money, as the market has more than doubled since he started pronouncing it overvalued. To such a journalistic and well sounding (but senseless) argument, Shiller was unable to respond expect to explain that the fact that he was wrong in one single market call should not carry undue significance."

The above is a quote from Taleb's book, "Fooled by Randomness". The case of Shiller is part of a pattern: an economist states that the stock market is over-priced. No-one believes them, they are ridiculed, and even lose their jobs, yet after some time, they are vindicated. Then, they are praised for having predicted a bubble, for their remarkable insight etc. And Taleb in this case defends Robert Shiller against George Will's criticism.

Now, I must confess that I have not researched the arguments of Shiller, nor any of the cases of economists being vindicated in the long run. Still, I cannot accept Taleb's criticism against Will. When an influential economist makes a prediction, they ought to provide an acceptable time limit for it to come true. To me, it does not seems a vindication if the stock market crashes five years after one has proclaimed it to be a bubble. Especially not if it doubles meanwhile!

I do not accept this for two reasons:

a) When an influential economist speaks, people listen, for better or for worse. People who did listen to Shiller when he first said that the market was forming a bubble could have been ruined financially. To them, it doesn't matter if Shiller is eventually vindicated.

b) The fact that the market declined does not necessarily mean that the prediction was correct. In fact, if the market crashes years after it was claimed to be overpriced, it is very likely that the prediction was false. Over a long time span, you can expect the market to go through cycles. Obviously, the markets are bound to crash at some point. And it may crush for reasons different to the ones that caused one to describe the market as overvalued.

Best,

Predicting bubbles?

2009-06-20T12:54:00.000-07:00

I just read an interesting article on the New Scientist, about how science can fix the markets. One scientist from the Federal Institute of Technology (ETH) of Zurich claimed that a method that could help in forecasting market "bubbles", such as the recent housing bubble, is to plot the price of a commodity/good/stock against time. If the increase in price belongs to a class higher than exponential, then there may trouble ahead. A simple way to check this is to plot the logarithm of a price against time, and if the resulting graph is not a straight line, a bubble may occur in the near future.

Now, this method does not hold for all sorts of goods or stocks, obviously. There may be cases of justified rapid expansion, potentially due to technological advances etc. Still, such higher-than-exponential growth ought to be examined if spotted.

Another interesting idea (which I think I will examine in detail in the near future) is examining the relation between the market performance of a company's share and the times the company is being searched in google. I assume that if a company is doing exceptionally well (or exceptionally badly), it will be searched more frequently. Indeed, the number of times "Lehman brothers" was searched reached a peak on September 15th, when it filed for bankruptcy. In fact, it was searched roughly 30 times more than it was at any other point in time.

This method could be perfected by taking a combination of a company's name and several key words, such as possibly "buy", "sell", "bankruptcy", "profits rise" etc. As I've said, I will look into it at some point!

Best,

"Monkeys on typewriters"

2009-06-14T06:45:00.000-07:00

Quite a few years back, when I first came across the term "law of probability", I asked my grandfather what it means. The response: "the law of probability states that if you give typewriters to an infinite number of monkeys, one of them will conduct the 'Encyclopedia Britannica'".

Taleb presents the exact same scenario (substituting Britannica for the Iliad), but not in order to explain the properties of probability, but to make a point about expert traders: he claims that the fact that one trader has produced profit for say 5 years in a row is more than coincidental: it would be unexpected if there were not a few traders performing that well. Why? Consider this: every year, there are hundred, possibly thousands of graduates joining financial institutions. Even if they place calls at random, say with a 50% probability of making a profit each year, you can expect half of them to have made a profit after the first year; a quarter of the original sample in the second year; and so on and so forth. Moreover, even if the population consists of bad traders, that is traders who have lower than a 50% chance of turning a profit, there will still be several of them with 5 successful years in a row.

In other words, the track record of a particular trader is unimportant; it is in all likelihood attributed to chance, simply because there is a very large sample of traders. If there were only 4 traders in the world, things would be different.

However, I slightly disagree with Taleb on this point; although I agree that in any given population, there will be many 'chance' winners if the population is sufficiently large, I believe that a problem with this (simplified) proposition lies in the fact that traders do not trade independently. As Taleb himself demonstrates, traders in their majority tend to exhibit a herd-ish behaviour. Thus, you can expect many of them doing well at the same time (as was the case before the crisis), or most of them doing badly together (as is the case after the crisis). Of course, there will be different magnitudes of doing well; some will be making thousands, others millions - and this is where Taleb's argument applies: the ones making millions could owe their success to chance, rather than skills.

If however you can identify a trader who has not exhibited herd mentality, and who has turned a profit when the majority of the population has not, then it is more probable that this trader owes his success to his abilities rather than luck.

Randomness in computer science

2009-06-12T14:27:00.000-07:00

Although on first sight this may seem boring to non-scientists, this post does have some interest - read on!

There is one field in computer science that truly benefits from black swans: genetic algorithms. Considered a subset of the field often referred to as "non-standard computation", these algorithms borrow concepts from biology to construct useful artifacts. These artifacts may be programs, (models of) objects such as lego designs, chips etc, or (more creatively) songs and even poems.

In genetic algorithms, we begin by defining a fitness function: this determines how good a produced artifact is. In case of a program, the fitness function may denote its complexity, its execution time, the amount of memory it consumes, or simply whether it terminates. When discussing lego bridges, we may want to measure their vigor; in the case of music, we can define elaborate functions, such as detection of patterns common in the works of great composers etc.

Then, we begin by generating a large population of random artifacts: in the case of music, for instance, we may simply generate random sequences of notes. We then evolve these artifacts by mutating them and "breeding" them. There are various ways to do this, such as switching random notes, swapping parts between two artifacts and so on and so forth. Each time, we eliminate the weakest scoring artifacts, and repeat the process. We terminate after a given number of iterations, or after an artifact of satisfactory score has been produced.

This method clearly depends on luck: quite often, genetic algorithms come up with truly novel solutions (in an experiment, a computer chip was produced which behaved in very peculiar ways: if some seemingly useless components were removed, the circuit would not work. It was still very efficient though). However, such genetic algorithms are very successful - and can benefit from black swans!

Best,

Black Swan and the music industry

2009-06-12T01:15:00.000-07:00

One of the sectors of the economy that have suffered the most from the appearance of a black swan, more than the financial sector, is the music industry. The name of the black swan to hit it is peer-to-peer networks.

One can argue that peer-to-peer networks were not really unexpected; it is only logical to expect that people would start trading their music files on-line. Still, their rise to prominence and their evolution took the music industry by surprise, and had a tremendous effect.

Beginning with Napster, continuing with Kazaa, I-mesh and lime-wire, before evolving to bit-torrents and rapidshare, the exchange of copy-righted material, an act labelled 'piracy', has had strong ramifications on record companies. Indeed, the record industry claims that piracy has caused a 23% decline in CD sales between 2000 and 2006, whilst the Times report that a teenager's iPod contains roughly 800 illegally downloaded songs.

This is not one of the Black swans that can be easily turned to one's advantage. However, the record companies are spending large amounts of money on new encryption mechanisms, which will be broken in a few months after they are released; they engage in lengthy and expensive law-suits, when every day more and more people trade their music on-line. The French government is trying to pass new laws, blocking the internet connections of illegal downloaders. This piece of legislation is possibly one of the most useless ones to ever been passed: people can easily disguise their IP address or use public computers to avoid been tracked down. Finally, even if the government manages to overcome these shortcomings, trust hackers to come up with new, undetectable ways to share files.

It can be seen that this fight against file-sharing is not only futile, but expensive and time-consuming. The record companies ought to try to adapt to the new status quo rather than try to shut down all file-sharing networks. Indeed, an opportunity lies for them in the new state of affairs : lately, more and more artists publish their music for free on the internet (Kate Perry, Lilly Allen and the Arctic Monkeys to name a few).

Instead of fighting file-sharers, record companies could simply set up their own website, storing all their artists' music online, free to download. There is large revenue to be made through online advertising. And these websites would be an ideal platform for all sort of advertisements. Who would choose to download music tracks of dubious quality through a file-sharing program, instead of visiting the official website and downloading a top-quality album for free?

Record companies would have nothing to lose, not any more that they already are. If people can download free music, they will; no matter what legislation passes, no matter how many downloaders get sued, no matter how many networks get shut down, more will spring up. Instead of playing a losing game, the music industry could, and should, turn the situation to their advantage.

Best,

References:

1. (2008) Average teenager's iPod has 800 illegal music tracks. The Times [http://technology.timesonline.co.uk/tol/news/tech_and_web/personal_tech/article4144585.ece]

2. (2007) Illegal music downloading still on rise. Afterdawn.com [http://www.afterdawn.com/news/archive/8623.cfm]

Know yourself

2009-06-10T01:52:00.000-07:00

In the very last pages on the Black Swan, Taleb makes one of his most important points, in my opinion. He narrates how once a friend of his prevented him from running to catch a train. This, in Taleb's opinion, is snubing one's destiny, it's a form of elegance. As he puts it, missing a train is only painful if you run for it. He then extrapolates from this, and asserts that you are above the rat race if you do so by choice. He concludes that it is difficult being the loser in a game, if you set up the rules.

Unfortunately, I believe Dr Taleb misses a crucial point in this argument. In order to be able to set up the rules, in order to be able to snub destiny, and all other factors that may seem to control your life, you must first start by understanding who you are, knowing yourself (γνώθι σεαυτόν, as the ancient Greeks would have it). Otherwise, no matter how hard you try to be aggressive, no matter how hard you try not to follow the rules others have set up for you, you are bound to fail.

Now, most people believe they do know themselves perfectly, that they never do things against their will, that they do not act in a way which is unlike their true nature. Unfortunately, this is not so. Most people fool themselves, try to present an image unlike their inner self to the outside world, and thus are unable to set up their own rules. They try to present an image that they believe the world will appreciate.

This theory is justified by the existence of trends: unless I am wrong, why do fashion trends come and go? Is it really this likely that suddenly everyone grasps the vanity and hardship of life at the same time, and this is why we have emos? Or is it possible that everyone realises at the same time that they are unconventional, thus giving rise to the punk movement?

I believe that in both cases people try to alter themselves in some way. I refuse to believe that most emos are deeply sad persons; I refuse to believe that most punks are really anti-establishment.

This aside however, there are more elegant forms of self-fooling that following a trend. You will see people who are dishonest with themselves, in many subtle ways. No matter how subtle though, such people will not manage to snub their destiny.

This is not to say that I personally am above all this. In many ways I know I am bound by rules set up by other people - difference being that I realise it and accept it, rather than trying to deny it. And again, most people will claim the same: "I know I don't always behave according to my true nature". But if you challenge any specific attitude or action, they will claim they really felt like doing what they did.

Here's an illustration of my point: I have always wanted to attend a prestigious university, such as Oxbridge or an Ivy league college. Why? Don't I know my own value or intelligence? Do I need an external ratification? As it turns out, I do know my own intelligence (according to most people, I may be a bit too confident about it), but still, I just want the prestige of attending such a university. My brother on the other hand just finished the IB programe. Although he had a prediction enough to qualify him for at least being considered for a place at such a university, he decided not to bother applying. When I asked him why, he just said it does not appeal to him.

"Wouldn't you want people to recognise your intelligence or abilites?" I countered.

"I don't care what people think of my intelligence. What I care about is being decent".

"But how can other people see your decency?"

"Doesn't matter, as long as the people who matter to me can see it".

So, if you really want to raise above the rat race, understand who you are, and act accordingly. Don't let people coerce you in doing things you don't appreciate. Do not wear a tie if you don't like them, just because society tells you too (unless you are my brother/family member), and do wear one if you like it, despite Dr Taleb's objections.

Best,

Complexity is increasing

2009-06-09T06:16:00.000-07:00

If we think of the world as a randomness generating process, we can claim that this process operates with positive feedback - the more complex it gets, the higher the rate of complexity production, as evidenced by the above video.

We need to consider this: in such a complex world, the slightest change in input variables will potentially cause an enormous difference in the output. Thus, even if it were possible to create a simulation of the economy as discussed in a previous post, i.e. even if we could identify all possible factors that influence the system, the slightest miscalculation in the numeric value of these factors would cause the simulation to fail.

Religion VS Science

2009-06-05T13:09:00.000-07:00

This topic may seem a bit unrelated to the rest of the blog at first sight. However, one needs to realise that the underestimation of unpredictability is linked to the more general problem of people taking their knowledge (and themselves) too seriously.
In "The Black Swan", Dr Taleb criticises scientists who are overconfident about their knowledge. This confidence manifests itself in the religion versus science debate. More and more scientists challenge religion on the grounds of it being irrational and unscientific. In doing so, however, they turn their own beliefs into dogma, much like religious institutions did a few centuries back.

An example of this is the argument against Intelligent Design. Now, before I proceed, I would like to declare that I do not believe in ID; however, I do believe it should be taught as an alternative to the Theory of Evolution. As any scientific theory, the theory of evolution does not account perfectly for the state of the world and the appearance of organisms. Proponents of intelligent design claim that evolution does not justify the degree of complexity witnessed in some organisms.
Most scientists, whether biologists or not, respond that this is an unjustified claim, a fallacy. The problem is that most of us simply cannot know this; unless we are taught intelligent design, and shown the weaknesses of the theory of evolution, we remain uninformed. In my school, Darwin's theory was presented as the conclusive explanation of the development of organisms. None of its weaknesses were pointed out. Up to this point, many of my friends and classmates believe in it completely, some times refusing to even consider alternative explanations.

An objection that is commonly raised to ID is that its proponents are politically motivated, and ID is merely a polished version of creationism. Now, to this, I have two things to say: firstly, ID has nothing to do with creationism. As far as I am aware, ID merely suggests that there may be a God behind evolution, and does not support the idea of Adam and Eve and the like. Secondly, the motivation of a proposer of a theory is irrelevant to the theory's validity. If I try to prove Goldbach's conjecture, then a) I may be doing it for the fame, b) I may be doing it for the sake of mathematics, and c) I am in for a disappointment, because I am terrible at maths (surprisingly, you don't have to be good at maths to be good at computer science).

I repeat that this is not an attack against the theory of evolution. It is merely a short diatribe against "scientists" who, contrary to the foundation of science, refuse to doubt their theories and beliefs.

A.

P.S. I have a question on evolution: it is accepted by the scientific community that Lamark's theory (that an organisms lifestyle may alter its genotype) is invalid. How is the bee's dance explained then? It is a mechanism hard-wired in them, in order to point to food. However, how did they learn to do that? Is it really plausible that it was chance? And even if one bee managed to be somehow genetically mutated, and learned to do a complex dance to indicate a source of food, how could the other bees understand it? And even if an entire hive managed to get the extremely improbable same mutation at the same time, how did it spread throughout the whole world? I would appreciate it if someone could answer this!

On unpredictability

2009-06-05T08:51:00.000-07:00

Dr Taleb discusses in "The Black Swan" the fundamental unpredictability of technological innovation. His argument goes like this: if we knew what we are about to predict, we are half way in predicting it already, thus its discovery, when it comes, is not a black swan (i.e. unexpected with a huge impact).

This is a point on which I disagree. Whilst for many technological advances the argument stands (if you have predicted the invention of the wheel, you already know what it looks like, thus you are on your way already), there are many others on which it does not. The fact that we know something is bound to be invented does not mean at all that we are on our way. Jules Verne wrote his novel "From the Earth to the Moon" in 1865, but this does not mean that rocket development was on its way, nor was it any less surprising when real rockets were built and sent in space. Similarly, there are many other inventions one can predict, without knowing how to go about materialising them. Examples could include a computer program that solves chess (i.e. can determine all possible effects of all possible moves, thus never losing), perfect virtual reality video-gaming, traveling beyond the edge of our galaxy etc. Of course, one cannot predict when these discoveries are bound to take place; chess may be solved within the next 10 years, whilst it may take centuries to master inter-galactic travel. Virtual reality video-gaming is hopefully closer.

In addition, Dr Taleb stresses the importance of luck when it comes to technological advances, providing the classic example of penicillin. Whilst this is true, meaning that many discoveries have been accidental, it is untrue to claim that most inventions happen by accident. Examples such as the invention of machine printing or the difference engine (arguably the first computer) corroborate my point that many inventions have been planned.

In conclusion, in his quest to demonstrate the importance of randomness in today's world, Dr Taleb exaggerates its effects, sometimes seemingly neglecting the importance of method.

Note: he still has a point in that complete planning does not allow room for experiment and innovation. This is probably why firms like Google give their employees free time to experiment on their own ideas.

On ties...

2009-06-05T05:55:00.000-07:00

Having provided the basic argument in Dr Taleb's book, I will begin offering some of my own thoughts on the matter, in no particular order. Some of these will be trivial and minor, and some (I hope!) will be more insightful.

The first minor objection I raise concerns ties. Dr Taleb seems to be quite angry (and rightly so) with all the so-called experts of financial markets, the "empty suits". Building on this, he often offers the advice "never trust men wearing ties" (which apparently, several firms, Google included, seem to follow). Personally, I disagree with this advice for several reasons. Primarily, I really like ties.
Moreover, throughout both his books, Taleb supports that we humans are controlled by our prejudices, and should try to overcome them. However, labelling someone inefficient because they wear a tie demonstrates prejudiced behaviour. It is of course understandable: most people who wear ties nowadays only do so in order to look "serious" and convincing. Most of them actually resent ties (they tend to take them off at the first chance!)

However, I am sure that Dr Taleb, an avid advocate on aesthetics, will understand that one can wear ties out of a sense of fashion, rather than an attempt to appear "no-nonsense". Indeed, unlike women, men have very limited opportunities to express their personality through their clothing - the only items which can be truly individual are ties and cufflinks.

In a nutshell: not everyone who wears ties is self-absorbed and inefficient. They could just be very vain (like me).
A.

What is the importance of Black swans?

2009-06-05T05:00:00.000-07:00

The Black swans can have both positive and negative effects. The internet for example has facilitated communication and sharing of information, whilst a war is obviously detrimental.

Unfortunately, the fact that most of us pretend that black swans do not exist, or are extremely unlikely, or that they cannot affect us, causes them to have significant negative impacts when they occur.

The credit crisis is a prime example: economists, traders and insurers were certain that their "Collateralised Debt Obligation" products were completely secure against defaults. They did not expect the housing market to collapse, and were completely vulnerable when that happened.

Another problem that we face is similar to the one explained in the previous post: not only do we try to justify past events, but in general we tend to play down the importance of randomness in the world, believing that we can find and exploit patterns, and thus make accurate predictions about the future. To this end, many formulas have been developed by economics or financiers, to calculate the value of various securities etc. These formulas do not account for what we do not know. As Warren Buffet put it, "beware of geeks bearing formulas"!

I read an article the other day on the New Scientist: http://www.newscientist.com/article/mg20227112.100-can-science-reinvent-the-economy-5-an-economy-in-a-computer.html

It claims that in a few years time it will be possible to build a computer powerful enough to simulate our entire economy. I believe that such a venture is doomed to fail. Economy and society are chaotic systems - there are so many factors that affect them, that in order to properly model them in a computer, all possible factors need to be taken into account, raging from election results to butterflies flapping their wings (potentially causing hurricanes!)

However, once we realise our weakness to predict the future accurately, we can turn black swans to our advantage. Consider being a venture capital: you invest small amounts in many firms, often turning out to be unprofitable. But you only need one to become the next Google to become a millionaire!

Best,

A Brief Summary

2009-06-05T01:26:00.001-07:00

In his books, Dr Taleb discusses the idea of randomness, and explains how this notion often eludes human understanding. In other words, us humans tend to try and find causal links whenever possible, often leading to errors.

First things first: what is a black swan? Let me present Dr Taleb's story: before the discovery of Australia, people believed that all swans are white. When Australia was discovered however, and black swans were first sighted, this view drastically changed. Drawing from this, a black swan is defined as an unexpected event which has a potentially huge impact. Of course, being a student in York, where black swans are (unfortunately) very common, this analogy seems a bit out of place to me, but I guess this does not hold for the rest of the world!

Examples of black swans include the invention of the internet, the world wars, Harry Potter's success, and other similar unexpected events.

Now, the problem with humans is that we tend to try to "whiten" these black swans, that is , we try to justify events that have already transpired, presenting them as having been more predictable than they actually were. This is known as the hindsight bias. As an example of this, Dr Taleb mentions the Great War: in our schools, we are being taught that the war was predictable, as at the time there was a mounting of tensions that was to inevitably lead to conflict. However, the historian Niall Ferguson demonstrated this to be completely false - the war actually came as a surprise to people at the time. Ferguson examined the prices of imperial bonds, and realised that they their price was quite stable up to the beginning of the war. Had the war been anticipated, the price of the bonds would have dropped, or fluctuated significantly. Thus, point one: avoid claiming that events were predictable. They most probably were not.

Next, Taleb introduces the imaginary lands of Mediocristan and Extremistan. The distribution of all events that belong to Mediocristan can be approximated by the Bell curve: the more an event deviates from the average, the more unlikely it is (for example, if the average height for a man is 1.70 meters, with a standard deviation of 10 cm, the probability of finding a man measuring 1.85 meters is about 4%. The probability of finding someone measuring 1.90 is about 1%. The furthest from the average, the lower the probability (no-one expects to find a 4-meter man!). An important property in Mediocristan is that no single measurement can alter the average in any significant way. If we collect a large sample of men, their average height is bound to be around 1.70. Even if we find a 3 meter man, his height will not affect the average. This property is enforced even further as the sample increases in size.

In Extremistan on the other hand, a single measurement can significantly alter the average. If instead of physical qualities (such as height or weight) we measure artificial ones (such as wealth), we find ourselves in Extremistan. If I take 100 people, but accidentally pick Warren Buffet amongst them, their average wealth will be hugely increased. In other words, Extremistan is a land of extreme variation, whilst Mediocristan there is only mild deviation from the average. Clearly, black swans are far more common in Extremistan.

Now, the importance of these facts will be explained in my next post. At the moment, I need to study Natural Language Processing for computer science, as I have an exam in 5 days...

Best,

Introduction

2009-06-04T13:55:00.000-07:00

Hello,

my name is Aris Catsambas. Last year, I read Dr Nassim Nicholas Taleb's book "Fooled by randomness", and more recently, the best-seller "The Black Swan". Although I strongly agree with Dr Taleb's theory in general, there are numerous minor, some times even pedantic, points on which I disagree with him. I will be discussing these points in this blog, although the first few posts will be a summary of his ideas. If you find any of these interesting, I would recommend you buy the books, as they are very well written and interesting. Hope you enjoy this blog!

Best,