Sunday, October 26, 2008

Celestial Economic Sphere, DataViz for the Finance Biz, Truthiness, Behavioral Finance, Gordon Gekko, Quants..

I'm hoping to revisit this topic to create a dynamic, interactive time-line of the history we are living through right now, whatever the outcome.

FYI: This long post has been sub-divided and incorporated into the Economic Sounds and Sights blog, where resources for the time-line will can be found.

The Celestial Economic Sphere, DataViz for the Finance Biz, Truthiness, Behavioral Finance, and Greed.

I came across this imag
e of the Celestial Economic Sphere on a post by Shae Davidson one of the authors of the Creative Synthesis Blog, and thought it was a good symbol for the topics of this post.

The Celestial Economic Sphere was created by Lise Autogena and Joshua Portway for the Black Shoals Stock Market Planetarium. The sphere relies on financial data from 4,000 public companies to create a real-time visual display of stock trading:

"Within this environment, a complex ecology of glowing amoeba-like “artificial life” creatures emerge. The creatures live in a world composed entirely of money and they feed on trading activity. Whenever a stock is traded its’ equivalent star produces food for the creatures – the bigger the trade, the more food is produced. . . .Because the stock market has the kind of cybernetic properties of biological systems and other complex phenomena (feedback loops etc.), it can be studied in the same was as biological systems. This tends to give rise to a sense that the market is somehow a “natural” expression of some fundamental forces.

According to the Black Shoals creators, the planetarium was a response to the collapse of a capital management firm in 1998:

"Watching the news we would hear that the FTSE 100 had slipped today, and we knew that meant bad things would happen in the future, but the connections were invisible and mysterious, like the forces the ancient Babylonians thought were exerted on our lives by the stars of the zodiac. These were the time of the boom and the height of media interest in all things stock market related -- a time where the market was often equated to a kind of ecosystem with a life of its own, and where the internal dynamics of the markets appeared to be more important than it's ties to the real world....So Black Shoals was designed as a kind of parody of the trading desk of the ubermench-the Mount Olympus from which they would survey their creation."

The data visualization incorporates a complex artificial life algorithm or genetic algorithm, which is described in detail in Black Shoals: Evolving Organisms in a World of Financial Data.

Spore meets Wall Street!

Note: Black Shoals is a play on words. Shoals are a group of fish who swim together, but in this case. Black Shoals also refers to the Black Scholes formula based on the work of Fischer Black and Myron Scholes. Black and Scholes built the formula on the previous work of Louis Bachelier, known for the mathematical model of the stock market that gave birth to the concept of stock options.

Catherine Mulbrandon's Visualizing Economics blog provides a variety of data/information visualizations related to the US and World economy that are grounded in reality. She collects and creates interesting representations that make it easier to understand economic concepts. Topics include Nominal vs Real 3-Month Interest Rate 1934-2008, Percentage of World GDP - Past 500 Years, (take a look at the comments for the post), Two Thousand Years of Growth: World Income & Population, and US Inflation: Annual Percent Change (1774-2007).

Maybe part of the solution is better data-viz for the financial biz... and everyone else!

I've been scratching my head over past few weeks watching our nation's financial market tumble into widespread crisis. This post is just a reflection of my curiosity about the matter, as I am not a financial expert. I am a low-risk investor and I do not play the market.

When I first heard about the Wall Street storm, I wondered how things could have Saul Hansell, the author of "Where Were the Quants? How Wall Street Lied to Its Computers".

Were the quants mesmerized by their faulty numbers, insisting that no hurricane was coming our way?

I know that there are cool tools around to support the quantitative gymnastics required to propel Wall Street businesses. The quants responsible for this work on the "street" have graduate degrees from top-tier universities. They are very smart.

How could so many bright and talented people be asleep at the wheel?

It helps to understand how the actions of a few generated a ripple effect that went unnoticed by general public. I am just beginning to understand...

According to Hansell, part of the problem is that the computer models related to risk analysis in the financials simply did not account for the complex changes in the industry. The information entered into the computers was faulty, and in turn, the data modeling was flawed. Important decisions were made based on what streamed out of the computers. One ofHansell's quotes indicates that this was not a mistake:

“There was a willful designing of the systems to measure the risks in a certain way that would not necessarily pick up all the right risks,” said Gregg Berman, the co-head of the risk-management group at RiskMetrics, a software company spun out of JP Morgan. “They wanted to keep their capital base as stable as possible so that the limits they imposed on their trading desks and portfolio managers would be stable.”

I've heard numerous financial experts say that no one ever saw the current storm coming. This difficult for me to believe.
Some experts say that it boils down to psychology. Really?

Perhaps people were too scared to take a serious look at the numbers. Others might have been deluded by their greed.

Maybe no-one felt comfortable speaking out. And now it is clear. The Wall Street empire (emperor) has no clothes.

Memory Triggers
A quick Internet search triggered my memory of various events that were going on over the past few years that contributed to our present situation. I'd like to put all of this in an interactive timeline at some point:

As I mentioned previously, the Celestial Economic Sphere/Black Shoals Stock Market Planetarium was created in response to a 1998 collapse of a capital management firm that sent the London FTSE 100 spiraling downward. If you visit the Black Shoals website, you'll see that there was plenty of information that was available to the public at the time that outlined the economic problems going on at the time.

Did we forget the lessons from Enron?
There was quite a bit of publicly available information around related to the Enron fiasco in 2001. Available on-line is the Enron Explorer e-mail visualizer, which was created from the database of all of the email messages between Enron's senior management team as things were falling apart between 1999-2002.

The following NY Times article (February 2002) will refresh your memory: Enron's Many Strands: The Company Unravels; Enron Buffed Image to a Shine Even as it Rotted from Within

Punishment was not swift, as the trial ended in 2006:
Two Enron Chiefs are Convicted in Fraud and Conspiracy Trial (NY Times, 2006)

"... the executives had sanctioned or encouraged manipulative accounting practices and then crossed the line from cheerleading into outright misrepresentations of financial performance."

"Enron's fall had a far greater impact than on just the energy industry by heightening nervousness among average investors about the transparency of American companies. "The Enron case and all the other scandals and cases that trailed after it may have finally punctured that romance with Wall Street that has been true of American culture for a while now," said Steve Fraser, a historian and author of "Every Man a Speculator: A History of Wall Street in American Life."

Not long after the Enron scandal, questionable practices involving accounting irregularities were discovered at Freddie Mac in 2003, resulting in firing of David W. Glenn, the president, and forced resignations of Leeland C.Brendsel, the chairman and CEO, and Vaughn A. Clarke, the CFO. Initially, the executives were rewarded for bad decision making.


Leeland C. Brendsel's severance package was $24 million, which was eventually protested:
Freddie Mac Severance Pay is Protested

(Alex Berensen, NY Times, 6/13/2003)

" ...Standard & Poor's cut its stock rating on Freddie Mac to avoid from hold. ''We believe the company has been less than forthright in giving investors adequate information regarding recent investigations,'' Standard &Poor's said. ''We are concerned about the magnitude of the investigation and its potential political fallout.'' "

At the same time, Fannie Mae was in trouble. An article in the NY Times, written by Jennifer Lee in 2004, outlined the findings of Federal regulators regarding significant problems in the accounting practices used by Fannie Mae related to amortization and derivatives:

Overseer Says Fannie Mae Due for a Shake-Up

"The accounting techniques used by Fannie Mae effectively resulted in off-balance sheet reserves that were used to smooth earnings to meet the expectations of financial analysts, according to the report."

KPMG was the firm that certified Fannie Mae's books at the time. Fannie Mae fired KPMG, and later sued the company, according to a 2006 NY Times article.

KPMG also certified the books of New Century Financial, which failed in 2007:
Inquiry Assails Accounting Firm in Lender's Fall

"New Century Financial, whose failure just a year ago came at the start of the credit crisis, engaged in “significant improper and imprudent practices” that were condoned and enabled by auditors at the accounting firm KPMG, according to an independent report commissioned by the Justice Department...E-mail messages uncovered in the investigation showed that some KPMG auditors raised red flags about the accounting practices at New Century, but that the KPMG partners overseeing the audits rejected those concerns because they feared losing a client."


In my quest for answers, I found an article written by Floyd Norris that explained a few things related to our current state of economic affairs.

In his August 2007 NY Times post, "The Quants Explain Disaster", Norris reflects on a hedge fund manager's comments regarding some of the unexpected financial bumps experienced on Wall Street at the time, and shares fund manager's comments about a report from Goldman Sachs, "The Quant Liquidity Crunch".

Here is an excerpt from the Goldman Sachs report:
"The speed in which the market reacted to the dislocation was unprecedented and it is not clear that there were any obvious early warning signs. With the benefit of hindsight, there were a few clues before last week that might have hinted at problems to come, including the dramatic rise in implied volatilities and the disruption in other markets and the related potential for contagion. No one, however, could possibly have forecast the extent of deleveraging or the magnitude of last week’s factor returns. "

And another:
"In the coming weeks, we will continue to analyze this extraordinary period. We will also re-evaluate and re-prioritize our research agenda in light of recent events. Stay tuned. As we continue to study these events, we hope to gain additional insights that will help us avoid similar problems in the future."

And from the hedge fund manager:
"Translation: we don’t know what happened to us or what we’re going to do about it, but we really, really, really don’t want to admit that the fundamental premise of our business is fatally flawed and shut down, so we’ll come up with something.
...In other words, what they do works 99% of the time, but the other 1% of the time they blow up — especially since they insist on using a ton of leverage because their brilliant models tell them that what happened last week was a 28-standard deviation event. Hint: IT WASN’T!"


This is where psychology comes into play.

Remember the "Go Go '80's"?
Reaganomics helped improve the economy during that decade, and as a result, which helped us forget how bad it was in 1981 and 1982.


Some of us forgot that this led us to 1987, the year of Black Monday. The date was October 19, 1987. The stock markets around the world crashed to the ground, but by the end of the year, there was an uptick.
Carlson, Mark (2007) "A Brief History of the 1987 Stock Market Crash with a Discussion of the Federal Reserve Response,"Divisions of Research & Statistics and Monetary Affairs Federal Reserve Board, Washington, D.C.

Black Monday Ten Years After: The Motley Fool's 1987 Timeline

(The above resources were found on Wikipedia.)

1987 is also known for Gordon Gekko, a fictional corporate raider played by Michael Douglas in the movie Wall Street. Gekko firmly believed that greed is good. Ironically, the events in the movie forshadow the scandals that have been played out on Wall Street and corporate America over the past 21 years. Below is a quote from Gekko's speech, and the corresponding video clip I found on YouTube.

"The point is, ladies and gentlemen, that greed, for lack of a better word, is good. Greed is right, greed works. Greed clarifies, cuts through, and captures the essence of the evolutionary spirit. Greed, in all of its forms: greed for life, for money, for love, knowledge, has marked the upward surge of mankind". - Gordon Gekko
(Memorable Quotes from Wall Street)

(I will take down this video if I'm notified that it violates someone's copyright.)

Humans are complex creatures. It would be challenging to create a behavioral finance application that could account for and predict various psychological and sociological scenarios. What would represent a constant? What characteristics, traits, behaviors, and inclinations would play as variables?

Here are a few:
It is appropriate to say that "greed" is an important human variable that should be incorporated into this affective/behavioral/financial applications.

Here are a few more:
"herd mentality", "politics", "power", "control", "need for constant adrenaline rush", "sins of omission", "sins of commission", "consumer confidence", "illusion of stability", "cluelessness", and of course, "

I don't intend this to be a joke. In real life, this would be a serious endeavor. I''m realistic to know that Wall Street quants would not waste their time trying to figure out how to quantify the concept of truthiness.

Maybe they should!

Here is an example of something I think touches upon the usefulness of the "truthiness" concept.

Although the following article was written in 2003, it holds up well in 2008.
The seeds of truthiness were planted well before Stephen Colbert came up with the word:

What the Quants Don't Learn in College
(Emanual Derman, Risk Magazine-Trends July 2003/Volume 16/No7

"The only universally applicable law is that of approximate similarity, which states that the best estimate of the unknown market value of a security is the price of another security that's closely similar to it. You need to find (or invent) a model to establish the similarity between two securities by demonstrating the equivalence of their future payouts under a wide range of circumstances....

Suspend disbelief
Although all you have is the limited power of this simple law, you must take your model of similarity seriously. Temporarily, like a fiction reader, you must suspend disbelief in your model. Then, when it's complete, remind yourself that economics and valuation involve the behaviour of people, and think hard about what could go wrong...

On Wall Street, no-one knows what the correct model is, but they go ahead and price and trade anyhow. It's a bit like the trial in Alice in Wonderland...Academics often overemphasise models, but much of the success of a model depends on software engineering....You need live market data, historical time series, databases, input screens and calibration. As a result, for every financial engineer who works on a model you may need three or four more software engineers to make it usable.."

I wonder if any Wall Street companies hired additional software engineers. From what has transpired over the past months, if they did, they were not the right software engineers!

To take a closer look at how software problems might have played a small part in the current situation, I chose to browse the Calyx Software website. Calyx provides software that is used by Freddie Mac and Fannie Mae. There is a treasure trove of information about the quality of this companies mortgage processing software on its support pages.

From my armchair analysis of the types of errors, it seems that it is not difficult for mortgage brokers to unknowingly make errors when determining a potential borrower's risk. The Freddie Mac Loan Prospector looks like it was rolled out before important errors were discovered. In my opinion, it was designed without the capacity to prevent critical errors.

Because the Catlx support page lists many of the problems as common, it is likely that the Freddie Mac Loan Prospector was designed with a lower level of error prevention than expected for this sort of transaction. To fix this problem after the fact, the support pages offer help solutions, but some of the solutions are quite complex.

Common Problems with the Freddie Mac Loan Prospector
Second home is not included in ratios

Credit Agency Missing from the Freddie Mac Loan Prospector
Credit Agency or Lender is Missing from Point

Little things that waste time

Must-watch video
If you have about an hour, watch this discussion between Charlie Rose, Floyd Norris, Mohamed El-Erian, Gretchen Morgenson, and Nouriel Roubini, which aired at the time of the Fannie Mae and Freddie Mac "bailout" decision.

According to Floyd Norris, "The senior managers of the banks assumed that the wizzes under them had their financial models which proved that they did not have any value at risk, and had nothing to worry about, and they believed all of this. And now what they believed looks like nonsense, and you wonder why they did. And that left those banks very exposed... In a lot of cases, they thought they were making money, but they really weren't."

The one thing that is clear to me is that our current economic models no longer function. We can't put the blame on the quants, or the politicians, or the greedy Wall Street leaders. We can't put the blame on new homeowners with low-incomes, or pressured lenders.

Right now, there is a high level of uncertainty, and we do not have anything tangible that ensures that things will be OK. This problem can not be solved quickly. We need better models that can support effective economic decision-making on Wall Street, Main Street, and everywhere inbetween.

The problem of preventing future economic disasters won't be solved by politicians, government officials, economists, and Wall Street leaders. The general public is strongly against the rescue bailout. It simply is too difficult to trust those we've blamed.

My solution at the moment?

We need to stretch our thinking and cast a wide net. This will require an interdisciplinary approach, and include people from a variety of disciplines, who are untainted by monetary scandals and have innovative minds, who care about future generations, and who believe strongly that in a democratic society, all citizens must have access to accurate, understandable information in order to make effective decisions - in all aspects of their lives.

Who might these people be? University researchers, practitioners in the workplace, graduate students, soccer moms, grandpas...with experience in areas such as finance, psychology, history, geography, urban planning, business, economics, banking, sociology, computer science, human-computer interaction, information visualization, graphic arts, & communications.

It is up to everyone to wake up and take action in some way.


Behavioral Finance: Benefiting from Irrational Investors
(Julia Hanna, Harvard Business School)
"Behavioral finance replaces the traditional and idealized idea of rational decision makers with real and imperfect people who have social, cognitive, and emotional biases. The resulting inefficiencies in the capital markets can create opportunities for investment managers and firms."

Behavioral Finance: A Review and Synthesis pdf
(Avanidhar Subrahmanyam, 2006)

The Behavioral Finance Hoax pdf
(Richard Michaud)

Detailed presentation, includes theories and formulas:

A Survey of Behavioral Finance
(Nicholas Barberis, Richard Thaler, presented by Ryan Samson, CalTech)

Behavioral Finance at JP Morgan

(Malcolm P. Baker, Aldo Sesia Jr. 2007, Harvard Business Publishing)

Ivan Boesky

Black Shoals/Scholes Related
The Midas Formula (BBC, 1999)
Midas Formula Program Transcript
Exibition Puts Stocks in Lights
Cefn Hoile

Jean-Philippe Rennard
Book: Handbook of Research on Nature Inspired Computing for Economics and Management
(Rennard is the editor)
Ants Viewer
Social Insects and Self Organization
Genetic Algorithm Viewer
Artificial Life and Genetic Algorithm Links and Resources

21 Ways to Visualize and Explore Your E-mail Box
(Flowing Data)