(Un)Calculated Risk | by Peter Orr of Intuitive Analytics

I know evolutionary biology comparisons to business are a bit
tired at this point, but having just checked off an item on my personal bucket list (diving in Galapagos!) I can’t resist reciting a few quotes from my friend Chuck Darwin. After seeing some of his famous finches firsthand one day and dealing with a balance sheet market risk analysis the next, I can’t help myself:

"Programming today is a race between software engineers stirring to build bigger and better idiot-proof programs, and the universe trying to produce bigger and better idiots. So far, the universe is winning."            - Unknown

Many public finance businesses are grappling with the "build vs buy" question as it relates to their analytic tools.  I've commented here on the challenges of building good financial software and frankly, most firms that aren't specifically in software development are poorly equipped to do so.  And this is a tangentially related and important question.

Of course, build vs buy is not a new question generally but it may be new to some in financial services. The most succinct variant of the common wisdom on this is in this infoworld article . Here's the bottom line from the article:

"Decades of trial, error, and egghead analysis have yielded a consensus conclusion: Buy when you need to automate commodity business processes; build when you're dealing with the core processes that differentiate your company."

There's an interesting dynamic about the technology "backbone" behind a public finance business (providing accurate/current debt profiling, refund screens, historic prices and reset histories from EMMA, bond and option valuation functionality, etc).  Although these may be commodity business processes that every banking/advisory/investment firm must do in one form or another, that data backbone serves as the foundation for proprietary analytic tools and reports that can differentiate one from the rest. 

At IA, we do both automation and innovation so that the workflow public finance analytics need to support are fully addressed. This allows us to be far more efficient and effective in designing each.    

Other good "build vs buy" articles here:

techrepublic 

MIT Free Software: Build or Buy Dilemma 

In the 2^nd post on the topic, Dan Cooreman of the Sunday Business section indicated that the problems Gretchen Morgenson spoke to in her article,The Swaps That Swallowed Your Town, were in fact related to the variable rate instruments.  He seemed to agree with exactly my point but ultimately, my final words (below) to the Times editors on the topic have gone unanswered.

One of the (few) benefits of leaving a perfectly functional Street job is that I'm now free to have a free dialogue with the free press using my now free(er) speech.  To that end, I decided to engage the NYT a few weeks back regarding Gretchen Morgenson'sThe Swaps That Swallowed Your Town article.  This is the first of 3 posts recounting the transcript as surprisingly, they did engage.   

Two people were examining the output of the new computer in their department. After an hour or so of analyzing the data, one of them remarked: "Do you realize it would take 400 men at least 250 years to make a mistake this big?"       Unknown 

I'm a big fan of Riccardo Rebonato. From the book on interest rate models, a required text in my grad school, to the papers he's done on interest rates measures in the "real-world", he's an extremely clear thinker on otherwise murky stuff. I can't recommend more highly his recent book, Plight of the Fortune Tellers. If you or your clients are in the business of making tough financial decisions, it's a must read and enjoyable to boot. Enough gushing (I need payment to go any further ...)

One extremely important concept woven throughout Plight is the difference between the traditional "probability as frequency" concept and the more general Bayesian or "subjective" probability. Probability as a pure frequentist concept is a special case of Bayesian/subjective probabilities that would be appropriate when looking at the likelihood of a head after a coin flip. Outside of a belief the coin is fair, no prior knowledge is necessary to reliably assess the likelihood of such an event. Contrast that with say, the probability that the Jets win the SuperBowl in 2011, or the Republicans retake the House in November, or even that gold goes over $1,500 an ounce by year end. These are all events to which we could also assign a probability, though analyzing purely historical data in a frequentist sort of way will yield few helpful results. We are much more inclined to include and use other relevant information such as the Jets strong defense going into the next season, the anti-incumbent mood of the electorate, and the growth of global money supplies.

What does this have to do with the use of raw historical data in financial decisions support analytics? A lot. Certain financial questions are better answered using frequentist concepts. Others are far more judgment-based relying on more subjective criteria and professional experience. But how do you know which situations are which? Though no hard and fast rules exist, there are basically four criteria:

Data frequency - The more relevant data you have, the more inclined towards a frequentist approach.

Time horizon - the longer the horizon of analysis, the more likely a subjective analysis will be more relevant.

Rarity of event - the more rare the event, the more the analysis calls for a Bayesian/subjective approach. 

Time homogeneity of data - Were there no regime changes or other tectonic shifts in the underlying phenomena from which data was gathered? If so, analysis will tend more towards frequentist methods.

So for long time horizons, a scarcity of data, significant changes through time in the realm in which the data lives, and highly improbable events, we land squarely in the realm of subjective probabilities. Though historical/frequentist data isn't ever completely irrelevant, in these circumstances professional judgment of the situation at hand trumps pure number crunching. Unfortunately, from rating agencies to regulators to a large swath of finance professionals, this is not well understood. Things are just much more clean and simple if we allow ourselves to believe that 100 data points and a fancy model will yield 99.97% confidence precision. This is a particularly dangerous type of belief in finance, as acutely borne out over the last 18 months.  

The good news is that whether frequentist or subjective, widely available probability-based models should always be used to capture risk metrics, evaluate best and worst outcomes, assess breakevens, and ultimately to avoid the ever pervasive flaw of averages.

"The first and most important thing to understand about Monte Carlo is that it is a numerical technique, not a model."

It's 2010 and time to raise the bar.  Public finance software has remained substantially unchanged for over a decade, and probably more like two.   In this day and age your public finance software, in addition to all the other stuff it's done since TRA86, must add the following five features: 

Lots has been studied and even reported on the decision criteria involved in pulling the trigger on a public finance refunding. When is the right time? What bonds do I choose? What savings target do I use? The old rule of thumb that present value (pv) savings should be at least 3% of refunded par is taking some criticism which I won't repeat here. Suffice it to say it's a threshold originally conceived by bankers and as such, the bar for a "Go" decision is not very high. I personally think it's age discrimination; this calculation does have a good three decades under the belt...

"Knit Cap Creates Huge Hangover" is Not a Good Headline