Neural Networks for High Accuracy Forecasts

In USING NEURAL NETWORKS: AN ANALYSIS. OF METHODS AND ACCURACY, Jason E. Kutsurelis accomplishes some really amazing empirical runs, predicting 10-day returns of the S&P500 index. It seems incredible that he would make this awesome literature freely available to the world, and yet very few have yet to embrace the concepts successfully simply because it takes some brain labor.

Key points from his Neural Network model 10-day forecast tests:

Condition Probability of predicting a market rise: could reach 93%
Condition Probability of predicting a market decline: could reach 88%

When compared to traditional multiple regression, the NN outperformed it in several ways. Lower error bounds and higher accuracy.

Predictive daily variables used for forecasting 10-day future SPX closing price:
1) SPX Day-High Price
2) SPX Day-Low Price
3) SPX Closing Price
4) DWT, DOW Transportation Index Closing Price
5) DWU, DOW Utilities Index Closing Price
6) XOI, AMEX Oil Index Closing Price
7) CRB, Commodity Research Bureau Closing Value
8) XAU, Gold and Silver Mining Index Closing Value

A lot of mathematical or statistical software today have capability to do neural network work, SPSS and Matlab have pretty user-friendly interfaces. Lots of stuff to play with!

*Quick note, a lot of theoretical stuff goes behind the above, and some studies have found Support Vector Regression to outperform Backpropogation Neural Networks in financial forecasting. This area of research has still got a long way to go.

IT Outsourcing

I attended Manda J.’s Girl Geek Dinner last night, and encountered some interesting points regarding outsourcing issues in the industry today. As computer science graduates diminish, partly due to fear of declining demand at home, those in the industry must create and embrace some added value to make outsourcing undesirable.

Efficiency

A distance in communication naturally results in efficiency sacrifices. E.g. issues requiring a day to resolve locally may linger unaccounted for, unclearly defined, or incorrectly relayed by an outsourced party for weeks or months.

Aligned perspectives

This presents a big issue dealing with 3^rd world manufacturing facilities. With large cultural, language discrepancies, misinterpretations on product/service looks, functions, core values occur, sometimes at relatively high cost.

Corruption

Under the table transactions occur at all times. Having conducted business in China, everyone is hungry to make a buck, and everybody speaks in cryptic tongue with lots of implications. Needless to say, conflicts of interest surface and expand inevitably over time.

Making outsourcing undesirable

Notwithstanding the foregoing, many firms remain willing to make the said sacrifices in pursuit of greater profit margin. The job then comes to the next generation of IT people to convince industry leaders that the risks, the necessary sacrifices, outweigh gains off higher paid local minds. With accurate incentives, anyone can be influenced.

Social Security Payments to Shrink

For the first time, Social Security payments will shrink next year due to negative inflation, just when the American people needs it the most. Ironically, the politicians whose reckless spending brought on this snafu were all democratically elected by the average American folks. So now that crap has stroked the fan, how will this thing unfold?

“

Millions of older people face shrinking Social Security checks next year… The trustees who oversee Social Security are projecting there won't be a cost of living adjustment (COLA) for the next two years…

”(Yahoo News)

Not surprised

Many folks had anticipated this a mile away. Remember that movie IOUSA from half a year ago? I remember reading Ron Paul’s The End of Dollar Hegemony back in 2006, it explains partly the emerging American economic turmoil. Even Warren Buffet, successful trader (who made initial millions off classical arbitrage), implies that America may not viably handle existing, growing debt.

Survival

It pays to become educated in economics in this heavily finance driven society today. As retiring off social security does not appear practical anymore, the next generation of retirees must step that extra step, exploit market demands to boost income, while living within the means.

Oh, and perhaps vote for more responsible politicians, instead of some slick talking, big business advocate?

Corporate earnings decline

Robert Schiller at Yale Department of Economics updates the above, an inflation adjuted S&P500 earnings value. This explains the presently extraordinarily high P/E value of 140+. Simply unprecedented.

Some propose the Federal Reserve interventions lately have upheld the bubble despite the deteriorating economy, evidenced with GDP at -3.9%, inflation at -2.1%, and escalating Jobless Claims (tradingeconomics.com). Sooner or later, reality will set in and the horrifying mean reversion will come.

It will take place with either extreme inflation, dramatic equity price drops. Given that inflation stands negatively today, the decline will probably occur first, then the dollar crisis. Nice to have some excitement to look forward to, huh!

Vocabulary and Ingenuity

Perpetual vocabulary growth helps to improve creativity. As novel words trigger unprecedented emotions, ideas, imageries, naturally ingenuity ensues. Of course it also takes work, sacrifice, like most rewards in life.

Examples

“Arbitrage” remains a word many newbie market participants have not encountered, a concept never even considered; and once introduced, new possibilities, strategies, hopes and dreams, IMMEDIATELY arise.

“Expectancy”, a term from statistics, explains why casinos win over time, why arbitrary buy-and-hold tactics do not end well for equity positions, and more importantly, the necessary numbers to create a winning strategy. Winning, after all, is everything.

“Charisma” remains really important in society today. Social status makes people feel good about themselves and those in their lives. Understanding the word, following up on it and enhancing personal magnetism could potentially make a world of difference from corporate promotions to making new, fun friends.

Simple steps

Read every day, and look up, UNDERSTAND any unfamiliar words. Think about what they really mean, and how this new concept could help your world in any manner. Life can only expand from this point, exhilarating adventures await!

Rugby brawl!

I live really close to that high school. Off the camera rumour remains that one of the guys made an offensive ethnic remark, and before long even the spectators (fathers, mothers, and siblings haha) all wanted a piece of each other. Another awesome day in Auckland!

FDIC running empty already?

Noticed this at Mich's blog.

So what will likely happen next? The market will likely selloff hard again, then the US government will reiterate the whole "bailout the system (e.g. FDIC) or it'll get worse..." thing again. History does repeat; just never in identical fashion!

S&P Stats 7/31/09 (PE at 143.95)

This is from the StandardAndPoors.com

Things look pretty shaky from here.

S&P 500 Exchange Representation As of July 31, 2009
	Number of Cos.	% of Market Capitalization
NYSE	411	80.7 %
NASDAQ	89	19.3 %
AMEX	0	0.0 %

---

S&P 500 Statistics As of July 31, 2009
Total Market Value ($ Billion)		8,660
Mean Market Value ($ Million)		17,319
Median Market Value ($ Million)		7,096
Weighted Ave. Market Value ($ Million)		71,956
Largest Cos. Market Value ($ Million)		343,483
Smallest Cos. Market Value ($ Million)		712
Median Share Price ($)		30.050
P/E Ratio*		143.95
Indicated Dividend Yield (%)		2.18

NM - Not Meaningful *Based on As Reported Earnings.

Gamma Scalping with Stock Options

Joe Troccolo here discusses the process of "Gamma Scalping". It would be nice for him to also spend some time on direct risks as well. Nevertheless, I like this video for the brevity and precision.

Troubled Assets Still Around

The credit derivatives still linger, alongside future ARM defaults. The New York Times had the following to say,

" The Federal Reserve estimated in May that American banks still had about $599 billion in assets to write down. Goldman Sachs and the International Monetary Fund estimated the total at about $1 trillion. And RGE Economics, headed by Nouriel Roubini, has estimated the total at $1.27 trillion. "

So until Americans sort out the above, the crisis will worsen. Pretty obvious right?

Socialism and Politics

Special interest groups have historically applied impractical socialism ideals to captivate supporters amidst economic hardships. China, North Korea, and the former Soviet Union come to mind. While utopian theories sound nice, they do not offer feasibility as these ideas forget to factor in corruption related risks.

Political process rundown

The scheme from the (evil) interest group’s perspective.

1. Allow or influence existing political will to undermine the economy, therefore lowering living standard. As power generally leads to corruption, even Rome had fallen while its enemies needed nothing but patience.

2. Initiate campaign, appear empathetic of the peoples’ outrage at the existing government, business owners for job cuts. Pacing remains crucial in bypassing the critical factor, i.e. by appearing to agree on issues off various levels, the audience becomes more inclined to accept any following suggestions readily. (A common speech pattern utilized by US politicians.)

3. Through all media channels, promote needs for equality, sharing, friendship and helping… This is where the distraction, brainwashing takes place; “…give us a chance to lead, and we will make other people work and pay for your life style, yay!”

4. Once in power, terminate all potentially opposing forces so as people begin to see the truth, they have already become helpless. When Mao’s communist party triumphed in China, they killed anyone with more than a high school education, business owners, economists, national party members, and etc.

What about wealthy areas like Europe and Australia?

2 words: Global Warming

Business School Grads Have It HARD

So this NY lady who just graduated with $70K in debt, no employment prospect, decides to sue the school. While empathetic, she could have seen this coming what with the vast amount of information available regarding outsourcing and economic conditions.

" Thompson, a graduate of Monroe College, is suing her school for the $70,000 she spent on tuition because she hasn't found solid employment since receiving her bachelor's degree in April, according to a published report. "
Full Story


It really begs the question, considering risks off tuition and opportunity cost, do business oriented schools offer any REAL value? Having gained no unique or technical skills, how does any graduate expect to even find adequate employment or freelance work?

Interesting Forecast Research with SVR Network

Li, Hu, and Hirasawa made some interesting findings forecasting USD/JPY exchange rate moves in “Financial Time Series Prediction Using a Support Vector Regression Network”, or SVRN. Their Next-Day (t+1), Next-Week (t+5), and Next-2Week (t+10) directional predictions resulted with close to 80% accuracy. It rocks. For those unfamiliar with SVR, a detailed guide sits here.


SVRN test rundown

With the t+1, t+5, and t+10 USD/JPY exchange rates as output, input variables included historical USD/EUR exchange rates, the NIKKEI225, and oil prices. They essentially split the training data into three making up the “Transformation Layer”, whose outputs then became input variables for the “Prediction Layer” where the magic takes place.


What made this research interesting

Through the SVRN, the predictions displayed significantly lower SSE (Sum of Square Error) than those performed by SVR alone, especially with the longer time steps. This makes it much easier to create and support a practical trading strategy.


Did I mention they showed a close to 80% hit rate for all three forecasts? Keep in mind the SVRN derived forecast models remain INDEPENDENT. Just applying basic probability theory, one could theoretically wait until all three forecasts pointing to the same direction, bet on it and have a 98%+ hit rate (1- 0.2^3). Pretty cool huh!