[Click here to return to journal index]
© Copyright 2003 by Commodity Systems Inc. (CSI). All rights are reserved.
Website Links: This CSI website and Unfair Advantage system contain hyperlinks and automatic links to websites operated by parties other than CSI. No association with or endorsement of third-party websites should be inferred through these links. They are provided for your convenience and reference only. CSI is not responsible for and has no control over the content of third-party websites.
|
October 2003
CSI Technical Journal Volume XXI, Number 10 |
Page 1 |
Topics discussed in this month's journal:
A Military Approach to Tracking Down Profits and Using Computers to Enhance the Art of Analysis Tech TalkA Military Approach to Tracking Down Profits and Using Computers to Enhance the Art of Analysis
Four decades ago I was working as a contractor to the U.S. Government at a remote U.S. Air Force base in Pennsylvania. My assignment, as a member of an analytical team of engineers, was to report to the President on the likely coordinates of a possible nuclear explosion on the territory of the United States. I was hired to write computer code that would take incoming data from various electromagnetic, seismic, and optical sensors to compute the coordinates of a confirmed nuclear burst, and to report event findings.
The mathematical analysis was straightforward, but the computer equipment was in the dark ages compared to today’s technology. Nevertheless, we completed our assignment of developing the software, which fortunately was never called upon in a non-test situation. Each of us who participated was left with a much broader view of potential computer applications than we had imagined in civilian endeavors.
That experience eventually led me to the career that has sustained me for much of my adult life, because a colleague there introduced me to the commodity markets. Working at that base in Pennsylvania, I often wondered how it might be possible to harness available computer power to fashion a system for trading. In my nuclear detection research assignment the goal was clear, and the events required to render a confirming report were few and manageable. I thought at the time that a similar algorithm aimed at solving a market prediction problem with computers would be all but impossible to construct. All the same, I was hopeful that some day something could be done to conquer the markets with similar relative ease to this disaster reporting system.
A single personal computer equipped with today’s technology is far more powerful than the computers of the late 50s and early 60s. In fact, either of the two PCs that sit on my desk could replace a 1960s era computer that took acres to house and tons of air conditioning systems to keep functional. It is interesting to note that here at CSI headquarters we recently converted half of our old computer room into a conference room to better utilize what had become wasted space. The Convergent Technologies mini computers and racks of TI systems that filled the room throughout the 80s and into the 90s have gradually been replaced by banks of efficient microcomputers.
With the computer capability I currently have the privilege to employ, I can easily view nearly 60 years of daily market data on related markets, and build a model that will learn how best to trade a given market based upon the causal interaction of all markets, with a dependant variable and a collection of all other pertinent independent markets. In my analysis, the dependant variable is the one that I want to trade – the one whose future price I seek to predict.
I call this sort of modeling “Sequential Multi-Correlated Variable Analysis,” although there may be better ways to describe it. This discipline involves analyzing a dependant price series and measuring the causal influence of independent time series that are strongly correlated or strongly uncorrelated. My input stream of information can include any market factor that is known to relate or suspected of relating to the movement of the dependant variable.
The resulting model is actually a mixture of many disciplines that come together to predict the future movement of some market. It draws upon many technical and fundamental aspects of various associated markets and facts that prove to influence the directional movement of the chosen dependant market. The sequential aspect of the analysis tends to identify and weigh the importance of the many independent time series supplied to the process. The more independent markets supplied, the greater will be the computer resources required. This is why today's computers have a better chance of actually solving such a complex problem.
This is a novel approach that depends upon large amounts of data to arrive at a solution. Theoretically, the more data supplied the better should be the result. It views all markets and variables forward in time. There is no hindsight optimization testing. Analysts correctly believe that basing future decisions on results optimized in the past tends to produce unrepeatable performance in real-time trading. Such complaints boil down to the derogatory label of "curve fitting." None of that is done here. This model learns by studying only future (or forward) facts in time. Only one pass through all of the supplied data defines the trading model that will be forecasting tomorrow’s prices.
The model-building stage of the process includes a feedback mechanism that measures the daily causal effects of each independent variable upon the movement of the dependant variable. The feedback logic tends to emphasize the positive and negative effects of each variable upon the dependant variable. The decision-making process therefore becomes increasingly heuristic over time as more and more price history is reviewed. The model is encouraged by predictive success and discouraged by the apparent failures of all independent time series supplied. The basic intent of the process is to teach the computer to profitably trade a dependant variable based upon the predictive characteristics of all independent variables.
If one were to trade the corn market, thus making it the dependant variable, then the wheat market would be relevant due to the correlation resulting from both products’ use of the same soil, similar growing seasons, and the cost of energy resources to power the farm equipment. Energy costs are too often overlooked in market forecasting, but they are very important. Products such as unleaded gas or crude oil would likely be used to plant, plow, and harvest the crop, so the availability of energy products would be important to the corn market and nearly any other crop.
Additional time series that might be seen as independent variables to corn futures are corn cash prices, live hog futures and live cattle futures. The livestock are included because they depend upon corn as their principal feed. The USDA Commitments of Traders (COT) data for corn and the seasonal cycle of corn futures are other variables that are helpful in understanding the dynamics of supply and demand. Data on volume and open interest are of great technical importance in the short term movement of corn prices. Inflation and interest rates are also considered independent variables because agricultural products are affected by prevailing economic conditions.
As you may well understand, there is a rank and hierarchy of products and other factors that affect each market. Once the important elements are defined and introduced into the heuristic model, the decision-making process can be exercised through a single pass over many years of past history.
The computer code written to build the comprehensive model requires extensive memory and disk resources, but we chose a flexible language to make the process work efficiently. We wanted to engage plenty of pertinent variables that can be called upon to forecast a given market.
As with any statistical model, there is a chance that performance may not consistently produce profits. For this reason we drew upon the same digital signal processing theories that R.E. Kalman, PhD. introduced in 1960. Those theories were important to my colleagues in the Advanced Analytical Methods Unit of the Heavy Military Electronics Department at General Electric, where I worked in Syracuse, N.Y. in 1962. The Kalman filter was the buzz word of the day. Those same ideas were used by G.E. to work on a proposal in a project which later became known as the Patriot Missile Defense System. G.E. built radar equipment which sampled the moving three dimensional coordinants of a target missile moving through space. The Kalman filter smoothed the data by filling in data points between samples. This enabled our team to predict a future intersection point where the incoming missile was predicted to reside. A similar scenario might be used to find a critical point where a market has reversed direction.
The Kalman filter is employed in a simplified form in the model we are building to track the dependant variable. This tool will guide every position, and will assist the trader to quickly exit an unprofitable position should the market take an unexpected turn.
We have been working on this comprehensive model for a few months to accomplish the above, but we are not yet ready to demonstrate the efficacy of the developing system. However, we expect to have something to show readers soon. When completed, this comprehensive model might appear to be a natural extension of CSI’s services, but it will likely not be available directly from CSI as part of Unfair Advantage®. We strongly believe that our position as a data vendor must not be compromised nor the main focus of attention of the CSI staff diverted from providing the best possible market data. Therefore, any release of the system will likely be through a third-party vendor. We hope to have more information to share on this exciting project in the weeks and months ahead.
Fortunately, my search for nuclear blasts - almost a lifetime ago - was not fruitful. The exercise itself led me to a fulfilling career with many opportunities to help others gain financial security and control risks. I hope that today’s military engineers are soon able to convert modern technology into peaceful civilian endeavors.
Bob Pelletier
Tech Talk
Each month in this column, the CSI technical support staff addresses issues that may be of interest to many subscribers through this question-and-answer forum. This month they discuss the Northeastern blackout, various aspects of using Position Manager and proper handling of MS (MetaStock) format files.
Q. I trade commodities and want to get involved with Single Stock Futures. Where do they fall in CSI’s rate schedule? Are they considered futures markets or stocks?
A. “Single Stock Futures” is a separate classification on our rate schedule. Within UA they are classified as Futures. Personal and Private Use subscribers who have contracted to receive this market category can view and analyze an unlimited number of contracts involving up to 79 single-stock futures markets daily without incurring extra fees.
Q. A friend tells me he uses Unfair Advantage for intraday updates. Isn’t CSI still a daily update service?
A. Yes. CSI still provides daily updates of market summary data, showing the open, high, low and close for the given trading day. In recent years we have become more liberal and flexible in our updating policies, however. We no longer advise subscribers to update “once a day after the close of the markets.” Now you can request data as many times as are necessary to update all markets of interest in a timely manner, which may involve multiple accesses as markets close around the world. We also offer preliminary stock data, which means it is hot off the wire, but not yet screened and checked. Preliminary stock data can be retrieved as early as 4:45 p.m. Official stock quotes should be downloaded each evening to collect any revisions to the preliminary data. True intraday quotes can be viewed and used for portfolio valuations through Unfair Advantage’s Position Manager module. You’ll find it on the UA Trading Tools menu.
Q. I would like to detrend the markets I chart from my portfolio. No matter how many times I select Detrend on the charting preferences screen, the charts still display with non-detrended data. What’s going on?
A. When you select an item in your portfolio and click the [Chart] button in the Portfolio Manager panel, UA refers to the Portfolio settings (set through the Portfolio Menu) and creates the chart based upon the preferences shown there. The chart settings you select through the “Preferences” screen affect charts that are created without benefit of a portfolio. These are the charts you define individually after clicking the “Chart” icon from the toolbar. Go through the Portfolio menu to make changes to the settings for your portfolio content.
Q. I’m trying to do a “what if” study using some back-adjusted data, and would like to substitute a few hypothetical prices for real ones. I think I’m doing it right: I create a chart, click the “Table” tab and then double click the cell I want to edit. UA displays the price, but it won’t let me edit it. What am I doing wrong?
A. Unfair Advantage allows you to edit market data through the price table as you described, but this feature only works on actual market data, not computed contracts such as back-adjusted and Perpetual Contract Data. If you want to enter a hypothetical price, you must do so for an actual futures contract (or other tradeable). When you select the cell you want to edit, you can either double-click it or press
Q. The CSI support staff keeps talking about “UA Preferences.” My software doesn’t have such a menu. What are they talking about?
A. Unfair Advantage version 2.7.2 and above have a “Preferences” screen that replaces the “Options” menu. This change was made to minimize confusion between program options and tradeable options. Current subscribers can download a free upgrade through the CSI website at www.csidata.com.
Q. I’ve noticed that my new Unfair Advantage upgrade has a “Download/Distribution Selection” screen that offers the choices of “Add List,” Edit List,” “Delete List” and “Run.” What would I use this for?
A. If you typically update the database just once a day, you probably won’t ever want to use anything but the default choice of “Download Everything.” Simply click the [Run] button to download and distribute everything in your subscription categories.
The other choices are most useful for traders who access multiple times daily, updating recently closed markets throughout the day. For example, if you download at 10:45 a.m. eastern time, the only markets with today’s current updates would be from the Pacific Rim. If these markets are included in your CSI subscription, you might make a list of just these exchanges, and run that list for your first download. On your second download at 3:30 p.m., you might want to distribute prices for only the European exchanges that were just updated. By making a list with just these exchanges, you avoid re-distributing the Pacific Rim markets and assure that UA does not attempt to distribute other markets that haven’t been updated yet.
800-274-4727 | 561-392-8663 | 561-392-1379 (Fax)