December 2002

[Click here to return to journal index]

Notice: The views and information expressed in this document reflect the opinions and experience of the author Robert C. Pelletier. Neither CSI nor the author undertake or intend to provide tax advice or trading advice in any market or endorse any outside individual or firm. All recommendations are provided for their informational value only. Readers should consult competent financial advisors or outside counsel before making any software purchase or investment decision. CSI does not stand behind or endorse the products of any outside firms.

© Copyright 2002 by Commodity Systems Inc. (CSI). All rights are reserved.

Website Links: This CSI website and Unfair Advantage system contain hyperlinks and automatic links to websites operated by parties other than CSI. No association with or endorsement of third-party websites should be inferred through these links. They are provided for your convenience and reference only. CSI is not responsible for and has no control over the content of third-party websites.

December 2002
CSI Technical Journal
Volume XX, Number 12

Page 1

Topics discussed in this month's journal:

Holiday Schedule

Making Futures Markets Look Like Stocks

Tech Talk

Holiday Schedule
CSI will be closed for voice communication from Tuesday, December 24 at 6 p.m. through Wednesday, December 25, and from Tuesday, December 31 at 6 p.m. through Wednesday, January 1 for the Christmas and New Year’s holidays, respectively. The CSI host computer will be accessible without interruption throughout the holidays and data will be available for any market that remains open on these days. All of us at CSI extend our best wishes for the season and the coming new year.

Making Futures Markets Look Like Stocks
The introduction of Single-Stock Futures in November ushered a new breed of traders into the realm of futures trading. This hybrid stock-futures trader is, no doubt, accustomed to the world of stock charting and analysis, which, although rife with its own subtle complexities, can be a cake walk when compared with technical analysis of futures. The way stocks are traded leaves a relatively straightforward picture of performance over time. Except for the minor complications of handling dividends and splits, stock prices are typically displayed in their raw form, as they are quoted by the exchange. Technical analysis of futures markets would certainly be more convenient if futures left stock-like footprints. Today we will discuss efforts to make just such a transition. The nature of futures trading requires that futures contracts be subject to the supply and demand situation at a forward point in time, when successive contracts are targeted for delivery. A July 2003 corn contract, for example, will be priced according to the expected situation in July 2003. Successive contracts are priced according to the supply and demand conditions of later dates and other circumstances. This forward-looking perspective would be easier to deal with if futures weren't cursed with a short contract life. Every contract is destined to expire within a few months or years. Liquidity, measured in terms of volume and open interest, rises and falls over the course of each contract's life, forcing each successive delivery month to stand on its own.

E Pluribus Unum
From many one. It is hoped that single contracts derived from many will be useful in defining price behavior over an extended period. These lengthy continuous series do look like stocks. Assuming that simulated buying and/or selling can be done in hindsight, one can use these series to simulate real market conditions over time. To that end, the analysis community has come up with several ways by which a single time series can be fashioned from many. CSI's Unfair Advantage® (UA) software includes the ability to create several types of computed (stock-like) contracts. Every computed contract produced by UA requires some factors be identified before building the file. These rules, which may be simple or complex, are followed during construction of each series. Identity details of computed contracts are available through UA's portfolio display. These include statistics such as the symbol, portfolio name, directory, description, exchange, format type, file name, period (daily, monthly, annually, etc.), the current delivery month, the previous delivery month, and the date when last rolled. This display covers all of the components of each portfolio. Traders have found it to be very useful to track new contract progress over time, as new contracts are sequentially entered based on predetermined rolling rules.

Click Image to Enlarge
Caption: An expanded portfolio listing. Use your mouse to resize the portfolio to see these details.

Nearest Future Data
One of the first methods used for producing continuous commodity series was to simply concatenate the prices of the Nth nearest (usually the first nearest) future contract. To prepare a continuous series of the Nth nearest future contract requires showing the prices of the rotating delivery month that is precisely N contracts forward. This is the electronic equivalent of stringing beads on a necklace. A chart of a nearest future display will outline distinct segments showing actual market movement over time, but it holds little attraction for analysis purposes because the data inevitably shows jumps or drops in price as contracts expire and later contracts are reported. The beads on this necklace are of different colors and sizes, so the effect is not one of smooth continuity. Even though these displays have limited analytical value, viewing them may be worthwhile because they show market movement that may be helpful in one's quest to understand market dynamics.

Perpetual Contract Data
Another, more viable way to present long-term historical prices for futures contracts is through the use of Perpetual Contract® data (a CSI exclusive). A Perpetual Contract series represents a time-weighted average of the pair of forward contracts that lie before and after a date that lies a specified number of days ahead of the current date. This forward date will lie an exact number of days after the expiration date of the earlier member of the series, and before the exact expiration date of the next more distant contract. The time-weighted average of these two forward future contracts will determine the Perpetual Contract data price. This calculation is repeated for every data point. Here’s a corn example: Let's look back to the date of May 1, 2002 and pretend it is the current date. A perpetual period of ninety days forward of the current date of May 1 would be July 29, which is roughly nine days after the July corn contract is set to expire. The expiration of the September corn contract (the next distant) will occur on approximately September 20, which is 53 days from July 29. Given this information, the 90-day forward Perpetual Contract price for corn on May 1 would be calculated as (53 times July price + 9 times September price) divided by 62, which is the time span in days from July 20 to September 20. As you can see from this example, the 90-day forward Perpetual Contract price is weighted proportionately more for the July contract because the perpetual period point in time of July 29 is closer to the July contract’s expiration of July 20 than to the September contract’s expiration. Perpetual Contract data offers many advantages in the development of a trading system for commodities. By carefully setting the forward perpetual period, one can arrange to capture pricing statistics that lie near the center of volume and open interest liquidity for the market. The appropriate period is usually based upon the frequency of trading months, and can be determined through a small amount of experimentation. Perpetual Contract data particularly lends itself to simultaneous analysis of multiple markets for intermarket spreads and straddles that involve a variety of dissimilar products. In such an environment, it makes good sense to view related markets from the same perspective. Perpetual Contract data is an excellent way in which to paint and compare all series from an equivalent future perspective, provided the perpetual period is a constant across all series. This is one important reason why UA’s MultiMarket Analyzer employs Perpetual Contract data to analyze a related mix of markets.

Back-Adjusted Data
If your testing method deals with one market at a time, the back-adjusted contract may be a good data choice. Looking backward in time, these computed contracts use real contract pricing, plus or minus a "delta" factor that cumulatively adjusts prices as earlier contracts move into view. The delta differences can be applied to earlier prices as an arithmetic addition to or subtraction from such prices, or as a proportional multiplier that will raise or lower earlier prices by the ratio of the later price to the earlier price. Proportional adjustments tend to avoid a possible bias that could force early prices to become negative. A proportional adjustment is, in effect, a percentage change in price that could move earlier prices to near zero, but never less than zero. When computing delta changes as one contract is joined with an earlier contract, the arithmetic or proportional delta effect may be computed by taking the difference between earlier and later contract opens, earlier and later contract closes, or an earlier contract open and the later contract close. Looking backward in time, this calculation always involves rolling from the later contract into the earlier contract, and is always cumulative. Delta is computed based upon both the earlier contract’s date and price, and the later contract’s date and price for the very same date. This could be the date before the actual roll or the date of the roll, depending upon your preferences. If the rolling rules involve a close-to-open adjustment, then two days of data are necessary to compute the current delta. An advantage in building a back-adjusted file is that the series will represent actual traded prices with an arithmetic or proportional adjustment factored into the data. The end portion of the time series will always reflect current prices for the most appropriate contract of the given commodity. You can be confident that any back-adjusted contract represents the unadjusted prices of a current contract in your UA data files. During inflationary periods, commodity prices tend to rise, resulting in positive delta values. Therefore, back-adjusted series are likely to lose some of their overall trend as cumulative delta values inflate earlier prices.

The Importance of Detrending
When forming computed contracts of any type, an important option open to UA users is the ability to detrend your results. This manipulation (which literally removes the trend from the data) is often overlooked by market analysts, but is strongly recommended. All markets are subject to levels of historical supply and demand that depend more on the uniqueness of the product than on overall conditions of the economy. Therefore, asking for a mix of products to be detrended over the period of interest for your model may be important to the success of your analysis.

Click Image to Enlarge
Caption: Detrended data (top) and non-detrended data (bottom). To graphically illustrate the effects of detrending, we analyzed over 36 years of world sugar data from 1996 to the present. This chart shows the earliest eight months of both series. (Eight months conveniently fit into the space allocated for this comparison.) As the chart illustrates, the detrending process did not distort the market's day-to-day behavior, but it did accomplish the desired result by bringing all prices up to the levels of today.

In the detrend process, UA focuses upon the closing price on the beginning day of the series and on the closing price on the end day of the penultimate contract in the series. The difference in prices between those two dates will identify the total period change in price. UA gradually (linearly) adjusts all prices proportionately over the detrend period to distribute the price change over time. Example: If the price rose during a given period, and the period covered 1,000 days, then the first open-high-low-close would be elevated by the (end price minus the start price) times 1000 divided by 1000. (The first day would therefore, be increased by the maximum delta difference.) The second day's O-H-L-C price would be elevated by the (end price minus the start price) times 999 divided by 1000, etc., etc. Finally, the last day's O-H-L-C price would be elevated by the (end price minus the start price) times 1 divided by 1000, which would result in minimal distortion to this nearby price. While traders may use decades of historical data to prove their proposed system will produce profits, they often fail to understand that results in the distant past may drastically understate profits and losses in today’s terms. Detrending forces the conditions of today to gradually appear equivalent to the conditions of the past, correcting a basic economic flaw of treating all dollars equally in long-term technical analysis. Detrending works quite well for some markets, but the Green Revolution (when farm machinery replaced the horse and plow) improved farm efficiencies, and helped compensate for an otherwise inflationary bias. Consider that in 1966 sugar was priced at about 2.5¢ per pound. Sugar today ranges around 7¢ per pound. A 100% return in 1966 could move the price of sugar from 2.5¢ to 5¢. This would be booked as a 2.5¢ profit without detrending. A 100% gain at today’s prices would show a 7¢ profit. Without detrending, a highly significant profit in 1966 could register as a small aberration in your long-term analysis. If you don't detrend, you may well fail to learn history's lessons. Money has relative time value that should be brought forward to today’s conditions. When you detrend a series under the UA system, you force the prices at the beginning of the chosen series in your econometric model to appear equivalent to the prices at the end of the series. This may seem drastic - which it is, but it is also vital. Although stock price series have a standard appeal that derives from their relative simplicity, we believe the benefits of futures are worth the extra analysis efforts. We welcome those who are new to the world of futures trading. We hope this Journal will serve as a helpful guide when you explore ways to transform futures contract data into meaningful time series conducive to long-term analysis.

Bob Pelletier

Tech Talk
Each month in this column, the CSI Technical Support staff addresses topics of interest to many CSI subscribers in a question-and-answer format. This month they present data and time management tips for enhanced database enjoyment.

Q. I update the full Unfair Advantage database on a DSL line and have found the downloads have gotten lengthier as time goes by. Why? Is there anything I can do to speed things up?
A. That depends upon what you consider to be a "lengthy" download and which factors you may have manipulated that affect the update time. At a recent test in our office using the optimal conditions of a T1 line and well equipped computer, it took 1 minute to append to all commodity contracts in the world database, 2 minutes to append new data to all stocks, 1/2 minute to update and add to all options, 1/2 minute to append to all the items in a few modest portfolios and 1 minute to update a substantial market scanner file. In addition, it took 3 minutes to capture adjustments for mutual funds such as dividends, and capital gains, splits and earnings or dividends for stocks, and updates for econometric fundamentals.

When you consider that the UA database supports 12,751 stocks, 15,578 mutual funds, all contracts on 645 world futures markets, about 190 cash markets, 140 Commitments of Traders series, 560 economic fundamentals and 200 Single-Stock Futures, these relatively low updating times are really quite remarkable.

If your updates take longer than seven minutes or so (for everything), consider the following and make adjustments where feasible:

Portfolio Content
Writing and making adjustments to export files for use with other programs can be time consuming. Unfair Advantage automatically processes all export-format files listed in your portfolio(s) with adjustments for splits, dividends, corrections, etc. Depending upon the quantity of markets that require adjustments, this can be a lengthy process. To avoid unnecessary delays, we recommend you periodically check your portfolio(s) to see if you have requested ASCII, MS, CSI or CSM format files that you aren't using. Remove any unnecessary file formats for each market.

MarketScanner™ Use
UA's MarketScanner is a great way to uncover hidden opportunities in the markets and run your own custom studies. Regular MarketScanner users can save time by having UA automatically scan the database each day. However, if you aren't reviewing the MarketScanner results, then running the scan is a waste of your time. Make sure this option is set correctly for your needs through MarketScanner's Scan Settings screen.

Accessing Method & Hardware
If you're still using a phone line and standard modem for data retrieval, please consider upgrading to DSL via your phone system or cable, or satellite. You'll be glad you did. This state-of-the-art software performs best on state-of-the-art equipment.

Time Management
The reality is that the maintenance of UA's massive database can be time consuming. The greater your data needs, the more time it will take. Whether you spend that time cursing the software and wasting your time or doing something productive (or relaxing) is up to you.
Be aware that once per month, mutual fund companies publish dividends and capital gains, so on that one day there is always a longer append step. This month UA reported some 4,000 dividend, earnings and capital gains reports. On quarterly month ends and typically after the 20th of the month, mutual fund downloads are quite a bit slower because the entire history could be adjusted for dividends. End-of-year maintenance after December 20th creates the longest delays because mutual funds tend to even up on just about every adjustment possible.

Unattended Retrieval
Did you know your computer can download updates when you're not anywhere near your computer? There are two ways to initiate an unattended UA retrieval session. Why not use one of them so updates can occur while you're otherwise occupied? One option is to use the AutoSchedule feature on the Communication Options screen. This lets you set your desired time of day (such as 6:00 PM) for regular updates. Make sure UA is running at that time specified, and your data will be updated automatically. An alternative is to use Windows' AutoScheduler to automatically launch UA and initiate a retrieval session. See the UA manual for details.