Demo: Diagnostic Test of Returns (StockEval)

(Posted December 1, 2001)

The Trading Rules of StockEval have been developed using the Diagnostic Test to evaluate the results of applying the Trading Rules over a two year (500 day) interval.  The daily stock data are downloaded each night after market hours, then the recommended positions are calculated.  In the Diagnostic Test, it is assumed that market orders are placed the next morning, before the market opens, so that they go through at the opening price.  The market orders are set to adjust the position in the stock to the recommended N-day Position (or Constant Position).  The positions are changed in multiples of a round lot (100 shares), and commissions are not taken into account.  Also, the buy/sell stops are not taken into account.

The method of adjusting your position every day to the recommended position is probably the best method if your portfolio is large enough.  This is the method that will keep the variance of the returns, and hence the risk, to a minimum.  However, if your portfolio is not that large, or you do not want to trade every day, then this method can be approximated by adjusting the positions at the approximate buy/sell points.  These points are not completely precise due to the short-term random volatility.  However, they will indicate the approximate points at which the trades are most profitable, leading to the optimal trading strategy for N-day trading.  This will be your best strategy where commission costs are an important factor, or where each position consists of only a few round lots.

The Diagnostic Test shows that the Trading Rules yield excellent results when used as a mechanical trading system.  However, we do not recommend that you use them in this manner.  Instead, it is always better to use the indicators and position recommendations of StockEval in conjunction with all other available information, such as market sentiment and the business situation of the company in question and the overall state of the economy.  You should always make the final trading decisions yourself, using your own judgment and all the information and analysis at your disposal.

The Diagnostic Test uses only past data for each day of the calculation.  No future data are used, as far as we are aware.  We have done our best to try to plug any future leaks that might exist in the Diagnostic Test.  However, it is best to interpret the Diagnostic Test as a practical test for evaluating the Trading Rules, not as a rigorous test that would re-enact history exactly.  But the Diagnostic Test is a reasonable and very stringent test of the Trading Rules, and should be interpreted as such.

Results of the Diagnostic Test

Listed in the table are the results of running the Diagnostic Test on 14 of the 20 stocks in the Sample portfolio included with StockEval.  These are the 14 stocks in the Sample folder that use 500 days of data for the Diagnostic Test.  To provide a fair test, the files that use less than 500 days of data for the test have been omitted (see below).  The Market Orders test was run for time scales of 2, 5, 10, 20, and 40 days, and compared to the result for the Constant Position (Trading Parameters on 0%) and Buy & Hold returns, and the three Trading Parameter settings were the maximum (100%) settings (see below).  The numbers in the table are all simple returns, and are the average annualized simple returns for the given time period of the trading rules (500 days), compared with the annualized simple buy-and-hold return for the same time period, in the last column.  It is assumed that a set of trading rules is used in which market orders are placed each morning, and the trades take place at the opening price for the day.  This is a trading strategy that is realistic and easy to maintain.

The Buy & Hold (simple) return is calculated by starting with the ending price over the 500-day interval divided by the starting price.  This gives the "compounded" return over the interval.  This may be converted to an average daily price ratio by taking this ratio to the 1/500 power.  Starting with this average daily price ratio, we may instead convert it to a (percentage) daily simple return by subtracting one and multiplying by 100%.  When this daily return is multiplied by 256, it gives the average annualized simple return.  This return may be compared with the return due to maintaining a constant position, which is almost always higher.  Evidently there is an actual gain in buying when the price goes down and selling when the price goes up, in order to maintain the constant position, and it is not just due to a different method of calculation.  Sometimes the return from the constant position is positive while the average annual simple Buy & Hold return is negative.  If the difference were due solely to different methods of calculation, they would both be either positive or negative.  The Constant Position corresponds in trading simply to maintaining the Core Position, as determined by the slope of the robust long-term Regression line, together with the correction that you add in using the Adjust Expected Return dialog box.  For many traders and investors, simply maintaining a constant (in dollars) Core Position by trading every day, might be a viable strategy.

Here are the results of the Diagnostic Test for StockEval, run on the Sample files (pre-bear market) with at least 1000 days of price data (the full 500 days of data for the Diagnostic Test).  The Buy & Hold and Constant Position returns are at a margin percentage of 100%, while the N-day returns are at a variable margin percentage:

Stock

2-Day

5-Day

10-Day

20-Day

40-Day

Const.

B & H

SUNW

106.42 %  

165.55 %  

131.85 %  

121.72 %  

119.31 %  

140.26 %   

123.57 %  

YHOO

63.83 %  

69.85 %  

221.94 %  

223.58 %  

287.68 %  

126.29 %   

87.42 %  

AOL

166.47 %  

28.34 %  

158.67 %  

164.91 %  

67.22 %  

101.84 %   

73.35 %  

CSCO

141.02 %  

142.81 %  

171.42 %  

180.10 %  

135.87 %  

83.27 %   

69.03 %  

INTC

169.38 %  

154.93 %  

210.74 %  

107.81 %  

95.27 %  

76.39 %   

64.94 %  

AAPL

34.25 %  

74.44 %  

165.30 %  

154.05 %  

150.98 %  

67.90 %   

50.27 %  

MOT

57.46 %  

29.43 %  

33.27 %  

95.86 %  

3.78 %  

58.56 %   

44.54 %  

IBM

136.42 %  

115.16 %  

81.04 %  

37.25 %  

74.30 %  

41.41 %    

33.82  %  

MER

25.97 %  

15.82 %  

55.83 %  

-11.65 %  

-9.58 %  

48.01 %   

31.56 %  

LUV

34.59 %  

90.69 %  

35.87 %  

50.51 %  

37.46 %  

41.86 %   

29.75 %  

MSFT

-14.20 %  

35.18 %  

35.37 %  

5.53 %  

50.67 %  

23.01 %   

13.43 %  

DELL

43.65 %  

-34.32 %  

80.26 %  

84.94 %  

120.30 %  

29.44 %   

10.99 %  

MRK

111.00 %  

106.09 %  

79.95 %  

59.38 %  

31.99 %  

14.25 %   

7.21 %  

LU

38.48 %  

31.07 %  

65.27 %  

80.72 %  

92.09 %  

19.38 %   

2.16 %  

It will be noticed that the returns for some stocks vary drastically on different time scales.  The N-day Trading Rules function as a type of oscillator indicator that detects overbought and oversold conditions, that is most sensitive to cycles with a time period of 2N days.  It could be that for some stocks, there are pronounced cycles with well-defined periods that the Trading Rules are picking up.  But for the lower time scales, it should also be expected that there will be increased variance of the returns, due to the increased rapidity of the trading.

In the case of Merrill Lynch (MER), there was a sudden V-shaped decline in September and October of 1998.  The Diagnostic Test does not take into account buy/sell stops.  Probably if the position had been stopped out soon after the rapid decline started, the final results would be much better.  Nevertheless, this illustrates that a sudden change of trend can confuse the Trading Rules, which work on the basis of correlation in the price data.  Be sure to set tight buy/sell stops to protect yourself from such sudden trend changes.

You can do your own statistical analysis of the results.  However, it is clear just from a glance that the Trading Rules of StockEval yield a highly significant statistical advantage.  But due to the considerable variance of the returns from short-term trading, it is recommended that you do your trading within a diversified portfolio to minimize risk.

Gains from Short-Term Trading

There are actually two different sources of gains (over the Buy & Hold return) from trading. The gains come from the Trading Rules and from maintaining a Constant Position:

  1. Trading Rules: You can measure the gain due to the short-term Trading Rules, relative to the Buy & Hold return, by varying the three sliders that control the Trading Parameters, and the Time Scale. When the sliders are set to zero, this yields the gain due to the Constant Position. You can measure the results due to the Market Orders, Day Limit Orders, or GTC Limit Orders strategies. All three of these strategies result in increased variance of the returns, or risk. Hence there will be a distribution of results about the average due to these short-term strategies, due solely to random chance. It will be found that the short-term trading rules result in gains for the vast majority of stocks, although for a few stocks it will result in a loss. This is due simply to the increased variance of the returns due to the rapid short-term change in the position.
  2. Constant Position: You can measure this gain by running any of the tests with all three of the sliders set to zero. In that case, the Diagnostic Test attempts to maintain a constant dollar amount invested in the stock. When the price goes down, stock is bought, and when the price goes up, stock is sold. This results in a net gain over Buy & Hold due to buying on the short-term dips and selling on the short-term peaks. This is almost the same as the well-known Dollar-Cost Averaging strategy, except that you are maintaining a constant dollar amount of the investment, not adding a constant amount periodically.

The overall returns will be the return due to the Buy & Hold strategy, plus a combination of the gains over Buy & Hold listed above. Both of these components are important. But both sources of gains over Buy & Hold depend on varying the position smoothly from day to day. If you vary the position suddenly and radically in response to any kind of trading signals (with the exception of the sell stops and buy stops, of course), then you will end up with reduced returns and greatly increased risk.

Trading Methods

There are four different types of trading methods that you can test:

  1. Market Orders: This is the best way to trade, with the highest returns and the least risk. It is also a very simple way to trade. You simply place a market order every day, adjusting the position to that recommended by StockEval. In the Diagnostic Test, the position is adjusted at the opening price for the day, corresponding to placing a market order before the open each morning. However, it does not seem to matter very much at what point in the day you adjust the position with a market order. This type of trading results in the greatest returns with the least risk, because the position is varied smoothly from one day to the next.
  2. Day Limit Orders: In this type of trading, one buy and one sell day limit order is placed, at the expected volatility limits for the day as calculated by StockEval. This calculation is done using the Linear Prediction routine on the daily average prices and the volatilities.  You can still select the Time Scale for the smoothing of the trading rules, just as in the Market Orders test.  This test can then be compared with the Market Orders test to determine the effectiveness of the estimates of daily average price and volatility in the Short-Term Trades dialog box.
  3. GTC Limit Orders: In this type of trading, when the calculated buy/sell points for N-day trading appear within N days of the present trading day, a GTC limit order is placed (one buy and one sell order at a time). The order is maintained until it is either filled or replaced by a new buy/sell point a further time in the future. This type of trading leads to significant gains, but is less effective than the two daily trading methods.
  4. Random Numbers: In this (hypothetical) trading method, the daily position is varied at random around the average position (100%). The variation of the position is changed by changing the three Parameters sliders, and is proportional to the sum of the sliders. At their maximum settings, the position varies at random between -50% and +250%. This variation of the position is more extreme than the other trading methods, and as a result the variance of the returns is expected to be greater. Since the random numbers are different each time the test is run, you can see the variance of the returns by repeatedly running this test. The average return (from repeatedly running the test) should equal the Constant Position return (which is what you see when all three sliders are set to zero). 

Parameters

There are four parameters you may set each time you run the Diagnostic Test, in order to determine their effect on the outcome. The Time Scale is in an edit list box, and there are three sliders to set the Trading Parameters. These four controls are the same ones that are in the Trading and Portfolio Parameters dialog on the Main Toolbar. When you first open the Diagnostic Test dialog, these four parameters are pre-set to their values that were set in the Trading and Portfolio Parameters dialog. You may then vary these parameters any way you want to run the Diagnostic Test.

  1. Trading Time Scale: This sets the time scale for N-day trading, at convenient intervals from 2 through 40 trading days. When testing the N-day Trading Rules, this sets the time scale of the smoothed prices used to calculate the recommended position. The longer the time scale, the more slowly the recommended position changes.
  2. Relative Price: This sets the component of the Trading Rules that is derived from the Relative Price indicator in the Harmonic Oscillator. This component corresponds to the Buy Low--Sell High trading strategy. It is the least aggressive of the three components, and it works best for the longer time scales. This indicator measures the expected N-day price move.
  3. Velocity: This sets the component of the Trading Rules that is derived from the Velocity indicator in the Harmonic Oscillator. This component corresponds to the Buy High--Sell Higher, or Momentum trading strategy. It is of intermediate aggressiveness, and it works well for all time scales. This indicator directly measures the expected return over the immediate N-day time interval.
  4. Acceleration: This sets the component of the Trading Rules that is derived from the Acceleration indicator in the Harmonic Oscillator. This component corresponds to a "Calling Tops and Bottoms" trading strategy. It is the most aggressive of the three components, and it works best for the shorter time scales. This indicator measures the rate at which the price trend is turning upward or downward.

Note Concerning Short Data Sets

You might get erratic results from the Diagnostic Test if you are running it on a stock with a very short data set. The data set for most stocks will be 1000 days, so the Diagnostic Test will run for 500 days. However, if the data set is less than, say, 500 days, the Diagnostic Test will run for less than 250 days. This is not enough time to reliably compute the long-term robust Regression Line, and in addition there is not enough time for the short-term volatility to be averaged out. As a result, for short data sets the Diagnostic Test will depend greatly on the short-term volatility, and will not be a valid long-term test of the trading rules.

return to Demonstrations page