Demo: Diagnostic Test of Returns (StockEval)
(Posted December 1, 2001)
The Trading Rules of StockEval have been developed using the Diagnostic Test to evaluate the results of applying the Trading Rules over a two year (500 day) interval. The daily stock data are downloaded each night after market hours, then the recommended positions are calculated. In the Diagnostic Test, it is assumed that market orders are placed the next morning, before the market opens, so that they go through at the opening price. The market orders are set to adjust the position in the stock to the recommended N-day Position (or Constant Position). The positions are changed in multiples of a round lot (100 shares), and commissions are not taken into account. Also, the buy/sell stops are not taken into account.
The method of adjusting your position every day to the recommended position is probably the best method if your portfolio is large enough. This is the method that will keep the variance of the returns, and hence the risk, to a minimum. However, if your portfolio is not that large, or you do not want to trade every day, then this method can be approximated by adjusting the positions at the approximate buy/sell points. These points are not completely precise due to the short-term random volatility. However, they will indicate the approximate points at which the trades are most profitable, leading to the optimal trading strategy for N-day trading. This will be your best strategy where commission costs are an important factor, or where each position consists of only a few round lots.
The Diagnostic Test shows that the Trading Rules yield excellent results when used as a mechanical trading system. However, we do not recommend that you use them in this manner. Instead, it is always better to use the indicators and position recommendations of StockEval in conjunction with all other available information, such as market sentiment and the business situation of the company in question and the overall state of the economy. You should always make the final trading decisions yourself, using your own judgment and all the information and analysis at your disposal.
The Diagnostic Test uses only past data for each day of the calculation. No future data are used, as far as we are aware. We have done our best to try to plug any future leaks that might exist in the Diagnostic Test. However, it is best to interpret the Diagnostic Test as a practical test for evaluating the Trading Rules, not as a rigorous test that would re-enact history exactly. But the Diagnostic Test is a reasonable and very stringent test of the Trading Rules, and should be interpreted as such.
Results of the Diagnostic Test
Listed in the table are the results of running the Diagnostic Test on 14 of the 20 stocks in the Sample portfolio included with StockEval. These are the 14 stocks in the Sample folder that use 500 days of data for the Diagnostic Test. To provide a fair test, the files that use less than 500 days of data for the test have been omitted (see below). The Market Orders test was run for time scales of 2, 5, 10, 20, and 40 days, and compared to the result for the Constant Position (Trading Parameters on 0%) and Buy & Hold returns, and the three Trading Parameter settings were the maximum (100%) settings (see below). The numbers in the table are all simple returns, and are the average annualized simple returns for the given time period of the trading rules (500 days), compared with the annualized simple buy-and-hold return for the same time period, in the last column. It is assumed that a set of trading rules is used in which market orders are placed each morning, and the trades take place at the opening price for the day. This is a trading strategy that is realistic and easy to maintain.
The Buy & Hold (simple) return is calculated by starting with the ending price over the 500-day interval divided by the starting price. This gives the "compounded" return over the interval. This may be converted to an average daily price ratio by taking this ratio to the 1/500 power. Starting with this average daily price ratio, we may instead convert it to a (percentage) daily simple return by subtracting one and multiplying by 100%. When this daily return is multiplied by 256, it gives the average annualized simple return. This return may be compared with the return due to maintaining a constant position, which is almost always higher. Evidently there is an actual gain in buying when the price goes down and selling when the price goes up, in order to maintain the constant position, and it is not just due to a different method of calculation. Sometimes the return from the constant position is positive while the average annual simple Buy & Hold return is negative. If the difference were due solely to different methods of calculation, they would both be either positive or negative. The Constant Position corresponds in trading simply to maintaining the Core Position, as determined by the slope of the robust long-term Regression line, together with the correction that you add in using the Adjust Expected Return dialog box. For many traders and investors, simply maintaining a constant (in dollars) Core Position by trading every day, might be a viable strategy.
Here are the results of the Diagnostic Test for StockEval, run on the Sample files (pre-bear market) with at least 1000 days of price data (the full 500 days of data for the Diagnostic Test). The Buy & Hold and Constant Position returns are at a margin percentage of 100%, while the N-day returns are at a variable margin percentage:
It will be noticed that the returns for some stocks vary drastically on different time scales. The N-day Trading Rules function as a type of oscillator indicator that detects overbought and oversold conditions, that is most sensitive to cycles with a time period of 2N days. It could be that for some stocks, there are pronounced cycles with well-defined periods that the Trading Rules are picking up. But for the lower time scales, it should also be expected that there will be increased variance of the returns, due to the increased rapidity of the trading.
In the case of Merrill Lynch (MER), there was a sudden V-shaped decline in September and October of 1998. The Diagnostic Test does not take into account buy/sell stops. Probably if the position had been stopped out soon after the rapid decline started, the final results would be much better. Nevertheless, this illustrates that a sudden change of trend can confuse the Trading Rules, which work on the basis of correlation in the price data. Be sure to set tight buy/sell stops to protect yourself from such sudden trend changes.
You can do your own statistical analysis of the results. However, it is clear just from a glance that the Trading Rules of StockEval yield a highly significant statistical advantage. But due to the considerable variance of the returns from short-term trading, it is recommended that you do your trading within a diversified portfolio to minimize risk.
Gains from Short-Term Trading
There are actually two different sources of gains (over the Buy & Hold return) from trading. The gains come from the Trading Rules and from maintaining a Constant Position:
The overall returns will be the return due to the Buy & Hold strategy, plus a combination of the gains over Buy & Hold listed above. Both of these components are important. But both sources of gains over Buy & Hold depend on varying the position smoothly from day to day. If you vary the position suddenly and radically in response to any kind of trading signals (with the exception of the sell stops and buy stops, of course), then you will end up with reduced returns and greatly increased risk.
There are four different types of trading methods that you can test:
There are four parameters you may set each time you run the Diagnostic Test, in order to determine their effect on the outcome. The Time Scale is in an edit list box, and there are three sliders to set the Trading Parameters. These four controls are the same ones that are in the Trading and Portfolio Parameters dialog on the Main Toolbar. When you first open the Diagnostic Test dialog, these four parameters are pre-set to their values that were set in the Trading and Portfolio Parameters dialog. You may then vary these parameters any way you want to run the Diagnostic Test.
Note Concerning Short Data Sets
You might get erratic results from the Diagnostic Test if you are running it on a stock with a very short data set. The data set for most stocks will be 1000 days, so the
Diagnostic Test will run for 500 days. However, if the data set is less than, say, 500 days, the
Diagnostic Test will run for less than 250 days. This is not enough time to reliably compute the long-term
robust Regression Line, and in addition there is not enough time for the short-term volatility to be averaged out. As a result, for short data sets the
Diagnostic Test will depend greatly on the short-term volatility, and will not be a valid long-term test of the trading rules.
As always, "Past performance is no guarantee of future results."
Please read and understand the Disclaimer.
Go back to the Home Page.
Last modified 10/10/2006 .
Copyright 2014 Omicron Research Institute. All Rights Reserved.