Portfolio Optimization in QuanTek

(Revised January 30, 2014)


Robert Murray, Ph.D.

Omicron Research Institute


(Copyright Ó 2014 Omicron Research Institute.  All rights reserved.)


4.0           Contents

  1. Need for Portfolio Optimization
  2. Outline of the Optimization Problem
  3. Measures of Portfolio Performance
  4. Classical Markowitz Optimization Problem
  5. Modified (Unconstrained) Optimization
  6. Diagonalizing the Covariance Matrix
  7. Standard Markowitz Cost Function
  8. Another Reasonable Cost Function
  9. Conclusion
  10. References

4.1    Need for Portfolio Optimization

            The goal of the QuanTek program is to maximize returns and minimize risk in the portfolio. This is accomplished by a modification of the classical Markowitz method of portfolio optimization [SAB] This method uses quantities that are estimated or measured from the securities in the portfolio, namely the expected future return and the standard deviation or risk. The expected future return is a quantity that must be estimated from some kind of Price Projection, which in QuanTek is provided by means of a Linear Prediction filter. The standard deviation is measured, in the present version of QuanTek, by taking a long-term average of the average absolute deviation and multiplying by a certain factor to convert it (assuming the Gaussian distribution) into the equivalent standard deviation. In a future version of QuanTek, we hope to utilize the Linear Prediction method to estimate a time-varying future standard deviation, implementing a form of a GARCH (Generalized AutoRegressive Conditional Heteroskedasticity) [G] model.

            In QuanTek the portfolio is optimized for a given trading time horizon N, and the future returns are estimated for each security for this time horizon. As just stated, the standard deviation of each security is also measured, taking a long-term average. A third quantity that is measured is the correlation of the returns between all the securities, forming a (symmetric) correlation matrix. From the standard deviation, we take its square to obtain the variance. Then from the variance and the correlation matrix, we multiply the correlation matrix by the variance to obtain the covariance matrix. The quantities in the portfolio optimization calculation, for a portfolio of M securities, are the M expected future returns  and the M by M covariance matrix  (where the indices run from 1 to M).

4.2    Outline of the Optimization Problem

The portfolio optimization problem consists of calculating the fraction  of each security in the portfolio that results in the optimal portfolio, maximizing returns and minimizing risk. The fraction  can be positive, corresponding to a long position, or negative, corresponding to a short position. The sum of the absolute values of these fractions must add up to unity, for a portfolio with M securities:


Actually, we may relax this restriction below and let the sum of the positions (long or short) add up to a number slightly less than unity, for the sake of computational convenience. This corresponds to being not quite 100% invested. This will happen if the quantities  vary significantly from each other. This is a good thing and should actually help reduce risk in the portfolio when the equity is not evenly distributed.

            Given the fraction of each security in the portfolio, the portfolio expected return  is then the average over all the expected returns  of the securities in the portfolio:


 On the other hand, the total variance of the portfolio is given in terms of the covariance matrix  by the following formula:


From this formula we can see that the portfolio variance becomes smaller as the number of securities in the portfolio becomes larger. For example, if the variance of each security  is the same and all the securities are uncorrelated, and the fraction of each security is also the same so that , then the portfolio variance becomes:


Hence we see that the total portfolio variance has been reduced by a factor M compared to the variance of each individual security.

4.3    Measures of Portfolio Performance

Starting with the daily returns, the average N-day return may be estimated by taking a sum over N days of the daily returns. So the N-day (logarithmic) return grows proportionally to N. On the other hand, at least in the Random Walk Model, the daily variance also adds (because the returns are uncorrelated), so the average N-day variance is also estimated by taking a sum over N days of the daily variance. This means that the standard deviation of estimated future returns grows like the square root of N.

One measure of performance of a portfolio is the ratio of annual returns  to annual standard deviation , called the Sharpe ratio [SAB, FFK, G]:


However, the problem with this quantity is that it depends on the time interval over which the average is taken, since  grows proportionally to time while  grows like the square root of time. So I would like to propose a new quantity that I call the Quality ratio or Q-ratio:


This is the ratio of the average or expected return  to the variance , both of which grow proportionally to time (at least approximately). Not only is this quality measure relatively time-invariant, but it also further reduces the quality rating of those extremely risky securities with very high volatility, such as penny stocks. In my opinion, the Sharpe ratio tends to over-rate the quality of these types of securities. The Q-ratio will then be applied not only to the portfolio as a whole, but also to individual securities. Notice that, as before, for a portfolio of M securities with the same returns and variance, the Q-ratio of the whole portfolio will be multiplied by M relative to that of each security, due to the reduction of the variance by a factor of M (with the average return remaining the same).

4.4    Classical Markowitz Optimization Problem

There are, no doubt, many distinct approaches to solving the portfolio optimization problem. For one thing, these depend on what investors (on the average) consider to be the optimum solution. Almost all investors and traders wish to maximize returns and minimize risk. However, different investors and traders differ in their degree of risk aversion, which is the amount of return they are willing to sacrifice in order to reduce risk, or alternatively their risk tolerance, which is the amount of risk they are willing to accept for a given expected return. So it is somewhat arbitrary how one defines the exact trade-off between risk and return – or you could say it is a matter of understanding and anticipating the behavior of market participants.

The classical Markowitz solution [SAB, G] to the problem may be framed in terms of a cost function. The cost function is taken to be a simple combination of the portfolio variance and expected return – the difference of the two with a constant  representing the risk aversion:


When , the investor cares only about returns and not at all about risk. When , the greater the variance the lower the value of the cost function, so the variance counts against the cost function. If , then this means the investor is actually in favor of more variance or risk, so the investor actually has a risk appetite. Very few investors fall into this category, unless they are real risk-takers and don’t mind losing a lot of money!

            The optimization problem is solved by taking the partial derivative of  with respect to each  and setting this equal to zero, meaning that the cost function  is maximized. Then this equation may be solved for the fraction of each security . Unfortunately, this procedure is complicated by the constraint that all the positions  must add up to unity. So the problem reduces to one of constrained quadratic optimization [SAB]. Markowitz himself presented a solution to this problem, in 1956, but in general it needs to be solved numerically. (But also see the book by Gourieroux [G].) Another problem is that the covariance matrix  is difficult to compute if there are more than a few securities, and moreover it has been shown that this matrix is mostly stochastic noise anyway [LCBP]. So we are going to find an alternative method that is better for the purposes at hand and easier and more reliable to compute.

4.5    Modified (Unconstrained) Optimization

It is possible to modify the constrained optimization described above by introducing new variables such that the cost function is optimized in terms of these new, unconstrained variables. The variables  are constrained such that the  all add up to unity, as stated above. Let us define a new variable in terms of the old variables, in which the old variables are divided by a norm function, in such a way that the sum of the absolute values of the new variables is automatically less than or equal to unity. We define, for a portfolio with M securities:


Notice in this definition that we have lost one degree of freedom, because if all the  variables are multiplied by an arbitrary (positive) constant, the modified variable  remains unchanged. This is equivalent to the constraint imposed on the  variables. Then notice that if all the  variables are equal (ignoring the constraint, so they can be multiplied by any arbitrary positive constant) then we get , and then the sum of the  is again unity. However, it should not be too hard to show that if the  variables are not all equal, the sum of the  is less than unity. Also, if we try to make the  variable as large as possible, by making  much larger than all the others, we find that . So we arrive at the modified constraint on the absolute values of the  variables, which is implicit in the above definition:


We also find the following value for the norm of the vector :


This verifies the inequality  for each individual component . This is actually a good definition for a mix of securities in a portfolio, because it helps ensure that the mix is balanced. Just as in the case of a mutual fund, in which the maximum equity that can be allocated to a single security is something like 5%, in the present case the maximum that can be allocated is , which for 25 securities (a rather large portfolio) will be 20%.

            Now, to optimize the portfolio, we define a new cost function as , and optimize with respect to the original variables. This is done by taking M partial derivatives as before, and setting the results to zero to specify the maximum of the cost function:


To carry out this calculation, we may first calculate the partial derivatives of the  variables with respect to the  variables. Thus we have:


This quantity will be used in every optimization problem, no matter what the particular cost function is.

Let us see how this works in the example of the totally risk-tolerant investor with  in the example above. Then we have for the above cost function, in terms of the  variables:


The optimization condition becomes:


Since  is always positive, this then leads to the following condition on the  variables in terms of the expected returns :


This is a set of M equations, one equation for each value of the index k from 1 to M. The quantity in brackets is a scalar quantity that is the same for each component k. So we can easily conclude that the vector  is proportional to the vector . Setting the proportionality factor equal to K, we can compute:


Thus we find the solution for the  variables in terms of the returns :


So the result is that the  variables are the same functions of the returns  that they are of the  variables. In other words, the optimal mix of securities in the portfolio with no risk aversion at all is that the fractions of the equity  are simply proportional to the expected returns.

4.6    Diagonalizing the Covariance Matrix

Now let us consider the term in the cost function involving the covariance matrix, which is a measure of the risk of the portfolio. The derivative of this function takes the form:


Thus we have the following derivatives:


Both of the terms involve the covariance matrix , which is a positive definite symmetric matrix. Any such matrix may be diagonalized by means of an orthogonal matrix , so that afterward the covariance matrix  will have only (positive) diagonal elements, which are the eigenvalues corresponding to variances, and the off-diagonal elements corresponding to covariances will be zero:


This orthogonal transformation is a change of basis in the M-dimensional vector space, to a basis consisting of the eigenvectors of the covariance matrix . The vectors  and  likewise transform under this change of basis, to vectors  and . Then, in the diagonal basis, each of the M equations can be solved independently since there will be no mixing between the different components of the vectors.

            However, at this point we are going to make an approximation. We could go ahead and make the orthogonal transformation to the diagonal basis and solve the equations, then transform back at the end to the original basis. The original basis consists of a separate security for each component of the vectors, while the diagonal basis consists of some linear sum of securities for each component. But for the present purpose I view this as unnecessarily complicated, and so the approximation will be made of simply ignoring the correlations between securities and assuming the covariance matrix is approximately diagonal in the original basis. This should make only a small difference in the portfolio optimization problem, which should not be very significant. In order to ensure that this approximation is insignificant, investors and traders should select securities for the portfolio that are not very highly correlated. If a broad selection of securities in different sectors is chosen, so that there are no strong correlations between the securities, then this is the essence of good diversification in the portfolio [FFK].

            Thus, assuming the correlations between securities are approximately zero, so they are uncorrelated and the covariance matrix is approximately diagonal, we may express the above equation solely in terms of the diagonal components:


The term in brackets is the total portfolio variance, expressed in terms of the fractions , which as we recall add up to slightly less than unity in general. We may define this variance as follows:


Then the above equation is rewritten as follows:


So apart from this total variance, which is a constant, the k’th equation above depends only on the variance  and the fraction  of the k’th security in the portfolio.

4.7    Standard Markowitz Cost Function

If we go back to the original Markowitz cost function, this may be expressed in terms of the new variables as:


Then the optimization equation takes the form:


We once again use the following relations, derived earlier:


Substituting in these previously derived expressions for the terms, we get:


To solve this, it is clear that the equation is satisfied whenever the term in brackets is zero. Also, from the value for the norm of the vector  given previously, it can be seen that the quantity in parentheses on the right is actually a projection operator onto the hyperplane perpendicular to , or in other words, any vector proportional to  summed with this quantity is zero. Thus in general we may write, for any arbitrary constant K, that the terms in brackets are equal to:


However, the constant K is not arbitrary, because the norm constraint on  must also be satisfied. Thus to determine K, we find by multiplying the terms in the above equation that:


This equation fixes a particular value of K, which is then given by:


With , this reduces to the same equation as before, but now in the present case the equation is not so easy to solve for K. However, we may resort to another approximation. If we approximate that all the variances are equal, then we have:


Then we may write:


Then we have:


With this value of K we then have for the normalized weight vector:


It should be noted that the (approximate) portfolio variance  itself depends on the values of the weight vector , which is what we are trying to solve for. Hence to use this formula, we must approximate  beforehand. A proposal for how to do this is mentioned in the next section. Alternatively, we could just modify the solution for the weight vectors by setting the term involving  in the final formula to zero. This would ensure that the denominator is always positive.

So for  we arrive at the same result as before, and for  not too large the weights are reduced as the individual variances  are increased. If  is large and the variances are very uneven, there is the possibility of running into a zero denominator, indicating no solution to the optimal portfolio, so this limits the possible values of . (Of course, we also assume , corresponding to risk averse or risk neutral investors.) If all the variances are equal, then the approximation becomes exact and the term proportional to  is zero, and the result is the same expression we found previously for zero risk aversion.

4.8    Another Reasonable Cost Function

Instead of the Markowitz cost function, which is a weighted difference of the cost associated with the returns and risk, we can consider another cost function that is more closely related to the Q-ratio defined previously. Thus we define the following cost function:


The optimization condition then takes the form:


Again we make use of the following relations, derived earlier:


This then leads to the following result:


This is nearly the same result as before, except for an additional ratio of  to  multiplying the risk aversion constant . But these two quantities are the very ones we are trying to determine by the optimization procedure. They are just the portfolio return  and portfolio variance , respectively, and their ratio was previously defined as the Q-ratio. We may initially approximate these quantities by taking the weight vectors to be all the same, . Then we define a new risk parameter  as a renormalized version of the old one:


Then in terms of this modified risk parameter, the (approximate) solution is the same as the previous one:


For added safety in this equation, to ensure against the possibility of a zero denominator, we may simply drop the  term and write:


This will make the sum of the  a little smaller, and each  a little less dependent on . Alternatively, the average value  or some fraction of it can be used. The parameter  is computed as indicated using average values of  and , and with the original exponent  defined to be in the range . For  this just corresponds to the cost function taken as the Q-ratio (normal); for  the cost function is just the Sharpe ratio (riskier); for  the cost function is just the portfolio return alone (riskiest); and for  the portfolio variance is given extra weight (safest).

4.9    Conclusion

The portfolio optimization calculation is important in order to maximize returns and minimize risk for the portfolio as a whole. It should be emphasized that only with a well-balanced portfolio is it reasonable to expect to achieve an acceptably low level of risk. If stocks or other securities are traded individually, most likely the outcome will be a drastic increase in risk and very little increase in return. It is necessary to have the balanced portfolio so that the random fluctuations in the different securities will balance each other and average out to yield a reasonable degree of risk reduction. Then, the idea is that instead of trading in each security separately, trading is done within the portfolio as a whole by the act of portfolio rebalancing.

            Another point should be stressed. The problem of how to divide the equity in the portfolio evenly among the various securities is also non-trivial. With each security trading at a different price, and changing every day, even dividing the equity evenly among all the securities becomes a non-trivial computational exercise. In the portfolio optimization calculation, after computing the optimum fraction of the equity to invest in each security, the routine then goes on to divide by the price per share to obtain the optimal number of shares at any given time. Also, even if the equity were divided equally, the fact that the prices per share are continually changing means that the correct number of shares to own also changes continually. The optimal portfolio calculation computes not only the optimal fraction for each security, but also the number of shares based on the current price, so that the total equity invested in all the shares remains at a constant value. This is important in order to remain fully invested at all times, without running the risk of margin calls or wild swings in the total market value of the shares. So the optimal portfolio calculation actually fulfills two roles, one of optimizing the return/risk ratio, and the other of dividing the equity according to the desired proportions with security prices that are constantly changing.

4.10  References

 [BP]    Jean-Philippe Bouchaud & Marc Potters,

Theory of Financial Risks: From Statistical Physics to Risk Management, 2nd ed.,

            Cambridge University Press, Cambridge, UK (2000)   


[FFK]  Frank J. Fabozzi, Sergio M. Focardi, & Petter N. Kolm,

            Financial Modeling of the Equity Market: From CAPM to Cointegration,

            John Wiley & Sons, Hoboken, New Jersey (2006)


[G]       Christian Gourieroux, ARCH Models and Financial Applications,

            Springer-Verlag, New York (1997)


[LCBP]Laurent Laloux, Pierre Cizeau, Jean-Philippe Bouchaud, & Marc Potters,

            “Noise Dressing of Financial Correlation Matrices”,

            Physical Review Letters, Vol.83, No.7 (16 August 1999)


[SAB]  W.F. Sharpe, G.J. Alexander, & J.V. Bailey, Investments, 5th ed.,

Prentice Hall, Englewood Cliffs, NJ  (1995)