There is a difference between merely using an optimiser and trading optimally. Many index players would be better off focusing on stratified sampling, rather than worrying about optimizers.


Here we use the term optimiser to mean “an algorithm that seeks the minimum point of active risk subject to given constraints”. The active risk is a function of the weights of all positions in the portfolio and is invariably calculated using a covariance matrix. Optimisers are very sensitive to changes in the covariance matrix. You can get an optimizer to generate quite different trade lists simply by changing the covariance matrix a little bit.

The construction of a covariance matrix is complex. So complex, in fact, that nearly everyone leaves it down to a very few specialist ‘risk model’ providers. They have their proprietary recipes, and all their clients take the (same) data they are given, whether or not the recipe makes sense for the strategy being managed.

Covariance matrix construction can be done using a variety of methods ranging from brute-force to very subtle. You can use short return histories of 60 days or less, or you can use very long histories. You can use an equal weighting of returns, or you can weight recent returns higher than older returns, or you can even ignore some returns. All these variations and more are used in practice, and they all produce different covariance matrices and hence different ‘optimized’ results.

Even if God (or Barra) could hand you the ideal covariance matrix construction method there are still problems.

Covariances (or equivalently, correlations) are not stable. They change over time. A great example from currencies is the CHF vs EUR at the time of the Greek debt crisis. Within a few days, two assets with a 10+ year history of near-perfect correlation developed a significant negative correlation. The reason? An idiosyncratic problem with one asset (the Euro) not replicated by its close substitute (the Swiss Franc) drove many to sell EUR and buy CHF.

Investors should think hard about this. Regarding two assets as substitutes for each other just because of a history of correlation is dangerous. Another such example: Oil and Natural Gas. They were highly correlated until the widespread exploitation of abundant shale gas made the price of the latter decline steadily.

Investors should be fully aware that optimizers take advantage of these observed correlations to produce results. You ask an optimizer to match a 1000 security index with 100 securities, and it will place some big bets on those correlations. When correlations diverge, the user of a long-dated covariance matrix calculated with a long history may be the last to know. If your covariance matrix is based on two-years of daily returns and a correlation that was strong suddenly becomes negative, you will have to wait a year for your covariance matrix to reflect even 50% of the effect of the ‘new regime’.

In practice, we see a lot of skilled investment professionals spending a lot of time on rather mundane or senseless tasks ‘getting the optimizer to work’. These are summed up in these categories:

  • Getting my identifiers to match with those of the optimisers
  • Dealing with securities which are not in the risk model (these are often important)
  • Playing with the constraints to make the optimiser “do what I want it to do”.

The last category is particularly sad: if the portfolio manager knows what the right result looks like, what value is the optimizer adding?

Stratified Sampling

Stratified sampling is far easier to understand and implement than covariance-based optimization. It works by slicing the benchmark and fund up into segments and insisting that invested weight of fund and benchmark match each other, in aggregate, over each segment. This is the ‘stratified sampling rule’. Like optimization, it is best to consider several factors simultaneously. 

For example, in an international equity portfolio, segment the portfolio by

  • country,
  • industry,
  • size of the company,

and run an algorithm to bring the portfolio into obeyance of the stratified sampling rule, along each of these dimensions simultaneously.

This type of algorithm produces very satisfactory results in practice, and overcomes many of the objections to optimization:

  • The data required is just the data you have – you don’t need (to pay for) a risk model.
  • The multi-factor dimensions follow those used in designing the strategy, rather than those set by the risk-model provider.
  • It is not driven by observed correlations.
  • It can be implemented by inexpensive software which is not an external process and does not require interfacing and identifier mapping.



Ryedale publications do not offer investment or trade advice or make recommendations to use any particular investment strategy. Should you undertake any such activity on the basis of information contained in Ryedale publications, you do so entirely at your own risk, and Ryedale shall not be held responsible for any loss, damage, costs or expenses incurred by you as a result.  

The information we publish has been derived from or is based on sources that we consider to be accurate and primary. Although reasonable care has been taken, we cannot guarantee the accuracy or completeness of any information we publish. You should always carry out your own independent verification of information and seek a professional advisor before making any investment decisions.   

Jul 08, 2020

London, United Kingdom

About Ryedale

Ryedale delivers a modern, cloud-based, full-service IT platform for serious investment businesses. Its software provides industry-leading support for model-driven and quantitative investment strategies, embracing index investments, smart-beta, custom indices, asset allocation, overlays and hedging. Ryedale's customers include several globally recognized asset management firms and institutions. For more information about Ryedale investment management solutions, please visit