Scientific Beta

There is significant evidence that systematic equity investment strategies (so-called smart beta) outperform cap-weighted benchmarks in the long run. However, it is important to recognise that performance analysis is typically conducted using backtests that apply the methodology to historical returns. Concerning actual investment decisions, it is therefore important to question how robust any outperformance is. The paper makes a distinction between relative robustness and absolute robustness.

A strategy is assumed to be ‘relatively robust’ if it is able to deliver similar outperformance under similar market conditions by aligning well with the performance of underlying factor exposure it is seeking and reducing unrewarded risks.
‘Absolute robustness’ is the absence of pronounced state and/or time dependencies. A strategy shown to outperform irrespective of prevailing market conditions can be termed as robust in absolute terms.

Assessing the robustness of smart beta strategies should play a central role for investors in their due diligence process. Such strategies often experience an out-of-sample degradation of their performance compared to that presented in the historical in-sample period. Investors should always check that interesting in-sample results are complemented by a consistent construction framework and transparency on the methodology and implementation from the side of the strategy provider. They should also be able to measure the robustness directly using appropriate tools and metrics.

This paper discusses why robustness is relevant for investors in smart beta strategies and describes the sources of deficiencies. It further explains the need for robustness checks in performance analysis of such strategies and the various methods by which Scientific Beta improves robustness. It also discusses measurements of robustness and the protocol that we employ internally to assess the robustness of newly developed strategies. This toolkit of tests is quite relevant to investors and can be used in their evaluation of smart beta strategies. It is based on an analysis of conditional performance in a multi-dimensional context (market, volatility, sector, factors, and macroeconomic variables). It also draws on an evaluation of robust statistical inference and on out-of-sample robustness tests of the performance and risk of the indices proposed using long-term data. Finally, the paper applies our robustness protocol to a set of strategies and evaluates the outcome against their proposed objectives.