A backtested Sharpe ratio is the maximum over the configurations that were tried, and the maximum of N draws has a higher expectation than a single draw. This deflates an observed ratio for the size of the search that produced it, the length of the sample, and the skewness and kurtosis of the returns. Everything computes in the browser; the page makes no network request.
| Parameter | Meaning | Example |
|---|---|---|
sharpe | Observed Sharpe ratio per period, not annualised. Divide an annualised figure by the square root of the periods per year before entering it, or supply periods and set annualised=1. | 0.2724 |
trials | Number of independent configurations tested, including every parameter value and every variant discarded during development. This is the size of the search, not the number of strategies presented. | 10 |
length | Number of return observations T in the sample. | 24 |
skew | Skewness gamma3 of the periodic returns. Defaults to 0. | -0.508966 |
kurtosis | Kurtosis gamma4 of the periodic returns, not excess kurtosis. Defaults to 3, the normal value. Entering excess kurtosis here inflates the result. | 2.368633 |
trialsd | Standard deviation of the per-period Sharpe ratios across the trials, measured from the trial set. If omitted, the threshold is computed under the true-zero null with a standard deviation of the square root of one over length, which is a lower bound on the correct threshold. | 0.15 |
periods | Periods per year. Used to report annualised figures and the minimum backtest length in years. Defaults to 12. | 12 |
annualised | Set to 1 if the figure in sharpe is already annualised. Defaults to 0. | 0 |
target | Target annualised Sharpe ratio for the minimum backtest length. Defaults to the observed annualised ratio. An extension to the published parameter set, added because the minimum backtest length is a function of the target rather than of the observation. | 1.0 |
oos | Comma-separated out-of-sample relative ranks of the in-sample-best configuration, one per cross-validation split, each strictly between 0 and 1. Required for the probability of backtest overfitting; without it, PBO is not reported. | 0.42,0.31,0.58 |
Example, the published base series treated as the best of ten trials:
https://quants.wiki/calc/deflated/?sharpe=0.2724&trials=10&length=24&skew=-0.508966&kurtosis=2.368633&trialsd=0.15&periods=12
A researcher who tries N configurations and reports the best one has not estimated that strategy's Sharpe ratio. They have reported the maximum of N estimates, which has a higher expectation and a smaller variance than any one of them. Treating it as a single estimate is a category error rather than a small bias, and it gets worse rather than better as computing power grows. The count N is almost never known and is always larger than the figure a researcher reports: a grid search over three parameters at ten values each is a thousand trials whether or not it was thought of that way, and every abandoned data-cleaning decision adds more. The independence assumption in the correction cuts in the researcher's favour, because correlated trials on a smooth parameter surface have a lower expected maximum than independent ones; the right input is an effective number of independent trials, which is smaller than the raw count and hard to estimate. Both should be reported, because the gap between them is where the judgement lives.
Bailey, Borwein, López de Prado and Zhu give the approximation used here. Writing gamma for the Euler-Mascheroni constant, 0.5772156649, the expected maximum of N independent standard normal draws is (1 − gamma) times the inverse standard normal at 1 − 1/N, plus gamma times the inverse standard normal at 1 − 1/(N times e). Their paper states this as equation 5, with the scaling by the standard deviation of the annualised Sharpe estimator folded in, and notes an upper bound of the square root of twice the natural log of N. Both are reported here so the gap is visible. The approximation is asymptotic in N and poor at small N: at N = 2 it returns 0.519755 against the exact value of one over the square root of pi, 0.564190, an understatement of about eight percent. At N = 1 the first inverse-normal term is negative infinity, so the expected maximum is defined here as exactly zero, which is the correct value for a single draw of a mean-zero variate and is what makes the deflated ratio collapse to the ordinary probabilistic Sharpe ratio when nothing was searched.
Source retrieved in full: David H. Bailey, Jonathan M. Borwein, Marcos López de Prado and Qiji Jim Zhu, “Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance”, Notices of the American Mathematical Society 61(5), May 2014. Equation 5 is the expression above; the paper states the square-root-of-two-log-N upper bound, gives 1.57 as the expected best in-sample Sharpe ratio after only ten configurations, and records that with five years of data no more than forty-five independent configurations should be tried before an expected best annualised Sharpe ratio of 1 arises from no edge at all, and that two years of data needs only seven.
The probabilistic Sharpe ratio is the probability that the true Sharpe ratio exceeds a stated threshold, given the observed ratio, the sample length and the higher moments. It is the standard normal cumulative distribution evaluated at the observed ratio minus the threshold, times the square root of T − 1, divided by the square root of one minus skewness times the observed ratio plus (kurtosis minus 1) over four times the squared observed ratio, with kurtosis the full fourth standardised moment rather than excess kurtosis. Substituting excess kurtosis there is a common implementation error and it inflates the result. Negative skewness raises the denominator, because the skewness term enters with a minus sign, and therefore lowers the probability, which is the correct direction: negative skewness makes a given Sharpe ratio less trustworthy. When skewness is zero and kurtosis is 3 the denominator reduces to the square root of one plus half the squared ratio, and the statistic becomes the ordinary normal-theory test on the Sharpe ratio with T − 1 rather than T. A probabilistic Sharpe ratio against a zero threshold is not a defence against overfitting; it corrects for sample length and higher moments only.
The deflated Sharpe ratio is the same statistic evaluated against a threshold set by the size of the search instead of against zero. The threshold is the standard deviation of the Sharpe ratios across the N trials, multiplied by the expected maximum of N standard normals. All ratios are per period, not annualised. The result is a probability in the interval from zero to one and is not comparable with a Sharpe ratio in any way, despite the name. The trial dispersion is the single most influential input and the one least often measured: it comes from the Sharpe ratios of every configuration tested, which is exactly what a researcher who kept only the winner cannot supply. A deflated Sharpe ratio computed with a guessed dispersion is a guess, and this page therefore defaults it to the square root of one over T, the true-zero null value, and says so rather than inventing a number.
Source retrieved: David H. Bailey and Marcos López de Prado, “The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality”, Journal of Portfolio Management 40(5), 2014, pages 94–107, doi 10.2139/ssrn.2460551 — the SSRN record and abstract were retrieved, along with the publisher's abstract at Portfolio Management Research. The article itself is behind a paywall: the abstract confirms that the measure corrects for selection bias under multiple testing and for non-normally distributed returns, and lists Minimum Backtest Length and Probabilistic Sharpe Ratio among its keywords, but the algebra above was not read from the published page images. The probabilistic Sharpe ratio itself is attributed to Bailey and López de Prado (2012), “The Sharpe Ratio Efficient Frontier”; that paper was not retrieved and is named on the strength of the citation alone.
The minimum backtest length is the sample length below which a claimed Sharpe ratio cannot be distinguished from the best of N trials on data with no edge. It is the square of the expected maximum of N standard normals divided by the target annualised Sharpe ratio, expressed in years, with the approximation given in the source being twice the natural log of N over the squared target. The relationship is inverse-square in the target, which is why low-Sharpe strategies are effectively unverifiable: demonstrating a true annualised Sharpe ratio of 0.5 against even forty-five trials needs about twenty years of data. The formula is stated in years and uses an annualised ratio, so the sampling frequency does not enter. A daily backtest and a monthly backtest over the same calendar span face the same minimum, which is correct, because sampling the same period more often does not add independent information about the mean. This is a necessary condition and not a sufficient one: a backtest longer than the minimum has not been shown to be valid, it has merely not been ruled out on multiple-testing grounds. Look-ahead bias, survivorship bias and regime change are separate failures with separate tests.
The probability of backtest overfitting is the probability that the configuration selected as best in sample performs below median out of sample. It is estimated by combinatorially symmetric cross-validation: divide the sample into contiguous groups, form every balanced train-test partition, pick the in-sample winner in each, and record its relative out-of-sample rank. With that rank written as omega, the logit is the natural log of omega over one minus omega, and the statistic is the fraction of splits in which that logit is at or below zero. A value near one half means the in-sample ranking carries no out-of-sample information at all; above one half means the in-sample winner is worse out of sample than a randomly chosen configuration, which is the signature of an overfit selection procedure. It measures the selection procedure and not the selected strategy: a low value says the method of choosing between candidates generalises, not that the winner has an edge, and it is not a probability of losing money.
Computing it properly needs the full matrix of performance for every configuration over every group, which is why it is rarely reported and why this page asks for relative ranks rather than raw out-of-sample performance. If raw performance figures for the winner alone are supplied, the reference distribution they should be ranked against is missing, and no honest rank can be constructed from them. In that case this page declines to report a figure and says so, rather than substituting the median of the winner's own results, which would return one half by construction and mean nothing.
Source: Bailey, Borwein, López de Prado and Zhu, “The Probability of Backtest Overfitting”, Journal of Computational Finance, 2017. That paper was not retrieved. What was retrieved is the same authors' earlier Notices of the AMS article linked above, which sets out the multiple-testing framework and the minimum backtest length but not the cross-validation statistic; the definition above is the one published in the corpus this calculator belongs to, and it is stated here as such.
The inverse standard normal distribution function is Wichura's algorithm AS 241, the double-precision variant conventionally called PPND16, which is a pair of rational approximations in the central region and two further pairs in the tails. Its documented relative accuracy is about one part in ten to the sixteenth. Checked here against reference values, the returned quantile at probability 0.975 differs from 1.959963984540054 by 4.4 times ten to the minus sixteen, and at probability ten to the minus ten it reproduces −6.361340902404056 exactly in double precision. Upper-tail arguments are evaluated as the negative of the quantile at the complementary probability, so that the inverse normal at one minus one over N is computed as minus the inverse normal at one over N. That matters: forming the argument as 1 − 1/N directly loses all significance once N exceeds about ten to the sixteenth, and computing the trial count needed for an expected best annualised Sharpe ratio of 2 over twenty years of monthly data requires N near three times ten to the eighteenth, where the naive form silently returns infinity.
The standard normal cumulative distribution function is Hart's 1968 double-precision rational approximation, in the arrangement published by Graeme West as “Better approximations to cumulative normal functions”. It uses a seven-term over seven-term rational form for arguments below the square root of fifty and a continued fraction above it. Checked against reference double-precision values at arguments of 0, 1, 1.96, −3, 5 and −8, the largest absolute discrepancy observed was 2.1 times ten to the minus sixteen, which is the limit of double precision; the relative error in the far left tail at −8, where the value is about 6.22 times ten to the minus sixteen, was under one part in ten to the eighth. No approximation on this page is the binding source of error. The binding source of error is the trial count, which nobody knows.
The corpus this calculator belongs to publishes worked figures from a constructed 24-month return series with a monthly Sharpe ratio of 0.2724, skewness −0.508966 and kurtosis 2.368633, assuming a trial-Sharpe standard deviation of 0.15. Those are the default inputs. The published values are below, for comparison against the live tables.
| N trials | E[max Z] | Threshold SR0, monthly | Deflated Sharpe ratio |
|---|---|---|---|
| 1 (no selection) | 0.000000 | 0.000000 | 0.887023 |
| 2 | 0.519755 | 0.077963 | 0.806286 |
| 4 | 1.052123 | 0.157818 | 0.694739 |
| 6 | 1.300141 | 0.195021 | 0.634561 |
| 10 | 1.574598 | 0.236190 | 0.563936 |
| 20 | 1.900708 | 0.285106 | 0.477480 |
| 30 | 2.073392 | 0.311009 | 0.431868 |
| 100 | 2.530603 | 0.379590 | 0.316870 |
| 1000 | 3.255122 | 0.488268 | 0.168640 |
Also published: the probabilistic Sharpe ratio against zero is 0.887023 with the observed higher moments and 0.900220 under normal moments, so the observed skewness and kurtosis cost about 1.3 percentage points of confidence. The expected maximum of N standard normals is 0.519755 at N = 2, 1.192594 at 5, 1.900708 at 20, 2.235640 at 45, 3.052528 at 500, 3.860665 at 10,000, 4.390829 at 100,000 and 4.867860 at 1,000,000. Minimum backtest length at an annualised target of 1 is 2.4794 years for 10 trials, 4.9981 for 45, 6.4040 for 100, 10.5958 for 1,000 and 14.9047 for 10,000; the twice-log-N approximation gives 4.6052, 7.6133, 9.2103, 13.8155 and 18.4207. At a sample length of 24 months, the expected best annualised Sharpe ratio is 1.1134 from 10 trials, 1.7894 from 100, 2.3017 from 1,000 and 2.7299 from 10,000, and roughly 243 trials produce an expected best of 2.