--- title: "Statistical Foundations for Quant Investing" description: "Fat tails, p-values, the multiple testing problem, and why most backtests lie. Statistical foundations every systematic investor needs. By Tanmay Kurtkoti." source_url: "https://www.rupeecase.com/learn/path-5/module-5-1-statistical-foundations" --- [Path 05 1 of 5](/learn/path-5/)Statistical Foundations for Quant Investing[Next 5.2](/learn/path-5/module-5-2-time-series-analysis) Quantitative finance has a statistics problem. Not too little statistics; too much confidence in it. Every factor, every strategy and every signal arrives dressed in t statistics, p values and [Sharpe](/learn/glossary#sharpe) ratios. Most of them mean far less than they appear to. What follows is about healthy scepticism, starting with my own research. The factor premia are real and the evidence for them is substantial. The methods used to find and validate them are routinely misused, and knowing those misuses is the first step to research that survives contact with a live market. Test enough useless strategies and one will look significant 1. 100 tested**99.4%** 2. 100 tested**99.4%** 3. 100 tested**99.4%** 4. 100 tested**99.4%** Chance that at least one strategy with no edge passes a p < 0.05 test: 1 − 0.95ⁿ, for n independent tests. ## The problem with normal distributions Standard portfolio theory and most back test statistics assume returns follow a normal distribution, the bell curve. The assumption is mathematically convenient and empirically wrong. Real returns have **fat tails**: extreme days happen far more often than a bell curve predicts. You do not need decades of data to see it. I ran the numbers on the Nifty 50's daily closes from 7 Oct 2024 to 9 Oct 2026, 498 daily returns including the two Budget day sessions. | Statistic, Nifty 50 daily log returns | Actual | If returns were normal | | --- | --- | --- | | Annualised volatility | 13.1% | | | Kurtosis | 5.3 | 3.0 | | Days beyond 3 standard deviations | 6 | about 1.3 | | Days beyond 2 standard deviations | 22 | about 22.6 | | Largest one day fall | −3.3% | | Computed by RupeeCase from NSE daily index closes, 7 Oct 2024 to 9 Oct 2026. Price index, dividends excluded. Look at the pattern. The moderate moves, beyond 2 standard deviations, arrive almost exactly as often as a bell curve says. The extreme ones, beyond 3, arrive about four and a half times as often. That is what fat tails mean in practice: the model is fine on ordinary days and wrong on the days that matter. 5σ Under a normal distribution, a 5 standard deviation daily fall should happen about once in 13,800 years of trading. Major markets see moves of that size every decade or so. 5.3 Kurtosis of Nifty 50 daily returns over the two years to 9 Oct 2026. A normal distribution has 3. The practical consequence: any risk number that assumes normality, such as a confidence interval built from [standard deviation](/learn/glossary#standard-deviation) or a parametric value at risk, understates tail risk. Maximum [drawdown](/learn/glossary#drawdown) is a more honest measure precisely because it assumes nothing. It just reads the worst fall that actually happened. ## P values and what they actually mean A back test shows a t statistic of 2.3 and a p value of 0.02. "Significant at the 5% level", so the strategy works? Not necessarily. The p value answers a narrow question: *if this strategy had no edge at all, how likely would results this good be by chance?* Here, 2%. It sounds convincing. Two problems: * **The p value is not the probability that the strategy works.** It is the probability of these results if it does not work. Those are very different statements, and confusing them is the most common error in research. * **Statistical significance is not economic significance.** A strategy with a t statistic of 2.1 and 0.05% a year of excess return is significant and worthless after STT and charges. t statistic of a strategy's mean return t = mean / (std dev / √n) ≈ Sharpe × √years With annual Sharpe S measured over T years, t is roughly S × √T. A Sharpe of 0.5 needs 16 years of data to reach t = 2, and 36 years to reach t = 3. A Sharpe of 1.0 needs 4 and 9 years. That second formula is worth memorising. It tells you that a five year back test, which is all most Indian strategies can show with clean data, can only confirm a high Sharpe ratio. A modest edge cannot be proven in five years at all. It has to be supported by economic reasoning and by out of sample evidence instead. Harvey, Liu and Zhu (2016) counted hundreds of published factors and argued that, given how many have been tested, a new factor should clear a t statistic of about **3.0**, not the traditional 2.0, before anyone takes it seriously. [Harvey, Liu and Zhu (2016), ...and the Cross-Section of Expected Returns](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2249314) **Myth**A p value of 0.05 means a 95% chance the strategy works. **Fact**It means a 5% chance of results this good if the strategy had no edge. With many strategies tested, some will pass that bar by luck alone. ## The multiple testing problem This is the most important statistical idea in quantitative research and the most ignored. Test 100 completely random signals on the [Nifty 500](/learn/glossary#nifty-500) and about 5 will look significant at the 5% level by pure chance. Publish the best three as discovered factors and you have done what researchers call p hacking, whether you meant to or not. Most research, published work included, does not say how many signals were tried before the winner was chosen. The back test shows the best of many trials, and the statistics are computed as if it were the only one. Bonferroni correction Adjusted α = α / n To keep a 5% chance of any false discovery across n tests, each test needs p < 0.05 / n. With 20 signals, each needs p < 0.0025. It is conservative, but it shows how fast the bar rises with every extra test. ### Worked example: twenty moving averages Say you test a moving average crossover on the Nifty 50 with 20 different look back lengths, 10 days to 200 days, and the best one shows a p value of 0.01. Alone that looks strong. Across 20 tries, Bonferroni says you needed 0.0025. The best of twenty random rules would pass 0.01 about 18% of the time if the tests were independent (1 − 0.99²⁰), and look back lengths are related, so treat that as a rough guide. Your winner is not evidence yet. It is a candidate, and it needs data it has never seen. **The honest check:** for any factor or strategy, ask how many signals were tested to find this one. If the answer is many, and no correction was applied, treat the result with real scepticism, including your own research. ## Stationarity and regime change A signal that worked from 2005 to 2015 may not work from 2015 to 2025. Financial time series are **non stationary**: their statistical behaviour changes as market structure, participants and information change. That creates a tension. You want long histories for statistical power, but old data may describe a market that no longer exists. The Nifty 50 makes the point inside just two years. Its 21 day volatility ranged from 5.2% in early September 2026 to 28.3% in early April 2026 (computed from NSE daily closes). A risk model fitted on either month alone would describe a different market. Momentum: comparatively durable Medium term momentum has been documented across many markets and decades, India included. Persistence across regimes is evidence of a structural effect rather than a fit to one period, though momentum still has sharp crashes. Value: definitions age Price to book value was built for a world of factories. As capital light businesses in IT, pharma and consumer brands grew, book value said less about worth. A value signal that never adapts its definition can decay. Market structure shifts Retail participation, demat accounts and small cap liquidity in India have grown enormously over the last decade. A small cap strategy tested on the thin markets of the past faced different execution than it would today. How to detect it Rolling window analysis and structural break tests, such as the Chow test, show when a signal's power changed. Judge a strategy across sub periods, not just on its whole period average. 1. **Hold out data**Judge the final rule on data it never saw. Module 5.5. 2. **Hold out data**Judge the final rule on data it never saw. Module 5.5. 3. **Hold out data**Judge the final rule on data it never saw. Module 5.5. 4. **Hold out data**Judge the final rule on data it never saw. Module 5.5. 5. **Hold out data**Judge the final rule on data it never saw. Module 5.5. ## Autocorrelation and inflated t statistics Standard tests assume independent observations. Strategy returns often are not. A momentum strategy holds similar positions from one month to the next, so its monthly returns are related. That overstates the effective sample size and makes t statistics look stronger than they are. Overlapping windows make it worse. A five year back test reported as 49 rolling one year returns has 49 numbers but only about five independent years. The fix: use Newey West standard errors for time series t statistics, or judge performance on non overlapping periods such as calendar years. Treat any chart of rolling returns as a picture of consistency, not as 49 separate pieces of evidence. ## Common mistake: trusting the Sharpe of a short back test The common mistake is reading a back tested Sharpe ratio as a fact about the future. A Sharpe ratio estimated over five years has a wide error band. Using the rule of thumb above, the standard error of an annual Sharpe estimate is roughly 1 / √years, about 0.45 over five years. A measured 1.6 is consistent with a true Sharpe anywhere from below 1 to above 2. Add selection: the strategy you are shown is usually the best of several variants, so its measured Sharpe is biased upwards. Expect live results to come in lower than the back test. The good strategies are the ones that still work after that haircut. ## How this shows up in RupeeCase Statistics in RupeeCase factsheets Machine learning is used in RupeeCase research to search and optimise the rules behind each strategy, tested out of sample; the rule that ships is then frozen and published, so the live strategy is a fixed algorithm, not one that keeps refitting itself. The factsheets show calendar year returns, which do not overlap, next to rolling 1 year and 3 year windows, which do; read the rolling ones as consistency, not as extra evidence. Every return is labelled model, back tested, with its window. See an example on the [Allcap factsheet](/strategies/allcap), and the full record on the [track record page](/track-record). ## Questions readers ask What t statistic should I look for in a back test?For a new idea, at least 3 rather than 2, because so many ideas have already been tested. Remember that t is roughly the Sharpe ratio times the square root of the years tested, so short back tests rarely reach it unless the Sharpe is very high. Are Indian market returns normally distributed?No. Daily Nifty 50 returns over the two years to October 2026 had a kurtosis of 5.3 against 3 for a normal distribution, and about four and a half times as many 3 standard deviation days as a bell curve predicts. How long a back test do I need?Long enough to include at least one serious fall and, ideally, more than one market regime. Five years is a minimum; for a modest edge the statistics alone will not be conclusive, so out of sample evidence and an economic reason matter more. What does Statistical Foundations for Quant Investing cover?Most backtests are wrong. Not because the data is bad, but because the statistics are misapplied. This module covers the core statistical concepts that separate rigorous quant research from elegant-looking noise. It works through The problem with normal distributions, P values and what they actually mean, The multiple testing problem, Stationarity and regime change and more. It is module 5.1 of Path 5, Advanced Quant Methods, about 9 minutes of reading, free. TTry it ## Statistical Foundations for Quant Investing, in numbers. Optimal capital fraction given edge and odds Your numbers, the module's formula, nothing stored. Calculator ### Kelly Criterion Calculator Maximises geometric growth of capital. Most professionals use a fraction of full Kelly to soften drawdowns. Win probability (%)Average win (%)Average loss (%) [Full calculator: Sharpe and volatility](/learn/tools/sharpe-volatility-calculator) AApply it on RupeeCase ## 5Paisa Focused Alpha 5 Five stocks, three-week rotation. The most concentrated card on the lineup. An index built on the rules this path teaches, concentrated momentum book, highest octane on the lineup, held stock by stock in your own demat. Risk band: Very High. [See the strategy](/strategies/5paisa-focused-alpha-5)[Sign up for RupeeCase](https://invest.rupeecase.com/signup?strategy=5paisa-focused-alpha-5) [CAGR54.1%Sharpe1.60Max DD−24.2%Minimum22000Model, back tested, as of 27 Apr 2026. Not client returns, not a promise.](/strategies/5paisa-focused-alpha-5) QQuick check ## 10 questions, 7 to pass. Pick an answer and the explanation appears. Your result stays on this device and counts towards the Path 05 certificate. CCheck yourself ## Path 05 so far. Your quick check results on this device, stop by stop. 1. [TPath test, 24 questionsNot yet](/learn/path-5/module-5-5-out-of-sample-strategies#pathTestSection) 2. [TPath test, 24 questionsNot yet](/learn/path-5/module-5-5-out-of-sample-strategies#pathTestSection) 3. [TPath test, 24 questionsNot yet](/learn/path-5/module-5-5-out-of-sample-strategies#pathTestSection) 4. [TPath test, 24 questionsNot yet](/learn/path-5/module-5-5-out-of-sample-strategies#pathTestSection) 5. [TPath test, 24 questionsNot yet](/learn/path-5/module-5-5-out-of-sample-strategies#pathTestSection) 6. [TPath test, 24 questionsNot yet](/learn/path-5/module-5-5-out-of-sample-strategies#pathTestSection) Explore next ## Put the module to work. * [CalculatorSharpe and volatilityFree, with the formula and a worked example](/learn/tools/sharpe-volatility-calculator) * [Strategy5Paisa Focused Alpha 5 strategyNifty Alpha 50, 5 stocks, every 3 weeks](/strategies/5paisa-focused-alpha-5) * [Free module 5.2Time Series Analysis8 min read](/learn/path-5/module-5-2-time-series-analysis) * [Learn guideSystematic Investing in India: RulesA pillar guide across the paths](/learn/systematic-investing) Community ## Join my WhatsApp group. Scan the code, fillFill one short form, and join my WhatsApp group. Slides, sources, factsheets and the rebalance logs live there, and members join a daily Google Meet with me. [Sign up for RupeeCase](https://invest.rupeecase.com/signup) [Join the WhatsApp group](https://forms.gle/C6HMtM1Z2GfeocQV7) [One form, one group](https://forms.gle/C6HMtM1Z2GfeocQV7)Scan with your phone camera SSources ## Where the figures come from. 1. NSE daily Nifty 50 index data, 7 Oct 2024 to 9 Oct 2026, statistics computed by RupeeCase 2. NSE daily Nifty 50 index data, 7 Oct 2024 to 9 Oct 2026, statistics computed by RupeeCase 3. NSE daily Nifty 50 index data, 7 Oct 2024 to 9 Oct 2026, statistics computed by RupeeCase 4. NSE daily Nifty 50 index data, 7 Oct 2024 to 9 Oct 2026, statistics computed by RupeeCase GTerms ## 5 terms from this module. Fat tailsThe tendency of financial return distributions to have more extreme observations than a normal distribution predicts. Formally measured by excess kurtosis. P-valueThe probability of seeing results at least as extreme as the data, assuming no true effect. Does not equal the probability that a strategy works. Multiple testingThe inflation of false positive rates when many hypotheses are tested. Testing 100 signals at 5% produces ~5 false positives even if none have genuine edge. [Stationarity](/learn/glossary#stationarity)A time series is stationary if its statistical properties don't change over time. Financial returns are typically non-stationary, complicating historical analysis. [Autocorrelation](/learn/glossary#autocorrelation)Correlation between a time series and lagged versions of itself. Serial correlation in strategy returns inflates apparent statistical significance if not corrected. [All 91 terms in the glossary](/learn/glossary) [From the podcast archiveEP30: Sensex Expiry. Sell The Straddle. Brent CooledThe Tanmay Edge, 30 Apr 2026, graded 2/5](/podcast/episodes/2026-04-30)[Pillar guideSystematic Investing18 modules across paths 2, 3, 5](/learn/systematic-investing) Education only, not investment advice. RupeeCase is a technology platform operated by QC Alpha Technologies Pvt Ltd; it is not a fund, a PMS or a SEBI registered investment adviser. Model returns are back tested and not a promise of future returns. Figures carry the date they were checked.