Sentiment, cycles and system testing

Reading for India · about 5 min

What this cluster is for

This looks like 9 topics. It is 1 topic, asked 9 times.

How do you tell whether something you believe about markets is true?

Everything here is a version of that question. Sentiment asks whether a crowd measure tells you anything. Seasonality asks whether a calendar pattern is real or was found by searching. Backtesting asks what past prices can and cannot prove. Statistics asks what your numbers mean. Optimisation asks how a search manufactures results that were never there. Performance metrics ask which number describes a strategy honestly. Correlation and hypothesis testing give you the arithmetic. Volatility gives you the one thing in markets that is genuinely forecastable.

Two sentences carry the whole cluster, and they are worth reading before anything else.

A backtest is a hypothesis you have not tested yet. It tells you what a rule would have done on data you already have. It says nothing about a period you have not seen, and if you adjusted the rule while looking at the result, it does not even say that.

Drawdown, not return, is what ends people. Nobody leaves the market because their annual return was disappointing. They leave during a fall they had not planned for, at the worst point of it, and the size of that fall was decided by their position size long before it arrived.

By the end of this cluster you will be able to:

Write a rule you already trade precisely enough that a stranger could execute it tomorrow
Count how many things you have tested, and know what that number does to your result
Compute your own expectancy, payoff ratio and worst drawdown from your broker's export
See what your worst loss would have looked like in a different order, using your own trades
Say what a p-value does and does not mean, without notation
Convert today's VIX into an expected daily move, and check it against what the index actually did

Nothing here names a stock or a strategy to trade. Everything here is a way of checking claims, including your own.

The reading order

The first 2 articles are about claims other people make. The next 5 are about claims you make. The last 2 give you the tools underneath all of them. Read in order the first time.

Reading the crowd

Sentiment and contrarian indicators — sentiment measures what the crowd already did. It tells you how crowded, never when.
Market cycles and seasonality — which calendar patterns have a mechanism, and how many patterns were searched before the impressive one appeared.

Testing your own claims

Backtesting and system design — a backtest is a hypothesis, not evidence. Data used to build a rule cannot also confirm it.
Statistics for technicians — the quiet arithmetic inside every indicator, and the days your risk model says are impossible.
Designing a rules-based trading system — 6 components, no room for opinion. Most people discover they never had a rule.
Optimisation, robustness and Monte Carlo — any strategy can be tuned to look perfect on the past. Shuffle your own trades and see the drawdowns you avoided by luck.
Performance metrics — expectancy, payoff ratio, drawdown and recovery time. Return is the least informative number about a strategy.

The tools underneath

Correlation, regression and hypothesis testing — correlation rises towards 1 in the crashes when you were relying on it to be low. And 20 tests at the 0.05 threshold produce 1 false winner on average.
Volatility analysis — the VIX is a priced expectation of movement, not a fear gauge, and it is usually higher than what happens. Volatility is forecastable. Direction is not.

The checklist

Every article ends with something you do on your own data. Collected here, they are a weekend, and together they are an audit of your own trading that nobody else will ever do for you.

Once, with your broker's export open

Find the dates of your 3 largest buys and 3 largest sells, and mark them on a 2-year index chart. (1)
Compute your win rate, average win, average loss, expectancy and payoff ratio. (7)
Compute your worst peak-to-trough loss and how long recovery took. Look at the dates. (7)
Shuffle your trades into a random order 5 times and recompute the worst drawdown each time. Several will be worse than the one that happened. (6)
Ask whether you would have continued through the worst of those. If not, cut your position size today. (6, 7)

Once, on paper

Write 1 rule you already trade, precisely enough for a stranger to execute it. Mark every place they would have to ask you a question. (3)
Complete the other 5 components: setup, filter, exit, stop, size. (5)
Count every number in that document. Circle every word requiring judgement: strong, clean, confirmed, holding. (5)
List every rule, setting, indicator and strategy you have tried in 2 years. Count them. (6, 8)

Once, with a spreadsheet

Take 6 months of daily closes on 1 holding. Compute the average change and the standard deviation, then count the days beyond 3 standard deviations. (4)
Pick 2 holdings you believe are different. Compute their correlation over the last year, then over the worst single month. (8)
Test 1 calendar claim on 1 holding, then count how many other calendar claims you could have tested on the same data. (2)

Today, in 5 minutes

Look up India VIX or the CBOE VIX. Divide by 15.9. Compare against the average daily move of the index over the last 20 days. (9)

India and the United States, equally

Every article covers both markets at the same depth, then says where they differ and what the difference tells you.

Two divergences run through this whole cluster and both change what you should do.

The data is not equal. American readers have decades of clean, survivorship-corrected, free price history. Indian readers have under 30 years of index history, patchy constituent records and inconsistent corporate action adjustment in free sources. The same test therefore carries less evidence in India, which means the Indian version of this discipline has to be stricter, not looser. Common practice is the reverse.

The evidence about people like you is not equal either, and here India is ahead. SEBI has published population-scale studies of individual derivatives traders, covering crores of accounts, updated within the last few years. There is no American equivalent. The best evidence anywhere about what happens to individual traders without risk control was produced in India, about Indian traders, and Indian readers are the least likely to have read it.

A note on what this is not

These lessons are free and they will stay free. They are the text version of the ideas in our video course, written for anybody who cannot spend money on learning right now.

This is the least glamorous cluster in the entire track and it is the one that decides whether anything else you learned is worth anything. A method you have not tested is a preference. A preference you have tested 40 times and kept the winner of is a coincidence with a story attached. The point of all 9 articles is the same one it has always been: fewer people in the 95%.