Sentiment and contrarian indicators
The answer
A sentiment indicator measures how the crowd feels right now, using something the crowd has already paid for — option prices, survey answers, cash balances, trading volume. It is a thermometer, not a clock.
Extreme readings tell you that a lot of positioning is already in one direction. They do not tell you when that positioning will reverse, and the gap between "stretched" and "over" has bankrupted more careful people than any other idea in this cluster.
Why this costs you money
Here is the sequence almost every retail investor has lived through at least once.
The market falls for 3 months. Every headline is bad. Your holdings are down 30%. You stop opening the app. Eventually you sell some of it, because the losing has to stop somewhere.
Then the market rises for 9 months. By month 7 you feel safe again. You put money back in, and you put in more than you took out, because now the story is good and you are behind.
You have just bought high and sold low, and at no point did it feel like a mistake. It felt like caution, then like confidence. Sentiment is not a measurement you apply to other people. It is the thing that moved your own hand.
The second cost is the mirror image, and it hits people who have read one book about contrarian thinking. They see a high fear reading and they buy. The fear reading goes higher. They buy more. They are correct about direction and wrong about time, and time is what kills a leveraged or impatient position. Being early is indistinguishable from being wrong until the money runs out.
How it works
Every sentiment indicator does the same thing in a different costume. It finds a number that people generate as a by-product of acting on their feelings, and it reads that number back.
They fall into 3 groups.
| Group | What it measures | Example |
|---|---|---|
| Priced | What people paid to protect themselves or to speculate | VIX, put-call ratio, option skew |
| Surveyed | What people say they believe | Investors Intelligence, AAII, fund manager surveys |
| Positioned | What people have actually done with money | Cash balances, margin debt, flows, delivery volume |
Priced measures are the most honest, because somebody had to pay real money to create the reading. Surveyed measures are the weakest, because saying you are bearish costs nothing. Positioning sits in between and is usually the slowest to arrive.
Three worth knowing in detail.
The VIX. The CBOE Volatility Index is calculated from the prices of S&P 500 index options. It expresses, as an annualised percentage, how much movement the options market is pricing into the next 30 days. India has its own version, India VIX, calculated from NIFTY 50 option prices. When people call it a "fear index" they are being loose. It measures expected movement in both directions, and it happens to rise when people buy protection, which happens when they are frightened. Article 9 in this cluster takes it apart properly.
The put-call ratio. A put option pays off when a price falls. A call option pays off when it rises. Divide the volume of puts traded by the volume of calls traded and you have a crude measure of which way people are leaning. A high ratio means heavy put buying, which usually means fear. Exchanges publish this daily. The catch is large: a put can be bought as insurance by somebody who is extremely bullish and simply wants to protect a large position, so the same number can mean 2 opposite things.
Bull-bear surveys. Weekly polls of investment advisors or individual investors, asking whether they are bullish, bearish or neutral. Long-running series exist in the United States. The useful reading is not the level but the spread between bulls and bears, and its extremes relative to its own history.
The contrarian logic underneath all 3 is one sentence: if almost everybody has already bought, the remaining buyers are few, and the marginal seller becomes powerful. It is a statement about who is left to act, not about who is right.
What it tells you, and what it does not
Sentiment tells you about fuel, not about ignition.
An extreme reading says that positioning is crowded. A crowded position falls harder when it unwinds, because everybody wants the exit at once. That is real and it is useful for sizing.
It does not tell you when. Sentiment can stay extreme for months. In a strong trend, extreme readings are normal readings — a market that goes up for 2 years will print "too bullish" many times on the way, and every one of those readings would have been a losing short.
It does not tell you level. There is no number at which fear becomes a buy. Every indicator has to be read against its own history, and its own history changes. The average level of the VIX in the 2010s was not the average level in the 1990s.
And it does not survive being counted twice. When you look at the VIX, the put-call ratio and a bull-bear survey and see the same thing, you have not found 3 confirmations. You have found 1 fact reported 3 times, because all 3 are driven by the same recent price move.
The decision rule
Sentiment changes how much you do, almost never whether you do it.
If your own analysis says buy, and sentiment is at a fear extreme, that is a reason to buy in more than 1 instalment, because the extreme can get more extreme.
If your own analysis says buy, and sentiment is at a greed extreme, that is a reason to take a smaller position and to set your exit before you enter.
If sentiment is your only reason, you do not have a reason. You have a feeling about somebody else's feeling.
The one place sentiment is close to decisive is at the extreme of the extreme, and those happen perhaps 2 or 3 times in a decade. If you find yourself identifying a generational panic twice a year, you are not measuring the market. You are measuring your own mood with extra steps.
Try this now
This takes 5 minutes and it is uncomfortable, which is the point. You are going to build a sentiment record of yourself.
- Open your broker's Reports, Tradebook or P&L section. Set the period to the last 2 years and sort by trade value.
- Write down the date of your single largest buy by rupee or dollar value. Then the date of your single largest sell.
- Open a 2-year chart of the index your market follows — NIFTY 50 for India, S&P 500 for the United States.
- Find both dates on the chart. Mark where they sit relative to the highs and lows of those 2 years.
- If you have more than 10 trades, do the same for your 3 largest buys and 3 largest sells.
What you should see. For most people, the largest buys cluster after a run of good months, near a local high, and the largest sells cluster after a fall, near a local low. Not always, and 1 trade proves nothing. But if you see the pattern across 3 buys and 3 sells, you have just measured something no indicator can tell you: your own position size rises with your confidence, and your confidence rises with recent prices.
That is the whole of sentiment analysis, applied to the only participant whose behaviour you can change.
If you have no trade history, do this instead. Write down, in 1 sentence, what you believed about the market 6 months ago. Then check what the index actually did in those 6 months. Keep the note. Repeat every 6 months. In 2 years you will have a sentiment series with a sample size of 1, and that 1 is you.
Three real cases
1. "The Death of Equities", BusinessWeek, 13 August 1979 (United States) — the cover as a sentiment reading A major business magazine ran a cover story arguing that inflation had permanently destroyed the appeal of shares and that a generation of investors had abandoned them. The article was a fair description of how people felt. The long bull market of the 1980s began roughly 3 years later. The lesson is not that magazines are contrary indicators, which is a superstition with a very small sample. The lesson is that a widely shared conclusion had already been paid for by the time it was printed.
2. Warren Buffett's op-ed, 16 October 2008 (United States) — right and early Buffett published an article in The New York Times saying he was buying American shares, and gave the rule: be fearful when others are greedy, and greedy when others are fearful. He was right about the decade that followed. He was also about 5 months early — the S&P 500 fell substantially further and did not bottom until March 2009. Anybody who read that article, borrowed money and bought would have faced a further large fall first. Same view, same evidence, completely different outcome depending on leverage and patience.
3. India, 4 June 2024 — sentiment collapsing inside 1 session Exit polls released after voting suggested a large majority for the governing party. Indian markets rose sharply on the following session, and India VIX, which had risen through the campaign, was elevated. When the actual counting on 4 June produced a different result, the index fell steeply intraday, and India VIX collapsed over the following days as the uncertainty resolved. Two things happened at once, and separating them is the skill: uncertainty fell while prices fell. A falling VIX is not a bullish signal. It is a statement that the event has passed.
The question that resolves it
A novice looks at a fear reading and asks: is everybody scared?
An expert asks: scared enough to have already sold?
The first question is about emotion and is easy to answer from headlines. The second is about positioning, and it is the only one that matters, because a market falls on selling, not on worry. People who are frightened but still fully invested are fuel. People who are frightened and already in cash are finished sellers, and a market with no sellers left rises on very little.
What would make this wrong
If sentiment extremes reliably marked turning points, then a simple rule — buy when the VIX crosses some high level, sell when a bull-bear survey crosses some high level — would produce steady profits. Anybody can test this on 20 years of data in an afternoon. The result is usually a strategy with a small number of good trades, a large number of early trades, and a drawdown that most people could not sit through.
The honest limits are 3.
First, the sample is tiny. Genuine sentiment extremes happen a handful of times per decade. A rule fitted to 6 events is fitted to 6 events, and article 8 in this cluster explains why that is close to worthless as evidence.
Second, the indicators change meaning as markets change. The put-call ratio in a market where retail investors buy far more options than they used to is not measuring the same population it measured 20 years ago.
Third, sentiment is mostly a lagging function of price. Most sentiment series are close to a smoothed, inverted version of the last few weeks of returns. If you already look at the price, you have most of the information.
In India
India VIX is calculated by NSE from the order book of NIFTY 50 index options, using a methodology licensed from CBOE. It has been published since
- It is the single most watched sentiment number in the Indian
market and it is available free on the NSE website and inside most broker apps.
Beyond it, India offers several sentiment measures that have no clean American equivalent.
- FII and DII daily flows. The exchanges publish, every evening, how much foreign institutional investors and domestic institutional investors bought and sold in the cash market. This is a positioning measure, published daily, free. Very few markets give retail investors this.
- F&O participant-wise open interest. NSE publishes open positions split by category: foreign investors, domestic institutions, proprietary desks and clients. "Clients" is largely retail. Watching what that category is doing in index futures is a direct positioning reading.
- Delivery percentage. For each stock, the exchange reports what fraction of traded volume was actually taken to delivery rather than squared off the same day. A rising price on a falling delivery percentage tells you the move is being driven by intraday activity, not by people accumulating.
What India lacks is a long-running, respected retail sentiment survey. There is no widely followed Indian equivalent of a weekly individual-investor poll going back to the 1980s.
In the United States
The United States has the deepest sentiment toolkit in the world, and most of it is free.
- CBOE VIX, computed from S&P 500 options, published since 1993 in its original form and revised to the current method in 2003. Related indices exist for other periods and other markets.
- CBOE put-call ratios, published daily and split into equity options, index options and total. The split matters: index puts are heavily used as institutional insurance, so the index ratio says something quite different from the equity ratio.
- Investors Intelligence Advisors' Sentiment, a weekly survey of investment newsletter writers running since the 1960s.
- The AAII Sentiment Survey, a weekly poll of individual investors running since 1987.
- Margin debt, reported monthly by FINRA — how much investors have borrowed against their portfolios. A slow positioning measure with a long history.
- Composite indices such as the widely quoted fear and greed gauge, which combine several of the above into 1 number. Convenient, and it hides the disagreement between the components, which is often the interesting part.
There is also an academic literature. Baker and Wurgler built a composite investor sentiment index and found it related to the later returns of small, young, unprofitable and highly volatile companies more than to the market as a whole. That is a more precise claim than "sentiment predicts the market", and more useful: sentiment bites hardest where valuation is most a matter of opinion.
Where they differ, and what that tells you
The two markets give you different halves of the same picture.
The United States gives you opinion. India gives you position.
American sentiment data is rich in surveys — what people say they think, weekly, for decades. Indian sentiment data is rich in flows — what institutions and retail clients actually did with money, published daily by the exchange.
That difference is not cosmetic. A survey can be wrong about the person answering it. A settlement record cannot. An Indian investor reading American sentiment commentary is often reading about a survey series that has no Indian equivalent, and concluding that Indian sentiment cannot be measured. The opposite is true. India's sentiment data is harder to find and better once found.
The second difference is comparability. India VIX and CBOE VIX are computed from options on 2 different indices with 2 different levels of underlying movement. A reading of 20 does not mean the same thing in both places. NIFTY 50 and the S&P 500 do not move by the same amount in a typical year, so their option markets do not price the same expectations. Comparing the 2 numbers directly is the most common error in Indian market commentary, and article 9 deals with it in full.
Carry this
- Sentiment measures fuel, not ignition. It tells you how crowded, never when.
- Priced measures beat surveyed measures, because somebody paid for them.
- The 3 indicators agreeing is 1 fact counted 3 times.
- Use sentiment to change position size, not to change direction.