What Explains Profits and What Doesn’t: Insights from 150 Years of Market Data – CFA Institute Enterprising Investor

Stock prices and corporate earnings move closely together over longer horizons, a relationship confirmed by more than a century of data collected by Robert Shiller. This analysis examines the strength of that long-term link and tests whether changes in the correlation between earnings and price provide insight into future stock market returns.

The results show that while earnings help explain market behavior over time, fluctuations in correlation themselves do not provide a useful basis for predicting returns. The sections that follow document empirical patterns over multiple rolling periods and assess the limits of using correlation measures as market timing tools. The findings can also help financial advisors map long-term market behavior for clients in a grounded and intuitive way.

What this analysis aims to clarify

I examine the long-term relationship between stock prices and corporate earnings for two main reasons.

First, the findings provide a simple way to explain stock market behavior over a long investment horizon. I define a long horizon as more than ten years, which is a useful minimum time frame for retirement planning and asset allocation decisions.

Second, after calculating the correlations between prices and earnings, I tested whether changes in correlation over time could serve as a leading indicator of future returns. I specifically wondered whether periods of unusually low historical correlation were followed by stronger or weaker subsequent stock market performance.

Correlation results

The analysis uses monthly averages of S&P Composite earnings per share and S&P Composite price. The reported monthly earnings, stock prices and returns data for the S&P Composite companies are based on Shiller’s data from 1871 through December 2024.

Across multiple periods, correlations between revenues and prices were consistently high.

I have chosen common time periods to examine the data and note the following:

• One starting point is the Investors Act of 1940, which was used to test whether results differed after investor protections and more uniform accounting standards were introduced. The difference appears to be negligible.
• The past ten- and twenty-year periods are included to represent what is often considered a typical retirement planning horizon.

Correlation changes over time

The correlation between earnings and stock prices fluctuates over time, especially over shorter horizons such as the five-, ten-, and twenty-year horizons. The rolling 50-year correlations also vary, although within a much smaller range.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

The lowest rolling 50-year correlation occurred in the first half of the 20th century, when data series reached 0.6. Given the backdrop of two world wars, the Great Depression and limited market regulation before 1940, it is remarkable that the correlation did not decline further.

The variability increased as the time horizon became shorter. In the twenty-year rolling series, correlations fell below 0.50 for an entire decade between February 1918 and December 1928, and again briefly in December 1948.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

The rolling ten-year correlations fell below zero during three periods: at the end of World Wars I and II, and during the high inflation era of the late 1970s and early 1980s.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

The five-year rolling correlations naturally showed the most volatility, with deeper declines and more frequent swings, including multiple periods of negative correlation. Both mean and median five-year rolling correlations were lower than those observed over longer horizons.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

Does the variability in correlations correspond to returns?

To test whether variation in the correlation between earnings and price has any predictive value for stock returns, we ran regressions of the correlation levels against subsequent annualized returns.

The R² between the profits of S&P Composite and the price from 1871 to 2024 is very high at 0.95. Given the strength of this long-term relationship – and the relative rarity of low-correlation periods – it’s reasonable to wonder whether these periods can function as buy or sell signals. In other words, does variation in the correlation between earnings and price help predict future returns?

I evaluated this question over multiple rolling time horizons. The resulting R² values ​​– which link correlation levels to subsequent annualized returns – were much lower than the R² between earnings and price itself. For the rolling ten- and five-year periods, the R² fell near zero, indicating virtually no predictive relationship.

The rolling 50 year period showed the strongest relationship with an R² of 0.53.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

For the rolling 20-year windows, the R² was 0.24, reflecting significantly more variability.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

The variability increased further in the rolling ten-year series, with the R² decreasing to 0.06.

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

The rolling five-year periods do not show a consistent pattern. R² is almost 0.0 (actual: 1.27E-07).

Source: Robert J. Shiller S&P data; Archer Bay Capital LLC

Overall, I have found no evidence that changes in the correlation between earnings and price predict future annualized returns. The data show that the two measures do not interact in any meaningful way for horizons shorter than 50 years.

Predictive power of correlation

The strong long-term relationship between earnings and prices provides a clear explanation for the rise and fall of stock markets over long periods of time. It provides a simple and intuitive framework for understanding long-term stock trends.

However, the second goal – to determine whether changes in correlation can serve as a predictive measure of annualized returns – was not achieved. The evidence shows that factors other than the earnings-to-price ratio determine the rate of change in annualized returns, even though the two series move close together over long horizons.

Key Takeaways

• Earnings and stock prices move closely together over a longer horizon. More than 150 years of Shiller data show a consistently strong relationship between the two series.
• Shorter windows cause significant noise. Correlations fluctuate significantly over five-, ten-, and twenty-year periods, reflecting wars, inflation shocks, and structural changes.
• Correlation strength does not imply predictive power. Shifts in the correlation between earnings and price provide little ability to predict subsequent returns at horizons relevant to most investors.
• Only the longest windows have limited explanatory power. Even the 50-year regressions, with an R² of 0.53, provide only modest insight, while shorter horizons are close to zero.

Profits help explain long-term market behavior, but they do not help in timing the market.

The author is a registered investment advisor with Archer Bay Capital LLC/Integrated Advisors Network – an SEC registered investment advisor. The information contained herein represents Campbell’s independent view or research and does not represent a solicitation, promotion or research of Integrated Advisors Network or Archer Bay Capital LLC. It has been obtained from or is based on sources believed to be reliable, but its accuracy and completeness are not guaranteed. This is not intended as an offer to buy, sell or hold any security.