Kalshi's first research report is out: How collective intelligence beats Wall Street think tanks in predicting CPI.

null

This article is from: Kalshi Research

Compiled by | Odaily Planet Daily (@OdailyChina); Translator | Azuma (@azuma_eth)

Editor’s Note: Leading prediction market platform Kalshi announced yesterday the launch of a brand new research column, Kalshi Research, aimed at providing Kalshi’s internal data to scholars and researchers interested in topics related to prediction markets. The first research report of the column has been released, titled “Kalshi Outperforms Wall Street in Predicting Inflation” (Beyond Consensus: Prediction Markets and the Forecasting of Inflation Shocks).

The following is the original content of the report, compiled by Odaily Planet Daily.

Overview

Generally, in the week prior to the release of important economic statistical data, analysts and senior economists from large financial institutions provide estimates for the expected values. Once these predictions are compiled together, they are referred to as the “consensus expectation,” which is widely regarded as an important reference for insights into market changes and adjustments in position layout.

In this research report, we compare the performance of consensus expectations with the implied pricing of the Kalshi prediction market (sometimes referred to as “market prediction” below) in predicting the actual value of the same core macroeconomic signal - the year-over-year overall inflation rate (YOY CPI).

Core Highlights

Overall accuracy is superior: Under all market conditions (including normal and shock environments), Kalshi's average absolute error (MAE) is 40.1% lower than consensus expectations.

“Shock Alpha”: When a significant shock occurs (greater than 0.2 percentage points), Kalshi's prediction within a one-week forecast window has an MAE that is 50% lower than the consensus expectation; if the shock occurs the day before the data is released, the MAE will further expand to 60%. In the case of a moderate shock (between 0.1 - 0.2 percentage points), Kalshi's prediction within a one-week forecast window also has an MAE that is 50% lower than the “consensus expectation”; the day before the data is released, this will expand to 56.2%.

Predictive Signal: When the deviation between market predictions and consensus expectations exceeds 0.1 percentage points, the probability of a prediction impacting the market is about 81.2%, rising to approximately 82.4% the day before the data is released. In cases where market predictions are inconsistent with consensus expectations, market predictions are more accurate in 75% of cases.

Background

Macroeconomic forecasters face an inherent challenge: the most critical moments—when markets become disordered, policies shift, and structural breaks occur—are precisely the periods when historical models are most likely to fail. Financial market participants usually release consensus forecasts a few days before key economic data is announced, aggregating expert opinions into market expectations. However, these consensus views, while valuable, often share similar methodological paths and information sources.

For institutional investors, risk managers, and policymakers, the stakes of prediction accuracy are asymmetrical. In non-controversial times, slightly better predictions can only provide limited value; but during periods of market turmoil—when volatility soars, correlations break down, or historical relationships fail—superior accuracy can yield significant Alpha returns and limit drawdowns.

Therefore, understanding the behavioral characteristics of parameters during periods of market volatility is crucial. We will focus on a key macroeconomic indicator — the year-on-year overall inflation rate (YOY CPI) — which is a core reference for future interest rate decisions and an important signal for measuring economic health.

We compared and assessed the predictive accuracy over multiple time windows prior to the official data release. Our core finding is that the so-called “impact Alpha” does indeed exist — that is, in tail events, market-based predictions can achieve additional predictive accuracy compared to consensus benchmarks. This outperformance does not merely imply pure academic significance, but can significantly enhance signal quality at critical moments when predictive errors carry the highest economic costs. In this context, the truly important question is not whether prediction markets are “always right,” but whether they provide a signal with differentiated value that is worth incorporating into traditional decision-making frameworks.

Methodology

data

We analyzed the daily implied prediction values of market traders on the Kalshi platform, covering three time points: one week before the data release (matching the consensus expectation publication time), the day before the release, and the morning of the release. Each market used is (or was) a real tradable market in operation, reflecting real fund positions at different levels of liquidity. For the consensus expectations, we collected institutional-level YoY CPI consensus forecasts, which are typically published about a week before the official data release by the U.S. Bureau of Labor Statistics.

The sample period is taken from February 2023 to mid-2025, covering over 25 monthly CPI release cycles, spanning various macroeconomic environments.

Impact Classification

We categorize events into three types based on the “unexpected magnitude” relative to historical levels. “Shock” is defined as the absolute difference between consensus expectations and actual published data:

Normal Event: The forecast error of YOY CPI is less than 0.1 percentage points;

Moderate impact: The forecast error of YOY CPI is between 0.1 and 0.2 percentage points;

Significant impact: The forecast error of YOY CPI exceeds 0.2 percentage points.

This classification method allows us to examine whether the predictive advantage exhibits systematic differences as the difficulty of prediction changes.

Performance Indicators

To evaluate the predictive performance, we use the following metrics:

Mean Absolute Error (MAE): The main accuracy metric, calculated as the average of the absolute differences between predicted values and actual values.

Winning Rate: We will record which prediction is closer to the final actual result when the difference between consensus expectations and market forecasts reaches or exceeds 0.1 percentage points (rounded to one decimal place).

Forecasting Time Span Analysis: We track how the accuracy of market valuations evolves gradually from one week prior to the release date to the release date itself, in order to reveal the value brought by the continuous incorporation of information.

Result: CPI forecast performance

Overall accuracy is more advantageous.

In all market environments, market-based CPI forecasts have an average absolute error (MAE) that is 40.1% lower than consensus forecasts. Across all time spans, market-based CPI forecast MAE is lower than consensus expectations by 40.1% (one week ahead) to 42.3% (one day ahead).

In addition, when there is a divergence between consensus expectations and market implied values, Kalshi's market-based predictions demonstrate statistically significant win rates, ranging from 75.0% a week in advance to 81.2% on the day of release. If we also consider the cases where the predictions align with consensus expectations (to one decimal place), the market-based predictions are approximately 85% likely to be in line with or outperform consensus expectations a week in advance.

Such a high directional accuracy indicates that when there is a divergence between market predictions and consensus expectations, this divergence itself carries significant informational value regarding “whether a shock event is likely to occur.”

“Impact Alpha” does exist.

The difference in prediction accuracy is particularly evident during shock events. In moderate shock events, when the release time is consistent, the market's predicted MAE is 50% lower than the consensus expectation, and this advantage expands to 56.2% or more the day before the data is released; in major shock events, when the release time is consistent, the market's predicted MAE is also 50% lower than the consensus expectation, and can reach 60% or more the day before the data is released; while in a normal environment without shocks, the market predictions are roughly on par with the consensus expectations.

Despite the small sample size of impact events (which is reasonable in a world where “shocks are inherently unpredictable”), the overall pattern is very clear: the market's information aggregation advantage is most valuable when predicting the environment is most challenging.

However, what is even more important is that not only does Kalshi's prediction perform better during periods of shocks, but the divergence between market predictions and consensus expectations may itself be a signal that a shock is about to occur. In cases of divergence, the win rate of market predictions compared to consensus expectations reaches 75% (within comparable time windows). Furthermore, threshold analysis indicates that when the deviation between market and consensus exceeds 0.1 percentage points, the probability of a shock occurring is approximately 81.2%, and this probability further rises to about 84.2% the day before the data is released.

This significant difference at the practical level indicates that prediction markets can serve not only as a competitive forecasting tool alongside consensus expectations, but also as a “meta-signal” regarding the uncertainty of predictions, transforming divergences between the market and consensus into a quantifiable early indicator for warning potential unexpected outcomes.

Derivative Discussion

An obvious question arises: why do market forecasts outperform consensus forecasts during shocks? We propose three complementary mechanisms to explain this phenomenon.

Heterogeneity of market participants and “collective intelligence”

Traditional consensus expectations, while integrating the views of multiple institutions, often share similar methodological assumptions and information sources. Econometric models, Wall Street research reports, and government data releases form a highly overlapping common knowledge base.

In contrast, prediction markets aggregate positions held by participants with different information bases: including proprietary models, industry-level insights, alternative data sources, and intuition based on experience. This diversity of participants is grounded in a solid theoretical basis of the “wisdom of crowds” theory. This theory suggests that when participants possess relevant information and their prediction errors are not completely correlated, aggregating independent predictions from diversified sources can often yield better estimation results.

When there is a “state switch” in the macro environment, the value of this information diversity is particularly prominent - individuals with scattered, local information interact in the market, and their information fragments can be combined to form a collective signal.

Differences in participant incentive structures

Consensus forecasters at the institutional level often operate within complex organizational and reputational systems, which systematically deviate from the goal of “purely pursuing predictive accuracy.” The professional risks faced by forecasters create an asymmetric payoff structure — significant prediction errors can incur substantial reputational costs, while even highly accurate predictions, especially those achieved by significantly deviating from peer consensus, may not necessarily yield proportionate professional rewards.

This asymmetry induces “herding behavior,” where forecasters tend to cluster their predictions around the consensus value, even when their private information or model outputs suggest different outcomes. The reason is that, in professional systems, the cost of “making a mistake in isolation” is often higher than the benefit of “being right in isolation.”

In stark contrast, the incentive mechanisms faced by participants in prediction markets achieve a direct alignment between predictive accuracy and economic outcomes — accurate predictions mean profits, while incorrect predictions mean losses. In this system, reputational factors are almost nonexistent; the only cost of deviating from market consensus is economic loss, which entirely depends on whether the prediction is correct. This structure imposes stronger selective pressure on predictive accuracy — participants who can systematically identify consensus prediction errors will continuously accumulate capital and enhance their influence in the market through larger position sizes; whereas those who mechanically follow the consensus will continually suffer losses when the consensus is proven wrong.

During periods of significant rising uncertainty, the differentiation of this incentive structure is often most pronounced and economically significant when the occupational costs for institutional forecasters deviate from the expert consensus at their peak.

Information Aggregation Efficiency

A noteworthy empirical fact is that even a week before data is released — a time point that aligns with the typical time window for consensus expectations — market predictions still exhibit a significant accuracy advantage. This indicates that the market advantage does not merely stem from the “speed of information acquisition advantage” that is often cited about market participants.

In contrast, market forecasts may more efficiently aggregate those fragments of information that are too dispersed, too industry-specific, or too vague to be formally incorporated into traditional econometric forecasting frameworks. The relative advantage of prediction markets may not lie in accessing public information earlier, but rather in their ability to more effectively synthesize heterogeneous information within the same time scale — and consensus mechanisms based on surveys often struggle to process this information efficiently, even when they have the same time window.

Limitations and Precautions

Our research results require an important limitation. Due to the overall sample covering only about 30 months, significant shock events are inherently rare, which means that the statistical power for larger tail events remains limited. A longer time series would enhance future inferential capability, although the current results have strongly suggested the superiority of market predictions and the variability of signals.

Conclusion

We recorded a significant performance of the prediction market relative to expert consensus expectations in terms of systematic and economic significance, especially during shock events when prediction accuracy is most critical. The market-based CPI forecast has an overall error that is about 40% lower, and during periods of major structural changes, the error reduction can reach about 60%.

Based on these findings, several future research directions become particularly important: first, to investigate whether “shock Alpha” events can be predicted through volatility and predictive divergence indicators using a larger sample size and across various macroeconomic indicators; second, to predict the liquidity thresholds above which the market can consistently outperform traditional forecasting methods; third, to explore the relationship between the market's predicted values and the predicted values implied by high-frequency trading financial instruments.

In an environment where consensus forecasting heavily relies on models with strong correlations and shared information sets, prediction markets provide an alternative information aggregation mechanism that can capture state transitions earlier and handle heterogeneous information more efficiently. For entities that need to make decisions in an economic environment characterized by increasing structural uncertainty and the frequency of tail events, “Shock Alpha” may not only represent a gradual improvement in forecasting ability but should also be considered a fundamental component of their robust risk management infrastructure.