Conditional Forecasts and Proper Scoring Rules for Reliable and Accurate Performative Predictions

Performative predictions are forecasts which influence the outcomes they aim to predict, undermining the existence of correct forecasts and standard methods of elicitation and estimation. We show that conditioning forecasts on covariates that separate them from the outcome renders the target distribution forecast-invariant, guaranteeing well-posedness of the forecasting problem. However, even under this condition, classical proper scoring rules fail to elicit correct forecasts. We prove a general impossibility result and identify two solutions: (i) in decision-theoretic settings, elicitation of correct and incentive-compatible forecasts is possible if forecasts are separating; (ii) scoring with unbiased estimates of the divergence between the forecast and the induced distribution of the target variable yields correct forecasts. Applying these insights to parameter estimation, conditional forecasts and proper scoring rules enable performatively stable estimation of performatively correct parameters, resolving the issues raised by Perdomo et al. (2020). Our results expose fundamental limits of classical forecast evaluation and offer new tools for reliable and accurate forecasting in performative settings.

then M ∈ M p has no correct forecasts. This phenomenon is well known:

1 An introduction to causal models is given in Appendix A. 2 For ease of exposition we don't let forecast F depend on covariates X, nor let it be confounded with Y . In Appendix E, we show that the results from the main paper extend to the setting where F depends on covariates X, which are allowed to be confounded with Y .