Asymptotically Exact Error Characterization of Offline Policy Evaluation with Misspecified Linear Models

r e t p a h C Misspecified Linear Models Arguably, the strongest assumption that we made in Chapter 2 is that the regression function f (x) is of the form f (x) = x ⊤ θ * . What if this assumption is violated? In reality, we do not really believe in the linear model and we hope that good statistical methods should be robust to deviations from this model. This is the problem of model misspecified linear models. Throughout this chapter, we assume the following model: where ε = (ε 1 , . . . , ε n ) ⊤ is sub-Gaussian with variance proxy σ 2 . Here X i ∈ IR d . When dealing with fixed design, it will be convenient to consider the vector g ∈ IR n defined for any function g : IR d → IR by g = (g(X 1 ), . . . , g(X n )) ⊤ . In this case, we can write for any estimator f ∈ IR n of f , Even though the model may not be linear, we are interested in studying the statistical properties of various linear estimators introduced in the previous ˆˆ˜ˆĉ hapters: θ ls , θ ls K , θ ls X , θ bic , θ L . Clearly, even with an infinite number of observations, we have no chance of finding a consistent estimator of f if we don't know the correct model. Nevertheless, as we will see in this chapter something can still be said about these estimators using oracle inequalities.