1. How are we computing? Invalidation Certificates Consider invalidating the constraints (prior info, and N dataset units) The invalidation certificate is a binary tree, with L leaves. At the i’ th leaf –coordinate-aligned cube –Polynomial/rational functions (“surrogate models”) & error bounds, which satisfy –sum of squares certificate proving the emptiness of Moreover Caveat: with each M j relatively complex, these error bounds are generally heuristic, implicitly assuming regularity in M j

2. Building Quadratic, rational surrogates on Given L samples of M, namely Relaxing the denominator condition number bound leads to a quasi-convex problem, solved with bisection and semidefinite programming. –Richer approximation than quadratics only –Parametrization is only twice as large (as quadratics) –Non-explicit control of behavior off sample points (with κ) peak fitting error using rational, quadratic function on samples with condition number bound on denominator over the domain (not just sample points) minimize

3. Quadratic, rational surrogates on Given L samples of M, namely Enforce using the S-procedure: Find nonnegative and such that Quadratic forms… nonnegative everywhere Two (1+n)x(1+n) matrices that depend affinely on coefficients of D and λ and τ must be positive semidefinite

4. Quadratic, rational surrogates on Given L samples of M, namely But ensures that, so

5. Quadratic, rational surrogates on Given L samples of M, namely For fixed t –linear constraints on (coefficients of) D and N –Semidefinite constraints on (coefficients of) D and λ and τ –linear constraints on λ and τ –Check feasibility with, e.g., SeDuMi Bisect on t to optimal

6. Prediction with Rational surrogates Given 1+P models and data “Bound”: Inner and Outer bounds to minimum and maximum value that can take on Local search S-procedure, weak duality

7. Prediction with rational surrogates The prediction problem is But so Minimize such that the S-procedure proves Fixed, SDP. Bisect to optimal outer bound So, in this context, many computations involve proving empty intersections of systems of quadratic inequalities.