Validation Geometry for Reward Optimization under Proxy Misspecification
A finite validation protocol can miss precisely the proxy errors that an optimizer amplifies. This project develops an optimizer-conditioned geometry for certifying a specified reward-optimization endpoint.
- Core result
- For any fixed optimizer endpoint, the centered density ratio relative to the reference distribution is the exact sensitivity direction for proxy overstatement.
- Design insight
- Average risk, restricted worst-case coverage, and feasible measurement selection require distinct validation-design objectives.
- Evidence and scope
- Experiments verify sharp kernel witnesses, identifiability transitions, noisy certificates, and transfer failures under shifted or unseen optimizer directions. The result certifies a declared endpoint rather than universal safety.
