Towards Comprehensive Patent Approval Predictions: Beyond Traditional Document Classification

Predicting the approval odds of a patent application is a challenging problem involving multiple factors. The most important factor is arguably the novelty -35 U.S. Code § 102 rejects applications that are not sufficiently differentiated from prior art. Novelty evaluation distinguishes the patent approval prediction from conventional document classification -toosimilar newer submissions are considered as not novel and would receive the opposite label, thus confusing standard document classifiers (e.g., BERT). To address this issue, we propose a novel framework AISeer that unifies the document classifier with handcrafted features, particularly time-dependent novelty scores. Specifically, we formulate the novelty scores by comparing each application with millions of prior art using a hybrid of efficient filters and a neural bi-encoder. Moreover, we impose a new regularization term into the classification objective to enforce the monotonic change of approval prediction w.r.t. novelty scores, From extensive experiments on a large-scale USPTO dataset, we find that standard BERT fine-tuning can partially learn the correct relationship between novelty and approvals from inconsistent data. However, our time-dependent novelty feature and other handcrafted features offer a significant boost on top of it. Also, our monotonic regularization, while shrinking the search space, can drive the optimizer to better local optima, yielding a further small performance gain.