The Wi-Fi 6 Objective PAtent Landscape (“OPAL”) tool objectively scores the statistical essentiality of patent publications to Wi-Fi 6 functionality. It was created using a machine learning algorithm trained on a large set of expert reviewed Wi-Fi 6 patents. A short summary of the methodology follows:
- Universe of Patents Subject to Analysis - 3.6M+ applications and granted patents potentially relevant to Wi-Fi 6.
- Patents Evaluated Manually by Experts - 3,000+ unique families evaluated manually by technical experts at Scintillation Research.
- AI Training - Patents were vectorized using FastText, and a binary classification algorithm was trained to predict essentiality to the universe of patents.
- ML Performance - Training complied with accepted machine learning practices and the results achieved a high F-1 score of 0.99.
Basis for Using Machine Learning to Predict Essentiality
The question of who owns Wi-Fi 6 standard-essential patents is an issue that many institutions and companies are grappling with as these important technologies are deployed. Many have claimed through declarations that they own Wi-Fi 6 essential patents, but there is currently no economically sensible way to evaluate these claims. Furthermore, experience shows us that there can be many potentially essential patents that are never declared and that are not encumbered by any FRAND obligations. Unified Patents' Wi-Fi 6 “OPAL” landscape not only identifies self-declared Wi-Fi 6 patents but also these undeclared and FRAND-unencumbered patents.
According to OPEN, Unified Patents’ IEEE standards submissions database, over 7,182 technical contributions have been submitted. These top 10 account for 27% of all technical contributions.
However, unlike 3GPP/ETSI, where companies self-declare – and tend to over declare – their patent portfolios, no such phenomenon exists in IEEE’s standard-setting system. Instead of self-declarations, companies will issue Letters of Assurances (LOAs) that often include no patents and are blanket declarations. This makes the landscape very difficult to ascertain. The LOAs received are not even remotely close to those participating in the technical contributions as demonstrated below.
Recognizing this, Unified Patents turned to machine learning based analytics to predict the Wi-Fi 6 essentiality for hundreds of thousands of patents relevant or tangential to telecom. The criteria for this analytics were unwavering objectiveness, transparency, cost-efficiency, and consistency and sufficient reliability.
Creating a Training Set
The experts at Scintillation first begin to determine essentially of Wi-Fi related patents by using the following keywords:
- Orthogonal Frequency Division Multiple Access (OFDMA)
- Multi-User Multiple Input Multiple Output (MU-MIMO)
- Basic Service Set Coloring (BSS Coloring)
- Target Wake Time (TWT)
- 160 MHz (i.e. the channel bandwidth)
- 1024 QAM (Quadrature Amplitude Modulation)
Patent publications were searched using the keywords above, restricted to those with a priority date from January 1, 2013 through February 2, 2021 and with an IEEE citation or a CPC code belonging to class H04W.
The keyword search resulted in several thousand candidate patent families for essentiality review by the experts at Scintillation. From there, irrelevant patents – e.g. patents that clearly did not belong to telecom – were discarded. What remained was then categorized into discrete technology buckets corresponding to the keywords enumerated above.
Fourteen experts from Scintillation were divided into four teams, with each team specializing in a handful of technical categories. For each patent, an expert would identify any claim that could be mapped to a relevant section of the Wi-Fi 6 standard. If a claim could be found, the patent was marked as essential. If no such claim could be mapped to the specification, then the patent would be marked as non-essential. In corner-cases, a hierarchical review process allowed for closer inspection. An overall quality-control regime was implemented for all determinations, across the board. Internal review and quality control involved different teams, i.e. any team conducting a review was independent of the team that had made the initial determination.
This rigorous process resulted in a training set of a little over 3,000 patent families, with roughly ~1000 positive labels (i.e. essential families) and ~2000 negative labels.
Predicting Essentiality via Machine Learning
With Scintillation’s training set, Unified Patents used the positive and negative labels to train a binary classification algorithm to predict potential essentiality to Wi-Fi 6. Textbook machine learning techniques and good practices were adhered to in training this binary classification algorithm.
FastText was used to vectorize the title, abstract, claims, and CPC codes of each patent in the training set. Initially, 400 dimensions were used to distinguish the vectors but this was reduced to 10 to reduce the risk of overfitting. An ensemble of the best performing models – such as XGBoost and shallow extra-randomized forest – were aggregated together to make the ultimate essentiality prediction.
In training the model, a stratified K-fold cross-validation process was deployed. This stratified resampling was used to correct any optimistic errors resulting from imbalanced data sets as well as to preserve the proportionality among the cross-validation testing and training sets. The class weights of the positive labels and negative labels were also balanced.
Applying the Trained Model to the Universe
With the training set in place, Unified then had to find a relevant universe to determine essentiality rates. This was done in a way to capture the largest set of relevant patents. The first way was to use the reviewed set of patents from Scintillation as a seed set. Using a supervised machine learning approach, the universe was then expanded to identify other patents. Unified then looked at individual patents that had non-patent literature of cities for IEEE and IEEE 802. Then, a third step was to look at patents that cited CPC H0W4 - Wireless Communications Networks. The final expansion looked at the actual IEEE submissions documents, found the authors for 802.11ax submissions and then matched corresponding patents using the individuals names as inventors and that the filing date and upload date of the submission document were in a similar time frame.
Each of these steps were ultimately combined to create the largest Wi-Fi 6 universe of patents and applications. This resulted in a universe of 3,615,100 patents and applications that would then be trained to determine essentiality on the reviewed Scintillation patents.
The trained model was applied to this universe and a score was assigned to each patent publication. In the figure below, the gray depicts the distribution of scores in the universe. The red distribution corresponds to the negative labels and the blue to the positive labels.
OPAL Training Wi-Fi 6 Essentiality Scoring Distribution
The performance of the ML algorithms resulting from the training earned very high F-1 scores of 0.99 for Wi-Fi 6. The F-1 scoring captures the harmonic mean of precision and recall where precision equals the number of true positives divided by the number of all positive results and recall equals the number of true positives divided by the number of all samples that should have been identified as positive.