The WordPress plugin directory has no categories. It has tags — a handful of free-text labels per plugin, typed by whoever wrote it — and there are 64,231 distinct ones in use across the live directory.
Why that is not a classification
Nothing connects “ecommerce” to “woocommerce”, or stops “post”, “posts” and “gutenberg” from all describing the same plugin. There is no hierarchy, no definitions, and no review. It works well enough as search keywords and fails completely as a way of counting anything.
Try to answer “what share of the directory is about security” from a tag cloud and you will find you cannot. The labels overlap, compete and repeat, and the ones that matter most are the ones nobody bothered to type.
So we asked a model to build one
There was no starting list. The first plugin through the classifier had an empty taxonomy and proposed the first category; every plugin since has either matched something that already existed or added to it. That has produced 14 categories and 51 sub-categories so far.
The interesting part was not getting an answer. A language model will always give you an answer. The interesting part was whether it gives you the same answer twice.
The first design was not consistent
It was not close. Re-running a plugin through the same model, with the same prompt, changed its top-level category two times in three. And it invented a brand-new category for roughly 40% of plugins, which means the taxonomy never stopped growing — and a taxonomy that never stops growing is just a tag cloud with extra steps.
The fix was structural rather than clever. The original design had the model look up candidate categories with a tool call and then place the plugin in a second call, so it was always choosing from a shortlist it had fetched a moment earlier. The prompt now carries the entire taxonomy, both levels, in the single call that does the placing. The model sees everything that exists before it decides whether anything fits.
After: 40 out of 40
- Top-level agreement on re-run went from about 1 in 3 to 40 out of 40, with a single sub-category flip.
- The new-category rate fell from a flat 40% to 15%.
- Cost per plugin halved, from two calls to one — a side effect, not the goal.
A similarity gate sits in front of every proposed category and folds a near-duplicate onto the existing row. It deliberately compares across category boundaries, so a sub-category proposed under the wrong parent gets matched to the right one instead of duplicating under the wrong one.
What we still cannot tell you
Consistency is not correctness. We measured whether the model agrees with itself; we did not measure whether it is right, and we cannot, because there is no ground-truth categorisation of the WordPress plugin directory to measure against. If there were, we would not have needed to build one.
Every placement is published so it can be judged. That is the only honest form of quality control available here.
The other caveat is coverage. Classification costs a model call per plugin, so it advances in batches — 114 plugins have been through it so far out of 73,049. The shape of the taxonomy is meaningful; the plugin counts inside it are counts of that sample, and the categories page states the coverage figure at the top rather than at the bottom.
Full method, including the prompt change and the three data-integrity bugs that cost real debugging time, is on the method page.