How the Skill Score is computed

When a skill has been benchmarked, the Skill Score is a weighted composite of an effectiveness measurement plus three hygiene sub-scores, each rated 0 to 100. Until then it is provisional (see below). Nothing is secret — we publish the rubric because a score whose method is hidden is just an opinion with a logo on it.

Benchmarked: 0.40·effectiveness + 0.25·safety + 0.15·maintenance + 0.15·documentation + 0.05·adoption.

Provisional (not yet benchmarked): 0.40·safety + 0.30·maintenance + 0.30·documentation — popularity zeroed, and the listing is clearly labelled provisionalso you know effectiveness hasn't been measured yet. We never invent an effectiveness number. The band — High Quality, Solid, or Caution — comes from the overall: ≥7.5, ≥5.5, otherwise low. Both regimes rescale to the 0–10 display range.

Effectiveness is the part GitHub can't replicate, so it is worth explaining in full. For each benchmarked skill we assemble a small suite of verifiable tasks in its capability — contract clauses with a known risk, invoices with known line items, code with a planted bug, datasets with a known answer. Then, for every task:

• Arm A runs the skill (executed in a sandbox where we can, otherwise faithfully reproduced from its instructions — the page says which).
• Arm B gives the same frontier model a strong, fair baseline prompt and no skill.
• A separate judge model compares A and B blind, scoring correctness, completeness, specificity, and format. We run the comparison twice with the order flipped and only count a win if the same arm wins both times.

The skill's win rate over the baseline, with a 95% confidence interval, is its effectiveness. We pre-register the bar for “materially better” before running, so we can't move the goalposts. We also report reliability (does it win consistently across repeated trials, or only sometimes?) and cost — a 1% quality gain at 10× the cost is not a win.

Crucially, this catches the skill that is really just a good prompt. If the skill's edge disappears once we hand its own instructions to the bare model, we say so on the page: most of this skill's value is its prompt. That is the most honest, least gameable thing we can tell you, and it is the whole point of the number.

The Skill Score answers "How good is this skill?", not "Can I trust this person?". It is a quality signal — our opinion, derived from public signals — about whether the skill does what it says, whether it does it well, and whether it will keep working as the surrounding ecosystem changes.

We are explicit about that framing because the score is not a security guarantee. The safety sub-score reads code statically and flags the most common patterns of trouble; it cannot catch a skill that does something dangerous only when triggered by a specific input. Treat the score as a strong signal that lets you skip the obviously bad skills and prioritise the obviously good ones — not as a substitute for reading the install command before running it.

For each indexed skill we collect a fixed set of signals from the source repository and supporting registries. Nothing we publish on a skill page is invented; every fact comes from one of these places.

• Source URL, license file, and topic tags from the canonical repository.
• Stars, forks, contributor count, open and closed issue counts.
• Last commit date and release-tag cadence.
• The README and any marker file (SKILL.md, mcp.json, .cursorrules, .windsurfrules, etc.).
• The skill's own source code, read for safety review.
• Package-registry metadata (npm, PyPI) where the skill is also published.
• Reviews left by verified installers through this site, once the skill has accumulated any.

A skill can carry a flag at one of three severities. Flags are surfaced at the top of the skill page so you see them before you copy an install command.

The effectiveness benchmark measures output quality on representativetasks — not your exact task. The score tells you whether a skill tends to beat the model alone and is safe to run; it can't predict whether it's the right fit for your specific workflow.

• We grade output quality by blind comparison against a baseline — not aesthetic or stylistic preference. There is no “we liked the tone” sub-score.
• We don't compare paid skills against free ones on cost alone. License and pricing are surfaced as facts, not weighted into the score.
• We don't penalise low star counts on their own. A new skill with strong maintenance and clean documentation can sit at High Quality.
• We never let a vendor pay for a higher score, and the score is recomputed on a schedule independent of any commercial relationship.

The score feeds into a tier decision. Three tiers exist:

• Tier A — listed and indexed by search engines. Requires complete metadata, a valid license, no critical flags, an overall score of 5.5 or higher, and at least one real-world signal (≥25 stars, recent commits within 180 days, presence in a known registry, or a verified submission).
• Tier B — listed in our directory but noindex until it earns indexing. A real skill that hasn't yet accumulated enough signal to be ranked.
• Tier C — rejected. Broken, empty, joke, malicious, near-duplicate of a canonical entry, or carrying a critical safety flag.

Borderline decisions go to a moderation queue rather than getting auto-published. That is the only manual step in the pipeline — every other decision is reproducible from the published rubric.

Every Tier-A skill is re-scored at least weekly. Repository signals (stars, commits, issues) refresh nightly. Skill scans re-run when the source code's content hash changes, so a skill that ships a security fix sees its score move quickly. When a skill's tier or critical-flag status changes, the affected listing pages rebuild automatically.

The "Last verified" date on each skill page is the timestamp of its most recent successful refresh. If you see a date older than two weeks, the refresh job failed for that skill and we surface that as an info flag.

A few things to keep in mind so the score is useful rather than oversold:

• The safety review reads code statically. A skill that does dangerous things only when it sees a specific input may pass review and still be unsafe. Treat the score as a strong signal, not a guarantee.
• Documentation grading uses a language model. We bias toward false positives there — a poorly-described skill is easier to live with than a missed-warning one.
• For skills that aren't open source — paid Custom GPTs, OpenAI Apps, hosted Zapier templates — we score what we can see. The skill's actual implementation is closed, so the safety sub-score weights the platform's vendor-level controls rather than line-level static analysis.
• Brand-new skills with no signal default to Tier B until they accumulate either real-user activity or a verified author submission. A new skill being at Tier B is not a judgment about its quality.