YDMmobile / oss

I Vendored Two Third-Party Claude Code Skills. A 49-Day Census Found Zero Calls.

2026-10-21

A counter-list to the "plugins every developer should install" posts: five things my own transcripts taught me

My feed keeps serving me lists of Claude Code plugins and skills I apparently cannot live without. I read them. I also install things: at my 2026-09-03 census, my global skills folder held 36 user skills.

So here is a different kind of list. Not "what to install", but what happened after I did — measured, not remembered. The short version: the skills I chose carefully were never called, and the newest one is a copy I had to fix myself.

1. The filter passed. The census failed.

On 2026-07-11 I reviewed MengTo/Skills, an MIT-licensed collection of 75 agent skills. Roughly 90% of it was web- or Codex-specific — Tailwind, GSAP, Three.js, Netlify, plus account automations for Gmail and X. I write mostly mobile code, so I filtered hard: per skill, not per collection, keeping only what would make sense in any repo.

Two survived: design-first-ui-prompting and video-to-superprompt. I symlinked them on 07-11 and replaced the symlinks with real copies on 07-15, because my ~/.claude is a git repo and copies survive a machine move while symlinks into a throwaway clone do not. Each copy ended with a Provenance section: "Vendored from MengTo/Skills (MIT) on 2026-07-15; maintained locally since". Responsible. Tidy. Very proud of myself.

The first warning came on 2026-08-19. Across 2,200 transcripts from 2026-07-10 onward, holding 319 Skill calls, neither skill had been invoked once. Neither had eight other skills in my folder, and together those ten descriptions cost about 810 tokens in every session. So I made all ten user-invocable only, which removed them from the model's skill list until I typed their slash command.

That 08-19 count is the solid evidence, because both skills were still model-invocable for the whole stretch it measured.

Then, on 2026-09-03, I ran a proper skill census: a parse of every transcript under ~/.claude/projects/ — 2,605 files covering 2026-07-16 to 2026-09-03 — counting model-initiated Skill calls per skill. Both skills landed on its zero-calls list. (For the last 15 days of that window they were slash-only, so the model could not have called them anyway — which is exactly the kind of zero item 2 is about.)

They went to _gc_trash/2026-09-03/ rather than being deleted, so the undo is one mv. They have not come back.

Since then, adoption has been two gates instead of one. The first gate (my judgement at install time) decides what gets in. The second gate (a measured call count) decides what stays. A skill can pass the first on taste and fail the second on arithmetic.

2. A zero over three days is not a zero

The census did not only catch vendored skills. It soft-deleted 14 user skills with zero calls in the window.

Less than two hours later I restored five of them. All five had been added between 2026-08-28 and 08-31, so the 49-day window covered three to six days of their life. Zero calls in three days says nothing. The nine that stayed retired had existed for 18 to 75 days of the window and still drew nothing.

One of the restored five is audit-reference-originality — the only MengTo-derived skill I still have, and it did not even come from my careful review. It was ported from my Codex skills folder on 2026-08-28, with its own Provenance section pinning the upstream commit. (So the one MengTo skill that survived is the one I did not pick through the filter. Make of that what you will.)

So a census number only counts when you divide it by how long the skill existed. Read the dates in your trash folder, not just the counts.

3. Project-scoped is not the same as used

This one stung more, because I did it "right".

ACT (agentic-coding-toolkit) is a paid, Flutter-only toolkit — 39 skills and 3 agents in the checkout I had. By the time I measured it, it was installed per project and turned on in 16 repositories. Exactly the kind of careful scoping a good install guide recommends.

Across every transcript retained from 2026-07-18 to 2026-09-02, a full grep with no truncation found zero act: Skill-tool calls. That is 46 days, 16 repos, nothing. My own honest read afterwards: I had really only used it for its spec interview, and the brainstorming and plan-writing skills I already had covered that. I retired it on 2026-09-02.

To be clear, that is a fact about my usage, not a review of the toolkit. It just did not fit how I actually work.

Scoping limits how much an unused skill can confuse other sessions, but it proves nothing about whether you will reach for it. Only the census does that.

4. Upstream was wrong, so my copy is not upstream's

The most recent skill I vendored is apple-design, from emilkowalski/skills at commit e8a175d, on 2026-10-11. This time I did not copy it verbatim.

Checking it on 2026-10-11 turned up two defects that break at runtime. Springs set by duration or bounce drop the gesture velocity in Motion 14.1.0, and a transform: none !important reduced-motion rule left a closed sheet on screen in headless Chrome. There were also several misattributed sources.

So the copy carries the fixes, and says so in its own frontmatter (read 2026-10-12):

disable-model-invocation: true
metadata:
  category: design-media
  origin: emilkowalski/skills (MIT), skills/apple-design at e8a175d; vendored 2026-10-11; differs from upstream only by the corrections listed under "Local changes" at the end and by prettier formatting

I reported the fixes upstream as issue #32 instead of a PR. As of 2026-10-11, that repo had 17 PRs and had merged no outside PR that changed skill content; the three earlier apple-design PRs were closed without comment within one minute on 2026-08-09. Issue reports, on the other hand, were sometimes applied by the maintainer. So: an issue with runnable evidence, not a PR.

A corrected copy also adds a third obligation. Before re-syncing from upstream, check #32 and diff upstream against e8a175d, so a lazy sync does not quietly reinstall the bugs. And if /apple-design draws no calls, it goes on the next census's retirement list like everything else.

5. Install count is a first filter, not evidence

I also have a find-skills skill. Its rules are not mine — they are its author's — and, as of 2026-10-12, they read like a "top N plugins" post turned into policy:

1. **Install count** — Prefer skills with 1K+ installs. Be cautious with anything under 100.
2. **Source reputation** — Official sources (`vercel-labs`, `anthropics`, `microsoft`) are more trustworthy than unknown authors.
3. **GitHub stars** — Check the source repository. A skill from a repo with <100 stars should be treated with skepticism.

It also suggests checking a popularity leaderboard before searching, and its install command adds -g -y: install globally, skip the confirmation.

Popularity tells you that other people installed something. It does not tell you that it is correct (see item 4), that it fits your stack (see item 3), or that you will ever call it (see items 1 and 2). And "globally, no prompt" is the opposite of how I want anything to arrive.

These days I treat install counts as a way to shorten a search, not as a reason to trust anything. And find-skills itself was one of the ten skills I made user-invocable only on 2026-08-19, so it runs when I ask for it and not when a question merely sounds like "how do I do X".

Lessons Learned

  1. Start every vendored skill slash-only. disable-model-invocation: true takes the skill out of the model's list, so it only runs when you type the slash command, and its description stops costing tokens in every session. My 2026-09-23 config review found that every remaining zero-call skill was already slash-only, so they cost no context while they wait to be measured.
  2. Keep provenance in the file, not in your head. Upstream repo, commit, license, and what you changed. Future you will re-sync without remembering why.
  3. Retire, don't delete. A trash folder with dated subfolders made the five wrong calls in item 2 a same-day mistake instead of a re-download.
  4. Count both kinds of call. A slash-only skill is called through its slash command, not by the model, so a model-call count alone will call it unused.

If you want a rough first pass on your own machine, these two greps worked on my transcripts on 2026-10-12, with Claude Code 2.1.296. They are crude (they match one JSON key order and one tag shape, and the second also lists built-in commands like /clear), but they will tell you which skills you actually reach for:

# Skill-tool calls (skills the model invoked)
grep -rhoE '"name":"Skill","input":\{"skill":"[^"]+"' ~/.claude/projects --include='*.jsonl' | sort | uniq -c | sort -rn

# Slash-command calls (skills you invoked)
grep -rhoE '<command-name>/[^<]+</command-name>' ~/.claude/projects --include='*.jsonl' | sort | uniq -c | sort -rn

Then compare the output against your skills folder. Whatever is in the folder and missing from both lists is your own "plugins every developer should install" — the ones you installed and never called.

What is the oldest skill in your setup that you have never called? Run the greps before you answer, and tell me in the comments if the result surprised you.