Methodology

How Lightyear evaluates a Claude Code setup.

No black box. Everything below is derived from constants in the codebase. Current analysis model: claude-opus-4-7. Prompt version: v6.

1 · The pipeline

01Sign inGitHub OAuth via Convex Auth. Your identity is the GitHub numeric id; no password storage.
02Zipcd ~ && zip -r dotclaude.zip .claude. The browser unpacks the zip locally — nothing leaves your machine yet.
03WalkJS-side filter keeps the 14 canonical paths, strips binaries / oversize files / per-plugin cache + source. Hard caps: 2 MB per file, 500 files, 10 MB total.
04Scrub28+ regex patterns detect secrets. Blocks are always redacted (and server-rejected). Warns default to keep (mock paths, demo env vars rarely contain real secrets); you can flip them.
05Hash & uploadsha256 over the canonical-serialized bundle and (bundle_hash + tldr + loomUrl + prompt_version + model). The blob goes to Convex storage; the analysis_hash gates idempotency.
06VerifyServer re-hashes the stored blob; mismatch → reject. Asserts tldr/loomUrl are already canonical; refuses non-canonical input.
07Structural passBucket each file into the matching canonical category, count items / bytes / lines.
08Claude reviewclaude-opus-4-7 via Anthropic tool-use. One schema-bound submit_analysis tool — no free-form text. Untrusted bundle content is wrapped in sentinel markers and a policy preamble.
09Post-parseZod validates shape + enums + min lengths. clampAnalysis truncates near-edge string overshoots. Scrubber runs over Claude's output prose as defense in depth.
10StoreReports table upserts per submission. status: error → revivable in place on resubmit.

2 · Detected upload root

The walker detects what kind of folder you zipped. The score and the prompt then treat unreachable categories as N/A, not as missing — so submitting only your home ~/.claude doesn't penalize you for the lack of a project-level CLAUDE.md.

project_root

Project root

Has CLAUDE.md, CLAUDE.local.md, .claude/, or .mcp.json at top level.

Reaches: All 14 categories.

dotclaude_dir

Bare .claude/ dir

Top-level entries match .claude internals (hooks/, commands/, skills/, agents/, settings.json…) with no enclosing project.

Reaches: 10 categories under .claude/. Project-root artifacts (CLAUDE.md, .mcp.json, .gitignore) are N/A.

user_dotclaude

User-level ~/.claude

Top-level contains user-only signals like projects/, sessions/, paste-cache/, file-history/.

Reaches: Same 10 .claude/ categories. The four project-root-only entries are explicitly excluded from scoring.

3 · The 14 canonical categories

The bar. Weights sum to 100 by construction. Coverage scoring divides only by weights reachable from your detected root.

Path	Kind	Weight	Target	Why it matters
`CLAUDE.md`	file	12	1	Project memory that survives every session and steers Claude's behavior across the whole repo.
`CLAUDE.local.md`	file	4	1	Personal overrides for your machine — gitignored — for paths, tokens, and quirks unique to you.
`.gitignore`	file	2	1	Prevents local files (and secrets) from leaking into the repo's history.
`.mcp.json`	file	8	1	Project-scoped MCP server configuration — extends Claude with custom tools beyond its built-ins.
`.claude/hooks/`	dir	9	3	Deterministic shell hooks (PostToolUse, SessionStart, PreCompact) that enforce policy automatically.
`.claude/commands/`	dir	10	5	Slash commands — reusable prompts and workflows you trigger by name.
`.claude/skills/`	dir	12	5	Model-readable capabilities Claude auto-invokes when the situation matches.
`.claude/agents/`	dir	10	4	Subagents with isolated context — used for code review, log triage, and protecting the main thread.
`.claude/output-styles/`	dir	6	2	Custom response styles — bend Claude's voice and brevity to match the surface.
`.claude/plugins/`	dir	9	2	First-class plugin bundles — commands, agents, and skills versioned together.
`.claude/rules/`	dir	6	3	Path-scoped policy that loads only when files in that subtree are touched.
`.claude/statusline`	file	3	1	Bottom-bar UI customization — at-a-glance status, model, branch, cost.
`.claude/settings.json`	file	5	1	Permissions, model, hook registry — the spine of Claude Code's behavior in this repo.
`.claude/settings.local.json`	file	4	1	Personal, gitignored settings layered on top of settings.json.

4 · Coverage score formula

Deterministic. Claude does not see or set this number — it's pure structure + weights.

per_category(c)
  = present
    ? min(1, log1p(itemCount) / log1p(targetCount)) * weight
    : 0

earned   = Σ per_category(c) for c in reachable(rootKind)
possible = Σ weight(c)        for c in reachable(rootKind)

score = clamp(0..100, round((earned / possible) * 100, 1))

The log1p ramp means going from 0 → 1 item buys most of the credit; the next few items round it up; piling on past target adds nothing. Designed to reward having any of a category at all without inflating expert tiers further.

5 · Sophistication tier

Claude picks one of these three. Not deterministic — it's a structured tool-use output, validated against the same enum the schema enforces.

Exploratory

Trying things out. Few categories populated, mostly defaults. Low depth.

Intentional

Deliberate, layered choices. Multiple categories populated. Patterns visible across files.

Expert

Self-sustaining system. Hooks enforce policy, agents protect context, skills auto-fire, plugins bundle workflows.

6 · Secret scrubbing

The scrubber registry runs in two places: in your browser before upload, and in the Convex action before Claude sees anything. Same patterns, no drift.

— Anthropic / OpenAI / Google / Hugging Face / Replicate keys
— GitHub / GitLab personal tokens, gh* prefixes
— AWS access + secret keys (with adjacent env-var context)
— Stripe live + test keys, Slack tokens + webhooks
— Supabase service_role JWTs (decoded payload check)
— PEM-format private keys (RSA, EC, OPENSSH)
— JWTs, bearer headers, URL-embedded tokens
— .mcp.json env maps, dotenv exports, high-entropy strings
— Absolute home paths rewritten to ~/

Block-severity matches are mandatory redactions and the server rejects a bundle that still contains any. Warn-severity (paths, demo env vars, email) defaults to keep — the false-positive rate on real .claude folders is high enough that auto-redacting them shreds signal. You can flip the whole warn class on the review screen.

7 · Idempotency

analysis_hash = sha256(
  bundle_hash + " " +
  canonicalize(tldr) + " " +
  canonicalize(loomUrl) + " " +
  PROMPT_VERSION + " " +
  ANALYSIS_MODEL
)

Same five inputs → same analysis_hash → preflight short-circuits to your existing report. Change any of them — re-zip, edit the TL;DR, paste a different Loom URL, model bump, prompt rewrite — and you get a fresh analysis. Errored runs revive in place on resubmit, preserving the URL.