Agent skill security scanner

Audits an agent skill before installation: a static scan for prompt injection, hidden Unicode, secrets, risky scripts and structural traps, then a review of intent and permissions ending in a verdict. Use when a skill comes from an outside source.

SecurityPack: Sentry skills209 linesskill-scanner
Author
Source
getsentry/skills/skills/skill-scanner @ a582c51
License
Apache-2.0 · Copyright 2025 Functional Software, Inc. dba Sentry

About this skill

What it does

The skill audits an agent skill before you install it, in two layers. First scripts/scan_skill.py scans the folder and prints JSON: prompt-injection phrases, hidden Unicode (zero-width, right-to-left override and tag characters), API keys and private keys, risky script calls (HTTP requests, sockets, eval, shell=True, writes to CLAUDE.md, settings.json or git hooks), URLs outside a short trusted list, and structural traps: symlinks leading out of the folder, hooks in frontmatter, inline shell commands that run when the skill loads, conftest.py and other test files a runner would execute, postinstall scripts, text hidden in PNG metadata.

Then the agent judges the hits. The key question is whether a pattern attacks the agent or only describes an attack: a security skill listing injection phrases in references/ is documenting threats. The agent checks that the description matches what the instructions really do, looks for edits to agent config and memory, scope creep and needless data gathering, reads every script in full and rates allowed-tools on a six-tier scale from read-only to *. Only high-confidence findings are reported; medium ones go under "Needs verification". The report ends with a verdict: safe to install, install with caution, or do not install.

Three references back these steps: prompt-injection-patterns.md with a false-positive guide, dangerous-code-patterns.md, and permission-analysis.md with the tool tiers.

When to use it, and when not

Use it before installing a skill from a catalog, a colleague or a pull request, or to audit a repository's whole skill tree ("scan all skills").

It reviews skills, not applications. Pattern matching plus a reading of the instructions raises or lowers confidence; it cannot prove a skill safe.

How to run it

The install command below puts the whole folder in place. The scanner needs the uv CLI, which also pulls its one dependency, PyYAML; if the script fails, the agent falls back to grep with the reference patterns. The agent picks the skill up by its description; to call it yourself:

  • Claude Code: /skill-scanner, or ask for the skill by name.
  • Codex: $skill-scanner.
  • Cursor: type / in the Agent chat and pick skill-scanner.

Example: /skill-scanner Scan ./downloads/pdf-tools before I copy it into .claude/skills.

How it differs from similar skills

Nothing in our catalog does the same job. Security review, Sentry looks for exploitable bugs in application code and has no notion of instructions aimed at the agent. The quick_validate.py in Skill writer by the Agent Skills spec checks only structure: frontmatter, name, referenced files. Claude Code settings audit proposes permissions for a repository, not a check of the skills you add to it.

Install

Run one command — it saves SKILL.md and the files it uses into your agent's skills folder. Claude Code and Codex pick up a new skill without a restart; if it does not show up, restart the agent.

Claude Code — macOS, Linux
curl -fsSL --create-dirs -o ~/.claude/skills/skill-scanner/SKILL.md https://anymodel.org/en/skills/sentry/skill-scanner/SKILL.md -o ~/.claude/skills/skill-scanner/LICENSE https://anymodel.org/en/skills/sentry/skill-scanner/LICENSE -o ~/.claude/skills/skill-scanner/references/dangerous-code-patterns.md https://anymodel.org/en/skills/sentry/skill-scanner/references/dangerous-code-patterns.md -o ~/.claude/skills/skill-scanner/references/permission-analysis.md https://anymodel.org/en/skills/sentry/skill-scanner/references/permission-analysis.md -o ~/.claude/skills/skill-scanner/references/prompt-injection-patterns.md https://anymodel.org/en/skills/sentry/skill-scanner/references/prompt-injection-patterns.md -o ~/.claude/skills/skill-scanner/scripts/scan_skill.py https://anymodel.org/en/skills/sentry/skill-scanner/scripts/scan_skill.py
Codex, Cursor — macOS, Linux
curl -fsSL --create-dirs -o ~/.agents/skills/skill-scanner/SKILL.md https://anymodel.org/en/skills/sentry/skill-scanner/SKILL.md -o ~/.agents/skills/skill-scanner/LICENSE https://anymodel.org/en/skills/sentry/skill-scanner/LICENSE -o ~/.agents/skills/skill-scanner/references/dangerous-code-patterns.md https://anymodel.org/en/skills/sentry/skill-scanner/references/dangerous-code-patterns.md -o ~/.agents/skills/skill-scanner/references/permission-analysis.md https://anymodel.org/en/skills/sentry/skill-scanner/references/permission-analysis.md -o ~/.agents/skills/skill-scanner/references/prompt-injection-patterns.md https://anymodel.org/en/skills/sentry/skill-scanner/references/prompt-injection-patterns.md -o ~/.agents/skills/skill-scanner/scripts/scan_skill.py https://anymodel.org/en/skills/sentry/skill-scanner/scripts/scan_skill.py
Claude Code — Windows PowerShell
foreach ($f in 'SKILL.md','LICENSE','references/dangerous-code-patterns.md','references/permission-analysis.md','references/prompt-injection-patterns.md','scripts/scan_skill.py') { $p = Join-Path "$HOME\.claude\skills\skill-scanner" $f; New-Item -ItemType Directory -Force (Split-Path $p) | Out-Null; Invoke-WebRequest "https://anymodel.org/en/skills/sentry/skill-scanner/$f" -UseBasicParsing -OutFile $p }
Codex, Cursor — Windows PowerShell
foreach ($f in 'SKILL.md','LICENSE','references/dangerous-code-patterns.md','references/permission-analysis.md','references/prompt-injection-patterns.md','scripts/scan_skill.py') { $p = Join-Path "$HOME\.agents\skills\skill-scanner" $f; New-Item -ItemType Directory -Force (Split-Path $p) | Out-Null; Invoke-WebRequest "https://anymodel.org/en/skills/sentry/skill-scanner/$f" -UseBasicParsing -OutFile $p }

Any other agent: save the file into its skills folder, or give it the link to the file: https://anymodel.org/en/skills/sentry/skill-scanner/SKILL.md

SKILL.md

Raw

Skill Security Scanner

Scan agent skills for security issues before adoption. Detects prompt injection, malicious code, excessive permissions, secret exposure, and supply chain risks.

Requires: The uv CLI for python package management, install guide at https://docs.astral.sh/uv/getting-started/installation/

Important: Run all scripts from the repository root. Script paths like scripts/scan_skill.py are relative to this skill's root directory (the directory containing this SKILL.md), not relative to the target repository.

Bundled Script

scripts/scan_skill.py

Static analysis scanner that detects deterministic patterns. Outputs structured JSON.

uv run scripts/scan_skill.py <skill-directory>

Returns JSON with findings, URLs, structure info, and severity counts. The script catches patterns mechanically — your job is to evaluate intent and filter false positives.

Workflow

Phase 1: Input & Discovery

Determine the scan target:

  • If the user provides a skill directory path, use it directly
  • If the user names a skill, look for it under .agents/skills/<name>/ first, then other established layouts such as skills/<name>/ when the repo uses a canonical root skill tree, .claude/skills/<name>/, plugins/*/skills/<name>/, or another repo-managed skill root with clear prior art
  • If the user says "scan all skills", discover all */SKILL.md files and scan each

Validate the target contains a SKILL.md file. List the skill structure:

ls -la <skill-directory>/
ls <skill-directory>/references/ 2>/dev/null
ls <skill-directory>/scripts/ 2>/dev/null

Phase 2: Automated Static Scan

Run the bundled scanner:

uv run scripts/scan_skill.py <skill-directory>

Parse the JSON output. The script produces findings with severity levels, URL analysis, and structure information. Use these as leads for deeper analysis.

Fallback: If the script fails, proceed with manual analysis using Grep patterns from the reference files.

Phase 3: Frontmatter Validation

Read the SKILL.md and check:

  • Required fields: name and description must be present
  • Name consistency: name field should match the directory name
  • Tool assessment: Review allowed-tools — is Bash justified? Are tools unrestricted (*)?
  • Model override: Is a specific model forced? Why?
  • Description quality: Does the description accurately represent what the skill does?

Phase 4: Prompt Injection Analysis

Load references/prompt-injection-patterns.md for context.

Review scanner findings in the "Prompt Injection" category. For each finding:

  1. Read the surrounding context in the file
  2. Determine if the pattern is performing injection (malicious) or discussing/detecting injection (legitimate)
  3. Skills about security, testing, or education commonly reference injection patterns — this is expected

Critical distinction: A security review skill that lists injection patterns in its references is documenting threats, not attacking. Only flag patterns that would execute against the agent running the skill.

Phase 5: Behavioral Analysis

This phase is agent-only — no pattern matching. Read the full SKILL.md instructions and evaluate:

Description vs. instructions alignment:

  • Does the description match what the instructions actually tell the agent to do?
  • A skill described as "code formatter" that instructs the agent to read ~/.ssh is misaligned

Config/memory poisoning:

  • Instructions to modify CLAUDE.md, MEMORY.md, settings.json, .mcp.json, or hook configurations
  • Instructions to add itself to allowlists or auto-approve permissions
  • Writing to ~/.claude/, ~/.agents/, or any agent configuration directory
  • Scripts that append to global config files — the poisoned instructions persist after skill removal

Scope creep:

  • Instructions that exceed the skill's stated purpose
  • Unnecessary data gathering (reading files unrelated to the skill's function)
  • Instructions to install other skills, plugins, or dependencies not mentioned in the description

Information gathering:

  • Reading environment variables beyond what's needed
  • Listing directory contents outside the skill's scope
  • Accessing git history, credentials, or user data unnecessarily

Structural attacks (check scanner output for these):

  • Symlinks: Files that resolve outside the skill directory — can disguise reads of ~/.ssh/id_rsa, ~/.aws/credentials, etc. as "example" files
  • Frontmatter hooks: PostToolUse/PreToolUse hooks in YAML — execute shell commands automatically, the model cannot prevent it
  • !command`` syntax: Runs shell commands at skill load time during template expansion, before the model sees the prompt
  • Test files: conftest.py, test_*.py, *.test.js — test runners auto-discover and execute these as side effects of pytest or npm test
  • npm lifecycle hooks: postinstall scripts in bundled package.json — run automatically on npm install
  • Image metadata: PNG files with text in metadata chunks (tEXt/iTXt) — multimodal LLMs can read hidden instructions from image metadata

Phase 6: Script Analysis

If the skill has a scripts/ directory:

  1. Load references/dangerous-code-patterns.md for context
  2. Read each script file fully (do not skip any)
  3. Check scanner findings in the "Malicious Code" category
  4. For each finding, evaluate:
    • Data exfiltration: Does the script send data to external URLs? What data?
    • Reverse shells: Socket connections with redirected I/O
    • Credential theft: Reading SSH keys, .env files, tokens from environment
    • Dangerous execution: eval/exec with dynamic input, shell=True with interpolation
    • Config modification: Writing to agent settings, shell configs, git hooks
  5. Check PEP 723 dependencies — are they legitimate, well-known packages?
  6. Verify the script's behavior matches the SKILL.md description of what it does

Legitimate patterns: gh CLI calls, git commands, reading project files, JSON output to stdout are normal for skill scripts.

Phase 7: Supply Chain Assessment

Review URLs from the scanner output and any additional URLs found in scripts:

  • Trusted domains: GitHub, PyPI, official docs — normal
  • Untrusted domains: Unknown domains, personal sites, URL shorteners — flag for review
  • Remote instruction loading: Any URL that fetches content to be executed or interpreted as instructions is high risk
  • Dependency downloads: Scripts that download and execute binaries or code at runtime
  • Unverifiable sources: References to packages or tools not on standard registries

Phase 8: Permission Analysis

Load references/permission-analysis.md for the tool risk matrix.

Evaluate:

  • Least privilege: Are all granted tools actually used in the skill instructions?
  • Tool justification: Does the skill body reference operations that require each tool?
  • Risk level: Rate the overall permission profile using the tier system from the reference

Example assessments:

  • Read Grep Glob — Low risk, read-only analysis skill
  • Read Grep Glob Bash — Medium risk, needs Bash justification (e.g., running bundled scripts)
  • Read Grep Glob Bash Write Edit WebFetch Task — High risk, near-full access

Confidence Levels

Level Criteria Action
HIGH Pattern confirmed + malicious intent evident Report with severity
MEDIUM Suspicious pattern, intent unclear Note as "Needs verification"
LOW Theoretical, best practice only Do not report

False positive awareness is critical. The biggest risk is flagging legitimate security skills as malicious because they reference attack patterns. Always evaluate intent before reporting.

Output Format

## Skill Security Scan: [Skill Name]

### Summary
- **Findings**: X (Y Critical, Z High, ...)
- **Risk Level**: Critical / High / Medium / Low / Clean
- **Skill Structure**: SKILL.md only / +references / +scripts / full

### Findings

#### [SKILL-SEC-001] [Finding Type] (Severity)
- **Location**: `SKILL.md:42` or `scripts/tool.py:15`
- **Confidence**: High
- **Category**: Prompt Injection / Malicious Code / Excessive Permissions / Secret Exposure / Supply Chain / Validation
- **Issue**: [What was found]
- **Evidence**: [code snippet]
- **Risk**: [What could happen]
- **Remediation**: [How to fix]

### Needs Verification
[Medium-confidence items needing human review]

### Assessment
[Safe to install / Install with caution / Do not install]
[Brief justification for the assessment]

Risk level determination:

  • Critical: Any high-confidence critical finding (prompt injection, credential theft, data exfiltration)
  • High: High-confidence high-severity findings or multiple medium findings
  • Medium: Medium-confidence findings or minor permission concerns
  • Low: Only best-practice suggestions
  • Clean: No findings after thorough analysis

Reference Files

File Purpose
references/prompt-injection-patterns.md Injection patterns, jailbreaks, obfuscation techniques, false positive guide
references/dangerous-code-patterns.md Script security patterns: exfiltration, shells, credential theft, eval/exec
references/permission-analysis.md Tool risk tiers, least privilege methodology, common skill permission profiles

Connect through AnyModel API

This skill runs on whatever model your agent uses. Point Claude Code, Codex or Cursor at AnyModel to run it on Claude, GPT, Gemini and more with one key — pay per token, no subscription.

More from the pack Sentry skills

AGENTS.md and CLAUDE.md writer

Writes or trims a repository's AGENTS.md to under 60 lines of commands, conventions and pointers to existing docs, checks every path, and makes CLAUDE.md a symlink to it. Use when a repo needs agent instructions or its current file has grown long or stale.

Claude Code settings audit

Detects a repository's stack and proposes a Claude Code settings.json allow list of read-only commands for the tools it actually uses, plus docs domains and Sentry or Linear MCP entries. Use when setting up Claude Code permissions or auditing existing ones.

Code review, Sentry practices

Reviews a change by Sentry's guidelines: runtime errors, performance and N+1 queries, side effects, compatibility, security, design and tests, with a polite tone and approval once only minor issues remain. Use when reviewing a pull request or branch.

Code simplifier

Simplifies recently changed code without changing behaviour: less nesting, no nested ternaries, clearer names, no redundant abstractions, while following CLAUDE.md conventions. Use when code works but reads heavier than it should and needs a cleanup pass.

Conventional commit messages

Commits in Sentry's Conventional Commits style: a feature branch instead of main, a typed capitalised subject up to 70 characters, Fixes or Refs footers, no personal data. Use when asked to commit changes or draft a commit message.

Django access control review

Audits Django and DRF code for IDOR and broken access control by learning the app's authorization model and tracing each endpoint from ID to query, reporting only confirmed gaps with enforcing fixes. Use when reviewing views, viewsets or tenant isolation.

Find bugs in branch changes

Reviews the commits on the current branch for security holes, bugs and quality issues with an eleven-item checklist and reports verified findings with file, line, severity and fix, without editing code. Use when you want a security-first pass before a PR.

GitHub Actions security review

Audits GitHub Actions workflows for attacks an outsider without write access can run: pwn requests, expression injection, comment commands, token theft, poisoned agent configs, unpinned actions. Use when reviewing CI security or changing workflows.

Iterate a PR until CI is green

Drives an open PR to green: fixes failing CI and high and medium review comments, verifies locally, commits, pushes and repeats, stopping at human approval or draft gates. Use when a PR is red or has review feedback to work through.

PR title and description writer

Writes or refreshes a PR title in Sentry's conventional format and a short reviewer-facing description without template headings or test logs, then opens a draft or updates the PR. Use when opening a PR or after commits change its scope.

Prompt optimizer

Writes, repairs or ports system and agent prompts against a small eval set, comparing candidate rewrites on the same cases and checking the winner on held-out ones. Use when a prompt is unreliable, moves to another model family or needs evals.

Secret leaks through serialization

Finds secrets that leak when a credential field on a dataclass, pydantic model or JS object meets code that logs or traces whole objects, searching the whole repository for the other half. Use when a change adds such a field or a wholesale sink.

SECURITY_MODEL.md for review tools

Writes and maintains SECURITY_MODEL.md, telling security review tools a repository's exposure, invariants, reportable finding classes and owner-confirmed exclusions, with a script that flags stale references. Use when preparing a repo for automated review.

Skill writer by the Agent Skills spec

Creates and improves agent skills by Sentry's workflow: sources with provenance, the simplest execution shape, SPEC.md and SOURCES.md, precision passes after each change and a structural validator. Use when building a skill that a team will maintain.

Security review, Sentry

Reviews a file, diff or component for exploitable vulnerabilities with OWASP-based references, researches the codebase before flagging and reports only findings reached by attacker-controlled input. Use when a change needs a low-noise security review.

Django performance review

Reviews Django and DRF code for N+1 queries, unbounded querysets, missing indexes and write loops, validating table size and hot paths before reporting. Use when a Django view or endpoint is slow or a change touches large tables.