Skill Dedup Scanner 🔍

Detects duplicate or similar skills that may cause model confusion.

When to Use

• Before publishing a new skill
• When skills trigger incorrectly
• Auditing existing skill collection
• Resolving naming conflicts
• Checking for similar skill names/descriptions

Usage

Basic Scan

# Auto-detect language
python3 scripts/main.py ~/.openclaw/workspace/skills/

# Specify language
python3 scripts/main.py ~/.openclaw/workspace/skills/ --lang en
python3 scripts/main.py ~/.openclaw/workspace/skills/ --lang zh

Advanced Options

# Custom threshold (0-1)
python3 scripts/main.py ~/.openclaw/workspace/skills/ --threshold 0.8

# Output to file
python3 scripts/main.py ~/.openclaw/workspace/skills/ -o audit_report.md

# JSON output
python3 scripts/main.py ~/.openclaw/workspace/skills/ --json

# Verbose mode
python3 scripts/main.py ~/.openclaw/workspace/skills/ -v

Options

Output

Generates a detailed report with:

• Similarity scores (name + description)
• Conflict analysis
• Actionable recommendations
• List of safe/unique skills

Similarity Thresholds

Language Support

• English (--lang en)
• Chinese (--lang zh)
• Auto-detect (--lang auto) - Detects system locale

Examples

Example 1: Quick Scan

$ python3 scripts/main.py ~/.openclaw/workspace/skills/
🔍 Scan Directory: /home/user/.openclaw/workspace/skills
📦 Total Skills: 15

✅ No Duplicate Skills Found

All skills have distinct names and descriptions. No optimization needed.

Example 2: Find Conflicts

$ python3 scripts/main.py ~/.openclaw/workspace/skills/ --threshold 0.6
🔍 Scan Directory: /home/user/.openclaw/workspace/skills
📦 Total Skills: 15

⚠️ Found 2 Groups of Similar Skills

### 1. 🔴 High Similarity Warning

**Skill A:** `tushare-finance`
**Skill B:** `stock-analyzer`

**Similarity Score:** 85%

**Analysis:**
- Name Similarity: 60%
- Description Similarity: 85%

**Recommendations:**
- Clarify the distinction in descriptions
- Rename to make purposes more distinct

Project Structure

skill-dedup-scanner/
├── scripts/
│   ├── main.py                 # Main entry point
│   ├── skill_scanner.py        # Skill scanner
│   ├── similarity_checker.py   # Similarity calculator
│   ├── report_generator.py     # Report generator
│   └── locale_loader.py        # Multi-language support
├── locales/
│   ├── en.json                 # English translations
│   └── zh.json                 # Chinese translations
├── references/
│   └── best_practices.md       # Skill naming best practices
└── output/                     # Generated reports

Requirements

• Python 3.7+
• PyYAML (pip install pyyaml)

License

MIT