/model-migrate-flagos <model_name> [upstream_folder] [plugin_folder]
| Argument | Required | Default |
|---|---|---|
| --- | --- | --- |
model_name | Yes | — |
upstream_folder | No | /tmp/vllm-upstream-ref |
plugin_folder | No | current working directory |
Extract from user input:
{{model_name}} = first argument (required, snake_case){{upstream_folder}} = second argument or /tmp/vllm-upstream-ref{{plugin_folder}} = third argument or current working directoryIf {{upstream_folder}} doesn't exist, ask user whether to clone it. If {{plugin_folder}} doesn't exist, error out.
→ Tell user: Confirm parsed model name and paths.
Read these files (relative to this SKILL.md):
references/procedure.md — step-by-step migration procedurereferences/compatibility-patches.md — 0.13.0 patch catalogreferences/operational-rules.md — communication, TaskList, bash rules, resilienceThe procedure references executable scripts in scripts/:
scripts/validate_migration.py — automated code review (Step 6)scripts/benchmark.sh — benchmark verification (Step 9)scripts/serve.sh — serve model locally (Step 10.1, also used for E2E)scripts/request.sh — test request (Step 10.2)scripts/e2e_eval.py — E2E correctness verification (Step 11)scripts/e2e_test_prompts.json — test prompts for E2E (5 text + 5 multimodal)scripts/e2e_config.template.json — E2E config template (copy to e2e_config.json and fill in)scripts/e2e_remote_serve.sh — manage GT server on remote machine via SSHThen investigate upstream source + HuggingFace to resolve all placeholders:
| Placeholder | How to derive |
|---|---|
| --- | --- |
{{model_name}} | Direct from argument |
{{model_name_lower}} | Lowercase of model_name (usually identical, e.g. qwen3_5) — used in file paths |
{{MODEL_DISPLAY_NAME}} | From upstream code or HF model card |
{{ModelClassName}} | From upstream model class (PascalCase) |
{{model_type}} | From HF config.json model_type field |
{{ConfigClassName}} | From upstream or derive from model_type |
{{skill_root}} | Absolute path to this skill's folder (the directory containing this SKILL.md) |
Naming conventions vary per model — always verify from actual source, never guess.
→ Tell user: Present all resolved values. Use AskUserQuestion if anything is ambiguous.
With placeholders resolved, execute every step in procedure.md sequentially. Apply patches from compatibility-patches.md during the copy-then-patch step. Follow operational-rules.md throughout.
→ Tell user: Before starting, output a numbered plan. Report progress at each step boundary.
| Script | Step | Description |
|---|---|---|
| --- | --- | --- |
validate_migration.py | 6 | Automated import/API/registration checks |
benchmark.sh | 9 | vllm bench throughput with dummy weights |
serve.sh | 10, 11 | Start local vLLM server (port 8122, VLLM_FL_PREFER_ENABLED=false) |
request.sh | 10 | Quick smoke-test request |
e2e_eval.py | 11 | Token-level comparison vs upstream GT server |
e2e_test_prompts.json | 11 | 5 text + 5 multimodal test prompts |
e2e_config.template.json | 11 | Config template (GT machine, local port, eval params) |
e2e_remote_serve.sh | 11 | SSH-based GT server lifecycle (start/stop/status/logs) |
Example 1: Typical new model
User says: "/model-migrate-flagos kimi_k25"
Actions:
1. Parse → model_name=kimi_k25, defaults for upstream/plugin paths
2. Clone upstream, find vllm/model_executor/models/kimi_k25.py
3. Discover it wraps DeepseekV2 → follow kimi_k25 (wrapper) pattern
4. Copy file, apply P1+P2 patches, create config bridge
5. Register, validate, test, benchmark, serve+request
6. E2E verification against upstream GT
Result: kimi_k25 fully working in plugin, all 11 steps passed
Example 2: Re-run after upstream update
User says: "migrate qwen3_5 again, upstream updated"
Actions:
1. Idempotent re-run — overwrite existing files with fresh upstream copy
2. Re-apply patches, re-validate, re-test
3. Re-run E2E to confirm no regression
Result: qwen3_5 updated to match latest upstream, no regressions
General principle: When any runtime error occurs, first compare vLLM upstream code against both the plugin adaptation and the installed 0.13.0 environment. The diff is the fastest path to root cause. See operational-rules.md § Debugging Priority: Upstream-First for the full protocol.
| Problem | Typical Cause | Fix | |
|---|---|---|---|
| --- | --- | --- | |
ImportError after copy-then-patch | Missing P1 fix (relative→absolute imports) | Verify all from .xxx converted to from vllm. or from vllm_fl. | |
AttributeError: module 'vllm' has no attribute X | API doesn't exist in 0.13.0 | Check P3 in compatibility-patches.md; stub or remove | |
| Config not recognized by vLLM | model_type mismatch or config bridge missing | Verify _CONFIG_REGISTRY[model_type] matches HF config.json exactly | |
| Registration has no effect | Class name or import path typo | Compare with existing registrations in __init__.py | |
Benchmark KeyError on config field | Config bridge missing a field | Compare upstream config class vs bridge; add missing fields with defaults | |
| Benchmark/Serve fails with OOM or "insufficient memory" | GPUs occupied by other processes | Kill GPU processes: `nvidia-smi --query-compute-apps=pid --format=csv,noheader \ | xargs -r kill -9` then retry. Never skip these steps. |
| Model outputs garbled/gibberish text | ColumnParallelLinear used for merged projections with different sub-dimensions (TP sharding mismatch) | Override __init__ to use MergedColumnParallelLinear(output_sizes=[...]). See P8 in compatibility-patches.md | |
AssertionError: Duplicate op name | Child class imports custom op from different module path than parent | Use same import path as parent module (e.g. vllm_fl.ops.fla not vllm_fl.models.fla_ops). See P11 | |
AttributeError on fused_recurrent_* during CUDA graph warmup | __init__ override with nn.Module.__init__(self) missed attributes used by inherited _forward_core | Create ALL attributes from parent's __init__, especially custom ops. See P12 | |
| E2E: local server not reachable | serve.sh port doesn't match e2e_config.json local port | Ensure both use same port (default 8122) | |
| E2E: GT server not reachable | GT machine down or docker/conda env wrong | Check e2e_remote_serve.sh status or SSH manually | |
| E2E: early token divergence (first 5 tokens) | Weight loading bug, TP sharding error | Check load_weights, stacked_params_mapping, MergedColumnParallelLinear | |
| E2E: late minor divergence (token #15+) | Numerical noise from different op implementations | Usually acceptable; document in report | |
resolve_op fails with VLLM_FL_PREFER_ENABLED=false | Op not registered in dispatch, no fallback | Add try/except fallback to flag_gems in op import code |
共 2 个版本