← 返回
未分类 中文

Web Fetch as Markdown

Fetches web pages from specific URLs and converts them to clean, structured Markdown via trusted APIs, enabling Agents to parse and extract data more effecti...
通过可信 API 从指定 URL 获取网页并转换为简洁、结构化的 Markdown,使 Agent 能更高效地解析和提取数据。
wuruofan wuruofan 来源
未分类 clawhub v1.2.1 1 版本 99777.3 Key: 无需
★ 0
Stars
📥 448
下载
💾 0
安装
1
版本
#latest

概述

Web Fetch as Markdown

Fetches any web URL and converts it to clean, structured Markdown — stripping ads, navigation, and clutter to leave only readable content, making it far easier for Agents to parse and extract data compared to raw HTML.

Conversion Services & Priority

This skill uses reputable third-party APIs to facilitate conversion. Always be transparent with the user about which service is being used.

  1. Primary — https://markdown.new/: Official Cloudflare edge conversion service. Highly reliable for general sites, especially Cloudflare-hosted ones, and optimized for token reduction.
  2. Fallback 1 — https://r.jina.ai/: Jina AI's official Reader API. Excellent for parsing clean markdown from complex pages when the primary service fails.
  3. Fallback 2 (Requires Consent) — https://markdownforagents.com/r?url=: General-purpose converter returning Markdown with YAML frontmatter. Only use when primary services are unreachable, and after informing the user.

Execution Flow & Graceful Degradation

When tasked with fetching a URL, follow this strict sequence:

  1. Attempt Primary Service: Construct the URL by prepending the target to the primary service:

https://markdown.new/https://example.com/article

Use the built-in WebFetch tool or equivalent to retrieve the content.

  1. Graceful Fallback to Jina: If the primary service fails, times out, or returns a domain safety warning (e.g., "Unable to verify if domain is safe"), gracefully fall back to Jina Reader:

https://r.jina.ai/https://example.com/article

  1. User Consent for Unverified Services: If both Cloudflare and Jina are unreachable (e.g., due to regional network policies), prompt the user:

> "The primary markdown conversion services (Cloudflare & Jina) are currently unreachable. Would you like me to attempt fetching via markdownforagents.com, or would you prefer to provide the content locally?"

Do not proceed to markdownforagents.com without explicit user approval.

  1. Final Limitation Handling: If all API routes fail or are denied by the user, inform the user that the content cannot be fetched automatically due to network or security constraints. Do not attempt to use curl with silent flags or suggest network workarounds.

Trigger Scenarios

User Intent (Chinese):

  • "上网搜索"、"帮我查一下"、"帮我看看这个网页"、"fetch 网页"
  • "帮我抓取这个页面"、"帮我解析一下"、"打开这个链接"

User Intent (English):

  • "search online", "look up", "fetch this page", "scrape this website"
  • "parse the data from", "extract information from"

System Scenarios:

  • The built-in WebFetch tool returns a domain safety restriction (e.g., "Unable to verify if domain * is safe to fetch").
  • The user asks to read/extract content from a specific URL where direct HTML parsing is inefficient.

Safety & Compliance Guidelines

  • Respect Platform Boundaries: If the platform's built-in tools block a domain for safety reasons, do not instruct the agent to bypass these safeguards. Use the approved fallback APIs instead.
  • No Network Evasion: Never suggest configuring proxies, VPNs, or other methods to circumvent network restrictions or regional blocks.
  • Data Transparency: Be aware that sending URLs to conversion services means the URL path (and potentially any non-authenticated content on that page) is exposed to the API provider. Avoid fetching pages known to contain sensitive authentication tokens in the URL.

版本历史

共 1 个版本

  • v1.2.1 当前
    2026-05-03 10:36 安全 安全

安全检测

腾讯云安全 (Keen)

安全,无风险
查看报告

腾讯云安全 (Sanbu)

安全,无风险
查看报告

🔗 相关推荐

data-analysis

Data Analysis

ivangdavila
{"answer":"数据分析与可视化。查询数据库、生成报告、自动化电子表格,将原始数据转化为清晰可行的见解。适用于:(1) 您……"}
★ 208 📥 67,336
data-analysis

Stock Watcher

robin797860
管理和监控个人股票自选列表,支持利用同花顺数据添加、删除、列出股票及汇总近期表现。适用于用户希望追踪特定股票、获取表现汇总或管理自选列表时。
★ 112 📥 45,966
data-analysis

Tavily 搜索

jacky1n7
通过 Tavily API 进行网页搜索(Brave 替代方案)。当用户要求搜索网页、查找来源或链接,且 Brave 网页搜索不可用时使用。
★ 273 📥 100,158