← 返回
未分类 中文

Browse website - Crawls sites automatically and mounts pages as markdown files you can grep, diff, cat, and explore with standard Unix commands — over SSH or HTTP

Turn any website into a filesystem. Crawls sites automatically and mounts pages as markdown files you can grep, diff, cat, and explore with standard Unix com...
将任意网站转换为文件系统,自动抓取网站并将页面挂载为 Markdown 文件,可使用 grep、diff、cat 等标准 Unix 命令进行搜索、比较、读取和浏览。
bigmindai
未分类 clawhub v1.0.0 1 版本 100000 Key: 无需
★ 1
Stars
📥 315
下载
💾 0
安装
1
版本
#latest

概述

Browse Website

Use OpenObj to access any website's content. Pages are automatically crawled, converted to markdown, and mounted as a virtual filesystem you can explore with standard Unix commands.

Network Access

All commands in this skill require network access to openobj.com.

Use required_permissions: ["full_network"] for all SSH and curl commands (they need to reach openobj.com).

How to Use

Preferred: SSH exec mode (works in terminals with network access):

# required_permissions: ["full_network"]
ssh {domain}@openobj.com "{command}"

Fallback: HTTP API via curl (works in sandboxed environments):

# required_permissions: ["full_network"]
curl -s -X POST https://openobj.com/exec \
  -H 'Content-Type: application/json' \
  -d '{"site": "{domain}", "command": "{command}"}'

Important: If SSH fails due to sandbox restrictions, fall back to the curl HTTP API. Both methods return the same results.

Available Commands

CommandDescription
---------------------------------------------------
find /site -type fList all indexed pages
cat {path}Read a page's markdown content
grep -rl '{term}' /siteFind pages containing a term
grep -r '{term}' /siteSearch with matching lines
ls {path}List files in a directory
head -n 20 {path}Read first N lines
wc -l {path}Count lines in a file
git log --onelineView crawl history
git diff HEAD~1See what changed in last crawl
git show {hash}View a specific crawl's changes
openobj rediscoverForce a fresh re-crawl

Examples

Via SSH

# required_permissions: ["full_network"]
ssh docs.stripe.com@openobj.com "find /site -type f"
ssh docs.stripe.com@openobj.com "grep -rl 'webhook' /site"
ssh docs.stripe.com@openobj.com "cat /site/docs/webhooks.md"

# Change tracking
ssh docs.stripe.com@openobj.com "cd /site && git log --oneline"
ssh docs.stripe.com@openobj.com "cd /site && git diff HEAD~1"

# Force re-crawl and see what changed
ssh docs.stripe.com@openobj.com "openobj rediscover && cd /site && git diff HEAD~1"

Via HTTP API (curl)

# required_permissions: ["full_network"]
# List all pages
curl -s -X POST https://openobj.com/exec \
  -H 'Content-Type: application/json' \
  -d '{"site": "docs.stripe.com", "command": "find /site -type f"}'

# Search for a term
curl -s -X POST https://openobj.com/exec \
  -H 'Content-Type: application/json' \
  -d '{"site": "docs.stripe.com", "command": "grep -rl webhook /site"}'

# Read a page
curl -s -X POST https://openobj.com/exec \
  -H 'Content-Type: application/json' \
  -d '{"site": "docs.stripe.com", "command": "cat /site/docs/webhooks.md"}'

Workflow

  1. Discover — Run find /site -type f to see all available pages
  2. Search — Use grep -rl '{keyword}' /site to find relevant pages
  3. Read — Use cat {path} to read the full content of a page
  4. Refine — Use grep -r '{term}' {path} to search within specific files
  5. Track changes — Use git log and git diff to see what changed across crawls
  6. Re-crawl — Use openobj rediscover to force a fresh crawl and update pages

Behavior

  • First access to a domain triggers an automatic crawl (may take 10-30 seconds)
  • Subsequent accesses use the cached version (refreshed every 24 hours)
  • Use openobj rediscover to force a fresh crawl before the 24h window
  • Pages are converted from HTML to markdown automatically
  • Up to 200 pages per site are indexed
  • The virtual filesystem mirrors the site's URL structure
  • Each crawl is tracked as a git commit for change diffing

Credits

  • Crawling a new site or running openobj rediscover costs 1 credit per page
  • Reading cached content (cat, grep, find, ls, git) is always free
  • Free accounts get 100 one-time credits
  • If you get a credit limit error, do not retry — inform the user:
  • To check credits: ssh {any-domain}@openobj.com "openobj credits"
  • To upgrade: tell the user to run ssh auth@openobj.com in their terminal
  • Prefer reading cached sites over re-crawling to conserve credits

Response Format

The HTTP API returns JSON:

{
  "stdout": "...",
  "stderr": "...",
  "exitCode": 0
}

Use the stdout field for the command output. A non-zero exitCode indicates an error.

版本历史

共 1 个版本

  • v1.0.0 当前
    2026-05-07 18:47 安全 安全

安全检测

腾讯云安全 (Keen)

安全,无风险
查看报告

腾讯云安全 (Sanbu)

安全,无风险
查看报告

🔗 相关推荐

data-analysis

Stock Analysis

udiedrichsen
利用Yahoo Finance数据深度分析股票和加密货币。支持投资组合管理、关注列表与提醒、股息分析、八维度股票评分、热门趋势扫描(热点扫描器)及谣言/早期信号检测。适用于股票分析、投资组合追踪、财报反应、加密货币监控、热门股票发现及在主流
★ 281 📥 58,152
data-analysis

Data Analysis

ivangdavila
{"answer":"数据分析与可视化。查询数据库、生成报告、自动化电子表格,将原始数据转化为清晰可行的见解。适用于:(1) 您……"}
★ 216 📥 71,264
data-analysis

AdMapix

fly0pants
AdMapix 原始数据层,提供广告创意、应用、排名、下载/收入及市场元数据。返回 AdMapix API 的结构化 JSON;调用方...
★ 298 📥 142,762