Social Media Data Hub

Use Apify to collect social media data from TikTok, Instagram, X/Twitter, and YouTube through one consistent workflow.

Core Capabilities

1. Profile lookup - Retrieve profile metadata such as follower count, content volume, and bio.
2. Bulk post retrieval - Pull recent or popular posts for an account together with engagement metrics.
3. Single-post lookup - Input a post URL and return a normalized record with key metrics.
4. Comment retrieval - Fetch comment threads for a supported post URL.
5. Cross-platform normalization - Map all supported platforms into one shared schema for analysis.

Actor Mapping

Usage

All scripts live under {baseDir}/scripts/, are executed with python3, and return JSON.

Profile Lookup

python3 {baseDir}/scripts/fetch_profile.py --platform tiktok --username "khaby.lame"
python3 {baseDir}/scripts/fetch_profile.py --platform instagram --username "natgeo"
python3 {baseDir}/scripts/fetch_profile.py --platform twitter --username "elonmusk"
python3 {baseDir}/scripts/fetch_profile.py --platform youtube --channel-url "https://www.youtube.com/@MrBeast"

Bulk Post Retrieval

python3 {baseDir}/scripts/fetch_posts.py --platform tiktok --username "khaby.lame" --count 20
python3 {baseDir}/scripts/fetch_posts.py --platform instagram --username "natgeo" --count 30
python3 {baseDir}/scripts/fetch_posts.py --platform twitter --username "elonmusk" --count 100
python3 {baseDir}/scripts/fetch_posts.py --platform youtube --channel-url "https://www.youtube.com/@MrBeast" --count 50

Single-Post Lookup

python3 {baseDir}/scripts/fetch_single_post.py --url "https://www.tiktok.com/@user/video/123456"
python3 {baseDir}/scripts/fetch_single_post.py --url "https://www.instagram.com/p/ABC123/"
python3 {baseDir}/scripts/fetch_single_post.py --url "https://x.com/user/status/123456"
python3 {baseDir}/scripts/fetch_single_post.py --url "https://www.youtube.com/watch?v=ABC123"

Comment Retrieval

python3 {baseDir}/scripts/fetch_comments.py --url "https://www.tiktok.com/@user/video/123456" --count 50
python3 {baseDir}/scripts/fetch_comments.py --url "https://www.instagram.com/p/ABC123/" --count 30
python3 {baseDir}/scripts/fetch_comments.py --url "https://x.com/user/status/123456" --count 100

Cross-Platform Normalization

normalize.py is used internally by the fetch scripts and can also be run directly:

echo '<raw_json>' | python3 {baseDir}/scripts/normalize.py --platform tiktok --type post

Unified Data Model

Normalized Post (NormalizedPost)

Normalized Profile (NormalizedProfile)

Cost Reference (BRONZE Tier)

See {baseDir}/references/apify_actors_reference.md for actor inputs, selection guidance, and pricing details.

Notes

• TikTok has a $0.50 minimum per run, so bulk retrieval is usually more cost-effective than single-item lookups.
• For X/Twitter, use tweet-scraper when you need 50 or more items and twitter-scraper-lite for smaller batches or single tweets.
• Instagram comments are limited to 50 comments per post.
• YouTube comments are not handled by these scripts; use a dedicated YouTube Comments Scraper actor instead.
• All scripts call the Apify REST API directly and do not rely on the apify-client SDK.