Instagram Public Profile Scraper

Fetches public Instagram profile data using Instagram's internal API. No login required, no browser needed, no Playwright overhead. Returns structured profile stats and recent post data.

Requirements

pip install httpx

Usage

python instagram_scraper.py <username>
python instagram_scraper.py google
python instagram_scraper.py nike

What It Returns

Profile Data

Recent Posts (up to 12)

Output

Results are saved to ./storage/instagram/.json:

{
  "profile": {
    "username": "google",
    "full_name": "Google",
    "followers_count": 12500000,
    ...
  },
  "recent_posts": [
    {
      "url": "https://www.instagram.com/p/ABC123/",
      "likes_count": 45230,
      "caption": "...",
      ...
    }
  ]
}

Implementation

The scraper uses Instagram's internal web API endpoint (i.instagram.com/api/v1/users/web_profile_info/) with a standard browser User-Agent and the public Instagram app ID. This is the same endpoint the Instagram web client calls on first page load.

class InstagramScraper:
    BASE_URL = "https://i.instagram.com/api/v1"
    APP_ID = "936619743392459"  # Public Instagram web app ID

    async def get_profile(self, username: str) -> dict:
        """Fetch a user's public profile data."""
        ...

    def parse_profile(self, raw_data: dict) -> dict:
        """Extract key information from raw profile data."""
        ...

    async def get_recent_posts(self, raw_profile: dict, limit: int = 12) -> list:
        """Extract recent posts from profile data."""
        ...

Error Handling

Rate Limiting

• Instagram's internal API is not publicly rate-limited, but aggressive scraping triggers IP blocks.
• Add a 2-5 second delay between requests when scraping multiple accounts.
• Private accounts return only public metadata (follower count, bio) but no posts.

Limitations

• Only works on public profiles
• Returns up to 12 most recent posts (Instagram's default page size)
• Does not return Stories, Reels metadata beyond what appears in the timeline grid
• Video view counts are not included in the internal API response

Use Cases

• Competitive analysis: track follower growth over time by running daily
• Influencer vetting: check engagement rate (likes+comments / followers) before partnerships
• Content research: analyze which post formats drive highest engagement for a niche
• Brand monitoring: track competitor posting frequency and content themes