> ## Documentation Index
> Fetch the complete documentation index at: https://docs.insforge.dev/llms.txt
> Use this file to discover all available pages before exploring further.
# Web Scraper overview: Apify integration for InsForge
> Connect Apify to InsForge to let your coding agent run scrapers on demand, manage actors and runs, and land scraped datasets in your app.
Use InsForge Web Scraper to give your coding agent live access to external data: connect your own Apify account once, and your agent can run scrapers (Apify calls them actors) on demand, while the dashboard shows your actors, run history, and scraped datasets without leaving InsForge.
Connect Apify with one click, then paste the scrape prompt into your coding agent. The agent authenticates with your InsForge-managed Apify token, picks the right actor for the job, and returns the results.
Apify remains the source of truth for actors, runs, and datasets. InsForge surfaces a focused subset for everyday checks, then deep-links into the Apify console for anything beyond it. The Web Scraper integration is available on InsForge Cloud; self-hosted deployments return `501 Not Implemented` on these routes.
```mermaid theme={null}
flowchart TB
Agent["Coding Agent"] --> Skill["Apify Skill + CLI"]
Admin["Dashboard"] --> ScraperAPI["Web Scraper API"]
Skill --> Apify["Apify"]
ScraperAPI --> Apify
Apify --> Actors["Actors"]
Apify --> Runs["Runs"]
Apify --> Datasets["Datasets"]
Actors --> Pages["Web Scraper pages"]
Runs --> Pages
Datasets --> Pages
style Admin fill:#1e293b,stroke:#475569,color:#e2e8f0
style Agent fill:#1e293b,stroke:#475569,color:#e2e8f0
style Skill fill:#1e40af,stroke:#3b82f6,color:#dbeafe
style ScraperAPI fill:#166534,stroke:#22c55e,color:#dcfce7
style Apify fill:#c2410c,stroke:#fb923c,color:#fed7aa
style Actors fill:#0e7490,stroke:#06b6d4,color:#cffafe
style Runs fill:#0e7490,stroke:#06b6d4,color:#cffafe
style Datasets fill:#0e7490,stroke:#06b6d4,color:#cffafe
style Pages fill:#6b21a8,stroke:#a855f7,color:#f3e8ff
```
## Features
### One-click Apify connection
Connect Apify from the Web Scraper page in the dashboard. InsForge walks you through the Apify OAuth flow, stores credentials server-side, and keeps the access token refreshed for you. The raw token never lands in your repo or your frontend; agents and functions fetch a live token from the backend when they need one.
### Scrape via your coding agent
After connecting, the empty state ships a scrape prompt you can paste into your coding agent:
```
Use the insforge webscraper apify skill to scrape and return the results.
```
Behind the prompt, `npx @insforge/cli webscraper apify login` fetches your InsForge-managed Apify token, authenticates the local Apify CLI headlessly (no browser OAuth), and installs the Apify agent skills. From there the agent picks an actor from the Apify Store, starts runs, and reads the results back.
### Actors
The actors you have used or created recently, with their last run time and total run count. Each row deep-links into the Apify console for full actor configuration.
### Runs
Recent scraper executions with status (succeeded, failed, running), start time, and cost in USD. Useful for a quick "did last night's scrape work and what did it cost" check without opening Apify.
### Dataset
Datasets produced by your runs, with item counts, creation time, and the actor that produced them. Deep-links into Apify storage where you can inspect or export the items.
### Landing scraped data in your database
Scraped results live in Apify datasets by default; nothing is written to your project's Postgres unless you want it there. For small scrapes, your agent can just return the results. For anything you want to keep or refresh on a schedule, have the agent deploy an [edge function](/core-concepts/functions/overview) or [compute service](/core-concepts/compute/overview) that fetches the dataset from Apify and upserts rows into a table.
### Settings and disconnect
The Web Scraper Config dialog (the gear icon in the sidebar) shows the connected Apify account, plan, and data retention, links into the Apify console, and lets admins disconnect. Disconnecting only stops InsForge from using your Apify credentials; your Apify account, actors, and datasets stay intact, and you can reconnect anytime.
## Concepts
The serverless scrapers behind every run, from ready-made Store actors to your own.
How datasets store scraped items and how to export or fetch them via API.
## Build with it
`npx @insforge/cli webscraper apify connect` links your project to Apify, then logs your local agent in.
Thousands of ready-made actors for common targets, from Google Maps to social platforms.
Call actors and read datasets from your edge functions or compute services.
## Next steps
* Open the Web Scraper page in the dashboard and click **Connect Apify**.
* Paste the scrape prompt into your coding agent and tell it what you want to scrape.
* When a scrape is worth keeping, ask your agent to land the dataset in a table via an [edge function](/core-concepts/functions/overview) or a [schedule](/core-concepts/functions/schedules).