Event-Driven Profile Automation: Triggering Antidetect Sessions From Webhooks

Three weeks ago I watched an engineer debug why their scheduled scraper missed a flash sale. Their cron job ran every 15 minutes. The sale started at 2:03 PM, ended at 2:11 PM. The scraper kicked off at 2:15 PM. Eight minutes too late.

"We need faster polling," they said.

No. You need to stop polling.

Event-driven architecture flips the model. Instead of asking "is there new data?" every N minutes, you wait for something to tell you "there's new data." A webhook fires when a lead lands in your CRM. A queue message arrives when a competitor updates their price. A GitHub action triggers when a repo pushes new code. This approach pairs well with antidetect browser automation since profiles launch only when needed.

Your antidetect browser doesn't wake up on a schedule. It wakes up when there's work to do.

This tutorial walks through building that architecture: inbound events trigger profile launches, tasks run against fingerprint-protected sessions, and results post back to wherever you need them.

What We're Building

By the end of this, you'll have:

1. A webhook endpoint that receives events (new lead, price change, whatever)
2. A job queue that buffers incoming events
3. A worker that pulls jobs, launches JustBrowser profiles via REST API, runs tasks, and posts results back
4. Error handling that doesn't leave orphaned browser processes

The whole thing runs on a single server. No Kubernetes. No Terraform. Just code. (I genuinely hate how "just spin up a K8s cluster" has become the default answer to every scaling question. Most of you don't need it. Neither did I.)

Prerequisites

• Python 3.9+ with FastAPI (or Node.js with Express — I'll show both)
• JustBrowser Pro ($9.99/mo) — team seats have no REST API access
• Redis for job queuing (or use a simple in-memory queue for testing)
• A webhook source — Zapier, a CRM, or just curl for testing
• Basic familiarity with async programming

Step 1: Set Up Your Webhook Endpoint

First, we need something to receive events. Here's a minimal FastAPI server:

from fastapi import FastAPI, Request, BackgroundTasks
import redis
import json

app = FastAPI()
r = redis.Redis(host='localhost', port=6379, db=0)

@app.post("/webhook/lead")
async def receive_lead(request: Request, background_tasks: BackgroundTasks):
    payload = await request.json()

    # Validate — don't trust incoming data
    if not payload.get("lead_id") or not payload.get("target_url"):
        return {"error": "Missing required fields"}, 400

    # Push to queue instead of processing directly
    job = {
        "type": "lead_scrape",
        "lead_id": payload["lead_id"],
        "target_url": payload["target_url"],
        "profile_id": payload.get("profile_id", "default-scraper-01"),
        "callback_url": payload.get("callback_url")
    }
    r.rpush("job_queue", json.dumps(job))

    return {"status": "queued", "lead_id": payload["lead_id"]}

For Node.js/Express folks:

const express = require('express');
const Redis = require('ioredis');

const app = express();
const redis = new Redis();

app.use(express.json());

app.post('/webhook/lead', async (req, res) => {
  const { lead_id, target_url, profile_id, callback_url } = req.body;

  if (!lead_id || !target_url) {
    return res.status(400).json({ error: 'Missing required fields' });
  }

  const job = JSON.stringify({
    type: 'lead_scrape',
    lead_id,
    target_url,
    profile_id: profile_id || 'default-scraper-01',
    callback_url
  });

  await redis.rpush('job_queue', job);
  res.json({ status: 'queued', lead_id });
});

app.listen(3000);

Notice we're not launching browsers in the webhook handler. That's the mistake most people make. I made it too — my first version launched profiles inline and I couldn't figure out why HubSpot kept marking my endpoint as "unreliable." Webhook handlers should be fast. Accept the event, queue it, return. If you try to launch profiles synchronously, you'll hit timeouts, lost events, and angry upstream systems wondering why your endpoint takes 30 seconds to respond.

Step 2: Build the Worker Process

The worker is where the actual browser work happens. It pulls jobs from the queue, launches profiles, runs tasks, posts results.

import requests
import time
import json
import redis
from playwright.sync_api import sync_playwright

JUSTBROWSER_API = "http://localhost:9222/api"
r = redis.Redis(host='localhost', port=6379, db=0)

def launch_profile(profile_id):
    """Start an antidetect profile and return connection details."""
    response = requests.post(
        f"{JUSTBROWSER_API}/profiles/{profile_id}/launch",
        json={"headless": False}
    )
    response.raise_for_status()
    data = response.json()
    return data["wsEndpoint"]  # For Playwright/Puppeteer

def stop_profile(profile_id):
    """Clean shutdown of profile."""
    try:
        requests.post(f"{JUSTBROWSER_API}/profiles/{profile_id}/stop")
    except Exception as e:
        print(f"Warning: Failed to stop profile {profile_id}: {e}")

def run_task(ws_endpoint, job):
    """Execute the actual scraping/automation task."""
    with sync_playwright() as p:
        browser = p.chromium.connect_over_cdp(ws_endpoint)
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        try:
            page.goto(job["target_url"], wait_until="networkidle")

            # Your actual task logic here
            # This example scrapes basic lead info
            title = page.title()
            description = page.locator('meta[name="description"]').get_attribute("content")

            return {
                "lead_id": job["lead_id"],
                "title": title,
                "description": description,
                "scraped_at": time.time()
            }
        finally:
            browser.close()

def post_results(callback_url, results):
    """Send results back to the source system."""
    if callback_url:
        requests.post(callback_url, json=results, timeout=10)

def worker_loop():
    print("Worker started. Waiting for jobs...")

    while True:
        # Blocking pop — waits for a job
        _, job_data = r.blpop("job_queue")
        job = json.loads(job_data)

        profile_id = job["profile_id"]
        print(f"Processing job: {job['type']} for lead {job['lead_id']}")

        try:
            ws_endpoint = launch_profile(profile_id)
            time.sleep(2)  # Let browser initialize

            results = run_task(ws_endpoint, job)
            post_results(job.get("callback_url"), results)

            print(f"Completed: {job['lead_id']}")

        except Exception as e:
            print(f"Failed: {job['lead_id']} — {e}")
            # Could re-queue with retry count here

        finally:
            stop_profile(profile_id)

if __name__ == "__main__":
    worker_loop()

This is a single-threaded worker. Each job runs one at a time. For higher throughput, run multiple workers — each pulls from the same queue, each manages its own profile. If you're new to Playwright with antidetect browsers, start there first.

(I spent an embarrassing amount of time debugging why my multi-threaded version kept crashing. Turns out I was sharing one browser instance across threads. Don't do that. One worker, one profile. Scale horizontally.)

Step 3: Handle Concurrent Jobs Without Colliding

Real systems get bursts. Thirty leads arrive in 10 seconds. You've got 5 profiles. What happens?

Without throttling, you try to launch 30 sessions simultaneously. Profiles crash. Memory explodes. Your server reboots at 3 AM.

Here's a worker pool pattern:

import threading
from queue import Queue
import time

class ProfilePool:
    def __init__(self, profile_ids, api_base="http://localhost:9222/api"):
        self.available = Queue()
        for pid in profile_ids:
            self.available.put(pid)
        self.api_base = api_base
        self.lock = threading.Lock()

    def acquire(self, timeout=60):
        """Get an available profile, blocking until one is free."""
        return self.available.get(timeout=timeout)

    def release(self, profile_id):
        """Return profile to pool after use."""
        self.available.put(profile_id)

def worker_with_pool(pool, job_queue):
    while True:
        job = job_queue.get()
        profile_id = None

        try:
            profile_id = pool.acquire(timeout=120)
            ws_endpoint = launch_profile(profile_id)
            time.sleep(2)

            results = run_task(ws_endpoint, job)
            post_results(job.get("callback_url"), results)

        except Exception as e:
            print(f"Job failed: {e}")
        finally:
            if profile_id:
                stop_profile(profile_id)
                pool.release(profile_id)

# Start worker threads matching your profile count
PROFILE_IDS = ["scraper-01", "scraper-02", "scraper-03", "scraper-04", "scraper-05"]
pool = ProfilePool(PROFILE_IDS)

for _ in range(len(PROFILE_IDS)):
    t = threading.Thread(target=worker_with_pool, args=(pool, job_queue))
    t.daemon = True
    t.start()

Each worker grabs a profile from the pool, runs the job, returns the profile. If all profiles are busy, new jobs wait in the queue. No collisions. No resource exhaustion.

Honestly? This is the part most tutorials skip. They show you the happy path — one job, one browser, success. Real traffic doesn't work that way.

Step 4: Post Results Back via Webhook

The callback_url in your job payload lets results flow back to whatever triggered the event:

def post_results(callback_url, results):
    if not callback_url:
        return

    try:
        response = requests.post(
            callback_url,
            json=results,
            headers={"Content-Type": "application/json"},
            timeout=10
        )
        response.raise_for_status()
    except requests.exceptions.RequestException as e:
        # Log failure, maybe retry later
        print(f"Callback failed for {callback_url}: {e}")

Now your CRM gets scraped lead data pushed back. Your price monitoring system receives competitor updates. Your CI/CD pipeline knows the visual regression test passed.

The loop closes: event in, browser work, event out.

Common Errors and Fixes

Error: Connection refused on profile launch

Cause: JustBrowser isn't running, or the API port changed.

Fix: Make sure the JustBrowser app is open. Check settings for the actual API port — default is 9222, but if something else is bound there (another Chromium tool, maybe), it'll be different. See our REST API for port configuration.

Error: Jobs pile up faster than workers can process

Cause: Your inbound event rate exceeds processing capacity.

Fix: Add more worker processes. Each worker needs its own profile, so you'll need enough profiles to match your concurrency target. Pro gives unlimited profiles, so the limit is your hardware. Budget ~500MB RAM per active browser. If you're running 10 workers, that's 5GB just for browsers. (Yeah, browsers are memory hogs. Always have been. No way around it.)

Error: Orphaned browser processes after worker crash

Cause: Worker died before calling stop_profile.

Fix: Run a cleanup job periodically that checks for profiles that have been running longer than expected and kills them. JustBrowser's API has a list endpoint — iterate through running profiles and stop any that look stuck.

def cleanup_stale_profiles(max_age_seconds=600):
    response = requests.get(f"{JUSTBROWSER_API}/profiles/running")
    running = response.json()

    for profile in running:
        if profile["running_since"] < time.time() - max_age_seconds:
            stop_profile(profile["id"])

Error: Results never reach callback URL

Cause: Callback endpoint is down, or timeout is too aggressive.

Fix: Implement retry with exponential backoff. After 3 failures, push to a dead-letter queue for manual review. Don't block your worker waiting for slow callbacks — fire and continue, handle failures async.

Next Steps

You've got event-driven browser automation running. Here's where to take it:

Add observability. Every job should log start time, end time, success/failure, and any error messages. Ship these to Datadog, Grafana, or even just a structured log file. When something breaks at 2 AM, you'll want to know which job failed and why. I learned this the hard way after staring at silent failures for a week, convinced the system was working fine. It wasn't.

Implement dead-letter queues. Jobs that fail 3 times get moved to a separate queue. Review them manually. Sometimes the failure reveals a bug in your task logic. Sometimes it's a site that changed its DOM. Sometimes it's a profile that needs reconfiguration.

Consider geographic distribution. If you're scraping sites that geo-block, you'll need profiles configured with matching proxy locations. A lead scrape for a UK site should use a UK residential proxy. Build this into your job routing — the profile_id in your job payload determines which geo setup runs. Our proxy setup guide covers configuration details.

Scale horizontally. One server running 10 workers is fine for moderate loads. Beyond that, spin up multiple worker servers pulling from the same Redis queue. Each server runs its own JustBrowser instance with its own profiles. Queue distributes work automatically. And please — don't try to run 50 workers on one box "because cloud is expensive." Rent the extra $20/month server. Your sanity is worth more than that.

For ad traffic analysis where you need to verify what users actually see, ClickzProtect handles click fraud detection. For analytics on your scraped data without leaking to third parties, JustAnalytics provides privacy-first tracking. Both integrate with webhook architectures like the one you just built.

FAQ

Can webhooks trigger antidetect browser sessions automatically?

Yes. JustBrowser's REST API lets you launch profiles programmatically. When your webhook endpoint receives an event — a new lead, a price change, a queue message — your handler calls the API to spin up the right profile, runs the task, and posts results back. No polling required.

How do I prevent webhook floods from overwhelming my profile capacity?

Implement a job queue between your webhook endpoint and profile launcher. Redis, RabbitMQ, or even a simple in-memory queue works. The queue buffers incoming events, and a worker process pulls jobs at a rate matching your available profiles. If you have 10 profiles, cap concurrent jobs at 10.

What happens if a task fails mid-execution in an event-driven flow?

Your worker should wrap task execution in try-catch, log failures to your monitoring system, and always call the profile stop endpoint in a finally block. For critical tasks, implement retry logic with exponential backoff. Dead-letter queues catch events that fail repeatedly.

Do I need Pro for webhook-triggered automation?

The REST API needs a paid plan or an active trial — team seats have no API access. At $9.99/mo you get unlimited profiles plus the API endpoints needed for programmatic profile control.

---

Try JustBrowser

Native Chromium antidetect browser — not extension-based. Real C++ engine patches at the canvas / WebGL / font / TLS layer, so 40+ identity parameters are genuine, not faked. REST API for Playwright, Puppeteer, Selenium. 7-day free trial, card required — then $9.99/month or $99.99/year, unlimited profiles, free team seats.

Get started → · How it differs from Multilogin / GoLogin / AdsPower