Event-Driven Profile Automation: Launching Browser Profiles From Your Own Code

A scheduled scraper can miss a flash sale entirely. If the cron job runs every 15 minutes, a sale that starts and ends between two runs is never seen.

The instinct is faster polling. The better answer is to stop polling.

Event-driven architecture flips the model. Instead of asking "is there new data?" every N minutes, you act when something tells you "there's new data." A lead lands in your CRM. Your price monitor sees a competitor change a price. A CI job finishes a deploy. Your own code already sees each of those moments. This approach pairs well with multi-account browser automation, since profiles launch only when needed.

Your multi-account browser doesn't wake up on a schedule. It wakes up when there's work to do.

This tutorial walks through building that architecture: events in your own code create jobs, tasks run in isolated profiles, each with its own cookies, storage and proxy, and results go back to wherever you need them. Everything here is your code: JustBrowser has no webhooks, so nothing calls into it except your worker, through its local REST API.

What We're Building

By the end of this, you'll have:

1. A small function your code calls when something happens (new lead, price change, whatever)
2. A job queue that buffers those jobs
3. A worker that pulls jobs, launches JustBrowser profiles via REST API, runs tasks, and posts results back
4. Error handling that doesn't leave orphaned browser processes

The whole thing runs on one machine: the Windows or Mac computer where JustBrowser runs, because its local API listens only on 127.0.0.1. Redis can live there too, or on a small server the worker can reach; either way the worker that drives the browser sits next to JustBrowser. No Kubernetes. No Terraform. Just code. ("Just spin up a K8s cluster" has become the default answer to every scaling question. Most of you don't need it.)

Prerequisites

• Python 3.9+ (or Node.js — both are shown)
• JustBrowser ($9.99/mo or $99.99/year, and the 7-day trial includes the API) — team seats have no REST API access
• An API token from the AI page in the app
• Redis for job queuing (or use a simple in-memory queue for testing)
• Something in your own code that knows when work is due: a CRM sync, a price monitor, a CI step, or a test script
• Basic familiarity with async programming

Step 1: Queue a Job From Your Own Code

First, the code that knows something happened needs a cheap way to hand the work off. A function that checks the job and pushes it onto a Redis list is enough:

import json
import redis

r = redis.Redis(host='localhost', port=6379, db=0)

def enqueue_lead_job(lead_id, target_url, profile_id="default-scraper-01", callback_url=None):
    # Check the job, even though your own system produced it
    if not lead_id or not target_url:
        raise ValueError("lead_id and target_url are required")

    # Push to the queue instead of launching a browser here
    job = {
        "type": "lead_scrape",
        "lead_id": lead_id,
        "target_url": target_url,
        "profile_id": profile_id,
        "callback_url": callback_url,  # where in your system the results go
    }
    r.rpush("job_queue", json.dumps(job))
    return {"status": "queued", "lead_id": lead_id}

For Node.js:

const Redis = require('ioredis');
const redis = new Redis();

async function enqueueLeadJob({ lead_id, target_url, profile_id = 'default-scraper-01', callback_url }) {
  if (!lead_id || !target_url) {
    throw new Error('lead_id and target_url are required');
  }

  await redis.rpush('job_queue', JSON.stringify({
    type: 'lead_scrape',
    lead_id,
    target_url,
    profile_id,
    callback_url  // where in your system the results go
  }));
  return { status: 'queued', lead_id };
}

module.exports = { enqueueLeadJob };

Call it from wherever the event happens: the code that saves a new lead, the price monitor's change handler, the last step of a CI job.

Notice we're not launching browsers in the code that creates the job. That's the easy mistake. The caller should stay fast: check, queue, return. Launch a profile right there and a slow page or a busy machine holds up the part of your system that only wanted to record a new lead.

Step 2: Build the Worker Process

The worker is where the actual browser work happens. It pulls jobs from the queue, launches profiles, runs tasks, posts results.

import os
import requests
import time
import json
import redis
from playwright.sync_api import sync_playwright

JUSTBROWSER_API = "http://127.0.0.1:36542/api/v1"   # the local API; port configurable in the app
HEADERS = {"Authorization": f"Bearer {os.environ['JB_API_TOKEN']}"}
r = redis.Redis(host='localhost', port=6379, db=0)

def launch_profile(profile_id):
    """Start a profile and return its CDP URL."""
    response = requests.post(
        f"{JUSTBROWSER_API}/profiles/{profile_id}/start",
        json={"headless": False},   # the API starts headless unless told otherwise
        headers=HEADERS
    )
    response.raise_for_status()
    return response.json()["data"]["cdp_url"]  # for Playwright/Puppeteer

def stop_profile(profile_id):
    """End the session. browser.close() only disconnects."""
    try:
        requests.post(f"{JUSTBROWSER_API}/profiles/{profile_id}/stop", headers=HEADERS)
    except Exception as e:
        print(f"Warning: Failed to stop profile {profile_id}: {e}")

def run_task(ws_endpoint, job):
    """Execute the actual scraping/automation task."""
    with sync_playwright() as p:
        browser = p.chromium.connect_over_cdp(ws_endpoint)
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()

        try:
            page.goto(job["target_url"], wait_until="networkidle")

            # Your actual task logic here
            # This example scrapes basic lead info
            title = page.title()
            description = page.locator('meta[name="description"]').get_attribute("content")

            return {
                "lead_id": job["lead_id"],
                "title": title,
                "description": description,
                "scraped_at": time.time()
            }
        finally:
            browser.close()

def post_results(callback_url, results):
    """Send results back to the source system."""
    if callback_url:
        requests.post(callback_url, json=results, timeout=10)

def worker_loop():
    print("Worker started. Waiting for jobs...")

    while True:
        # Blocking pop — waits for a job
        _, job_data = r.blpop("job_queue")
        job = json.loads(job_data)

        profile_id = job["profile_id"]
        print(f"Processing job: {job['type']} for lead {job['lead_id']}")

        try:
            ws_endpoint = launch_profile(profile_id)  # answers once the browser is up

            results = run_task(ws_endpoint, job)
            post_results(job.get("callback_url"), results)

            print(f"Completed: {job['lead_id']}")

        except Exception as e:
            print(f"Failed: {job['lead_id']} — {e}")
            # Could re-queue with retry count here

        finally:
            stop_profile(profile_id)

if __name__ == "__main__":
    worker_loop()

This is a single-threaded worker. Each job runs one at a time. For higher throughput, run multiple workers — each pulls from the same queue, each manages its own profile. If you're new to driving a profile with Playwright, the walkthrough on testing CAPTCHA flows on your own site with Playwright shows the connection end to end.

(Don't share one browser instance across threads. One worker, one profile. Scale horizontally.)

Step 3: Handle Concurrent Jobs Without Colliding

Real systems get bursts. Thirty leads arrive in 10 seconds. You've got 5 profiles. What happens?

Without throttling, you try to launch 30 sessions simultaneously. Profiles crash. Memory explodes. Your server reboots at 3 AM.

Here's a worker pool pattern:

import threading
from queue import Queue
import time

class ProfilePool:
    def __init__(self, profile_ids):
        self.available = Queue()
        for pid in profile_ids:
            self.available.put(pid)

    def acquire(self, timeout=60):
        """Get an available profile, blocking until one is free."""
        return self.available.get(timeout=timeout)

    def release(self, profile_id):
        """Return profile to pool after use."""
        self.available.put(profile_id)

def worker_with_pool(pool, job_queue):
    while True:
        job = job_queue.get()
        profile_id = None

        try:
            profile_id = pool.acquire(timeout=120)
            ws_endpoint = launch_profile(profile_id)

            results = run_task(ws_endpoint, job)
            post_results(job.get("callback_url"), results)

        except Exception as e:
            print(f"Job failed: {e}")
        finally:
            if profile_id:
                stop_profile(profile_id)
                pool.release(profile_id)

# A local queue fed from Redis by one reader thread
job_queue = Queue()

def redis_reader():
    while True:
        _, job_data = r.blpop("job_queue")
        job_queue.put(json.loads(job_data))

threading.Thread(target=redis_reader, daemon=True).start()

# Start worker threads matching your profile count
PROFILE_IDS = ["scraper-01", "scraper-02", "scraper-03", "scraper-04", "scraper-05"]
pool = ProfilePool(PROFILE_IDS)

for _ in range(len(PROFILE_IDS)):
    t = threading.Thread(target=worker_with_pool, args=(pool, job_queue))
    t.daemon = True
    t.start()

Each worker grabs a profile from the pool, runs the job, returns the profile. If all profiles are busy, new jobs wait in the queue. No collisions. No resource exhaustion.

Honestly? This is the part most tutorials skip. They show you the happy path — one job, one browser, success. Real traffic doesn't work that way.

Step 4: Send Results Back to Your System

The callback_url in the job tells the worker where in your own system the results belong:

def post_results(callback_url, results):
    if not callback_url:
        return

    try:
        response = requests.post(
            callback_url,
            json=results,
            headers={"Content-Type": "application/json"},
            timeout=10
        )
        response.raise_for_status()
    except requests.exceptions.RequestException as e:
        # Log failure, maybe retry later
        print(f"Callback failed for {callback_url}: {e}")

Now your CRM gets scraped lead data pushed back. Your price monitoring system receives competitor updates. Your CI/CD pipeline knows the visual regression test passed.

The loop closes: job in, browser work, result out.

Common Errors and Fixes

Error: Connection refused on profile launch

Cause: JustBrowser isn't running, or the API port changed.

Fix: Make sure the JustBrowser app is open, and that the worker runs on the same machine: the API listens only on 127.0.0.1. Check the app's settings for the actual API port — the default is 36542, and it's configurable. The API docs cover every endpoint.

Error: Jobs pile up faster than workers can process

Cause: Your code creates jobs faster than the workers can process them.

Fix: Add more worker processes. Each worker needs its own profile, so you'll need enough profiles to match your concurrency target. The plan includes unlimited profiles, so the limit is your hardware. Measure memory per active browser on your own workload before you size the machine. (Yeah, browsers are memory hogs. Always have been. No way around it.)

Error: Orphaned browser processes after worker crash

Cause: Worker died before calling stop_profile.

Fix: Run a cleanup job periodically that checks for profiles that have been running longer than expected and stops them. The list endpoint filters by status, and the CDP endpoint reports when each browser started.

from datetime import datetime, timezone

def cleanup_stale_profiles(max_age_seconds=600):
    response = requests.get(f"{JUSTBROWSER_API}/profiles", params={"status": "running"}, headers=HEADERS)
    running = response.json()["data"]["profiles"]

    for profile in running:
        cdp = requests.get(f"{JUSTBROWSER_API}/profiles/{profile['id']}/cdp", headers=HEADERS).json()["data"]
        launched = cdp["browser"].get("launched_at")
        if not launched:
            continue  # start time not recorded; check it by hand
        started = datetime.fromisoformat(launched.replace("Z", "+00:00"))
        if (datetime.now(timezone.utc) - started).total_seconds() > max_age_seconds:
            stop_profile(profile["id"])

Error: Results never reach callback URL

Cause: Callback endpoint is down, or timeout is too aggressive.

Fix: Implement retry with exponential backoff. After 3 failures, push to a dead-letter queue for manual review. Don't block your worker waiting for slow callbacks — fire and continue, handle failures async.

Next Steps

You've got event-driven browser automation running. Here's where to take it:

Add observability. Every job should log start time, end time, success/failure, and any error messages. Ship these to Datadog, Grafana, or even just a structured log file. When something breaks at 2 AM, you'll want to know which job failed and why.

Implement dead-letter queues. Jobs that fail 3 times get moved to a separate queue. Review them manually. Sometimes the failure reveals a bug in your task logic. Sometimes it's a site that changed its DOM. Sometimes it's a profile that needs reconfiguration.

Consider geographic distribution. If the content you collect differs by country, use profiles configured with proxies in the matching location. A lead lookup on a UK site should run from a UK profile. Build this into your job routing — the profile_id in your job payload determines which geo setup runs.

Scale horizontally. One machine running 10 workers is fine for moderate loads. Beyond that, add more worker machines pulling from the same Redis queue. Each one runs its own JustBrowser instance with its own profiles. Queue distributes work automatically. And please — don't try to run 50 workers on one box "because cloud is expensive." Add another Windows or Mac machine. Your sanity is worth more than that.

For ad traffic analysis, ClickzProtect handles click fraud detection. For analytics on your scraped data without leaking to third parties, JustAnalytics provides privacy-first tracking.

FAQ

Can my own code trigger multi-account browser sessions automatically?

Yes. JustBrowser's local REST API lets you launch profiles programmatically. When something happens in your system — a new lead, a price change, a finished deploy — your code puts a job on a queue, and a worker calls the API to start the right profile, runs the task and sends the result back. No polling required.

How do I stop a burst of jobs from overwhelming my profile capacity?

Put a job queue between the code that creates jobs and the worker that launches profiles. Redis, RabbitMQ, or even a simple in-memory queue works. The queue buffers incoming jobs, and a worker process pulls jobs at a rate matching your available profiles. If you have 10 profiles, cap concurrent jobs at 10.

What happens if a task fails mid-execution in an event-driven flow?

Your worker should wrap task execution in try-catch, log failures to your monitoring system, and always call the profile stop endpoint in a finally block. For critical tasks, implement retry logic with exponential backoff. Dead-letter queues catch jobs that fail repeatedly.

Do I need a subscription for API-triggered automation?

The local REST API needs a subscription or an active 7-day trial — team seats have no API access. At $9.99/mo or $99.99/year you get unlimited profiles plus the API endpoints needed for programmatic profile control.

---

Try JustBrowser

A multi-account browser for teams, built on a custom Chromium engine with native C++ integration (not extension-based). Each profile is its own browser with its own cookies, storage, proxy and 40+ identity parameters. Local REST API and CDP for Playwright, Puppeteer, Selenium. 7-day free trial, card required — then $9.99/month or $99.99/year, unlimited profiles, free team seats.

Get started → · How it differs from Multilogin / GoLogin / AdsPower