Wiring a Multi-Account Browser Into an Apify Actor Pipeline

The run log said SUCCEEDED. The dataset had zero items. The gap between those two facts is the reason to check what the browser behind an Actor is actually rendering.

Picture an Apify Actor written months earlier, collecting public listings on a 4-hour schedule, quietly returning empty pages. No errors, no 403s, just a rendered shell with the product grid missing. Sometimes the cause is a changed layout. Sometimes the page waits for a signed-in session, a cookie the run never had, or a consent state that a fresh container throws away on every run.

Apify's own browser stack is good, and Crawlee handles queues and retries well. But an Actor container is ephemeral: every run starts with an empty browser, no cookies, no storage and a default configuration. If your workflow depends on a logged-in session in an account you own, or on a stable, consistent browser identity across runs, a throwaway container is the wrong place to keep it.

The job splits naturally. Apify keeps what it is great at: the request queue, the dataset, the schedule and the run history for the Actor that feeds the queue. JustBrowser, a multi-account browser for teams, supplies persistent profiles, each its own browser with its own cookies, storage, proxy and identity parameters. A small worker on the JustBrowser machine joins the two: it drives a profile over CDP and reads and writes Apify storage through Apify's API.

Use this for data you are allowed to collect: public pages, your own accounts and storefronts, client accounts you manage with permission, and your own QA. If a site forbids automated access, use its official API or data feed instead.

This tutorial walks through the wiring.

What We're Building: a Multi-Account Browser + Apify Actor

Two pieces:

• A feeder Actor on Apify. It runs on an Apify Schedule, reads its start URLs from Apify's standard input schema, and adds them to a named request queue.
• A worker on the JustBrowser machine. It starts a profile through JustBrowser's local REST API, connects to it over CDP with Playwright, takes requests from the same queue, pushes rows to an Apify Dataset, and stops the profile when the queue is empty.

The important architectural bit: the browser does not live on Apify, and nothing on Apify calls into your machine. The browser lives on a box you control — a spare Mac mini, a Windows machine you never turn off, whatever — and the worker beside it only makes outbound calls to Apify's API. Apify Actors are ephemeral. Profiles are not supposed to be.

Prerequisites

• An Apify account and its API token (APIFY_TOKEN) on the JustBrowser machine. Check Apify's pricing page before you budget anything — it changes from time to time.
• A persistent host you leave running. JustBrowser ships for Windows x64 and macOS on Apple Silicon, so that's a desktop or a Mac mini rather than a Linux VPS. Size for the fact that every active profile is a real Chromium process, not a browser tab, and leave headroom.
• JustBrowser — $9.99/mo, or $99.99/year. One plan, and the local REST API and CDP are in it, trial included.
• Node 20+ on the host, and the Apify CLI (npm i -g apify-cli) to deploy the feeder.

Step 1: Keep the Browser API Local

JustBrowser's API listens on 127.0.0.1:36542 (the port is configurable in the app). It is local by design: there's no hosted API, and reaching it from another machine isn't a supported setup. In this design nothing needs to. The worker runs on the same machine and calls the API there; the cdp_url a start returns is also a 127.0.0.1 address, which only a process on that machine can use.

So there's no inbound port to open and no tunnel to secure. The host's only outside traffic is the worker calling Apify's API with your token, plus the browser's own traffic through its proxy.

Check the API from the host before you touch Apify:

curl -H "Authorization: Bearer $JB_API_TOKEN" http://127.0.0.1:36542/api/v1/profiles

The token comes from the AI page in the app.

Step 2: Set Up the Profile

One profile per worker. Create it through the API:

curl -X POST http://127.0.0.1:36542/api/v1/profiles \
  -H "Authorization: Bearer $JB_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "apify-marketplace-01",
    "os": "windows",
    "locale": "de-DE"
  }'

The response carries the profile's id. Next, give it its proxy. A one-off setup script on the host can turn an Apify Proxy URL into a JustBrowser proxy and assign it to the profile:

// setup-proxy.js — run once on the JustBrowser machine
import { Actor } from 'apify';

await Actor.init();   // APIFY_TOKEN in the environment

const JB_BASE = 'http://127.0.0.1:36542/api/v1';
const JB_TOKEN = process.env.JB_API_TOKEN;
const PROFILE_ID = process.env.JB_PROFILE_ID;

async function jb(path, init = {}) {
  const res = await fetch(`${JB_BASE}${path}`, {
    ...init,
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${JB_TOKEN}`,
      ...(init.headers ?? {}),
    },
  });
  if (!res.ok) throw new Error(`JustBrowser ${path} → ${res.status} ${await res.text()}`);
  return res.json();
}

const proxyConfig = await Actor.createProxyConfiguration({ groups: ['RESIDENTIAL'] });
const url = new URL(await proxyConfig.newUrl(`jb_${PROFILE_ID}`)); // session-sticky per profile

const { data: proxy } = await jb('/proxies', {
  method: 'POST',
  body: JSON.stringify({
    name: `apify-${PROFILE_ID}`,
    protocol: 'http',
    host: url.hostname,
    port: Number(url.port),
    username: decodeURIComponent(url.username),
    password: decodeURIComponent(url.password),
  }),
});

// The proxy belongs to the profile, not to the start call.
await jb(`/profiles/${PROFILE_ID}/proxy`, {
  method: 'PUT',
  body: JSON.stringify({ proxy_id: proxy.id }),
});

// Match the profile's time zone and language to where the proxy exits.
await jb(`/profiles/${PROFILE_ID}/geo-match`, {
  method: 'POST',
  body: JSON.stringify({ proxy_id: proxy.id, apply: true }),
});

await Actor.exit();

The last call matters. The profile's timezone and locale should match the exit IP; a Berlin proxy behind an America/Chicago profile makes localised sites behave inconsistently.

If the job needs a logged-in session, open the profile once by hand, sign in to the account you own or manage, and close it. The session persists in the profile, so every later run starts already signed in.

Step 3: The Feeder Actor

Create the project:

apify create jb-feeder --template project_empty
cd jb-feeder
npm i apify

The feeder never runs a browser, so you don't need the Playwright base image. Use the slim Node image.

FROM apify/actor-node:20
COPY package*.json ./
RUN npm --quiet set progress=false \
 && npm install --omit=dev --omit=optional
COPY . ./
CMD npm start --silent

.actor/input_schema.json, kept short:

{
  "title": "JustBrowser feeder input",
  "type": "object",
  "schemaVersion": 1,
  "properties": {
    "startUrls": { "title": "Start URLs", "type": "array", "editor": "requestListSources" }
  },
  "required": ["startUrls"]
}

And the Actor itself:

// src/main.js — runs on Apify, on a schedule
import { Actor } from 'apify';

await Actor.init();

const { startUrls = [] } = (await Actor.getInput()) ?? {};
const queue = await Actor.openRequestQueue('jb-work');
const day = new Date().toISOString().slice(0, 10);

// A request queue skips a uniqueKey it has seen before, so key each URL by day
// to queue it again on the next scheduled run.
await queue.addRequests(startUrls.map(({ url }) => ({ url, uniqueKey: `${url}#${day}` })));

await Actor.exit();

Step 4: The Worker, Connected Over CDP

This runs on the JustBrowser machine. It uses the same apify package, with forceCloud: true so the queue and dataset are the ones on Apify, not local folders.

// worker.js — runs on the JustBrowser machine
import { Actor } from 'apify';
import { chromium } from 'playwright';

await Actor.init();   // APIFY_TOKEN in the environment

const JB_BASE = 'http://127.0.0.1:36542/api/v1';
const JB_TOKEN = process.env.JB_API_TOKEN;
const PROFILE_ID = process.env.JB_PROFILE_ID;

async function jb(path, init = {}) {
  const res = await fetch(`${JB_BASE}${path}`, {
    ...init,
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${JB_TOKEN}`,
      ...(init.headers ?? {}),
    },
  });
  if (!res.ok) throw new Error(`JustBrowser ${path} → ${res.status} ${await res.text()}`);
  return res.json();
}

const queue = await Actor.openRequestQueue('jb-work', { forceCloud: true });
const dataset = await Actor.openDataset('jb-results', { forceCloud: true });

const { data: { cdp_url: wsEndpoint } } = await jb(`/profiles/${PROFILE_ID}/start`, {
  method: 'POST',
  body: JSON.stringify({ headless: false }),
});

const browser = await chromium.connectOverCDP(wsEndpoint);

Two details worth pausing on.

headless: false is deliberate. The API starts profiles headless unless told otherwise. Leave the host logged into a real desktop session and run headed: a headless session has no real window, so its screen and viewport follow headless defaults. (The Playwright and Puppeteer guide goes through the details.)

And the proxy went on the profile in Step 2 — via PUT /profiles/{id}/proxy, or POST /profiles/batch/proxy if you're spreading proxies across a pool — not in Playwright's options. JustBrowser applies it at the engine level, alongside WebRTC protection and per-profile DNS-over-HTTPS. Set it in Playwright instead and the profile and the network path can disagree.

Step 5: Scrape, Push, Release

const context = browser.contexts()[0];          // the profile's own context
const page = context.pages()[0] ?? (await context.newPage());

try {
  let request;
  while ((request = await queue.fetchNextRequest())) {
    try {
      await page.goto(request.url, { waitUntil: 'domcontentloaded', timeout: 60_000 });
      await page.waitForSelector('.product-card', { timeout: 15_000 });

      const found = await page.$$eval('.product-card', (cards) =>
        cards.map((c) => ({
          title: c.querySelector('.title')?.textContent?.trim() ?? null,
          price: c.querySelector('.price')?.textContent?.trim() ?? null,
        })),
      );

      await dataset.pushData(found.map((row) => ({ ...row, url: request.url })));
      await queue.markRequestHandled(request);
    } catch (err) {
      console.error(`${request.url}: ${err.message}`);
      if (request.retryCount < 3) {
        request.retryCount += 1;
        await queue.reclaimRequest(request);   // back in the queue for a later try
      } else {
        await queue.markRequestHandled(request); // give up on it
      }
    }
    await page.waitForTimeout(3000);            // pace requests to the site
  }
} finally {
  await browser.close();                        // only disconnects
  await jb(`/profiles/${PROFILE_ID}/stop`, { method: 'POST' }).catch(() => {});
  await Actor.exit();
}

That finally block matters more than it looks. browser.close() only disconnects Playwright; the profile keeps running until /stop. If the worker dies mid-run and never calls it, the profile stays up on your host holding RAM, and the next start gets a 409.

Pushing to the dataset per page rather than once at the end means an aborted run still keeps whatever it already collected. Apify's dataset is append-only — use that.

Step 6: Schedule It

Deploy the feeder:

apify push

In the Apify console: Schedules → Create → cron expression → attach the feeder Actor. Set its memory low; it only adds requests to a queue.

Then schedule the worker on the host itself, to run shortly after the feeder: launchd on macOS, Task Scheduler on Windows. Or keep it running and have it sleep and check again when the queue is empty. Give it a timeout either way, so a hung page can't hold the profile forever.

The feeder's runs show in Apify's run history; the worker's logs live on your host, so send them somewhere you will actually look.

For the reverse direction, you can poll the profile's health score over the local REST API (GET /profiles/{id}/health) on a schedule and decide whether a profile should be paused or rebuilt, rather than relying on a fixed clock. If a site has blocked your access, stop and use its official data access or contact the owner.

Common Errors and How to Fix Them

connectOverCDP: connect ECONNREFUSED

The profile isn't running, or the worker isn't on the JustBrowser machine. Run the worker on the same host, then check the app is up with curl -H "Authorization: Bearer $JB_API_TOKEN" http://127.0.0.1:36542/api/v1/system/status on the host itself. If that fails, the app isn't running or the port in the app's settings differs from the one in your script.

409 Conflict on launch

{"error":{"code":"ALREADY_RUNNING","message":"..."}}

A previous run left the profile up. Call /stop manually, then add the finally block from Step 5 if you skipped it. Branch on error.code, never on the message. If you run several workers, give each its own profile so two can't start the same one.

Run succeeds, dataset is empty

Your selector is gone, or the page is showing a login wall, consent dialog or verification step. Screenshot before you guess:

const kv = await Actor.openKeyValueStore('jb-debug', { forceCloud: true });
await kv.setValue('debug.png', await page.screenshot(), { contentType: 'image/png' });

The screenshot in the key-value store tells you in two seconds what an hour of log-reading won't.

Launches failing under load

Every start spawns a full browser process, so back off exponentially rather than retrying tight — the concurrency and backoff patterns apply here the same as anywhere else.

Next Steps

• More profiles, more workers. Run one worker per profile, keyed by target site, so runs don't collide and each site's cookies and storage stay together. Before several workers drain one queue, check Apify's documentation on sharing a request queue between clients. Only scrape public data the site permits, and respect its terms and robots.txt.
• Crawlee, if you want its crawler. You can subclass PlaywrightPlugin to connect instead of launch and keep Crawlee's retry and queue machinery in the worker. It works, but it's a bit hacky, and only worth it if you're already deep in Crawlee.
• Check profile consistency. Every few weeks, re-test the profile's proxy in the app's proxy checker and confirm that timezone, locale and exit IP still agree. Proxies change, and profiles accumulate state.
• Know where to look when a profile misbehaves. The troubleshooting checklist for profiles that keep getting verification prompts and the browser detection signals glossary explain what a site is reading. If a site is challenging your traffic, slow down or ask for access.

If the scraped data feeds a dashboard, JustAnalytics covers the metrics side without shipping anything back to the platforms you're measuring. And if you're running paid traffic next to the scraping infrastructure, ClickzProtect is the traffic-quality half of the same problem.

One caveat to close on: this setup has a single point of failure, and it's your host. Apify's reliability is no longer the ceiling — yours is. Budget for a health check and a restart policy before you schedule anything you actually depend on.

Frequently Asked Questions

Can I install JustBrowser inside the Apify Actor container itself?

No. Apify Actors are Linux containers and JustBrowser ships Windows x64 and macOS (Apple Silicon) builds only, so there's nothing to install in there. It would be the wrong shape regardless: Actors are ephemeral and get torn down after every run, so a profile living inside one starts cold every time — no cookies and no history. Run JustBrowser on a persistent host you control, with a small worker beside it that drives the profile over CDP and talks to Apify's API. Apify stays the scheduler, queue and dataset; the browser identity stays put.

Does Apify Proxy still work if the browser runs on my own machine?

Yes, but read the path carefully. Traffic goes from your host, out through the Apify Proxy endpoint, to the target — so the exit IP is Apify's, not yours. Add the Apify Proxy endpoint to JustBrowser as a proxy (POST /proxies) and assign it to the profile (PUT /profiles/{id}/proxy) instead of configuring it in Playwright. Whatever exit geo you pick should match the profile's timezone and locale, or sites that localise content will behave oddly and your data will be inconsistent.

Do I need a subscription for this?

Yes. There is one plan: $9.99/mo, or $99.99/year, for unlimited profiles with the local REST API and CDP included. The 7-day trial is full access, API and all, so you can build this whole pipeline and watch it run before the first charge. Card goes in at checkout; cancel inside the seven days and you aren't billed.

How many workers can share one profile?

One at a time. A profile is a single browser instance with a single cookie jar, and a second start of a running profile is refused with a 409 (ALREADY_RUNNING). To work through a queue faster, create more profiles and run one worker per profile, or key each profile to a specific target site so runs never collide.

---

Try JustBrowser

A multi-account browser for teams, built on a custom Chromium engine with native C++ integration, not an extension. Real C++ engine patches at the canvas / WebGL / audio / font / screen layer, with 40+ identity parameters rendered by the engine itself. Each profile keeps its own cookies, storage and proxy, and teammates can share profiles. Local REST API + CDP for Playwright, Puppeteer, Selenium. $9.99/month or $99.99/year. 7-day free trial, card required — cancel any time in the seven days and you are not charged. Unlimited profiles.

Get started → · How it differs from Multilogin / GoLogin / AdsPower