About the Project

We are building a high-performance cloud platform for parallel execution of hundreds and thousands of browsers (Chromium/Chrome/Firefox) in containers or micro-VMs, controlled via:

Chrome DevTools Protocol (CDP)

WebSocket

Selenium WebDriver API

Puppeteer / Playwright

CEFSharp (embedded browsers for .NET)

The platform is used for large-scale distributed parsing, bypassing anti-bot protection, web interface automation, and writing complex user scenarios.

We are looking for a specialist capable of building a multithreaded distributed data collection system that uses remote browsers and hybrid control protocols.

Tasks

Developing a Multithreaded Distributed Parser

• managing dozens/hundreds of browser sessions simultaneously

• working directly via Chrome DevTools Protocol (CDP)

• executing JS in the page context

• collecting HTML, JSON, network responses

• parallel task queues, URL deduplication, retry logic

• site traversal planning (BFS/DFS, segmentation)

Working with Automation Frameworks

Puppeteer:

• connect() mode to a remote browser

• developing complex scenarios, hooking network requests

Playwright:

• connectOverCDP to remote sessions

• integrating page events, route interception

Selenium:

• developing a bridge layer to remote browsers

• integrating WebDriver API on top of CDP

CEFSharp:

• working with embedded Chromium Embedded Framework

• integrating the parser with a C# environment (if required)

• advanced browser events, hiding automation

• analyzing the behavior of embedded CEF under load

Anti-bot & Stealth

• bypassing challenge pages

• simulating user behavior (scroll, input, delays)

• analyzing WebGL/Canvas/DOM changes

• retrieving data from JS frameworks (SPA, React, Angular)

• working with Residential/Mobile proxies

Reliable Distributed Architecture

• managing large pools of WebSocket connections

• fault-tolerance: restarts, retries, session recovery

• optimizing network requests

• reducing latency when working through proxies

• integration with a cloud system for creating/terminating browsers

Mandatory Requirements

1 Chrome DevTools Protocol (mandatory)

• deep knowledge of Network.*

• DOM.*

• Runtime.*

• Page.*

• direct work via WebSocket

• experience with CDP without Puppeteer is critically important

2 Puppeteer

• experience with complex scenarios

• intercepting requests

• working via browserWSEndpoint

• bypassing some anti-bot checks

3 Playwright

• experience with Playwright API

• ability to connect to a remote browser (connectOverCDP)

• knowledge of route/network hooks

4 Selenium / WebDriver

• deep understanding of Selenium API

• experience integrating Selenium with non-standard environments

• understanding of the W3C WebDriver protocol limitations

5 CEFSharp / Chromium Embedded Framework

• experience with implementing and configuring CEFSharp

• managing pages and resources via CEF API

• understanding of DevTools integration with CEF

• experience working with embedded browsers for .NET

• plus: experience with customizing renderer/JS binding

6 Programming Languages (one of):

• Go (preferred)

• Node.js

• Python

• Rust

• C# (for CEFSharp modules)

7 Multithreading and High-load

• working with task queues

• thread synchronization (mutex, channel, async-await)

• ability to write code that can handle 100–1000 parallel sessions