Data Engineer/ Backend developer
Company: QLAN - a technology integrator that implements end-to-end AI solutions into the business processes of large companies and the public sector.
Tasks:
- Write custom parsers for social networks, media hosting sites, and marketplaces
- Implement services in Python
- Develop and maintain data collection services
Requirements:
- 3+ years of experience in developing and supporting high-load parsers/scrapers;
- Python 3+ years;
- Understanding of various proxy server types (proxy rotation, quality monitoring), CAPTCHA principles;
- Working with rate limiting;
- Playwright;
- AsyncIO, aiohttp/httpx
- Experience in reverse engineering internal APIs of web and mobile applications
- Experience with databases (PostgreSQL / ClickHouse / Redis)
- Excellent understanding of HTTP/TCP, CDN operation, REST API, GraphQL, anti-bot systems, and methods for bypassing them;
Will be a plus:
- Experience developing parsers for platforms: Instagram, YouTube, Tik Tok, Trip Advisor, Spotify, X, Flickr, LinkedIn, Reddit, and similar
- Experience with Grafana/Prometheus
- Kubernetes;
- Kafka/RabbitMQ;
We offer:
- Project-based employment with a flexible work format
- Direct communication with technical leadership without bureaucracy
- Opportunity to grow into a full-fledged technical role within the company
