Senior Web Scraping Engineer

About usWe're building a product that relies heavily on collecting structured data from a number of known websites. We need someone experienced who can own this part of the system end-to-end; from writing scrapers to making sure they scale and stay reliable.This is a senior role. We're looking for someone who has already dealt with the real-world problems of web scraping and can work independently.What you'll doBuild and maintain scrapers for 24-30 websites (starting with 6).Handle authenticated sites, sessions, cookies, tokens, etc.Work with search endpoints and dynamic content.Architect the data ingestion pipeline, caching, retries, and queues.Make the scrapers stable and resilient even as websites change.Collaborate closely with our backend team to define and use APIs.Handle scaling, rate limits, and performance.What we're looking for (hard requirements)5 - 7+ years of building production-grade web scrapersStrong experience with both API-based scraping and browser automation (Playwright/Puppeteer)Familiarity with proxies/IP rotation and basic anti-bot handlingComfortable with auth flows (cookies, sessions, tokens)Ability to design pipelines: ingestion → clean → store → cacheCan work independently and own the scraper system end-to-endNice to have (not required)Familiarity with Cloudflare's ecosystem.Python, Node.js (TypeScipt only), or Go; any backend language is fine, TypeScript is preferred.Who will do well hereYou enjoy scrapers that work reliably, and predictably.You think in terms of systems, not quick patches.You can work remotely and manage your own time.You appreciate straightforward communication and minimal bureaucracy.CompensationBase: ₹3,00,000 - 4,00,000 per month (36 - 48 LPA). Equity/options are available if you choose to take a lower cash component.

Apply Now