Senior Engineer (Web Crawling & Automation)
- Tracxn
- 1 - 6 years
- Bengaluru
- 3 months ago
- Email to a friend
- Report this job
Job Description
Total Experience: 1+ years
Mode of Hire: Permanent
Required Skills Set (Mandatory): JavaScript/Node.js/Python, Web Crawling (Puppeteer/Playwright), Strong knowledge of web technologies
Desired Skills (Good if you have): HTML, Distributed Systems, Data pipelines, Proxy Management, Anti-bot Techniques
Job Responsibilities
- Design scalable web crawling solutions using modern technologies like Puppeteer and Playwright.
- Develop optimal and cost-effective methods for efficiently crawling diverse websites at scale.
- Implement robust proxy management and sophisticated anti-bot techniques.
- Create comprehensive debugging and monitoring systems for crawler performance.
- Actively contribute to technical discussions and architectural decision-making processes.
Job Requirements
- Experience in end-to-end management of large-scale distributed web crawling.
- Deep understanding of website internal mechanics, enabling innovative data extraction strategies.
- Expert-level proficiency in JavaScript/Node.js, web technologies, and anti-bot measures.
- Demonstrated ability to understand, adapt, and extend current system architectures.
- Strong collaborative skills with the ability to work effectively within existing technical frameworks.
Preferred Qualifications
- Advanced system optimization skills.
- Proven experience with cloud platforms, databases, and distributed systems.
- Comprehensive knowledge of data pipelines and data warehousing concepts.
Culture
- Work with performance-oriented teams driven by ownership and passion.
- Learn to design systems for high accuracy, efficiency, and scalability.
- No strict deadlines, focus on delivering quality work.
- Meritocracy-driven, candid culture. No politics.
- Very high visibility regarding which startups and markets are exciting globally.
About Tracxn
Tracxn (Tracxn.com) is a Bangalore-based product company providing a research and deal sourcing platform for Venture Capital, Private Equity, Corp Dev, and professionals working around the startup ecosystem.
We are a team of 600+ working professionals serving customers across the globe. Our clients include Funds like Andreessen Horowitz, Matrix Partners, GGV Capital, and Large Corporations such as Citi, Embraer & Ferrero.
Founders
- Neha Singh (ex-Sequoia, BCG | MBA - Stanford GSB)
- Abhishek Goyal (ex-Accel Partners, Amazon | BTech - IIT Kanpur)
About Technology Team
Tracxn's Technology team is 50+ members strong and growing. The technology team is subdivided into multiple smaller teams, each owning one or more services/components of the technology platform. Ours is a young team of motivated engineers with a minimal management structure where almost everyone is actively involved in technical development and design activities. We have a team-centric culture where the ownership and responsibility of a feature or module lie with a team compared to an individual.
We work on an array of technologies, including but not limited to ReactJS, Next.js, Storybook, Webpack, Node, Mongo, AWS Lambda, Spring, Elastic Stack, MySQL, Kafka, Redis, Ansible, etc.
We value ownership, continuous learning, consistency, and discipline as a team.
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QARole Category: Software DevelopmentRole: Software Development - OtherEmployement Type: Full timeContact Details:
Company: TracxnLocation(s): Bengaluru
Keyskills: Javascript Proxy Management Web Technologies Web Crawling Node.js Distributed Systems Data pipelines Python
Similar positions
Tracxn
Founded in 2013 by ex-Venture Capitalists, Tracxn is one of the worlds largest platforms for tracking 1.4 million entities through 1,800 feeds categorized across industries, sectors, sub-sectors, geographies, affiliations and networks globally.
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in