Sr Python Developer -Immediate Joiners- Work from Office
- Blackstraw
- 5 - 10 years
- Chennai
- 2 months ago
- Email to a friend
- Report this job
Job Description
About Us: We are a dynamic and innovative team specializing in building scalable and efficient web scraping solutions for complex e-commerce platforms and diverse web environments. Our team is proficient in a wide range of technologies and advanced web scraping techniques, ensuring high-quality data extraction and delivery
Job Summary: We are seeking a talented Web Scraping Engineer / Data Extraction Specialist to join our growing team. The ideal candidate will have a strong background in web scraping, data extraction, and backend technologies. You will be responsible for designing, developing, and maintaining robust web scraping solutions, handling dynamic content, and overcoming anti-crawling measures.
Responsibilities:
Develop and maintain scalable web scraping scripts using Python, JavaScript, and related frameworks (e.g., Scrapy, Selenium, Puppeteer, Beautiful Soup, Cheerio.js).
Implement advanced web scraping techniques, including API interception, sitemap parsing, and handling dynamic content.
Design and build data pipelines for efficient data extraction, processing, and storage.
Manage and optimize data extraction workflows, ensuring high speed and accuracy.
Implement anti-crawling solutions, including IP rotation, proxy management, and CAPTCHA bypassing.
Collaborate with cross-functional teams to gather requirements and deliver complex data solutions.
Utilize backend technologies such as Flask, FastAPI, Django, Node.js, Spring Boot, and relational databases (PostgreSQL, MySQL) for data storage and API development.
Work with cloud platforms like Azure and leverage services such as AzureML and ADLS GEN2.
Employ data processing techniques using libraries like NumPy and Pandas.
Use tools like Postman, MITM, and DevTools for API testing and network traffic analysis.
Apply machine learning and NLP techniques for data analysis and processing (e.g., sentiment analysis, content classification).
Set up and manage server-side scraping environments.
Monitor and troubleshoot scraping scripts to ensure optimal performance.
Required Skills:
Strong proficiency in Python and/or JavaScript.
Experience with web scraping libraries and frameworks (Scrapy, Selenium, Puppeteer, Beautiful Soup, Cheerio.js).
Knowledge of backend technologies (Flask, FastAPI, Django, Node.js, Spring Boot).
Experience with relational databases (PostgreSQL, MySQL).
Understanding of HTTP/HTTPS protocols and API communication.
Familiarity with cloud platforms (Azure).
Ability to handle dynamic content and JavaScript-heavy websites.
Experience with anti-crawling techniques (IP rotation, proxies, CAPTCHA bypassing).
Data processing and analysis skills (NumPy, Pandas).
Experience with API testing tools (Postman).
Knowledge of machine learning and NLP concepts is a plus.
Strong problem-solving and debugging skills.
Excellent communication and collaboration skills.
Experience:
5+ years of professional experience in web scraping or related fields. Education:
Bachelor's degree in Computer Science, Engineering, or a related field
Job Classification
Industry: IT Services & ConsultingFunctional Area / Department: Engineering - Software & QA
Role Category: Software Development
Role: Back End Developer
Employement Type: Full time
Contact Details:
Company: BlackstrawLocation(s): Chennai
Keyskills: Web Scraping Python Flask RestAPI Data Extraction Web Development
Similar positions
Blackstraw
\n\n Tech Mahindra is an Indian multinational information technology services and consulting company. Part of the Mahindra Group, the company is headquartered in Pune and has its registered office in Mumbai. Tech Mahindra is a US$6.0 billion company[6][7] with over 158,000 employees across 90 countr...
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in