Senior Python Data Scraping Engineer – Freelance

About the role

Mindrift is seeking a Senior Python Data Scraping Engineer to support the Tendem project on a freelance basis. You will design and operate end‑to‑end web extraction workflows that deliver high‑quality, structured datasets for real‑world AI applications. This remote, part‑time position is ideal for professionals with deep experience in web scraping and data processing.

Key responsibilities

• Develop and maintain complete data extraction pipelines for complex, dynamic websites.
• Utilise and extend tools such as Apify, OpenRouter, Selenium and custom Python scripts to accelerate collection and validation.
• Handle JavaScript‑rendered content, AJAX calls, infinite scroll and API interactions while managing proxy rotation.
• Implement rigorous data quality checks, cross‑source validation and formatting compliance before delivery.
• Scale scraping operations for large volumes using batching, parallelisation and failure monitoring.

Required profile

• Minimum 5 years of experience in data engineering, web scraping or software development.
• Proven ability to solve non‑trivial extraction problems independently.
• Detail‑oriented approach with strong debugging and optimisation skills.
• Comfortable working remotely and managing own schedule.

Required skills

• Python programming.
• Web scraping libraries/frameworks: BeautifulSoup, Selenium (or equivalents).
• Experience with Apify, OpenRouter or similar automation platforms.
• Proficiency with proxies, JavaScript, AJAX and API integration.
• Data validation, cleaning and structuring techniques.

What we offer

• Freelance contract with flexible remote work.
• Opportunity to contribute to cutting‑edge AI projects.
• Competitive compensation based on deliverables.