Data Pipeline Engineer – SQL Expert
• *About the job**
• *Mercor** connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include **Benchmark**, **General Catalyst**, **Peter Thiel**, **Adam D'Angelo**, **Larry Summers**, and **Jack Dorsey**.
• *Position:** Software Engineer II
• *Type:** **Full-Time**
• *Compensation:** **$70–$95/hour**
• *Location:** **Remote**
• *Role Responsibilities**
• Drive technical implementation and testing of complex data pipeline features within the **SPECTRA** platform. Ensure robust end-to-end functionality and meet project release timelines.
• Design and develop data processing pipelines using **SQL** and **Python**. Handle large-scale data collection, transformation, and quality monitoring workflows.
• Build and maintain dashboards and visualization tools. Provide visibility into data quality, rater performance, and platform metrics for stakeholders across multiple teams.
• Create comprehensive end-to-end tests and monitoring alerts. Ensure platform reliability, data integrity, and proactive identification of issues in the data collection workflow.
• Debug and troubleshoot complex data flow issues across the **SPECTRA** platform. Identify root causes and implement solutions to maintain high system availability.
• Collaborate closely with Technical Leads and engineering teams. Implement backend data processing logic and ensure adherence to development standards and best practices.
• *Qualifications**
• *Must-Have**
• Experience in **SQL** development writing complex queries, data transformations, and working with large datasets in production environments.
• Proficient in **Python** with experience in data processing, automation, and pipeline development using object-oriented programming principles.
• Experience with data engineering pipelines including **ETL** processes, data modeling, and building scalable data processing workflows.
• Hands-on experience with data visualization and dashboard tools such as **Tableau** or similar platforms.
• Strong analytical and problem-solving skills with demonstrated ability to debug complex data issues and implement effective solutions.
• Experience with data quality monitoring and automated testing frameworks.
• *Preferred**
• **Degree in Computer Science** or related field.
• **Machine Learning** experience with understanding of ML model training data requirements, feature engineering, and model evaluation metrics.
• Experience building and scaling large products or systems.
• Experience with building privacy and ads-related products.
• Experience working with human labeling and annotation systems including workflow design, quality assurance, and vendor management in data collection operations.
• Prior experience working with the company or its internal tools.
• *Compensation & Legal**
• Pursuant to the California Fair Chance Act, Los Angeles County Fair Chance Ordinance for Employers, Los Angeles Fair Chance Initiative for Hiring Ordinance, and San Francisco Fair Chance Ordinance, qualified applicants will be considered for assignment with arrest and conviction records.
• *Application Process (Takes 20–30 mins to complete)**
• Upload resume
• AI interview based on your resume
• Submit form
• *Resources & Support**
• For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
• For any help or support, reach out to: [email protected]
• PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.*
Apply tot his job
Apply To this Job
• *Mercor** connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include **Benchmark**, **General Catalyst**, **Peter Thiel**, **Adam D'Angelo**, **Larry Summers**, and **Jack Dorsey**.
• *Position:** Software Engineer II
• *Type:** **Full-Time**
• *Compensation:** **$70–$95/hour**
• *Location:** **Remote**
• *Role Responsibilities**
• Drive technical implementation and testing of complex data pipeline features within the **SPECTRA** platform. Ensure robust end-to-end functionality and meet project release timelines.
• Design and develop data processing pipelines using **SQL** and **Python**. Handle large-scale data collection, transformation, and quality monitoring workflows.
• Build and maintain dashboards and visualization tools. Provide visibility into data quality, rater performance, and platform metrics for stakeholders across multiple teams.
• Create comprehensive end-to-end tests and monitoring alerts. Ensure platform reliability, data integrity, and proactive identification of issues in the data collection workflow.
• Debug and troubleshoot complex data flow issues across the **SPECTRA** platform. Identify root causes and implement solutions to maintain high system availability.
• Collaborate closely with Technical Leads and engineering teams. Implement backend data processing logic and ensure adherence to development standards and best practices.
• *Qualifications**
• *Must-Have**
• Experience in **SQL** development writing complex queries, data transformations, and working with large datasets in production environments.
• Proficient in **Python** with experience in data processing, automation, and pipeline development using object-oriented programming principles.
• Experience with data engineering pipelines including **ETL** processes, data modeling, and building scalable data processing workflows.
• Hands-on experience with data visualization and dashboard tools such as **Tableau** or similar platforms.
• Strong analytical and problem-solving skills with demonstrated ability to debug complex data issues and implement effective solutions.
• Experience with data quality monitoring and automated testing frameworks.
• *Preferred**
• **Degree in Computer Science** or related field.
• **Machine Learning** experience with understanding of ML model training data requirements, feature engineering, and model evaluation metrics.
• Experience building and scaling large products or systems.
• Experience with building privacy and ads-related products.
• Experience working with human labeling and annotation systems including workflow design, quality assurance, and vendor management in data collection operations.
• Prior experience working with the company or its internal tools.
• *Compensation & Legal**
• Pursuant to the California Fair Chance Act, Los Angeles County Fair Chance Ordinance for Employers, Los Angeles Fair Chance Initiative for Hiring Ordinance, and San Francisco Fair Chance Ordinance, qualified applicants will be considered for assignment with arrest and conviction records.
• *Application Process (Takes 20–30 mins to complete)**
• Upload resume
• AI interview based on your resume
• Submit form
• *Resources & Support**
• For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
• For any help or support, reach out to: [email protected]
• PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.*
Apply tot his job
Apply To this Job