Global Data Factory Consultant

We're Hiring : Data Factory Specialist – Data Integration (Global Pharma Leader)

Are you passionate about building scalable, high-performance data pipelines in a global environment? Our client, a leading pharmaceutical company, is seeking a Data Factory Specialist to join their Data & Analytics team and help drive their enterprise data strategy forward.

About the Role

As a Global Data Factory Specialist , you’ll be responsible for designing and maintaining robust data pipelines that integrate data from diverse sources into clean, structured layers. You’ll work closely with data analysts and business stakeholders to deliver high-quality, curated data that powers decision-making across the organization.

This is a hands-on technical role with a strong focus on cloud-native data engineering, data quality, and production-grade reliability.

Key Responsibilities

Design, develop, and maintain scalable data pipelines using AWS Glue, Lambda, and Airflow.

Integrate data from APIs, databases, and file systems, ensuring robust error handling, retry logic, and monitoring.

Implement data quality checks, validation logic, and transformation processes.

Work with Iceberg format to support schema evolution and time travel capabilities.

Collaborate with cross-functional teams to understand data requirements and deliver fit-for-purpose solutions.

Document workflows and communicate clearly with data consumers regarding pipeline performance and incident remediation.

Must-Have Experience

AWS Glue and Lambda : Proven hands-on experience building and deploying production-grade pipelines.

Iceberg Format : Deep understanding of schema evolution and time travel features.

Data Quality : Experience implementing validation logic and quality checks.

Source System Integration : Skilled in connecting to APIs, databases, and file systems with robust error handling.

Airflow : Experience building and maintaining DAGs in production environments.

Qualifications and Skills

Bachelor’s degree in Computer Science, IT, or related field.

Strong experience in data engineering or data integration.

Strong proficiency in PySpark, SQL, and cloud platforms (AWS, Snowflake).

Experience with pharma datasets and platforms (e.g., Veeva CRM, Adobe Campaign, MDG) is a plus.

Excellent communication, collaboration, and problem-solving skills.