Senior Data Engineer

Overview :

We are seeking a highly skilled and experienced Senior Data Engineer with a minimum of 5 years working with Databricks and Lakehouse architecture to join our team in the consumer finance industry. The successful candidate will play a critical role in mapping requirements, designing, implementing, and maintaining scalable data solutions, and will ensure seamless integration and operation of CI / CD pipelines.

Key Responsibilities :

• Design, develop, and optimize robust data architectures using Databricks and Lakehouse principles to support large-scale and complex data analytics needs.
• Implement and maintain CI / CD pipelines to ensure continuous integration and delivery of data solutions, ensuring data quality and operational efficiency.
• Collaborate with cross-functional teams (ideally Finance) to understand data requirements, map data through distributed systems, and translate these into technical solutions that align with business objectives.
• Manage and optimize data storage and retrieval systems to ensure performance and cost-effectiveness.
• Develop, maintain, and document ETL / ELT processes for data ingestion, transformation, and loading using industry best practices.
• Ensure data security and compliance, particularly within the context of financial data, adhering to relevant regulations and standards.
• Troubleshoot and resolve any data-related issues, ensuring high availability and reliability of data systems.
• Evaluate and incorporate new technologies and tools to improve data engineering practices and productivity.
• Mentor junior data engineers and provide technical guidance to the wider team.
• Contribute to the strategic planning of data architecture and infrastructure.

Required Qualifications and Experience :

Desirable Skills :