Project Description:
We are seeking a highly skilled Databricks Platform Engineer with strong experience in data engineering. The candidate will have a deep understanding of both data platforms and software engineering, enabling them to effectively integrate and operate the platform within a broader IT ecosystem.
This role requires a hands-on individual contributor who takes full ownership of deliverables end-to-end, including design, development, testing, deployment, and ongoing support.
Responsibilities:
• Manage and optimize Databricks data platform including workspace setup, cluster policies, job orchestration, Unity Catalog, cost controls, multi-tenancy.
• Design, write and maintain APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management
• Ensure high availability, security, and performance of data systems which includes access control, secrets management, RBAC, monitoring, alerting, RLS, incident handling, performance tuning.
• Provide valuable insights about the data platform (Databricks) usage which includes cost attribution, usage analytics, workload patterns, telemetry.
• Implementing new features of Databricks, including serverless, Declarative Pipelines, Agents, lakebase , etc.
• Design and maintain system libraries (Python) used in ETL pipelines and platform governance (Databricks).
• Optimize ETL Processes - Enhance and tune existing ETL processes for better performance, scalability, and reliability.
Mandatory Skills Description:
• Minimum 10 Years of experience in IT/Data.
• Minimum 5 years of experience as a Databricks Data Platform Engineer.
• 3+ years of experience in designing, writing, and maintaining APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management
• Bachelor's in IT or related field.
• Infrastructure & Cloud: Azure, AWS (expertise in storage, networking, compute).
• Programming: Proficiency in PySpark for distributed computing.
• minimum 4 years of Python experience for ETL development.
• SQL: Expertise in writing and optimizing SQL queries, preferably with experience in databases such as PostgreSQL, MySQL, Oracle, or Snowflake.
• Data Warehousing: Experience working with data warehousing concepts and Databricks platform.
• ETL Tools: Familiarity with ETL tools & processes
• Data Modelling: Experience with dimensional modelling, normalization/denormalization, and schema design.
• Version Control: Proficiency with version control tools like Git to manage codebases and collaborate on development.
• Data Pipeline Monitoring: Familiarity with monitoring tools (e.g., Prometheus, Grafana, or custom monitoring scripts) to track pipeline performance.
• Data Quality Tools: Experience implementing data validation, cleaning, and quality frameworks, ideally Monte Carlo.
Nice-to-Have Skills Description:
• Containerization & Orchestration: Docker, Kubernetes.
• Infrastructure as Code (IaC): Terraform.
• Understanding of Investment Data domain (desired).
Languages:
English: C1 Advanced
Data Engineer (Databricks) • Valencia, Província de València, Spain
A technology company specialising in IT services and products, built on the talent and skills of our people – over 5,000 professionals developing technology with a human touch.We’re expanding our b...Mostrar más
Our goal is to strengthen the brand in Spain.We are looking for an expert to work as.What do you need to fulfill this job?.PL/SQL procedures, packages, and functions.API-driven integrations, and ba...Mostrar más
En OPTIMISSA tenemos claro que el éxito de nuestros proyectos se debe a las personas que forman nuestro equipo.Por eso, si tienes experiencia como.Ingeniero de Soporte a Producción.Quieres construi...Mostrar más
Senior Data Engineer – AI Labs at Insud Pharma.Data Engineering or Analytics Engineering roles.AI Labs is Insud Pharma’s transversal team for.Artificial Intelligence, Data Science, and Machine Lear...Mostrar más
Porque trabajar en Sopra Steria, también es sentir.Somos un reconocido líder europeo en consultoría, servicios digitales y desarrollo de software, con cerca de 56.Nos enfocamos en las personas, en ...Mostrar más
Data Engineer (Data Platform & Cloud).Barcelona, especializada en tecnología aplicada a servicios urbanos.La misión de la empresa es ayudar a las ciudades a mejorar la gestión de los servicios urba...Mostrar más
En Ioon estamos convencidos de que la tecnología será el eje de la transformación de nuestras vidas,.Te animas a ser parte de esta revolución?.Te proponemos esta posición por si te quieres sumar a ...Mostrar más
Sigue leyendo porque tenemos lo que buscas….Mostrar más
Buscando nuevos retos? ¿Quieres formar parte de nuestro equipo?.Mostrar más
Would you like to join an innovative project where technology is used to make a positive impact by combating misinformation and fake news?.We’re seeking Principal Data Engineers to join our Data te...Mostrar más
Capitole Consulting sigue creciendo y queremos hacerlo contigo!.Horario flexible y jornada intensiva.SQL avanzado, con foco en: BigQuery SQL.Plataforma Cloud Google Cloud Platform.Diseño y desarrol...Mostrar más
Senior AI & Data Systems Engineer.Biome Makers is a global AgTech on a mission to restore soil health! We are a fun, multicultural and energetic team, composed of industry experts, top scientists, ...Mostrar más
Tu misión será diseñar, construir y mantener soluciones de datos robustas para clientes de sectores como.Trabajarás en proyectos de largo recorrido, optimizando desde el seguimiento avanzado (.Mostrar más
Company Description Obtenga más información sobre las tareas generales relacionadas con esta oportunidad a continuación, así como sobre las habilidades requeridas.Miratech helps visionaries change ...Mostrar más
Quiénes somos Aunque la experiencia profesional y las cualificaciones son clave para este puesto, asegúrese de comprobar si posee las habilidades interpersonales preferibles antes de solicitar, si ...Mostrar más
Take your career to the next level with Amaris Consulting as a Data Engineer.Become part of an international team, thrive in a global group with €800M turnover and 1,000+ clients worldwide, and an ...Mostrar más
We’re Hiring: Software Engineer / Data Engineer (Azure, Databricks, Python).We’re looking for a skilled Software/Data Engineer to join a high-impact project focused on modernising applications with...Mostrar más
Data Engineer - Industrial Digital Platform.Our Industrial Digital Platform team is looking for a Data Engineer to build and scale the data backbone that powers decision-making across engineering, ...Mostrar más
