Chief Data Architect

Job Description

As a senior software engineer, you will play a pivotal role in designing and developing scalable backend infrastructure for our data-intensive web application. This involves optimizing storage / database workloads, ensuring high availability for our AI / ML backend services, scaling integrations with multiple tools for ingesting different data types, and building robust data pipelines for data transformation and processing.

Key Responsibilities :

• Design and build scalable backend services for our data-intensive web app and AI / ML model serving
• Make strategic database and storage trade-offs to optimize performance when managing large amounts and varieties of data, including vector stores
• Build and maintain batch and real-time data pipelines that can scale with increased load and data
• Develop and maintain data integrations with different data sources and platforms (source and sink)
• Design and manage the evolution of flexible data models and database infrastructure to handle different types of data sources and data points, optimizing for fast reads / writes
• Implement and manage a robust workflow orchestration for multiple data jobs
• Set up robust testing practices, monitoring, and observability to ensure the reliability of all services and data pipelines

Required Skills and Qualifications :

Minimum qualifications include a bachelor's or master's degree in computer science or a related field, solid CS fundamentals, distributed systems knowledge, experience building scalable data pipelines, orchestrating workflows, and optimizing pipelines, and experience working with data warehouse technologies.