Lead – Databricks ML
Job Details
Job Title:
Lead - Databricks ML
Department:
Artificial Intelligence - AI
Location:
Abu Dhabi, UAE
Job Type:
Full Time
Salary:
TBD
Job Description of Lead - Databricks ML
✅ We are looking for an experienced Lead – Databricks ML to own the architecture and design of our data and analytics platforms in Abu Dhabi. In this senior role, you will shape modern Lakehouse solutions on Azure Databricks that power analytics, BI and AI/ML use cases. You will set technical direction, define platform standards and mentor data engineering teams, so your decisions will directly influence how we build and scale data-driven solutions.
✅ If you are a hands-on architect who enjoys balancing performance, cost and governance while growing a team, this role is for you.
Key Responsibilities of Lead - Databricks ML
✅ Lead architecture and design for data and analytics use cases.
✅ Design scalable Lakehouse architectures using Azure Databricks and ADLS Gen2.
✅ Define frameworks for data ingestion, transformation, orchestration and consumption.
✅ Architect batch and real-time data processing solutions.
✅ Develop data models for analytics, BI and AI/ML requirements.
✅ Establish data governance, security, monitoring and platform standards.
✅ Drive performance, cost optimization and scalability.
✅ Define CI/CD, DevOps and Infrastructure-as-Code standards.
✅ Provide technical leadership and mentor data engineering teams.
✅ Support technical roadmaps, solution design and effort estimation.
Qualification Required for Lead - Databricks ML
✅ Bachelor’s degree
✅ 10–12 years in data engineering and architecture
✅ Strong, hands-on experience with Azure Databricks and data architecture (essential)
✅ ADLS Gen2, Lakehouse architecture and the broader Azure Data Platform
✅ Batch and real-time data processing
✅ Data modeling for analytics, BI and AI/ML
✅ CI/CD, DevOps and Infrastructure-as-Code practices
✅ Data governance, security and cost optimization
✅ Proven technical leadership, mentoring ability and clear communication with both technical and business stakeholders.