Lead – Databricks ML
Job Details
Job Title:
Lead - Databricks ML
Department:
Artificial Intelligence - AI
Location:
Abu Dhabi, UAE
Job Type:
Full Time
Salary:
TBD
Job Description of Lead - Databricks ML
Key Responsibilities of Lead - Databricks ML
✅ Lead architecture and design for data and analytics use cases.
✅ Design scalable Lakehouse architectures using Azure Databricks and ADLS Gen2.
✅ Define frameworks for data ingestion, transformation, orchestration and consumption.
✅ Architect batch and real-time data processing solutions.
✅ Develop data models for analytics, BI and AI/ML requirements.
✅ Establish data governance, security, monitoring and platform standards.
✅ Drive performance, cost optimization and scalability.
✅ Define CI/CD, DevOps and Infrastructure-as-Code standards.
✅ Provide technical leadership and mentor data engineering teams.
✅ Support technical roadmaps, solution design and effort estimation.
Qualification Required for Lead - Databricks ML
✅ Bachelor’s degree
✅ 10–12 years in data engineering and architecture
✅ Strong, hands-on experience with Azure Databricks and data architecture (essential)
✅ ADLS Gen2, Lakehouse architecture and the broader Azure Data Platform
✅ Batch and real-time data processing
✅ Data modeling for analytics, BI and AI/ML
✅ CI/CD, DevOps and Infrastructure-as-Code practices
✅ Data governance, security and cost optimization
✅ Proven technical leadership, mentoring ability and clear communication with both technical and business stakeholders.