AI engineer-Data Databricks Pltform

Databricks Platform Engineer responsible for designing, building, configuring, and operationalizing enterprise Data & AI solutions on the Databricks platform across development, testing, and production environments. The role involves platform administration, data pipeline integration, AI & ML enablement, DevOps, security, governance, monitoring, and collaboration with stakeholders to ensure reliable, secure, and scalable data services.

• Design, build, configure, and maintain Databricks workspaces across development, testing, and production environments
• Implement platform standards, reusable frameworks, templates, and best practices
• Manage Unity Catalog, clusters, SQL Warehouses, compute policies, workspace configurations, and access controls
• Automate platform deployment and configuration using Infrastructure as Code (Terraform, CI/CD pipelines)
• Build and integrate scalable data pipelines using Databricks Lakehouse architecture
• Design ingestion frameworks for batch, streaming, API, database, and file-based integrations
• Implement Delta Lake, Structured Streaming, and medallion architecture patterns
• Integrate Databricks with enterprise data platforms such as Snowflake, SAP, Oracle, SQL Server, Azure Data Factory, Kafka, and cloud storage services
• Enable ML and Generative AI workloads on Databricks (MLflow, model lifecycle management, feature stores, model serving)
• Integrate Databricks with LLMs, vector databases, AI gateways, and enterprise AI platforms
• Establish CI/CD pipelines for data and AI workloads; implement automated testing, deployment, monitoring, and rollback mechanisms
• Create reusable deployment frameworks and engineering accelerators; support release management and environment promotion processes
• Implement enterprise security controls, RBAC, data masking, encryption, and audit logging; configure and manage Unity Catalog governance policies
• Ensure compliance with enterprise security, privacy, and regulatory requirements; partner with cybersecurity teams for platform hardening
• Implement platform observability, monitoring, and operational dashboards; configure logging, alerting, performance monitoring, and incident management processes
• Optimize platform performance, cost, scalability, and reliability; support production operations and resolve platform issues
• Collaborate with architects, data engineers, AI engineers, security teams, and business stakeholders; provide technical guidance and platform best practices
• Participate in architecture reviews and platform roadmap planning; mentor junior engineers and contribute to engineering excellence initiatives

• Bachelor's or master's degree in Computer Science, Engineering, IT, or a related field
• 5+ years of experience in Data, Platform, or Cloud Engineering
• 3+ years hands‑on experience with the Databricks Lakehouse Platform
• Strong expertise in Delta Lake, Unity Catalog, MLflow, Databricks Workflows, Structured Streaming, PySpark, and Spark SQL
• Experience with cloud platforms such as Azure, AWS, or GCP
• Proficiency in Python and SQL
• Strong knowledge of DevOps, CI/CD, Infrastructure as Code (IaC), and platform automation
• Hands‑on experience with Terraform, GitHub Actions, Azure DevOps, or equivalent CI/CD tools