Azure Data Engineer - Databricks/PySpark

Pravi HR Advisory is hiring an Azure Data Engineer to build and maintain a large‑scale enterprise data lake for a leading FMCG/CPG organization. The role focuses on designing, developing, and optimizing ETL/ELT pipelines in Azure Databricks, integrating SAP S/4HANA and other enterprise sources, and ensuring data quality and governance across the platform.

• Design, develop and maintain scalable ETL/ELT pipelines using Azure Databricks, PySpark, SQL and Azure services
• Integrate data from SAP S/4HANA and other enterprise applications into Azure Data Lake
• Develop curated data layers using Delta Lake and lakehouse architecture principles
• Implement incremental data processing, pipeline orchestration, monitoring and error handling
• Build reliable datasets for business reporting, analytics and downstream applications
• Implement data quality checks, reconciliation and validation against source systems
• Optimize data pipeline performance, processing efficiency and cloud resource utilisation
• Apply version control, CI/CD, automated deployment and testing practices
• Support data governance, access controls, lineage and technical documentation
• Troubleshoot production issues and support ongoing data platform operations

• 6–12 years of professional experience in Azure Data Engineering
• Strong proficiency in Azure Databricks, Python, PySpark, and SQL
• Hands‑on experience with Azure Data Lake Storage and Azure Data Factory (ADF) or Databricks Workflows
• Deep understanding of Delta Lake, ETL/ELT pipelines, and dimensional data modeling
• Experience integrating or processing SAP S/4HANA data, including knowledge of SAP business objects and extraction approaches such as CDS Views, ODP, or SLT
• Ability to implement incremental data processing, data quality controls, and performance optimization
• Familiarity with CI/CD, infrastructure as code, and data governance practices