Data QA Engineer

The ETL/Data QA Engineer will be responsible for ensuring the accuracy, reliability, and quality of data processed through the organisation's data warehousing and integration platforms. This role involves validating ETL processes, conducting detailed source-to-target data verification, and contributing to continuous improvements in data quality practices using Python, SQL, and Azure tools.

• Validate ETL mappings, workflows, and transformations to ensure compliance with business and technical requirements
• Perform functional and non-functional testing for data validation using Azure components such as Azure Databricks and Azure Data Factory
• Execute ETL test cycles, including data-quality checks for both full and incremental data loads
• Conduct smoke, integration, regression, and other tests on products under development
• Collaborate with developers, business analysts, and users to understand testing needs and identify opportunities for automation
• Utilize Informatica PowerCenter for ETL validation, debugging, and workflow monitoring
• Use Unix/Linux commands for job monitoring, log analysis, and operational support

• 5+ years’ experience in data automation using Python, PySpark
• Hands-on experience in building customized automation frameworks using Python, PySpark, and SQL
• Extensive experience with ETL concepts, data migration testing, data warehouse testing, and BI Reporting
• High level of proficiency in writing complex SQL queries in Azure Databricks
• Experience with Azure Data Factory, Azure Databricks, and Azure DevOps
• Bachelor's Degree in IT, Business, Computer Science, or a related field