Python, AWS Services, Pyspark Developer

CGI is seeking an experienced Python, AWS Services, and PySpark Developer to design, develop, optimize, and maintain scalable data engineering solutions on cloud platforms. The role involves developing robust ETL/ELT pipelines, processing large volumes of structured and unstructured data, and collaborating with cross-functional teams to deliver enterprise-grade data solutions.

• Design, develop, test, and maintain scalable data processing applications using Python and PySpark
• Build and maintain ETL/ELT pipelines for data ingestion, transformation, validation, and integration
• Work with AWS services such as Amazon S3, AWS Glue, AWS Lambda, Amazon EMR, Amazon Redshift, and AWS Step Functions
• Optimize Spark jobs through partitioning, caching, join strategies, and resource tuning
• Monitor pipeline execution, troubleshoot failures, and resolve performance bottlenecks in production environments

• 5+ years of experience in Python programming and distributed data processing using PySpark
• Strong hands-on experience building ETL/ELT pipelines and cloud-based data pipelines using AWS services
• Proficiency in SQL for data extraction, analysis, validation, and reconciliation
• Experience with version control and CI/CD tools such as Git