Specialist Software Engineer - Data Engineer Databricks pySpark

Societe Generale Global Solution Centre is seeking a Specialist Software Engineer - Data Engineer with strong expertise in Databricks and pySpark. In this role, you will design, build, and optimize scalable data pipelines within a complex Big Data environment, collaborating with global cross-functional teams to deliver robust data engineering solutions.

• Translate business and technical requirements into scalable, maintainable data engineering solutions.
• Design and develop robust, high-performance data pipelines using Databricks and pySpark.
• Build and optimize data flows within the Hadoop ecosystem.
• Ensure performance tuning, resource optimization, and efficient cluster usage on the Databricks platform.
• Apply industrialization best practices including CI/CD, unit testing, and pipeline validation.
• Participate in Agile ceremonies and collaborate with Product Owners, architecture, and Ops teams.

• Bachelor of Engineering in Computer Science, Information Technology, or equivalent streams.
• 4+ years of experience in data engineering or large-scale distributed systems.
• Strong hands-on experience in pySpark for distributed processing and performance optimization.
• Solid understanding of Hadoop ecosystem tools including Hive, HDFS, YARN, Oozie, and Starburst.
• Proficiency in Python, Git, and modern CI/CD practices.