Lead Data Engineer Full-time Job

What are your contributions to the team?

Lead/Administer the enterprise data platform (DWH, Data Lake). 

Create and maintain optimal/reliable data pipeline architecture to meet business needs. 

Define and operate the infrastructure required for optimal extraction, transformation and loading  (ETL) of data from a wide variety of data sources using SQL and ‘big data’ technologies as needed (Hadoop, MapReduce, Hive, Spark, Kafka, Pig, data streaming, NoSQL, SQL, programming) 

Design and implement life cycle management processes (DevOps) to enable continuous integration and continuous deployment (CICD) of data systems. 

Integrate data from various resources (including external data sources and IoT) and manage the big data as a key enterprise asset.  

Create and maintain backend data solutions for data analysts and data scientists. Assist them in unlocking insight from enterprise data. 

Identify, design, and implement internal process and framework (e.g. elimination of manual processes, optimizing data delivery, evolving data infrastructure capabilities, etc.) 

Work with stakeholders including product, data and architecture SME to assist with data-related technical issues and support their data infrastructure needs. 

Ensure compliance to data architecture and security requirements. 

Identify data quality issue and make recommendation for addressing root causes.  

Setup observability/monitoring to measure reliability of the data pipelines and act quickly (e.g. operational support) in case of incident. 

How to thrive in this role?

You hold a bachelor’s degree in computer science, Statistics, Informatics, Information Systems or another quantitative field. 

You have 10 years of experience in a Data Engineer / Data Specialist role . 

You have experience coaching/leading a small team (technical leadership). 

You have knowledge of Agile / SCRUM project delivery, DevOps and CICD practices. 

You have advanced knowledge of SQL, query authoring and relational databases. 

You have experience optimizing ‘big data’ pipelines (storage, file format, partitioning, spark, python, streaming) 

You are efficient at performing root cause analysis to address issues and applying long-term fix. 

You have experience designing and building data transformation, data structures, metadata framework and automated workload management. 

You have experience in data protection measures, data privacy and collaborating with Cyber team 

You have good knowledge of Azure data services (Azure Data Factory, Synapse, Azure Data Lake Storage, Event Hub, Polybase, Databricks, Delta lake, Cognitive Services, …) 

You have good knowledge of Object-oriented/object function scripting languages: Python, Java, C++, Scala, etc. 

You have good people skills and are a team player, motivated by developing other people. 

You are a good communicator, who can simplify complex technical issue for non-technical people. You can work in an environment with a mix of French and English languages.