Senior Data Engineer

Senior Data Engineer Remote iO Associates is looking for aSenior Data Engineerfor one of our leading clients to join the Innovation Technology team. In this role, you'll be a key player in designing and maintaining data pipelines on Azure using Databricks, ADF, and PySpark. They ensure data is secure, reliable, and optimised for analytics, while mentoring engineers and driving innovation to improve the data ecosystem. Required 8 years of progressive data engineering and 3+ years of experience in a leadership experience supporting customer-facing products at scale. Deep expertise with Microsoft Azure architecture, including PaaS databases, Synapse, ADF pipelines, and Synapse stored procedures. Required 3+ years of experience working with Databricks (Delta Live Tables, Unity Catalog mandatory). Strong experience of building and managing Azure Data Factory pipelines. Strong skills in Python and PySpark, with additional knowledge of Spark, MapReduce, and Scala, a plus. Proficiency in developing batch ETL pipelines, real-time pipelines and a strong understanding of dimensional modelling, ETL, data governance, warehousing, and working with structured/unstructured data. What You'll Do Design, build, and optimise large-scale data pipelines using Databricks, Delta Live Tables, ADF, and PySpark. Develop and maintain data ingestion and transformation frameworks that integrate multiple sources into Azure-based platforms. Implement and manage Unity Catalog for data governance, security, and lineage. Collaborate with developers, analysts, and product teams to deliver high-quality, reliable datasets for analytics and customer-facing applications. Monitor and tune performance across Azure SQL, Cosmos DB, and Azure Data Lake Gen2 to ensure scalability. Be a hands-on leader: review code, enforce best practices, and mentor junior engineers. This is the position for you if you want to work for a company that values its employees' health and well-being and invests in developing and rewarding talent. If you are interested in learning more about the opportunity, please email

Apply Now