Lead Data Engineer
<p><b>Salary: £88,000 - 97,900 per year</b></p>
<b>Requirements:</b>
<ul><li>Proven experience as a Lead or Senior Data Engineer in enterprise-scale environments</li><li>Hands-on expertise with Hadoop, Spark, and Splunk</li><li>Advanced Python development skills</li><li>Experience designing and optimising high-performance data pipelines</li><li>Strong understanding of CI/CD, source control, and automated testing</li><li>Analytical, problem-solving, and leadership skills</li><li>Experience working in regulated, enterprise environments (e.g., banking, fintech, government)</li><li>Agile delivery experience (Scrum/Kanban)</li><li>Ability to mentor and uplift team performance</li><li>Strong communication and stakeholder engagement skills</li><li>Collaborative, delivery-focused mindset with high accountability</li><li>Knowledge of control, compliance, and regulatory requirements</li><li>Up-to-date awareness of modern tools, cybersecurity, and data privacy regulations</li><li>Champions innovation, advanced technologies, and best practices</li></ul>
<b>Responsibilities:</b>
<ul><li>Lead the design, development, and optimisation of enterprise data engineering platforms</li><li>Build and maintain robust ETL/ELT pipelines integrating large, complex datasets</li><li>Work with structured, semi-structured, and unstructured data across SQL and NoSQL technologies</li><li>Develop solutions using Hadoop, Spark, and Splunk in large-scale environments</li><li>Write maintainable Python code, applying object-oriented and functional programming principles</li><li>Implement and maintain CI/CD pipelines, automated testing, and version control</li><li>Collaborate with BI, Analytics, and downstream teams to support reporting and insights</li><li>Pair program and mentor other engineers to promote knowledge sharing and code quality</li><li>Define and maintain technical test plans including unit and integration tests</li><li>Promote SRE principles to ensure service resilience, sustainability, and recoverability</li></ul>
<b>Technologies:</b>
<ul><li>CI/CD</li><li>ETL</li><li>Hadoop</li><li>Support</li><li>Kanban</li><li>NoSQL</li><li>Python</li><li>SQL</li><li>Spark</li><li>Splunk</li></ul>
<p><b>More:</b></p>
<p>We are seeking an experienced Lead Data Engineer to design, develop, and optimise enterprise-scale data platforms for large, regulated organisations, ideally in banking, financial services, or other regulated sectors. This role requires hands-on technical expertise, leadership, and a consulting mindset to deliver scalable, resilient data solutions while promoting best practices and operational excellence. The position is a hybrid role based in Sheffield, offering a contract for 6 months with the possibility of extension at a rate of £400 per day inside IR35.</p>
<p>last updated 8 week of 2026</p>
<b>Requirements:</b>
<ul><li>Proven experience as a Lead or Senior Data Engineer in enterprise-scale environments</li><li>Hands-on expertise with Hadoop, Spark, and Splunk</li><li>Advanced Python development skills</li><li>Experience designing and optimising high-performance data pipelines</li><li>Strong understanding of CI/CD, source control, and automated testing</li><li>Analytical, problem-solving, and leadership skills</li><li>Experience working in regulated, enterprise environments (e.g., banking, fintech, government)</li><li>Agile delivery experience (Scrum/Kanban)</li><li>Ability to mentor and uplift team performance</li><li>Strong communication and stakeholder engagement skills</li><li>Collaborative, delivery-focused mindset with high accountability</li><li>Knowledge of control, compliance, and regulatory requirements</li><li>Up-to-date awareness of modern tools, cybersecurity, and data privacy regulations</li><li>Champions innovation, advanced technologies, and best practices</li></ul>
<b>Responsibilities:</b>
<ul><li>Lead the design, development, and optimisation of enterprise data engineering platforms</li><li>Build and maintain robust ETL/ELT pipelines integrating large, complex datasets</li><li>Work with structured, semi-structured, and unstructured data across SQL and NoSQL technologies</li><li>Develop solutions using Hadoop, Spark, and Splunk in large-scale environments</li><li>Write maintainable Python code, applying object-oriented and functional programming principles</li><li>Implement and maintain CI/CD pipelines, automated testing, and version control</li><li>Collaborate with BI, Analytics, and downstream teams to support reporting and insights</li><li>Pair program and mentor other engineers to promote knowledge sharing and code quality</li><li>Define and maintain technical test plans including unit and integration tests</li><li>Promote SRE principles to ensure service resilience, sustainability, and recoverability</li></ul>
<b>Technologies:</b>
<ul><li>CI/CD</li><li>ETL</li><li>Hadoop</li><li>Support</li><li>Kanban</li><li>NoSQL</li><li>Python</li><li>SQL</li><li>Spark</li><li>Splunk</li></ul>
<p><b>More:</b></p>
<p>We are seeking an experienced Lead Data Engineer to design, develop, and optimise enterprise-scale data platforms for large, regulated organisations, ideally in banking, financial services, or other regulated sectors. This role requires hands-on technical expertise, leadership, and a consulting mindset to deliver scalable, resilient data solutions while promoting best practices and operational excellence. The position is a hybrid role based in Sheffield, offering a contract for 6 months with the possibility of extension at a rate of £400 per day inside IR35.</p>
<p>last updated 8 week of 2026</p>