Lead Data Engineer
<b>Requirements:</b>
<ul><li>Proven experience building and owning data platforms on Azure</li><li>Deep, hands-on knowledge of Databricks (lakehouse architecture, cluster management, performance and cost optimisation)</li><li>Strong opinions on data modelling, schema design, and standardisation</li><li>Experience treating data pipelines as software: version control, CI/CD, automated testing</li><li>Comfortable challenging architectural decisions and explaining why</li><li>Ability to translate technical trade-offs into business impact for non-technical stakeholders</li><li>Natural mentoring capabilities to raise the team through pairing, guidance, and example</li><li>Experience with industrial, sensor-driven, or time-series data is highly desirable</li><li>Background in high-volume or highly variable data environments will also transfer well</li></ul>
<b>Responsibilities:</b>
<ul><li>Architect and rebuild the data transformation layer in Databricks</li><li>Design robust data flows that support both real-time operational views and deep historical analysis</li><li>Move pipelines from ad-hoc scripts to software-engineering standards (CI/CD, testing, modular design)</li><li>Define clear data models, schemas, and standards across a complex data estate</li><li>Establish a stable, high-performance serving layer for analytics, visualisation, and data science workloads</li><li>Work closely with data scientists to remove bottlenecks and enable better modelling and experimentation</li><li>Engage in pair-programming and mentoring to elevate the engineering standards of the team</li><li>Set direction beyond just delivering features</li></ul>
<b>Technologies:</b>
<ul><li>Architect</li><li>Azure</li><li>CI/CD</li><li>Databricks</li><li>Support</li><li>Cloud</li><li>Grafana</li><li>Machine Learning</li><li>MQTT</li><li>Python</li><li>SQL</li><li>dbt</li></ul>
<p><b>More:</b></p>
<p>We are a forward-thinking company based in Sheffield, offering a hybrid working model with two days per week in person. As a Lead Data Engineer, youll have the autonomy to shape the architecture of our modern data platform, with a salary of up to £85k depending on experience. We value professional development, providing five weeks of paid holiday plus bank holidays, tax-efficient stock options, a company pension scheme, and training support. Our team is committed to transparency and recognition, and we invite diverse applicants to join us in this impactful role.</p>
<p>last updated 8 week of 2026</p>