Lead Data engineer (Lead II - Data Engineering)

Job summary

Role description Role: Lead Data Engineer Fixed term contract Contract Length: Initial 3-6 months with possible extensions Start Date: ASAP Experience range - 10- 12 years Location Requirement: Onsite (3 days per week in the office and 2 days remote ) Applicants mUST*** be legally authorized to work in the United Kingdom without the need for current or future visa sponsorship Responsibilities - Desig

Qualifications

• Experience range - 10- 12 years
• Location Requirement: Onsite (3 days per week in the office and 2 days remote )
• into technical data solutions.
• Support data discovery, profiling and validation activities to understand source data structures, data quality issues and data gaps.
• Develop and maintain data transformations, curated datasets and data models required to support reporting and analytical use cases.
• Monitor, troubleshoot and resolve data ingestion issues, defects and enhancements identified during development, testing, UAT and production support.
• Ensure solutions comply with data architecture standards, engineering best practices, security
• and governance frameworks.
• Produce clear technical documentation for data ingestion processes, data flows and operational support
• Provide comprehensive handover documentation and knowledge transfer to the Data Support team following delivery of data ingestion pipelines.
• Collaborate within Agile delivery teams, actively contributing to sprint planning, stand-ups, retrospectives and continuous improvement activities.
• Identify opportunities to improve pipeline performance, automation, scalability and maintainability through process and technology enhancements.

Responsibilities

• Design, build and optimise robUST*** data ingestion pipelines to acquire, transform and load data vendor platforms into client's data ecosystem.
• Develop scalable data engineering solutions using Databricks, PySpark, SparkSQL and associated cloud

Industry

Company size

Contract length