Senior Databricks Engineer

Job summary

About This Role Bou*** is a premier end-to-end digital transformation consultancy dedicated to partnering with ambitious brands to create digital solutions for today’s complex challenges and tomorrow’s opportunities.

Benefits

Qualifications

• With uncompromising standards for technical and domain expertise, we deliver innovative and strategic solutions in Strategy, Analytics, Digital Engineering, Cloud, Data & AI, Experience Design, and Marketing.
• into reliable data products.
• Establish engineering best practices, conduct code reviews, and mentor junior data engineers.
• Monitor production pipelines, troubleshoot failures, and drive root-cause analysis and continuous improvement.
• Required
• 5+ years of data engineering experience, with 3+ years building production solutions on Databricks and Apache Spark.
• Expert proficiency in Python (PySpark) and advanced SQL.
• Deep hands-on experience with Delta Lake, Unity Catalog, and the medallion architecture pattern.
• Strong experience with at least one major cloud platform (AWS, Azure, or GCP) and its core data services.
• Proven track record optimizing Spark performance and managing cluster cost.
• Experience with data modeling, warehousing concepts, and building dimensional/analytics-ready datasets.
• Proficiency with Git-based version control, CI/CD, and infrastructure-as-code.

Responsibilities

• Promote and enforce awareness of key information security practices, including acceptable use of information assets, malware protection, and password security protocols
• Identify, assess, and report security risks, focusing on how these risks impact the confidentiality, integrity, and availability of information assets
• Understand and evaluate how data is stored, processed, or transmitted, ensuring compliance with data privacy and protection standards (GDPR, CCPA, etc.)
• Ensure data protection measures are integrated throughout the information lifecycle to safeguard sensitive information
• ROLE AND
• Architect, build, and maintain scalable ETL/ELT pipelines on the Databricks Lakehouse Platform using PySpark, Spark SQL, and Delta Lake.
• Design and implement medallion (bronze/silver/gold) data architectures and enforce data quality, governance, and lineage standards.
• Optimize Spark jobs and cluster configurations for performance and cost, including partitioning, caching, and autoscaling strategies.
• Implement and manage Unity Catalog for access control, data governance, and cross-workspace asset sharing.
• Build and orchestrate workflows using Databricks Workflows, Delta Live Tables, and CI/CD pipelines.
• Collaborate with data scientists, analysts, and business stakeholders to translate

Degrees

Industry

Company size

Security clearance