Data Migration s/w engg

Job summary

Job Summary Design and optimize robust data platforms as an Architect with extensive experience in Python Databricks SQL Databricks Workflows and PySpark working in a hybrid model during day shifts and focusing on scalable analytics solutions that power critical business decisions while improving data reliability performance and usability across the enterprise.

Benefits

Qualifications

• and provide robust access control within the Databricks environment.
• Mentor less experienced data engineers and architects by sharing best practices in Python development PySpark optimization and Databricks platform usage to elevate overall team capability.
• Monitor production data pipelines and workflows to proactively identify performance bottlenecks capacity issues or failure patterns and implement sustainable technical remediation.
• Demonstrate extensive hands on experience designing and implementing data engineering solutions using Python with a strong focus on clean coding practices and reusable components.
• Show deep proficiency in Databricks SQL including writing complex analytical queries tuning execution plans and working with large scale structured and semi structured datasets.
• Apply advanced PySpark
• of the successful candidate.
• This position is also eligible for Cog***’s discretionary annual incentive program, based on performance and subject to the terms of Cog***’s applicable plans.
• Medical/Dental/Vision/Life Insurance
• Paid holidays plus Paid Time Off
• 401(k) plan and contributions
• Long-term/Short-term Disability

Responsibilities

• Design optimized data models and lakehouse structures on Databricks that improve query performance reduce compute costs and enhance reliability for business intelligence and advanced analytics use cases.
• Configure and orchestrate Databricks Workflows to automate complex data processes enforce dependencies and provide predictable repeatable execution patterns for critical data jobs.
• Collaborate with product owners and business stakeholders to translate analytical needs into practical technical architectures that align with organizational goals and deliver measurable business value.
• Implement strong data governance practices including data quality rules validation checks and monitoring frameworks to increase trust in enterprise data assets and reduce operational risk.
• Optimize Python and PySpark code for performance scalability and maintainability by applying best practices in modular design resource management and efficient data handling approaches.
• Guide teams on effective use of Databricks SQL for analytics reporting and interactive exploration ensuring queries are tuned and consistent with established data modeling standards.
• Establish and document architecture patterns standards and guidelines for hybrid work delivery enabling distributed teams to collaborate effectively while maintaining secure access to data platforms.
• Evaluate new features within Databricks and related data ecosystem

Degrees

Work schedule

Industry

Company size