Data Engineer - Spark, Hadoop, Ozone CH, Flink | Berkley Heights, NJ (Onsite) | Contract W2 only

Job summary

Job Title: Data Engineer - Spark, Hadoop, Ozone CH, Flink Location: Berkley Heights, NJ (5 Days Onsite) Duration: long term Type: Contract W2 only Experience: 7 - 10 Yrs Job Description We are seeking a highly skilled Big Data Engineer with strong experience in Apache Spark, Hadoop ecosystem, and Apache Ozone.

Qualifications

• Experience: 7 - 10 Yrs
• We are seeking a highly skilled Big Data Engineer with strong experience in Apache Spark, Hadoop ecosystem, and Apache Ozone.
• Strong expertise in Apache Spark (Core, SQL, Streaming).
• Hands-on experience with Hadoop ecosystem (HDFS, YARN, MapReduce).
• Proficiency in Apache Ozone for object storage and integration with Hadoop.
• Solid programming
• Bachelor's degree in computer science, Software Engineering, or a related field.
• Proficiency in business process modeling and documentation

Responsibilities

• Design and implement distributed data processing solutions using Apache Spark, Hadoop, Flink
• Develop and maintain Spark applications for data transformation, aggregation, and ETL processes using Scala, Java, or Python
• Utilize Apache Ozone for storing large-scale datasets, ensuring efficient data access and management in a distributed environment
• Manage and optimize HDFS and Apache Ozone, Kafka for scalable and fault-tolerant storage.
• Develop ETL pipelines for batch and real-time data ingestion and transformation.
• Implement and ensure data validation, data security, integrity, and compliance across big data platforms.
• Monitor and troubleshoot performance issues in large-scale clusters.
• Collaborate with data scientists, analysts, and application teams to deliver high-quality data solutions.
• Automate workflows and improve operational efficiency using scripting and orchestration

Degrees

Industry

Company size

Contract length