Data Engineer (Ab Initio, Flink) | Berkeley Heights, NJ (5 Days Onsite) | Contract W2 only

Job summary

Job Title: Data Engineer (Ab Initio, Flink) Location: Berkeley Heights, NJ (5 Days Onsite) Duration: long term Type: Contract W2 Experience: 7+ Yrs Job Description: We are seeking a highly skilled resource to design and implement high-performance, event-driven data pipelines, ensuring low-latency data processing and high availability system for the large credit card processing system.

Qualifications

• Experience: 7+ Yrs
• Deep understanding of credit card process system
• Deep knowledge of GDE (Graphical Development Environment), EME (Enterprise Meta>Environment), Conduct>It, and Continuous Flows.
• Understanding of Kafka, message queues, and real-time stateful services.
• Proficiency in Unix/Linux shell scripting, SQL, and database
• Bachelor's degree in computer science, Software Engineering, or a related field.
• Proficiency in business process modeling and documentation

Responsibilities

• Create complex Ab Initio continuous flow graphs, including stateful joins, sliding time windows, and aggregations.
• Implement event-driven data pipelines using Kafka, MQ, and file streams.
• Ensure the resilience of continuous flows, including checkpointing and recovery, to guarantee "exactly-once" processing.
• Apply advanced Ab Initio components (e.g., Reformat, Rollup, Join, Partition) to ensure low-latency performance.
• Proactively monitor live production streams to ensure 24/7 reliability and troubleshooting data issues
• Develop ETL pipelines for batch and real-time data ingestion and transformation.
• Implement and ensure data validation, data security, integrity, and compliance across big data platforms.
• Monitor and troubleshoot performance issues in large-scale clusters.
• Collaborate with data scientists, analysts, and application teams to deliver high-quality data solutions.
• Automate workflows and improve operational efficiency using scripting and orchestration

Degrees

Work schedule

Industry

Company size