Big Data Engineer

All vacancies of Australia • Information & Communication Technology • Big Data Engineer

My client is a global consultancy and seeking a Big Data Engineer to work at a large banking client.

Summary about this job

Developers/Programmers

Company: Genesis IT&T P/L

Location: Sydney

Work type: Full Time

Salary: 150000

Phone: +61-3-5130-8140

Fax: +61-8-6517-3905

E-mail: n\a

Site: n\a

Detail information about job Big Data Engineer. Terms and conditions vacancy

CBD Location
Permanent Role
Work-Life Balance

The Big Data Engineer would be engaging with key stakeholders and providing knowledge, experience and thought leadership in big data development and engineering solution implementation.

Responsibilities:

Quick POC implementation for technical feasibility
Generic Framework component development
Application development and unit testing
Design documentation for the sprint tasks
Run book and deployment plan preparation for sprint task
Production deployment and handover to operational team

Experience:

Hadoop data engineer/developer should have minimum of 4-8 years of total IT experience with at least 2 big data project development and production deployment experience in data lake environment. Must have Java, Shell script programming experience.
Candidate with any big data developer certification will be an added advantage.
Experience of working in an Agile Development environment.
Familiarity with Jenkins, Hudson, Bamboo, JIRA and other continuous integration tools.
Familiarity with Agile Scrum, Kanban, or Lean software development practices
Participate in Grooming sessions and other Agile ceremonies to help groom the User stories from technical stand point.
Good understanding of HDP/HDF Hadoop components
Good experience in handling data loading from disparate data sources
Good experience in handling different data formats
Good experience in handling different data load patterns in Hadoop such as – full data, incremental data and delta data handling
Good experience in data acquisition using HDP native tools like Sqoop, Flume, Hadoop copy etc or using any custom built framework using any other Hadoop native tools
Good experience in writing complex map reduce and spark programs for data transformations
Good experience in writing complex Hive hqls, and custom hive udfs
Good experience in writing complex Oozie workflows, coordinators and bundles
Good experience in Shell script/Python
Good experience in Java and REST web services
Good knowledge of any enterprise scheduler like (control-m)
Good knowledge on the performance tuning aspects in Hadoop components

More similar vacancies

Responds for Big Data Engineer on FaceBook

Read all comments for Big Data Engineer. Leave a respond Big Data Engineer in social networks. Big Data Engineer on Facebook, LinkedIn and Google+