Big Data Specialist (ETL Hadoop)

Hadoop Data Movement Expert

Main objective/Context

Within Global Technology, the Product Development team is in charge of developing the company's Global Information Systems and maintaining them aligned with company's business needs.

In the scope of a ground-breaking Business Intelligence project, we are looking for a Hadoop data movement expert to take ownership of the data feeding part from this new platform by reviewing and automating integration feeds.

The Hadoop data movement expert will be a member of a cross-functional highly skilled Agile Scrum Team.

Main responsibilities & duties

Provide automation on diverse data feeds into our lake
Review feeds design
Check for PII or PCI data items and take needed actions to ensure compliance with standards
Industrialization and orchestration of the loads
Interface with third parties to setup secure feed connections
ETL flows between DBs and the Lake
External interfaces via API connections
Ensure performance of the data sets stored in the lake
Perform performance analysis on a regular basis
Assist the Hadoop specialist with tuning activities (shredding & sharding)
Take proactive actions to meet customer's performance expectations
Integrate with an Agile Scrum team
Effectively communicate with the rest of the Scrum team (Product Owner, Developer, QA, and Business Analysts)
Provide update on activities as well as known impediment/issues
Express issues in a clear, consistent and understandable manner (written, verbal)
Take ownership of all assigned tasks and project related assignments
Ensure direct communication over email/ticketing chain to ensure timely resolution of identified issues

Technical Skills

Hadoop data movement toolset
Knowledge of the Cazena product or Cloudera distribution
Experienced with SQOOP, HIVE and Spark
Experienced with Oozie and Hue
Experience working with
Development Collaboration Suite (Rally, JIRA, HP QC, SharePoint or equivalent)
ETL tools (Informatica or equivalent)
RestFul APIs for provisioning data into a big data system
Experience working with sensitive data (PCI, PII) and how to secure them for data analytics usage
Able to moderate conference calls
Written technical documentation authoring skills (user manuals, how to)
MS Office, PowerPoint, Outlook
English fluent: written and verbal
Abilities to deal within a multicultural and distributed environment (ability to communicate effectively in a conference call setting)
Service oriented (takes pride in maintaining a top notch environment for internal and external users)
Excellent communication skills (written and verbal)
Creative solving of technical problems
Able to focus on deadlines and deliverables in an autonomous manner
Enthusiastic about his/her field and related technologies
Highly autonomous individual

Education/Experience

Bachelor or Masters of Science in Computer Science or Computer Engineering or equivalent
3 years+ experience in industrialization of data feeds to a Hadoop cluster in a managed way
Multi-cultural experience on geographically distributed projects
Travel Industry related experience is a plus.