Big Data Engineer (m/f)

REFERENZNUMMER:

340973/3

IHRE AUFGABEN:

-Provide architectural guidance: planning, estimating cluster capacity, and creating roadmaps for Hadoop Cluster deployment
-Install Hadoop distributions, updates, patches, version upgrades
-Design, implement and maintain enterprise-level security (Kerberos, LDAP/AD, Sentry, etc.)
-Develop business relevant applications in Spark, Spark Streaming, Kafka using functional programming Methods in Scala
-Implement statistical methods and machine learning algorithms to be executed in Spark applications
-Identify new components, functions and features and drive from exploration to implementation
-Create run books for troubleshooting, cluster recovery and routine cluster maintenance
-Troubleshoot Hadoop-related applications, components and infrastructure issues at large scale
-Design, configure and manage the strategy and execution for backup and disaster recovery of big data
-3rd level support (DevOps) for business-critical applications and use cases

IHRE QUALIFIKATIONEN:

-Excellent hands-on working experience with Hadoop ecosystem including Apache Spark, Spark Streaming, Kafka, Zookeeper, Job Tracker, HDFS, MapReduce, Impala, Hive, Oozie, Flume, Sentry, but also with Oracle, MySQL, PSQL
-Strong expertise in functional programming, object oriented programming and scripting, i.e. in Scala, Java, Ruby, Groovy, Python, R
-Proficiency with IDEs (IntelliJ IDEA, Eclipse, etc.), build automation (Maven, etc.) and continuous integration tools (Jenkins, etc.)
-Strong Linux skills; hands-on experience with enterprise-level Linux deployments as well as shell scripting (bash, tcsh, zsh)
-Well-versed in installing, upgrading & managing distributions of Hadoop (CDH5x), Cloudera Manager, MapR
-Know-how regarding Hadoop cluster design, cluster configuration, server requirements, capacity scheduling, installation of services, name node, data node, zookeeper, job tracker, yarn, etc.
-Hands-on experience with automation, virtualization, provisioning, configuration and deployment
-Experience with the following technologies: Chef, Puppet, Ansible, OpenStack, VMware, Docker, etc.
-Business fluent English in (written and spoken), German is a plus

WEITERE QUALIFIKATIONEN:

Data engineer, Software developer