Big Data Engineer (English Speaker)

DESCRIPTION

Templeton and Partners Limited is currently is looking to hire a group of English speaking experienced Big Data Engineers to join one of our international clients in Netherlands for a 12 months+ project.

THE TASK

- Design and implement distributed data processing pipelines using Spark, Hive, Sqoop, Python, and other tools and languages prevalent in the Hadoop ecosystem.
- Ability to design and implement end to end solution.
- Build utilities, user defined functions, and frameworks to better enable data flow patterns.
- Research, evaluate and utilize new technologies/tools/frameworks centered around Hadoop and other elements in the Big Data space.
- Define and build data acquisitions and consumption strategies
- Build and incorporate automated unit tests, participate in integration testing efforts.
- Work with teams to resolving operational & performance issues
- Work with architecture/engineering leads and other teams to ensure quality solutions are implements, and engineering best practices are defined and adhered to.

SKILLS & REQUIREMENTS

- MS/BS degree in a computer science field or related discipline
- 6+ years' experience in large-scale software development
- 1+ year experience in Hadoop or big data technologies.
- Strong Java programming, Python, Shell Scripting, and SQL
- Strong development skills around Hadoop, Spark, Hive, and Pig
- Good understanding of file formats including JSON, Parquet, Avro, and others
- Experience with performance/scalability tuning, algorithms and computational complexity
- Ability to understand relational database schemas
- Proven ability to work cross functional teams to deliver appropriate resolution
- Experience with AWS components and services, particularly, EMR, S3, and Lambda
- Automated testing, Continuous Integration/Continuous Delivery

NICE TO HAVE

- Experience with open source NOSQL technologies such as HBase, DynamoDB, Cassandra
- Experience with messaging & complex event processing systems such as Kafka and Storm
- Scala
- Machine learning frameworks
- Statistical analysis with Python, R or similar
- Experience (at least familiarity) with data warehousing, dimensional modelling and ETL development