Data Engineer (m/f/d)

REFERENCE NUMBER:

589791/1

MY DUTIES:

- Design and Develop data ingestion pipelines and processes based on requirements in Python and PySpark
- Create error handing, exception management and data quality routines to expose the anomalies in the data
- Profile and analyze data to identify gaps and potential data quality issues
- Identify relationships between disparate data sources
- Use Python, Databricks and Spark to code the data Engineering routines
- Perform unit and integration testing
- Work with the group of data scientists and business SMEs to get the requirements and present the details in data.
- Design and jointly develops the data architecture with data architect and ensures security and maintenance
- Explore suitable options, designs, and creates data pipeline (data lake / data warehouses) for specific analytical solutions
- Identify gaps and implements solutions for data security, quality and automation of processes
- Build data tools and products for effort automation and easy data accessibility
- Support maintenance, bug fixing and performance analysis along data pipeline
- Diagnoses existing architecture and data maturity and identifies gaps
- Gather requirements, assess gaps and build roadmaps and architectures to help the analytics driven organization achieve its goals

MY QUALIFICATIONS:

- Experience in data engineering using Python, PySpark and/or Spark
- Bachelor’s Degree in Computer Science, Engineering, and/or background in Mathematic and Statistics; Master’s or other advanced degree a plus
- Experience on Big Data platforms (e.g. Hadoop, Map/Reduce, Spark, HBase, HDInsight, Data Bricks, Hive) and with programming languages like UNIX shell scripting, Python etc.
- Have used SQL, PL/SQL or T-SQL with RDBMSs like Teradata, MS SQL Server, Oracle etc in production environments
- Experience with reporting and BI packages e.g. PowerBI, Tableau, SAP BO etc.

MY BENEFITS:

- Exciting project in an international company

ABOUT HAYS:

Our many years of recruitment experience and specialist knowledge of the engineering job market make us a strong partner for engineers and technicians. We maintain intensive relationships with customers across all industries and so can provide engineering specialists with exciting projects and attractive job opportunities. Whether it's working with an international automotive manufacturer or a regional market leader in mechanical engineering, drawing on your interests and experience, we will find the right job to suit you. Because with us, specialists find their ideal working environment – all completely free of charge. Register with us and reap the benefits of job offers that are both interesting and relevant to your skills and experience.

MY CONTACT AT HAYS:

My contact person
Jacqueline Montag

Reference number
589791/1

Contact
E-Mail: [email protected]