Senior Big Data Engineer

Gaming Innovation Group

Responsibilities

Design, create, and maintain ETL processes, data integrations, and data pipelines
Ensure seamless handling of vast amounts of data.
Leverage extensive knowledge, the engineer develops high-performance services to efficiently process, store, and analyse large datasets
using optimised code.
Aim to maintain data accuracy and reliability, takes the lead in designing and implementing data validation and quality checks tailored to meet specific business needs.
Troubleshoot live production systems is a regular part of the engineer's responsibilities, ensuring smooth operations and swift issue resolution.
Demonstrate an innovative spirit, the engineer actively tests and
implements new technologies, staying abreast of the latest advancements relevant to current and future projects.

Strong level of expertise with big data technologies such as Spark, Kafka, Nifi, ClickHouse and Big Data datastores.
Strong understanding of schema design methodologies and data models
Expert in extracting large datasets using SQL, familiarity with SQL
databases
Experience with object oriented programming skills - ideally Python and/or Java
Strong experience with cloud computed and/or distributed systems