jobapplier

Senior Data Engineers

VIDAA (Netherlands) International Holdings B.V.

Skills

Required

  • Software Engineering
  • Data Science
  • big data development
  • real-time
  • data warehouse implementation
  • data ingestion
  • system construction
  • streaming task development
  • Python
  • Scala
  • Java

Preferred

  • Google Cloud
  • Azure
  • cloud
  • big data platforms
  • data governance
  • user behavioral data
  • Kafka
  • Spark
  • SQL
  • Job Responsibilities 1. Own the construction and maintenance of the company’s business data ingestion system.
  • Lead the implementation and iterative optimization of the real-time data warehouse architecture.
  • Job Requirements 1.

About VIDAA (Netherlands) International Holdings B.V.

  • VIDAA VIDAA is a leading TV OS distributor, pioneering the next-generation platform-to-the-home.
  • Starting with a Smart TV OS, we aim to encompass various aspects of consumers' digital and connected lives.
  • With millions of households reached daily, we are looking for people that want to drive excellence and innovation.
  • We are a diverse and passionate company looking for the best and the brightest people to lead sucees.
  • We value people and their needs and we provide a safe and secure environment for everyone.
  • If you are self-motivated, entrepreneurial and know how to thrive in a fast growing environment, then we should talk.

Responsibilities

  • Responsible for the full lifecycle development of real-time collection, cleaning, and warehousing of business logs, user behavioral data, and database operational data, ensuring the completeness, timeliness, and stability of end-to-end data ingestion pipelines.
  • Design multi-layer real-time data models, develop and tune streaming ETL pipelines, and support the stable output of real-time metrics, dashboards, and business monitoring data.
  • Collaborate closely with business teams, data analysts, and data product managers.
  • Fully understand business logic and statistical calibers, independently complete data requirement decomposition, logical sorting, development implementation, release verification, and continuous iteration.
  • 4.
  • Maintain and optimize Kafka streaming pipelines for real-time data ingestion and consumption.
  • Troubleshoot and resolve core online issues including message backlog, data skew, data loss and duplication, and ensure high availability and stability of real-time data pipelines.
  • Develop, tune, and operate Spark real-time tasks.
  • Continuously optimize computing resource utilization, data latency, and throughput, and improve the overall performance and stability of streaming data pipelines.
  • Participate in data standardization construction, real-time data quality governance, and pipeline redundancy optimization.
  • Summarize and standardize data ingestion specifications and development best practices to improve the overall team’s data development efficiency.
  • Provide stable real-time data support for downstream business scenarios including BI dashboards, data services, user portrait platforms, and business effectiveness measurement, ensuring reliable online data service delivery.
  • Expert-level knowledge of Kafka.
  • Familiar with real-time data ingestion, partition strategy, consumer mechanism, and capable of diagnosing and optimizing online risks such as message backlog, data skew, data loss and duplication; experienced in large-scale streaming pipeline stability governance.
  • Hands-on experience with Spark, Spark SQL, Flink and Flink SQL.

Requirements

  • Bachelor’s degree or above in Computer Science, Software Engineering, Data Science, or related majors.
  • 3+ years of professional big data development experience in the internet industry, with proven experience in end-to-end real-time data warehouse implementation and data ingestion system construction.
  • 2.
  • Proficient in English reading, writing and verbal communication skills, capable of collaborating with overseas teams and business stakeholders.
  • Solid capabilities in streaming task development, resource tuning, latency and throughput optimization, and online troubleshooting.
  • Familiar with real-time data warehouse layered architecture and dimensional modeling methodologies.
  • Deep understanding of the big data ingestion system.
  • Experienced in full-process real-time processing of multi-source data, including user behavior logs and business database data, proficient in real-time collection, CDC synchronization, data cleaning, and exception handling with standardized implementation capabilities.
  • 6.
  • Practical experience with cloud-native data tools including BigQuery (BQ), DBT and Databricks.
  • Capable of data warehouse modeling, pipeline orchestration, metric iteration and dashboard development in cloud data environments.
  • Proficient in at least one programming language among Python, Scala and Java with solid hands-on big data development capabilities.
  • Familiar with Linux commands and Shell scripting, able to independently complete task deployment, log analysis and daily operation maintenance.
  • 8.
  • Familiar with mainstream relational databases and Binlog/CDC real-time synchronization principles.
  • Experienced in business database real-time ingestion implementation.
  • Knowledgeable in real-time data SLA governance, data quality monitoring, alert configuration and incident review mechanisms.

Preferred

  • Overseas study or work experience is highly preferred.
  • Experience with Google Cloud, Azure or other mainstream cloud big data platforms, as well as overseas team collaboration and real-time data governance experience is highly preferred.
Easy Apply