jobapplier

AVP - System Development Manager - Data Platform - LME

HKEX

Skills

Required

  • data/platform engineering / backend infrastructure
  • Helm
  • authoring
  • RBAC
  • PV/PVC
  • offset management
  • monitoring
  • Kafka
  • schema
  • registry
  • Apicurio / Confluent
  • Spark
  • SQL

Preferred

  • Infrastructure
  • Kafka cluster operations
  • broker tuning
  • partition rebalancing
  • disaster recovery
  • Airflow / Dagster
  • Prometheus
  • Python
  • Scala / Java
  • Flux / Docker

Responsibilities

  • CN-Shenzhen-HyQ Shift: Standard - 40 Hours (China) Scheduled Weekly Hours: 40 Worker Type: Permanent Job Summary: Design, build, and operate critical subsystems of the data platform.
  • Take ownership of major components — streaming, batch, storage, or serving — drive them from design to production, and continuously improve their reliability and performance.
  • Responsibilities Streaming Infrastructure: Kafka cluster operations (broker tuning, partition rebalancing, monitoring, disaster recovery); manage schema registry and Kafka Connect connectors
  • Build and optimize Spark batch jobs and Flink/Spark Structured Streaming pipelines; contribute reusable job frameworks and tuning guides
  • Manage Iceberg tables (compaction, snapshot expiration, orphan file cleanup, schema evolution); operate MinIO at scale (lifecycle rules, tiering, performance tuning)

Deploy and operate

  • Trino clusters (connector config, resource groups, query monitoring); manage StarRocks/ClickHouse clusters (sharding, replication, materialized views)
  • Build and maintain Airflow/Dagster DAGs for platform operations; extend custom operators and sensors
  • Implement monitoring with OpenTelemetry, Prometheus, Grafana, and Loki across all stack layers; build dashboards and alerting rules
  • Write Helm charts, manage operator lifecycles, configure resource quotas, node affinity, pod disruption budgets CI/CD & GitOps: Own ArgoCD application sets, Helm-based deployments, and promotion pipelines from dev to prod

Participate in on-call rotation; write post-mortems and runbooks

  • Mentor mid-level engineers through pairing, design discussions, and code reviews
  • table maintenance, time-travel, catalog integration Hands-on Trino or Presto: connector configuration, query tuning, resource group management
  • StarRocks, ClickHouse, or Doris — table design, ingestion pipelines, query optimization
  • Proficient in Python and either Scala or Java Solid CI/CD and GitOps: ArgoCD or Flux, Helm, Docker Airflow or Dagster for pipeline orchestration

Requirements

  • Skills & Experience 6+ years in data/platform engineering or backend infrastructure
  • Helm chart authoring, RBAC, network policies, storage (PV/PVC), operators
  • topic design, consumer group management, offset management, monitoring lag, Kafka Connect, schema registry (Apicurio or Confluent) Solid Spark: Dataframe/Dataset API, Spark SQL, performance tuning, troubleshooting in production
  • Working knowledge of Flink or Spark Structured Streaming for real-time pipelines
  • SCC, Routes, ImageStreams, BuildConfigs dbt project experience for data transformation and modeling
  • Great Expectations, Soda, or Deequ Kafka Streams or ksqlDB for stream processing
  • Exposure to DataHub/Atlas for data discovery and lineage
  • ITD SZ 港交所科技（深圳）有限公司 ，是2016年12月28日于深圳市前海自贸区成立的外商独资企业。 作为港交所的技术子公司， 港交所科技（深圳）有限公司 主要是为集团及其附属公司提供计算机软件、计算机硬件、信息系统、云存储、云计算、物联网和计算机网络的开发、技术服务、技术咨询、技术转让；经济信息咨询、企业管理咨询、商务信息咨询、商业信息咨询、信息系统设计、集成、运行维护；数据库管理、大数据分析；以承接服务外包方式提供系统应用管理和维护、信息技术支持管理、数据处理等信息技术和业务流程外包服务。

Every job at HKEX