jobapplier

Data Engineer

SpyCloud

Skills

Required

  • AWS
  • data pipeline
  • machine learning
  • data engineering
  • deployment
  • DynamoDB
  • Terraform
  • AI
  • Python
  • Elasticsearch / OpenSearch
  • Go

Preferred

  • Lambda
  • S3
  • ECS
  • EKS

Join SpyCloud as a Data Engineer II, where you'll play a crucial role in evolving our data platform using cutting-edge technologies like Databricks and Spark. You'll design, build, and improve scalable data systems, optimize data pipelines, and collaborate with various teams to develop effective technical solutions. This hands-on role offers competitive salaries, equity, and a remote-friendly culture.

Responsibilities

  • Design, build, and improve scalable data systems with a focus on performance, and reliability.
  • Build, optimize, and maintain real-time and batch data pipelines using AWS technologies such as Lambda, S3, DynamoDB, ECS, EKS, and Terraform.

Requirements

  • Independently own complex data engineering initiatives from design and development through deployment and production support.
  • Practical experience with DynamoDB and other AWS-native data services
  • Experience building and operating Lakehouse, Databricks, or large-scale distributed data architectures in AWS
  • Strong development skills in Python, particularly when building data-intensive systems
  • 5+ years of professional experience in data engineering, software engineering, or a closely related discipline
  • Experience troubleshooting complex data pipeline, performance, and data quality issues in production environments
  • Experience preparing and managing data for machine learning and LLM workflows
  • Strong communication and collaboration skills across technical and non-technical stakeholders
  • Hands-on experience designing, building, and maintaining ETL and data-processing pipelines
  • Experience building and managing Elasticsearch/OpenSearch data stores

Experience with infrastructure-as-code tools such as Terraform

  • Ability to independently own complex technical projects from design and development through deployment and production support
  • Strong understanding of distributed systems as they relate to data storage, processing, and computing
  • Ability to evaluate technical trade-offs and contribute effectively to architectural and design decisions
  • Familiarity with AI/ML tools and technologies such as SageMaker, LangChain, or vector databases
  • Understanding of data governance, privacy, and compliance considerations within AI and ML environments
  • Experience with observability and monitoring tools for large-scale data platforms

Experience with Go or C programming languages

  • Comfort working in fast-paced environments with evolving product and technical requirements
  • Experience working with cybersecurity or security-related datasets and technologies
  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related STEM field

Pay

USD 104,000 - 135,000