jobapplier

Backend Engineer – Data Acquisition

Lighthouse

Responsibilities

  • Design, build and improve backend systems for data acquisition that handle close to 200 million requests a day.
  • Work on our in-house proxy layer and browser tooling: request routing, connection handling and protocol-level behaviour.
  • Investigate hard crawler problems, test alternative approaches when one gets blocked, and turn what works into robust, reusable systems.
  • Build clear interfaces and guardrails so other teams and AI agents can use what you build reliably.
  • Own the quality of your code, including code written with AI assistance: tests, observability and maintainability.
  • Work closely with crawler engineers, DevOps and Security to ship solutions that are secure and scalable.

Requirements

  • You have hands on experience building, deploying and running production backend services in a cloud environment.

You're experienced with a systems language such as

  • R&D mindset. You're curious about how browsers and network stacks work internally, you're comfortable with ambiguity, and you'd rather prove a hypothesis than assume it.
  • You have strong analytical and debugging skills. You're comfortable with packet captures, profilers and reading other people's source code.
  • You communicate clearly, share your opinions, welcome feedback and work well in a team.
  • You're fluent in English.

Job Details

  • 💼 Engineering
  • 📍 Spain - Barcelona
  • 🕜 Full-Time/Remote

What you will do

You'll join our Crawler Engineering team as a Backend Engineer focused on data acquisition. This is a building role: you'll design and develop high-performance systems, mostly in Rust, that collect billions of data points from the web. The work sits where backend engineering meets how the web works: HTTP stacks, proxies and browser tooling. When a data source blocks an approach, you'll dig into what's happening on the wire and find another way forward. You'll also help make our infrastructure "agent-friendly", so other teams can safely build on it with AI.

About our team

The Crawler engineering team makes sure Lighthouse customers get fresh, high-quality rates, ranking and supply data. We handle close to 200 million requests a day across 200+ data sources, so the team carries a big responsibility for keeping our crawling infrastructure reliable, efficient and ahead of a web that keeps changing.

What's in it for you?

  • Flexible working environment: Work from home or at one of our global offices.
  • Flexible time off: Autonomy to manage your work-life balance.
  • Alan Healthy benefits: 160€/month for food or nursery.
  • Payflow App: Access your earned salary in advance at no cost to you.
  • Flexible retribution: Optional benefits through tax-free payroll deductions for food, transportation and/or nursery.
  • Wellbeing support: Subsidized up to 80% ClassPass subscription.
  • Comprehensive health insurance: 100% Alan coverage for you, your spouse, and dependents.
  • Collaborative team: High-bar, friendly, creative, and passionate colleagues.
  • Career development: Workshops, frameworks, tools, training, and processes to realize your full potential.
  • Impactful work: Shape products relied on by 85,000+ users worldwide.
  • Competitive compensation: Proactively maintained to value your work.
  • Referral bonuses: Earn rewards for bringing in new talent.

Who you are

Rust or

C/C++

  • , or you have a strong track record in another backend language and a real drive to go deep in Rust.
  • You have a solid grasp of how the web works under the hood: HTTP, TCP/IP, TLS and proxies.

You have an

Technologies you will work with

Rust, TypeScript/Node.js, Python, Google Cloud, Kubernetes, Prometheus, Grafana, Claude Code.

Diversity, equity, inclusion, and belonging

Thank you for considering a career with Lighthouse. We are committed to fostering a diverse and inclusive workplace that values equal opportunity for all. We welcome candidates from all backgrounds, regardless of age, gender, race, religion, sexual orientation, and disability. Our commitment to equality is part of our culture. If you require reasonable accommodation at any point during the application or interview process, please notify your recruiter.

Not ticking every box? No problem! We value diverse backgrounds and unique skill sets, and we encourage individuals from all walks of life to apply. If your experience looks a little different from what we've described, but you're passionate about what we do and are a quick learner, we'd love to hear from you!

Learn more about our careers and recruitment process

Our company and leadership team

Beware of recruitment scams