Software Engineer, Backend (Massive Graph Processing)

San Francisco

LiveRamp


Role Location

  • San Francisco

Employees

501+ people

Address

225 Bush St Ste 1700
San Francisco, CA, 94104, US

Tech Stack

  • Java
  • MySQL
  • Go
  • Kubernetes
  • Hadoop
  • Apache Spark
  • Kafka
  • React
  • Ruby on Rails
  • Google Cloud
  • Apache Spark

Role Description

We’re LiveRamp Identity Engineering, and we maintain a massive graph that connects together the different identifiers for consumers (e.g., anonymized email addresses and phone numbers) and the devices they use online. Help us process a trillion edge graph as quickly and efficiently as possible.

The engineering systems we’ve developed are constantly ingesting new edges from thousands of different sources and finding numerous types of relevant paths to power LiveRamp’s suite of core products.

You will get to work on projects such as:

-Pregel Path Computer: This system finds relevant graph paths using the pregel graph computation framework as implemented in Apache Giraph. There are challenges in running Giraph at the scale of our graph and we’re constantly looking to refine our Pregel algorithms.

-Edge Ingestion and Partitioning Framework: We could never process all trillion edges at once and luckily we don’t have to. Instead we process subgraphs that contain specific types of edges. Our edge ingestion and partitioning framework manages different Hadoop datastores for different types of edges and automates the ingestion of new edge data. It leverages LiveRamp’s Seek MSJ framework to efficiently incorporate new data into existing edge stores.

-Path Computation as a Service: We provide a service to other LiveRamp engineering teams for finding specific types of paths within our massive graph. It handles 20,000 requests a day and this is possible due to its use of caching and intelligently batching similar request together.

About LiveRamp

LiveRamp's product is a massive identity graph connecting each individual to their online identifiers like cookies and device IDs.

Our engineering workflows can be broken into three parts: Data Ingestion, Data Manipulation, and Data Distribution. We ingest behavioral data indexed by offline and online identifiers from our clients, resolve every index to a single LiveRamp ID through our Identity Graph, and then distribute ingested data to other technology platforms in the industry.

Company Culture

All LiveRampers are smart, nice, and get things done. We move quickly and value autonomy and project ownership over all else. We empower our people to use their best judgment when making decisions. We grow our talent by challenging them with stretch projects and supporting them with mentorship.

Interested in this role?
Skip straight to final-round interviews by applying through Triplebyte.