Senior Software Engineer , Big Data Infrastructure
- San Francisco, CA, United States
- Apache Spark
- Ruby on Rails
- Google Cloud
LiveRamp is looking for a Senior Software Engineer to help us maintain and scale our Big Data infrastructure. Our data infrastructure team helps application teams manage several Hadoop clusters of different sizes, many with tens of thousands of cores and terabytes of RAM. We’re also responsible for some miscellaneous tools that we’ve picked up along the way and plan to replace with open source alternatives.
The LiveRamp engineering team has spent the last year planning and executing the massive migration of our entire environment to GCP which has been outlined on our engineering blog. We also had the opportunity to share our experience in more detail at Google Cloud NEXT which can be viewed here. We just crossed the finish line on our GCP migration, and we're ecstatic to have the opportunity to rethink how our applications work using a wide array of new tools available to us such as Dataproc and Bigtable.
Running this many data applications takes a heavy support stack. Zookeeper coordinates data deliveries. HashiCorp Consul exposes services. Vault stores secrets. We use open-source tools when they make sense, but we aren’t afraid to get our hands dirty when they don’t -- we work at a scale that only a few companies in the world match, and our most interesting tools come from solving those challenges. Some of these are open source (like our key-value datastores and log journaling tools), but other sophisticated internal workflow and YARN job coordinators aren’t publicly released ....yet.
The most valuable part of our infrastructure is the 60+ engineers writing hundreds of big data applications on our infrastructure, across a dozen application teams. These engineers are our customers, and our job is to work closely with them to support building efficient applications via both education and building the tools they need to get the job done.
If all of this sounds exciting, we are looking for you!
You will: Build big data applications and tooling to assist 60+ fellow developers writing hundreds of big data applications on our infrastructure. Ensure operational excellence and empower the organization to achieve a high level of technical productivity, reliability and simplicity. Act as technical point of contact for large re-architectures that touch all LiveRamp systems Foster a positive environment of integrity, empowerment, initiative and teamwork. Take ownership of projects - You don’t let bug reports fall through the cracks. Closely work with and iterate on feedback from your customers -- other developers. You err on the side of shipping fast and iterating. Your team will: Maintain and scale our Big Data infrastructure. Engineer scalable data processing engines of stunning complexity. Build tools to empower LiveRamp’s developers to operate effectively and efficiently. Deliver on the technical roadmap for LiveRamp’s core product offerings. About you: You have 4+ years developing large scale applications. You want to build tools to keep other engineers from reinventing the wheel. Turning your experience into reusable tooling excites you. You keep up-to-date with cutting edge big data frameworks; you enjoy digging into the tradeoffs of Spark vs Flink vs Mapreduce for a new project. You are both productive and pragmatic -- software is only useful if it is used. You have world-class debugging skills Get excited by when you make a Hadoop application more efficient - but you get even more excited when you make developers more efficient. Bonus points: You have built big data applications and have run into scale problems. You have open-source projects on GitHub. If you’ve contributed to someone else’s project… even better. Benefits: People. Work with talented, collaborative, and friendly people who love what they do. Food. Enjoy catered meals, boundless snacks, and the occasional food truck. Fun. We host events such as game nights, happy hours, camping trips, and sports leagues. Stock. Every employee is a stakeholder in our future. Health and Saving. Receive the benefits of comprehensive health, dental, vision and disability insurance along with a 401k matching plan. Location. Work in the heart of San Francisco and take advantage of our commuter benefits. More about us:
LiveRamp is the leader in data connectivity, helping the world’s largest brands use their data to improve customer interactions on any channel and device. We thrive on mind-bending technical challenges and value entrepreneurship, humility, and constant personal growth.
There is so much more that we want to build and that we could continue to improve. We value strong engineers who are agile enough to hit the ground running and tackle challenges.
To all recruitment agencies: LiveRamp does not accept agency resumes. Please do not forward resumes to our jobs alias, LiveRamp employees or any other company location. LiveRamp is not responsible for any fees related to unsolicited resumes.
LiveRamp is an affirmative action and equal opportunity employer (AA/EOE/W/M/Vet/Disabled) and does not discriminate in recruiting, hiring, training, promotion or other employment of associates or the awarding of subcontracts because of a person's race, color, sex, age, religion, national origin, protected veteran, disability, sexual orientation, gender identity, genetics or other protected status. Qualified applicants with arrest and conviction records will be considered for the position in accordance with the San Francisco Fair Chance Ordinance.
About LiveRamp Inc.
LiveRamp's product is a massive identity graph connecting each individual to their online identifiers like cookies and device IDs.
Our engineering workflows can be broken into three parts: Data Ingestion, Data Manipulation, and Data Distribution. We ingest behavioral data indexed by offline and online identifiers from our clients, resolve every index to a single LiveRamp ID through our Identity Graph, and then distribute ingested data to other technology platforms in the industry.
All LiveRampers are smart, nice, and get things done. We move quickly and value autonomy and project ownership over all else. We empower our people to use their best judgment when making decisions. We grow our talent by challenging them with stretch projects and supporting them with mentorship.
- Apache Spark
- Ruby on Rails
- Google Cloud
Skip straight to final-round interviews by applying through Triplebyte.