Senior Software Engineer - Platform

East Bay, CA, United States, San Francisco, CA, United States, Silicon Valley, CA, United States

Databricks


Role Locations

  • East Bay, CA, United States
  • San Francisco, CA, United States
  • Silicon Valley, CA, United States

Employees

501+ people

Address

160 Spear St Fl 13
San Francisco, CA, 94105-1546, US

Tech Stack

  • Scala
  • Python
  • JavaScript
  • Node.js
  • Modern Front-End Frameworks
  • Go
  • Apache Spark
  • AWS
  • Azure
  • Google Cloud

Role Description

Mission

 

If you are looking for an unparalleled opportunity to build the next generation big data processing platform, and learn how to launch hundreds of thousands of VMs a day at scale while running thousands of Kubernetes clusters, you have come to the right place. The platform team builds and manages the core systems powering Databricks, allowing it to seamlessly scale and run across various geographic regions/clouds, and making Databricks the go-to product for big data processing in the cloud.

 

This is where you come in. You will be a senior software engineer responsible for architecting scalable systems to power Databricks, making it the de-facto platform for running Big Data and AI workloads. You will build and extend the Databricks cloud platform, which is based on a micro service architecture and includes systems for managing thousands of Kubernetes clusters at scale, systems for streaming and consuming gigabytes of log data per minute, onboarding and managing thousands of data scientists on Databricks, scalable API gateway, rate limiting framework, network security and encryption, build infrastructure (we use Bazel), and scalable CI/CD framework among many others.

 

Outcomes

Develop and extend the Databricks platform. This implies, among others, writing clean, efficient code in Scala or Python and/or interacting with: cloud APIs (e.g., compute APIs, cloud formation, Terraform), with open source and third party APIs and software (e.g., Kubernetes) and with different Databricks services Be responsible for the full software development lifecycle - designing, development, testing, and operating scalable systems in production Architect solutions to achieve a high level of reliability, scalability and security Communicate effectively with other engineers in the same team, with other teams and with various other stakeholders including product managers and compliance teams Operate in an Agile and fast moving environment Competencies

5+ years of experience writing production code in one of: Java, Scala, JavaScript, C++ or Python Experience architecting, developing, deploying and operating large scale distributed systems at scale Experience with cloud APIs (e.g., a public cloud such as AWS, Azure, GCP or an advanced private cloud such as Google, Facebook) Experience working on a SaaS platform or with Service Oriented Architectures Good knowledge of Linux fundamentals Good knowledge of SQL (MySQL preferred) Experience with Continuous Integration/Continuous Deployment frameworks Experience with Docker and Kubernetes Benefits

Medical, dental, vision 401k Retirement Plan Unlimited Paid Time Off Catered lunch (everyday), snacks, and drinks Gym reimbursement Employee referral bonus program Awesome coworkers Maternity and paternity plans About Databricks

Databricks’ mission is to accelerate innovation for its customers by unifying Data Science, Engineering and Business. Founded by the original creators of Apache Spark™, Databricks provides a Unified Analytics Platform for data science teams to collaborate with data engineering and lines of business to build data products. Users achieve faster time-to-value with Databricks by creating analytic workflows that go from ETL and interactive exploration to production. The company also makes it easier for its users to focus on their data by providing a fully managed, scalable, and secure cloud infrastructure that reduces operational complexity and total cost of ownership. Databricks, venture-backed by Andreessen Horowitz, NEA and Battery Ventures, among others, has a global customer base that includes Viacom, Shell, and HP.  For more information, visit www.databricks.com.

 

Apache, Apache Spark and Spark are trademarks of the Apache Software Foundation.

About Databricks

Databricks Unified Analytics Platform, from the original creators of Apache Spark™, unifies data science and engineering across the Machine Learning lifecycle from data preparation, to experimentation and deployment of ML applications.

Interested in this role?
Skip straight to final-round interviews by applying through Triplebyte.