- New York, NY, United States
Note: This role is based in NYC, DC, or Remote
At Arthur, we are building the first platform for Responsible AI. We’re looking for an experienced Data Engineer response for the design & implementation of pipelines that will crunch petabyte-scale data in order to power our ML monitoring platform. The ideal candidate is a hands-on-keyboard engineer who has built streaming & distributed data architectures, is always on the lookout for the next evolution in data processing technology, and is excited to mentor others to ensure the entire team is able to build & maintain data pipelines.
As a Senior Data Engineer, you will:
- Design & build a high-throughput capabilities with particular emphasis on data engineering components, in close coordination with frontend & API engineers
- Work closely with the SRE team to ensure that data pipelines are resilient, performant, well-monitored, and scalable for both our SaaS and on-prem product offerings
- Exhibit continuous curiosity in understanding emerging technology that could improve our platform
- Mentor teammates on best data engineering practices
- 4+ years software engineering experience on a SaaS platform with emphasis on large-scale data systems
- Experience building large-scale data systems using distributed file storage technologies such as hdfs and s3 and distributed processing frameworks such as Spark, EMR, and HDFS.
- Experience with event processing and streaming data technologies including message queues such as Kafka and stream processors such as Spark streaming, Storm, Kinesis, etc.
- Experience with multiple RDBMS & NoSql technologies
- Proficiency with Python (preferred) or other commonly used data processing languages such as Java or Scala
- Understanding of multi-tenant platforms, providing secure & controlled access to data
- Experience working with cloud environments such as AWS and GCP
- CS (preferred) or other technical degree, or equivalent practical experience
- 1+ year experience as a technical lead or principal in data engineering
- Experience with machine learning & AI and related tools such as Tensorflow, and Sci-kit learn
- Experience with analytics or data visualization architectures
- Experience with on-prem deployment architectures
- Experience running a 24x7 SaaS platform with an SLA
- Working with a small, fast-growing team, lots of opportunity to take ownership and run with projects
- The opportunity to get in on the ground floor of a rapidly growing startup, working with a cutting-edge technology stack
- Generous equity
- A culture that empowers great people to accomplish great things
- Full benefits package
- Flexibility to work out of our NYC, DC, or remote
Arthur AI is the first production AI monitoring platform, giving enterprises the tools to detect model issues proactively, in real-time to maximize their effectiveness. The Arthur AI platform brings auditability and transparency to black box models, and can be configured to monitor for unwanted bias.
Arthur’s Trusted AI platform offers a single pane of glass to all of your production models, outlining where models may be inaccurate due to any one of many statistical metrics
We are a scrappy, highly motivated team with varied & diverse backgrounds, who value honest feedback, transparency, and collaboration in order to make our product & our processes better. We are early enough in our journey that our next set of hires can influence the culture!
Skip straight to final-round interviews by applying through Triplebyte.