- New York
Note: This role is based in NYC.
As a Data Engineer, you will:
- Design & build a high-throughput SaaS platform with particular emphasis on data engineering components
- Be responsible for production delivery of your components , including integration & coordination with your teammates responsible for the user-facing web application, CICD pipelines, and SRE infrastructure
- Be forward-thinking in designing for future petabyte-scale, ensuring a performant & resilient architecture
- Exhibit continuous curiosity in understanding emerging technology that could solve our challenges
- Mentor others on best data engineering practices & guide decision-making
- 3+ years software engineering experience on a SaaS platform with emphasis on large-scale data systems
- Experience building large-scale data systems using distributed file storage technologies such as hdfs and s3 and distributed processing frameworks such as Spark, EMR, and HDFS.
- Experience with event processing and streaming data technologies including message queues such as Kafka and stream processors such as Spark streaming, Storm, Kinesis, etc.
- Experience with multiple RDBMS & NoSql technologies
- Proficiency with Python (preferred) or other commonly used data processing languages such as Java or Scala
- Understanding of multi-tenant platforms, best practices for managing multiple organizations on a shared platform, providing secure & controlled access to data, and role-based access control (RBAC)
- Experience working with cloud environments such as AWS and GCP
- CS (preferred) or other technical degree, or equivalent practical experience
- 1+ year experience as a technical lead in data engineering
- Experience with machine learning & AI and related tools such as Airflow, Tensorflow, and Sci-kit learn
- Experience with analytics or data visualization architectures
- Experience with on-prem deployment architectures
- Experience running a 24x7 SaaS platform with an SLA
About Arthur AI
Arthur AI is the first production AI monitoring platform, giving enterprises the tools to detect model issues proactively, in real-time to maximize their effectiveness. The Arthur AI platform brings auditability and transparency to black box models, and can be configured to monitor for unwanted bias.
Arthur’s Trusted AI platform offers a single pane of glass to all of your production models, outlining where models may be inaccurate due to any one of many statistical metrics
We are a scrappy, highly motivated team with varied & diverse backgrounds, who value honest feedback, transparency, and collaboration in order to make our product & our processes better. We are early enough in our journey that our next set of hires can influence the culture!
Skip straight to final-round interviews by applying through Triplebyte.