Materialize Inc
Materialize gives you instant query results from your streaming data cluster. Compatible with widely deployed open-source stream platforms such as Apache Kafka, you can write live queries in an interactive SQL shell, or create materialized views to programmatically query using your favorite language’s database adaptors. Views stay incrementally updated within milliseconds.
Stream platforms promise lowered latency compared to batch ETL systems, but come at the cost of operational complexity, and manual orchestration. Building a compute pipeline in existing stream platforms requires manually building streams. Existing Streaming ‘SQL’ solutions have limited expressiveness, which hinders declarative programming, forcing the user to still think in terms of the component streams they are building.
Instead, Materialize allows users to declare materialized views in ANSI-standard SQL, and keeps them incrementally updated within milliseconds, no matter how complex the SQL query.
Materialize is built on top of a powerful stream processing engine: Timely Dataflow. Inspired by the award-winning Naiad research project, Timely Dataflow has been in open-source development for 4 years and has been used with large sets of data and under heavy load.
Why join us?
Our co-founder and Chief Scientist Frank McSherry has co-invented various breakthroughs in computing such as Naiad, Differential Privacy, and the technology at the core of Materialize: Timely Dataflow.
We just raised an $8.5 million Series A led by Ravi Mhatre of Lightspeed Venture Partners.
Our head of engineering was the leader of much of Cockroach Labs' engineering team, the founding head of engineering for Dropbox's NYC office, and the third engineer at YouTube.
All our code is written in Rust.
Engineering at Materialize Inc
We have a head of engineering leading a small but growing team. We're light on process, moving quickly but with deliberation and theoretical rigor. Engineering process will be added as the need arises, in a pragmatic manner.
We have many challenges relating to distributed systems, algorithmic optimization, and compilers.
We are writing a SQL layer on top of Timely Dataflow and Differential Dataflow, two fast, well-tested systems originally created by our co-founder Frank McSherry. To do this, we need a SQL parser, SQL type system, and the ability to perform various SQL-related operations efficiently. In addition, we'll need to create a means to easily to deploy our software on a cluster, as well as run multiple Timely clusters in active-active settings. Later, there will be substantial challenges in other areas such as query optimization, message durability, and checkpointing and fast replay.
We have begun writing a SQL parser and hooking it up to an execution engine to allow users to create SQL materialized views that are backed by data flows that incrementally update with very low latency.
The SQL engine will need a type system.
We'll need to an easy way to create and monitor a high-availability deployment of Materialize. This will likely involve container orchestration or some other form of automated deployment.
We need to make durable the indexed in-memory log-structured merge tree used in Differential Dataflow. Differential builds and maintains LSMs, but these are not durably written to disk. Writing this to disk will greatly speed up cluster restart times, rather than reading and backfilling them from the input queues.
Working at Materialize Inc
We value engineers who can iterate efficiently, desire impact, demonstrate resourcefulness, and strike a balance between autonomy and collaboration. Our culture is still embryonic, and you can help shape it if you join as an early engineer.
-
Free Food
We have lunch together as a team, picking different restaurants in SoHo to visit.
-
Company Retreats
We plan to do company retreats twice a year, starting this June. You will be part of the very first one!
-
Health Insurance
We provide full medical, dental, and vision coverage.
-
Workshops/Conferences
We encourage everyone to spend approximately one week a year on personal development, including going to or presenting at conferences and workshops, fully paid for. Our academic backgrounds mean that we've greatly benefited from this over our careers, and we'd like to continue that.
-
Flexible Hours
Some of us keep 10-6, some of us keep 9-5. Do what works for your productivity, including working from home as needed.
-
Beautiful Office
While beautiful might be a stretch for our WeWork offices, we don't do open offices. We have multiple small rooms that seat 2-3 employees each, and we think that's optimal for productivity.
-
Generous Vacation
We have unlimited vacation. We roughly aim for:
20days/4 weeks vacation 5 days of personal days (things happen) 5 days of personal development (go to conferences/workshops)
-
Gym/Fitness
Discounted memberships at NYHRC
Interested in this company?
Skip straight to final-round interviews by applying through Triplebyte.