Solving hard problems with cutting edge technology is our business. We rely heavily on open source solutions in our large and distributed global infrastructure that includes several large high performance computing environments, a large low latency trading infrastructure and significant supporting infrastructure. We are looking to add to our team in Chicago or New York.
Example projects currently in progress include a many hundred-TB data warehouse project based on Clickhouse, a migration from a legacy Sybase environment onto MySQL 8, and scaling a small MySQL cluster that at peak times returns >100GiB/sec of data to thousands of clients. Database engineers "own" many parts of the stack, including embedding with trading and other teams.
The ideal candidate will be passionate about open source database technology, with experience in more than one (for example MySQL, Postgres, Clickhouse, InfluxDB, Prometheus). The candidate will have a good ability to automate (using a language such as Python or Go) and understand the full database system when deployed on Linux. Experience managing ETL pipelines and managing petabyte scale data warehouses is valuable, but not mandatory.
What you’ll do:
- Work on and lead major projects involving a mix of database technology.
Some example projects in flight are listed above. In general these projects span database engineering (e.g. deploying, scaling or managing database infrastructure) and working with other teams to solve business problems (for example Schema design, data architecture planning, technology selection)
- Be part of a global “on-call” schedule for database technology, including a legacy Sybase environment.
No previous Sybase experience is required and this environment is being aggressively replaced with open source technology.
- Develop automation to minimize operational toil, in accordance with SRE principles.
- Significant engineering experience working with open source database, solving challenging business problems with cutting edge technology.
- Comfort using a programming language for automation tasks (Python, Go, etc.)
- Comfort using the Linux shell to understand and troubleshoot an overall system
- Proficiency with SQL
Useful (but certainly not mandatory) skills:
- Experience working on petabyte scale data warehouses and distributed database systems
- Industry experience in finance/trading, particularly supporting post-trade processing systems
- Python, pandas, etc.
- Kafka experience
- ETL pipeline experience