Distributed transactions suffer from poor performance due to two major limiting factors. First, distributed transactions suffer from high latency because each of their accesses to remote data incurs a long network delay. Second, this high latency increases the likelihood of contention among distributed transactions, leading to high abort rates and low performance. The authors present Sundial, an in-memory distributed optimistic concurrency control protocol that addresses these two limitations.
An interview with Andy Pavlo, an assistant professor of databaseology in the Computer Science Department at CMU to discuss his current research and outlook on the future of database management.
The author presents CelFS, Two Sigma’s geo-distributed file system. Although CelFS has scaled to serve tens of petabytes of data, it uses physical partitioning to provide quality of service guarantees, it has a high replication overhead, and cannot take advantage of outsourced cold storage The talk further describes our response to these limitations in Jaks, a new storage system to reduce the TCO of CelFS and serve as the backend for other systems at Two Sigma.
This presentation discusses the design and implementation of Smooth at Two Sigma, our experience running it over the past two years, ongoing challenges, and future directions.
This presentation discusses each part of the durable storage stack, from the hardware on up, and how usage numbers can take on different meanings at each layer. It covers what's important to know at each layer, and how to think about and talk about concepts like compression, fragmentation, write amplification, and wear leveling. Finally, it examines different ways benchmarketers present data deceptively, and provides some techniques for identifying and cutting through those kinds of misrepresentations.