samskivert: Operating System Support for Database Management – Stonebraker

12 October 2009

Various operating system services correspond to services needed by DBMSs and yet lack just enough functionality or performance that they end up being reimplemented by DBMS vendors. The paper outlines those features and the mismatches in question.

Limitations on having the operating system provide transaction exist in relation to crash recovery. Paged virtual memory systems also suffer some of the same problems as buffer pool management, and have additional page table memory overhead.

This paper dates from 1981 and is talking mainly about early versions of UNIX on PDP-11s and VAXen, thus its appeal is mainly historical. Many of the issues have simply disappeared thanks to Moore’s law applied to CPU speed and memory size. Having a 100K page table when mmap-ing a 100M file would seem no longer to be an issue. Similarly context switching overhead and IPC message sends have taken a back seat from a performance standpoint to efficiently utilizing multiple processors and optimally making use of multi-tier memory architectures.

One problem that has not gone away is dealing with disk subsystems. For a time DBMS vendors tried using raw disk I/O to achieve maximum performance, but advances in file system technology coupled with an explosion in disk drive technologies (RAID, drives with huge memory caches and complex controllers, SAN, NAS, SSD, etc.) have reduced the viability of that approach. It remains the purview of familiar-with-the-state-of-the-art DBAs to know how to best configure a database and a disk subsystem to achieve good performance.

Source: PDF ACM

©1999–2015 Michael Bayne