open source database ten times faster than closed source rivals

This story is very interesting given it's significance from both an open and closed source perspective given the high performance and disruptive potential involved.

For full disclosure up front, I am an employee of Ingres. My blog is my own opinion however and thus I am not speaking for my employer here. I believe I've done a pretty good job at keep my blog even handed towards all open source technology including databases.

An article at the Register today reminded me I am overdue to blog about this.

Relational database software is interesting in that it revolutionized the industry when it was introduced. It was one of those disruptive technology changes that for the most part became the standard way of storing, managing, and querying data. Since then features have been added to various database (or more correctly perhaps - relational database management system) software. These features added robustness, more data type, more functions, more drivers for various languages, more clustering and replication options, and so forth. Under the covers, the technology remained the same however. It was still rows of data being written to some sort of storage - first tape, and then hard disk. The market leaders in this industry have been very successful making lots of money and thus occupied with keeping their customers happy. If you read the Innovator's dilemma, it will not surprise you to learn the technology has not changed that much in the past couple  of decades.

Enter the team at Centrum Wiskunde and Informatica (CWI). Their research went back to first principles to examine what the best case scenario should be. How fast should a database be if everything worked perfectly and overhead was minimized? They discovered a massive difference. In some cases, a relational database was ten to one hundred times slower than a C program coded specifically to load data from disk and query it in the optimal scenario. As well, they examined modern hardware capabilities and discovered they were not fully being exploited by database technology. What followed was some clever R&D to implement technology that is a significant advancement in relational database technology - leading to Ingres Vectorwise.

I've seen live demos of Ingres Vectorwise in action at various public events. It is extremely impressive. The performance of Ingres Vectorwise on an off the shelf laptop rivals and even bests performance of high end servers. Query results that would take minutes on a typical server running database technology from the closed source market leaders complete in less than a few seconds... often sub-second. This is stunning. It makes my imagination race to think of what types of applications are now feasibly or practical now that were not so just a few years ago. I am particularly interested in the application of this technology to my work with geospatial data given the rapid increase in gps/spatially aware devices and applications - very cool!