Ingres aims to exploit modern CPUs

Ingres has announced it is working with VectorWise to improve database performance through fully exploiting the capabilities of modern CPUs. They are planning to create Ingres/VectorWise, a new product which incorporates the new techniques, for release in mid-2010. VectorWise is a commercial spin off of the, Ingres funded, Amsterdam based CWI (Centrum Wiskunde & Informatica) database research team. Intel are supporting the project with engineering expertise and hardware.

Ketan Karia, Senior Vice President of Marketing, said "Early results show a 10x improvement in application performance" and that the benefits for business applications should be available "just by moving to Ingres". Detailed benchmarks will be presented at the Intel Developer Forum (IDF) in the first week of September. Karia points out that the design process and research behind VectorWise probably represent the first time that a database query engine has been designed with the physical processor in mind.

At the core of the VectorWise technology is a new query processing architecture, dubbed the "X100 Engine", which breaks down complex queries into vectors. These can be fed large numbers of values from the stored data, which can then be fed through the query processing pipeline and processed within the CPU's cache memory. The main memory, the traditional RAM, is used as an I/O buffer and store for intermediate data structures. The engine's design exploits the fact that a modern CPU has a large cache capable of holding the substantial amounts of data required to execute the query plan.

VectorWise have developed vectorised versions of selection, project, join and sort and even a binary search; these primitives are simple enough to compile using SIMD instructions for higher performance. This is combined with a new storage manager, called ColumnBM, which uses lightweight compression and clustering to provide high bandwidth access to on-disk data, resulting in high performance queries.

An early example given by Ingres of processing six million TPC-H style "LineItem" (with records buffered in main memory to remove I/O delays from the comparison) has Ingres 9.0 completing the query in 16.5 seconds and a hand written C++ version of the query working in 0.040 seconds. The VectorWise engine managed the task in 0.206 second; much faster than the traditional query architecture.

The technology holds the promise of performance improvements for intensive and complex queries. Ingres's plan, according to Karia, is to introduce the technology into the Ingres database, alongside the traditional query optimisation, as a new type of table which database administrators can use where appropriate to provide a performance boost. Karia says that eventually the technology will be open sourced, but in the development phase it will only be available to Ingres partners. Ingres has announced the project now to attract new partners who want early access to the VectorWise enhanced database offering.

(djwm)