Bi-directional connection for Membase and Cloudera Hadoop

At Hadoop World 2010 currently taking place in New York City, Membase Inc., formerly known as NorthScale, has announced that it has entered into a new partnership with Hadoop service provider Cloudera. Under the new partnership, the two companies have launched a new bi-directional connection for Membase and Cloudera's Distribution for Hadoop.

According to a post on the Membase blog by Membase co-founder James Phillips, the two companies have been "working very closely" over the last year in several customer environments where both Membase and Cloudera have been evaluated and deployed. Phillips says they have built "two mechanisms for integrating Membase and Cloudera Distribution for Hadoop (CDH)". The first consists of a Membase NodeCode module that streams data from Membase to CDH in real-time, while the second consists of a Sqoop-derived batch loader utility that allows for the loading of data to and from Membase and CDH.

The integration is already being used at AOL Advertising and ShareThis. Discussing the integration, AOL Chief Architect Pero Subasic says that "Traditional databases lack the scalability required to support our goal of five milliseconds per read / write. Creating user profiles with Hadoop, then serving them from Membase, reduces profile read and write access to under a millisecond, leaving the bulk of the processing time budget for improved targeting and customisation."

Subasic will join ShareThis archictect Manu Mukerji, along with Phillips at Hadoop World today to present "Better ad, offer and content targeting using Membase with Hadoop.” The session will take place in Sutton South at the New York Hilton at 1:45pm EST.

See also:

• Membase-Cloudera Integration Joins Leading Hadoop Distribution and Real-Time NoSQL Database, press release from Membase.
• Membase Server 1.6 released, a report from The H.
• Apache Hadoop goes Enterprise, a report from The H.

(crve)