In association with heise online

12 April 2012, 12:37

10gen announces MongoDB Hadoop Connector

  • Twitter
  • Facebook
  • submit to slashdot
  • StumbleUpon
  • submit to reddit

MongoDB logo 10gen, the company behind MongoDB, has announced the general availability of a connector for its open source NoSQL database and Apache Hadoop, the MapReduce framework and distributed computing platform. According to its developers, version 1.0 of the connector is the "culmination of over a year of work to bring our users a solid integration layer between their MongoDB deployments and Hadoop clusters for data processing".

Using the connector, developers can transfer MongoDB data into Hadoop MapReduce jobs and receive the results of MapReduce jobs and transfer it back to MongoDB. Support for writing to MongoDB from Apache Pig and from the Flume distributed logging system is included in the initial release, as is support for using Python to MapReduce to and from MongoDB via Hadoop Streaming.

The MongoDB Hadoop Connector currently supports MongoDB 2.0 or later, but the developers note that "it should (mostly) work with 1.8.x". Future releases of the connector will add Pig input support and Ruby support for Streaming, as well as support for reading and writing MongoDB Backup Files for offline batch processing.

More details about the connector can be found in the announcement blog post. Version 1.0 of the MongoDB Hadoop Connector is available to download from GitHub; a full list of requirements, documentation and examples are provided.

See also:


Print Version | Send by email | Permalink:

  • July's Community Calendar

The H Open

The H Security

The H Developer

The H Internet Toolkit