The Two Elephant Problem
When an open source project wants to make major changes in its core code, such as revamping a user interface library, it faces the two elephant problem. The first elephant in the room is the existing community and its familiarity with what had already been implemented. The second elephant, which the developers want to bring into the room, is all the changes that they want to incorporate into the project. The difficulty is moving the second elephant into the room without disturbing the first or just filling the room completely with elephant.
There are a number of strategies to approaching the two elephant problem. The "push the new elephant in, in one go" approach is the most radical and disruptive. A new version of the project is presented to the community, complete and ready to roll, and it is up to the community to accept the changes. The danger for the project though is that the first elephant may decide to leave. This was the approach taken by the KDE developers when they committed to creating KDE 4.0. The backlash saw many users sit back on older versions of KDE or look for alternative desktops. It is only now, after a number of subsequent releases, that KDE is winning back friends.
Another approach is to mitigate the disruption by disassembling the new elephant and introduce it bit by bit into the room, with the hope that the community will adopt the new elephant parts as their own. The problem with this approach is that it is very hard to disassemble elephants, let alone reassemble them, and any vision for the new version may not survive the rebuilding process. A Frankenstein elephant may work, but the elegance of the original plans could well be lost and new parts may well be rejected. This is partly the approach of the GNOME developers, who have delivered a preview version of the GNOME Shell, a highlight of the proposed GNOME 3.0, in version 2.28. According to some reports, this is a hard to get working preview and as such the effort to get it running may outweigh the benefits of the early preview.
Get Another Room
The way forward may well be to not even think about trying to get the new elephant in the room, but to make a new room and put it the elephant in there. It is an option few projects seem to wholeheartedly adopt, but forking a project into a new development effort does have some major benefits. When we say fork, in this case, we are talking about Project Popular spinning out a Project Next Generation and moving the radical redesign plans into "Next Generation". Ideally, the name should not have too much in common with the lead project. The objective of the process is to reduce confusion. With an established community, this is a definite signal that the next update the user runs will not change the software radically. It also allows developers to be more radical with their changes in "Next Generation". They also have the advantage of being able to create, and abandon, multiple forks which can explore different approaches to development, from an open collaboration to a focussed Skunk Works style operation.
This is not the fracturing fork created by community disputes but more a willing fork with which the community and developers look forward to the point when a well engineered development fork has sufficient traction that the developers can invest engineering time into folding the development fork back into the mainstream. If the development fork is good enough, the community will demand it. The obvious caveat with this approach is that the project must actively be measuring community demands and needs to make this work.
In the past, one of the biggest worries with doing forks like this has been the concern that it can potentially leave the core project with fewer developers and fragment the development effort. This is a risk, but less so now that open source developers are already making use of distributed source code control systems which can assist greatly in keeping common code bases in sync. Bug fixes from the development versions can then, if required, be back-ported to the core project, and vice versa.
Managing Two Rooms
Open source developers have given themselves the tools to allow for this kind of development model. Systems like github and Launchpad unite those tools with tools for non-code collaboration such as bug and issue tracking, translations and documentation, in a way where it is possible for a community to create new forks and spaces for major revisions. What's missing is usually the will within project leadership to create the space and manage major research forks. More often than not, an announcement of a fork is usually preceded by some debate over the future of the project and in turn leads to worries about fragmentation, demands that the developers focus on the core or just a disgruntlement that enhancements are being kept away from the community, even if they will be disruptive. With the trend towards regular release cycles, the window for medium to long term research has become vanishingly small or is carried out quietly by a small set of developers with an itch to scratch.
By adopting the "room per elephant" model, that medium to long term research can be carried out in full view of the community but without the regular release clock ticking away. The "when it's done" mantra, which has guided many projects to release final code only when it's felt to be complete, can be applied to the research forks, allowing innovation to thrive in the thinking and developing space provided while a project keeps to a regular release schedule for the community core. A research fork can have it's own release schedule, to maintain its own development momentum. The "room per elephant" model has already been successful for one vendor: Red Hat. Fedora is a room for the developers to create new features, to experiment and to tune and hone existing code. Red Hat's first elephant, their customers, can see the development happening, give early feedback and still know that those developments will be carefully managed and tested when they are incorporated into Red Hat's enterprise software.
By creating space to research and develop, in the open, without a project's existing wider community feeling that they are being pushed into a fait accompli by the project leadership means that the wider community can participate more in one or more research projects. It's a model that can empower the community, as a social market, to select future features with their feet, or at least their selected checkboxes on their package installer.
Have you experienced the Two Elephant problem? Do you think the Two Rooms solution is appropriate? Is there a better solution? Have your say in our forums.