26 August 2009, 10:36

Kernel Log - Coming in 2.6.31 â Part 4: Tracing, architecture, virtualisation

Kernel Log Penguin

by Thorsten Leemhuis

New performance counters allow developers to take a detailed look at the runtime behaviour of program code to target specific areas for optimisation. The recently introduced tracing infrastructure has been further modified and improved. Other changes affect the architecture, the memory subsystem, and various virtualisation solutions.

Last weekend, Linus Torvalds released the seventh release candidate of Linux 2.6.31. In his release email, Torvalds highlights some of the corrections; two days earlier, the list of known new problems contained 29Â unsolved problems.

Unlike the week before, Torvalds didn't make a statement about the final release of 2.6.31 â it is likely to take another week or two. The Kernel Log continues its discussion of the major new features of 2.6.31 by looking at the areas of tracing, architecture, memory management and virtualisation.

Detailed road test

Operating partly in the kernel and partly in the userspace, the "performance counters" code can be used for retrieving the performance data offered by modern processors. This performance data quantifies various CPU processes that have an impact on CPU performance â providing an in-depth analysis and allowing developers to optimise the code segments relevant to processing speed down to the last detail for each CPU.

Occasionally abbreviated to "perf_counter", performance counters should not be confused with Perfmon ("hardware-based performance monitoring interface for Linux"), which has offered similar functionality for years â the kernel hackers didn't include it in the Linux source code because they were dissatisfied with some of Perfmon's properties. This is probably one of the reasons why Ingo Molnar and several other developers pursued the idea of performance counters, which use a slightly different approach. The chief developer of Perfmon repeatedly criticised some aspects of the performance counter concept when the counters were first introduced in late 2008 and during the months that followed; when the developers began to integrate the required more than 600Â patches into the kernel sources, he added further criticism which was discussed in greatest detail. The article "Duelling performance monitors" published on LWN.net in late 2008 provides some details concerning this competitive situation.

Use of the performance counters is described in the more recent article "Perfcounters added to the mainline" on LWN.net. Further background information can also be found in the documentation. The relevant userspace tools were integrated directly into the kernel sources in a newly established tools/ subdirectory after a lengthy debate, during which Linus Torvalds expressed his opinion in no uncertain terms.

For other articles on 2.6.31 and links to the rest of the "Coming in 2.6.31" series, see The H's Kernel Log - 2.6.31 Tracking page.

A different kind of analysis

Ingo Molnar has introduced not only the performance counters, but also numerous other changes affecting the tracing area he maintains. In his git-pull request, he suggests that the main development phase of trace points is likely to be completed in the near future. He also highlights improved filters, internal restructuring measures and performance optimisation. Some of the possibilities offered by the tracing infrastructures of current kernels are explained in Sony developer Tim Bird's presentation "Measuring Function Duration with Ftrace", which we already mentioned in the article "The nitty gritty â The proceedings of the Linux Symposium 2009 ....".

The kernel hackers also integrated a profiling feature that uses the GCC Coverage Testing Tool (gcov). Kernel developers and testers can from now on use Kmemleak (documentation) to track down memory leaks â however, the tool misinterprets certain situations and constructions, and the results it produces shouldn't be trusted implicitly. Another new tool, Kmemcheck (documentation), detects the use of non-initialised memory areas.

In brief

The changes described above are only the most significant of those recently made by kernel hackers in terms of the Linux architecture and infrastructure. Here is a short overview of further changes:

Architecture code:

From 2.6.31, Linux will address 2^46 instead of 2^44Â bytes of RAM on x86-64 CPUs â however, this currently affects very few people, as systems offering 16 to 64Â Tbytes of RAM are still rather rare.

That hibernation (suspend-to-disk/software suspend/swsups) and power-saving techniques are important not just for notebooks, but also for mainframes, is demonstrated by several of Hans-Joachim Picht's patches to provide hibernation support for IBM's S390 architecture (now SystemÂ z) being incorporated into the main development branch. Additional changes introduced by Michael Holzheu further improve the use of several power-saving techniques for S390 systems.

Memory management:

A number of patches submitted by Mel Gorman are designed to optimise the page allocator responsible for memory allocation â a few benchmark results can be found in the commit comment.

Virtualisation:

The KVM developers have reworked the interrupt code and reportedly improved SMP performance.

The Xen code in the kernel now also provides some details of the /sys/hypervisor directory. A new addition to the kernel is the evtchn Xen driver for sending and receiving events on "event channels". The integration of Dom0 support, on the other hand, seems to have slipped away into the far future again â there has been no further discussion of the topic on LKML since the discussions just before and after the release of Linux 2.6.30.

After things went quiet around the Lguest "Simple x86 Hypervisor", this time around developers have made several further substantial changes; additions include PAE support and improved multi-threading.

Minor gems

Many additional minor, but by no means insignificant, changes can be found in the list below. Like many of the references in the text above, the links lead to the relevant commits in the web front end of the main Linux development branch, where the commit comments and the patches themselves provide extensive further information on the respective changes.

Architecture Support

...ARM: