Running through a fog
The SMB protocol is derived from NetBIOS, which began as a joint project between Microsoft and IBM. NetBIOS made use of a non-routable protocol, NetBEUI, for small computer networks, and allowed early PCs to talk to one another. Subsequently, NetBIOS was implemented over TCP/IP, and this unhappy union became Microsoft's standard protocol, dignified by the portentous label, CIFS, (the Common Internet File System). For better or worse, SMB/CIFS has become the standard implementation within enterprises to allow communication on PC networks running Microsoft systems, intermingled with a bewildering mix of proprietary protocols and add-ons used by Microsoft.
Working on Samba through the years must have been like running through a fog, as the developers have worked to disentangle the published standards from the implemented realities. Nonetheless, anecdotal wisdom and industrial surveys have consistently suggested that Samba on Linux outperforms Microsoft's own implementations of SMB/CIFS on Windows, probably because Samba has a Unix-like modular structure, which allows for structured development and tidier code.
Jeremy Allison, who has been part of the Samba Team since before it became Samba says that "in the days when Novell Netware dominated the file serving world Microsoft was a great supporter of standards. They published the specifications of their own protocols (then called Server Message Block, or SMB) and supported implementations on other platforms... They even proposed the SMB protocol (now renamed the Common Internet File System, CIFS) as an Internet(IETF) standard," but once Netware was defeated by Windows NT "their attitudes changed, and the flow of information stopped. Proprietary modifications to core protocols like the Kerberos authentication protocol followed, and these changes were treated as trade secrets, patented if possible, and only released under restrictive non-disclosure agreements, if released at all."
The interoperability dance
Each new release of Windows has brought new extensions to SMB/CIFS which the Samba developers have been obliged to re-engineer. The Samba Team has interpreted these deviations as bugs, and Samba is seen as "bug-for-bug compatible" with Microsoft's implementation of the protocol. As is stated unequivocally in the Samba how-to "The goal behind the project is one of removing barriers to interoperability."
The original concept of Samba had been to build the code around the X/Open specification for SMB, the publicly available description of the protocol. "We had the X/Open spec. This was how it was supposed to work, so we'd code up to that, and we'd look at how NT behaved and test it with clients. If we weren’t quite accurate we would modify our code and put little exception conditions in, or whatever."
By the time of the release of Windows 2000 the team came to the inevitable conclusion that "the code was not long-term tenable," because there were too many special cases for the various extensions and bugs included in the Microsoft implementation. "The spec was a fiction, and made no sense. The complexity of SMB is one of the main reasons that NT was so unstable." As the X/Open specification had become an untenable model for Samba, large chunks of the code had to be rethought and rewritten to allow for the tidy inclusion of untidy changes to CIFS.
Disentangling CIFS sometimes had the making of a B-movie detective story. The password encryption for LanMan on Windows NT, for instance, included a "magic word", which Microsoft refused to disclose without an NDA (non-disclosure agreement). The Samba team's response was: "We're not giving you an NDA, because we're publishing source." As nobody would give way, the word had to be discovered by detection and analysis and through conversations with Microsoft personnel, which gave enough clues to allow the developers to search backwards through the code, and find the word for themselves. The magic word was KGS!@#$%.
The necessity for such contortions has been removed by the 2007 judgement of the European Commission, which requires Microsoft to publish its protocols in their entirety - without evasion, inconsistencies, broken standards and obfuscations - and to comply with the commonly understood meaning of open standards and protocols as they have been implemented by other participants in the computing industry.
Although, as Allison notes, "We now have specs. But we still have to work out what Windows does, so that's not changed. If we try to figure something out we go to Windows and write a test for it, and see how Windows does it. But it's a lot easier now, because we can go to the spec and see how it's supposed to work. At that point we have recourse to go to Microsoft and say, 'Hey, we think there's a bug in the spec here', and they will fix it, and they do."
"It's a different kind of work, and we're hoping it will change the kind of people who work on Samba, because before you had to be weird. You had to have a particular mindset that enjoyed doing that stuff, and most programmers don't. Most programmers don't like playing guessing games. They want to be able to write easy code that just works. Most people don't want to work out how something is supposed to work before they can write a piece of code..."