[GUFSC] MS paper touts Unix in Hotmail's Win2k switch

[Rafael R Obelheiro]

[http://www.theregister.co.uk/content/4/28226.html]

MS paper touts Unix in Hotmail's Win2k switch 
By Thomas C Greene in Washington
Posted: 21/11/2002 at 11:39 GMT

An older MS internal whitepaper from August 2000 on
switching Hotmail, which MS acquired in 1997, from front-end
servers running FreeBSD and back-end database servers
running Solaris to a whole farm running Win2K, reads like a
veritable sales brochure for UNIX, but concludes that the
company ought to set the right example by ensuring that each
division "should eat its own dogfood." 

The whitepaper, by MS Windows 2000 Server Product Group
member David Brooks, has been posted on the Web by Security
Office, which says it discovered the item and numerous other
confidential MS documents on a poorly protected server. There
are a number of other fascinating documents posted, in which
the careful reader will find a veritable treasure map for hacking
the citadel, but the one I enjoyed best was the comparison
between Win2K and UNIX. 

Among the observations is a very basic one about security: "A
fact about UNIX is that it is easy for an administrator to ensure
that there are no irrelevant services running. As well as giving
the potential for maximizing performance, it is useful to be sure
that there are no random TCP/IP or UDP ports open that could
be used as a basis for an attack," the paper notes. 

Next there's kernel stability: "Both the UNIX kernel, and the
design techniques it encourages, are renowned for stability. A
system of several thousand servers must run reliably and
without intervention to restart failed systems," the author notes,
and adds that, "Apache is also designed for stability and
correctness, rather than breadth of features or high
performance demands." 

Then of course there's the cost of ownership, which MS insists,
against overwhelming contradictory evidence, gives Windows
an advantage: "FreeBSD is free. Although there are collateral
costs (it's not particularly easy to set up) the freedom from
license costs is a major consideration, especially for a startup." 
And it's easy to minimize a UNIX system: "It is particularly easy
to cut down the load on the system so that only the minimum
number of services is running. This reduced complexity [and]
aids stability and transparency." 

Whereas: "A Windows server out of the box is an elaborate
system. Although it performs specific tasks well (such as being a
web server) there are many services that have a complex set of
dependencies, and it is never clear which ones are necessary and
which can be removed to improve the system's efficiency." 

Another good thing about UNIX is that everything is out in the
open, for admins, anyway: "It's easy to look at a UNIX system
and know what is running and why. Although its configuration
files may have arcane (and sometimes too-simple) syntax, they
are easy to find and change." 

Whereas in Win2K: "Some parameters that control the system's
operation are hidden and difficult to fully assess. The metabase
is an obvious example. The problem here is that is makes the
administrator nervous; in a single-function system he wants to
be able to understand all of the configuration-related choices
that the system is making on his behalf." 

Another strike against Windows is the GUI: "GUI operations
are essentially impossible to script. With large numbers of
servers, it is impractical to use the GUI to carry out installation
tasks or regular maintenance tasks." 

Then we have the ease of UNIX administration: "Most
configuration setups, log files, and so on, are plain text files with
reasonably short line lengths. Although this may be marginally
detrimental to performance (usually in circumstances where it
doesn't matter) it is a powerful approach because a small,
familiar set of tools, adapted to working with short text lines,
can be used by the administrators for most of their daily tasks.
In particular, favorite tools can be used to analyze all the
system's log files and error reports," the author explains, and
notes further that: 

"Over the years, UNIX versions have evolved a good set of
single-function commands and shell scripting languages that
work well for ad-hoc and automated administration. The shell
scripting languages fall just short of being a programming
language (they have less power than VBScript or JScript). This
may seem to be a disadvantage, but we must remember that
operators are not programmers; having to learn a
block-structured programming language is a resistance point."
Furthermore, "PERL ... is more of a programming than scripting
language. It is popular for repeated, automated tasks that can be
developed and optimized by senior administrative staff who do
have the higher level of programming expertise required." 

We find also that the Windows image size can be a real
inconvenience on a big farm: "The team was unable to reduce
the size of the image below 900MB; Windows contains many
complex relationships between pieces, and the team was not able
to determine with safety how much could be left out of the
image. Although disk space on each server was not an issue, the
time taken to image thousands of servers across the internal
network was significant. By comparison, the equivalent
FreeBSD image size is a few tens of MB." 

And finally, we're reminded that Windows often needs a re-boot
when a UNIX admin can simply edit a configuration file, stop
the process in question, and immediately run it again with the
new configuration. 

This is also a great advantage when things go wrong: "A service
may be hung, and rather than take the time to find and fix the
problem, it is often more convenient to reboot [a Windows
machine]. By contrast, UNIX administrators are conditioned to
quickly identify the failing service and simply restart it; they are
helped in this by the greater transparency of UNIX and the small
number of interdependencies." 

Another item worth mentioning, though not directly related to
a UNIX comparison, is the cost of load-balancing technology
and its supporting software. Using Windows load balancing
service requires Advanced Server, whereas using Cisco's Local
Director needs only Server. The costs, we discover, are
dramatically different: 

"Although Hotmail uses Microsoft software without license
fees, we must consider this project as a model for real
customers. Use of WLBS requires Advanced Server, but Server
provides all the other features used by Hotmail. Using list prices,
the cost comparison for a farm of 3500 servers is: Using WLBS
(hence Advanced Server): $15M+ / Using LD and Server: $6M+" 

Also very entertaining is the dramatic difference between the
internal whitepaper and its public version on MS TechNet in
terms of facts. 

For example, TechNet assures us that, "administrators generally
find benefit from porting 'cron' jobs to Windows Task
Scheduler events. Both Microsoft Interix 2.2 and SFU allow
administrators to port 'cron' files to Windows 2000 without any
changes in most cases, allowing administrators to gradually
transition scheduled events and scripts without impacting
operations i.e. at migration scheduled events can still run as
'cron' jobs. After the migration, the 'cron' jobs can be migrated
to Windows Task scheduler events. The Windows task
scheduler has better integration with event logs." 

But the whitepaper had found that, "using FreeBSD, such tasks
are scheduled by the cron service. Jobs are scheduled by being
listed in a file, one line per job. Changing the file is easy to
accomplish using the command line (or rdist), and replacing the
entire file is a good way to ensure that each server has exactly
the schedule of jobs that the administrator intended. Jobs can be
scheduled to execute once, or at intervals down to one minute. 

"Although the Windows Task Scheduler service is
fundamentally able to look after such jobs, the interfaces
provided in Windows does not measure up to the task. The usual
interface is the GUI, which is appropriate for setting up jobs on
a machine at a time, is labor-intensive and error-prone. 

"The command at is deprecated, is not able to schedule repeated
jobs at a frequency of less than one day. 

"The command jt was offered by the Task Scheduler team, but it
is unsupported and awkward to use (it was intended for testing). 
"None of the three interfaces offers an easy way to replace the
current task schedule entirely. The team met the need by
running the cron service provided in Services for UNIX. As
described earlier, relying on Services for UNIX (or any other
package subject to extra license costs) provides a bad model for
other customer deployments." 

So once again we see that TechNet is more a source of rhetoric
than information, just in case their painfully-cheerful security
bulletins had left anyone in doubt. 

It is terrifying to contemplate the efficiency bonus MS would
have enjoyed if it had only been willing to base its entire
corporate operations on UNIX instead of eating its own dog
food. The software monopolist might today be in the bizarre
position of being the world's only consumer of unices. ®