ChangeLog interviews Jeremy Allison

The following interview was performed by Maya Tamiya of ChangeLog [ http://www.changelog.net/ ] during the Japan Linux Conference '99 in December, 1999.

We also would like to thank Takaaki Higuchi who helped us to realize this interview and JLA members for managing the conference. 

CL: Our first question is: Have you become rich?

Jeremy: (laugh) On paper, yes.

CL: But in six months?

Jeremy: Well, but it doesn't quite work like that. I've been in a California company that went public before... and it's like watching a roulette wheel go round. When the ball finishes, you'll never know, it might finish on zero, perhaps wins. So, yes, theoretically, on paper I'm very rich. Practically, I'm ignoring it, because it isn't real. It isn't really real.

CL: You have VA's shares, anyway?

Jeremy: Yeah.

CL: Who will be employed by VA next?

Jeremy: I don't know. I have several people I'm trying to hire, but I cannot say anything until they signed on... but yes, we're hiring! We are sucking up Linux talent as fast as possible! :-)

CL: What made you move from SGI to VA?

 Jeremy: It's actually personal reasons. I needed to buy a house in Silicon Valley... It's just that I wasn't going to be able to buy a house if I stayed there... But I like SGI. I think they do good products. I think they're doing tremendous amount of Linux work. Not as much as VA is doing, but for an old-style Unix company, they are actually doing a lot of work. They don't actually get as much credit as they need to.

I really like SGI, I like the people, I liked working there. I really like VA as well. I really enjoy it. VA is almost like the Silicon Valley Linux Users Group turned into a company :-) Almost everybody who was in the Silicon Valley Linux Users Group is at VA. So I was both with SGI and VA.

CL: Are you a full-time "developer" at VA?

Jeremy: Yes. I'm officially in professional services and I develop Samba. VA actually has quite large professional services. What you've got to realize is that, and this is wonderful to see, many companies who now want to have Linux so they are able to say "yeah we do Linux", have a lot of Windows NT code, and need to have that ported to Linux. And that's one of the things that VA does particularly well, basically to advise people on how to port what we love to call "legacy Windows code" to a modern platform :-) There are so many people trying to do that right now. It's wonderful because I remember, about five or six years ago, everyone was porting the other way. In fact, I used to make my living porting Unix code to Windows, and that's why I got involved in Samba, because it was making my job as a developer porting Unix code to Windows easier. So it's wonderful to see it going like that, the other way.

CL: In practice, about how many hours do you write codes a day?

Jeremy: Umm... it depends. I've been on the road a lot recently... I did a bunch of SGI Linux University commitments.

Some days, no code at all. Some days, it's all email. I get around 300 emails in just a day. I'm drowning in email, I really do. I just have to relate. I just cannot do anything about it.

On good days, I just don't look at email whatsoever and I just code all day. That is a day I think "Oh, what a great day! I had a wonderful day today!" :-)

Most days, it's an average of probably half time coding, half time just managing stuff. Trying to manage the release of Samba, merging other people's patches and communicating with those team members, it just takes an enormous amount of time, like Dirk says. Dirk is becoming a manager to manage XFree86, and that is beginning to happen to me, and I hate it. It drives me nuts, because I love writing a code. Writing a code is why I do this, because that's the fun part.

I used to have a job where I could go and write a code eight hours a day everyday. I can't do that anymore. I still try to have at least one day in a week to do that, but it's very hard, it really is. There are just so many pressure on time, and especially now that the press and the corporations have discovered open source, there are so many people who say "Will you just come and give us a generic open source's good talk", and to be honest, I can probably fill all of my days up every week, just giving talks. And I won't get any work done whatsoever. That would be very depressing.

 CL: If you really gets to hate it and if you escape, what happens to Samba?

Jeremy: Well, Andrew is still working on it, there's Gerald, there's Luke, there's Volker.... Open source projects don't live or die by one individual, or even by a group of people. There's no irreplaceable person, even Linus, actually! I remember a Linus's talk at one of the Linux expos, where he gave the frequently answered questions, and one of those was, "What would happen to Linux if you get ridden on by a bus?" and his answer was "I live in Silicon Valley, and there aren't any buses in Silicon Valley, because everybody drives. So it's very unlikely for me to get ridden on by a bus, but even if I got ridden by a bus, I'd be dead then, and I wouldn't care, would I?" :-) and that's pretty true. Big projects have a life of their own. They're bigger than anyone, they are bigger than people who started them. So if Samba is important, then it will continue.

Most people say "What do you want to do with Samba?" and Andrew and I have exactly the same answer. Our answer is "We want Samba to go away", because we want Linux to be so successful that people run Linux on all of their desktops. And if people are all running Linux on their desktops, then they won't need Samba to serve files, because then there won't be any Windows desktops anymore! And I can go back to be a more productive and useful citizen to do stuff that I enjoy doing. Well, I enjoy Samba, but I would be able to go back to do other interesting stuff. So... the goal for Samba is for Samba to go away :-)

CL: Which operating system do you do your work on?

Jeremy: My desktop is on Linux. VA is entirely almost all Linux. In fact, this has been a joke that I'm the only person at VA who's allowed to use Windows :-) because I have to use Windows. I have to test Samba!

But because Samba is very cross-platform, I actually have access to... Sun has donated a Solaris box, HP has donated two HP-UX machines, IBM has donated various versions of AIX, SGI has donated not only the server but the develop machines, and they pay for the bandwidth. Whistle gave me a FreeBSD machine, and I think DEC is in the process of donating a Digital Unix box. So although I develop on Linux, I actually have access to many many different kinds of Unixes, because it's in all of these vendors' interests to make sure that Samba works well on their platforms.

One of the amazing things was... I was at an HP conference to give a Samba talk, and I was sitting down with HP-UX development managing people. They were saying "How can we make your lives easier?" There was basically another Linux guy other than myself and he essentially said "Look, why don't you try and make HP-UX follow the Linux Standard Base? Because most of the software that you want to run on your HP-UX system is now being developed on Linux. So if you want that stuff to be ported to HP-UX, and if you want HP-UX to have a chance of survival in the future, you'd better make it look as much like Linux as possible, so that people would have an easier job porting their software to it", and they leaned to it! Two or three years ago, every Unix vendors would say "Ours is the one true Unix, ours is the one that's going to win." and we actually have them say "oh yeah, maybe if we made our Unix look more like Linux, we do better." It is just amazing. They actually have committed to looking seriously at making HP-UX look a lot more like Linux. That was wonderful.

But actually some of the areas of Samba are best developed on various platforms. So for instance, the 64-bit testing of Samba is done almost exclusively done on IRIX boxes, although they are done on Solaris boxes and AIX boxes as well, because IRIX has a really clean 64-bits implementation.

CL: Did you become to develop on Linux after you came to VA?

 Jeremy: Actually yeah, I was developing Samba mainly on IRIX. Because Unixes are so similar, it really doesn't matter what you have in front of you. Writing potable code, is the important thing. We've had five years of experience in Samba in writing portable code. So I could have FreeBSD, Linux, Solaris, IRIX or whatever, on my desktop and it really wouldn't matter much, because we have the POSIX standards to follow, and we follow those. We test everything to work on every platform, so we just need portable code.

A lot of the vendors who are having problems porting their code are being punished for lazy coding in the past, basically. Once you bite the bullet and say "No, we're going to write a portable code" then essentially, it's really easy to work on almost any Unix platform out there.

CL: Have you ever heard of VA coming to Japan?

Jeremy: I cannot make any comment on that, sorry.

 CL: Where do you position yourself among these three: Free software movement (like RMS), Open source movement (like ESR), and Free beer (like Linus? :-)?

Jeremy: I personally agree with Richard's ideals, but I like Eric's methods :-)

CL: So you pick up the best part of all?

Jeremy: Yeah, and I'd always go with free beer :-) But I think what Richard is trying to do is wonderful and I really admire it. I really would like to get to the world where Richard is happy. But I don't think that doing it the way that Richard want us to do it would get us there. I think doing it in the way Eric wants to do it will get us there.

So when I'm talking to corporations, I would sit there, and I would say "open source", "reliability", "better return on investment", all those things that people wearing suits want to hear. But what I'm actually trying to do is what Richard is trying to do, which is "free the software". So... I think I'm probably closer to Eric Raymond than Richard Stallman, but I still want to see free software. I started working on free software sort of like eight or nine years ago... no... it must be older than that... maybe ten or eleven years ago now... I was honored to have my first patch rejected by Richard Stallman :-)

I've been in the free software movement for a long time, but I've sensed well enough to keep my mouth shut and say "open source" when it suits, when we need to do better. I think a lot of corporations are now discovering the positive sides of open source and what I'm really pleased about is that it seems that many of the corporations are not only saying "oh, open source, give them all the source" or whatever, they're actually embracing the ideas of the free software movement as well, which I never thought would happen, I'm really pleased to say. Look at Netscape, look at what SGI is doing, look at the number of people, Intel, for heaven's sake, I think they released drivers for their EEpro cards under GPL! I mean, Intel! I just cannot imagine that! If someone would have said five years ago "Can you imagine Intel releasing GPL'ed software?" I would have gone "You're insane!" :-) I think the critical issue is that people actually now understand what they are trying to do with the GPL. It's just amazingly wonderful.

 CL: Would you give us your comment on the recent problem between Sun and the Blackdown team?

Jeremy: I think that was just a mistake from Sun's marketing folks. It wasn't deliberate, at least. That kind of thing happens all the time.

But in general term, Sun's marketing organization doesn't really understand them. That's something they're just going to have some experience with.

The Blackdown team did all the work, so I can understand why they were quite annoyed, I would be annoyed as well. But then again, essentially, if you work under the Sun Community Source License.... you know, you signed onto it, they can do that to you... I don't like the Sun Community Source License, and so I think what happened to the Blackdown team was a good lesson for people who might think of working on stuff under a discourteous license. I don't think it's a very friendly license for open source.

CL: What do you think of the future of open source software and proprietary software? Do you think proprietary software will survive in the future, too?

Jeremy: Yes, I think it will, but in a much more diminished extent.

I remember talking with Applix. Their engineering manager was saying to me, "We've tried to think of how we could open-source Applixware, and we can't see a way of doing it." I would very much like them to do it because there's an occasional bug that I would quite like to fix but I can't because I don't have the source code... I'm not trying to knock it. I use Applixware for all of my presentations, it is my office suite.

But to be honest, I thought about it and I essentially said "You're right. There is absolutely no reason for you to open-source your software right now", because this was before Sun was to open-source StarOffice. "Right now, you don't really have any open source competition, and you're making money on it, you're doing quite well, there's no reason for you to open source it." People tend to open-source stuff when there's pressure on them to do so. So in the future, when KDE office suite is very popular, when Gnome office suite is very popular, when Sun does open-source StarOffice, then, Applixware will have to open-source or they will all die. But until then, there's no real benefit for them in doing it.

I think there's always be some pieces of software that's so weird and obscure that three people in the world use it and those people are willing to pay ten million dollars each :-) That kind of stuff, I think will survive. But I think generic proprietary software is going to find themselves under more and more pressure, because once a piece of open source software reaches a certain level of complexity and performance, then it tends to eat proprietary software alive. We are seeing an evidence of this happening in the Samba sphere. I don't want to talk too much about it, but essentially, it's too expensive for them to compete with us, when we get all the development done by series of different companies for free, and they are having staff, marketing, and everything to pay by themselves.

 So I think you'll see once open source software colonizes a niche, it will completely dominate it. And as time goes on, the ecological niches where open source will live in will get more. They'll find more and more areas where you wouldn't even think of using anything but open source software.

I think that's the view of the future. If I was a proprietary software vendor, I would be very scared right now. For instance, I've been thinking they have to do the same for Solaris. SGI knows that they have to give away their IRIX technologies and put them into Linux or die. You'll see that pressure happening in the operating systems level first, because that's the most obvious thing, but it will happen to the applications, too. Look at OpenGL. It is about to be colonized by open source software. And so at that point, main 3D graphics rendering will become open source, and then you'll see it expanding from there.

One of the interesting possibilities is, let's say, games vendors. They have very good proprietary games platforms, but what they are really selling, is an experience, a story, a game. Once open source 3D graphics reaches a certain level, what people would be willing to pay for is almost like a movie. If you want to play Quake or whatever, you're not really paying for the graphics software, you're paying for the story, effort and artists' work or whatever. So I see that sort of stuff surviving. You'll have open game engine platforms, and people make their money by selling you a story and characters and that kind of things. So it's becoming more like the movie business, basically.

CL: What do you think of O'Reilly's Using Samba?

Jeremy: It's a wonderful book! A lot of people are working on it. The best thing about it is they have given us basically the source code for the book. I think that's an absolutely wonderful move on their part. I am so happy they did that, because essentially what it gives us is a very high quality documentation that we can keep current as we extend the software.

When I was working at SGI, one of the things that I realized was that if we, SGI were going to do a commercial Samba product, we have to fix the problem that software would advance the documentation. So I spent three months rewriting the documentation, and that was a really miserable thing to do because I really wanted to be writing code and doing other stuff...

We also think using the O'Reilly book to keep up to date with the software is going to make immense difference, because we will basically be able to ship a complete publish quality book along with the software, and you can get it all for free.

Most people would still go out and buy the O'Reilly book, because it's just so convenient, you can read them in the bath... I mean, I do. I buy a lot of books all the time. It's much nicer than having stuff online.

But the fact remains that, let's say you want to teach a Samba course in Bangladesh. To be able to actually give your students copies of the book is just essential if you are going to be able to successfully teach, to be able to expand their knowledge. Now they would have a publish quality book in order to use for their course work, so I'm very happy with it.

One of my idea for open source is that the third world countries can use it, so that the third world doesn't have to depend on large US corporations. They can use it legally and they can have localized software that they themselves can control. Having really good quality documentation is an essential part of that.

CL: In 1999, there were a lot of FUDs. Looking them back now, do you still "trust no one"?

Jeremy: Mindcraft actually were desperate to restore their reputation. Their original article was a really nasty piece of work, actually. I still think so... I was talking on the phone to Bruce Weiner who is the Mindcraft guy who did it, and I told with him a lot... If you look at Mindcraft, their reputation is all they have to sell, and their reputation got a severe bashing, because they did such an obviously one-sided report. So they were desperate to restore their credibility, because that's all they have to sell.

 I've known about Mindcraft for quite a long time, and actually I was pleased when they did what they did, because previous Mindcraft reports have been showing that Windows NT is a better web server than Solaris, for instance, and they've shown that Windows NT was a better file server than Novell NetWare, a bunch of these things were just, so untrue that they were just literally sort of Microsoft-sponsored propaganda turned into a so-called independent report. They've done a bunch of hatchet jobs on Sun and Novell... So when they did it to Linux and Samba and Apache, I thought "Oh, good. At least we were some very good company!"

But having said that, they did pointed out some problems, which I'm pretty grateful to them for.

Now the whole thing, benchmarks, especially for Windows file serving, are really terrible. There are new good benchmarks for Windows file serving. I admit I have played the benchmarking game in just the same way that Microsoft did it in the past, when we released Samba for IRIX when I was at SGI.  It's a game. I've played that game. So for instance, you can come up with the benchmark result that shows that the file server is capable of serving files faster than the physical speed of the wire that you have between the two. You know, it makes no sense.

When Mindcraft came out with their second report, I knew that Linux and Samba were going to lose, because after that first report, we ran the same benchmark in SGI's lab. We knew what the numbers were, and I knew we were going to lose, so I decided not to play. That's why I refused to take part in the second set of benchmarks, because, you know, for that kind of thing, you only play if you know you're going to win.

I'm being completely honest here. To my mind, much much better benchmarks than the Mindcraft thing were done by a print magazine in the US called "Smart Reseller". There's a guy at Smart Reseller called Steven von Nicholus. The reason why I like him is because he did a similar set of benchmarks, but what he did was very different to what Mindcraft did. Mindcraft did their benchmarks up at Redmond. They had the Microsoft engineering on site, they were local. They came in and they tuned the NT server. Admittedly for the second set of benchmarks, they said "You can come up and tune your server", but I knew we were going to lose at that point so I didn't want to play.

But what Steven von Nicholus did was, he didn't ask anyone for tuning advice. He just said "OK, if you were in the real world, you don't get Samba team members to fly to your site to tune your server, you don't get Microsoft engineers on site to tune your server. In the real world, you buy a box from, say, Dell or somebody, you put software on it, and you just run it. So let's see what the performance would look like, if you just take a PC box, put NT on it and run the test, put Samba and Linux on it and run the test." And if you do that, and you don't tune anything, Samba outperforms NT by factor of two.

 Yes, admittedly, there are some high end things that need fixing, but actually the performance bottlenecks are not in Samba. The reason I've known that is because Sun actually requested that I help them out with the benchmarks that they did at PC Week. PC Week were doing benchmarks with Windows NT, NetWare, Linux and Solaris running on x86. Now Sun had the product that they call PC NetLink that essentially does what Samba does, but they declined to use that product for the benchmark. They asked to use Samba instead, which tells you something... So I went up there to tune Samba for them. We didn't get great results, and what we found was that on Solaris, if we removed the disk subsystem from the problem, we beat NT on the same hardware. On Intel x86 hardware running Solaris we actually had better file serving numbers than NT on the same hardware. However, as soon as you actually stored the data on the Solaris disk subsystem, the numbers halved. The reason is because Solaris disk subsystem is quite slow. I believe that's been fixed for Solaris 8. So I would like to redo the benchmarks on Solaris on x86 for you.

On Linux, the problem was different. The problem was the TCP stack. Solaris has a wonderfully multi-threaded TCP stack, so you can get multiple packets going through the stack at the same time. The Linux 2.2 TCP stack basically has a large global lock. What that means is that you cannot put the throughput through the stack that you can on Solaris or, in fact, on NT. So that's one of the things that's been very heavily worked on for Linux 2.4. In fact, they really have a lot of very interesting plans that will probably not come to fruition until Linux 2.6. But I'm expecting to close the gap with NT for Linux 2.4, and to be able to beat them on Linux 2.6. But it isn't really a Samba issue. It's actually a kernel issue with the TCP stack.

So those benchmarks were very interesting, but didn't actually prove anything. Remember, it was a very carefully crafted hardware platform. So I would still say "trust no one", especially not me :-) Because if I'm going to do any benchmarks, the only way I would love to do the benchmarks, is when I know I'm going to win. So if you see me going to be doing benchmark, you know what the outcome is going to be, because if I'm not going to win, I'm not going to do the benchmark. You know, that's one of the things you learn when you are a commercial vendor, the only way to win if you can't win is not to play the game. And, yeah, I still stand by the article.

I've done a lot of work on that area, and there are a lot of issues with those benchmarks...

CL: What was the biggest (or most surprised) affair/event of the year 1999, with regard to Samba?

Jeremy: I was disappointed that we didn't get the PDC support. I really wanted to do that within 1999, and we haven't done that.

I was surprised at Veritas adopted Samba. Veritas is basically saying, "Yes, we've evaluated all your competitors, and our customers want us to officially support Samba as an official product on Solaris."

That was probably the most positive and pleasant news, and sort of the fact that we didn't get the PDC support done is the most negative news, and there are many reasons for that. A lot of it is time and effort that we need to sink into that code.

I was also quite pleased to see Dell, because Dell is now a Samba vendor. They probably don't know it, but they are. Dell announced an appliance that is basically a revised Cobalt Cube. Essentially, because that is using the Cobalt software, and Cobalt software depends on Samba, Dell has become a Samba vendor. Actually there are quite a few top manufacturers that are now Samba vendors, including IBM. So we're actually colonizing the top-level Unix vendors...!

 CL: Then, what do you think was the biggest (or most surprised) affair/event of the year 1999, with regard to Linux and open source as a whole?

Jeremy: The Mindcraft benchmarks were the most disappointing thing. They were probably the most negative thing, but in some ways that was positive, because it showed that we are their serious competitor and Microsoft is willing to spend lots of money to have a go, which in itself is a big news.

But I think for sort of most positive, has to be the incredible snow-balling of Linux support from everybody!

Why it happened, I don't know, but I think what we've seen in 1999, is Linux actually going across the chasm (as in Geoffrey Moore's "Crossing the Chasm") and becoming a mass-market product.

I've expected it to happen, but I didn't expected it to happen so soon!

I remember I was on a panel called, "The Future of Linux", with Linus and guys from Red Hat and Intel. We did a big presentation in Silicon Valley, and I knew it was going to be big, because we had about two thousand people there. But it was a few weeks after that Oracle announced that they were going to support Linux, and Sybase and a bunch of the database vendors followed. That was like late '98, and after that, things just snow-balled in every increasing way. I can measure it by the growth in email in my inbox daily. If I plot the curve of how much email I'm getting, it's going up and it's going up at the ever increasing rate, and I can't cope with it! But it's incredibly exciting.

I knew we were going to do well! I remember I was at the Atlanta Linux Showcase, and we were at a hotel and I was sharing a car with guys of Linuxcare and Linux.com, and we were all chatting. Then one of the luggage people, the bell clerks, came up to us and said "Are you part of that Linux conference?" and I said "Yeah" then he said "I'm thinking about installing that on my home machine." He was a kid going to college and was in these bellhop's uniform, and I knew that we were going to go mass-market at that point! It was an amazing experience. The bellhop's using Linux. We've got to win! It was a wonderful feeling.

 CL: What do you think will be the biggest thing in 2000 with regard to Samba, and Linux/open source as a whole?

Jeremy: I think we'll see Samba adopted as the appliance vendors' platform of choice. What I'm expecting to see is a lot of people coming out with basically things to compete with Cobalt, sort of like all file servers in a box. And I think what we will see is that 90% of those people will pick Samba. So it will essentially colonize its niche of these file server appliance software. That's probably the most positive thing I think we'll see happening around Samba.

For Linux, I'm really looking forward to... I know this is proprietary software, I shouldn't be :-) but I'm really looking forward to having Corel's office suite on Linux. Because at that point, you've got Corel Office, StarOffice, Applixware, K-Office... (Gnome would be going to be a little further behind), and all of those will be really full-featured, robust office suites that would read/write Microsoft Office file formats. I think what we will see is significantly large corporations ripping out Windows desktops, and replacing them with the Linux desktops, to perform the same functionality. I think the year 2000 will be the year that we will start to see Linux colonize the desktop.

Who knows? If the Department of Justice pokes Microsoft up we may even see Microsoft Office on Linux. It could happen. Because if Microsoft Office would be a separate business unit, and would be untied from the Microsoft operating system, I bet you that the people in charge of that would say, "Get me a Linux port now, because we want money."

So I think the year 2000 will be the year of Linux on desktop, or, at least, even if it won't take over the desktop, you will see an increasing presence on the desktop. I'm really looking forward to going to my local computer store in Silicon Valley and buying shrink-wrapped Linux games. You'd never really know how that could feel. Buying Quake III for Linux, in a shrink-wrapped box. I don't get a lot of time to play games, but I just want to know I don't have to boot into Windows to go and play games. That's a big part in taking over the desktops, and I think that will happen in 2000. I'm hoping!

Copyright 1999