Jim Hugunin - interview

From LXF Wiki

ON THE BEGINNING "When I first looked at Java, I thought it was a total joke."

Ironman Hate .NET? So did Jim Hugunin until he tried it ­ then he wrote IronPython. How do the Python community feel about his irrepressible urge to hack?

Jim Hugunin loves programming in Python, the dynamic language he first heard about while juggling with developer Pavel Curtis. But he also loves extending it, and a curiosity about Java led to his creating Jython, an implementation for Java developers. The same thing happened with .NET ­ he couldn't believe it was as bad as he had heard, found that it wasn't and created IronPython to use with APIs such as .NET and Mono. This curiosity may have taken him away from the open source framework Curtis backed, and Hugunin now works for Microsoft. It's not something he regrets, he tells Paul Hudson.

Linux Format: What instinct do you think it is in you that seems to need to reimplement Python?

Jim Hugunin: That's a complicated question. Before I worked on Jython I worked on a project called Numeric Python. At the time I was at MIT in their graduate programme, doing a lot of scientific computing. I'd been using languages like Matlab before, and I was really frustrated by the fact that Matlab was such a great language for doing numerical calculations and the kind of things scientists needed to do... but if ever I needed to do a real programming task, and I wanted dictionaries, lists and classes, it just broke down and my code would get really ugly and horrible.

I started messing around with a lot of different programming languages at MIT:Scheme was one of them, Perl, Python, a lot of that class of languages. And I really liked Python, and I was playing around with this object extensibility it's fairly easy to extend Python with new objects ­ and I found that I could add to Python these numeric arrays that were the key feature of Matlab. By adding the numeric arrays, now it felt like I was programming in Matlab when I was writing Python code. When I wanted to go above and beyond, I had all the power of Python. That was how I first got hooked on Python. That was the positive side of it.

LXF: So it worked!

JH: Yes, it worked great, and Numeric Python is still used pretty widely today. But I got really frustrated with extending the language in C. Because, and this carries the story all the way through, according to Microsoft's terminology...`managed language' mainly means a language that has garbage collection, reflection, real solid-type safety and dynamic loading.

That's the same sort of features that Python has had forever. But C doesn' have that. So there was this huge mismatch between the C world and the Python world that whenever I had to program my extensions to Python in C, it got very frustrating because all of a sudden I had to think about memory management again, or type safety. As I was doing that project, Java came along. When I first looked at Java, I thought it was a total joke because it ran about as fast as Python did. Even for doing heavy-duty numeric things like fast Fourier transforms [FFTs], which are the kind of algorithm that C runs about a hundred times faster than Python, I looked at doing it in Java, and it ran at Python speed. So I thought, "Why are they bothering with this whole new platform when it runs at the same speed?"

And then, and this is true although it sounds made up since I'm now at Microsoft, it was Microsoft's Java implementation that convinced me that wasn't true any more. Because Microsoft was the first company to make a very fast, just-in-time compiler for Java, and I played with that and my numbers all totally shifted. Now I was seeing through these fast, heavy-duty computing applications that Java was roughly as fast as C.

So I saw all the advantages that Java had for the C side and I was curious to know if I could make Python and Java work well together. What really clinched the deal for me for doing Jython was Aaron Waters, who I owe a great debt to, because I said, "This would be a cool thing to do" and Aaron Waters said, "There's no way you could possibly ever make Python run on top of the JVM [Java Virtual Machine]." I think Aaron figured out one of the tricks to getting me to do things, which is you say something is impossible, and that's... encouraging.

I sat down and tried to do it, and hacked it out, and Jython was one of those fabulous projects. It is probably the best project I have ever worked on, because it kept getting easier, which is a very unusual thing in software development projects. They usually have this honeymoon period, when everything is working really well, and then you realise, "Oh my God, I forgot about these things."

I realised that the JVM had these features that I was looking for to match up with Python with comparable performance with C and C++. I discovered more about how reflection worked and how class loaders worked and all of this stuff worked, and the project just got easier as time went on. I found there was this beautiful mesh between Java as an extension model, so now I didn't need C any more; now I had this very nice smooth transition. When I wanted to do C-like things I could do them in Java and get all of that performance. That is why I got into Jython. I thought that was exciting.

LXF: You spent time outside the Python community after that on AspectJ. Where did IronPython come from?

JH: IronPython was really started on a lark. I had read all of the things about Microsoft's new common language framework, this great new thing that Microsoft had that could run all of these languages. And I had totally laughed at them, because I knew Java could run hundreds and hundreds of languages quite effectively. In particular, I was struck by the fact that everybody was saying that it couldn't run dynamic languages. Some of this had to do with some early experiments trying to get Python and Perl running on top of .NET that concluded you couldn't, and I really wanted to know how Microsoft could have screwed up .NET so badly that it was worse than the JVM for running dynamic languages.

I wanted to write a short, pithy little paper saying, "Why .NET is a horrible platform for dynamic languages." It would win me some points in the Java community, which was a community I was having a lot of fun in, and help me understand how they'd screwed up, because I was curious. So I started messing around with doing it, and, again, some of it was like the early days of Jython, where I found that actually it was working really well. There were some things that weren't there in the early days of Jython: there was a very nice byte-code generator, which shipped with the runtime. And the performance was really good ­ 70% faster than CPython.

So I think the reason why I did [IronPython] wasn't so much this hook on needing to reimplement Python again and again, it was getting my teeth into this challenge that people were saying you can't do it...

LXF: Made you want to do it?

JH: Yes, and finding that it works, and going down this slope of, "Well that worked, so if I do this will that work even better?"

I went down that same road with .NET, and it was learning the ways of trying to be a multi-language platform. My first shock was, so, the bytecode format in Java really assumes that you have Java source code. When you generate your debugging information, you never specify what language did this code come from, because 99% of the people assume it came from Java. But there's a flag in the bytecode for .NET, which is, `Give me the GUID that you're registered for your language and your language implementation this is coming from because we expect all things to be coming from different languages.' That again just led me down the path of things seeming to be working well when it didn't seem like they would before.

LXF: Did you find that doing Jython and IronPython caused animosity towards you from the Python community?

JH: You know, no. And I thought it might have, quite frankly. Jython I did for a while, and then I left Jython in a good way ­ made sure that I had somebody set up to be a successor, who was going to be able to run the project and make sure that all the licensing and the hosting was all set up and other people could continue running it.

It's always a little painful to leave a project behind and not be able to keep working on it, but you've got to do it, you can't stay attached to one project forever. I can't; maybe other people can. Guido [van Rossum, Python creator] does, but even he's not attached to Python at the closest level, and he can do that because of the size of the community. Jython has built up a good community, but it's not nearly as big as the CPython community is, so it doesn't work quite as well.

But nobody saw IronPython as taking away from Jython. What I love about the Python community is that I think it is the language community that is the most open to multiple implementations. They really see the language ­ and this comes from Guido ­ as being this one cool thing, but the implementation can be separate from the language, and they really embrace other platforms. Python has had the best Windows support of the `P' languages for a long time. A lot of that's due to [Microsoft consultant] Mark Hammond, who did some amazing work, but some of it is due to the community saying, "Windows is great, we want Python to run really great on Windows. We also want it run really great on Linux and Macs..."

Jython was just seen as another opportunity for Python to get more deeply embedded with the Java developers. And IronPython: people seem to see it as, "Here's a great opportunity to keep Python going in this other, exciting space of .NET." I didn't get any grief from people doing the Jython stuff and IronPython. There are people who would love to see me go back and devote myself full time to Jython, but there are people who would love to see me go do a lot of things.

LXF: Was there any, "Why are you moving to Microsoft?"

JH: There wasn't that much, actually. I presented IronPython to the Python community in March 2004, and I presented it as I was an independent consultant at the time trying to figure out what I was going to do next. Then it was four or five months later at OSCon that I announced I was moving to Microsoft. People saw IronPython without knowing I was moving to Microsoft, and they were thinking, "How is IronPython really gong to fly?" They saw it as a cool thing but they weren't quite sure how it was going to happen. I was spending my time doing AOP consulting ­ that was what was paying the bills, and that was where most of my time was going. So my moving to Microsoft I think they saw as a real chance for IronPython to do really well.

I'd be curious to know what would have happened if I'd done things in a different way. If my very first announcement of IronPython had been, "I'm going to Microsoft" and people hadn't had a chance to get used to one before hearing the other, I don't know if the response would have been the same.

LXF: As you work on IronPython at MS you go through the GotDotNet [www.gotdotnet.com] Shared Source scheme.

JH: Yes.

LXF: What your thoughts on that; on the difference between before Microsoft and now with Microsoft?

JH: I'm not going to pick any bones about it: GotDotNet is not my favourite Shared Source site. I really don't think that it's in the same league as most of the open source/shared source development sites. I think that's something that Microsoft needs to do a better job at. But Microsoft is learning how to play in this space. It's never been Microsoft's core business, and Microsoft has done more and more ­ we did the the WTL [Windows Template Library] and the Flex Wiki stuff.

We are continuing to try to learn how effectively Microsoft can play in this space without totally changing what Microsoft does. GotDotNet is an example of where there is a long way to go in my personal opinion.

LXF: But at least they're on their way!

JH: The fact that GotDotNet is there is huge. It's great step in the right direction. But I hope that we'll be moving a lot further. We've certainly gone out of our way... we've got a lot of feedback from people, a lot of our users really didn't like GotDotNet. The mailing list was filled with people complaining about it. And we probably addressed half of their concerns. We shut down the forums but we kept the mailing list running so that people could still have the effective discussion.

People want us to have an open source repository. We don't have the infrastructure to do that in a reasonable way right now, so we're making releases every two weeks. It's not quite as good as having an open source tree but we're still trying to get the feel that it's very interactive ­ when you suggest something it's quite possibly fixed within a week and back in your hands. To figure out how to do a lot of the .NET stuff effectively, we need people's feedback on what's working for them, and that only works if there's a feedback loop.

LXF: I'm interested in what the downsides of the Shared Source License actually are ­ you say it's short and clear. Are there any bad things about it from the open source point of view?

JH: The fact that it's called Shared Source, and I think that's an unfortunate thing. It's the perception rather than the reality of the licence. I don't think there's anything wrong with the reality of the licence. It's essentially a BSD-style licence, the style that I prefer, but you can put it in commercial code and you don't need to release all your commercial code to the world, so I happen to like the terms of the licence very much.

LXF: So it's quite like LGPL?

JH: More like the BSD License. MPL [Mozilla Public License] is probably the closest licence out there.

LXF: What are your thoughts on IronPython running on Mono?

JH: This is one of those that I preface: everything I've told you before has been my personal opinion. What I'm going to tell you now is even more so my personal opinion. When I first started talking about IronPython, my very first talk on it said, there's this cool .NET framework thing out there. It's cool for these two reasons. One of them is that Microsoft is putting this huge effort behind it, and therefore I see this great momentum coming behind it and it's going to be very successful, powerful and effective. The second reason is that it's this international standard, it's an ECMA/ISO standard and it's a real standard, because there are people out there building alternative implementations of the standard, rather than coming from the original person. So for me, Mono is part of why, as a language designer, I find .NET particularly interesting. It's a platform, it's very broad. At the end of my first talk at OSCon 2004, IronPython had only ever run on my laptop. The only computer in the world that it had ever run on: my laptop running Windows. By the end of my talk, people had downloaded it and run it on OS X and Linux ­ effectively. And I thought that was awesome. And the Mono folks have been doing a great job. The performance gets better with every release, and I like to see that. That's where I see it.

LXF: So what is your future at Microsoft? Are you going to continue with IronPython or are you going to do, I don't know, SchemePython?

JH: No, no, PhyrricPython would be the perfect one, right?

LXF: PHPython?

JH: Python on top of the PHP engine? You probably don't want to hear me talk about PHP... LXF