[00:00.820 --> 00:04.620]  The grid using grid computing systems.
[00:04.920 --> 00:09.240]  So we hooked up a little earlier in the year and came up with this talk idea.
[00:10.020 --> 00:19.160]  And some of you know, we'll lay the groundwork and stuff, but some of you know that the TeraGrid was hacked sort of famously in about April.
[00:19.160 --> 00:24.600]  And they had some ongoing security problems and got a lot of publicity.
[00:24.960 --> 00:27.680]  And I want to let you know at the outset, we know nothing about that.
[00:27.840 --> 00:28.960]  Okay, we don't know anything.
[00:29.220 --> 00:29.640]  I'm not...
[00:29.640 --> 00:38.380]  Even though I work at a supercomputing center and I know people at some of the supercomputing centers that were affected, I have no inside knowledge whatsoever and we don't know who done it.
[00:38.600 --> 00:44.600]  And we came up with our proposal for the talk before those things happened.
[00:45.040 --> 00:45.980]  So we're innocent.
[00:47.680 --> 00:49.120]  Just to let you know.
[00:49.240 --> 00:59.220]  And I did get email from someone at NCSA Security, the National Center for Supercomputing Applications, when the Fifth HOPE speaker schedule went up.
[00:59.220 --> 01:00.000]  It was up.
[01:00.240 --> 01:02.200]  It was up, but it got slash dotted.
[01:02.660 --> 01:11.380]  And he said, dude, could you let us know what you're going to talk about before you go and talk about it at the conference?
[01:11.500 --> 01:12.100]  And I said, yeah, dude.
[01:13.800 --> 01:16.080]  Unfortunately, we didn't know what we were going to talk about until just recently.
[01:16.560 --> 01:20.460]  So I'm not quite on with that promise.
[01:21.860 --> 01:22.720]  Okay, here we go.
[01:25.320 --> 01:27.280]  And do one of these.
[01:28.560 --> 01:29.260]  Okay, here we go.
[01:29.360 --> 01:30.820]  This is our official slide.
[01:33.040 --> 01:33.360]  Great.
[01:33.840 --> 01:34.440]  Okay, so that's us.
[01:34.480 --> 01:35.480]  We got a couple URLs.
[01:37.080 --> 01:37.780]  Here we are.
[01:38.420 --> 01:42.360]  We've been involved with the conference, but as I said, we do other stuff as well, and this is part of it.
[01:42.360 --> 01:50.500]  So there's a lot of excitement about grid computing, and there's also a lot of things that are kind of difficult to understand.
[01:50.980 --> 01:51.940]  And it takes a while.
[01:52.160 --> 01:54.000]  It's like a lot of these technical things.
[01:54.080 --> 01:55.880]  It's sort of a steep learning curve.
[01:56.080 --> 02:03.640]  So we're going to talk a little bit about the general grid environment, some of the standards that are coming out, talk about what grid computing is, all that sort of general stuff.
[02:03.640 --> 02:12.440]  We also want to share with you some of our experiences in getting grid software, particularly the Globus Toolkit and related components, up and running.
[02:12.860 --> 02:15.000]  And we thought those would be pretty informative.
[02:15.180 --> 02:22.280]  And we also have a bunch of comments, and as I said, no new exploits, no announcements of heretofore unknown weaknesses.
[02:22.280 --> 02:35.160]  But a few comments about weaknesses in the grid computing infrastructure, the software, the deployment, the practice, the policy, a lot of different aspects with potential and, in some cases, actual security weaknesses.
[02:35.160 --> 02:56.180]  And, of course, some recommendations for either, if you're looking to be the next person that hacks the terror grid, things to look for, or probably a little more relevant for people in the community listening to this talk is, what would you do to make your system stronger,
[02:56.340 --> 02:58.340]  you know, more resilient to some sort of attack?
[03:00.560 --> 03:03.700]  So, we got, as usual, more stuff than we can do.
[03:03.880 --> 03:04.900]  This is kind of the outline.
[03:06.000 --> 03:08.340]  What is, no review, talk about Globus.
[03:09.920 --> 03:15.100]  Let's go ahead and do the what is, and then what I'm going to do is, no, actually, I want to go ahead and get that started now.
[03:15.580 --> 03:16.400]  I'll get this started now.
[03:16.580 --> 03:27.180]  So, one of our, one of the things we want to talk about is the fact that Globus software, which is kind of the standard, if you're, if you want to say what's grid, one of the things you can say is, well, you know, we're using Globus.
[03:27.180 --> 03:30.340]  That's what makes a grid computing environment versus something else.
[03:32.340 --> 03:35.920]  Globus is a huge, difficult to install, difficult to administer package.
[03:36.180 --> 03:39.500]  So, I'm operating on my 17-inch Power Mac G4 here.
[03:40.120 --> 03:42.340]  So, I'm not going to try to actually install Globus.
[03:42.480 --> 03:44.560]  I think it can be done, but I haven't ever tried.
[03:44.800 --> 03:47.320]  But let's go to the directory.
[03:52.280 --> 03:57.160]  And I know that you might not see that entirely, but let me show you what's here.
[03:57.160 --> 03:58.400]  And I'll just explain it to you.
[03:58.480 --> 03:59.460]  And then we'll get something running.
[03:59.660 --> 04:02.440]  And then we'll talk and we'll come back and it'll probably still be running.
[04:02.800 --> 04:07.180]  These are the packages, the Globus toolkit packages.
[04:07.500 --> 04:11.780]  So, the Globus toolkit source installer, which is basically what you need.
[04:11.900 --> 04:15.620]  If you get a binary distribution for this package that we'll talk about.
[04:15.940 --> 04:17.260]  Have you tried the binaries, right?
[04:17.860 --> 04:20.760]  They're not really all that much faster than the source to install.
[04:22.320 --> 04:28.020]  And they're less, well, they're sometimes more reliable and sometimes they really screw up.
[04:28.180 --> 04:28.760]  Right, exactly.
[04:28.940 --> 04:35.830]  I mean, and usually if you get a binary and you're having trouble and you go out on the mailing list, they say, dude, you got to install from source.
[04:36.180 --> 04:36.620]  Right?
[04:36.900 --> 04:38.100]  And this is all Java code.
[04:38.240 --> 04:40.980]  So, you know, Java compilers usually aren't blazingly fast anyway.
[04:42.240 --> 04:44.120]  And these are many, many packages.
[04:44.120 --> 04:52.480]  So, the Globus toolkit version 3.3.0, the source installer is, whatever that is, 123.
[04:53.900 --> 04:58.960]  I'm sorry, that's what I thought it was, about 64 meg or so compressed, gzipped, of source code.
[04:59.180 --> 05:00.060]  So, that's a lot.
[05:00.200 --> 05:01.940]  And we can, I already decompressed it.
[05:02.000 --> 05:05.540]  So, we can go down to the source code installer directory.
[05:06.500 --> 05:10.980]  And we see we have, I already did a build, build hyphen old, and I moved it over.
[05:10.980 --> 05:13.270]  And so, we can do, this is zshell.
[05:13.420 --> 05:14.400]  This is why I'm running zshell.
[05:14.500 --> 05:19.160]  We can do wc star star slash star Java.
[05:19.480 --> 05:22.330]  And it will recursively descend all of the directories.
[05:22.480 --> 05:23.560]  It's a nice little zshell trick.
[05:23.800 --> 05:24.800]  You can't use zshell.
[05:24.920 --> 05:27.720]  Did you try using anything other than born shell with your experiences?
[05:27.920 --> 05:28.820]  No, I've only used born.
[05:29.000 --> 05:32.560]  Yeah, it's like, it's like, you're very constrained in many cases.
[05:32.560 --> 05:33.540]  Born shell is it.
[05:33.960 --> 05:38.520]  It counted 239,000 lines of Java code.
[05:38.840 --> 05:41.900]  That's including, you know, blank lines and comments and stuff like that.
[05:42.080 --> 05:46.000]  So, depending on how you count your lines of code, you know, might be a smaller number.
[05:46.160 --> 05:47.320]  And that's just the Java code.
[05:47.420 --> 05:51.420]  There's also, of course, the build.xml files that the ant installer uses.
[05:51.420 --> 05:57.580]  There's also the, some WSDL files and just all kinds of different things that are down there.
[05:58.100 --> 06:00.860]  So, big, huge, monstrous, complicated package.
[06:01.240 --> 06:07.860]  And if you look in, if you look in that directory, you see that it's basically a package of packages.
[06:08.120 --> 06:12.080]  These are, this is from a build I did just the other day just to see how this would work.
[06:12.520 --> 06:18.100]  And you see it's just these bunch of tar.gz stuff and then the actual installation directories.
[06:18.100 --> 06:22.000]  So, the message here is that Globus is complicated and really unduly so.
[06:22.280 --> 06:27.200]  And, of course, complication when you have many lines of code means statistically there are bugs, right?
[06:27.440 --> 06:31.100]  And, again, statistically some of those bugs will be security related.
[06:31.340 --> 06:34.860]  So, let's just go ahead and we'll get this running.
[06:34.980 --> 06:36.120]  There's a little install script.
[06:37.500 --> 06:38.700]  You tell it where to go.
[06:40.320 --> 06:43.060]  Just an arbitrary place because I'm not going to...
[06:43.060 --> 06:44.700]  Well, actually, did I remove that directory?
[06:44.840 --> 06:46.460]  Let me make sure I remove that directory first.
[07:03.290 --> 07:06.750]  Okay, so you just run this little install script that basically knows all the many, many packages.
[07:07.070 --> 07:08.270]  You give it a destination directory.
[07:08.450 --> 07:10.970]  So, we are now building the Globus toolkit, okay?
[07:11.210 --> 07:12.810]  Now we're going to go on to the rest of our talk.
[07:13.010 --> 07:20.530]  And I think, unless we forget to look, we're going to come back later and the Globus toolkit will still be building on this pretty spiffy, reasonably fast laptop computer.
[07:20.530 --> 07:30.210]  And this is part of why they're... part of my identification of security holes in grid computing, because it's painful to fix.
[07:30.430 --> 07:36.930]  And as I said, you go out and say, gee, you know, I think there's an error in this little package and I tried doing this little tweak and I tried doing this little tweak.
[07:36.930 --> 07:41.770]  The answer on the Globus mailing list is reinstall from source, right?
[07:41.930 --> 07:45.310]  And if you have to go back and apply whatever changes you discovered the first time.
[07:45.910 --> 07:51.190]  All right, let's talk about what is grid computing and that'll make more sense for you as we go on.
[07:52.790 --> 08:00.870]  So, there's a relationship between grid computing, parallel computing, cluster computing, other, you know, shared memory computing.
[08:01.190 --> 08:12.370]  Essentially what you're doing is using multiple processors together, getting data and commands back and forth somehow, and then getting results back to whoever needs them at the end.
[08:13.030 --> 08:20.490]  With grid computing, typically what you're doing is having a slower network, so it's not a cluster with like a mirror net or something like that.
[08:20.490 --> 08:27.050]  You have the ability to have a hybrid set of systems, so rather than having, you know, in a cluster, you have a cluster of itanium 2s, right?
[08:27.090 --> 08:28.510]  You have an optron cluster.
[08:29.210 --> 08:35.250]  With the grid, you could have a grid which is all kinds of different sorts of computers.
[08:36.590 --> 08:38.450]  And so you're not going to guarantee latency.
[08:38.610 --> 08:43.510]  You're not going to require a homogenous or relatively homogenous computational architecture.
[08:46.110 --> 08:51.670]  And what that means is that you're going to look at problems that don't require a whole lot of communication.
[08:51.930 --> 09:03.450]  If you have a whole lot of... let's say you have a shared memory problem where you need to get data from place to place, like from one processor to the next processor to the next processor, that would be your typical parallel, massive parallel computing environment,
[09:03.670 --> 09:05.170]  shared memory, that sort of thing.
[09:06.790 --> 09:12.750]  That's where you're having very low access to do communication or transfer data from one node to the other.
[09:12.950 --> 09:23.650]  That's not a good type of problem for a grid system, because you have separate systems, and they're using this grid infrastructure, which is like web services, except it's called grid services.
[09:23.830 --> 09:27.270]  So you're not using like a high-speed interconnect, low latency, all that stuff.
[09:27.470 --> 09:34.390]  So you're looking at problems for grid computing, which are essentially coarse-grained parallel, as opposed to fine-grained parallel, as opposed to serial.
[09:38.360 --> 09:39.780]  The word grid is everywhere.
[09:40.020 --> 09:45.020]  We don't have the types of vendors here, but when you go to one of these trade shows, you know, everything's grid.
[09:45.260 --> 09:47.120]  You know, Oracle 10G is Oracle 10 Grid.
[09:47.340 --> 09:48.320]  Sun is doing grid.
[09:48.420 --> 09:49.500]  IBM is big on grid.
[09:50.720 --> 09:55.820]  The word grid is overused to mean essentially any distributed... sometimes just any distributed environment.
[09:55.980 --> 09:56.820]  Forget about heterogeneous.
[10:00.100 --> 10:06.900]  For our purposes, sort of the standard that we're going to stick with, even though, as I say, there are other definitions, they're reasonably acceptable.
[10:07.140 --> 10:08.900]  But we're going to talk about the Globus Grid.
[10:09.120 --> 10:20.860]  The Global Grid Forum is this group that's formed after the model of the IETF and W3C and these other standards organizations, trying to form grid standards.
[10:21.400 --> 10:26.660]  And Globus and other components that we'll touch on a little bit here are part of those standards.
[10:26.660 --> 10:36.480]  Now, the standards, just like way back in the browser war days, the standards are emerging at the same time that applications and products are emerging, which means that really is kind of a free-for-all.
[10:36.620 --> 10:38.680]  But we're going to focus in on Globus.
[10:45.930 --> 10:46.290]  Okay.
[10:47.090 --> 10:48.810]  I'm not going to go through all this in great detail.
[10:48.830 --> 10:54.850]  I'll get this up at Petascale over the next little while, and you can email me if you'd like the presentation link.
[11:00.900 --> 11:05.660]  The Globus Toolkit I mentioned, OGSA is this Systems Architecture Design.
[11:05.660 --> 11:06.980]  There are some design documents.
[11:07.020 --> 11:13.180]  They're undergoing a major change now, which they've done before, from one sort of way of doing things to another sort of way of doing things.
[11:13.360 --> 11:18.700]  More, you know, potential for problems to happen.
[11:18.700 --> 11:27.200]  And one of the things that's kind of cool about the current change is a move to basically things on web services.
[11:27.540 --> 11:30.080]  And probably some people program with web services.
[11:30.240 --> 11:31.500]  There's a lot of web services out there.
[11:32.020 --> 11:38.200]  Web services is built into a product like WebSphere, at least one of some of their incarnations.
[11:38.540 --> 11:43.780]  But it's also implemented as part of the set of Apache Software Foundation products.
[11:43.780 --> 11:51.360]  This is a whole package of Tomcat, and Jakarta, and Ant, and some other things that they sort of loosely group together as web services.
[11:51.660 --> 11:57.600]  Essentially, this is a way of sharing work among computers using something pretty similar to HTTP.
[11:58.100 --> 12:00.200]  You know, just sort of port 80 communication.
[12:01.740 --> 12:06.780]  With some abilities to build things like quality of service, encryption, and so forth on it.
[12:06.980 --> 12:08.400]  But not really built in.
[12:08.540 --> 12:10.740]  What Globus is intending to do...
[12:10.740 --> 12:11.060]  What?
[12:11.720 --> 12:12.600]  Are we optimistic?
[12:12.600 --> 12:13.160]  I don't know.
[12:13.720 --> 12:20.440]  Is they're going to take a web services framework and add the security layer and add the other types of things.
[12:20.680 --> 12:23.500]  Authentication and so forth that are part of Globus.
[12:23.700 --> 12:24.660]  They'll manage to do it.
[12:24.780 --> 12:28.300]  What stands to be seen is whether or not they'll do it well.
[12:28.920 --> 12:33.220]  They're actually pretty good about getting things done that they say they're going to get done.
[12:34.320 --> 12:38.120]  It's just that there's always extra bugs that you never really expect.
[12:42.180 --> 12:49.260]  The main thing that is a component of grid security is this highlighted phrase here, this idea of virtual organization or VO.
[12:50.820 --> 12:52.480]  So you're on the web, right?
[12:52.660 --> 12:58.840]  If you want to make your stuff available on the web, you have this nice little CGI program that's going to do some processing, produce some results.
[12:59.900 --> 13:10.580]  You're kind of stuck in terms of your ability to restrict access to that somehow because you have .htaccess file, some other sort of password system.
[13:10.580 --> 13:17.700]  You might use secure web transfer, you know, HTTPS with certificates and so forth.
[13:18.080 --> 13:20.460]  But you don't have a lot of fine-grained control.
[13:20.600 --> 13:26.960]  What the virtual organization does is it basically gives you an access control list on an application level.
[13:27.120 --> 13:38.320]  And it says, okay, here are the... and we layer on that certificates so that you say, okay, here are the different systems that are out there that are allowed to get access to this service.
[13:38.320 --> 13:41.020]  And you can even have more granularity than that.
[13:41.140 --> 13:48.420]  But it's basically a finer-grained level of access control than you see with the generic web server, the generic web services.
[13:48.720 --> 13:59.160]  This concept is really important because what it means is, at least conceptually, and I don't know that many places are really doing it for a variety of reasons.
[13:59.260 --> 14:04.180]  But conceptually, you could use this for enterprise types of communication or computing.
[14:04.180 --> 14:10.920]  You could use it for even secure computing environments, you know, the extent to which you trust the implementation and so forth.
[14:11.060 --> 14:15.600]  But it's designed to be able to set up essentially an arbitrary group of systems.
[14:15.940 --> 14:17.680]  They might all be in the same organization.
[14:17.680 --> 14:26.840]  They might be across the internet, you know, anywhere, into a virtual organization and have a level of trust and, you know, logging and authentication and all that stuff within the virtual organization.
[14:27.220 --> 14:32.280]  That's kind of the cool selling point about grid services that make it a lot different than web services.
[14:32.460 --> 14:34.020]  There's some overhead associated with that, of course.
[14:34.420 --> 14:36.800]  You want to talk about authentication and authorization?
[14:37.520 --> 14:46.240]  There is one point that's very important to know about Globus and to know about some of these things, is that there's a very big difference between authentication and authorization.
[14:48.020 --> 14:50.900]  I had a little poem that I accidentally created.
[14:51.780 --> 14:55.920]  Authentication is the verification of the identity of a remote entity.
[14:56.760 --> 15:01.600]  The remote person, knowing that you are Bob, is authentication.
[15:02.080 --> 15:05.080]  That doesn't necessarily mean that Bob is authorized to use this machine.
[15:06.540 --> 15:14.340]  With Globus, you have X509 certificates, let's say, and you can prove that you are this person.
[15:14.520 --> 15:16.180]  You have decrypted your side.
[15:16.420 --> 15:17.840]  You've given your password to your side.
[15:17.980 --> 15:23.420]  You've created a certificate which confirms that you are indeed either Bob or acting with Bob's authorization.
[15:25.360 --> 15:30.820]  And that doesn't necessarily mean that you can use... you can have this machine open a port for you.
[15:30.920 --> 15:34.420]  You can't necessarily use all the privileges of this resource.
[15:34.420 --> 15:43.680]  Similarly, resources cannot... resources, just because they are resources, doesn't necessarily mean that they are authorized to return information to you.
[15:43.820 --> 15:46.640]  They're not necessarily the correct person.
[15:46.820 --> 15:55.180]  I mean, this is all very string... all very meticulously done with mutual authentication between all the parties involved.
[15:57.120 --> 16:04.020]  And it's important to note, we'll come into circumstances later, where we talk about the different... where we talk about authorization.
[16:04.180 --> 16:06.420]  And we're not necessarily talking about authentication there.
[16:06.600 --> 16:08.320]  It's something you have to keep in mind.
[16:08.480 --> 16:10.020]  They're very different ideas.
[16:10.880 --> 16:14.260]  And they work together here, unlike where they work in normal schemes.
[16:15.000 --> 16:19.640]  Yeah, the certificate part is a lot of the basis for that.
[16:19.800 --> 16:29.820]  And that's... basically, you can... in terms of layers of trust, you know, using certificates gives you a pretty good starting layer for doing a lot of other things.
[16:32.800 --> 16:34.280]  I think we got this.
[16:35.180 --> 16:37.220]  Oh, I had an example here also.
[16:37.520 --> 16:39.440]  So, as I mentioned... you can't really see it that way.
[16:39.580 --> 16:59.720]  I mentioned the idea of enterprise-level computation sharing, you know, even doing something like business-to-business, you know, electronic data interchange, that sort of thing, could be enabled reasonably by this virtual organization instead of doing something like a virtual private network or internal network.
[16:59.720 --> 17:02.060]  or other ways that you might decide to do it.
[17:02.620 --> 17:07.660]  Just an example of what could be one of these heterogeneous data flows.
[17:07.840 --> 17:12.220]  So, let's say you have a, you know, database server that's doing data selection.
[17:12.380 --> 17:14.680]  Then you have a super computer that's doing computation.
[17:14.900 --> 17:17.060]  You have a big graphics machine that's doing your visualization.
[17:17.420 --> 17:21.380]  Then you have some storage backend that's storing the output in various ways.
[17:21.660 --> 17:28.760]  Well, it's very reasonable to use, again, not that many places are doing it in the real world because the software is problematic.
[17:28.760 --> 17:30.500]  And a lot of things are problematic.
[17:30.820 --> 17:41.680]  But the model is a good model for having this whole virtual organization chain of processing happening and the type of thing that grid computing is meant to enable.
[17:44.540 --> 17:48.740]  Okay, we talked about VOs, talked about certificates authentication.
[17:50.240 --> 18:01.260]  Mentioning here, as Porkchop was saying, that basically when you're talking about authentication, a lot of times the authorization is something that you deal with separately.
[18:03.820 --> 18:07.700]  Also, the language is a little bit slippery here.
[18:07.900 --> 18:12.580]  Even when you hear the people that are like writing this software, they don't always use quite the same terms.
[18:12.780 --> 18:15.100]  But you have this thing called a service or a port type.
[18:16.060 --> 18:18.260]  And these actually have their own level of control.
[18:18.380 --> 18:20.760]  So, it's not like a system level of control or a user name.
[18:20.960 --> 18:24.640]  It's more like essentially a service level of control.
[18:29.000 --> 18:29.860]  Let's see.
[18:30.120 --> 18:41.540]  Yeah, and I mentioned things like file systems and so forth as a... and logging as a possible element of infrastructure.
[18:44.120 --> 18:44.840]  All right.
[18:45.460 --> 18:47.340]  Talked about Globus a little bit already actually.
[18:47.540 --> 18:48.940]  These are some of the changes in Globus.
[18:48.940 --> 18:52.340]  As Porkchop said, we know that 4.0 is coming.
[18:52.580 --> 18:54.900]  And what you have to understand is version 1 came out.
[18:55.040 --> 18:56.580]  Then version... and people did development.
[18:56.740 --> 18:57.400]  And there was conferences.
[18:57.420 --> 18:58.160]  And there were talks.
[18:58.300 --> 19:00.220]  And people were doing software and proof of concept.
[19:00.500 --> 19:05.860]  I'm actually a co-chair of one of the working groups in the global grid forum for grid information retrievals.
[19:05.880 --> 19:10.120]  We're like doing reference implementations and requirements documents and all that formal stuff.
[19:11.580 --> 19:14.440]  It was... I started this in the version 2, so...
[19:14.440 --> 19:17.000]  But so version 2 comes along and everything changes, right?
[19:17.000 --> 19:18.900]  Version 3 came along and everything changes.
[19:19.080 --> 19:22.700]  You know, like when I say everything, I mean like just none of your code is going to work anymore.
[19:22.960 --> 19:24.060]  You know, you won't be able to compile it.
[19:24.060 --> 19:24.900]  You won't be able to run it.
[19:25.700 --> 19:28.060]  Version 4 is going to be more of the same essentially.
[19:28.620 --> 19:32.720]  The advantage is moving from an environment with Globus.
[19:32.860 --> 19:33.880]  Let's take a look over here.
[19:34.340 --> 19:48.080]  With Globus being this huge monstrous package that's still compiling because it includes all this stuff which is in many cases, not all cases, replicating what you can get from grid... web services, Tomcat and others.
[19:49.580 --> 19:55.360]  They're going to be leaving a bunch of that stuff out and essentially layering over web services.
[19:55.540 --> 19:57.360]  And we have yet to see really how that's going to work.
[19:57.500 --> 20:00.200]  But the thing that's going to be really cool about it is it's going to...
[20:00.200 --> 20:02.880]  It should, anyway, shrink the code base considerably.
[20:03.420 --> 20:04.560]  And that's going to be nice.
[20:04.900 --> 20:08.440]  It will also allow you to do a lot of very, very interesting things.
[20:10.720 --> 20:15.640]  It's kind of hard to explain how unless you're really into the details.
[20:16.440 --> 20:22.060]  But the more and more I look at Globus, the more and more it seems to be made for web services.
[20:22.260 --> 20:24.420]  I mean, I don't know if this is something we've been planning for a while now.
[20:24.560 --> 20:26.400]  But they really seem to go together very well.
[20:26.960 --> 20:33.600]  And it's really unfortunate that you basically have to burn everything that you've done in order to go from 2 to 3 and 3 to 4.
[20:34.300 --> 20:37.780]  There are some people I know... Does TeraGrid use 3?
[20:38.220 --> 20:39.260]  They're 2.4.
[20:39.500 --> 20:41.060]  TeraGrid is 2.4 still.
[20:41.440 --> 20:44.460]  They'll probably be 2.4 for the next year or two.
[20:44.460 --> 20:45.200]  And then they'll jump to 4.
[20:45.480 --> 20:46.940]  They'll probably jump to 4, true.
[20:47.280 --> 20:52.220]  But in order to maintain usability, you have to stay at older versions.
[20:52.380 --> 20:55.400]  And this gets into what we're going to be talking about a little bit later.
[20:55.540 --> 21:03.300]  One of the biggest enemies to keeping your systems up to date and secure is the length of time that you're going to be installing this stuff for.
[21:03.680 --> 21:10.340]  Yeah, so it's like using an old version of your operating system or an old version of some application software that's a little supported.
[21:10.680 --> 21:11.800]  You know, there's people using it.
[21:11.940 --> 21:14.800]  But it's not where all the developers are paying attention, certainly.
[21:14.960 --> 21:17.920]  Just because you don't have 400 hours to upgrade.
[21:20.520 --> 21:21.720]  See, I left this little to-do.
[21:21.840 --> 21:22.680]  I already said these other things.
[21:22.900 --> 21:26.520]  But the idea here was there's a couple of papers on this.
[21:26.640 --> 21:29.260]  And I had a citation and didn't find the one I was looking for.
[21:29.260 --> 21:36.940]  But essentially, if you have 1,000 lines of code, you're going to have, statistically speaking, a bug or two in that 1,000 lines of code.
[21:37.100 --> 21:43.320]  And you get rid of that bug or two in those 1,000 lines of code by going and testing, basically.
[21:43.520 --> 21:45.240]  And then you get-you cut that in half.
[21:45.380 --> 21:47.680]  So if you have 100,000 lines of code, you have 100 bugs.
[21:47.720 --> 21:48.500]  You can get down to 50.
[21:48.600 --> 21:50.840]  You can get down to 25 by doing revisions, right?
[21:52.820 --> 21:53.540]  And testing.
[21:53.800 --> 22:04.140]  Well, so statistically, if you have a highly complex amount of code, you know, lines of code is just one way of counting it, but a whole lot of code, a whole lot of complexity.
[22:04.740 --> 22:06.340]  Statistically, you got bugs, right?
[22:06.460 --> 22:11.820]  And you can do revision after revision after revision, test after test after test, and statistically, you still got bugs.
[22:12.280 --> 22:15.100]  And again, chances are that some of those bugs are security related.
[22:15.500 --> 22:18.220]  One thing that's in their favor, actually, I linked to this.
[22:18.340 --> 22:19.440]  I'm not positive this is good.
[22:19.520 --> 22:20.480]  I'll check it before I upload.
[22:22.920 --> 22:25.240]  One thing that's in their favor is most of this is written in Java.
[22:25.800 --> 22:36.940]  So Java doesn't have nearly the problems of C or C++ with buffer overruns and unallocated memory and all that sort of stuff that's plagued so many applications.
[22:42.960 --> 22:46.240]  Actually, those were installation instructions you had available there, right?
[22:46.620 --> 22:48.700]  Yeah, I don't think that link works, but yeah.
[22:48.880 --> 22:53.220]  But it's interesting that you had to write installation instructions, because I also had to write installation instructions.
[22:53.580 --> 22:56.240]  And I put mine up on the web, so to help some people, but...
[22:56.240 --> 22:57.820]  I mean, I created it.
[22:57.820 --> 23:02.860]  I work for an undisclosed educational institution north of here.
[23:03.480 --> 23:06.100]  And we had to...
[23:06.720 --> 23:08.940]  There was a lot of documentation about Globus.
[23:09.480 --> 23:12.100]  Not all of it, very little of it is up to date.
[23:12.700 --> 23:21.000]  And very little of it is coherent in that it doesn't make sense when you read lots of the different documentation, because everybody uses different words to mean the same things.
[23:21.000 --> 23:27.860]  And some people have different versions of what reality is, depending on which version of Globus they're familiar with.
[23:28.280 --> 23:29.280]  You know the story.
[23:30.120 --> 23:33.120]  It's a huge cluster, you know what.
[23:35.260 --> 23:45.320]  The point I was getting to is that since nobody really has any great document, the closest thing to a great document I've ever seen, I don't know if you've ever seen one, but IBM has a red book.
[23:45.320 --> 23:45.840]  Oh, yeah, absolutely.
[23:45.840 --> 23:49.000]  If you go to redbooks.ibm.com, you can find...
[23:49.740 --> 23:51.360]  It's redbook.ibm.com.
[23:51.520 --> 23:52.200]  Those red books.
[23:52.420 --> 23:53.260]  Red books, yeah.
[23:53.440 --> 23:55.620]  I think it's red books, but it could be red book.
[23:55.760 --> 23:56.640]  Hey, what do you know?
[23:58.040 --> 24:03.320]  .ibm.com, and you can go search for Globus-related stuff, and you should come up with about six hits.
[24:04.020 --> 24:04.920]  It should be pretty obvious.
[24:05.140 --> 24:09.200]  There's one that's 85 pages, and there's one that's like 130 pages.
[24:10.380 --> 24:13.480]  And then there's a different version of that one that's 380, whatever.
[24:13.860 --> 24:17.960]  You can find a lot of different... and that's probably the best documentation that's out there.
[24:18.220 --> 24:31.560]  Unfortunately, when you're teaching a class, or actually, in my case, I'm supporting a class that's being taught by someone else, you can't really give them a 500-pound manual and expect students to be able to actually get through it.
[24:31.640 --> 24:35.620]  Well, I mean, you should be able to expect students to get through it, but, you know, this is reality here.
[24:36.100 --> 24:37.560]  No students really do that anymore.
[24:38.920 --> 24:55.660]  So I had to rewrite the documentation into an 18-page version, which was a step-by-step trying to explain, you know, and trying to give notes of what to do if this didn't work, trying to explain those concepts behind it, but you really can't do that in 18 pages.
[24:57.680 --> 25:07.620]  And for political reasons, that didn't get used, and the teacher wrote up a two-page version, which was just line, you know, type this, type that, type that.
[25:07.740 --> 25:09.640]  And it was just line after line of type this, type that.
[25:12.340 --> 25:18.440]  It's... the point I'm trying to make here is the documentation for the user level is absolutely hard.
[25:18.880 --> 25:22.120]  And if you want to write this stuff, you pretty much, you have to read.
[25:22.400 --> 25:24.740]  And correct me if I'm wrong, you have to pretty much read the source.
[25:25.880 --> 25:34.200]  Yeah, and study the documentation, get it on the mailing list, and, you know, sort of follow up on the documentation.
[25:34.200 --> 25:38.460]  Now, on the mailing list, how many questions would you say are posed a day on the mailing list?
[25:38.880 --> 25:39.640]  20 to 40.
[25:39.840 --> 25:40.500]  How many are answered?
[25:41.260 --> 25:50.340]  I... when I statistically looked, I didn't look and do like a formal analysis, but I was looking for answers to my questions, and I always found my questions, and I found my answer about one in five times.
[25:51.120 --> 25:51.860]  Sounds about right.
[25:51.940 --> 25:54.100]  I'd say 20 to 30 percent might be answered.
[25:55.100 --> 25:55.640]  Any questions?
[25:56.540 --> 25:56.900]  Yes.
[25:56.980 --> 25:57.640]  All the mailing list, yeah.
[25:57.720 --> 26:00.660]  I had a link in here that... that... I think it's... it's not quite right.
[26:00.800 --> 26:02.320]  It's unix.globus.org.
[26:02.500 --> 26:03.100]  They have...
[26:05.040 --> 26:05.760]  www-unix.
[26:05.760 --> 26:06.220]  www-unix.
[26:06.220 --> 26:06.940]  That's what it is, yeah.
[26:07.100 --> 26:08.560]  And so they have archives all the mailing lists.
[26:08.640 --> 26:15.300]  There's globus-devel, globus-announce, and a bunch of other ones for the sub products.
[26:15.300 --> 26:18.380]  There's something called Gram for authentication.
[26:18.780 --> 26:21.740]  Globus-java, globus-globus-python?
[26:22.060 --> 26:23.160]  Yeah, just everything.
[26:25.300 --> 26:33.960]  So anyway, but I mean, we're sticking with it because we see some of the promise, and we think it's really exciting technology.
[26:34.140 --> 26:40.940]  I mean, there's a lot of... there's a lot of really worthwhile things that have been talked about.
[26:40.940 --> 26:46.380]  Some of those have actually happened, and I'm pretty optimistic for the future.
[26:46.700 --> 26:48.780]  But boy, you know, it's a rough ride.
[26:50.380 --> 26:51.880]  I want to talk about the TeraGrid.
[26:52.060 --> 26:53.640]  I think I use it both ways.
[26:53.760 --> 26:56.300]  I can't remember if it's an uppercase G or a lowercase g in TeraGrid.
[26:56.800 --> 27:03.720]  This was the National Science Foundation-funded effort to do sort of a next-generation internet thing.
[27:03.720 --> 27:07.160]  So most people know history of the internet in the middle 80s.
[27:07.320 --> 27:13.780]  NSFnet was the network which sort of was like DARPAnet, the earlier network.
[27:14.260 --> 27:21.700]  And NSFnet linked the originally five National Science Foundation-sponsored supercomputing centers, NCSA and San Diego and Princeton and so forth.
[27:23.140 --> 27:27.320]  Pittsburgh and NCAR, actually.
[27:31.740 --> 27:34.200]  So what do you do after you've done that for a little while?
[27:34.400 --> 27:39.580]  Well, first you, in the early 2000s, you commodify the internet so the NSF isn't running the internet anymore.
[27:39.840 --> 27:42.300]  And then you think, well, what are we going to do with these supercomputing centers?
[27:42.840 --> 27:45.980]  And one of the answers was, well, let's do this next-generation stuff.
[27:46.120 --> 27:47.780]  Let's do really ultra-high-speed networking.
[27:48.180 --> 27:51.380]  Let's look at how we can share and do, like, metacomputing across the centers.
[27:51.560 --> 27:53.240]  So anyway, they have this thing called the TeraGrid.
[27:53.240 --> 27:58.820]  The TeraGrid is sort of overlapping with different places, different people, different projects.
[27:59.420 --> 28:00.560]  Globus is part of this.
[28:00.720 --> 28:05.940]  As I said, they're using version 2.4 on some of the big systems.
[28:06.700 --> 28:08.340]  And there are other projects.
[28:08.420 --> 28:09.740]  I think, yeah, I have a couple of links here.
[28:09.840 --> 28:13.520]  Something called NPACI and the National Middleware Initiative.
[28:13.820 --> 28:14.760]  A lot of different acronyms.
[28:15.860 --> 28:21.860]  Moderate amount of software, not a whole lot of sort of standards activity.
[28:21.860 --> 28:23.960]  It's more like we're researchers.
[28:24.200 --> 28:25.640]  We're going to do stuff, figure out how to do things.
[28:25.780 --> 28:31.880]  And then eventually that will maybe turn into whatever product standards merge into existing services and so forth.
[28:32.000 --> 28:38.900]  In general, researchers really only care enough about what you have to do in order to get their stuff done.
[28:39.000 --> 28:40.960]  They don't really care about anything past that.
[28:40.960 --> 28:45.240]  You find there's the ground shaking grid.
[28:45.600 --> 28:52.220]  There's this grid for computing effects of earthquakes on structures.
[28:52.900 --> 28:57.460]  And they call it a ground shaking application.
[28:59.320 --> 29:00.740]  So they have a sense of humor.
[29:00.900 --> 29:01.960]  But they're not...
[29:01.960 --> 29:05.180]  They wrote their little application and that's it.
[29:05.180 --> 29:07.820]  Those people are no longer concerned about what happened with Globus.
[29:07.920 --> 29:09.660]  They've had their bad experience and they've gone away.
[29:10.340 --> 29:12.620]  They're not really interested in continuing...
[29:12.620 --> 29:15.280]  I mean, this is of course a broad generalization.
[29:15.320 --> 29:18.560]  And I don't know these earthquake people specifically.
[29:18.920 --> 29:21.040]  But in general, all the...
[29:21.040 --> 29:24.860]  Everyone is just interested in getting their little piece of work done.
[29:25.200 --> 29:27.080]  And by the time it's over, it's over.
[29:27.080 --> 29:29.080]  And that's true in several ways.
[29:29.260 --> 29:31.540]  Because once your grant money is over, your grant is done.
[29:33.660 --> 29:35.800]  I have a list of some other stuff here as well.
[29:37.180 --> 29:38.920]  Let's just show you just for a moment.
[29:38.940 --> 29:40.120]  I have some screenshots.
[29:40.500 --> 29:41.400]  We're still compiling over there.
[29:41.940 --> 29:42.840]  Oh, that was a screenshot.
[29:42.980 --> 29:44.420]  Sorry, it looked like I was in my login window.
[29:45.160 --> 29:47.920]  So I'm a researcher at a university.
[29:48.200 --> 29:54.700]  And I have a login on the TeraGrid to do some of my data information retrieval research.
[29:54.700 --> 29:57.580]  So this is the NCSA TeraGrid.
[29:57.680 --> 30:04.420]  I think this is actually Mercury, which is somewhere in the top 10, top 500 supercomputers of the world.
[30:04.560 --> 30:07.800]  You know, in other words, one of the world's fastest supercomputers ranked in the top 10.
[30:09.000 --> 30:11.640]  I believe, as I said, I didn't...
[30:11.640 --> 30:15.160]  I meant to and then I didn't go back and look up and see which system this is actually going to.
[30:15.340 --> 30:22.400]  But I believe this is a 1500 node Dell I-10 EM2 system at the National Center for Supercomputing Applications in Urbana-Champaign, Illinois.
[30:22.820 --> 30:23.860]  So I have a user name there.
[30:24.320 --> 30:28.640]  And just for curiosity, no, it's totally legitimate.
[30:28.740 --> 30:29.640]  Like I said, we're not...
[30:30.320 --> 30:32.280]  I had nothing to do with all that hacking stuff.
[30:34.440 --> 30:36.180]  But I just thought these were some nice numbers.
[30:36.280 --> 30:38.400]  This is just a DF, you know, list of the file systems.
[30:38.560 --> 30:39.660]  This is the one down here.
[30:39.900 --> 30:49.260]  So on the GPFS, which is short for General Parallel File System, it's just a way of doing, basically, parallel data file systems, right?
[30:49.260 --> 30:52.880]  So parallel data reading and writing across a bunch of different systems on some sort of cluster.
[30:53.660 --> 30:55.300]  So that's the way they do some scratch space.
[30:55.440 --> 31:01.240]  So this is a 23 terabyte scratch drive, of which 13 terabytes are available.
[31:01.420 --> 31:07.400]  So most people haven't seen those types of numbers in your local, you know, laptop and stuff like that.
[31:07.480 --> 31:08.120]  So I thought that was nice.
[31:08.120 --> 31:17.980]  Also, they have, as you see here, I did, you name mine and say, basically they're taking a variation on SUSE Linux and doing a few different things.
[31:18.160 --> 31:23.920]  But it's a fairly, fairly vanilla operating system on, as I said, Itanium 2.
[31:24.080 --> 31:30.160]  So fairly vanilla, at least for 64-bit systems that they're building this on.
[31:30.160 --> 31:38.460]  And then the hard part, as we mentioned, is the, you know, who knows how long, how many hours it's going to take when a patch comes out.
[31:38.660 --> 31:46.900]  You know, when a new version of Globus comes out, when a patch is something like OpenSSL that might be linked into other things on your system comes out.
[31:46.900 --> 31:53.480]  And so the question is, you know, are you going to apply that patch and risk breaking things?
[31:54.640 --> 31:59.880]  Or are you going to not apply that patch and risk breaking things, but also risk being broken in 2?
[32:01.200 --> 32:02.320]  Oops, that wasn't what I wanted to do.
[32:04.960 --> 32:05.780]  This is the other one.
[32:05.920 --> 32:08.620]  So I have just this, this is the same system, the Mercury system.
[32:08.700 --> 32:12.140]  I have this little program I use to do memory allocation.
[32:12.140 --> 32:16.480]  Because when you do 64-bit programming, I don't know how many people have experience doing this.
[32:16.480 --> 32:20.420]  When you do 32-bit programming, you know, an int is 4 bytes and you're done.
[32:20.600 --> 32:25.700]  You know, so an int has a range, you know, unsigned from 0 to 4 billion, signed from negative 2 billion to positive 2 billion.
[32:25.840 --> 32:26.400]  It's like you're done.
[32:26.480 --> 32:27.120]  You know this.
[32:27.340 --> 32:30.960]  On a 64-bit environment, all of a sudden you have to look at compiler switches.
[32:31.320 --> 32:34.760]  You have to look at whether you're using a long or a long-long occasionally.
[32:36.420 --> 32:37.980]  Double precision, all these types of things.
[32:38.380 --> 32:41.980]  And so all of a sudden you go on one system and you say, how long is an int?
[32:42.060 --> 32:44.380]  And it says, you know, 8, or 4 bytes.
[32:44.380 --> 32:46.100]  And you go on another system and you say, how long is an int?
[32:46.180 --> 32:46.760]  And it says 8 bytes.
[32:47.080 --> 32:48.980]  And you don't necessarily know all the time what you're getting.
[32:49.160 --> 32:56.680]  So I have this cheapo little program, 10-line C program, that does memory allocation to see how much memory you can get before it craps out.
[32:56.840 --> 33:09.740]  So if I compile it and it craps out at 2 gig, you know, assuming I have, like, gigs and gigs of memory, if it stops running at 2 gig of memory allocated, then I say, oh, okay, something's wrong because I should be getting closer to 4 gig, right?
[33:09.740 --> 33:12.780]  It's like I'm using signed integers rather than unsigned integers.
[33:12.980 --> 33:14.100]  You know, just a little debugging technique.
[33:15.220 --> 33:16.240]  Is it called brute force?
[33:17.620 --> 33:18.200]  Brute force.
[33:18.240 --> 33:20.360]  I don't know if I would necessarily call it anything in particular.
[33:20.540 --> 33:22.200]  It's just called trial and error.
[33:22.740 --> 33:24.820]  You know, it's basically the program runs until it breaks.
[33:25.040 --> 33:27.140]  And where it breaks tells me how much I was able to get.
[33:27.400 --> 33:32.760]  So it's like if you had a car, an engine in your car, and you were driving until you couldn't drive any faster.
[33:33.140 --> 33:34.620]  You know, and then you'd know your top speed.
[33:37.440 --> 33:48.320]  So the 64-bit environment, so if you have 12 gig of memory on your machine with 64 bits, you should be able to address with one program essentially all of that 12 gig, right?
[33:48.340 --> 33:58.140]  If it's a 32-bit machine, a Xeon, you know, Athlon, all that stuff, then you're really going to be able to access within a single address space, in other words, within a single program, up to about 4 gig.
[33:58.140 --> 33:59.720]  And in practice, you don't get quite that much.
[34:01.180 --> 34:05.880]  Anyway, my point is on the Mercury system, I got 17 terabytes of RAM.
[34:10.020 --> 34:11.120]  And these are Calix.
[34:11.320 --> 34:11.960]  It's not Malix.
[34:12.160 --> 34:15.420]  In Linux systems, sometimes you can do a Malix, and it says success.
[34:15.800 --> 34:17.560]  And it turns out it was exaggerating.
[34:18.860 --> 34:20.980]  With Calix, it actually zeroes out the memory.
[34:21.100 --> 34:22.380]  So supposedly, this is trustworthy.
[34:22.440 --> 34:23.360]  I'm not sure.
[34:23.480 --> 34:25.740]  I didn't actually go back and try to write all those...
[34:27.240 --> 34:28.820]  write to all those spots in the array.
[34:30.260 --> 34:32.200]  This is the TeraGrid user news.
[34:33.440 --> 34:35.220]  This is their security incident.
[34:35.360 --> 34:36.000]  This is on the web.
[34:36.100 --> 34:38.520]  Like I said, no, I don't have any inside information.
[34:39.120 --> 34:41.560]  Actually, I have a teeny weeny little bit.
[34:41.680 --> 34:45.500]  But I'm not going to share it with you because that would be unfair to the fine folks at San Diego.
[34:47.020 --> 34:51.300]  But this was an announcement that said, gee, folks, we were broken into in a pretty bad way.
[34:51.680 --> 34:53.880]  And therefore, all user names...
[34:53.880 --> 34:54.460]  Not all user names.
[34:54.520 --> 34:56.980]  All passwords on all TeraGrid systems are changed.
[34:58.260 --> 35:04.080]  And by the way, the systems are all down right now while we figure out what's going on and what we can do about it.
[35:04.080 --> 35:06.760]  And when some of those systems came back up...
[35:07.680 --> 35:08.440]  Not sure.
[35:08.740 --> 35:09.540]  This is it.
[35:09.760 --> 35:11.620]  No, I think it was the one I just had.
[35:11.820 --> 35:12.480]  Let me go back.
[35:14.080 --> 35:27.880]  When the systems came back up, one of the things that happened was they said, oh, rather than having a user name and you log in via SSH, some systems are going to continue to let you log in by SSH.
[35:28.140 --> 35:30.280]  Other systems are going to require Kerberos.
[35:30.280 --> 35:36.860]  Other systems are going to require that you set up a certificate and send us your digital certificate and we'll do the signature.
[35:37.080 --> 35:38.860]  And then procedurally...
[35:38.860 --> 35:41.440]  Well, we had a little bit of a key signing, I think, here.
[35:41.720 --> 35:48.260]  Procedurally, if you want to accept someone's key, you know, in a public key encryption system, you can just do it, right?
[35:48.540 --> 35:58.040]  But what you're supposed to do, of course, is you pick up the phone or you go and you meet them face to face and you say, okay, I verified that this digital thing matches some physical, actual person.
[35:58.040 --> 36:00.360]  And so they're actually doing that in the TeraGrid.
[36:00.460 --> 36:07.780]  You know, I don't think they're sending around people to visit or saying you have to come and stop by our office at San Diego or actually it's Caltech that's doing it that way, I think.
[36:08.640 --> 36:10.000]  You know, they just phone you up on the phone.
[36:11.120 --> 36:16.880]  But anyway, so they made big changes to their authentication system on the TeraGrid.
[36:22.810 --> 36:25.970]  All right, let's talk about this security stuff finally.
[36:28.110 --> 36:30.630]  This is fairly general stuff.
[36:30.810 --> 36:46.650]  We do know and it's, you know, no secret how the TeraGrid systems got broken into, or at least not in great detail, but basically it was operating system, the underlying operating system on these computers had patches that had not been addressed.
[36:46.650 --> 36:55.450]  And the main reason we suspect, we don't know for a fact, that they weren't addressed is because of basically fear or because of maybe labor or cost.
[36:55.650 --> 37:07.250]  But doing the upgrade might have been more painful than not doing the upgrade and they made a decision that, or maybe they made a decision or maybe they didn't make a decision, but regardless the fact of not doing it backfired.
[37:11.180 --> 37:13.500]  All right, so we already talked about this first point a fair amount.
[37:13.640 --> 37:14.700]  Actually, the first and the second point.
[37:15.700 --> 37:17.900]  Code complexity means opportunity for bugs.
[37:18.160 --> 37:30.040]  The fact that you have to rebuild from source every time you have a problem and, you know, the answer, when you get an answer to those 20 or 30 percent from the Globus developers often is the effect that, well, we don't really understand.
[37:30.520 --> 37:42.180]  You know, we know there's some kind of relationship, but we don't have a way of really describing the relationship between these different packages that you're using and the underlying operating system and things like the version of Java that you're using and the version of Ant that you're using and so forth.
[37:43.560 --> 37:48.020]  This complexity leads to bugs and some of those bugs are security related.
[37:49.040 --> 37:50.380]  Rebuilding is painful.
[37:51.320 --> 37:57.720]  Next generation, doing transition to next generation is painful, but it also means that there's fewer developers for the older versions.
[37:59.140 --> 38:10.720]  Certificate servers, aside from the just practical difficulties of doing what I said, you know, doing this like call you up and read a digital signature and find out if the person that, you know, you have on the phone you can really trust matches the signature.
[38:12.200 --> 38:16.280]  People need to install a certificate server even on a test basis.
[38:16.440 --> 38:22.980]  If you want to just do like a little, you know, little mini installation of grid software for your own testing, you have to have a certificate server, right?
[38:23.140 --> 38:25.780]  And people botch certificate servers all the time.
[38:25.980 --> 38:42.880]  The software for doing it is, you have some different choices unless you want to pay money to one of the companies that does it, you're stuck with some solutions that maybe aren't ideal or maybe require a lot of reading or more critically require installation on a system which is rock solid secure.
[38:43.160 --> 38:46.660]  So if your certificate server is compromised, then, you know, you lost, right?
[38:48.980 --> 38:50.500]  OS weaknesses we talked about.
[38:50.720 --> 38:53.540]  And then you have the whole area of applications.
[38:53.780 --> 39:01.480]  So if I'm developing on the TeraGrid, right, you can run essentially any system, any program, I mean, that you want under Globus.
[39:01.480 --> 39:03.820]  You just have to do a little work to kind of grid enable that.
[39:05.100 --> 39:05.740]  Well, okay.
[39:05.940 --> 39:09.520]  So the developers then could introduce various types of vulnerabilities.
[39:10.400 --> 39:20.980]  And maybe these would be sort of the equivalent of local exploits because people might need to come in through the virtual organization and get sort of authorized to access a particular service.
[39:21.140 --> 39:24.840]  But as that service is weak, those weaknesses could yield who knows what.
[39:24.840 --> 39:31.340]  You know, maybe a denial of service, maybe even a local sort of a tunnel to a local exploit.
[39:33.780 --> 39:37.440]  And this, I think, is one of the biggest ones.
[39:37.940 --> 39:41.760]  What's... actually, I meant to ask you before, what's the certificate server that you're using?
[39:41.840 --> 39:46.600]  Because you have something that's closer to a production environment than what I have, even though it's not really production.
[39:46.700 --> 39:47.900]  I would not call it a production environment at all.
[39:47.900 --> 39:50.320]  Talk about your system, actually, because I don't want to run out of time.
[39:50.320 --> 40:03.800]  My systems are... well, I've got a whole bunch of just normal Pentium machines, Pentium 4s, ranging from 2.4 gig to 3 gig or so.
[40:05.720 --> 40:07.880]  There are probably 14 or so of those.
[40:08.320 --> 40:14.240]  Also thrown in the mix for a good measure, a mainframe or two, S390 from IBM.
[40:15.000 --> 40:20.400]  Right now, we're using a G5, Z990 specifically.
[40:21.140 --> 40:22.380]  Z900, excuse me.
[40:22.580 --> 40:24.160]  Z990 is something different.
[40:26.160 --> 40:33.600]  Putting all that together and making it work together is somewhat harder than you might expect.
[40:34.380 --> 40:36.800]  And part of that is actually because of network access.
[40:37.180 --> 40:38.900]  You have to leave some ports open.
[40:38.900 --> 40:40.600]  You have to allow some things to get through.
[40:40.600 --> 40:47.640]  And when you have a mainframe environment involved, you generally don't have them in the same collision domain, not in the same subnet, etc.
[40:47.960 --> 40:49.720]  So that's caused some interesting issues.
[40:50.080 --> 40:56.800]  We're also, in the future, linking up to other schools, other institutions who want to do basically the same thing.
[40:57.380 --> 41:01.280]  And the theory is if you can get 10 small guys together, you'll make one big guy.
[41:04.660 --> 41:07.360]  And you had asked about specifically...
[41:07.360 --> 41:08.260]  Certificate server.
[41:08.380 --> 41:09.100]  Certificate server.
[41:09.560 --> 41:10.460]  We're using OpenSSL.
[41:11.160 --> 41:11.360]  Right.
[41:12.100 --> 41:15.920]  With pretty much the standard CA.PL.
[41:16.320 --> 41:16.820]  Right, right.
[41:16.900 --> 41:17.820]  I'm using the same thing.
[41:17.960 --> 41:18.240]  Exactly.
[41:18.600 --> 41:20.040]  It's this basic thing.
[41:20.160 --> 41:21.460]  It comes with OpenSSL actually.
[41:21.760 --> 41:26.820]  It's just a Perl script which does all the actions that a certificate authority needs to do.
[41:26.820 --> 41:28.940]  So we just have one system.
[41:29.220 --> 41:31.180]  Or actually, it's a virtual machine of a system.
[41:31.420 --> 41:35.860]  Most of our machines are actually split up into a bunch of different machines just for testing.
[41:35.980 --> 41:37.600]  Because this is purely a testing environment.
[41:37.640 --> 41:39.320]  We don't run any production load as yet.
[41:42.420 --> 41:43.880]  It's divided up into...
[41:44.520 --> 41:46.880]  Some of them are divided into six machines or more.
[41:47.080 --> 41:53.800]  There's a bunch of five machine machines which is being used for the class I mentioned earlier.
[41:57.240 --> 42:03.220]  With that certificate server you collect all the certificate requests, sign them and distribute them again.
[42:03.400 --> 42:08.900]  And there's going to be some interesting problems when we actually link up with other institutions as to how we're going to manage all that.
[42:09.080 --> 42:19.180]  Whether our server is going to manage everybody's certificates or if we are just going to sort of figure out some way to mutually trust each other's servers and allow things to go that way.
[42:19.540 --> 42:21.520]  That brings up another interesting point.
[42:21.520 --> 42:26.740]  I don't know how many of you know how certificates work, how X.509 works.
[42:27.540 --> 42:29.080]  There's an idea of a...
[42:29.080 --> 42:30.680]  X.509.
[42:33.260 --> 42:37.460]  There's this idea of a CRL, a certificate revocation list.
[42:37.700 --> 42:55.340]  When a certificate is no longer good, whether this machine that had it is compromised or whether it's just not being used for any reason, you add the certificate name to a list of certificates which are no longer valid and should no longer be trusted.
[42:55.500 --> 42:57.360]  So if somebody comes up and says, hey, I'm this guy.
[42:57.480 --> 42:57.660]  Look.
[42:57.780 --> 42:58.260]  And they're right.
[42:58.440 --> 43:01.060]  You look at the CRL and find out if they're listed.
[43:03.440 --> 43:10.440]  So one of the big problems here is that Globus until 3.2 didn't support CRLs.
[43:11.620 --> 43:33.120]  So there's this giant glaring security problem or issue sort of arose out of, you know, once you figure out some way to add in CRL support on your own, you were sunk.
[43:36.660 --> 43:38.840]  Certificates are a big pain in the ass.
[43:40.940 --> 43:41.420]  There's...
[43:41.420 --> 43:45.600]  Some of you may have used them before for Apache or something like that.
[43:45.740 --> 43:47.180]  It's a lot like that.
[43:47.360 --> 43:52.260]  And also, all the certificates that you're issuing, they're used to sign other certificates.
[43:52.400 --> 43:53.840]  They're all code signing certificates.
[43:54.080 --> 43:55.100]  Your certificate...
[43:55.620 --> 43:59.740]  The certificate that I personally have is used to sign a proxy certificate.
[43:59.740 --> 44:05.060]  That the proxy acts in my place to allocate and deallocate resources somewhere on the grid.
[44:05.160 --> 44:09.440]  So that I don't have to sit there and say, yes, password, yes, password, yes.
[44:10.000 --> 44:13.580]  A different entity, which runs in your machine, has to do that for you.
[44:13.720 --> 44:16.820]  So you're signing another certificate and this generation is signing it.
[44:17.640 --> 44:19.600]  Resources are signing certificates all the time.
[44:19.740 --> 44:22.240]  Certificates are signing certificates are signing certificates are signing certificates.
[44:22.500 --> 44:28.780]  I think you go six or seven levels deep before from the CA to the deepest you can go is It's like seven levels.
[44:30.240 --> 44:31.180]  It's complex.
[44:32.260 --> 44:34.860]  If you didn't get that already, Globus is complex.
[44:35.340 --> 44:44.600]  As we said, the idea that you're running... basically people that are running these certificate servers maybe shouldn't be running certificate servers.
[44:45.320 --> 44:53.080]  Or maybe they're putting it on a system which really shouldn't have a certificate server on it because that system is, you know, whatever system that a lot of people could log into.
[44:53.220 --> 44:54.680]  And it's all these implementation mistakes.
[44:54.880 --> 44:56.520]  And it's not rocket science.
[44:56.520 --> 44:59.980]  I mean, people... you just go to a checklist somewhere and you know how to do these things properly.
[45:00.180 --> 45:01.300]  But you might not have the resource.
[45:01.440 --> 45:03.300]  You might be saying, oh, it's just kind of... we're just messing around.
[45:03.460 --> 45:04.200]  It's just kind of for research.
[45:04.440 --> 45:09.620]  And, of course, those are the types of things that historically have resulted in, you know, security problems.
[45:09.680 --> 45:10.980]  The TeraGrid is there for research.
[45:12.800 --> 45:14.680]  So we're just in our last couple of minutes here.
[45:14.900 --> 45:15.640]  A lot of growth.
[45:15.940 --> 45:17.200]  A lot more grid computing.
[45:17.360 --> 45:24.560]  I've been going to the Globus... not the Globus world, but the Global Grid Forum conferences for the last half dozen or so meetings.
[45:24.560 --> 45:26.080]  And they're all over the world.
[45:26.300 --> 45:27.980]  And they're pretty big conferences.
[45:28.200 --> 45:29.260]  A lot of corporate interests.
[45:29.340 --> 45:30.500]  A lot of corporate interests.
[45:31.560 --> 45:33.580]  And a lot of different standards groups.
[45:33.920 --> 45:35.640]  And software that's going on.
[45:35.760 --> 45:36.660]  So it's really very active.
[45:36.860 --> 45:39.320]  That's part of why... I mean, this is very real stuff.
[45:39.780 --> 45:44.140]  And yet, to some extent, we've sort of poked at Globus a fair amount.
[45:44.800 --> 45:46.480]  But that's not necessarily the only piece.
[45:46.620 --> 45:48.940]  But to some extent, it's a castle built on sand.
[45:49.380 --> 45:49.560]  Right?
[45:49.720 --> 45:59.720]  It's a very problematic... the foundation software for doing all the stuff you'd really like to do, and in some cases can do, with grid computing, is fairly problematic.
[46:05.480 --> 46:06.520]  Moving towards standards.
[46:06.760 --> 46:07.840]  Moving towards modification.
[46:10.460 --> 46:10.900]  Diversification.
[46:11.500 --> 46:15.840]  Looking at, like, you know, trying to get the software to run on embedded systems and so forth.
[46:17.280 --> 46:18.520]  And continued fragility.
[46:18.760 --> 46:20.300]  Because of all these changes.
[46:20.560 --> 46:24.680]  And because of all the, you know, the lack of, like, a set of regression tests for software installations.
[46:24.880 --> 46:26.840]  You know, if you could... if you install software...
[46:26.840 --> 46:29.180]  I mean, okay, you... everyone's done a Linux kernel install, right?
[46:29.380 --> 46:31.420]  And the 2.6, of course, they upgraded.
[46:31.480 --> 46:32.900]  So you don't even need to watch all that stuff.
[46:33.020 --> 46:34.340]  You don't need to do a make dep anymore.
[46:34.620 --> 46:34.920]  Wow.
[46:35.300 --> 46:36.240]  That was an improvement.
[46:37.580 --> 46:40.320]  So, with Globus, they don't even know dependencies.
[46:41.400 --> 46:41.760]  Right?
[46:41.760 --> 46:43.300]  So you need to recompile.
[46:43.740 --> 46:46.760]  And you have to, you know, you have to do, like, a make clean and a recompile almost every time.
[46:46.820 --> 46:48.300]  As we're seeing, this takes a while.
[46:49.720 --> 46:53.520]  Simple recommendations that, you know, are pretty common sense.
[46:54.520 --> 46:54.920]  Authentication.
[46:54.920 --> 47:01.260]  The notion of whether your systems with these potentially fragile services are exposed to the outside world.
[47:01.960 --> 47:03.700]  The notion that you have to be nimble.
[47:04.040 --> 47:06.080]  And just say, okay, there's a... whatever.
[47:06.240 --> 47:07.300]  There's a new OpenSSL.
[47:07.400 --> 47:10.100]  I'm just gonna have to go back and rebuild a whole bunch of stuff.
[47:10.420 --> 47:16.020]  And maybe follow good practices and have a, you know, development system and a... whatever production type of system.
[47:16.240 --> 47:17.580]  There was an interesting problem recently.
[47:17.820 --> 47:20.140]  As you mentioned, OpenSSL and having to recompile everything.
[47:21.620 --> 47:25.900]  OpenSSL, whatever current is, I, versus whatever the current is, E.
[47:26.520 --> 47:35.180]  Whatever the current is, I specifies email address as lowercase e, capital M, A-I-L, equals whatever.
[47:37.180 --> 47:45.380]  Same thing, lowercase e, you know, specifies capital E, mail, capital A, address, equals.
[47:45.660 --> 47:53.920]  And that, just that little tiny difference between if you have a different version on your CA versus a different version on your Globus clients, you're gonna have a problem.
[47:54.540 --> 47:55.940]  And little things like that.
[47:56.040 --> 47:57.760]  Little, little mess ups.
[47:58.080 --> 48:00.020]  Yeah, it's immensely complex.
[48:00.240 --> 48:04.380]  You think about, um, uh, one of the analogies that, like, um...
[48:05.180 --> 48:07.100]  I can't remember if this is a Schneier analogy or not.
[48:07.160 --> 48:09.300]  We're just in our last 30 or 40 seconds here.
[48:09.420 --> 48:13.740]  But, uh, it was thinking about complexity leading to being able to test, right?
[48:13.880 --> 48:24.320]  So, if you have a Boeing 747 with six million parts, um, and something's going wrong in this area of the plane, you say, okay, let's look at what connects there, and let's, you know, let's kind of focus in on that part of the plane.
[48:24.480 --> 48:26.600]  With software, you usually don't have that luxury, right?
[48:26.680 --> 48:38.080]  So you have, you have a package with a quarter million lines of code in, uh, you know, 80 or 90 independently developed sub-packages, which are gonna be built together, and you have problems.
[48:38.320 --> 48:40.480]  You don't even know where to start a lot of times.
[48:40.640 --> 48:41.760]  You know, you start on the mailing list, I guess.
[48:41.840 --> 48:45.740]  But you don't even know where to start with, uh, uh, trying to, trying to fix those errors.
[48:46.140 --> 48:54.720]  And that's really, um, uh, you know, a problem that, that, that, that, again, statistically, historically, by best practices in software development, you can't get around.
[48:54.940 --> 48:57.020]  You know, the only thing that you can do is tests.
[48:57.100 --> 49:06.420]  You have to have regression tests, you know, a suite of, of, uh, uh, uh, you know, tests, both for the general installation and also for your own particular applications, to have confidence.
[49:06.580 --> 49:07.160]  That's a lot of work.
[49:07.420 --> 49:10.160]  Um, I don't know, any closing thoughts?
[49:10.280 --> 49:11.680]  I think we're in the last few seconds here.
[49:11.680 --> 49:14.400]  Um, not ready for prime time, but soon?
[49:14.980 --> 49:16.260]  That sound about reasonable?
[49:16.580 --> 49:16.720]  Yeah, exactly.
[49:16.780 --> 49:17.540]  Absolutely, exactly, yeah.
[49:18.200 --> 49:20.380]  Um, and, and, and we're, I mean, we see promise.
[49:20.600 --> 49:21.400]  We definitely see promise.
[49:21.680 --> 49:24.540]  Uh, we'd be happy to get email and talk with you about this, uh, later.
[49:24.660 --> 49:29.000]  We, sorry, I'm sorry we ran out of time, uh, so we don't have, uh, really any question period today.
[49:29.580 --> 49:31.180]  But, uh, okay, we have one question.
[49:31.300 --> 49:32.120]  For more, where do you go?
[49:32.320 --> 49:35.880]  Okay, uh, Globus.org is a place for Globus Toolkit.
[49:35.880 --> 49:41.940]  Um, I had a link to, uh, uh, to, you can do TeraGrid.org for the national TeraGrid.
[49:42.400 --> 49:45.920]  Um, and PACI.org is the, uh, the place.
[49:46.180 --> 49:46.780]  Forget about it.
[49:46.840 --> 49:48.960]  Go to TeraGrid.org and they can link to these other places.
[49:49.160 --> 49:55.380]  And then Globus or GGF, which is Global Grid Forum, are the places that are doing these standards that I talked about.
[49:55.520 --> 50:01.060]  So TeraGrid.org for the NCSA and so forth stuff I talked about, which is one implementation, basically.
[50:01.060 --> 50:02.620]  Plus a bunch of other things.
[50:03.220 --> 50:06.980]  And then GGF.org or Globus.org and they link to each other.
[50:07.140 --> 50:10.080]  Which is this Globus Toolkit and the, uh, standards development.
[50:10.720 --> 50:11.780]  Okay, we better wrap it up.
[50:11.900 --> 50:15.120]  Thanks and, um, have fun with the grid.
[50:19.200 --> 50:19.940]  The slides.
[50:20.340 --> 50:21.700]  Yeah, I'm gonna, I'm gonna upload them.
[50:22.100 --> 50:22.620]  Thank you.