[00:00.000 --> 00:05.640]  Thank you so much for being here for our talk.
[00:05.900 --> 00:06.440]  I'm Ming.
[00:06.740 --> 00:07.560]  Hi, I'm Roy.
[00:07.900 --> 00:16.380]  And we're here to talk about this problem known as cybersecurity education.
[00:16.980 --> 00:39.920]  So I would like to ask everyone, by a show of hands here, it's hard to see anything, but how many of you here got interested in security, privacy, politics, you know, sometimes later, after, like, you know, when you were in school, when you were in your undergraduate days,
[00:40.000 --> 00:43.080]  how many of you got interested in the topic?
[00:44.580 --> 00:48.640]  It was supposed to be before or after the...
[00:48.640 --> 00:49.000]  High school.
[00:49.240 --> 00:50.360]  Whatever it was, grade school.
[00:50.660 --> 00:51.940]  Grade school, sure.
[00:52.240 --> 00:56.520]  Like, how many of you were interested during, like, the school period, like grade school?
[00:56.520 --> 00:57.540]  All right.
[00:57.680 --> 00:59.180]  A good number of you.
[00:59.320 --> 01:01.080]  A good number of you.
[01:01.300 --> 01:10.940]  But how many of you got interested in cybersecurity and security privacy after, like, in later years?
[01:11.320 --> 01:11.900]  Okay.
[01:12.100 --> 01:15.020]  A good number of hands, a good number of hands as well.
[01:15.920 --> 01:21.600]  So I want to tell you something that happened two years ago.
[01:21.600 --> 01:29.460]  So two years ago, I was taking a nap in Brooklyn at my aunt's house while hope was happening.
[01:30.300 --> 01:39.940]  And I woke up from my nap, turned on Twitter, and there was a tweet from Sam Bound that really got my attention.
[01:39.940 --> 01:42.620]  That really got my attention.
[01:42.860 --> 01:49.420]  How many of you here were at Sarah Zatko's talk at HOPE 10?
[01:51.460 --> 01:52.160]  Wow.
[01:52.460 --> 01:52.540]  Okay.
[01:52.600 --> 01:53.200]  A few of you.
[01:54.040 --> 01:58.380]  There's the tweet and this picture really got our attention.
[01:58.380 --> 02:13.440]  It was this graph here that she presented on, you know, where does security rank with regards to other subjects in computer science?
[02:14.360 --> 02:27.700]  The picture is a little dark, but I can show you that number one and number two at the very left, the highest bar, the highest emphasis, were on algorithms, native structures, and software design.
[02:28.200 --> 02:32.520]  Number one and number two topics of emphasis in a computer science program.
[02:33.260 --> 02:34.700]  Where is security?
[02:35.960 --> 02:38.640]  Ninth out of tenth topic.
[02:38.900 --> 02:41.900]  The tenth topic was compilers.
[02:45.780 --> 02:51.300]  This picture, this was the picture that really got us motivated.
[02:52.340 --> 02:58.600]  And I'm very, very thankful that Sarah presented her talk at HOPE 10.
[02:59.240 --> 03:24.000]  And I'm very grateful that she gave the talk because not only did it shed light on this problem, which we've always wanted to point fingers at, at why we are still experiencing the same problems as we have for so long, but it also confirmed or verified a lot of what we have thought and what we have believed in for so long.
[03:24.320 --> 03:37.140]  Which was, I'm not sure what's really, really coming out of computer science curriculars these days, whether it's at a two-year, four-year institution, not only here in this country, but elsewhere.
[03:37.140 --> 03:43.020]  So this was the motivation why we are here today and why we are doing this work.
[03:43.240 --> 03:47.560]  You can consider this as a continuation of Sarah's talk.
[03:47.900 --> 03:52.200]  And wherever you are, Sarah, we want to say thank you for your work.
[03:52.700 --> 03:58.340]  It's an acknowledgement and, you know, why we want to talk a lot more about it today.
[03:58.340 --> 04:04.920]  To help you understand, I'm sure how many people have heard that we're facing a crisis right now.
[04:05.280 --> 04:07.760]  A shortage in talent and cyber security.
[04:07.940 --> 04:09.300]  We've all heard that, okay.
[04:09.760 --> 04:13.380]  And why are we facing the same vulnerabilities that we have for decades?
[04:13.700 --> 04:19.440]  I mean, decades ago, you know, the vulnerability, SQL injection, cross-site scripting, buffer overflows, prevalent.
[04:19.730 --> 04:21.520]  Now, guess what?
[04:21.880 --> 04:23.480]  The same battles, okay.
[04:23.480 --> 04:31.360]  But not only are we talking about the education, we're not talking about the lack thereof of education, but also what else is missing?
[04:31.580 --> 04:35.360]  I mean, how big is this iceberg looking ahead?
[04:35.760 --> 04:38.080]  You know, or what else is missing?
[04:38.980 --> 04:52.000]  And just to let you know, there's a lot more missing than just informing students that, you know, vulnerabilities, you know, like the usual suspect of stuff.
[04:52.000 --> 04:58.790]  And we also want to have a discussion, a sustained discussion of what can we do to address these problems.
[05:00.680 --> 05:06.340]  Now, I'm going to talk about, you know, the key integration into most computer science courses.
[05:06.910 --> 05:16.790]  Again, we have to make sure that students think, you know, think about all the preconditions and variants and also kind of think like an attacker, right?
[05:16.790 --> 05:18.100]  It's very hard.
[05:18.500 --> 05:22.820]  It's a very way of a different way of thinking, thinking outside the box.
[05:23.000 --> 05:27.940]  But we also want to encourage students to think about security at the beginning of any project.
[05:27.940 --> 05:32.040]  Just like we have the software development lifecycle, OWASP Top 10.
[05:32.380 --> 05:34.900]  It's been... how many years has it been since it's been released?
[05:35.380 --> 05:35.820]  I don't know.
[05:35.940 --> 05:36.480]  A long time.
[05:36.580 --> 05:37.480]  I won't even say the number.
[05:37.480 --> 05:41.800]  And, you know, I think a lot of hands-on practice is required as well.
[05:41.920 --> 05:47.600]  Whether it be the different seesaws or different collegiate competitions as well.
[05:47.860 --> 05:53.040]  And we also want to... I think this is a point that is near dear to our hearts as well.
[05:53.120 --> 06:00.280]  Because a lot of people, a lot of students, especially professors, may not even know about the different opportunities in security.
[06:00.280 --> 06:01.820]  It's really sad.
[06:02.020 --> 06:07.240]  And quite frankly, I mean, as you will see, I mean, you will see evidence of students.
[06:07.500 --> 06:13.300]  There's a lot of students out there that just don't know that these opportunities exist.
[06:13.640 --> 06:17.300]  Including these types of conferences in get-together.
[06:17.500 --> 06:19.220]  They just don't know.
[06:23.380 --> 06:27.980]  And there's no excuse, right, to not integrate security into computer science courses.
[06:28.500 --> 06:31.440]  Especially systems and application-based courses.
[06:31.440 --> 06:36.620]  We want to inform students of the opportunities, especially security and privacy problems.
[06:36.840 --> 06:39.400]  And also, ask students to be good citizens.
[06:39.860 --> 06:42.460]  How many times have we seen in the news, right?
[06:42.600 --> 06:43.080]  Good, bad.
[06:43.640 --> 06:46.100]  And also, we want to encourage and challenge students.
[06:46.320 --> 06:47.780]  Develop the security mindset.
[06:48.140 --> 06:49.920]  Mindset of attacker, in particular.
[06:52.000 --> 06:55.960]  Next, we wanted to talk about, right, do not use any traditional teaching.
[06:56.160 --> 06:57.060]  I think we've seen it.
[06:57.540 --> 06:59.380]  Theoretical works for some.
[06:59.580 --> 07:00.940]  Works for many.
[07:00.940 --> 07:01.880]  Maybe not for all.
[07:02.120 --> 07:05.640]  I believe that... we believe that learning how to take tests isn't helping.
[07:06.060 --> 07:07.260]  You can memorize stuff.
[07:07.460 --> 07:11.400]  But what happens after, like, you know, 10 minutes, maybe you forget right away.
[07:11.740 --> 07:16.860]  You want to inform students about the mentorship, mentorship, and networking opportunities.
[07:17.280 --> 07:20.680]  For example, these conferences like these, like HOPE, for example.
[07:21.060 --> 07:25.640]  We want to also provide guidelines of lessons learned through presentations like this.
[07:26.280 --> 07:31.060]  In the next few slides, we'll also talk about more of these information as well.
[07:31.440 --> 07:34.760]  Also want to be able to work with younger students, kids of all ages.
[07:35.000 --> 07:35.540]  Doesn't matter.
[07:36.120 --> 07:39.140]  We'll show in the next few slides of some tweets as well.
[07:39.800 --> 07:50.080]  Well, I think one thing I also want to add, I mean, in terms of what we all can do, you know, all in this room that we can do, that would make a huge difference.
[07:51.160 --> 07:59.200]  You know, at the very basic level is to, in just the basic, inform, you know, people, you know, these, this is a real problem.
[07:59.400 --> 08:05.420]  Especially with now, everything is becoming connected to the Internet, and everything is getting, the network, everything is getting broken into.
[08:05.680 --> 08:06.780]  These problems are real.
[08:07.060 --> 08:10.960]  This is no longer, you know, you know, dreaming up such a fictitious scenario.
[08:11.540 --> 08:18.540]  You know, show them, you know, X, Y, you know, X, Y, and Z, these cases have happened over the last few weeks.
[08:18.540 --> 08:20.780]  I mean, every day, some new scenarios happen.
[08:21.060 --> 08:23.360]  So, what if you're also in computer science?
[08:23.500 --> 08:35.020]  Now, one of the things I also do, because I advise a lot of students, and a lot, there's a lot of students that go and study computer science, two year, four year institution.
[08:35.520 --> 08:38.380]  And, of course, when you graduate, you have to eat, right?
[08:38.580 --> 08:43.420]  You have to find a, you know, you have to determine what's next in your career.
[08:43.420 --> 08:53.700]  And it is absolutely amazing that the number of students that I work with, where they think they're going to go after they graduate.
[08:53.920 --> 09:02.420]  You know what places everyone, a lot of computer science majors think they, where they want to go, or where they think they practically have to go, which is their only option to go?
[09:04.080 --> 09:07.820]  Google, Facebook, Microsoft, the usual suspect of places.
[09:08.660 --> 09:17.520]  You know Paul Graham, he posted a tweet, a little while back that said in one institution, thou shall not name him...
[09:17.520 --> 09:19.160]  He's not, he didn't mention what it was.
[09:19.440 --> 09:20.620]  It was mind blowing.
[09:21.000 --> 09:27.200]  That large percentage of computer science graduates, only went to five, five places after graduation.
[09:28.280 --> 09:30.100]  The usual suspect of places.
[09:30.300 --> 09:31.740]  It should not be a surprise.
[09:32.360 --> 09:33.980]  So, think about that.
[09:34.100 --> 09:35.140]  Let that sink in.
[09:35.560 --> 09:40.300]  A lot of the students think that is the road to go after you do a computer science degree.
[09:40.600 --> 09:42.640]  The usual suspect of places.
[09:43.140 --> 09:46.060]  Now, this was from Carl Landwire.
[09:47.200 --> 09:51.460]  This was from New England Security Day of April 2016.
[09:52.820 --> 09:56.800]  And he gave a talk at a security education panel.
[09:56.800 --> 10:00.820]  And this will also shed light on what's also really, really missing.
[10:01.040 --> 10:04.260]  The tip of the real big iceberg ahead.
[10:04.540 --> 10:15.400]  Is we need to train future leaders, not just computer science students, to understand the principles behind public policy issues relating to cybersecurity and privacy.
[10:15.400 --> 10:22.560]  So that when they rise to decision-making position, they will be less susceptible to making poor choices.
[10:24.100 --> 10:31.120]  Have you all heard of people that have made a lot of poor choices with regards to security, privacy, and politics?
[10:31.820 --> 10:32.600]  Oh, yes.
[10:33.040 --> 10:41.960]  And it should not be a surprise that Carl teaches a course specifically on this matter called Cybersecurity for the Future United States President.
[10:44.320 --> 10:45.960]  That's the course that he teaches.
[10:46.340 --> 10:50.880]  And there is a big reason why, because that is the big iceberg that we are approaching.
[10:51.720 --> 11:05.360]  Brian Levine at the UMass Amherst has mentioned, you know, CSE students need exposure to outside topics such as finance, criminology, democracy, and censorship, because sometimes the best solution is not technical.
[11:05.720 --> 11:16.080]  And I see this quite a bit in a lot of computer science curricula, is that, you know, you do a lot of theory, theory, theory, programming, programming, programming, theory, theory, programming, programming, programming.
[11:16.080 --> 11:22.340]  How many of you here have gone to a four-year computer science institution and felt that's the way that this program was structured?
[11:22.600 --> 11:26.960]  A lot of theory, theory, theory, programming, programming, programming, theory, theory, theory.
[11:27.180 --> 11:30.900]  And so there's lack of that global view of technology.
[11:32.500 --> 11:34.120]  I'm not going to say anything here.
[11:34.120 --> 11:42.820]  So here are some folks in the information security community you probably know of, you know.
[11:43.060 --> 11:45.080]  And this has been a chatter, you know.
[11:45.240 --> 11:50.380]  This was, I believe, since Sarah's talk.
[11:57.050 --> 12:03.970]  And if these are the things that are coming out of, you know, that are on social media, you know, it's still a problem.
[12:06.150 --> 12:10.190]  This is from a colleague of mine who's a software engineer at...
[12:10.190 --> 12:10.970]  Sorry.
[12:11.250 --> 12:14.030]  He's a technical manager of software engineering at TripAdvisor.
[12:14.250 --> 12:21.670]  And he mentioned, look, he told my class that if you want to work at TripAdvisor, you got to know basic security.
[12:22.710 --> 12:30.410]  Some real basic security principles, such as SQL injection, buffer overflow, cross-site scripting.
[12:31.110 --> 12:41.890]  And one of the things I really liked that he did was, you know, there's a lot of students that are not coming out with security, as you'll see in the next slide, which is for your eyes only.
[12:43.030 --> 12:52.390]  And I think what is a good idea is, look, if you're a manager and you're looking to hire, not a bad thing to do is, look, if you don't know basic security, don't come work here.
[12:52.850 --> 12:54.030]  That's not a bad idea.
[12:54.030 --> 13:01.090]  And that's what he definitely, that's what he tried to aim at to scare a lot of my students, to pay attention to these issues.
[13:01.690 --> 13:08.090]  Now, this is a private conversation between myself and Chris Ng, who's the director of research at Veracode.
[13:08.650 --> 13:20.650]  And how this conversation started was, Chris asked me if I knew of students who would be interested in a website, in a content developer job for Veracode.
[13:21.230 --> 13:21.870]  That was it.
[13:22.650 --> 13:29.250]  And so I sent the news around, I asked a bunch of students if you want to work, and they said no, and I got no response.
[13:29.730 --> 13:33.450]  Three weeks later, I asked Chris, hey, Chris, what's the status on the job?
[13:33.510 --> 13:37.070]  And he said, yeah, we found someone, but we filled it internally.
[13:37.330 --> 13:39.170]  And so this led to this discussion.
[13:39.450 --> 13:44.030]  And so Chris said, no security at all is what's coming out of a lot of places.
[13:44.030 --> 13:51.270]  I don't hire entry-level people based on my team, the teams...
[13:51.270 --> 14:02.350]  But here's the thing that really bothered me, is that not only did he say no security is what's coming out of most places, never mind security, some have no idea how the web or the Internet works.
[14:02.570 --> 14:09.210]  So much is abstracted from developers, especially in a typical CS curriculum.
[14:11.290 --> 14:14.470]  And this is a conversation I had with a former classmate and I.
[14:14.570 --> 14:16.310]  He's now an assistant professor at Bucknell.
[14:16.570 --> 14:22.890]  And he quietly nodded his head, he's like, mm-hmm, we're fighting this problem internally as well.
[14:25.210 --> 14:29.770]  Not only if we talk about security, but also basic problems solving with code.
[14:31.210 --> 14:34.370]  And this is a recurring information, right, a recurring issue.
[14:34.370 --> 14:38.450]  We had dark reading this year, earlier this year, talk about top U.S.
[14:38.530 --> 14:43.110]  undergraduate computer science programs skip cybersecurity classes as well.
[14:43.450 --> 14:47.870]  And the one thing we wanted to mention was this was another one from April 2016.
[14:48.850 --> 14:50.270]  The slides will be available too.
[14:50.430 --> 14:55.970]  But some of the key findings on the bottom, right, it says, I won't read them all, but none of the top 10 U.S.
[14:56.110 --> 14:58.810]  computer science programs require a cybersecurity course for graduation.
[14:58.810 --> 15:05.750]  In fact, three of the 10 top university programs don't even offer an elective course in cybersecurity.
[15:06.310 --> 15:06.950]  Yep.
[15:07.830 --> 15:11.030]  Don't want to mention any names, University of Michigan ranks, you know.
[15:11.190 --> 15:14.050]  But we're going to actually provide all the slides.
[15:16.050 --> 15:17.330]  More stuff here.
[15:17.490 --> 15:17.850]  Okay.
[15:18.350 --> 15:19.090]  All right.
[15:19.170 --> 15:26.590]  This was from the assistant director of the National Science Foundation, Dr. James Kuros.
[15:27.170 --> 15:32.110]  Now, he gave this presentation in December of 2015, which I was lucky enough to go to.
[15:32.210 --> 15:35.330]  It was an innovation partnership meeting in Cambridge.
[15:35.890 --> 15:46.810]  And he even mentioned, I mean, he even mentioned, all right, we want to grow cybersecurity, especially with the security industry, we want to grow it.
[15:46.970 --> 15:48.790]  I mean, this is a huge problem.
[15:49.110 --> 15:49.750]  All right.
[15:49.790 --> 15:52.490]  We have a lot of opportunities out here.
[15:52.490 --> 15:56.690]  But it seems like there is a lot of opportunities and little takers.
[15:57.350 --> 16:01.670]  And one of the issues that he mentioned was his education challenge.
[16:01.930 --> 16:06.410]  Is that, look, you know, changing undergrad curriculum is difficult.
[16:08.330 --> 16:13.850]  But it's also, another thing is it's also targeted a lot of masters and advanced students.
[16:14.090 --> 16:15.730]  But how do we do about that?
[16:15.970 --> 16:22.970]  But even one of the things he mentioned was perhaps integration within the curriculum is the way to go, although it is not easy.
[16:23.150 --> 16:25.670]  So even the NSF has recognized this problem.
[16:26.910 --> 16:28.830]  Also on a high school level as well.
[16:29.530 --> 16:30.030]  Okay.
[16:30.310 --> 16:30.510]  Okay.
[16:31.130 --> 16:31.590]  Sure.
[16:31.930 --> 16:32.890]  Just show of hands.
[16:33.010 --> 16:37.250]  How many people know about the CSTA, the Computer Science Teachers Association?
[16:37.650 --> 16:38.150]  Okay.
[16:38.430 --> 16:38.850]  Oh, awesome.
[16:39.150 --> 16:39.250]  Great.
[16:39.690 --> 16:53.070]  But yeah, basically, they are creating this year, they created kind of like a measurement for people that teach or actually teachers that teach computer science and give information about, you know, measure how well they do.
[16:53.250 --> 16:54.830]  You can get more information there.
[16:55.270 --> 16:57.270]  This we actually just found out as well.
[16:57.270 --> 17:03.490]  NIST actually has NICE, which is the National Initiative for Cybersecurity Education.
[17:04.250 --> 17:11.890]  This actually, they have kind of like an annual presentation and we're trying to get a hold of that too, get in there too.
[17:12.910 --> 17:18.610]  So, if you go to a four-year institution and study computer science, most go through accreditation.
[17:19.610 --> 17:25.810]  And most go through the accreditation board known as ABET, the Accreditation Board for Engineering and Technology.
[17:25.810 --> 17:33.990]  Just to let you know, anyone here know of someone who has a PE, a professional engineer license?
[17:34.790 --> 17:35.570]  All right.
[17:35.730 --> 17:49.830]  So, just to let you know, if you are to get the professional engineer license, like to say, if you're a civil engineer, one of the requirements is you must go to an institution which is accredited by ABET.
[17:50.690 --> 17:55.230]  Now, computer science generally, a lot of computer science programs go through an ABET accreditation.
[17:55.870 --> 18:02.050]  And is there a line in the accreditation that has to do with security, privacy, and politics?
[18:02.270 --> 18:02.910]  Well, yes.
[18:02.910 --> 18:14.390]  One of the important outcomes that they have listed is an understanding of professional ethics, ethical, legal, security, and societal issues and responsibility.
[18:14.870 --> 18:19.250]  That's listed black and white as one of the requirements for accreditation.
[18:20.110 --> 18:27.290]  And a few years ago, in 2011, Tufts went through a re-accreditation and they dinged Tufts.
[18:27.710 --> 18:38.750]  One of the issues that they dinged us was, you know, we generally did well, but one of the issues that they noted was, you know, the understanding of security and legal and societal issues.
[18:38.750 --> 18:44.190]  You know, there is emphasis on it, but not at the required curriculum.
[18:44.410 --> 18:47.270]  You know, the required courses for the major.
[18:47.990 --> 18:48.450]  Okay?
[18:48.890 --> 18:59.310]  So, of course, which leads to now, alright, are there any resources for better integrating, you know, for integrating security into the existing curriculum?
[19:00.210 --> 19:05.390]  One thing is, you know, as part of this talk, we have done that.
[19:05.390 --> 19:10.350]  We have created a collection of examples and also pictures of success stories.
[19:10.730 --> 19:24.890]  At the end of this slide, you know, how you integrate security into courses such as web development, healthcare, senior capstone, game development, and even programming languages.
[19:25.230 --> 19:37.770]  Now, there's a long list of examples and success stories, but we don't want to put that in because that will detract from the real crux of the matter, which we really want to get through.
[19:39.290 --> 19:54.470]  And these are some examples of, again, the main message here is these are some of the examples that we've done, some assignments and papers that we've done at both of our universities, and we actually want your feedback as well.
[19:54.470 --> 20:08.710]  We've seen capture the flag games, code reviews, conduct risk assessments, information security policies, research papers, peer reviews, mock interviews, personal engagement projects, also attending the local conferences, different conferences as well, like HOPE as well.
[20:10.950 --> 20:16.370]  So Sriram is a very prominent professor in programming language at Brown University.
[20:16.590 --> 20:23.910]  And he had a bone to pick at the panel back in April about what really was missing in security education in computer science.
[20:24.090 --> 20:31.810]  He said, we don't spend enough in the curriculum doing code review or reviewing code despite its clear importance in the security field.
[20:32.370 --> 20:36.990]  Even he mentioned that he had such a bone to pick on this issue.
[20:38.150 --> 20:54.450]  So I want to let this sink in as well too, especially when it comes to CTFs and real practical assignments and how valuable, how they really, really can help someone get along, get along, really, they really help a long way.
[20:55.050 --> 21:00.470]  So, see, Facebook recently open-sourced a CTF engine.
[21:00.630 --> 21:01.570]  What CTF?
[21:01.710 --> 21:02.830]  Capture the flags.
[21:03.650 --> 21:08.410]  An engine to create capture the flag games, to hone your security skills.
[21:09.010 --> 21:11.950]  And this was in the official blog post.
[21:11.950 --> 21:12.770]  Okay?
[21:13.030 --> 21:23.090]  Not only do CTFs have the ability to teach you more technical skills than you'll get in your average computer science program, you know, they will also help you break in the security industry.
[21:23.330 --> 21:26.130]  But this line, this paragraph, bothered me.
[21:26.430 --> 21:35.450]  It was, when I joined Facebook's security team last year, it was large and part because of the experience I got through capture the, you know, CTF, capture the flag game.
[21:35.850 --> 21:50.730]  When I was a student at the University of Michigan, the TA from my security class introduced me to CTFs, which exposed me to a fun and practical side of security that I didn't get in class.
[21:51.190 --> 21:54.490]  This is black and white, word for word from that blog post.
[21:56.330 --> 21:59.830]  And that, and stuff like this, can really help someone go a long way.
[22:01.130 --> 22:07.970]  So, as we mentioned, you know, the education problem in just CS is only the tip of the iceberg.
[22:08.490 --> 22:09.710]  It's a lot more.
[22:16.400 --> 22:17.520]  Anyone see this?
[22:18.080 --> 22:20.620]  Did anyone recall this article?
[22:23.160 --> 22:25.660]  Programmers, stop calling yourselves engineers.
[22:27.860 --> 22:30.060]  And let's talk about the title of engineer.
[22:31.120 --> 22:38.620]  And the bottom line is from that article is the title of engineer has been cheapened by the tech industry.
[22:39.200 --> 22:42.940]  And I can say, and I hear, I think we're all guilty of that.
[22:43.320 --> 22:45.620]  We all have some part to blame on this.
[22:45.860 --> 22:46.340]  Okay?
[22:46.540 --> 22:48.740]  And of course, they go through a litany of cases.
[22:49.340 --> 22:50.000]  All right?
[22:50.000 --> 22:55.300]  So, the real further security education gaps are number one, okay?
[22:55.740 --> 23:01.840]  The awareness that what we're building in software is critical infrastructure, life and death.
[23:02.940 --> 23:07.740]  And second of all, is the problem with policy and leadership.
[23:08.640 --> 23:11.000]  I mean, this is what's really... you talk about bad laws.
[23:11.320 --> 23:14.240]  That's what's coming out, that's what's coming out these days.
[23:16.900 --> 23:22.020]  So, anyone was... was anyone here at Matt Blaze and Sandy Clark's talk yesterday?
[23:22.620 --> 23:27.980]  That was fantastic because it carried into today and to our talk.
[23:28.300 --> 23:37.720]  Because one of the things that Matt and Sandy mentioned at the end of their talk was, the real hard problem is the security and the legal question.
[23:37.920 --> 23:40.520]  That's the real hard problem, okay?
[23:40.520 --> 23:48.620]  And there's very... there's almost hardly any exposure to people who are even studying computer science in that realm.
[23:49.580 --> 23:50.140]  Okay?
[23:50.600 --> 23:56.980]  And perhaps, you know, maybe this is food for thought, a deep thought.
[23:57.180 --> 23:58.860]  Do we need to go down to this model?
[23:59.700 --> 24:00.840]  Anyone know what this is?
[24:01.800 --> 24:06.800]  I got this from my brother, who's a student, who's actually studying civil and environmental engineering.
[24:06.960 --> 24:08.120]  Does anyone know what that is?
[24:08.580 --> 24:09.380]  Civil and Environmental Engineering?
[24:09.380 --> 24:10.100]  Yeah.
[24:10.800 --> 24:12.820]  Do we have anything like that or treated...
[24:12.820 --> 24:16.400]  Is there anything like that treated seriously when we build software?
[24:17.060 --> 24:18.860]  Treated seriously is the big...
[24:18.860 --> 24:22.020]  There's the ISC2 code of ethics that we can all laugh at together.
[24:22.160 --> 24:23.240]  Okay, okay.
[24:23.560 --> 24:24.500]  Say no more.
[24:24.800 --> 24:25.640]  Say no more.
[24:26.460 --> 24:27.060]  Okay?
[24:27.740 --> 24:29.420]  And there is actually...
[24:29.420 --> 24:29.700]  Do you...
[24:29.700 --> 24:30.840]  If you don't believe this...
[24:30.840 --> 24:32.260]  My brother sent me this.
[24:32.680 --> 24:35.440]  There's actually a PE exam for software engineers.
[24:35.820 --> 24:37.080]  Does anyone have one here?
[24:37.760 --> 24:38.980]  I didn't know there was one.
[24:39.120 --> 24:39.400]  I didn't know there was one.
[24:39.580 --> 24:40.000]  There was one here.
[24:40.620 --> 24:41.180]  All right.
[24:41.320 --> 24:42.940]  Effective April 2013.
[24:43.240 --> 24:45.240]  Software Engineering Specification.
[24:45.380 --> 24:46.540]  This is by NCEES.
[24:47.120 --> 24:48.320]  Same people who give this...
[24:48.320 --> 24:48.800]  What is that?
[24:49.060 --> 24:49.500]  Huh?
[24:49.920 --> 24:51.140]  What is NCEES?
[24:51.140 --> 24:52.180]  Uh...
[24:52.180 --> 24:52.960]  This was...
[24:52.960 --> 24:54.140]  I forget...
[24:54.140 --> 24:55.000]  Uh...
[24:55.000 --> 24:56.420]  They're the same group that give the PE.
[24:59.200 --> 25:01.300]  So, you may be interested...
[25:01.300 --> 25:03.120]  How are we addressing this gap?
[25:03.660 --> 25:05.420]  So, one thing is...
[25:05.420 --> 25:05.900]  I'm...
[25:05.900 --> 25:07.280]  Well, I'm at Tufts.
[25:07.560 --> 25:09.140]  And one...
[25:09.700 --> 25:15.200]  You know, Tufts is famous for international relations, political science, and active citizenship.
[25:15.920 --> 25:19.460]  And recently, computer science became the largest major in the whole university.
[25:22.280 --> 25:22.800]  Um...
[25:22.800 --> 25:23.720]  You know...
[25:23.720 --> 25:26.120]  So, how are we tackling this problem?
[25:26.440 --> 25:34.240]  Because, as you can see already, not only do we have a technical problem in computer science, but we also have a policy problem as well, too.
[25:35.180 --> 25:42.420]  So, we're in the process of hiring a bridge professor that actually works for two groups.
[25:44.400 --> 25:48.180]  That's in computer science and also the Fletcher School of Law and Diplomacy.
[25:48.340 --> 25:52.240]  And this position is called Cybersecurity and Policy Bridge Professorship.
[25:52.400 --> 25:55.460]  So, it's a professorship that spans multiple departments.
[25:55.780 --> 26:00.300]  So, we're actually opening that position up, I think, within the next few weeks.
[26:00.300 --> 26:06.320]  So, if anyone is on the job market, you know, for a bridge professorship, let us know.
[26:06.400 --> 26:06.860]  Let me know.
[26:07.760 --> 26:08.840]  Number two.
[26:09.160 --> 26:17.420]  In spring of 2017, computer science and political science will create a joint course on cybersecurity and cyber warfare.
[26:18.660 --> 26:28.700]  Now, the target of this course is going to be undergrads in both computer science and also political science.
[26:28.700 --> 26:37.060]  And the goal of the program, number one, to get the technical folks an understanding of the policy and the legal side of things.
[26:37.160 --> 26:38.960]  Because that's currently almost non-existent.
[26:39.520 --> 26:44.860]  And the policy and the policy folks an understanding of the technical issues.
[26:44.860 --> 26:48.220]  And that, clearly, that's hardly happening.
[26:48.840 --> 26:49.160]  Okay?
[26:49.400 --> 26:53.020]  But we want to target the undergrads and get them informed.
[26:53.540 --> 26:54.280]  Why undergrads?
[26:54.340 --> 26:56.940]  Some people are saying, yeah, but we already informed master's students.
[26:57.060 --> 26:59.020]  We already have master's programs all over the place.
[26:59.160 --> 27:02.360]  But by the time you inform master's students, they're too late.
[27:02.780 --> 27:06.020]  Because there's a lot of firms out there, a lot of the companies, they like to hire undergrads.
[27:06.020 --> 27:09.140]  Like, they're paying, like, 100K right out of school.
[27:09.380 --> 27:12.300]  And then we can start to think about the certificate and master's program.
[27:13.360 --> 27:19.160]  This is TBD, or to be announced, because I'm part-time, unlike Ming, who's full-time.
[27:19.620 --> 27:22.080]  But, again, there's a lot of issues.
[27:22.720 --> 27:35.020]  Open up healthcare information security courses so that it's available to all students rather than, you know, have an elective instead of just be part of a, you know, degree program, for example.
[27:35.020 --> 27:40.760]  And also development of a new course to incorporate student projects, work directly with students throughout the course.
[27:41.020 --> 27:49.300]  And just some other things, too, including certification programs and integration with additional departments that are outside of computer science as well.
[27:50.460 --> 27:57.700]  One of the huge advocates of this, you know, of this, you know, of this whole grand schema problem is Weld Punt.
[27:58.380 --> 28:02.100]  So Weld has, you know, he's very passionate on this topic.
[28:02.260 --> 28:08.220]  And he even mentioned, I just wish that first-year computer science student can actually, well, walk through an exploit.
[28:09.140 --> 28:09.700]  You know?
[28:10.700 --> 28:11.740]  See it, Tacoma Bridge.
[28:12.140 --> 28:14.760]  Does anyone, have anyone ever seen that video of the Tacoma Bridge?
[28:14.880 --> 28:15.460]  That fail?
[28:16.160 --> 28:16.480]  All right?
[28:16.840 --> 28:19.920]  Do we have anything like that remotely close in computer science?
[28:19.920 --> 28:20.740]  Right now?
[28:20.920 --> 28:21.500]  I don't...
[28:21.500 --> 28:21.720]  Huh?
[28:22.160 --> 28:22.960]  Home Depot.
[28:23.420 --> 28:23.940]  Yeah.
[28:25.120 --> 28:25.880]  Home Depot.
[28:26.260 --> 28:26.520]  Sony.
[28:27.460 --> 28:27.980]  Yeah.
[28:28.860 --> 28:29.220]  Okay.
[28:29.680 --> 28:32.560]  Maybe we can walk through something, one of those situations.
[28:32.820 --> 28:33.420]  Ashley Madison?
[28:33.640 --> 28:34.080]  Yeah.
[28:34.600 --> 28:41.980]  We can walk them through one of those situations and see what happened when you just don't even think of, when you don't consider security in your design.
[28:41.980 --> 28:42.960]  Okay?
[28:45.100 --> 28:55.940]  Chris Wysopal, I mean, and Chris Ng also mentioned, you know, the talent gap is, there's a lot of changes that need to be made, including textbooks and professors.
[28:57.120 --> 28:58.520]  More of the conversation.
[29:00.920 --> 29:02.460]  This one just speaks for itself.
[29:02.760 --> 29:07.740]  By the time, you raise security in master's programs, graduate programs, it's too late.
[29:10.120 --> 29:15.260]  So, the whole problem of the security education problem is this.
[29:15.740 --> 29:19.540]  How many of you here, the analogy I love to use is this.
[29:19.740 --> 29:28.440]  How many of you here, when you were young, I hope they still have it these days, in like grade school or middle school, they had something called shop classes.
[29:28.940 --> 29:31.320]  How many of you, or how many went through shop class?
[29:32.320 --> 29:33.060]  All right.
[29:33.060 --> 29:41.320]  And when you went through shop class, or what they called tech ed back in the day, what was one of the first lessons that you got?
[29:41.800 --> 29:43.000]  Safety lessons.
[29:43.260 --> 29:43.940]  All right.
[29:44.180 --> 30:02.940]  Before you operated the chainsaw, and the power saw, and the sanders, and all the power equipment in the shop, you know, didn't you have to go through, didn't you have to go through a training on how to wear something like safety goggles, like glass safety goggles,
[30:03.520 --> 30:05.100]  gloves, you know?
[30:05.640 --> 30:13.920]  And do we have any of that right now in computer science and in the technical field?
[30:28.910 --> 30:37.130]  There is a fundamental difference here, which is, if you do something wrong in shop class, you get your fingers chopped off.
[30:37.370 --> 30:42.130]  If you do something wrong in computers, it's other people's fingers that get chopped off,
[30:49.230 --> 30:54.650]  but now I want to actually ask a, because, by the way, that's all that we have.
[30:54.830 --> 30:56.010]  Like, we're done with the talk.
[30:56.170 --> 30:58.250]  I mean, the rest of it is, thank you.
[30:58.370 --> 31:00.150]  The rest of it is a collection of examples.
[31:04.110 --> 31:04.710]  Thank you.
[31:04.910 --> 31:05.610]  Thank you.
[31:05.770 --> 31:09.010]  And we have a list of, a collection of examples.
[31:09.010 --> 31:11.870]  Like, we can run through them, like, data structures.
[31:12.230 --> 31:13.010]  You can talk about that.
[31:13.010 --> 31:13.730]  You can talk about that.
[31:13.730 --> 31:13.910]  Yeah.
[31:14.090 --> 31:14.430]  Okay.
[31:14.590 --> 31:16.090]  We're just going to go down.
[31:16.690 --> 31:17.470]  Right up.
[31:18.530 --> 31:22.030]  But you know about other people's fingers being chopped off.
[31:23.130 --> 31:25.470]  Yeah, that's been generally true.
[31:26.570 --> 31:30.390]  And it looks like it's going to be continually true for a long time.
[31:30.650 --> 31:34.350]  Because let's talk about a thing that is emerging right now.
[31:35.290 --> 31:36.370]  Self-driving cars.
[31:38.150 --> 31:41.830]  Not only will you get, probably, fingers chopped off and heads lobbed off.
[31:41.930 --> 31:43.310]  You know, you're going to get a loss of life, too.
[31:45.510 --> 31:47.210]  That's a hard problem to think about.
[31:48.910 --> 31:50.630]  Now we're entering that territory.
[31:50.630 --> 31:53.350]  If you don't think about those implications.
[31:55.190 --> 31:56.830]  It's like the Cherokee attack.
[31:57.130 --> 32:03.910]  The 2014 Cherokee is certainly, you know, somebody else could attack your system because they didn't take security seriously.
[32:04.410 --> 32:07.010]  And it's still an ongoing issue right now.
[32:07.210 --> 32:08.050]  I mean, I can't turn...
[32:08.050 --> 32:11.130]  I mean, I turned on my Twitter feed today and I think something about GM.
[32:11.610 --> 32:14.650]  You know, something about GM mentioning about something that they're going to do.
[32:15.950 --> 32:18.230]  So, I think now is a good time.
[32:18.810 --> 32:19.250]  Yeah.
[32:19.590 --> 32:23.790]  We want to open this up to a nice, sustained discussion and conversation.
[32:25.070 --> 32:26.690]  How are we going to do this?
[32:30.050 --> 32:31.930]  So, I think we're sold on the problem.
[32:32.770 --> 32:34.650]  It was suggested that we had...
[32:34.650 --> 32:36.670]  There was a conversation, actually, at B-Sides Connecticut recently.
[32:37.030 --> 32:47.070]  And it was suggested that another part of the problem, which is even separate than the security piece, is that the computer science degrees now are more of a vocational school.
[32:47.070 --> 32:56.710]  They're more just teaching people how to code in some of the high-level languages that are used commonly, rather than really teaching them engineering, teaching them the fundamentals, teaching them assembler, whatever.
[32:57.310 --> 33:02.490]  And so, you know, I guess my question to you is, first of all, is that true?
[33:02.490 --> 33:08.170]  You're far closer to it than I am, that there are other things that are missing from the curriculum.
[33:08.910 --> 33:24.270]  And then, you know, if we believe that's true, if we believe that we want to make this something that is much more commonly part of the syllabus, not just for Komsky, but for kind of the broader range of degrees, like you were saying with Karl-Lanware, what are you recommending?
[33:25.950 --> 33:31.390]  I mean, I think all of us here would like to play a part, one way or another, in changing this.
[33:31.390 --> 33:41.810]  But I think those of us who are not familiar with academia in the U.S., are not familiar with kind of how change can be made there, would love to know how we can help.
[33:44.170 --> 33:45.730]  And you had a hand up?
[33:46.010 --> 33:53.810]  Yeah, I basically think that, you know, a lot of things we need to do is stop, like, some people tend to attack others who don't know as much as them.
[33:53.850 --> 34:04.230]  And if we take them and embrace them and try to mentor them, and we do these in our workplaces, and our educational environments, I believe we can start eliminating this problem and bring people forward instead of down.
[34:10.930 --> 34:23.530]  So, and that's why I feel the same exact way, too, is we all have an obligation, because I think we can take... because computer science everywhere now is not only exploding in population.
[34:24.830 --> 34:30.390]  Computer science, I can tell you right now, across the country has a huge problem of oversubscribed courses.
[34:30.730 --> 34:35.170]  And there's so little man...so little power, faculty members.
[34:35.410 --> 34:38.130]  So, it's only so many faculty members that take on stuff.
[34:39.690 --> 34:47.830]  And so, there's a lot of things like mentorship that goes out the window, that actually goes up, that goes out, goes away because of how busy people are.
[34:49.490 --> 34:56.830]  And to also tack on is that there can be a lot of red tape in academia.
[34:59.550 --> 35:17.630]  So, how this problem has been solved by a lot of places like Google and Microsoft is that they're running their own institutions inside of the company to offer courses like in cyber security or in big data.
[35:17.830 --> 35:26.430]  Because they know that a lot of students walking out of those four-year degrees and paying a lot of money from them, they're not...paying a lot of money for them.
[35:26.830 --> 35:32.850]  They're not getting the right skills necessary to tackle not only the problems of today, but also tomorrow.
[35:35.070 --> 35:39.110]  You know, change happens very slowly in a four-year institution.
[35:39.110 --> 35:47.830]  And it's gotten to a point where a lot of people have actually thought the whole curriculum is living on borrowed time.
[35:49.370 --> 35:51.490]  There's a lot more students now studying this.
[35:51.610 --> 35:58.250]  There's a huge push for people who code, as you can see, not only at a college level, but also, you know, at a younger generation level.
[35:59.110 --> 36:01.450]  Change happens very, very slowly at the top.
[36:01.590 --> 36:05.590]  Usually they say in terms of academic and academia land.
[36:05.590 --> 36:11.570]  I mean, I was talking to a former classmate of mine who's now at the ACLU, and I had to break it to him.
[36:11.650 --> 36:17.470]  I hate to break it to you, but, you know, a lot of stuff, a lot of the courses that you and I went through, they're still taught the same way.
[36:17.970 --> 36:20.330]  The intro course is still taught in C and C++.
[36:20.690 --> 36:23.290]  It's the same exact curriculum that you and I took it 15 years ago.
[36:25.430 --> 36:27.030]  Yeah, and it's C and C++.
[36:30.050 --> 36:34.610]  And so, it's going to be... hold on.
[36:35.090 --> 36:42.810]  One thing to add, I mean, I couldn't go and vent about this issue, but a lot of it, you know, the whole curriculum is very different.
[36:43.150 --> 36:46.790]  And it hasn't kept up with the times, quite frankly.
[36:47.450 --> 36:52.130]  And, you know, like Ming was saying, right, we need more people from the industry.
[36:52.130 --> 36:54.650]  Think about all the colleges, they're looking for professors.
[36:54.990 --> 36:57.710]  So, anyone who wants to submit a resume, let us know.
[36:57.870 --> 37:00.830]  We need more people from the industry to teach as well.
[37:01.150 --> 37:03.390]  Teach in your organizations, right, learn.
[37:04.010 --> 37:07.890]  We need people, more people participating, all these different committees.
[37:08.470 --> 37:12.610]  Committees, advisory groups, different conferences as well.
[37:15.170 --> 37:17.710]  Do you seem to have the understanding where you...
[37:17.710 --> 37:21.410]  Do you get permission for your new course to be in the Tusks open courseware?
[37:23.250 --> 37:25.070]  The new course education is policy?
[37:25.270 --> 37:25.590]  Yes.
[37:25.790 --> 37:27.650]  Is it going to be part of MIT open courseware?
[37:28.510 --> 37:30.790]  As of right now, good question.
[37:31.570 --> 37:32.710]  I guess a great thought.
[37:33.670 --> 37:35.370]  I like that a lot.
[37:35.470 --> 37:36.910]  And we have not considered it.
[37:36.910 --> 37:40.350]  And because, in fact, you bring up another thing.
[37:41.010 --> 37:42.810]  Is Tufts in terms of...
[37:42.810 --> 37:46.110]  Because, you know, I'm actually teaching an online course this summer.
[37:46.270 --> 37:48.230]  And it's one of the very, very few.
[37:48.450 --> 37:52.010]  And there's a lot of schools that don't embrace online learning.
[37:53.210 --> 37:54.450]  For business reasons.
[37:55.930 --> 37:57.070]  It's a good thought.
[37:58.310 --> 37:58.790]  Anyone?
[37:59.210 --> 37:59.710]  Excuse me.
[38:00.330 --> 38:01.510]  So, I...
[38:04.130 --> 38:06.750]  So, I come in and out of education.
[38:06.830 --> 38:10.610]  I usually teach tech certifications, not universities.
[38:10.910 --> 38:13.830]  I have department chaired at a university before, though.
[38:14.650 --> 38:15.930]  I see two problems.
[38:16.110 --> 38:18.710]  One, like, especially with you guys, you're talking about security.
[38:19.310 --> 38:21.970]  Still, what, 75%, 80% teach Java?
[38:22.170 --> 38:23.190]  I mean, Java security?
[38:23.550 --> 38:24.270]  I mean, come on.
[38:24.550 --> 38:26.750]  So, you have that.
[38:26.930 --> 38:27.930]  So, that's a mixed message.
[38:27.930 --> 38:33.110]  And then the other thing is, is that, you know, I'm an electrical engineer by background and a physicist.
[38:34.590 --> 38:42.870]  And, you know, the way computer science is taught is not like any other engineering discipline.
[38:44.590 --> 38:45.390]  And so...
[38:46.630 --> 38:47.530]  Well, it could be.
[38:47.730 --> 38:49.050]  But, you know, I mean...
[38:49.050 --> 38:57.410]  So, I think the biggest impediment, and this is why I come, I go back to the industry, is the comp side department seems to...
[38:57.410 --> 38:58.790]  Like, you guys circle your wagons.
[38:58.970 --> 39:00.370]  Like, you ask for industry people?
[39:00.670 --> 39:01.430]  You don't...
[39:01.430 --> 39:02.910]  Comp side departments don't want me.
[39:03.030 --> 39:04.310]  I've been hacking since I was 12.
[39:05.070 --> 39:11.130]  But, because I don't have an MS and CS, you know, that you're like...
[39:11.130 --> 39:13.730]  You know, and I'm like, okay, well, go get a real job.
[39:14.210 --> 39:15.850]  And you're not going to get hired.
[39:16.330 --> 39:19.350]  You know, or you'll get hired only by, like, Google, who needs that 5%.
[39:19.350 --> 39:20.490]  I mean, you know.
[39:21.350 --> 39:23.710]  So, like, you know, you guys...
[39:23.710 --> 39:25.210]  I understand the point.
[39:25.410 --> 39:27.650]  I just think that you're focused on the wrong issue.
[39:28.230 --> 39:31.910]  You know, like, if you do want more inclusiveness, I'm all for that.
[39:32.110 --> 39:33.450]  But we're going to tell you, drop Java.
[39:33.970 --> 39:36.910]  You know, we're going to tell you, like, teach languages that people know.
[39:36.910 --> 39:40.770]  We're going to teach you closed-loop engineering, which is, like, fundamental.
[39:41.010 --> 39:42.290]  And that includes security.
[39:43.790 --> 39:44.350]  So...
[39:49.300 --> 39:49.980]  Thank you.
[39:50.160 --> 39:50.240]  Right.
[39:50.420 --> 39:52.140]  So, also, by trade.
[39:52.560 --> 39:54.560]  So, it's not just security.
[39:56.200 --> 39:57.640]  I've done research.
[39:57.840 --> 40:00.020]  I've taught at Columbia once in a while.
[40:00.400 --> 40:01.640]  I work at Google.
[40:01.920 --> 40:03.780]  And we see...
[40:03.780 --> 40:06.240]  And already the people who get hired are...
[40:06.240 --> 40:10.000]  You can imagine a very, very, you know, thin slice of the top.
[40:10.000 --> 40:12.780]  And even those kids...
[40:12.780 --> 40:14.900]  Some kids are fresh out of school.
[40:15.740 --> 40:22.700]  Not only do they know about security, they don't know elementary software engineering principles that they learn on the job.
[40:23.420 --> 40:26.480]  So, it's not just security that has this problem.
[40:31.410 --> 40:32.990]  I've got something to say.
[40:33.570 --> 40:36.830]  I would like to say, and...
[40:38.230 --> 40:41.030]  I see it because I've managed people before...
[40:41.030 --> 40:43.930]  I've managed someone before, and we see this all the time.
[40:44.630 --> 40:45.910]  I would like to say...
[40:45.910 --> 40:47.110]  I mean, you now...
[40:47.110 --> 40:53.910]  I hear so many stories from friends and friends that they say, I give a student a problem, they fall flat in their face.
[40:54.210 --> 40:58.890]  It happens more often, and I think a number of us actually can identify with that.
[41:01.890 --> 41:05.150]  You know, one of the things I have to say is that...
[41:06.430 --> 41:08.830]  This is a very controversial remark I'm going to make.
[41:09.010 --> 41:14.910]  And some people need to understand, you know, you got to look really, really hard.
[41:14.950 --> 41:16.170]  Why do you want to go to college?
[41:16.470 --> 41:19.090]  And some things in college, they're not going to teach you.
[41:19.410 --> 41:23.330]  For some people, it's probably not the right way to go as well.
[41:23.330 --> 41:31.130]  I mean, I see a lot of people who go pay $50,000, $70,000, think that will be the way that will get you the next thing in life.
[41:31.350 --> 41:33.690]  And now, that's no longer true.
[41:35.290 --> 41:36.430]  That's no longer true.
[41:37.390 --> 41:43.170]  When they heard SQL injection, buffer overflow, cross-site scripting, you got to know this shit or you don't get a job.
[41:43.390 --> 41:44.750]  What was the students' reaction?
[41:47.050 --> 41:47.770]  They just...
[41:47.770 --> 41:49.250]  Oh, no, no, no, no, no, it gets better.
[41:50.010 --> 41:51.550]  We have never heard of this before.
[41:51.550 --> 41:53.830]  Well, that's part of it is, but here's the thing.
[41:54.470 --> 41:56.610]  Granted, they're juniors and sophomores.
[41:57.110 --> 41:57.670]  So, I was...
[41:57.670 --> 42:03.210]  So, one of the outcomes on my web course is cross-site scripting and injection attacks.
[42:03.390 --> 42:04.190]  So, they have seen it.
[42:04.550 --> 42:06.170]  They have been exposed to it.
[42:06.550 --> 42:07.830]  They have actually had to do risk.
[42:07.930 --> 42:10.830]  They have to do a security analysis, you know, in writing.
[42:11.210 --> 42:12.850]  Like, in writing piece.
[42:13.630 --> 42:14.830]  And they find...
[42:14.830 --> 42:15.870]  And most...
[42:15.870 --> 42:17.850]  And one thing that I don't like...
[42:17.850 --> 42:21.410]  I don't like to say much about myself, and they have really built this big interest.
[42:21.850 --> 42:22.250]  Insecurity.
[42:23.250 --> 42:23.650]  And...
[42:24.390 --> 42:27.010]  And also, it's not only the courses...
[42:27.010 --> 42:28.370]  The course is like 80 students.
[42:28.610 --> 42:33.030]  And also, the male-female ratio is also pretty good as well, too, which is fantastic.
[42:36.010 --> 42:38.330]  And, you know, and most of them are scared.
[42:38.510 --> 42:42.850]  But sometimes, that fear factor helps to say, okay, what am I really missing in my learning?
[42:43.210 --> 42:44.390]  And that can be a good thing.
[42:44.390 --> 42:46.250]  Now, what about the Java problem?
[42:46.450 --> 42:47.470]  I think, Brent...
[42:47.470 --> 42:49.110]  Did you have a quick comment on that?
[42:49.450 --> 42:49.710]  Yeah.
[42:49.930 --> 42:51.530]  It feeds into a larger question I have.
[42:51.790 --> 42:51.930]  Yeah.
[42:52.110 --> 42:52.910]  So, okay.
[42:52.990 --> 42:53.870]  It's like, I...
[42:53.870 --> 43:00.530]  My computer science curriculum, I have two degrees from an ABED accredited school in computer science that are part of a school of engineering.
[43:01.710 --> 43:02.110]  They...
[43:02.110 --> 43:04.450]  It doesn't matter.
[43:04.630 --> 43:05.790]  It's a programming language.
[43:05.870 --> 43:09.290]  And if you graduate from an ABED school, you'll have four or five of the time you're done.
[43:09.290 --> 43:12.790]  The question I have is, how do we push forward rigor?
[43:13.450 --> 43:19.770]  Because security analysis is just defect analysis, which every other engineering discipline already does, just that we don't do it very well.
[43:20.190 --> 43:25.790]  And I've never even heard of the PE for computer science until today, and...
[43:25.790 --> 43:27.290]  Well, I saw in software development, yeah.
[43:27.390 --> 43:28.050]  But close enough.
[43:28.190 --> 43:28.430]  Exactly.
[43:28.610 --> 43:34.890]  And I have an actual ABED degree, and I could, in theory, take it, except that it's impossible to work under an ABED accredited...
[43:34.890 --> 43:36.870]  ...under a PE, because there aren't any...
[43:36.870 --> 43:37.570]  ...under a PE that comes high.
[43:37.970 --> 43:41.810]  So, how do we push forward rigor if we can't actually get a PE in real life?
[43:42.210 --> 43:42.930]  And if we...
[43:42.930 --> 43:48.070]  And nobody other program wants to do it, and instead we spend time yelling at it, why are you using the tool I don't like?
[43:48.450 --> 43:49.770]  You should use the tool that I like.
[43:50.030 --> 43:51.290]  Yeah, you should use Emacs.
[43:51.950 --> 43:52.390]  Or...
[43:54.490 --> 43:56.390]  I would say, bam...
[43:56.390 --> 43:58.090]  And that's how religious force starts.
[43:58.570 --> 44:00.010]  Okay, I think in...
[44:00.010 --> 44:06.670]  One thing is, about the programming language issue, in which I've heard many people have said...
[44:06.670 --> 44:10.330]  I've actually gotten emails from employees like, why aren't you teaching Java?
[44:10.770 --> 44:11.770]  Which we can't hire...
[44:12.110 --> 44:18.490]  You know, we actually had two candidates, one student from your school, another student from my school.
[44:18.710 --> 44:22.310]  They were both as sharp, but the only reason why we hired the other person is because they had Java.
[44:23.430 --> 44:23.870]  But...
[44:23.870 --> 44:24.410]  That's the other...
[44:24.410 --> 44:26.650]  On the flip side is, I...
[44:26.650 --> 44:27.770]  It shouldn't matter.
[44:28.410 --> 44:29.510]  I mean, you should know...
[44:29.510 --> 44:32.330]  I mean, especially if you do computer science, you should know n number of languages.
[44:32.330 --> 44:33.610]  N, number of languages.
[44:33.730 --> 44:38.610]  The more important things are like a big idea, such as like first class functions, you know, type systems.
[44:38.770 --> 44:40.210]  I think those are the important keys.
[44:40.330 --> 44:42.190]  If you can understand that, you can learn any language.
[44:43.710 --> 44:45.350]  That can, that can...
[44:45.350 --> 44:46.230]  But I think...
[44:46.230 --> 44:49.790]  But however, it does raise a question...
[44:49.790 --> 44:52.470]  It does raise bigger questions.
[44:54.590 --> 44:58.970]  You know, what should be taught or what isn't right now?
[44:59.870 --> 45:01.470]  And how do we test rigor?
[45:01.470 --> 45:03.210]  And I don't think we got a good answer for that.
[45:03.490 --> 45:05.370]  And this is one of the reasons why we're in this room.
[45:08.930 --> 45:12.530]  But rigor in computer science is not the same as rigor in other engineering disciplines.
[45:12.850 --> 45:15.670]  I majored in both electrical engineering and computer science.
[45:15.950 --> 45:17.550]  And, you know, I now...
[45:17.550 --> 45:18.210]  I work for Google.
[45:18.430 --> 45:19.830]  I have TA'd at the graduate level.
[45:19.950 --> 45:21.090]  I've seen all of these things.
[45:21.090 --> 45:31.370]  And the rigor that we need in programming or in computer science generally, or certainly in cryptography, which is what I do, is nothing even...
[45:31.990 --> 45:38.190]  It's only remotely, if at all, similar to the rigor that I was taught in electrical engineering.
[45:38.190 --> 45:42.530]  So, I think trying to approach this like it's engineering...
[45:42.530 --> 45:44.870]  Like, even calling it software engineering, I think, is a mistake.
[45:45.050 --> 45:47.070]  We should really be looking at a different approach.
[45:47.170 --> 45:48.310]  We've started calling ourselves...
[45:48.310 --> 45:48.930]  Yeah.
[45:49.870 --> 45:50.690]  We got rid of it.
[45:50.710 --> 45:51.690]  Can you explain how it's different?
[45:51.830 --> 45:53.550]  Because it seems to me to be exactly the same.
[45:53.870 --> 45:57.550]  Okay, so I'm gonna make a really controversial statement, and I'm probably gonna get booed for this.
[45:58.130 --> 46:03.210]  In my view, my professional view, computer science is really a field of applied math.
[46:04.390 --> 46:12.090]  And, you know, and if you look at things like formal methods, if you look at things like databases, like relational algebra, et cetera, all of it is just math.
[46:12.610 --> 46:20.250]  And if we were to approach this the way math departments approach things, we'd be a lot less productive, but we'd also be a whole lot more rigorous.
[46:21.310 --> 46:23.150]  So, that would be my answer to your question.
[46:24.670 --> 46:25.550]  Can we undo...
[46:26.330 --> 46:26.830]  Can we what?
[46:26.830 --> 46:28.050]  What's the opposite of bullying?
[46:28.210 --> 46:28.570]  Cherokee.
[46:31.230 --> 46:37.270]  You know, and also, just to add on to it, we'll also want to say, you know, the...
[46:37.270 --> 46:39.090]  I mean, here's the one thing which I...
[46:39.090 --> 46:39.910]  Now, here's that...
[46:39.910 --> 46:44.850]  I know that at some institutions, it's also very theory-driven as well, too, to your point.
[46:45.510 --> 46:51.370]  And that's absolutely the wrong way to take it, because I always have stressed the tool chain is also as critical.
[46:51.870 --> 46:53.330]  So, you gotta know that as well.
[46:53.610 --> 46:55.110]  You gotta educate really broadly.
[46:55.110 --> 46:57.010]  But that's why there should be a compilers course.
[46:57.230 --> 47:00.690]  That was one of my biggest complaints at my undergrad and graduate institution.
[47:00.890 --> 47:02.530]  Nobody was required to take a compilers course.
[47:02.910 --> 47:03.370]  Thank you.
[47:03.370 --> 47:09.950]  I have a question at the beginning when you asked when people got interested in this stuff.
[47:10.350 --> 47:13.870]  A lot of people raised their hand for high school and grade school.
[47:13.870 --> 47:19.890]  I have an opportunity to teach at the junior high, high school level.
[47:20.790 --> 47:23.470]  First, I don't have a curriculum.
[47:24.110 --> 47:34.630]  And second, I don't know quite how to present security outside the idea of you're teaching our kids how to be hackers.
[47:37.330 --> 47:38.650]  Ideas would be welcome.
[47:39.150 --> 47:39.570]  Okay.
[47:40.150 --> 47:42.130]  So, I am...
[47:42.130 --> 47:44.630]  First of all, thank you for bringing that up.
[47:45.570 --> 47:47.730]  So, I can tell you something I did back in January.
[47:47.730 --> 47:51.490]  So, I got invited to Girls Act, a former student of mine.
[47:52.110 --> 47:53.230]  It was in Cambridge.
[47:53.590 --> 47:56.630]  And she teaches on Wednesday evenings on Girls Who Code.
[47:57.130 --> 47:59.850]  And I got invited to talk about basic security.
[48:00.410 --> 48:01.830]  And what I did was...
[48:03.470 --> 48:04.950]  What I did was this.
[48:05.670 --> 48:06.090]  Alright?
[48:06.230 --> 48:07.010]  Here's a blog.
[48:07.870 --> 48:08.430]  Alright?
[48:08.750 --> 48:09.750]  Here's a blog.
[48:10.310 --> 48:11.450]  Try to break it.
[48:11.450 --> 48:13.390]  Do all the fun stuff with it.
[48:13.830 --> 48:14.450]  Break it.
[48:16.310 --> 48:17.630]  Break it whatever you want.
[48:17.810 --> 48:22.930]  And, of course, one of the things I showed was the good old-fashioned trick, which is still very applicable these days.
[48:23.450 --> 48:25.050]  Well, what the whole idea of a password attack?
[48:27.730 --> 48:28.170]  Password.
[48:28.290 --> 48:30.830]  Trying to break the login for the blog site for a WordPress site.
[48:32.010 --> 48:33.050]  And you know what?
[48:33.690 --> 48:35.810]  It's to prove that things could...
[48:35.810 --> 48:37.990]  Whatever can be built should be able...
[48:37.990 --> 48:39.150]  It's to build...
[48:39.150 --> 48:39.310]  Yeah.
[48:39.510 --> 48:41.770]  Not to build something, but to break it.
[48:42.050 --> 48:44.730]  But also to be aware of...
[48:44.730 --> 48:47.890]  Be aware, be mindful of what goes wrong.
[48:48.070 --> 48:50.270]  And to apply it to think, Hey, this is why...
[48:50.270 --> 48:51.150]  You know...
[48:51.810 --> 48:52.650]  A lot of our...
[48:52.650 --> 48:53.330]  You know, credit...
[48:53.330 --> 48:54.930]  A lot of our information have been stolen.
[48:55.250 --> 48:57.110]  And the kids probably see this stuff on the news.
[48:57.270 --> 48:58.570]  Like X, Y, and Z got hacked.
[48:59.090 --> 49:00.230]  And even kids...
[49:00.230 --> 49:01.770]  They use like Club Penguin.
[49:02.090 --> 49:02.810]  And I don't know...
[49:02.810 --> 49:04.390]  They're probably like kids that have a Pokemon account.
[49:04.490 --> 49:06.030]  They probably got broken into already.
[49:06.310 --> 49:07.830]  But that's how that stuff works.
[49:07.830 --> 49:09.110]  But also...
[49:09.110 --> 49:10.810]  And so start to think, Hey, you know what?
[49:10.930 --> 49:12.310]  I could not only get this...
[49:12.310 --> 49:13.410]  To do this for a living.
[49:13.710 --> 49:14.410]  But also...
[49:14.410 --> 49:15.910]  I can also get arrested for that.
[49:16.190 --> 49:18.210]  So to see the trade-off for that kind of stuff.
[49:18.650 --> 49:19.070]  Thank you.
[49:20.970 --> 49:21.150]  Oh.
[49:21.470 --> 49:21.990]  Let's get...
[49:21.990 --> 49:22.670]  And...
[49:23.310 --> 49:24.570]  I'm gonna take two more of them again.
[49:27.780 --> 49:28.580]  I don't think it'll reach.
[49:28.920 --> 49:29.880]  Yeah, I don't think it'll reach.
[49:30.020 --> 49:30.800]  Oh, come on up.
[49:31.700 --> 49:36.620]  I saw you put up the VW deal on one of your slides.
[49:37.360 --> 49:37.640]  And...
[49:37.640 --> 49:50.560]  I mean realistically, how many of you believe that an engineer or a software developer at VW just independently rogue engineer decided, hey, let's develop software to cheat the diesel emissions standards.
[49:50.900 --> 49:51.360]  Right?
[49:51.360 --> 49:52.320]  I mean...
[49:58.280 --> 49:58.680]  So...
[49:59.200 --> 50:00.720]  I think that this is an issue.
[50:01.080 --> 50:01.380]  I mean...
[50:01.380 --> 50:07.060]  How many developers come out of college knowing what they need to know to get a real job?
[50:07.380 --> 50:14.360]  I've hired developers and we don't place a whole lot of emphasis on developers who have gone through a formal degree program.
[50:14.560 --> 50:17.380]  Really, we look at what those developers can actually produce.
[50:17.660 --> 50:23.980]  And that's kind of one of the neat things about this particular industry is you don't have to have a formal degree to be good at what you do.
[50:23.980 --> 50:25.520]  Unless you want to teach.
[50:25.840 --> 50:26.400]  True.
[50:27.100 --> 50:28.760]  Well, in academia...
[50:28.760 --> 50:47.620]  So the point here is that until companies... until it affects their bottom line, in not just a punitive way, the way that it currently does, where if you happen to get hacked, if it happens to get exposed that you're storing credit card numbers, you know,
[50:47.680 --> 51:03.400]  in plain text or something, until companies are making more money by being secure, by developing secure applications, I don't really see the motivation driving engineers to learn the security practices that they need to know.
[51:03.540 --> 51:04.040]  They're smart.
[51:04.140 --> 51:07.720]  If they get paid, they'll learn the security they need.
[51:07.860 --> 51:13.820]  And right now, there's just not the financial motivation to practice security.
[51:14.620 --> 51:20.960]  And I've actually... and I don't have a good answer to that, other than that, that I have a colleague that joked, why we don't pay attention to security?
[51:20.960 --> 51:22.180]  Because it's not profitable.
[51:23.600 --> 51:26.900]  I just want to say, at the high school level, you cannot teach attack.
[51:26.900 --> 51:27.840]  That is true.
[51:28.040 --> 51:32.260]  And you shouldn't, because they are immature enough to hack somebody's Facebook and send us all to jail.
[51:32.420 --> 51:34.540]  But there are things all ready to go.
[51:34.760 --> 51:36.160]  Pico CTF is ready.
[51:36.640 --> 51:37.760]  Capture the flags are very good.
[51:37.860 --> 51:46.660]  Pico CTF, Easy CTF, and Cyber Patriot are all designed for high school students, and they're primarily defensive, and that's what you have to do.
[51:46.720 --> 51:51.480]  And they enjoy the competition aspect and learn stuff, but you can't teach attack, obviously.
[51:51.480 --> 51:52.460]  That's the way it should be.
[51:56.920 --> 52:01.020]  Yeah, I just... the gentleman before was asking about high school stuff.
[52:01.920 --> 52:10.460]  There's a tremendous amount of money being given right now through the NSF and NSA to Gen Cyber Camps.
[52:10.540 --> 52:13.300]  There are about 170 of them all over the country.
[52:13.900 --> 52:19.640]  And those are primarily for high school and junior high school students to learn exactly the security stuff.
[52:19.880 --> 52:20.340]  Thanks.
[52:23.840 --> 52:24.940]  Yeah, thanks, everyone.
[52:25.120 --> 52:29.940]  And, of course, we ran out of time, but we definitely want any information.
[52:30.340 --> 52:31.920]  Email us or Twitter us as well.
[52:32.060 --> 52:35.900]  Yeah, so I think having this discussion on...
[52:35.900 --> 52:41.760]  I mean, having this discussion on social media and email, I mean, we absolutely welcome that.
[52:41.860 --> 52:44.480]  And, of course, you can find me, you know, where to find me on social media.
[52:45.320 --> 52:52.180]  Unfortunately, one thing is I have to run because I have to go to a family event in Queens.
[52:52.500 --> 52:53.780]  And, actually, everything worked out.
[52:53.920 --> 52:55.240]  So I got to run right now.
[52:55.360 --> 53:01.140]  But, Roy, you can, you know, we can have the discussion outside and we can continue the discussion outside the hall.
[53:01.360 --> 53:01.620]  Sure.
[53:01.720 --> 53:03.740]  And it's not an easy question to address.
[53:03.740 --> 53:04.200]  Yep.
[53:04.700 --> 53:05.060]  Thanks, everyone.
[53:05.260 --> 53:05.540]  Thank you.
[53:05.660 --> 53:06.420]  Thank you.