[00:00.000 --> 00:00.580]  Okay, cool.
[00:01.040 --> 00:02.920]  Alright, my name's Tim.
[00:03.060 --> 00:07.260]  I'm going to start my talk, which is called Deconstructing Ad Networks for Fun and...
[00:07.260 --> 00:08.840]  Sorry, you will have to hold that.
[00:09.300 --> 00:10.580]  I have to hold it up here.
[00:11.280 --> 00:12.980]  There's no, like, taller dude.
[00:14.280 --> 00:15.420]  Alright, I have two hands.
[00:15.520 --> 00:16.080]  We can use them both.
[00:16.600 --> 00:18.440]  So anyways, my name's Tim Libert.
[00:18.620 --> 00:21.880]  I'm a PhD candidate at the University of Pennsylvania down in Philly.
[00:22.240 --> 00:26.920]  But Philly sucks, so right now I live in Berlin at the Alexander Von Humboldt Institute.
[00:27.740 --> 00:30.320]  In a former life, I worked in computer forensics.
[00:30.420 --> 00:32.100]  I was an IT drone and web developer.
[00:32.560 --> 00:34.420]  My future life is undecided.
[00:34.560 --> 00:36.340]  I'm an all-but-dissertation person.
[00:36.340 --> 00:39.140]  So if you have something interesting for me to do, let me know.
[00:40.180 --> 00:50.680]  My research interests is I'm interested in kind of the broader sociological implications of kind of the past couple decades of widespread technological change we're seeing.
[00:51.440 --> 00:59.180]  My real question is how do these kind of, like, very low-level technical decisions filter up and have macro social implication?
[00:59.800 --> 01:13.240]  The reason I study privacy in quotes is because it's kind of a topic area where you really get to see where there are some tensions between the things that are interesting to people and the things that maybe we don't like about computers in our lives.
[01:14.240 --> 01:29.720]  So the type of research I'm going to talk about today, kind of the premise that's overlying this, is that ad networks broker personal information, collect personal information, and they also broker the relationships that exist between users, companies,
[01:29.840 --> 01:31.220]  and even above that, the state.
[01:31.820 --> 01:37.940]  So if you understand how these ad networks work, you get a better view into how social problems are working.
[01:38.600 --> 01:47.560]  And the problem with this, to me, is that what you have are these information asymmetries where the users don't know shit about what the companies do.
[01:47.760 --> 01:49.900]  They often know even less about what the government does.
[01:50.040 --> 01:54.360]  The companies know a lot about what users do, and the governments can kind of sit on top.
[01:54.520 --> 01:58.600]  And, you know, basically, in the whole situation, some people know more than others.
[01:58.640 --> 01:59.500]  That's always the case.
[01:59.660 --> 02:04.960]  But the most volume of people knows about the least volume of people.
[02:05.840 --> 02:12.160]  So the goal of what I'm going to be talking about today is I want to document the exact scope and size of these tracking networks.
[02:12.160 --> 02:18.580]  And that way, users can hold them accountable, and users can also hold governments accountable who are using the ad networks.
[02:19.700 --> 02:21.240]  What makes this a hacker talk?
[02:21.720 --> 02:27.160]  Again, I think we're living in an increasingly complex world, and I like survey research a lot.
[02:27.280 --> 02:37.060]  If you look at survey research, you see people increasingly, year after year, feel like you're an average person in the world, that they're losing control over their lives as they're overrun with technology.
[02:37.900 --> 02:40.740]  But as hackers, we're people who understand the systems.
[02:40.740 --> 02:47.640]  So I think if you're a hacker, what you really have is responsibility to learn things and then disseminate the knowledge.
[02:47.780 --> 02:50.240]  And the reason for that is to get rid of these asymmetries.
[02:51.180 --> 02:59.380]  So an example of hackers kind of evening out these asymmetries is Donald Trump knows he's a lizard person.
[02:59.600 --> 03:02.340]  And hackers can even that out.
[03:03.600 --> 03:06.240]  And so, moving on from lizard people.
[03:06.860 --> 03:08.660]  So part one, I'm going to talk about techniques.
[03:08.880 --> 03:11.220]  So this is going to be more on the tech end.
[03:11.580 --> 03:13.320]  Part two, I'm going to get more into findings.
[03:13.440 --> 03:15.360]  That's more on the implications end.
[03:16.120 --> 03:20.460]  I promised I was going to talk about malware, but I decided talking about the NSA would be more fun.
[03:20.700 --> 03:22.540]  And I haven't talked about that publicly before.
[03:23.840 --> 03:25.380]  So, I'm going to take a sip of water.
[03:28.560 --> 03:30.260]  You can stay as long as you want.
[03:30.680 --> 03:32.980]  You can stay until the next HOPE conference two years from now.
[03:33.180 --> 03:35.080]  You don't have to leave at the end of the hour.
[03:35.880 --> 03:36.240]  Okay.
[03:36.380 --> 03:37.040]  There's your future.
[03:37.270 --> 03:38.860]  I don't want to actually stay that long.
[03:41.180 --> 03:42.520]  I got a f*ck ton of slides.
[03:42.760 --> 03:42.990]  All right.
[03:42.990 --> 03:48.210]  So, basically, I'm going to talk about HTTP requests.
[03:48.320 --> 03:50.540]  It's kind of how web pages move around the web.
[03:50.710 --> 03:52.740]  I'm going to assume not everybody knows that.
[03:53.900 --> 03:55.140]  I guess I've got to hold this thing.
[03:55.600 --> 03:57.770]  So, web pages, you get them.
[03:57.960 --> 03:59.110]  You make a request.
[03:59.260 --> 04:02.990]  Your browser makes a request using something called a hypertext transfer protocol.
[04:03.820 --> 04:06.990]  It's basically just kind of a thing that says, give me this piece of data.
[04:07.100 --> 04:07.920]  Send it back to me.
[04:08.020 --> 04:10.110]  And it's kind of like common language.
[04:10.110 --> 04:11.200]  So these things work together.
[04:12.500 --> 04:14.520]  Each web page is...
[04:14.520 --> 04:16.960]  The first thing you get is a source code of the page.
[04:17.160 --> 04:19.560]  And that isn't, like, the page, right?
[04:19.650 --> 04:24.770]  So if you get a PDF document with a bunch of stuff in it, it's all one chunk that comes to your computer.
[04:25.090 --> 04:26.940]  When you go to a web page, you get a recipe.
[04:27.020 --> 04:30.260]  And that recipe tells your browser, like, hey, go get some fonts.
[04:30.270 --> 04:32.000]  Go get some YouTube videos.
[04:32.200 --> 04:32.970]  Get all this stuff.
[04:33.140 --> 04:35.910]  And what your browser is kind of like the chef in the kitchen, right?
[04:36.200 --> 04:37.970]  You say, hey, you give it...
[04:37.970 --> 04:38.940]  Make me a cake.
[04:39.180 --> 04:40.240]  It goes to the store.
[04:40.240 --> 04:40.900]  It gets the eggs.
[04:41.020 --> 04:41.710]  It gets the flour.
[04:41.880 --> 04:43.260]  It puts them all in the thing.
[04:45.000 --> 04:45.400]  So...
[04:46.180 --> 04:47.960]  These things can kind of come from two places.
[04:48.090 --> 04:49.240]  Let's say your web browser...
[04:50.120 --> 04:53.300]  The first party is who you've asked for the thing from.
[04:53.760 --> 04:56.260]  So that party is the same.
[04:56.410 --> 04:58.080]  So you already have in your browser bar...
[04:58.080 --> 05:00.210]  You're talking to TimLibert.me.
[05:00.330 --> 05:01.580]  So you know you're at my site.
[05:01.760 --> 05:03.520]  You download an image from my site.
[05:03.590 --> 05:05.180]  You know who it's coming from.
[05:05.520 --> 05:10.330]  But the other thing that can happen is inside that page there might be like a picture or a video.
[05:10.520 --> 05:13.410]  And that picture or video can come from another server.
[05:13.710 --> 05:15.200]  Your browser knows this.
[05:15.240 --> 05:21.350]  And your browser figures this out for you because you don't want to be typing in like a million addresses for everything.
[05:21.920 --> 05:26.740]  But this is without user notification because it's supposed to be convenient and work well.
[05:27.210 --> 05:31.060]  So sometimes it's a hidden web because some of these things aren't videos you see.
[05:31.240 --> 05:35.460]  They're little one by one pixels or particularly nasty forms of JavaScript.
[05:36.580 --> 05:43.400]  And this, because these little things are getting stuck in there by people you don't know about, that's a vector for surveillance.
[05:44.640 --> 05:46.340]  This is a little bit better with pictures.
[05:47.400 --> 05:51.080]  So the first thing that happens, again, you tell your browser.
[05:51.280 --> 05:55.680]  Person has to tell a browser to go get the HTML document.
[05:56.340 --> 05:59.180]  Then the person's not doing anything anymore.
[05:59.320 --> 06:01.850]  The HTML document is continuing this process.
[06:02.020 --> 06:06.220]  And this is really kind of important to keep in mind for how people really interact with the web.
[06:07.120 --> 06:11.120]  So again, no person involved anymore.
[06:12.160 --> 06:15.740]  And these servers cannot, they can be anywhere in the world.
[06:16.140 --> 06:22.240]  And the person has really no way of knowing about this unless you're a weirdo like me and people in this room and you look at this stuff.
[06:23.940 --> 06:27.700]  So, what data actually gets disclosed with these requests?
[06:28.560 --> 06:33.340]  So first off, you get your IP address, type of computer or browser you're using.
[06:33.440 --> 06:34.280]  That's a user agent.
[06:34.420 --> 06:40.540]  So when you look at a web page on your phone, it displays the phone version on your laptop, the desktop version.
[06:40.740 --> 06:45.100]  The reason for that is because with the request, it actually says, hey, I'm a phone.
[06:46.040 --> 06:47.980]  And the address of the page you're visiting.
[06:48.840 --> 06:54.540]  As the second bullet point shows, sometimes the address can have sensitive material just in there.
[06:54.680 --> 07:04.260]  So even if this other third party doesn't know anything else about you, they can actually see from the URL in many cases what you're looking at without even like scraping the page on their own.
[07:05.120 --> 07:12.440]  There's been some really good research that shows on a fairly large volume of data sets.
[07:12.680 --> 07:19.420]  Just using like UA string and IP address, you can get about 80% re-identification, which is what's good with cookies.
[07:19.920 --> 07:23.500]  So there's some talks and people are always having cool talks about fingerprinting stuff.
[07:24.380 --> 07:30.240]  You can up that 80 much higher, but with really basic information, you can get pretty good.
[07:31.240 --> 07:39.580]  And again, because you have the pages people are looking at, these URL strings, and you can identify them, you get a really good window into people's lives.
[07:40.200 --> 07:43.080]  And that's really, again, what I'm personally interested in.
[07:43.140 --> 07:44.960]  It's like, how are we looking into people's lives?
[07:45.600 --> 07:48.300]  And again, this is the basic stuff.
[07:48.300 --> 07:51.200]  You can get even creepier and weirder and get really more specific.
[07:53.420 --> 07:59.300]  So again, just to recap, when you load a page, connections are made to these third parties by each of these P requests.
[07:59.680 --> 08:01.300]  Third parties can keep records.
[08:01.520 --> 08:03.480]  The records show what pages people are visiting.
[08:03.900 --> 08:12.300]  And the really key point here is if a company has many bits of code across many pages, that's when they start to build a picture of your life.
[08:12.460 --> 08:15.850]  So there's like a thing called, a legal theory called like the mosaic effect.
[08:15.850 --> 08:19.880]  So imagine each visit to a website is a tiny tile and a mosaic.
[08:20.140 --> 08:24.600]  The more places I put my code, the more pixels I get to make a picture of you.
[08:32.840 --> 08:36.600]  But my personal interest is not in tracking anybody here.
[08:36.720 --> 08:40.340]  I like to track the companies doing it because it's a little bit more fun.
[08:40.760 --> 08:45.040]  So what I wanted to do is start documenting this at scale.
[08:45.780 --> 08:50.900]  So kind of like broad objectives, I wanted to load a lot of pages quickly, about a million or so.
[08:50.900 --> 08:54.260]  I wanted to find all the requests that are made of these third parties.
[08:55.100 --> 08:57.960]  And then at the network level, you just get a domain name, right?
[08:58.060 --> 09:00.400]  So some domain names you know, like google.com.
[09:00.520 --> 09:04.180]  But sometimes you're going to get 2mdn.net, which also goes to Google.
[09:05.000 --> 09:10.160]  Facebook.com you know, but you also might get fbcdn.net, also Facebook.
[09:11.080 --> 09:15.500]  So what I really want to do, what matters to me, is not just getting the network traffic.
[09:15.500 --> 09:18.320]  I have to connect that network traffic to people.
[09:18.320 --> 09:23.220]  And so for me, the social factor is really figuring out domain name registrations.
[09:23.240 --> 09:26.080]  And also company registrations and weird stuff.
[09:26.820 --> 09:29.060]  The other thing I want to do is make it cheap.
[09:29.280 --> 09:33.700]  Because if that kind of goes to point one, I can do it a lot faster if I can do it cheaper.
[09:34.260 --> 09:38.180]  And I also wanted to design something because, A, I'm a poor grad student.
[09:38.800 --> 09:40.500]  So I don't have like a huge budget.
[09:40.680 --> 09:42.360]  And B, I want other people to be able to use it.
[09:42.460 --> 09:44.340]  And I'm going to talk about the software route in a second.
[09:45.440 --> 09:50.480]  So when I started trying to figure this out in 2013, there's different projects out there.
[09:50.660 --> 09:52.220]  They didn't work for my needs.
[09:52.920 --> 09:56.380]  First of all, they relied on GUI or Virtual Frame Buffer.
[09:56.480 --> 10:02.360]  So that's stuff like Selenium where you're using like a real browser, but you have a real browser's overhead.
[10:03.320 --> 10:11.460]  Some of the ones out there at the time, they weren't using like a big database like SQLite, which is cool, but you can't scale up too high.
[10:13.000 --> 10:18.820]  Nobody had anything that, I don't think anybody else does at this point, still have anything that's going to produce you a report about companies.
[10:18.980 --> 10:22.740]  People will output reports about technical stuff, but not the social stuff.
[10:23.460 --> 10:28.900]  The other reason, too, is there's a lot of stuff like Ghostry, but these are very Western-focused.
[10:28.960 --> 10:31.720]  You're going to get American and European ad tech companies.
[10:31.720 --> 10:33.440]  You're not getting a lot of the Chinese companies.
[10:33.440 --> 10:45.860]  And so I've been working with a Chinese colleague because in the same way that, you know, there's a Chinese clone of almost any American service, there's also a whole different version of Chinese web tracking companies.
[10:46.060 --> 10:47.440]  We'll talk about those in a minute.
[10:48.400 --> 10:51.800]  So how not to do this in case you're deciding to do it yourself.
[10:52.820 --> 10:55.260]  This is not the correct attitude.
[10:57.200 --> 11:01.400]  So your first response is a hacker maybe, let's do it, the shell script.
[11:01.400 --> 11:10.300]  So you could, in theory, really quickly just W get a f*ck-ton of pages and then grep out all the requests.
[11:12.120 --> 11:14.960]  And you don't even have to download the content, so it's super fast.
[11:15.200 --> 11:21.440]  The problem with this is a lot of times pages are downloading JavaScript, and the JavaScript's going to fire off more requests.
[11:22.500 --> 11:24.060]  JavaScript's code that lives in the page.
[11:24.180 --> 11:29.060]  And the other thing, there's something called iframes, which are kind of pages that live inside pages.
[11:29.420 --> 11:33.400]  So if you're scraping stuff really fast, you can't exactly do it that way.
[11:34.480 --> 11:39.520]  And I believe everybody has to have at least one XKCD slide per talk.
[11:40.920 --> 11:42.320]  This is the one I'm including.
[11:44.380 --> 11:44.940]  All right.
[11:45.120 --> 11:48.340]  So the central problem is we can't do this with a shell script.
[11:48.560 --> 11:49.520]  So what do we do?
[11:49.580 --> 11:50.360]  We need a browser.
[11:50.640 --> 11:54.160]  We need a browser, but I still don't want a GUI, and I don't have a lot of RAM because I'm poor.
[11:54.160 --> 11:55.000]  So what do I do?
[11:55.700 --> 11:59.300]  I use PhantomJS, which is, if you haven't heard of it, it's awesome.
[12:01.040 --> 12:02.880]  PhantomJS is what's called a headless browser.
[12:03.140 --> 12:13.280]  So basically, it's a full web browser in the sense that it's built on WebKit, which is the same thing that's under Safari, and in theory, mostly under Chrome.
[12:13.480 --> 12:15.400]  So it's pretty much the same engine.
[12:15.400 --> 12:17.280]  It can run JavaScript.
[12:17.980 --> 12:19.420]  It's really cheap to run.
[12:19.560 --> 12:23.780]  You can run, like, a bunch of instances, like, very low resource utilization.
[12:24.940 --> 12:33.680]  And it has these really cool API where you can actually hook in on two API calls for on-resource-requested and on-resource-received.
[12:33.800 --> 12:41.960]  So some of the other things that people have done with using Selenium to do this type of work, you have to still have a layer in there, which is grabbing the stuff out.
[12:42.000 --> 12:44.980]  Otherwise, you can, like, Wireshark a bunch of random traffic.
[12:45.700 --> 12:48.400]  So PhantomJS, like, just boom, it gives you that report.
[12:49.300 --> 12:51.640]  It's not, you know, without flaws.
[12:52.560 --> 12:57.980]  I had to do a lot of weird custom coding to get around, like, URI and redirect chains.
[13:00.180 --> 13:06.320]  Everybody's experienced, you know, the thing where the page icon keeps spinning, and that's because it's still downloading a bunch of crap in the background.
[13:07.200 --> 13:11.820]  So I actually have to tell Phantom, like, hold on, let all the garbage arrive.
[13:13.260 --> 13:16.460]  There's some issues that I can talk about in person.
[13:16.680 --> 13:18.800]  People want about picking user agent strings.
[13:19.280 --> 13:23.560]  Ad networks can block it because they don't want to be discovered.
[13:23.920 --> 13:27.380]  And also people probably use PhantomJS for click fraud, which I don't do.
[13:29.380 --> 13:33.200]  And Chinese character encodings are a huge pain in the ass.
[13:33.200 --> 13:35.280]  But I crossed that bridge.
[13:35.920 --> 13:40.440]  So anyways, my software platform that controls this is called WebXRAY.
[13:41.420 --> 13:45.640]  And this is really basically an automation and analysis platform.
[13:45.980 --> 13:51.340]  So what it does is basically says it sends URLs to PhantomJS.
[13:52.040 --> 13:54.460]  PhantomJS kicks back JSON file.
[13:55.160 --> 13:57.960]  This has to run over the command line, which kind of sucks, but it works.
[13:59.140 --> 14:03.500]  So what WebXRAY does is ingests a list of addresses.
[14:04.140 --> 14:04.800]  It feeds them.
[14:04.920 --> 14:07.100]  It gets the data back.
[14:08.080 --> 14:09.800]  I did this several years ago.
[14:10.000 --> 14:13.400]  And I understood there's a current version of Python was 3.
[14:13.580 --> 14:14.680]  So that's what I taught myself.
[14:14.800 --> 14:15.440]  And it's written in.
[14:15.680 --> 14:20.480]  I since discovered that Python 3 is a mirage and that you're supposed to learn Python 2.
[14:21.880 --> 14:24.960]  Apparently until 2037 when Python...
[14:25.400 --> 14:27.840]  When we'll start adopting Python 3.
[14:28.060 --> 14:31.620]  So anyways, there's a couple cool things in Python 3 that are useful.
[14:32.020 --> 14:33.480]  Subprocess and multiprocessing.
[14:34.340 --> 14:37.640]  So like I said, PhantomJS is pretty cheap to run.
[14:37.980 --> 14:45.460]  And the cool thing with multiprocessing is you don't have to be that smart to use all your resources.
[14:45.460 --> 14:50.360]  And basically two times the RAM gets you two times the Phantom instances.
[14:50.740 --> 14:53.160]  So you can scale it up really, really well.
[14:53.540 --> 14:58.040]  I also use MySQL in the back, which again, that scales really well.
[14:58.260 --> 15:01.740]  You dump millions and millions of records and MySQL is not going to complain too much.
[15:10.750 --> 15:12.610]  So getting back to tech stuff.
[15:13.310 --> 15:14.650]  So it loads the page.
[15:15.310 --> 15:17.950]  We get all of our requests received events.
[15:18.610 --> 15:25.470]  Then the tricky thing is it turns out there's not an algorithmic way to figure out what the actual domain is in a request.
[15:26.450 --> 15:28.090]  And so you actually need a list.
[15:28.150 --> 15:30.590]  And there's a public suffix list from Mozilla.
[15:30.790 --> 15:33.190]  And you actually have to use that to really separate the domains.
[15:33.790 --> 15:38.110]  The good example is here, sub-example.ac.uk.
[15:38.130 --> 15:41.750]  So the actual domain here we care about is example.ac.uk.
[15:41.750 --> 15:48.330]  If we were just picking the last two tokens like, you know, google.com, then we wouldn't...
[15:48.330 --> 15:52.190]  We would say ac.uk is a domain and everything...
[15:52.190 --> 15:53.790]  Every UK university would match.
[15:53.930 --> 15:54.570]  We don't want that.
[15:55.230 --> 15:59.090]  And then it's just a pretty simple conditional statement.
[15:59.090 --> 16:06.810]  If the domain of the page does not match the domain of the element request, keep that in our database and look at it later.
[16:07.930 --> 16:12.170]  So the important part, as I said before, is really to figure out the companies.
[16:12.330 --> 16:16.190]  Like, so I had the network traffic, we're building it from that layer, and now I want to know the companies.
[16:17.010 --> 16:19.070]  And this has kind of been like an uphill battle.
[16:19.230 --> 16:24.450]  So there's been some lists I could kind of seed with, like, a list of like a hundred Google domains and stuff.
[16:24.530 --> 16:32.470]  But basically what I started doing several years ago is I would do a run and I would kick out like a hundred most common domains.
[16:32.510 --> 16:34.130]  And I would start figuring out who owns those.
[16:34.130 --> 16:36.850]  And then I kept redoing different experiments.
[16:36.850 --> 16:39.390]  And over time I kept surfacing new domains.
[16:39.930 --> 16:42.650]  And over that time I'd pick them out.
[16:42.750 --> 16:44.770]  I'd find new companies, add them to my database.
[16:45.250 --> 16:47.450]  I also try to check fairly frequently.
[16:47.450 --> 16:51.790]  I'll check companies in Crunchbase, because there's a ton of acquisitions going on right now.
[16:52.750 --> 16:57.510]  Like a really major one is like AddThis, which is that stupid orange thing that does nothing.
[16:58.810 --> 17:03.430]  But apparently Oracle thought it was pretty cool, because they spent a bunch of money buying it.
[17:03.430 --> 17:07.430]  So I had to update the company thing to say Oracle.
[17:07.710 --> 17:10.250]  So if you download the software, it's pretty up to date.
[17:10.450 --> 17:14.010]  And I'm always happy for pull requests for new domains.
[17:15.370 --> 17:20.230]  So again, that's a lot of like basically start with who is, and you kind of waste your afternoon from there.
[17:21.090 --> 17:23.430]  My favorite one actually is there's a company...
[17:24.230 --> 17:32.390]  I'm not going to get too into it, but there's one company I actually had to find their developer documentation, which is the only way I could figure out who the f*ck the domain belonged to.
[17:32.950 --> 17:39.470]  And when I'm working on stuff with the Chinese web, I have a Chinese colleague because I can't speak Chinese, and he's a super smart guy.
[17:40.330 --> 17:44.150]  So again, this is really kind of the point of the talk is this slide.
[17:44.350 --> 17:48.790]  Like this is how we're linking over technical stuff to the social world.
[17:48.890 --> 17:49.930]  In this case, companies.
[17:50.890 --> 17:55.590]  And just a final tech note before I go on to what this stuff means.
[17:57.110 --> 18:00.230]  Again, the design consideration was to be cheap and fast to run.
[18:00.350 --> 18:06.750]  So you can get about 30,000 pages an hour if you're letting the pages have a 90-second timeout.
[18:06.870 --> 18:10.330]  Because you can do more if you have like a one second, but you don't get all the cool stuff.
[18:11.310 --> 18:15.670]  You can tell I last presented this slide in Germany because it's in Euro.
[18:16.490 --> 18:18.850]  But basically, I mean, you can go pretty cheap.
[18:18.850 --> 18:25.890]  And if you're impatient, you can get the Alexa 1 million in a day and a quarter for about 30 bucks.
[18:26.650 --> 18:32.250]  If you're patient, you can get the Alexa 1 million in five days for like seven and a half bucks.
[18:32.430 --> 18:33.490]  And we're in New York City.
[18:33.750 --> 18:39.730]  So I actually think at some places, 30 might get you one cocktail at a New York City bar.
[18:39.990 --> 18:47.930]  But even at a cheap place, you can get the Alexa 1 million full set of trackers for the price of a beer, which I think is cool.
[18:48.970 --> 18:52.690]  So anyways, what does the data tell us?
[18:54.310 --> 18:55.630]  Other than I need some water.
[18:59.420 --> 18:59.960]  All right.
[19:01.740 --> 19:05.200]  So, I've been using this software for a few years, doing a bunch of different stuff.
[19:05.380 --> 19:05.780]  It's fun.
[19:06.620 --> 19:10.600]  I tried to pick a few kind of like case studies that I thought would be interesting to the HOPE audience.
[19:11.480 --> 19:16.020]  I do apologize for misleading advertising the talk description.
[19:16.260 --> 19:18.340]  I said I was going to talk about newspaper websites and malware.
[19:19.020 --> 19:20.540]  I said I'm going to talk about the NSA.
[19:21.340 --> 19:22.360]  So health.
[19:22.780 --> 19:29.100]  This is a project that came out that I published in March of last year.
[19:30.400 --> 19:32.780]  I kind of like to start these with a quote.
[19:33.060 --> 19:39.460]  So this is what I think is the oldest privacy policy in the world, which is the Hippocratic Oath, which is physicians.
[19:42.100 --> 19:48.360]  So, whatever I see or hear in the lives of my patients, I will keep secret as considering all such things to be private.
[19:49.120 --> 19:51.240]  I actually do want to have like a quick side note.
[19:51.360 --> 19:59.420]  The reason I also like this is one thing that really worries me recently is you have a lot of these tech companies whose banner is like, disruption, like we disrupt everything.
[19:59.740 --> 20:08.920]  If we've been making doctors say the same thing since before Jesus, like it's probably, it's like a good, it's like something not to disrupt.
[20:08.920 --> 20:10.000]  It's kind of a good idea.
[20:12.300 --> 20:17.140]  So, this is, again, this is like the most universally recognized privacy context.
[20:17.140 --> 20:23.120]  You can go anywhere in the world and it's basically understood sick people have a special right when it comes to information.
[20:24.200 --> 20:29.920]  The problem with that is people are going to Google before they go to their actual doctor.
[20:30.440 --> 20:36.140]  And what they don't understand is Google is not your doctor and Larry Page did not take the Hippocratic Oath.
[20:37.020 --> 20:39.920]  Mark Zuckerberg probably burned it for fun.
[20:42.240 --> 20:49.160]  So, you leave the doctor's office, you're out on the web, and you're in trouble.
[20:49.620 --> 20:52.040]  But I wanted to know exactly how much trouble, right?
[20:52.860 --> 20:58.320]  So, what I did is I took, I made a list of about 2,000 common disease names or conditions.
[20:58.720 --> 21:00.880]  I got the top 50 search results for those.
[21:01.060 --> 21:02.580]  That gave me about 80,000 pages.
[21:02.960 --> 21:15.520]  The reason I did it that way is because some people had looked at, like, web portals, like, you know, like WebMD or stuff like this, but there's pure data and research that shows people actually go to search engines first.
[21:15.860 --> 21:22.120]  And if your doctor gives you a nasty diagnosis, you're probably going to go five pages deep into those results.
[21:22.260 --> 21:36.220]  So, I wanted to see kind of a dark vision, but somebody, they get a nasty, you know, it's a text string, but you get a nasty text string from your doctor, you're going to go home, plug into a search engine, and you're going to keep clicking links.
[21:36.220 --> 21:38.360]  So, I wanted to look at these pages.
[21:38.800 --> 21:40.560]  So, I loaded these in the software.
[21:40.880 --> 21:44.120]  I wanted to find, first, what companies got the data.
[21:44.400 --> 21:56.180]  And secondly, going back to that whole thing with the URLs can reveal stuff, I wanted to see how often the disease was included in the URL, which I'm going to tell you.
[21:56.920 --> 22:02.700]  So, anyways, first off, 91% of the pages that I looked at leaked data to a third party.
[22:02.700 --> 22:10.600]  70% of all the URLs have human readable diseases in them.
[22:10.820 --> 22:14.200]  So, it's like, slash cancer, slash what have you.
[22:14.360 --> 22:16.000]  It's really easy to see.
[22:16.540 --> 22:19.360]  The ad tracker doesn't have to go back and scrape the page.
[22:19.500 --> 22:23.400]  You can just, in many cases, just parse out the disease from the URL.
[22:24.020 --> 22:29.820]  A thing this audience will appreciate as bad is only 3% of sites force SSL.
[22:30.440 --> 22:41.800]  I should go back, usually what I force, with WebExtra, I usually force it to go for a non-secure HTTP, and then I record a redirect.
[22:42.140 --> 22:44.720]  So, only 3% of times is it getting redirected.
[22:45.680 --> 22:49.160]  Google is the gigantor in this space, as in many.
[22:49.300 --> 22:50.940]  They get 78% of pages.
[22:51.900 --> 22:58.580]  And I redid this study after some people had contacted me, and I did it in the UK, in France, Germany.
[22:58.840 --> 23:03.960]  It's pretty much the same thing all over the world, even though, in theory, they have tighter rules.
[23:05.380 --> 23:06.560]  So, policies, right?
[23:06.680 --> 23:12.000]  So, we talk about detect, but how does that fit into a world of rules that humans make?
[23:12.460 --> 23:15.720]  So, first off, you have industry self-regulation.
[23:16.980 --> 23:19.120]  So, let's assume these are in good faith.
[23:19.120 --> 23:20.620]  Well, these, which they're not.
[23:21.020 --> 23:26.620]  So, these, they have rules about medical information.
[23:27.300 --> 23:32.240]  The NII had one that says, it's okay to market to people based on headaches, but not cancer.
[23:32.500 --> 23:37.440]  So, there's a lot of stuff in the middle, and I guess that's for you to make up.
[23:37.880 --> 23:40.440]  In the U.S., this is legal.
[23:41.560 --> 23:44.880]  So, HIPAA does not protect you on the web.
[23:45.040 --> 23:53.820]  HIPAA was actually drawn up, I mean, I think probably the first discussions about drafting HIPAA were happening contemporaneously with Tim Berners-Lee, like, making the web.
[23:54.440 --> 23:56.880]  So, it's not like they saw this happening.
[23:57.640 --> 24:01.620]  So, the U.S., in many ways, the privacy protections are pretty weak.
[24:01.840 --> 24:05.740]  In Canada, they have a little bit further.
[24:06.120 --> 24:13.460]  They dinged Google because they had, they were retargeting a guy with sleep apnea ads, which is not cool in Canada.
[24:14.300 --> 24:28.840]  So, Google acknowledged that some of the advertisers using their ad service did not comply with the corporation's policies, which made me wonder who the hell's in charge, because it's not like the people buying the ads made the giant system to spy on people looking at sleep apnea devices,
[24:29.000 --> 24:30.580]  or targeted to their computers.
[24:30.960 --> 24:43.660]  So, Google's settlement with the Canadians somehow involved them, like, telling people not to use their service for what exactly it's designed to do, which is a mystery to me.
[24:44.300 --> 24:46.340]  So, you know, what's the problem with this, right?
[24:46.640 --> 24:47.480]  Why does it matter?
[24:48.260 --> 24:49.840]  First off, people can be identified.
[24:49.840 --> 24:53.340]  You might not want to be identified by having a medical condition.
[24:53.700 --> 24:55.420]  You know, I talked about Google a minute ago.
[24:55.540 --> 24:57.140]  Google has pretty bad-ass security.
[24:57.340 --> 25:09.020]  They have good security teams, but there's a ton of long-tail, small nobody ad tech companies that don't have Google security, and I don't think the reason they've been hacked is because most people bother with them.
[25:09.640 --> 25:20.120]  There's also some, and this is a technical term, low-life pieces of shit, who have sold lists of people classified as rape sufferers and domestic abuse victims.
[25:20.380 --> 25:29.980]  So, if you're the type of person who thinks it's cool to sell a list of people who've been raped, you're a piece of shit, but you're not breaking the law, which is even more disturbing.
[25:30.500 --> 25:44.600]  Another thing, if we're going to move more towards the realm of, you know, predictive analytics and AI and all this cool stuff we're doing now, research actually by Elizabeth Warren, who's now a senator, showed that the number one reason for bankruptcy in the United States is health expenses.
[25:45.060 --> 26:00.380]  So, if we have a system that's designed to discriminate commercially among people, who to give the best discount to, who to tell early about the sale in the morning, all this type of stuff, who to give a better insurance rate to, if we just close our eyes and use web browsing data,
[26:00.380 --> 26:09.380]  people who at way above the mean are looking at health information are also going to be more correlated with bankruptcy, which means they're not people you want to sell shit to.
[26:09.680 --> 26:12.380]  So, again, this is not like an obvious thing.
[26:12.460 --> 26:20.220]  This one's actually kind of difficult to prove, but based on how systems work, you'd have to work really hard for this not to happen.
[26:20.220 --> 26:24.960]  And, again, HIPAA does not cover this in the EU.
[26:25.300 --> 26:30.460]  It should be legal, but no one's really enforcing it to a degree.
[26:30.460 --> 26:31.260]  I'm happy with.
[26:32.360 --> 26:33.960]  You can read more of this.
[26:34.040 --> 26:38.720]  It was Communications for Associating Machinery, Association for Computing Machinery.
[26:39.140 --> 26:41.940]  It's on my website in violation of their copyright.
[26:43.120 --> 26:44.480]  Then I'll talk about China.
[26:53.150 --> 26:53.770]  All right.
[26:53.850 --> 26:54.970]  So, here's another quote.
[26:54.970 --> 27:09.930]  Webtern is dedicated to facilitating the development of digital governance in China and to exploring effective means to monitor influential and positional views and discussions concerning social events, which sounds very benign from a Chinese government contractor.
[27:11.590 --> 27:13.510]  So, again, you know, there's...
[27:13.510 --> 27:15.390]  I'm not the only person researching this stuff.
[27:15.730 --> 27:23.910]  There's a lot of people researching the Chinese Internet, but pretty much a lot of Chinese Internet research is on censorship.
[27:23.910 --> 27:27.570]  Not really that many people are looking at web tracking and this type of stuff.
[27:27.690 --> 27:28.950]  So, I thought, all right, I'll do it.
[27:30.010 --> 27:31.210]  So, I start with my...
[27:31.210 --> 27:33.930]  This buddy of mine, Bo Mai, who's Chinese.
[27:34.670 --> 27:36.450]  We actually were thinking, so we...
[27:36.450 --> 27:40.270]  Can we actually see the government doing this type of web tracking in China?
[27:40.750 --> 27:41.530]  And there's kind of...
[27:41.530 --> 27:41.990]  There's three possibilities.
[27:42.670 --> 27:47.610]  One is that we could find it in a directly attributable way, which kind of seems weird to a Western audience.
[27:47.610 --> 27:57.830]  But if the Chinese government tomorrow wanted to say, a basis of you having a license to have a website is you have to insert our JavaScript after the body tag, people would have to do it.
[27:57.910 --> 27:58.570]  It would happen.
[27:58.790 --> 27:59.570]  It's different.
[28:00.230 --> 28:01.430]  So, maybe that exists.
[28:01.530 --> 28:02.730]  We want to look for it.
[28:02.870 --> 28:05.270]  The second thing is maybe it's done via contractors.
[28:05.530 --> 28:13.390]  And there's a government procurement database, which is weird because they want competitive bidding because it's like they're more capitalist than communist at this point.
[28:13.390 --> 28:16.470]  So, they do have a competitive government bidding system for some things.
[28:17.030 --> 28:23.230]  And then the last thing is maybe there's some sort of behind the scenes coercion or interception like we saw with Snowden.
[28:23.410 --> 28:31.350]  So, for this one, it's actually pretty hard to get a good list of popular Chinese websites because there's so much like rampant click fraud.
[28:31.890 --> 28:37.970]  So, the best we could do is the Alexa 500 top sites.
[28:38.650 --> 28:40.330]  This data is fairly fresh.
[28:40.570 --> 28:41.690]  It's from September 2015.
[28:42.450 --> 28:44.730]  Of those 500, we got 467.
[28:45.630 --> 28:46.950]  So, it's like 92 percent loaded.
[28:47.510 --> 28:57.770]  A lot of the ones that didn't, if I recall, were like banks who were like too smart to be scraped and like domain squatting and stuff like this.
[28:58.470 --> 29:00.930]  So, in the government attributable condition.
[29:01.210 --> 29:08.610]  So, what we would see is a request would go to a government registered domain that we could find across at least two sites.
[29:08.610 --> 29:10.530]  So, we're saying tracking movement, right?
[29:10.950 --> 29:16.890]  So, really we just dump all the domains and we see do we have any that match Gov CN.
[29:17.590 --> 29:23.510]  Also, government media agencies like CCTV, China Central Television, I think, Xinhua.
[29:24.290 --> 29:27.750]  And so, it wouldn't... again, it wouldn't be surprising to find this.
[29:27.850 --> 29:33.970]  I mean, if you're in the U.S. and you opened up a webpage and said like NSA.gov slash evil JavaScript, you'd freak out.
[29:33.970 --> 29:36.610]  It wouldn't be that surprising in China.
[29:37.570 --> 29:39.450]  So, here's one of the pages we looked at.
[29:39.630 --> 29:42.450]  It's like a buy art website or something.
[29:43.290 --> 29:49.450]  Look under the hood there and you see there's a JavaScript from cert ebs.gov.cn.
[29:50.290 --> 29:51.910]  That goes to this agency.
[29:51.910 --> 29:54.890]  So, we did find a government agency JavaScript.
[29:56.130 --> 29:58.830]  But this is the Shenzhen Market Supervision Bureau.
[29:59.150 --> 30:01.230]  And they're kind of like an e-commerce watchdog.
[30:01.370 --> 30:03.230]  It's kind of like a better business bureau situation.
[30:03.510 --> 30:07.650]  So, they're not like, you know, spying on people for nefarious reasons.
[30:08.510 --> 30:10.230]  And we found a few others of these.
[30:10.370 --> 30:13.370]  We found like CCTV PIC was on quite a few.
[30:13.510 --> 30:16.270]  That's like image hosting CDN.
[30:17.050 --> 30:21.710]  And so, even with the ones we found being totally benign, they're only about 4% of the sites we looked at.
[30:22.690 --> 30:25.250]  So, the second condition, government contracted surveillance.
[30:26.010 --> 30:29.510]  So, here the requests go to companies that we know contract with the government.
[30:29.810 --> 30:33.210]  And we have this government procurement database so we can check against it.
[30:34.050 --> 30:37.790]  And I like this, too, because the guy I worked with is not a programmer.
[30:37.830 --> 30:39.910]  And I can't read or understand Chinese.
[30:40.750 --> 30:47.890]  So, it really shows like, yeah, I really think there's a lot of strengths to be gained with cross-disciplinary and collaborative work.
[30:48.670 --> 30:51.890]  Which I would say to other hackers in the room, find friends.
[30:51.890 --> 30:53.170]  You can do great things.
[30:53.930 --> 30:57.230]  So, in this case, we looked at a page.
[30:57.390 --> 30:58.730]  China.com.cn.
[30:58.830 --> 30:59.970]  Here you have.
[31:00.710 --> 31:01.670]  Under the hood.
[31:02.630 --> 31:03.110]  Webterran.
[31:03.110 --> 31:04.550]  It was the quote from the beginning.
[31:05.130 --> 31:11.730]  And I really hate it when people do the JavaScript and then do a document write to write more JavaScript.
[31:11.730 --> 31:12.810]  It's like, why?
[31:15.130 --> 31:15.990]  I know.
[31:16.190 --> 31:16.550]  It's...
[31:16.550 --> 31:17.810]  But I got them anyways.
[31:18.310 --> 31:18.830]  So...
[31:20.050 --> 31:20.570]  Anyway.
[31:20.910 --> 31:22.190]  Oh, actually, where is...
[31:22.190 --> 31:22.350]  Oh.
[31:23.090 --> 31:23.350]  Yeah.
[31:23.530 --> 31:24.170]  I actually...
[31:24.170 --> 31:25.590]  I like that display none.
[31:25.970 --> 31:28.850]  You are recording and then do the honest game.
[31:28.990 --> 31:29.410]  They do.
[31:29.630 --> 31:30.370]  I don't know.
[31:30.630 --> 31:31.370]  It's annoying.
[31:31.670 --> 31:32.890]  But I get it.
[31:33.030 --> 31:33.650]  I don't care.
[31:34.270 --> 31:35.190]  So, anyways.
[31:36.570 --> 31:38.050]  Government contracted surveillance.
[31:38.190 --> 31:41.710]  So, we found two companies in the data set that actually we also found...
[31:41.710 --> 31:43.630]  were government contractors.
[31:44.430 --> 31:44.950]  Bo...
[31:44.950 --> 31:46.470]  He went deep.
[31:46.670 --> 31:49.330]  And he learned everything he could about KnownSec and Webterran.
[31:50.590 --> 31:51.490]  KnownSec is...
[31:51.490 --> 31:53.470]  They're basically just kind of like your run of the mill.
[31:54.250 --> 31:58.830]  CyberSec, if you're a firm, they make sure you're not getting like DDoSed and stuff like that.
[31:59.210 --> 32:01.350]  They seem pretty much above the board.
[32:01.990 --> 32:03.550]  Webterran is a bit creepier.
[32:03.810 --> 32:11.370]  They do a specialized public opinion research, which they track online discussions to inform the government of netizen opinion.
[32:12.070 --> 32:15.310]  They don't have time for a longer discussion of Chinese censorship.
[32:15.570 --> 32:24.030]  But one thing that's usually surprising or surprising to me when I first started getting into the topic is you're allowed to criticize the government a lot on the Chinese Internet.
[32:24.310 --> 32:28.710]  You're just usually not allowed to call for you and your friends to go out in public and do it.
[32:28.870 --> 32:30.890]  You can say you hate the government all day.
[32:31.130 --> 32:39.570]  And they actually use that to kind of monitor where people are in the society and then kind of trying to make moves to adapt to make sure they don't have people come out on the street.
[32:40.030 --> 32:41.310]  And that's what Webterran does.
[32:41.510 --> 32:46.810]  Webterran puts in their tracking code and they say, hey, people are kind of pissed off about this.
[32:46.910 --> 32:48.490]  Maybe you should do something.
[32:48.630 --> 32:55.250]  And the government wanting much more to make small adjustments to the way they do power than getting kicked out of power does it.
[32:55.350 --> 32:58.370]  So Webterran and there's a whole little cottage industry.
[32:58.510 --> 33:00.590]  And that's what Bo is looking at for his dissertation.
[33:01.410 --> 33:04.450]  Anyways, we had 92% of pages being tracked.
[33:04.690 --> 33:06.810]  Only four by the government.
[33:07.170 --> 33:09.030]  Three by these contractors.
[33:09.930 --> 33:11.670]  So where is the rest coming from?
[33:15.250 --> 33:18.210]  Is it all possible someone could provide me with additional water?
[33:18.410 --> 33:20.810]  I don't know if there's water or something.
[33:23.230 --> 33:24.550]  Perhaps I'll have to make this go.
[33:26.890 --> 33:28.530]  So commercial surveillance.
[33:30.690 --> 33:31.830]  Awesome, thank you very much.
[33:31.830 --> 33:35.670]  So pretty much all the tracking we found in China is commercial.
[33:36.330 --> 33:46.670]  And not surprisingly, it's consolidated really heavily among kind of like a few big companies, which are actually ones you've probably heard of even if you're from the West.
[33:47.210 --> 33:52.550]  And you get a long tail of really kind of like small operators doing three or four sites.
[33:53.510 --> 34:01.470]  And like in the West, the primary reason that these companies are placing this code is they want to drive up their ad rev with targeted advertising.
[34:02.450 --> 34:04.690]  So here are the top ten trackers we found.
[34:05.010 --> 34:06.330]  Number one is Baidu.
[34:06.490 --> 34:08.130]  Then we have, thank you so much.
[34:09.070 --> 34:09.550]  Alibaba.
[34:09.890 --> 34:12.470]  Google was number three, even though they're from the U.S.
[34:13.030 --> 34:13.510]  Tencent.
[34:13.870 --> 34:14.230]  Sina.
[34:14.750 --> 34:17.470]  If you've heard of Weibo, that's the Sina product.
[34:17.470 --> 34:18.550]  All yes.
[34:19.190 --> 34:19.670]  Knet.
[34:19.810 --> 34:20.510]  Media AV.
[34:21.470 --> 34:21.950]  Menlo.
[34:22.170 --> 34:23.030]  And then you got Amazon.
[34:23.550 --> 34:25.570]  That's actually mostly Amazon Web Services.
[34:25.790 --> 34:27.910]  So it's not really like a tracking thing.
[34:28.090 --> 34:31.050]  It's more people offloading some of their content hosting.
[34:32.310 --> 34:38.790]  So at this point, we cannot say conclusively that the government is leveraging commercial surveillance behind the scenes.
[34:39.250 --> 34:40.830]  But there's pretty much two options.
[34:40.830 --> 34:47.650]  One, they're not interested in kind of the level of fine-grained data you're going to get with really good modern user tracking.
[34:50.510 --> 34:58.790]  Or, more likely, they have some sort of secret data exfiltration or coercion and some sort of sharing system in the background.
[34:59.610 --> 35:02.730]  And we don't have a Chinese Snowden.
[35:02.910 --> 35:05.870]  And I don't know if there's more room in Russia at the moment.
[35:07.390 --> 35:08.030]  But...
[35:08.030 --> 35:09.110]  So we'll never...
[35:09.390 --> 35:10.850]  We may never really know.
[35:11.230 --> 35:12.590]  I lean towards number two.
[35:12.890 --> 35:18.230]  But as someone who's an objective scientist, all I can do is give you two options and probabilities.
[35:19.250 --> 35:23.110]  So one more thing before we move on from China.
[35:23.730 --> 35:28.350]  Is pretty frequently you're going to see headlines like this in the New York Times.
[35:28.930 --> 35:32.190]  China clamps down on web, pinching companies like Google.
[35:32.190 --> 35:42.030]  And, you know, always the implication or actually stated position of such articles is that Google's like the good guy and that's why China's f*cking with them all the time.
[35:42.110 --> 35:43.430]  Because they're good and they love freedom.
[35:43.750 --> 35:44.370]  Like us.
[35:46.790 --> 35:47.510]  And from...
[35:47.510 --> 35:49.970]  Actually, I forgot to mention this server was based in Beijing.
[35:50.150 --> 35:51.590]  Which was kind of a pain in the ass to set up.
[35:51.730 --> 35:53.450]  But this was all done out of Beijing.
[35:53.450 --> 35:58.430]  And we tried to load Google domains, timed out.
[35:59.890 --> 36:03.950]  And so that really tells us that the leaders of China and Google are not friends.
[36:07.790 --> 36:09.570]  But most things are blocked.
[36:09.870 --> 36:14.810]  And there's this other thing called Google Analytics, which is anywhere, anytime.
[36:15.670 --> 36:20.950]  So one thing Web X-Ray does is not only it monitors the request event, it monitors the received event.
[36:21.110 --> 36:24.350]  So if the received event does not happen, we know something's being blocked.
[36:24.550 --> 36:31.250]  And in this case, 25% of sites had a Google Analytics code embedded.
[36:31.250 --> 36:32.690]  And that was not getting blocked.
[36:32.910 --> 36:38.390]  So if I tried from this server, if I tried to hit Gmail, Google search, it would get blocked.
[36:38.390 --> 36:42.390]  I could just send traffic back and forth to Google Analytics all day.
[36:43.150 --> 36:49.630]  It's also interesting to note, when Google reorganizes Alphabet, the Chinese government blocked Alphabet in like five hours or something.
[36:49.810 --> 36:52.070]  So this isn't like... it didn't slip through accidentally.
[36:53.270 --> 36:57.010]  So again, just recapping, I'm not seeing blocking.
[36:57.270 --> 37:00.390]  So I can say conclusively, it's not blocked.
[37:00.970 --> 37:03.930]  And this is despite other services being blocked.
[37:04.390 --> 37:07.110]  And because we have a tech crowd, it gets worse.
[37:07.110 --> 37:11.590]  86% of those Google Analytics requests are not encrypted.
[37:12.490 --> 37:20.550]  So that makes it really, really easy for the Chinese to just use Google Analytics as like an add-on to their intelligence agency.
[37:20.830 --> 37:23.310]  The Chinese government selectively blocks this.
[37:23.530 --> 37:25.310]  So they're gaining a benefit.
[37:25.490 --> 37:27.070]  This is mutually beneficial.
[37:27.350 --> 37:30.770]  Google will sooner or later move back to the Chinese market.
[37:30.770 --> 37:33.530]  When they do, they have a lot of user data.
[37:34.810 --> 37:36.750]  At some point, someone did...
[37:36.750 --> 37:38.070]  I can't get into too many details.
[37:38.210 --> 37:39.550]  Someone actually contacted me.
[37:39.970 --> 37:43.850]  And it seems at least one of the Google Analytics servers is in China.
[37:43.850 --> 37:45.310]  So it's physically there.
[37:45.570 --> 37:47.810]  So someone's paying the rent on that rack.
[37:49.090 --> 37:54.550]  So Google in China, instead of Google being the freedom fighter, I would say they are surveillance BFFs.
[37:57.250 --> 37:58.410]  Got you, fam.
[37:59.990 --> 38:05.970]  And if you want to read more about this, I wrote an article in The Guardian with my other friend Maria Rapnikova.
[38:06.410 --> 38:07.730]  So now I'll talk about the NSA.
[38:07.730 --> 38:07.790]  Okay.
[38:10.690 --> 38:12.050]  No such agency.
[38:18.070 --> 38:19.170]  So here's a good quote.
[38:19.310 --> 38:24.790]  I like, rather than looking for a single needle in the haystack, let's collect a whole haystack.
[38:24.950 --> 38:29.830]  Collect it all, tag it all, store it, and whatever it is you want, you go searching for it.
[38:30.130 --> 38:32.710]  And that is General Keith Alexander.
[38:33.130 --> 38:37.350]  Here, looking like someone you don't want to leave your children with unattended.
[38:39.070 --> 38:42.230]  And he has a passion to collect it all.
[38:42.370 --> 38:44.290]  Also a Pokemon player, so watch out.
[38:45.510 --> 38:46.990]  So my haystack.
[38:47.470 --> 38:51.310]  This is kind of older data, but I thought it was fun.
[38:51.510 --> 38:55.610]  I looked at Alexa top 1 million websites in May of 2014.
[38:56.010 --> 38:58.290]  Of those, I got 950,000.
[38:58.930 --> 39:02.470]  On that, I got 21 million unique third-party elements.
[39:02.670 --> 39:06.010]  And when I'm saying unique elements, that's actually the beginning part.
[39:06.010 --> 39:15.090]  So if there's like a site identifier at the end, so it's like Google Analytics, question mark, ID equals ABC, I drop all after that.
[39:15.190 --> 39:17.530]  I only count Google Analytics as one.
[39:17.670 --> 39:20.010]  And so there's 21 million unique things.
[39:20.570 --> 39:26.830]  And when you visit those 950,000, you're going to get 7.5 million unique cookies.
[39:27.270 --> 39:32.510]  And again, these figures are really aggressively low-bounded and filtered on purpose.
[39:32.510 --> 39:35.770]  So if you redefine it, the numbers go up considerably.
[39:36.970 --> 39:43.950]  So the general findings on these million sites, 88% of Alexa top 1 million has a third-party element.
[39:44.270 --> 39:46.230]  83% have third-party JavaScript.
[39:46.830 --> 39:51.070]  And 62% have a third-party cookie, which is actually kind of lower than I thought.
[39:51.070 --> 39:58.350]  On average, each page is going to ping 9.47 other third-party domains.
[39:59.710 --> 40:02.270]  So the question is, who are these third-parties?
[40:02.590 --> 40:06.410]  And I can't use this slide outside the U.S., so I'm happy to use it in New York.
[40:07.890 --> 40:08.350]  Anyways.
[40:09.010 --> 40:10.750]  So many of these are companies you've never heard of.
[40:10.810 --> 40:15.710]  So this is the distribution of the top 50 companies on the Alexa 1 million.
[40:15.710 --> 40:18.110]  So as you can see, it kind of tails off.
[40:18.290 --> 40:22.810]  And when you get down there, you have some, like, really sector-specific ones.
[40:22.890 --> 40:27.770]  There's even one that only does religious advertising called, like, FaithNet HD.
[40:28.070 --> 40:30.310]  So I think, like, what would Jesus advertise?
[40:32.130 --> 40:34.430]  So you haven't heard of a lot of these.
[40:34.690 --> 40:36.010]  Some of them you have.
[40:36.550 --> 40:40.910]  Here's the PRISM slide from our main man Snowden gave us.
[40:40.910 --> 40:53.490]  So as you can see in there, Microsoft, Yahoo, Google, Facebook, PalTalk, who apparently only terrorists know what that is, YouTube, Skype, AOL, and Apple, who don't show up in my data pretty much at all.
[40:54.310 --> 41:04.650]  So of the identified PRISM companies on the Alexa 1 million, Google's on 78% of sites, Facebook 32, Twitter, Yahoo, Microsoft, and AOL picking up less.
[41:05.710 --> 41:09.630]  So people like to kind of, like, throw around PRISM as a scary thing.
[41:11.630 --> 41:16.350]  But we don't actually know that much about what it means in a kind of a technical sense.
[41:17.550 --> 41:21.370]  It's kind of like that one slide has gotten a ton of attention, but it's not really clear what it means.
[41:21.930 --> 41:25.290]  But Snowden gave us more than just that one PRISM slide.
[41:25.390 --> 41:28.290]  And some of those slides have really juicy technical details.
[41:29.430 --> 41:34.390]  So here's one from a Washington Post report that was really well written.
[41:34.390 --> 41:38.930]  And if you look there, it says, you know, our approach, da, da, da, da, da, da.
[41:40.290 --> 41:42.570]  Accounts, cookies, Google Pref IDs.
[41:43.230 --> 41:46.950]  So there's a specific cookie the NSA likes, which is Google Pref.
[41:47.430 --> 41:54.790]  And there's another thing, another presentation called Tor Stinks, which I think the Tor developers are happy about.
[41:54.790 --> 42:00.470]  And here, cookie leakage, double-click, seen on Tor, non-Tor IPs.
[42:00.650 --> 42:05.510]  So the NSA has figured out that these cookies are really useful.
[42:05.750 --> 42:07.830]  And that's not my Photoshop, but it's funny.
[42:07.990 --> 42:08.890]  So I put it in.
[42:09.390 --> 42:13.310]  So the NSA, we know they like these specific cookies.
[42:13.410 --> 42:17.730]  So my question is how many cookies are in the cookie jar with the Alexa 1 million?
[42:18.430 --> 42:24.490]  8% of the ones I looked at had the Pref, and about 20% had Google DoubleClick.
[42:24.630 --> 42:32.930]  The same specific cookie that's mentioned in this cookie, in this slide, is on one in five sites.
[42:34.010 --> 42:37.750]  So I want to point out, too, like Snowden, I think, did a really good job.
[42:38.150 --> 42:43.110]  But it's up to us in the technical community to even a couple years that some of the stuff is stale.
[42:43.110 --> 42:46.610]  Keep trying to figure out what technical bits can we work on.
[42:47.510 --> 42:52.350]  And to reprise the earlier slide, we got gotchu NSA.
[42:55.330 --> 42:57.710]  So some future projects I'm working on.
[42:57.850 --> 43:01.970]  I'm looking at tracking on activist websites, Black Lives Matter.
[43:02.290 --> 43:07.030]  Years ago, one of the reasons I started getting into this is looking at Occupy sites way back when.
[43:07.530 --> 43:10.290]  The situation on activism websites is not good.
[43:11.010 --> 43:14.350]  That includes many of your favorite digital rights organizations.
[43:14.830 --> 43:19.530]  Not the EFF, but some of the other ones who haven't fixed their sites after been told.
[43:20.630 --> 43:23.890]  I'm also sitting on a big longitudinal data set.
[43:24.010 --> 43:27.410]  I've been doing the top 100,000 Alexa every two weeks.
[43:27.770 --> 43:30.750]  And there's some, like, really cool changes and things going on.
[43:30.990 --> 43:32.630]  But I haven't crunched all of it.
[43:33.150 --> 43:35.310]  There's a one-one version of the software.
[43:35.310 --> 43:38.750]  There's more of a focus on UX impact of tracking.
[43:39.510 --> 43:41.590]  There's going to be some more that I didn't talk about today.
[43:41.670 --> 43:43.730]  Some more stuff about bringing in malware detection.
[43:44.390 --> 43:49.390]  And also I'm working on making my implementation of PhantomJS a little bit harder to detect.
[43:50.610 --> 43:54.170]  So if you want to contact me, that's my email.
[43:54.830 --> 43:57.090]  You can read my publications there.
[43:57.410 --> 43:58.570]  You can get the software.
[43:59.830 --> 44:01.470]  And you can download the software.
[44:01.530 --> 44:03.850]  And if you want to contribute to the software, you can do that also.
[44:04.010 --> 44:04.970]  And that's all I have.
[44:17.570 --> 44:19.270]  So how do we do questions in this room?
[44:19.350 --> 44:20.050]  Just shouting.
[44:21.090 --> 44:21.450]  Shouting.
[44:21.750 --> 44:22.110]  Shout.
[44:22.110 --> 44:22.210]  Shout out.
[44:22.970 --> 44:26.570]  Are you familiar with a company called Chihu, a Chinese company with a toolbar?
[44:27.450 --> 44:29.950]  I'm not off the top of my head, but...
[44:29.950 --> 44:31.530]  It is the largest...
[44:32.650 --> 44:34.790]  It has the largest botnet in the world.
[44:35.170 --> 44:37.250]  It's constantly updating its software.
[44:37.590 --> 44:38.290]  It uses...
[44:38.290 --> 44:39.170]  How do you spell Chihu?
[44:39.350 --> 44:40.630]  Q-I-H-U.
[44:41.570 --> 44:43.650]  They were the inventor of Chinese malware.
[44:44.010 --> 44:44.310]  Okay.
[44:44.710 --> 44:50.410]  And they have a toolbar, which everybody, for their protection, does their searches through.
[44:50.410 --> 44:54.790]  So that's a different way that the government can get all the click streams.
[44:55.250 --> 44:55.290]  Yeah.
[44:55.450 --> 44:56.390]  There's another one.
[44:56.610 --> 44:58.630]  It's called the Green Dam software.
[44:58.890 --> 45:02.690]  The Chinese government wanted to package all new PCs with the spyware.
[45:03.010 --> 45:05.870]  And it's like one of the few times people were like, no.
[45:07.550 --> 45:07.910]  So that's it.
[45:08.090 --> 45:08.590]  That's another...
[45:08.590 --> 45:11.630]  Well, Chihu has more than 80% of the Chinese market share.
[45:11.770 --> 45:12.570]  I'll look into them.
[45:12.810 --> 45:15.050]  I know Baidu's browser has some f*cked up stuff, too.
[45:15.970 --> 45:16.870]  Open up this guy.
[45:21.130 --> 45:22.270]  I have a mic.
[45:24.690 --> 45:25.730]  It's my fault.
[45:26.590 --> 45:27.770]  I can shout.
[45:27.850 --> 45:29.430]  You can shout and then I'll repeat it.
[45:29.870 --> 45:34.390]  Have you looked at the Oracle Blue Kai data marketplace?
[45:34.750 --> 45:37.470]  Have I looked at the Oracle Blue Kai data marketplace?
[45:37.470 --> 45:45.710]  So they actually list all the things, if you're an advertiser, if you want to buy data, all the things that are for sale through their data marketplace.
[45:45.990 --> 45:47.570]  Like, who likes to drink?
[45:47.890 --> 45:51.910]  And who likes to, you know, visit things with their friends?
[45:52.050 --> 45:53.090]  And it's like very delicious.
[45:53.690 --> 45:54.130]  Yeah.
[45:54.350 --> 45:56.050]  I've looked at some of that stuff a little bit.
[45:56.210 --> 45:59.750]  It's kind of out of scope with most of the stuff I do.
[46:00.230 --> 46:05.770]  One of the things I like looking at, if you ever want to laugh, is there's Axiom, which is one of the big data brokers.
[46:06.010 --> 46:12.210]  They have lists of the types of people and they're like so thinly veiled racist.
[46:12.710 --> 46:13.650]  Like, it's unbelievable.
[46:13.990 --> 46:17.850]  Like, it's like urban hip hop lovers is like a category.
[46:18.250 --> 46:20.590]  It's like, you're selling lists of black people.
[46:20.590 --> 46:21.330]  That's what you're doing.
[46:22.190 --> 46:23.590]  And they have all kinds of stuff.
[46:23.810 --> 46:31.630]  Like, any tortured way you could describe a race in like a really demeaning way, go to Axiom and they have a thing you can buy.
[46:32.510 --> 46:33.130]  So, yeah.
[46:34.530 --> 46:42.090]  Another one that's actually kind of interesting is there's some that I'm seeing now which are kind of operating almost like dark pools, kind of.
[46:42.250 --> 46:45.370]  So, like Google, you can go buy Google Ads, right?
[46:45.530 --> 46:49.690]  But there's other services that, let's say, my website sells cars, luxury cars.
[46:49.690 --> 46:51.410]  You sell luxury clothing.
[46:51.690 --> 46:53.570]  We don't compete, right?
[46:53.810 --> 46:58.850]  But I want to know about people who buy luxury clothes and you want to know about people who buy luxury cars.
[46:58.970 --> 47:05.210]  So, we can actually put tracking code from one of these companies and they'll kind of route data between the two of us.
[47:05.350 --> 47:11.630]  And that never hits a company like Google, which for better or worse is big enough to have public scrutiny.
[47:11.790 --> 47:17.870]  So, a lot of these smaller companies are doing really cool slash evil things that, like, nobody notices.
[47:17.870 --> 47:19.950]  But that one I will look at.
[47:20.990 --> 47:22.250]  This gentleman, yes.
[47:22.530 --> 47:26.030]  What sort of things are you looking at to make PhantomJS less detectable?
[47:26.730 --> 47:33.630]  And if it supports user agent spoofing, are sites out there doing anything more complex than that with browser detection?
[47:34.150 --> 47:38.270]  You must know how I am making PhantomJS less detectable.
[47:38.970 --> 47:42.390]  I'm going to wait until I release the 1.1 version of the software.
[47:42.390 --> 47:45.430]  So, I want to actually, like, use it before I publish it.
[47:47.370 --> 47:48.670]  It's a little selfish.
[47:49.730 --> 47:50.170]  Yeah.
[47:50.670 --> 48:00.590]  I mean, some of the thing I didn't mention too is the downside of writing something that goes really fast is you really look like a bot when you're downloading 30,000 pages an hour.
[48:00.870 --> 48:02.670]  So, yes, sir.
[48:02.670 --> 48:10.550]  Have you thought of, like, group sourcing your data collection and then possibly doing, like, live analysis, maybe capturing it through, like, a browser plugin?
[48:10.710 --> 48:15.750]  I mean, I guess that's going the opposite direction that you want to go because technically you're collecting data from people's computers.
[48:15.750 --> 48:21.410]  But, I mean, it might be a powerful way to get a better snapshot of what people are actually looking at.
[48:21.410 --> 48:21.990]  Yeah.
[48:22.170 --> 48:25.730]  So, basically it's like crowd sourcing to solve this question.
[48:26.010 --> 48:26.370]  Yeah.
[48:26.610 --> 48:35.150]  I mean, the best I could do with that was just start up on GitHub and I have had a contribution which is very, very appreciated and I hope I get more contributions.
[48:36.430 --> 48:39.810]  One thing I've been looking at is actually pulling in other data sources now.
[48:40.090 --> 48:43.950]  So, I use the data scheme as MySQL, so it's relational.
[48:44.390 --> 48:52.550]  So, the cool thing about that is I've actually started working on looking at other services with APIs where you can query information about domains.
[48:53.190 --> 48:58.850]  And, with Web X-Ray, it's really simple to write a script that just essentially patches the database with new information.
[48:59.150 --> 49:00.770]  So, I am looking at stuff...
[49:00.770 --> 49:03.070]  I'm looking at ways to pull in crowd sourced data.
[49:04.030 --> 49:04.550]  Yes?
[49:05.890 --> 49:12.990]  I was wondering if you were doing any analysis, especially about certificates and differences based on .
[49:13.890 --> 49:14.850]  I have not...
[49:14.850 --> 49:19.790]  You want to know if I looked at differences in SSL certs in China based on...
[49:19.790 --> 49:20.610]  Source IP?
[49:20.890 --> 49:23.490]  I have not, but that sounds cool.
[49:24.070 --> 49:26.370]  And I would encourage you to do that.
[49:28.930 --> 49:30.190]  Got you once already.
[49:30.330 --> 49:31.490]  Did anyone knock down once?
[49:32.250 --> 49:33.130]  This gentleman...
[49:33.130 --> 49:34.830]  Did you do anything to make...
[49:34.830 --> 49:36.030]  It's a tool called Web X-Ray?
[49:36.330 --> 49:36.510]  Yeah.
[49:36.870 --> 49:40.910]  Did you do anything to make it try to mimic a mobile client?
[49:42.690 --> 49:45.410]  The question is, have I tried to make it look like a mobile client?
[49:45.410 --> 49:48.490]  The answer is no, not exactly.
[49:49.470 --> 49:53.630]  I'm doing my best just to pull off looking like a desktop browser.
[49:54.010 --> 49:56.810]  And with mobile UA strings, it gets like a little bit weirder.
[49:56.910 --> 50:06.050]  So, I'm trying to just shoot for like middle of the lane, billion plus user UA strings, which is easier.
[50:06.410 --> 50:07.230]  Yes, gentleman.
[50:07.850 --> 50:10.530]  So, you were saying before about the correlation tracking.
[50:10.530 --> 50:20.190]  I guess you would explicitly recommend that anybody that's running a controversial or activist or health site not to have trackers and specifically not to have any of those trackers.
[50:20.290 --> 50:21.350]  Is that probably your advice?
[50:22.170 --> 50:22.610]  Yeah.
[50:22.790 --> 50:24.570]  I mean, there's different ways to do it, right?
[50:24.670 --> 50:29.490]  So, he's asking if I would recommend that activists and health websites don't have tracking.
[50:30.090 --> 50:33.890]  So, one of the things that I see a lot is things like social media buttons, right?
[50:34.670 --> 50:35.690]  Like, yay, like...
[50:36.550 --> 50:38.630]  And they're using the Facebook code.
[50:38.630 --> 50:47.250]  There's also a Facebook-crafted PHP, like, share.php, question mark, whatever your URL is.
[50:47.350 --> 50:50.190]  And you can just include that as a link and get, like, functionality.
[50:51.190 --> 50:55.710]  Facebook doesn't advertise that as prominently as their NASILA JavaScript library.
[50:56.410 --> 51:14.130]  So, the first thing I would like to see is pretty much everybody, if you are in a domain and you're serving information that has a sensitive impact on people who are at risk, first off, get rid of all your social buttons and just change them slightly and they'll be privacy protective.
[51:16.750 --> 51:18.430]  Second, people need analytics.
[51:18.570 --> 51:20.910]  You need to know if someone's visiting your site or not, right?
[51:20.930 --> 51:27.510]  I'm not suggesting that in the interest of privacy, site operators go blind and don't keep logs or anything like that.
[51:28.270 --> 51:30.910]  But there's responsible ways to manage logs.
[51:31.110 --> 51:32.190]  I mean, you can rotate them out.
[51:32.310 --> 51:34.130]  You can keep them for X period of time.
[51:34.330 --> 51:35.490]  You can aggregate them.
[51:36.010 --> 51:40.370]  I personally use the PIWIC or P-WIC, or I don't know how you're supposed to say it.
[51:40.450 --> 51:44.130]  It's basically like a Google Analytics clone that you host yourself.
[51:44.370 --> 51:54.370]  And the thing I like about it is they have really spent so much time on the installer process that if you can install WordPress, you can install your own analytics platform.
[51:55.330 --> 51:58.530]  And I actually think it's better than Google Analytics, personally.
[51:58.850 --> 52:02.950]  But, you know, so that's the type of direction I think things can go in.
[52:04.090 --> 52:04.850]  This gentleman.
[52:05.430 --> 52:06.270]  One in a second.
[52:06.470 --> 52:06.670]  Yes.
[52:06.910 --> 52:17.990]  Hey, are you currently now or planning in the future to release any kind of databases to help system administrators or privacy activists block third-party tracking links by smaller companies?
[52:17.990 --> 52:21.450]  Like, because you're collecting all this domain and idea information.
[52:21.730 --> 52:22.290]  Yeah.
[52:22.330 --> 52:30.950]  So one thing I've been thinking about actually kind of more, like, bigger picture, with a lot of...
[52:30.950 --> 52:44.690]  I didn't talk about the malware stuff, but malvertising has become a thing, which malvertising is basically in order to get singing and dancing horrible ads on your page, people send you JavaScript they haven't vetted, which is insane.
[52:44.690 --> 52:49.090]  I think criminals have figured this out, so they're sending malware.
[52:49.670 --> 52:52.070]  And the companies act like they're the victims.
[52:52.090 --> 52:53.690]  Like, oh, poor us.
[52:53.810 --> 52:54.710]  We've been tagged targeted.
[52:54.850 --> 52:55.610]  Like, no, dude.
[52:55.770 --> 52:56.790]  Like, this is your fault.
[52:57.530 --> 52:58.390]  So I...
[52:58.390 --> 52:59.780]  One behavior I would...
[53:00.270 --> 53:05.910]  If anybody is managing a corporate firewall, I think that's a black list immediately.
[53:05.910 --> 53:13.050]  If your ad company has served malware, I think any corporate firewall should just drop you immediately.
[53:13.890 --> 53:17.570]  And a lot of people look at stuff at work that they're not supposed to and they're being advertised to.
[53:17.710 --> 53:25.970]  And if corporate IT really started enforcing that, I think the industry would lose so much, even above what they're losing with ad blockers.
[53:26.630 --> 53:29.930]  Which, by the way, I don't think are not wanting to see ads.
[53:29.990 --> 53:35.390]  I think that's not wanting to be hacked by some dude in Eastern Europe with a bunch of Bitcoins.
[53:36.970 --> 53:39.290]  Anyways, any other questions?
[53:40.110 --> 53:40.390]  Right?
[53:40.590 --> 53:41.190]  One in the back.
[53:41.410 --> 53:42.270]  Sorry, we'll get back to you.
[53:42.370 --> 53:43.030]  Yes, sir.
[53:43.590 --> 53:44.070]  Okay.
[53:44.610 --> 53:45.930]  Actually, Tom, I was...
[53:45.930 --> 53:46.070]  Yeah.
[53:46.370 --> 53:47.350]  Do you think it
[53:50.560 --> 54:01.640]  took the browser out and maybe change the data strength just a little bit and give them a completely bogus stuff until they believe things that are impossible to poison their data?
[54:02.220 --> 54:07.400]  So the question is, do we think obfuscation is a good way to kind of screw with...
[54:07.400 --> 54:10.240]  Or generating phony data is a good way to screw with the companies?
[54:11.240 --> 54:17.720]  I mean, I'm a lowly grad student and I can, like, do fun stuff on, like, cheap servers.
[54:17.720 --> 54:24.740]  But I don't really think that's a winning strategy when you're dealing with people who have billions of dollars.
[54:25.800 --> 54:27.900]  And billions is not like a joke.
[54:28.040 --> 54:30.880]  These companies are raking in billions of dollars.
[54:31.060 --> 54:36.440]  They're getting the best comp-side PhDs in the world to do this stuff.
[54:36.440 --> 54:43.960]  So if you think you have a really clever way to stop them that is a technical fix, I don't really think it's going to work.
[54:44.120 --> 54:47.500]  I think the only real fix for this stuff is laws.
[54:47.980 --> 54:57.060]  Because laws eventually result in a man with a... or a woman with a gun, somebody with a gun, telling you, if you don't do this, we're going to put you in a cage for a while.
[54:57.060 --> 55:00.820]  And that, I think, is more effective, ultimately, than ad blocking.
[55:01.580 --> 55:03.060]  So I would like to see laws.
[55:03.820 --> 55:06.020]  And I'm not saying there should be a law against advertising.
[55:06.020 --> 55:14.400]  But if you're collecting people's health information, selling lists of who's been raped, like, I think you should go to jail for a while.
[55:14.520 --> 55:15.240]  And it shouldn't be fun.
[55:16.440 --> 55:16.840]  Okay.
[55:17.540 --> 55:17.940]  Yes?
[55:18.140 --> 55:24.960]  Did you see any use of privacy proxies for WHOIS data among any of the companies who are using it for privatizing?
[55:24.960 --> 55:30.380]  So the question is, when I was looking at domain registration, did I see anonymous proxy services?
[55:30.600 --> 55:30.960]  Yes.
[55:32.600 --> 55:38.900]  Those were so annoying, because I'd be, like, ten minutes away from a lunch break, and I'd hit something, and it's, like, an anonymous proxy.
[55:39.180 --> 55:42.100]  And two hours later, someone had lunch, and I finally get it.
[55:42.160 --> 55:46.160]  So a lot of the ad tracking domains are registered anonymously.
[55:47.420 --> 55:52.700]  You know, there's, like, I could have a whole talk about ways to figure out who owns a domain.
[55:52.700 --> 55:56.580]  But in the end, it's just, it's, like, an investigative thing.
[55:56.940 --> 56:00.720]  Crunchbase is actually really helpful sometimes, an unexpected one.
[56:00.780 --> 56:04.240]  So what aggregate percentage ended up in the can't-tell-who-these-are bucket?
[56:04.860 --> 56:06.260]  Which wasn't in your graph, right?
[56:06.460 --> 56:09.040]  I pretty much managed to track down almost everything.
[56:09.280 --> 56:12.900]  I don't think any, I don't think there's been any I couldn't get at the end of the day.
[56:13.000 --> 56:18.840]  I think the most exotic one was really, I was, I had to look up the developer documentation, and they were a weird company.
[56:19.060 --> 56:24.540]  They, every time you load a web page, there's, the company has an 800 number, like, call now.
[56:24.820 --> 56:31.920]  They actually randomly, or pseudo-randomly, generated the phone numbers, and then the phone numbers were paired with the user browsing data.
[56:32.060 --> 56:39.440]  So when you call the company, it pops up on their screen who you are based on the phone number you called and the information.
[56:40.300 --> 56:44.240]  Yeah, those guys really had their domain.
[56:44.660 --> 56:49.680]  But, you know, pretty cool product if you have no scruples.
[56:50.760 --> 56:51.360]  Yes?
[56:51.860 --> 57:04.860]  I'd be interested to see the intersection between things that you've classified as non-venying and the things that are blocked by ad blockers or things of that sort.
[57:06.380 --> 57:13.680]  Yeah, so the question is, things that I saw that were or weren't blocked by ad blockers, more or less?
[57:14.060 --> 57:14.500]  Yeah.
[57:14.620 --> 57:19.640]  Okay, so there's, there's some, actually some other people have been doing some pretty good research on that.
[57:19.760 --> 57:23.480]  I think at the EFF Privacy Badger talk, they talk about a few.
[57:23.640 --> 57:26.640]  There's a good paper by, um, Rosner?
[57:26.900 --> 57:30.680]  I forget her first name, like, Francisca or something like that.
[57:30.680 --> 57:34.300]  She was at University of Washington, CompSide Department.
[57:34.400 --> 57:36.580]  They did a pretty cool thing.
[57:36.740 --> 57:39.060]  They basically ran the same type of scam that I do.
[57:39.220 --> 57:42.620]  And then they re-ran it with an ad blocker on or with Ghostery on.
[57:43.380 --> 57:45.200]  And there's a few people who have done stuff like that.
[57:45.260 --> 57:46.960]  And those are cool studies, too.
[57:48.320 --> 57:49.440]  I'm good on questions.
[57:49.900 --> 57:50.880]  I'm good on time.
[57:51.460 --> 57:52.380]  One last question.
[57:52.540 --> 57:56.860]  You may have said this, but is the database itself open-source and accessible?
[57:56.860 --> 58:01.320]  Yeah, the code is open-source for academic publications.
[58:01.540 --> 58:02.780]  I always publish the data sets.
[58:03.620 --> 58:04.020]  Yeah.
[58:04.020 --> 58:08.040]  Yeah, so if I, any time I put out a publication, I post the data.
[58:08.920 --> 58:11.700]  Right, but we query your MySQL database.
[58:11.960 --> 58:12.380]  Yeah, yeah.
[58:13.420 --> 58:23.120]  Usually when I release it, I release it as one big table that has, you know, because there's like some, if you want to look at the SQL schema, you can, but there's a bunch of weird plumbing.
[58:23.120 --> 58:26.440]  And basically when I post it, it's like a giant CSV.
[58:27.300 --> 58:28.440]  But you can check those out.