[00:00.000 --> 00:05.640] So, hi, and thanks to HOPE for having me back again this year. [00:06.320 --> 00:15.320] Today I'm going to talk about obfuscation and also debut some new software, which I'm hoping will generate a bit of conversation. [00:15.760 --> 00:17.980] I'm going to try to leave some time for that. [00:18.280 --> 00:27.660] I'm a bit jet-lagged having just arrived from Hong Kong, so forgive me if I don't make complete sense. [00:28.760 --> 00:30.760] But let's start with the definition. [00:31.640 --> 00:47.780] Obfuscation is used in a range of different ways, but what I mean here for this talk is specifically the deliberate creation and or inclusion of misleading information in an effort to evade, distract, or confuse an adversary. [00:49.840 --> 00:55.540] Another way to put it is the intentional injection of noise into data streams. [00:57.220 --> 00:58.640] Take your pick. [00:59.640 --> 01:11.260] By way of example, this image shows a recently discovered variety of orb spider found in the Peruvian Amazon in 2012. [01:11.260 --> 01:16.580] The spider itself is actually just the little part at the top. [01:17.560 --> 01:32.140] The rest of that gunk, which is constructed by the spider from leaves and garbage, is a decoy to attract predators to the web. [01:35.240 --> 01:50.740] This image, which is a bit hard to see, shows a bomber releasing radar, what's called radar chaff, to confuse radar systems, so that they see many bombers instead of just one. [01:50.740 --> 02:05.360] This was a technique used extensively in World War II, where strips of paper backed with aluminum foil were dropped from planes in large quantities as they flew over enemy positions. [02:07.280 --> 02:23.360] This is a still from one of my favorite films by Stanley Kubrick in 1960 called Spartacus, and we can watch a short clip here if the AV system cooperates. [02:36.230 --> 02:37.570] Have we a count of prisoners? [02:37.990 --> 02:38.050] Have we a count of prisoners? [02:38.230 --> 02:40.090] We haven't made the final count, sir. [02:41.110 --> 02:43.690] I bring a message from your master! [02:44.110 --> 02:44.730] It's like the computer... [02:44.730 --> 02:49.370] Marcus Licinius Crassus, commander of Italy. [02:50.030 --> 02:56.570] By command of his most merciful excellency, your lives are to be spared. [02:59.980 --> 03:03.660] Slaves you were, and slaves you remain. [03:04.340 --> 03:18.740] But the terrible penalty of crucifixion has been set aside on the single condition that you identify the body or the living person of the slave called Spartacus. [03:31.220 --> 03:33.100] I'm Spartacus! [04:12.950 --> 04:16.330] Uh, heart-wrenching stuff. [04:17.130 --> 04:18.070] Um... [04:19.230 --> 04:30.710] Of course, what these, each of these items, or what they have in common together is that they're all examples of obfuscation from the analog domain. [04:31.050 --> 04:42.870] The spider and the bomber creating fake decoys to draw away attackers, and the slaves in Spartacus hiding our hero by all claiming falsely to be him. [04:44.390 --> 05:04.180] So I started working in this area back in grad school when I was doing research on data surveillance, specifically third-party tracking via cookies, something that the folks from EFF were talking about yesterday afternoon in their excellent presentation. [05:05.630 --> 05:31.330] Here you can see a hacked version of Mozilla that provided real-time information on cookies, a little bit along the lines of Privacy Badger, dynamically evaluating the risk each cookie represented to the user, and also providing supporting materials to facilitate their understanding of the decisions that the software was making. [05:32.290 --> 05:38.630] And you can see the cookies there on the left with different ratings as to their potential danger. [05:40.030 --> 06:01.750] And in this paper with Bahia Friedman and Ed Felton, we specifically looked at the notion of informed consent in regards to data surveillance, and what this could or should mean when interacting with, you know, vast corporate entities like Google or Yahoo. [06:03.490 --> 06:16.910] And soon after this, I started to think about and work on web search because, well, of course, free access to information is crucial to the functioning of any democracy. [06:18.910 --> 06:33.170] And it was here that I first stumbled onto the strategy, and this was with my long-time collaborator, Helen Nissenbaum, strategy of obfuscation to counter data surveillance. [06:33.990 --> 06:49.230] And in 2006, we released TrackMeNot to much critical disdain at the time, but which was, as best we can tell, the first instance of obfuscation in the online space for preserving privacy. [06:51.150 --> 06:59.110] And the idea behind TrackMeNot was very simple, and this was maybe one of the reasons why it became quite popular for a while. [06:59.610 --> 07:16.310] It simply noticed when a user was searching on a search engine like Google or Yahoo or whatever, and then surrounded their real query with a set of fake but believable queries issued in the background and invisible to the user. [07:18.730 --> 07:35.390] Now, making queries believable to a competent adversary with significant resources is a non-trivial research problem, which I won't go into much today, but for those interested, there are a number of good papers that have come out since this on this. [07:35.590 --> 07:36.670] This is one of them. [07:36.870 --> 07:44.150] I'm also... I'll also leave some references at the end of this talk if anyone wants additional information. [07:45.890 --> 07:49.830] But if we can assume a good implementation, the strategy allows... [07:49.830 --> 07:56.350] These are some of the queries that TrackMeNot generated over a period of time. [07:57.210 --> 08:10.130] Assuming a good implementation, the strategy allows users to search and to receive the correct results without disclosing their true interests, which may be sensitive for any number of reasons, medical or otherwise. [08:10.530 --> 08:14.590] And it pollutes... and it pollutes the data profile stored by the search engine. [08:15.650 --> 08:18.730] But perhaps most... more importantly, it is expressive. [08:19.070 --> 08:30.130] It gives users a voice in their interactions with these vast corporate entities, specifically writing their descent, you know, all over the server logs of the company. [08:32.510 --> 08:43.790] And this is a sort of key point in understanding where obfuscation can be a useful strategy, specifically in contexts that involve information asymmetries. [08:45.740 --> 08:52.290] A term referring to the often lopsided power dynamics between those gathering data and those... [08:52.290 --> 08:54.530] the users who are providing it. [08:55.110 --> 08:55.670] Okay? [08:56.270 --> 09:10.530] In the case of search engines, for example, entities like Google have massive resources, and users have little say in whether they're monitored, what happens to the data collected, and what might happen to them later because of that collection. [09:12.230 --> 09:16.930] In fact, many users are not even aware that such surveillance is occurring. [09:17.190 --> 09:23.810] Even when they are, they have no idea what the data will be used for, nor when or with whom it might be shared. [09:25.350 --> 09:34.930] As such, the potential consequences of the surveillance appear vanishingly small at the decision time, when they choose to search. [09:35.770 --> 09:42.630] While the true cost only reveals itself over a much longer period, well after the moment of decision has passed. [09:43.910 --> 09:55.710] And this lack of ability to assess risk is critical because without such an understanding of consequences, it is impossible to give any kind of meaningful informed consent. [09:57.750 --> 10:11.110] So in these cases, when users have few resources at their disposal, and when other solutions like regulation seem unlikely, at least in the short term, obfuscation can be a useful tool. [10:12.890 --> 10:14.730] And it has a number of strengths. [10:15.330 --> 10:17.150] As I mentioned, it's expressive. [10:17.330 --> 10:21.210] It gives voice to people who are otherwise powerless. [10:21.210 --> 10:33.910] It is often communal, in that by polluting one's own data profile, we also lower the value of the aggregate data, and thus the attractiveness of the business model itself. [10:36.170 --> 10:39.550] It's relatively low in resource intensity. [10:39.890 --> 10:43.810] A simple prototype of TrackMeNot was written in a day or so. [10:45.370 --> 10:47.050] And it's intuitive for users. [10:48.010 --> 11:00.890] Unlike something like public key encryption, for example, users are often able to grasp the strategy immediately, and to understand what the software is doing, or at least what it claims to be doing. [11:02.050 --> 11:06.390] And so it can counter some of the asymmetries that I'm mentioning. [11:07.630 --> 11:11.530] We can even think of it as what anarchist political scholar James C. [11:11.710 --> 11:13.990] Scott calls a weapon of the weak. [11:14.350 --> 11:26.090] He used this term to explain how peasants in Malaysia resisted authority when they had very few resources and could not mount any kind of real rebellion. [11:28.570 --> 11:39.850] Of course, obfuscation also has its weaknesses, as critics have pointed out, which makes it less suitable and even perhaps unethical in certain situations. [11:39.850 --> 11:42.810] It often requires additional resources. [11:43.350 --> 11:46.390] Generated noise may be filtered by an adversary. [11:46.850 --> 11:50.850] It has the potential to escalate into an arms race. [11:51.530 --> 12:00.470] It's often difficult to measure the success of the strategy because we have no access to the adversary and their techniques. [12:00.790 --> 12:03.810] And there may be important ethical considerations. [12:05.690 --> 12:12.110] Is the obfuscator free-riding by obtaining services without paying the cost? [12:12.330 --> 12:14.210] Wasting valuable resources? [12:14.450 --> 12:17.970] Violating agreements, whether implicit or explicit? [12:18.410 --> 12:23.210] Or possibly even providing cover for criminals or terrorists? [12:25.790 --> 12:34.830] So when evaluating such projects, we can ask whether they work, but also whether they're ethically justified. [12:34.830 --> 12:39.730] And we devote a large section of the original TrackMeNot paper to this latter question. [12:39.990 --> 12:44.570] But we can answer it only by considering specific cases. [12:44.570 --> 13:04.930] And happily, in the years, well, ten years now that since we released TrackMeNot, there have been many, many projects leveraging obfuscation that we can look at spanning context from social media, to email, to location tracking, to anonymity and more. [13:08.470 --> 13:26.470] Private information retrieval and obfuscation-based search have developed into active research areas in CS, with papers evaluating the relative success of TrackMeNot and other solutions, as well as suggesting improvements and metrics for analysis and comparing implementations. [13:28.230 --> 13:37.850] Cash Cloak by Myritz and Chiduri is a nice example where obfuscation is used to protect location data from mobile providers. [13:37.850 --> 13:53.230] And the way this works is instead of sending only the request for data on a user's current path, Cash Cloak sends requests for a number of different plausible paths, but presents the user only with the result for the actual path. [13:53.430 --> 13:57.690] The provider is then left to guess which of the requests was the real one. [14:00.860 --> 14:13.140] There have also been a number of projects that I've called ArtWare, which sort of combine the functionality of a tool with elements of politically engaged artworks, a few of which you can see here. [14:14.540 --> 14:22.260] Scare Mail by Ben Grosser adds algorithmically generated narratives to the end of each Gmail message that you send. [14:22.540 --> 14:23.580] It's a plug-in. [14:24.040 --> 14:28.420] These narratives contain one or more trigger words monitored by the NSA. [14:29.780 --> 14:34.260] And so it works as a means of invalidating bulk surveillance techniques. [14:34.260 --> 14:40.140] You can see why I refer to it as part artwork and part tool. [14:40.440 --> 14:42.840] I'm not sure how many people use it as an actual tool. [14:44.020 --> 14:51.680] I Like What I See by Steve Kleiss is a browser plug-in that automatically likes every single post one... [14:52.820 --> 14:54.860] that one sees on Facebook. [14:55.780 --> 15:01.340] Hiding users' true interests and diminishing the accuracy of their data profile. [15:02.860 --> 15:12.820] also totally sort of destroying their news feed as it comes in, which is an interesting side effect. [15:14.600 --> 15:27.460] Invisible by Heather Dewey Hagborg brings obfuscation back to the physical world with a spray designed to obfuscate traces of one's DNA that might be left at a scene. [15:28.600 --> 15:33.800] And you can actually buy this product in certain museum gift shops. [15:34.780 --> 15:39.660] And there are many, many other examples in various contexts. [15:40.480 --> 15:43.090] How are we doing? [15:43.370 --> 15:55.850] But with this background, we can now get to the exciting part of the talk, which is our new software release, which uses obfuscation as a strategy to fight back against surveillance by advertisers. [15:57.870 --> 16:03.690] And, of course, state surveillance has been getting more than its share of attention in the media. [16:04.030 --> 16:15.290] But, as Cory mentioned a few hours ago, corporate surveillance is actually equally or more pervasive. [16:16.030 --> 16:21.030] And it actually doesn't matter which is happening as they're all combined eventually. [16:22.230 --> 16:27.790] And this happens as we transition to what has been called surveillance capitalism. [16:28.190 --> 16:36.890] A new subspecies of capitalism in which profits derive from unilateral surveillance and modification of human behavior. [16:37.330 --> 16:41.730] And I would probably add prediction as well to that definition. [16:44.720 --> 16:50.920] So, when we look at advertising, of course, we see a context with massive information asymmetries. [16:51.260 --> 16:55.560] We interact with vast corporations with whom we have little or no say. [16:55.880 --> 17:05.380] We have no idea how the data collected might be used at present or in the future after being aggregated, sold, or stolen. [17:06.340 --> 17:10.900] And we cannot gauge the long-term risks to which we are exposing ourselves. [17:13.900 --> 17:24.380] So, at HOPE 10, I presented our first prototype of this work called AdNauseam, which today we are happy to release in version 2.0. [17:26.580 --> 17:31.480] And here is our advertisement for the project. [17:32.020 --> 17:36.560] Of course, you should read our terms of service before watching this. [17:41.280 --> 17:56.220] As data-voting corporations keep sabotaging the constitution of the do-not-track standard, and ad networks keep selling our profiles behind our backs, we are left with no choice but to fight back. [17:57.080 --> 18:07.960] From the studios that brought you TrackMeNot, the browser extension that keeps searching to obfuscate your search queries, comes yet another data obfuscation plugin. [18:08.720 --> 18:09.860] AdNauseam. [18:10.220 --> 18:12.240] Clicking ads so you don't have to. [18:13.260 --> 18:17.720] The AdNauseam plugin works in concert with your favorite ad blocker. [18:18.080 --> 18:26.640] Every ad blocked is then silently clicked by AdNauseam, confusing your data trackers by virtually liking all ads. [18:28.240 --> 18:38.280] Within your browser, the AdNauseam add-on collects and visualizes your clickstream over time, giving you a small glimpse of the data you passively generate. [18:38.980 --> 18:43.540] AdNauseam works in the background and does not interfere with your browsing. [18:44.000 --> 18:48.760] Point your browser at this URL and install AdNauseam now. [18:49.720 --> 18:57.100] Spread the word and help us code as we work to make AdNauseam better and to bring it to more people and more browsers. [19:00.220 --> 19:07.300] Finally, to the ad industry we would like to say it is time to be respectful of our privacy expectations. [19:07.300 --> 19:13.440] Let's work together towards an honest and meaningful do not track standard. [19:13.860 --> 19:19.500] Until that day comes, be sure you will keep pushing your buttons. [19:20.240 --> 19:21.320] All of them. [19:50.990 --> 19:51.630] All of them. [19:51.630 --> 19:57.510] Okay, so that video was put together by our awesome designer, Moushan Zarevive. [19:58.170 --> 20:04.270] And as you probably gathered, AdNauseam works quite similarly to an ad blocker. [20:04.390 --> 20:09.270] Version 2, in fact, is built on the excellent Ublock origin source code. [20:09.270 --> 20:12.970] It was actually a complete rewrite from version 1. [20:14.490 --> 20:18.750] So it blocks or hides ads on the pages you visit. [20:20.190 --> 20:31.970] Then silently and automatically clicks each ad in the background, polluting your data profile and creating mistrust between advertisers and the networks they pay for clicks. [20:35.120 --> 20:44.720] It also presents these interactive visualizations of the ads you have been served and probably not actually seen. [20:45.780 --> 20:52.500] Providing a glimpse as to what we look like to the people who are tracking us. [20:54.060 --> 20:57.740] And for version 2.0, we have added a number of new features. [20:58.020 --> 21:02.680] Most important of which is the support for Chrome and Opera browsers. [21:02.680 --> 21:04.400] In addition to Firefox. [21:05.300 --> 21:10.760] And amazingly enough, as you can see here, AdNauseam is now available in the Google Chrome store. [21:22.000 --> 21:26.780] Other features include the ability to block malware and non-visual trackers. [21:26.900 --> 21:29.300] Right, so these are things that are not visual ads. [21:29.620 --> 21:32.100] Options to view or not view ads. [21:32.380 --> 21:34.420] To click or not click ads. [21:34.580 --> 21:39.760] As well as improvements to our real-time menu display and to the interactive vault. [21:39.760 --> 21:48.740] Here you can see a tracking protection test that AdNauseam, the default setup, has passed. [21:50.580 --> 21:54.580] But there are still a number of questions for AdNauseam. [21:54.840 --> 21:58.740] Some have been written about online in regards to the prototype. [21:59.380 --> 22:00.700] The first is safety. [22:00.900 --> 22:02.620] How safe is it to use? [22:02.620 --> 22:07.760] The answer here is that it is much safer than using nothing at all. [22:09.320 --> 22:15.720] But it is less safe than using, say, a high-quality blocker like uBlock. [22:16.840 --> 22:17.900] Why is that? [22:18.040 --> 22:34.640] Because to get the data that we need for the ads, we need to occasionally allow requests that uBlock would block to be able to show the ad in the visualization. [22:35.520 --> 22:38.040] And then also to have enough information to click on it. [22:39.500 --> 22:42.140] It's efficiency is quite close. [22:42.680 --> 22:43.740] I don't... [22:43.740 --> 22:46.120] We're doing metrics on this now. [22:46.360 --> 22:55.340] But in my own use, we haven't noticed much difference at all in the performance versus uBlock. [22:58.520 --> 23:04.260] It's a similar case where it will be much faster than using nothing at all, as most of the world has to do. [23:04.260 --> 23:12.240] And probably slightly slower than using uBlock, as they can block a few extra requests. [23:15.440 --> 23:18.420] Again, we have the problem of opacity of response. [23:18.780 --> 23:24.540] We don't know what ad networks are doing to counter this initiative. [23:25.320 --> 23:31.720] And one way they would do that is by filtering out those clicks. [23:32.260 --> 23:34.680] But what happens when they filter out those clicks? [23:35.220 --> 23:38.480] Well, then they're sort of just ignoring you as a user, right? [23:38.620 --> 23:42.260] And in that case, you can think of it as a win, right? [23:42.340 --> 23:45.620] You've opted out of tracking through ads. [23:46.660 --> 23:52.760] And then there are some of the same ethical critiques. [23:54.040 --> 23:54.900] Excuse me. [23:55.660 --> 24:15.160] In the question of safety, the metaphor I used in the FAQ was that it is probably, yes, it is probably slightly less safe than using uBlock in the same way that attending a protest is less safe than staying at home. [24:15.160 --> 24:17.820] And, you know, watching TV. [24:34.430 --> 24:34.890] Okay. [24:35.030 --> 24:41.590] Also, there are a bunch of new features in the pipeline that weren't quite ready for 2.0, but should be in the next release. [24:44.210 --> 24:50.670] The optional compatibility with privacy badgers do not track whitelisting, which I'm quite excited about. [24:51.170 --> 25:01.950] So that users can choose to automatically whitelist sites that have posted a do not track commitment, as well as some of the other things you see on this list. [25:08.490 --> 25:10.150] How are we doing on time here? [25:10.950 --> 25:11.310] Okay. [25:11.750 --> 25:19.870] And we can see how the various aims of the strategy here in this diagram and how they relate. [25:22.030 --> 25:26.030] We're obviously attempting to be expressive, as I mentioned before. [25:26.550 --> 25:46.590] We're trying to be subversive, undermining the larger economic system by creating distrust and also by making the data less valuable and thus the business proposition less valuable of making money off of personal data. [25:48.550 --> 25:57.150] And we might notice that it not only protects the user, but it protects the larger community as the aggregated data is less accurate. [25:58.030 --> 26:03.250] Solutions like encryption or even ad blocking by contrast do not have this property. [26:04.550 --> 26:12.210] They generally will hide your voice and are primarily for an individual and perhaps their correspondence. [26:14.050 --> 26:19.950] And there's some interesting relationships between these properties that I don't really have time to go into here. [26:20.150 --> 26:26.530] But for example, there appears to be an inverse relationship between the goals of subversion and expression. [26:26.870 --> 26:33.990] The more expressive obfuscation technology is, the less subversive it will be. [26:33.990 --> 26:38.130] And the more subversive it will be, the less expressive it will be. [26:38.710 --> 26:41.190] Or at least that's my hypothesis. [26:41.590 --> 26:50.470] If you're interested in this, you can read about it in this paper, which I've also listed in the reference section at the end. [26:56.110 --> 27:01.390] So in any case, I hope you'll try the software for your browser of choice. [27:01.810 --> 27:04.590] Leave feedback for us on GitHub. [27:05.310 --> 27:12.410] And hopefully I've left enough time for a discussion on obfuscation in general. [27:12.610 --> 27:15.230] On any of the projects that I've mentioned. [27:15.830 --> 27:17.310] Or on AdNauseam. [27:17.450 --> 27:19.190] And these can be technical questions. [27:21.870 --> 27:23.690] Or ethical questions. [27:23.990 --> 27:24.550] Or otherwise. [27:24.830 --> 27:26.670] And I'll do my best to answer. [27:29.190 --> 27:32.730] And I also have lots of cool stickers here. [27:33.470 --> 27:34.950] That we just had made. [27:35.490 --> 27:40.150] So you can, after the talk, if you like stickers, come up and grab some of those. [27:44.290 --> 27:45.550] And that's it. [27:45.710 --> 27:46.450] Thanks a lot. [27:53.470 --> 27:56.310] As I said, I'll post this stuff online. [27:57.630 --> 28:00.770] Including all these references for people who want information. [28:02.070 --> 28:02.630] Where? [28:04.210 --> 28:04.770] Where? [28:04.970 --> 28:05.810] That's a good question. [28:09.310 --> 28:12.830] Isn't there a hope place for talks? [28:14.630 --> 28:15.830] For the talk slides? [28:17.430 --> 28:18.710] Nobody's mentioned that. [28:18.830 --> 28:19.250] Alright. [28:19.450 --> 28:21.050] Well, it will be on my website. [28:21.270 --> 28:23.810] If you can't find it, just drop me an email or tweet. [28:24.730 --> 28:27.450] And I will send it your way. [28:37.020 --> 28:38.640] Are we using a microphone? [28:38.940 --> 28:39.780] Or just... [28:41.060 --> 28:41.580] Okay. [28:41.580 --> 28:43.900] So, person standing in the back. [28:52.340 --> 28:53.340] A link... [28:53.340 --> 28:54.700] A link to the... [28:54.700 --> 28:56.400] To the slides as well. [28:56.920 --> 28:57.400] Excellent. [29:04.790 --> 29:05.390] Okay. [29:05.710 --> 29:05.970] Great. [29:06.730 --> 29:08.170] I'm not sure if those are readable. [29:08.530 --> 29:11.550] But if not, I'm happy to place them somewhere else. [29:13.430 --> 29:14.630] Let's see... [29:14.630 --> 29:15.570] Who... [29:15.570 --> 29:15.830] Here? [29:25.580 --> 29:26.180] Okay. [29:26.180 --> 29:27.740] Can you just fire off both words? [29:28.940 --> 29:31.600] We've experimented with both approaches. [29:32.000 --> 29:37.320] At the moment, we look at the header... [29:37.320 --> 29:40.840] The packets that are being sent out on a normal search. [29:41.820 --> 29:43.840] And mimic those exactly. [29:44.440 --> 29:44.500] Right? [29:44.920 --> 29:45.580] So... [29:45.580 --> 29:53.800] And we do tests to see whether we can differentiate between a TrackMeNot request and a Google-type request. [29:54.060 --> 29:57.140] Because obviously, that would be an easy way of filtering out. [29:57.680 --> 29:58.820] That's a good question. [29:59.700 --> 30:01.580] I'll switch sides back and forth. [30:01.800 --> 30:01.980] You here. [30:05.780 --> 30:07.180] Will the... [30:07.180 --> 30:09.840] Will your extension... [30:09.840 --> 30:11.200] Either of them... [30:11.200 --> 30:19.400] Will they interfere with or be interfered by running other extensions such as go through your privacy badger? [30:20.220 --> 30:20.820] Yes. [30:21.440 --> 30:26.860] So, the question was whether you can use these in combination with each... [30:26.860 --> 30:33.620] Like, using Ghostry or AdBlock Plus or whatever in combination with AdNauseam. [30:33.940 --> 30:35.660] And the answer is yes because... [30:36.200 --> 30:37.560] I mean, they will still work together. [30:38.040 --> 30:38.480] But it will... [30:39.000 --> 30:46.820] Depending on which order the actions happen, you may block things that we need to capture the Ad. [30:47.020 --> 30:47.420] Right? [30:47.600 --> 30:49.140] So, then you wouldn't get the Ad. [30:49.420 --> 30:53.340] You'd still get the blocking, but you wouldn't actually collect the Ad. [30:53.680 --> 30:54.380] Follow up. [30:54.500 --> 30:55.660] Is it possible that... [30:55.660 --> 31:07.400] Is it possible to configure it in such a way on any browser that Ghostry is forced to look at the traffic and run after this or not? [31:07.940 --> 31:09.440] That is a good question. [31:09.940 --> 31:10.200] I don't... [31:10.200 --> 31:11.000] Maybe... [31:11.000 --> 31:12.520] I don't know the answer. [31:12.520 --> 31:23.500] The question was whether you can sort of chain extensions together in a way that will enforce an order so that the output of one goes into the input of the next. [31:24.260 --> 31:27.780] And I have never seen it part of the API that does that. [31:28.200 --> 31:29.800] It doesn't mean it doesn't exist. [31:30.020 --> 31:37.740] But my guess is that it certainly doesn't exist across all of the browsers, which is what the features that we're focusing on, right? [31:38.540 --> 31:40.400] Let's go back to this side. [31:41.600 --> 31:42.100] Someone... [31:46.640 --> 31:49.460] Okay, that's the only case we've found so far. [31:51.020 --> 31:57.420] If you're not logged into YouTube, you get the normal behavior of... that you would from an ad blocker. [31:59.300 --> 32:00.320] Someone on this side? [32:00.500 --> 32:02.260] Oh, wait, you asked already, didn't you? [32:02.460 --> 32:02.640] Yeah. [32:02.640 --> 32:04.380] Okay, let's try and get a couple of them. [32:04.540 --> 32:04.780] Go ahead. [32:05.080 --> 32:05.620] All right. [32:05.800 --> 32:16.700] So, I feel like when you interact with these things on the net, often there are privacy-related concerns due to linkage between your accounts where you are in front of them and these other services. [32:17.760 --> 32:18.200] So... [32:19.160 --> 32:26.760] And sometimes when you click on something, you actually trigger additional consequences for accounts that you have with the services that you're already using. [32:26.760 --> 32:38.600] So, how do you ensure that the user who has this installed isn't accidentally, say, spamming their friends with a meet me at blah blah blah or, you know, I like this thing or... [32:38.600 --> 32:42.560] So, the question is, if you were... [32:42.560 --> 32:43.440] Correct me if I'm wrong. [32:43.560 --> 32:48.580] If you were logged into a service like Facebook or something and you clicked on... [32:48.580 --> 32:57.100] auto-clicked on some ad that the... that ad wouldn't then trigger, say, spamming all your Facebook friends. [32:57.360 --> 32:58.000] For example. [32:58.220 --> 32:58.620] Yeah. [32:58.960 --> 32:59.160] Okay. [32:59.440 --> 33:09.240] So, we run the clicks in... the clicks are issued as Ajax requests and so they run in a sandbox. [33:09.240 --> 33:12.740] They're not allowed to run JavaScript, for example. [33:12.740 --> 33:18.340] they are not allowed to run any kind of flash or scripting. [33:18.460 --> 33:24.020] In fact, you know, they are not even embedded in a DOM. [33:25.160 --> 33:35.940] So, then, the folks who are detecting these clicks can distinguish an actual click from an ad nonceum click by simply finding out whether any script was executed, right? [33:37.200 --> 33:37.840] Yes. [33:38.320 --> 33:38.780] That is... [33:39.320 --> 33:40.300] That is possible. [33:40.580 --> 33:41.080] Right. [33:41.080 --> 33:41.940] But they... [33:41.940 --> 33:42.200] Even... [33:42.200 --> 33:42.620] Even... [33:42.620 --> 33:44.080] And this is the question of filtering. [33:44.440 --> 33:50.140] Similarly, with TrackMeNot, there are machine learning approaches that you can potentially use to filter those things. [33:50.300 --> 33:50.680] But... [33:50.680 --> 33:53.020] So, the question is, what does that then do? [33:53.160 --> 34:02.220] First of all, the company has recognized the problem, devoted resources to trying to solve it. [34:02.780 --> 34:05.280] And at some point... [34:05.280 --> 34:09.560] And the business proposition is therefore sort of diminished, right? [34:09.560 --> 34:21.020] So, at some point, if enough people are using this, you know, the hope is that a company might say, well, isn't it easier to just, like, pay attention to what our users want? [34:25.380 --> 34:25.780] Um... [34:25.780 --> 34:26.520] I mean, there... [34:26.520 --> 34:26.680] Yeah. [34:27.020 --> 34:27.220] It's... [34:27.220 --> 34:27.840] It's a good point. [34:28.400 --> 34:28.800] Um... [34:28.800 --> 34:29.300] And... [34:29.300 --> 34:29.820] And... [34:29.820 --> 34:31.000] It's something we've been talking about. [34:33.340 --> 34:33.740] Uh... [34:33.740 --> 34:34.280] This side... [34:34.700 --> 34:36.300] Can you talk a little more about that sandboxing? [34:37.080 --> 34:37.880] Like, what... [34:37.880 --> 34:48.760] If someone's looking at this to exploit it, knowing that someone's running AdNauseam, knowing it's going to click on everything, like, what are some of the protections for you to make sure that you don't accidentally click on some malware? [34:49.540 --> 34:50.880] Well, basically, that's... [34:50.880 --> 34:52.700] That's what I just mentioned. [34:52.900 --> 34:57.880] Because of the nature of how XML, HTTP requests work, um... [34:58.480 --> 35:01.920] None of that stuff actually runs in the context of a... of a browser. [35:01.920 --> 35:07.800] The other way we've experimented doing it is, like, an invisible tab, right? [35:08.160 --> 35:08.660] Uh... [35:08.660 --> 35:15.640] Where you're actually running all that stuff, or running some of it, at least, and just invisibly to the user. [35:15.860 --> 35:18.740] And that has a lot of the, um... [35:18.740 --> 35:19.520] Sort of... [35:19.520 --> 35:21.460] Security considerations that you mentioned. [35:22.700 --> 35:23.260] Um... [35:23.260 --> 35:26.060] Actually, it's not the clicks that are cause... [35:26.060 --> 35:26.920] Are... [35:26.920 --> 35:27.260] The... [35:27.260 --> 35:29.260] At least from my perspective, the concern. [35:29.260 --> 35:31.460] It's the, um... [35:31.460 --> 35:32.260] Running of... [35:33.060 --> 35:33.220] Uh... [35:33.220 --> 35:34.560] The loading of the ad itself. [35:34.980 --> 35:35.260] Right? [35:35.360 --> 35:38.160] Because that's when, uh... [35:38.160 --> 35:39.460] Some tracking can happen. [35:39.860 --> 35:40.100] And... [35:40.100 --> 35:42.440] And, like, something like, um... [35:42.440 --> 35:46.380] There was a recent case of ransomware injected through advertising. [35:46.660 --> 35:46.760] Right? [35:47.340 --> 35:47.940] So... [35:47.940 --> 35:48.320] It's... [35:48.320 --> 35:50.180] It's more the, uh... [35:50.180 --> 35:53.200] Ad loading than the ad clicking that is problematic. [35:53.820 --> 35:54.420] But... [35:54.420 --> 35:57.600] Is the ad actually rendered or loaded? [35:58.300 --> 35:58.840] Um... [35:58.840 --> 35:59.720] Before it's played? [36:01.220 --> 36:01.760] Like... [36:01.760 --> 36:02.360] You said... [36:02.360 --> 36:03.260] You mentioned the sandbox arm. [36:03.740 --> 36:04.280] But... [36:04.280 --> 36:04.880] You see what I'm saying? [36:05.240 --> 36:05.660] Is there... [36:05.660 --> 36:08.460] Is there a get request to the actual ad itself? [36:08.920 --> 36:09.340] Yes. [36:10.000 --> 36:11.750] So, the ad is, uh... [36:12.240 --> 36:13.560] Like, uh... [36:13.560 --> 36:15.560] Ad blockers do in some situations. [36:16.260 --> 36:18.900] In any case, the ad is low... [36:18.900 --> 36:21.020] Like, for all text ads, for example. [36:21.760 --> 36:22.280] Um... [36:22.280 --> 36:23.800] They are loaded and then hidden. [36:28.590 --> 36:29.070] Um... [36:29.070 --> 36:30.610] Do you know any aggregate... [36:30.610 --> 36:31.230] Any, uh... [36:31.230 --> 36:32.530] Obfuscators, um... [36:33.050 --> 36:36.960] That will obfuscate, like, um... [36:36.960 --> 36:39.200] What the timing of your scrolling is, um... [36:39.200 --> 36:40.060] Or when you paginate... [36:40.760 --> 36:42.320] When you click on the next page thing. [36:42.820 --> 36:43.200] Um... [36:43.200 --> 36:47.460] Or obfuscators that will automatically click on, um... [36:47.460 --> 36:49.820] Every article in a news aggregator site. [36:50.000 --> 36:50.680] Or something like that. [36:50.820 --> 36:52.720] Yeah, I don't know of any that exists. [36:52.720 --> 36:55.160] I've been actually thinking about, you know... [36:55.160 --> 36:57.740] There's tracking done on mouse position. [36:58.080 --> 36:59.820] There's tracking done on scroll. [37:00.520 --> 37:00.760] Uh... [37:00.760 --> 37:01.440] Things like that. [37:01.940 --> 37:02.380] Um... [37:02.380 --> 37:03.930] I've been thinking about, you know... [37:04.360 --> 37:05.140] Sort of random... [37:05.140 --> 37:07.040] Sending randomized mouse positions. [37:07.040 --> 37:08.960] When you're not actually using the mouse. [37:09.700 --> 37:10.010] Um... [37:10.320 --> 37:12.220] But I don't know of any that exists so far. [37:12.540 --> 37:13.160] Okay. [37:13.220 --> 37:13.920] Let me hear a question. [37:14.340 --> 37:14.960] Sure. [37:17.040 --> 37:17.660] Uh... [37:17.660 --> 37:22.240] The ad tech industry definition of click fraud would be, uh... [37:22.240 --> 37:24.060] Click on any ad that's not seen by a human. [37:24.720 --> 37:25.460] So, uh... [37:25.460 --> 37:28.100] In the contracts that govern who applies and sells ads. [37:28.380 --> 37:31.900] These ads are ads that shouldn't be paid for if they're clicked on. [37:32.380 --> 37:33.580] That's just, uh... [37:33.580 --> 37:36.600] An aside as far as the conversation about click fraud goes. [37:36.760 --> 37:48.440] But my question is, given so much as move into mobile right now, Are you aware of any tools or efforts that are helping people who are using mobile apps reduce, uh, clicking and, uh... [37:48.440 --> 37:51.140] Sorry, not reduce clicking, but reduce tracking and... [37:51.140 --> 37:55.060] Try to undertake the same type of efforts that you're taking on the desktop web? [37:55.360 --> 37:55.920] Uh... [37:55.920 --> 37:58.100] There's this great, um... [37:58.100 --> 37:59.100] Piece of software. [37:59.580 --> 38:00.240] I can't... [38:00.240 --> 38:00.960] It's called... [38:01.580 --> 38:02.660] Oh, AdNauseam. [38:03.020 --> 38:03.680] And it... [38:03.680 --> 38:05.720] It runs in Firefox mobile. [38:10.460 --> 38:11.020] Uh... [38:11.020 --> 38:11.520] That's... [38:11.520 --> 38:12.100] Right. [38:12.240 --> 38:13.340] People spend more and more time. [38:13.500 --> 38:14.820] And I wonder if there's anything we can do. [38:15.020 --> 38:15.460] Yeah. [38:15.720 --> 38:16.200] I mean, that's... [38:16.200 --> 38:17.940] That's sort of the next frontier, right? [38:18.420 --> 38:19.140] For sure. [38:19.460 --> 38:20.040] I mean, we were... [38:20.040 --> 38:20.860] We just got... [38:20.860 --> 38:24.360] We just got our stuff running on Firefox mobile, which was... [38:24.360 --> 38:25.680] We were happy about. [38:25.920 --> 38:26.400] Uh... [38:26.400 --> 38:29.780] I think uBlock now runs on Firefox mobile as well. [38:30.160 --> 38:30.780] But, yeah. [38:30.860 --> 38:32.640] Outside the browser is another... [38:32.640 --> 38:33.200] Uh... [38:33.200 --> 38:33.760] The next... [38:33.760 --> 38:35.020] The next battleground. [38:35.460 --> 38:35.640] Uh... [38:35.640 --> 38:35.840] To... [38:35.840 --> 38:37.040] To just, uh... [38:37.040 --> 38:39.180] Comment on your click fraud question. [38:39.380 --> 38:42.040] That may be the ad tech's definition of click fraud. [38:42.160 --> 38:45.420] But it is not the legal definition of click fraud. [38:45.540 --> 38:48.160] Which involves intent and... [38:48.160 --> 38:48.740] Uh... [38:48.740 --> 38:50.200] And personal gain. [38:50.760 --> 38:51.260] Uh... [38:51.260 --> 38:51.800] Depending on... [38:51.800 --> 38:52.720] You know, I'm not a lawyer. [38:52.900 --> 38:54.600] But depending on jurisdictions and whatever. [38:55.040 --> 38:55.540] So... [38:55.540 --> 38:55.980] Yes. [38:56.100 --> 38:57.680] I think that... [38:57.680 --> 38:58.460] Uh... [38:58.460 --> 39:00.400] There will be questions about... [39:00.400 --> 39:03.040] Between advertisers and ad networks. [39:03.320 --> 39:03.680] About... [39:03.680 --> 39:04.960] Should we be paying for this? [39:05.120 --> 39:05.800] Of course not. [39:05.920 --> 39:06.660] No one saw it. [39:07.040 --> 39:10.080] And that's exactly the kind of distrust we want to inject. [39:10.280 --> 39:12.080] Into what has been a very... [39:12.080 --> 39:14.980] Friendly lucrative business relationship to this... [39:14.980 --> 39:15.760] To this point. [39:16.000 --> 39:18.100] With the users being the losers in that... [39:18.580 --> 39:18.760] Uh... [39:18.760 --> 39:19.280] Proposition. [39:21.920 --> 39:22.440] Um... [39:22.440 --> 39:23.780] A direct address in that point. [39:23.940 --> 39:25.640] So we can go to another place on the staff. [39:26.340 --> 39:26.740] Um... [39:27.380 --> 39:27.900] To... [39:27.900 --> 39:28.800] Walk... [39:28.800 --> 39:28.860] Um... [39:28.860 --> 39:30.680] AdWords domains and malware domains. [39:31.640 --> 39:31.840] And... [39:31.840 --> 39:32.260] Uh... [39:32.260 --> 39:32.520] Um... [39:32.520 --> 39:32.720] To... [39:32.720 --> 39:33.680] one of your handsets... [39:33.680 --> 39:35.000] Won't allow you... [39:57.200 --> 39:57.880] Uh... [39:57.880 --> 39:59.140] It's not super transparent either. [39:59.280 --> 40:00.180] I don't know what they're blocking. [40:00.460 --> 40:05.180] Yeah, I was gonna say, it depends on how well you trust your VPN provider, of course. [40:05.380 --> 40:06.860] But yeah, that's another solution. [40:07.040 --> 40:16.840] There have also been sort of, you know, the microcomputer, like Raspberry Pi setups, where everything goes, all your traffic goes through that. [40:17.240 --> 40:19.860] And then it, you know, that's running a proxy that filters out. [40:20.320 --> 40:22.600] So there are solutions like that. [40:22.680 --> 40:23.940] And that's like a virtual hotspot. [40:24.060 --> 40:24.660] Right, right, right. [40:24.880 --> 40:25.100] Cool. [40:27.920 --> 40:32.860] I wanted to know about the scope of your intent for the obfuscation. [40:33.040 --> 40:40.120] It seems like you want an operator there using the web browser, and you'll obfuscate their interactions there. [40:40.360 --> 40:46.500] But we all have seen user pattern analysis, so there still needs to be an operator there. [40:46.700 --> 40:53.580] What are your recommendations for obfuscating a user over a full 24-hour period? [40:53.580 --> 41:07.880] Is there a good way to script this tool so that I could leave my computer open, simply have the computer fake search or surf the net, and use this obfuscator to cover up my tracks in a bigger pattern? [41:07.880 --> 41:11.820] Yeah, I don't know of anything that exists like that. [41:12.020 --> 41:21.280] So far, there are softwares to generate random identities, but not ones I know of that actively go out and do stuff. [41:21.280 --> 41:29.500] I've been thinking about that as a possible, you know, combining some of these tools into like a, you know, an agent, right? [41:29.500 --> 41:38.300] Your obfuscatory agent that you say, okay, go, you know, confuse the web about what I'm actually doing. [41:38.700 --> 41:39.780] That would be kind of cool. [41:40.020 --> 41:42.300] I think this person had their hand up first. [41:42.300 --> 41:58.060] Just on the user pattern analysis thing, if you were trying to fully obfuscate the user pattern analysis, wouldn't you have to be doing the maximum possible amount of usage at all times? [41:58.060 --> 42:22.080] So if you ever wanted to watch YouTube videos, whatever frequent, like, you would have to be watching YouTube videos, you know, at least some portion of the time with the obfuscation to be saying, to, you know, be obfuscating when or when you are not using that or other video streaming services? [42:22.340 --> 42:25.300] And could this have bandwidth problems for most users? [42:26.000 --> 42:29.020] Well, this is the resource use question that I brought up. [42:29.120 --> 42:33.160] But I'm not sure that what you said actually follows, right? [42:33.220 --> 42:48.420] If you wanted to obfuscate your video interest, right, all you'd have to do is wait until you were about to watch a YouTube video and then start, like, four YouTube videos at the same time, right? [42:48.560 --> 42:49.760] Or make requests for that. [42:49.940 --> 42:54.440] Of course, there are still bandwidth resource limitations to that. [42:54.440 --> 43:02.800] You know, you... but there are also massive resource questions with advertising in general, right? [43:02.920 --> 43:18.080] If you look at the amount of... if you have a pay-by-data plan on your mobile and you look at how much of that is being devoted to advertisements that you're downloading and running, it's actually quite high. [43:18.080 --> 43:26.580] There was a New York Times article a few months back, I think, where they did a cost analysis. [43:27.080 --> 43:41.580] They went through all the top news websites and did, like, a cost analysis, like, you spent $3 downloading ads when you went to this page and this much bandwidth and things like that. [43:41.580 --> 43:59.540] So, and that's actually one of the things we want to build into the ad vault, which is what the price in bandwidth and in actual money for each of these ads or the aggregate collection of ads would be to give a more tangible sort of metric for that. [44:06.580 --> 44:28.180] Do you think that there's value in obfuscating, like, between multiple users by perhaps creating programs that sort of create an identity, like we were talking about a few questions back, and sort of browse the Internet and create profiles on Facebook and whatnot, [44:28.360 --> 44:37.100] and sort of dilute sort of just the meaning of each user. [44:37.300 --> 44:38.500] I mean, just kind of... [44:38.500 --> 44:53.160] Yeah, so there's an interesting analog case that happened, or may still happen, where, you know, you have these loyalty cards that you take to your pharmacy or your supermarket to get discounts, and of course they track everything you buy over time. [44:53.160 --> 45:00.160] So there are these clubs where people would swap loyalty cards with their friends, right? [45:00.520 --> 45:10.140] And then, you know, you'd still get the discounts, but everyone would... everyone's profile would be sort of diluted by the total group. [45:15.220 --> 45:34.200] I've been thinking about doing a P2P version of obfuscation where you are swapping things like cookies with your sort of group that you're linked to, right? [45:34.200 --> 45:48.000] Of course, you have to be careful about logins and personal details and all that, but, you know, I think that would be an interesting way of solving the believability problem, right? [45:48.120 --> 45:55.420] Because those would be real queries that people were asking, as opposed to machine-generated queries. [46:00.510 --> 46:08.290] I'm still not totally understanding why someone would pick Adnazium instead of ViewBlog or, like, another ad blogger. [46:08.490 --> 46:14.870] Is this sort of like a media-generated awareness hack, or do you really expect that people are going to choose to use this? [46:15.590 --> 46:18.110] Well, I hope that people are going to choose to use this. [46:18.370 --> 46:24.930] Like, TrackMeNot had just about... just under a million downloads in its first two years. [46:24.930 --> 46:36.370] And I think what I would attribute that to is that, you know, you use U-Block and you kind of just disappear from the radar, right? [46:36.690 --> 46:41.030] You are basically... you're not getting ads, you're not showing up in databases. [46:41.730 --> 46:44.090] U-Block, first of all, is a great, great tool. [46:44.310 --> 46:45.210] It's really well-built. [46:45.530 --> 46:47.750] And the guys are doing a great job. [46:47.750 --> 46:50.210] The people who are doing a great job working on that. [46:52.010 --> 47:06.510] But sometimes you actually want to, you know, express a more active role in sort of protesting a system that you don't... that you believe is corrupt, right? [47:06.750 --> 47:18.330] And in this way, TrackMeNot or Adnazium, the companies are going to know that you are annoyed at them. [47:18.330 --> 47:23.390] And it's going to potentially have monetary costs to them, right? [47:23.570 --> 47:31.450] Not just, like, a lost revenue from some missing data, but, like, actual problems they have to deal with. [47:31.790 --> 47:40.470] So, you know, it's the same analogy of, like, safety versus wanting to make a stronger protest. [47:41.850 --> 47:43.610] Does that answer the question? [47:46.500 --> 47:47.020] Yeah. [47:47.280 --> 47:48.840] Follow up, if you... [47:48.840 --> 47:53.920] Well, what would you say most websites can see when you're using ad blockers? [47:54.020 --> 47:56.160] I just want to think that that would be important for it to itself. [47:56.460 --> 47:56.980] Right. [47:57.260 --> 48:03.600] I think it is a light... especially now when ad blocker adoption is getting so high, right? [48:04.540 --> 48:07.000] I think it's a lighter form of protest, right? [48:07.100 --> 48:19.160] You're not directly sort of subverting their economic model and causing this tension between advertisers and ad networks, but you are making a statement as well, I think. [48:19.420 --> 48:19.920] It's a good point. [48:21.960 --> 48:40.200] Does this stop... does this on some websites stop the full-page pop-ups that occur on some news websites when you are using an ad blocker that come from the website saying, we notice you're using an ad blocker, we're not going to let you load our page? [48:40.340 --> 48:48.220] Right, like the wonderful people at Wired or Wall Street Journal or whatever know you get the same. [48:48.220 --> 48:52.720] But there are... so of course this all works by lists, right? [48:52.940 --> 49:04.180] And there are these publicly maintained sort of crowd-sourced lists about different elements that can be blocked, and there are lists that block those things. [49:04.440 --> 49:13.760] So if you go into the subscription manager and check like block ad... anti-ad blockers, then they'll disappear. [49:13.760 --> 49:19.900] Yeah, like I also have... like, it doesn't fool those at all. [49:20.120 --> 49:20.460] Really? [49:21.020 --> 49:37.920] Like, I mean, does... your application does not fool those, AdNauseam won't prevent those from coming up simply by not... simply by not allowing the website to note that you are using that library that you can still look back to that. [49:38.320 --> 49:41.760] In that case, I mean, it's similar to uBlock where you have to... [49:41.760 --> 49:45.720] You are actually still blocking those elements. [49:45.900 --> 49:55.480] Because you're blocking all the trackers and potential malware that are... you know, most of what is downloaded onto a site is not advertising. [49:55.760 --> 49:59.520] It's sort of trackers and beacons and things like that. [49:59.520 --> 50:03.800] So, you still show up as... as a blocker. [50:03.960 --> 50:04.360] Okay. [50:06.460 --> 50:08.640] Does it block also affiliate links? [50:08.940 --> 50:14.280] Say, for instance, you want to purchase an item and give a donation to an affiliate? [50:16.360 --> 50:26.580] So the question was whether it blocks affiliate links, like whether you want to purchase a book related to some article or whatever. [50:26.580 --> 50:30.800] And this is all managed through the list subscription process, right? [50:31.020 --> 50:34.440] So there's a default list which is pretty conservative about what it blocks. [50:34.600 --> 50:35.540] Like it allows Twitter. [50:35.760 --> 50:37.780] It allows Facebook links. [50:37.940 --> 50:44.700] Then you can check off other kinds of lists that you want to add in the interface, which would block things like that. [50:45.180 --> 50:46.820] So it's really up to the user. [50:46.820 --> 50:48.300] And an important thing... [50:48.300 --> 50:49.880] Well, there's two important things, I think. [50:49.980 --> 50:57.400] One is that you can, of course, whitelist any site, any page or any site that you want to support, which I think is important. [50:57.460 --> 51:09.720] And then the new feature, which is in the works now, which is to support the EFFs do not track standard. [51:11.540 --> 51:12.500] Any last... [51:12.500 --> 51:15.100] Maybe we have time for one more question. [51:19.500 --> 51:23.680] Clicking on some percentage that they have to slow... [51:24.180 --> 51:24.660] Right. [51:25.080 --> 51:26.560] Yeah, we've talked about that. [51:26.800 --> 51:33.980] And I think at some point we will add a slider that says, like, how aggressive you want to be in your clicking. [51:33.980 --> 51:49.120] And that would be, like I mentioned, that's sort of a trade-off between how expressive you want to be, right, and how sort of indistinguishable you want to be between actual users. [51:49.300 --> 51:54.600] So you could maybe select, like, click once a day instead of clicking everything. [51:54.600 --> 52:02.180] But in the prototype and still in version two, the default is click everything. [52:04.440 --> 52:05.040] Okay. [52:05.320 --> 52:06.240] I think we're out of time. [52:06.580 --> 52:07.240] Thanks so much. [52:07.380 --> 52:11.040] If you want some stickers, help yourself up here.