I Tapped That... Tapping a Nationwide Telecommunications Network

by E Squared

For the record, all of this is purely made up and does not actually exist.

This is only a "theoretical" method that a telecommunications provider could use to monitor the network traffic of its subscribers.  On second thought, maybe this really does exist...

For many years, I have navigated the world of IT contracting and made a pretty good living at it.  For someone who has no college degree, I have spent endless hours studying for certification exams, learning how to lock down servers, configuring MPLS tunnels, racking network switches, and the like.  I have always been a technophile but lacked the "proper" education to turn this into a career.  Then one day I found myself working as a temporary employee for an experiment on an Army base.  Because of my willingness to learn, along with a small set of PC skills, I turned that role into a three year odyssey.  From there I gained some experience and certs, eventually landing a position as a network engineer.

As all contract work goes one day, unfortunately sooner than I expected, my contract ended.  I found myself once again back in the job market.  Little did I know that my next position would take me "behind the curtain" of a wireless provider.  I was hired as a network deployment engineer tasked with deploying a voice analysis platform at over 100 sites across the U.S.  This technology was like nothing I had ever worked with before.  I found myself learning the architecture of a 3G/4G nationwide wireless network.

For those of you who have no experience or knowledge of how your cell phone actually works, I recommend looking up a good overall description using Wikipedia or the like.

For me to go over the topology of these different circuit switched versus packet switched networks would take up way too many pages in this fine magazine.  I will touch briefly on the major network elements required for a person's User Equipment (UE) to use the wireless provider's network.  Besides, all of us technology enthusiasts know a thing or two about learning about a new skill/area of expertise using the net as our electronic library.

Mobile wireless networks are made up of two major parts: the Radio Access Network (RAN), and everything else.

The "everything else" part depends on what type of network you are using.  For people with smartphones this is 4G for data and 2G/3G for voice.  This will all change in the near future with the wide deployment of VoLTE which is Voice over LTE.  There are scenarios where a subscriber will get handed off to a legacy network due to tower limitations.  This is called Circuit Switched Fallback (CSFB).  If a user has a 4G handset or device, but the tower is not 4G capable, they will be handed off to the 3G network for all data/voice sessions until they are in the vicinity of a 4G tower.

In order for a provider to understand what is happening on their network, they must install software tools that can provide analysis of the traffic in near real-time.

This is different when you talk about a voice network or the data network.  There are a multitude of different signaling protocols used for both the control plane traffic as well as the user plane traffic.  For the voice side, this is primarily SS7 and SIP.  For the data side this is everything from S1, S11, S5, SGI, and many, many more.  What you need to understand at a basic level is that different network elements communicate with each other using these protocols for different kinds of traffic.

This is where the fun begins.

When I started to learn more about the platform we were installing, I soon understood how much these providers know about what we do with our phones.  Some of these tools are able to actually store and decode phone calls for up to 400 days, depending on storage capacity.  Most of these analysis tools slice the packet and only keep the header information (metadata).  Other tools keep a copy of the entire packet.  Each system usually contains some sort of storage array, with the excess data offloaded to a Storage Area Network (SAN) for later analysis.

But where do these systems get the data from?

I mean, there are thousands of circuits in a provider's network.  This is where the network TAPs come into place.  These are exactly what you think of when you hear the word phone tap.  The only difference is that with the evolution of networking, instead of clamping on a copper wire and reading the electrical impulses with a handset, these are full-fledged rack mountable pieces of hardware.

Depending on the media of the circuit (copper or fiber), some are unpowered passive elements while others are powered, providing active failover so as not to lose any data.  Your normal fiber optic TAP is a small 1U box mounted in a rack that consists of network ports, where the light travels to its intended destination, and monitor/tool ports where part of the light is redirected to an analysis platform.  It does this by using prisms which split the light, commonly in a 60/40 rule, where the 40 percent is sent to the tool port with the 60 percent of light continuing down its intended path.  Once the optical signal is split, this effectively copies the packet.

If there is a large amount of links being tapped, which is the normal scenario, an aggregation switch is used to collect all of these tapped links.

Several vendors provide boxes that do everything from collect the traffic to send to different analysis servers, to place traffic filters in place so the analysis platform only sees the specific type of traffic it needs.  Do a search for [Network Packet Brokers] using the big G and you will find a ton of info on this technology.  These pieces of equipment are probes that process the different voice or data traffic.  Some are passive, only reporting on the traffic.  And some are active, which can reroute or block certain types of traffic.

I have actually been on a troubleshooting call with a vendor while decoded SMS messages flowed across the screen.

Kind of unsettling, huh?  Once this project was about to end, I was approached by management who said that my contract was getting extended because they were installing another analysis platform.  This is where my eyes were really opened.

The previous system I described was for 3G voice analysis.

The next solution the provider purchased was for 3G/4G data.  This is where the mother load of subscriber data resides.  Just as with the previous scenario, network TAPs were installed and the traffic was fed to the new analysis platform.  From what I was told by the vendor, this is the largest deployment of the system in any wireless provider's network in the world.

The system is comprised of a server running Linux with multiple NICs: one for management, one for analysis traffic (which we will call production), an out-of-band console connection, and a fiber NIC for the traffic feed from the TAP aggregation switch.

Another server running Linux connected to the first system by a crossover cable actually processes the traffic flows.  This flow processor includes a storage array which can keep data for up to 400 days.  The last component of this system is a reporting server.  This actually queries a database residing on the other server that processes the traffic.  This reporting server contains a GUI which the user can log into to gain access to a full feature set of functionalities.  A nice Google Maps overlay plots the location of the provider's network along with relevant stats such as subscriber sessions, saturated throughout, TCP loss rate, and total amount of data flowing through a tower.

The most important function of the reporting server is the subscriber forensics it can provide.

This can be as simple as what the top ten mobile applications running on the network are, how many RF connection setups a certain app makes each day, or which mobile app is using the most data.  On and on and on.

There is even a section where the user, identified using their International Mobile Subscriber Identity (IMSI) can be monitored to see how much data they are actually using at a given time.  But why should I care about that?  This is information I get every month in my bill or by using a provider's app in my phone.

Well folks, this might be the case, but what they can now see and run a report on is each and every application you are running on your phone.

A forensics report can tell how much you used BitTorrent and where the traffic went.  Or how many times your Facebook app connected to the network.  This can even list the destination server whose IP address can be identified using a WHOIS lookup.

To me this opens up a whole new study of people and their mobile habits.

I think of it as Bit Level Sociology.  Using these kinds of analysis platforms, one can study millions of people's behavior.  It is a kind of unintentional transcendence.  People use their mobile devices as extensions of themselves.  Providers now know exactly what apps those people use at certain times of the day in order to market more services to those subscribers.  You would be surprised to see the stats on how many people are watching Netflix between the hours of 8 p.m. and 11 p.m. on any given weeknight.  All of this kind of data correlation is done each day.  Don't get me wrong; not all of this is bad.  As a network engineer, I understand the need to see what the network is doing at any given point in time.  This prevents outages and keeps the service up for people like you and me.

What's scary is the level of detail being reported on.

I thought it ironic that the Edward Snowden story broke as I was starting the deployment of these tools in this unnamed provider's network.  I even saw a circuit inventory list that has NSA listed next to the SS7 signaling.

Why would anyone need to scan an entire network when compromising one server can give them the keys to the kingdom?  All a domestic or foreign asset needs to do is place a person on the vendor's forensic service team and the provider is owned.  Data can be exported, reports run, all major network elements listed (with IP addresses), and the provider is none the wiser.

So what can we as users do?

Well, for one, use a VPN service.

All of this type of traffic is reported as encrypted and not subject to analysis like the rest of the mobile app traffic.

Two, uninstall all social network applications on your phone.

These apps, especially Facebook, send multitudes of data to home servers.  If you must use a social network app, use a VPN and the mobile browser using HTTPS to access the service.  This will also just be reported as HTTP/HTTPS traffic.

I was surprised to see the low percentage of overall user data that is in fact encrypted.  Tunnel everything, folks.  Websites like vpngate.net list VPN servers you can use around the world.

OpenVPN even has a great Android app that is free and simple to use.  With the advent of the Raspberry Pi, there are tons of tutorials online that can teach you how to set up your own VPN server in no time.

In closing, I hope to have pulled back the veil on wireless providers' networks a little bit for you.

I hope, in fact, I might have even taught one or two of you guys a thing or two.  I am by no means an expert on cellular networks.  I just know what I have experienced working on these large projects at one.

Magazines like 2600 provide an invaluable service to us all.  We get to read about all kinds of things (some techie, some not) that nobody else is reporting on.

Take what you read in these pages as an informal education.  Heck, it might even lead to a career.

I know it did for me!

Return to $2600 Index