How does this wireless authentication stuff work anyway?

My last couple of posts have been calling out some of the issues with wireless authentication on Android.   But, in my day job I deal with authentication on the most common operating systems you would see in the wild.  If you were only to read my blog posts this far, you would probably believe that Android is the worst device for wireless authentication there is.   The truth is, every operating system I have come across does something stupid when it comes to wireless authentication.

But, perhaps the worst component of authentication on wireless networks is the complete lack of understanding that most people have of how it works.   Now, you can argue that you don't need to know how your engine works in order to drive a car, so you don't really need to know how wireless authentication works in order to use your computer.  And, in general, I would agree with you.   However, most people would agree with me that knowing more about how their engine works results in an engine that generally works better because those people know what to look for and what types of behaviors can cause them trouble down the line.   With wireless authentication, it is also the same thing.   However, if you destroy the engine on your car, you will be out a lot of money to fix it or replace the car.   If you screw up with your authentication the damage can be far worse, and far more expensive.   So, do yourself a favor and learn a bit more about how the authentication on your wireless networks work.

My goal in this blog entry is to try to boil down the authentication process in to plain English, while mapping some of the technical jargon in to something that an average person can understand.  As a result, this blog entry will be very long.  But, hang in there with me.  I think it will be worth it.

We are going to look specifically at user name and password based authentication on networks.  This type of authentication is commonly used on enterprise wireless networks.  However, it can also be used on wired networks.  In fact, the method of authentication used in these networks was designed originally to be used on wired networks.  It was adapted to work with wireless networks, and that is where it seems to have found the most success.  This standard is called IEEE 802.1X.  Some people may now be yelling at their screen, "Nuh uh!  WPA-Enterprise or WPA2-Enterprise is used on wireless networks, not this 802.1X stuff!"  Those people are both right and wrong at the same time.   The WPA specifications outline how to use the encryption keys that are generated during an authentication with 802.1X.   So, we will be focusing on 802.1X, and leaving the encryption key stuff for another time.

There are two ways that an authentication starts.   One is that the network recognizes that you have connected to it and sends a request to start the authentication.  The other is the client machine (i.e. your laptop or phone) sends a message to the network requesting that it start the authentication process.  Both methods result in the network sending an identity request to the client station.   The station responds to this request with a message that contains some form of identification.   Because of the way that 802.1X has evolved, this identification is often referred to as the "outer identity" or "anonymous identity".  This terminology is technically questionable, but seems to be what the industry has settled on.  Probably because the "proper" terminology would just be confusing to anyone that doesn't understand how the various authentication methods works.   So, we will use the "outer identity" or "anonymous identity" to reference this through the rest of this blog post.

Following this identity request, the network will send a challenge message to start the authentication.   If you have ever configured authentication for a wireless network, and seen the choices such as (EAP-)PEAP, (EAP-)TTLS, (EAP-)TLS, this is where those settings come in to play.  The challenge message that the network sends includes an identifier that tells the client what type of authentication the network wants to use.  If the client is configured to allow that type of authentication, the process proceeds.   If not, then the client will respond to the network asking for a different type of authentication.   If the network allows the type of authentication the client requests then it sends another challenge, this time identifying the authentication method requested by the client.

At this point, the details of the methods used for authentication diverge wildly depending on which authentication system you use.   But, the basic ideas are all the same.   So, rather than dive in to the details of the various authentication protocols, we will discuss what the goals of the authentication are, and then work backwards.

The network, the client, and the user each have specific goals when they enter the authentication.  Those goals are :

• The networks wants to make sure the client is someone that is allowed to use the network.
• The client wants to make sure that it doesn't provide its credentials to anyone other than the networks that it believes it can trust.
• The user just wants to get on the network, everything else be damned.

So basically, the client and the network want to make sure that both the network and the user are protected.  But, as usual, the user is the weak link as they will probably do just about anything to gain access to the network.  (Facebook statuses won't update themselves!)     This is one of the biggest failures in the design of the various authentication protocols.   For methods such as PEAP or TTLS, the user has the ability to gut the security of the connection as an easy way of gaining access to the network.   

So, if we ignore the desires of the user and focus on the security aspects of the authentication, how do the goals of the network and the client get met?

The first step is for the network to identify itself to the client.   Currently, this is done using a word that strikes terror in to most network administrators.  "Certificates".   Certificates really aren't as scary as most admins think they are, but that is a topic for another blog entry.   For the purposes of this discussion, the network provides a certificate to the client that the client can run some checks on to verify that the network is who it claims to be.  (Side note : There are a lot of potential security issues with certificates and this type of authentication.  But, that is beyond the scope of this post.)   If the client decides it doesn't trust the certificate it will either send a message to the network indicating it doesn't like it, or simply disconnect from the network.   Assuming it does trust the certificate, various pieces of information are used to generate cartographic keys that allow all of the future messages to be encrypted when they cross the network.  This layer of encryption is usually referred to as the "inner tunnel" or sometimes just "the tunnel".

At this point, if the client believes it can trust the network, it will fulfill the desires of the network by proving it should be allowed access to the network.   This is usually done by using a second authentication method "inside the tunnel".   This portion of the authentication is known in most documentation as "phase 2" or "the inner phase".   At this point of the authentication, the client sends the user name and password back to the server using the encryption keys that were generated by the certificate exchange.   It is believed that the encryption set up by the certificate exchange is pretty secure, since it is basically the same type of encryption that is used when you purchase something from a secure web site.   

Taking a quick side trip back to the beginning of the authentication, it should now be more clear why the initial identity the network requests is called the "outer identity".   However, it isn't any more clear why it might be called the "anonymous identity".   The reason it can be called that is because the specifications for PEAP and TTLS explicitly state that the first identity request can be answered with an identity of "anonymous" because the users actual user name will be passed to the network using encryption later in the process.   The idea is that by returning "anonymous" for the first identity it makes it a little harder for a bad guy to use that identity to target a specific user, since the first identity is sent over the network with no encryption.  (Side note : The first identity isn't always sent unencrypted.   There are situations where it is encrypted, but the working assumption in the standards is that it is unencrypted.)

Once the user name and password are sent to the network, the network can decide if it wants to allow the client to access the network.  If the client is allowed, then the network sends a success message and opens the network up to be used.   If the client is not allowed, then the network sends a failure message and blocks the client from using the network.

While I don't want to get in to many of the security aspects of this type of authentication, I do want to point out the weakest link.   There is an assumption that the client will validate the certificate the server provides to it.   There are a lot of places that can go wrong, but by far the biggest one is the user disabling the checking of that certificate at all.   Sadly, some devices like the Nook tablets and Windows RT both default to not checking the certificate, or in the case of the Nook doesn't even allow the user to configure checking the certificate!  (Note : I have not looked at the latest Nooks, so it is possible this issue has been resolved.)  If the client doesn't verify the certificate, then the client will send the user name and password to any device that pretends to be the wireless network the user wants to connect to.   Depending on the authentication method used, this could mean that the password is sent to the bad guy with no encryption, or only lightly hashed.

Hopefully someone finds this post useful.   802.1X authentication can quickly go from something that seems simple enough to a wildly complex set of decisions with potentially huge negative consequences.   In the coming months, I plan to go in to more of a "deep dive" on the different authentication methods and what the known risks are with each of those methods.

Scary Warning Message on KitKat Part 2

Update 1/26/2015 - Google has refused to take this issue seriously.  They have closed several bugs posted on the Android bug tracker.   Fortunately, there are some other people out there that agree that the current implementation is broken.   Unfortunately, Google doesn't seem to think that it is worth fixing.   Since I know a lot of people find this page while looking for information on this annoying warning, I would encourage you to go to the current bug on the Android bug tracker, and star it to show Google that you want this resolved!   Perhaps if enough of us star it, Google will start to pay attention.   You can find the current open bug here.

It always amazes me which posts of mine tend to generate the most traffic.   Not that my blog gets a ton of traffic, but once in a while I get spikes of traffic to a specific post and it surprises me a little.   My previous post on KitKat seems to be one of those that has caused a bit of a spike.   So, I figured a follow up might be interesting.

A couple of hours ago, a coworker sent me a link to Android issue 62076.   Gotta say, I was quite surprised to see a link to my blog as supporting evidence!   I was equally surprised to see that the bug was marked "Works as Intended" and closed.   I read through the statements on how it is intended to work, and came to the conclusion that if it does indeed work that way, this may not be as big a deal as I thought.   But, what was being claimed didn't seem to match up with my experience using KitKat.

For those who don't want to sift through the bug chatter, here is a snip of the response from one of the Android team members :

The "User" portion of the trusted credential store is non-system CA certificates that have been installed and are trusted by the browser and other things that use the system Trusted Certificate Store. This warning is about protecting the user of the device.

Note that EAP-TLS and other Wi-Fi modes do not need to install a CA certificate to the Trusted Certificate Store. If you include your CA certificate in a PKCS#12 bundle when installing the credentials and select "Wi-Fi" as the destination for those credentials, you will not get this warning. You can also create a program that adds Wi-Fi credentials and configurations programmatically using the new WifiEnterpriseConfig API (see http://developer.android.com/reference/android/net/wifi/WifiEnterpriseConfig.html).

Asking for exceptions does not make sense. There is a process for getting a CA into the trusted list of browsers and operating systems. Please see http://www.mozilla.org/projects/security/certs/policy/ for an example of the process that needs to be followed.

This all seems pretty reasonable, although arguing that you can write an app to install certificates in a way that doesn't cause scary messages seems to be a bit of a stretch for most people.   But, lets run with the claim for the PKCS#12 bundle for installing the certificates for a second.

Lets start with a little test.   First, go download a root CA certificate from a known CA provider.  You might need to search around a bit to find one, or you can just use the GeoTrust certificate that I will be using as I run through this test.   I used the first certificate on the list, which is the "Equifax Secure Certificate Authority" certificate.   Next, lets convert that certificate to a PKCS#12 bundle so that we can attempt to install it.   You can either try to figure out the magical incantations to do it using OpenSSL, or you could just hop over to the SSL Shopper.com converter that I will be using.

Whoops.  You mean we have to have a private key to create the bundle that we need to install a root CA in a way that doesn't trigger that scary warning message?!   Awww, c'mon!

A little bit of searching will quickly turn up that a PKCS#12 bundle is intended to store a user certificate and any supporting CA certificates it may need.   Further, even if you could get a PKCS#12 file with just the CA certificates in it, the code in the Android certificate installer would prevent you from using it.  (I have spent FAR too much time digging around in that code.  So you may have to trust me on this one.)

Okay, so lets assume for a second that we are using an EAP-TLS network, and so we can create a PKCS#12 bundle that contains a user certificate and private key.   According to the statement from the bug, we should be able to do this, and as long as we select the "Wi-Fi" store, we should be good to go.   As I happened to have a small home grown CA set up on a Linux box, I tried this method.   After going through the install, I still saw the scary icon in the shade.   But, how could that be?   If the bug was closed because it works as intended, and someone with an @android.com e-mail address says the way it is intended to work shouldn't be showing that scary icon, we must have done something wrong.   Did I maybe forget to select "Wi-Fi" for the target store?   So, I decided to try again.   The first thing I would do is go an clear the "Credential Store" by going to Settings->Security->Clear Credentials.   Then, I tried it again, and again got the icon for the scary warning.   But this time, I made extra sure that I had selected the "Wi-Fi" store like the bug resolution said to.  Interesting...

While poking around with KitKat, I remembered coming across another location that I could use to install certificates.   If you go to Settings->WiFi, select the menu button at the bottom and select "Advanced" there is another option in that menu to install certificates.   But, before we do that, we need to clear out the certificates we had already installed.  So, we go to Settings->Security->Clear Credentials, but the option to clear the credentials isn't available.   Does this mean that the certificates weren't installed?  If you tap on "Trusted Credentials" and then select the "User" tab, you would find that the certificates you had installed are actually listed.   So, either the "Clear Credentials" option has a bug that prevents it from being used when a user has installed certificates to the "Wi-Fi" store, or the certificates are installed someplace other than the "normal" user key store.   The response to the bug indicated they were installed elsewhere, so lets assume that is the case.  (A quick aside.  They are actually stored in the same place as they always were, but they are flagged with a different user ID.   So, it could easily be argued that we are dealing with a bug.  Exactly what the bug is would depend on your view of how things should work, but the fact that historically using "Clear Credentials" has cleared all of the certificates that were installed on the "User" tab would seem to indicate that the same should continue to be true.)

Okay, so the only way to get rid of them appears to be to tap each certificate listed, scroll down and tap the "Remove" button.   Fair enough, lets do that.   Now, we are ready to install certificates using the installed option located in the advanced section of the WiFi settings.   Doing so acts almost the same as installing them through the security menu.  The only difference is that you are not given the option to install to the "Wi-Fi" store.   So, I guess we have to assume that it is going there by default when you install this way.  (Another aside, testing this on the emulator shows that it does go in to the "Wi-Fi store".)

But, look at that, the scary warning is showing up again!

Well crap.   Now what?   Lets try the other option outlined.   Let's write our own code.  But first, lets take a look at this bug.   Uh oh.   Looks like that isn't an option either.   And I won't even bother to point out that the API doesn't allow for certificate chaining.   So, even if it DID work, most people would be outta luck using it anyway.

So there you have it.  This is not a bug.  It works as intended.   Which means that you have *NO* way to access an authenticated wireless network without seeing a scary warning that someone might be monitoring your traffic.


But, before I go, let me relate a story that happened to me a few years ago that, while tangentially related seems like it makes a good warning.   Several years ago, I was on a conference call with a large mobile phone provider from the UK, and higher-ups at a phone manufacturer that had significant market share at the time.  I'm leaving out names because I suspect neither party would be overly happy to be identified publicly.  During this call, the mobile phone provider was asking the manufacturer for APIs to support a product they wanted to deploy.   No matter how they asked, or how they threatened, the manufacturer kept telling the provider that they didn't need those APIs because the product they wanted to use would be obsoleted by something they were working on.   After getting off the call, I talked to my boss (who was also on the call) and said, "Ignoring what your customers want seems to be a good way to kill a company."   In the years since that call, the manufacturer ignored more of what its customers wanted, and its market share has declined to a point that is shameful.   It doesn't matter how big your company is, or how your market share is today, if you ignore what your customers want, you will eventually die.

Since there seems to be some interest in how the wireless authentication all fits together on Android, I plan to post additional information here.   If you want to know far more about the certificate internals of Android than any reasonable person would, you might want to check back.

Android KitKat : Network may be monitored by an unknown third party

Update 1/26/2015 - Google has refused to take this issue seriously.  They have closed several bugs posted on the Android bug tracker.   Fortunately, there are some other people out there that agree that the current implementation is broken.   Unfortunately, Google doesn't seem to think that it is worth fixing.   Since I know a lot of people find this page while looking for information on this annoying warning, I would encourage you to go to the current bug on the Android bug tracker, and star it to show Google that you want this resolved!   Perhaps if enough of us star it, Google will start to pay attention.   You can find the current open bug here.

Let me start by making the disclaimer that these observations are based on an AOSP build of Android KitKat on a Galaxy Nexus.   (My Nexus 5 isn't due for two more days.)   However, things like this that exist in AOSP tend to also exist in the release builds.

Along with all of the great new features in KitKat, Google has introduced what is probably the WORST security hole possible.   A ham-fisted implementation of certificate pinning.

Certificate pinning itself is a good idea.   It verifies that when you visit a web site, it provides you with the same certificate every time.   In general, certificates shouldn't be changing on web sites, and if one does you should be made aware of it.

So, how could certificate pinning be a security hole then?   By creating a situation where a harmless certificate creates a scary, and unnecessary, warning message.  When you install any certificate in to the key store, you get a warning icon in the bar at the top of the screen.   Pulling down the shade presents the following screen :

Let's consider the average user at this point.   Given the revelations about all of the snooping by various governments, it seems that a warning like this would make uninformed users very concerned.   (And, lets face it, there are very few informed users when it comes to 802.1X on Android.)   But, okay, maybe I am freaking out over nothing.  What happens when we tap the warning?

Hmm... Maybe not.   So, the assumption that an uninformed user is to draw based on this is that having any form of third party "trusted credential" installed means that a 3rd party will probably be monitoring my traffic.

I'm going to skip over the obvious irony here that something from Google is warning me that a 3rd party might be monitoring the web sites I visit, and reading my e-mail.  (If the irony is lost on you, you may want to do some research.)

Google seems to be trying to argue that the only safe type of certificate is one that comes pre-installed on your device.   Which is a downright silly argument no matter how you slice it.   But, lets go ahead and let that one slide.   Being the security minded individual that I am, I make sure that all of my network connections are as secure as possible.   So, I make the (probably bad) assumption that purchasing a certificate for my RADIUS server from a public CA will provide me what I need in order to have a secure wireless network.

Once I get the network setup, I try to connect my Android device to the network.   Now, being that we have paid even a little attention to the security issues around wireless networks, we know that we need to validate the server certificate in order to have a secure connection.   No problem, we purchased from a public CA, so we will just select that in the configuration settings.   But, you can't.   The pre-installed certificates on Android can't be used with 802.1X authentication.   Okay, no problem.  I'll just install the CA certificate on my device and then use that.   Oh, what is this scary message?

Now, those of us that understand the meaning of this message will just dismiss it.   But, let's assume that you aren't a techno savvy individual that has enough time to spend learning about security.   This warning is going to freak you out!   If this conversation hasn't happened on a message board yet, it will soon :

"Hey, I got the upgrade to KitKat, and it is great!  But, now I get this warning saying that someone is monitoring my network connection!   How do I make that go away!?  I don't want someone monitoring my network connection!"

"Getting rid of that warning is easy.   Go to Settings->Security->Clear credentials.   After that, the warning will go away."

"Thanks!  The warning is gone.  But now I can't connect to my wifi network!  HELP!"

"Not a problem.  Go in to the configuration for your wifi network and change the 'CA certificate' setting to '(undefined)'.  Problem solved!"

"Perfect!  That solved my problem!  Thanks!"



And somehow Google either didn't consider this case, or they really want to decrease the security of wireless networks.

Anyway, if you happen across this post while looking for how to make this scary looking warning go away, and you use a secure wifi network, please just swipe the warning out of the shade.   It is really nothing to worry about.   (I'll do a follow-up post in the next week or so about how authentication on wifi works and why you should care.)

Edit 11/26/2013 : A coworker pointed out that my blog post was referenced in a bug posted to the Android bug tracker.   The same bug post had a response from someone using an @android.com e-mail address outlining why this issue isn't a problem.   I tested their solutions and wrote about it here.