Requirements
2 Computers (Physical or Virtual Machines):
Computer 1 – Client (In this example I used Windows 7 64bit Enterprise)
Computer 2 – Proxy (In this example I used Windows XP Pro SP3)
2 Computers (Physical or Virtual Machines):
Computer 1 – Client (In this example I used Windows 7 64bit Enterprise)
Computer 2 – Proxy (In this example I used Windows XP Pro SP3)
Software:
Wireshark (and WinPcap) – Network Analysis
Fiddler – Web Debugging Proxy
OpenSSL – A Great Suite of PKI/SSL Tools
Assuming Internet Explorer 8 on both computers
Download and install the current version of Wireshark along with the included version of WinPcap:
http://www.wireshark.org/download.html
For this example, I used version 1.2.9.
Download and install the current version of Fiddler:
http://www.fiddler2.com/Fiddler2/version.asp
For this example, I used version 2.2.9.7
Download and install the current version of OpenSSL:
http://www.slproweb.com/products/Win32OpenSSL.html
I had to install the Visual C++ 2008 Redistributables to get OpenSSL to install correctly.
For this example, I used version 1.0.0a.
On Both Computers
Setup a Local Certificate Management Console:
Start, Run: mmc
From the Microsoft Management Console:
File, Add/Remove Snap-in…
Click Add…
Select Certificates and click Add
Make sure My User Account is selected and click Finish
Again, make sure Certificates is selected and click Add
This time select Computer Account and click Next
My sure Local Computer is selected and click Finish
Click Close
Click OK
You should now have a Management Console that looks like this:
Optionally, you can save this:
File, Save
Enter a name: Local Certificate Management
Click Save
On the Proxy Computer
Open up Fiddler:
Click Tools, Fiddler Options…
Optionally you can disable HTTP protocol violation warnings. My experience has been that these warnings happen often and are more annoying than useful.
Click on the HTTPS Tab:
Click on the Decrypt HTTPS traffic option
This will bring up a dialogue box to trust Fiddler’s Root Certificate – Click Yes
Note: This will allow you to decrypt SSL sessions without the client browser displaying a certificate error/warning.
Click Yes to add the Fiddler Root Certificate
Next, enable the ignore server certificate errors (unless you want to see the warnings).
Click Export Fiddler Root Certificate to Desktop
Next, click on the Connections Tab
Click Allow remote computers to connect
Note: This is necessary because if you have your browser talk to Fiddler on the same host it will use a loopback/local connection and Wireshark will not be able to see the traffic between the browser and Fiddler. In order to decrypt the SSL traffic, Wireshark must be able to see the traffic between the browser and Fiddler. I accomplish this by having the browser connect to Fiddler from a different computer (the Client computer).
Important Note: In order for Fiddler to accept incoming proxy requests from remote computers, you will need to exit out of Fiddler and then re-start it.
Warning – Once you have this setup, any SSL traffic on this computer will be decrypted with all information such as usernames and passwords visible in Fiddler. Make sure this system is only used for analyzing client SSL traffic and only where you have permission.
Setup target web server:
Open up the Internet Explorer browser and navigate to the web server that needs to be analyzed:
I will use a test web server as an example:
Now, go to the Local Certificate Management Console that you setup earlier.
You will need to hit F5 to refresh the console. After doing this, you should see a Certificates Folder under the Personal Folder for the Current User. In this folder, you should see a certificate for the web server you just went to. Notice that it is issued by Fiddler. Right click on this certificate and select All Tasks, Export…
Click Next, select Yes, Export the Private Key, disable strong protection, leave the password blank, and save the file on your desktop. I called it msappsrv-fiddler.pfx.
Open a Command Prompt
Use the following sequence to extract the private key from the PFX file you just created. In this example, I use the msappsrv-fiddler.pfx file I just created.
Note: If the openssl binary is not in your path you will need to add it or specify the full path – e.g. c:\OpenSSL-Win32\bin\openssl …
Extract the private key from the PFX file:
openssl pkcs12 -in msappsrv-fiddler.pfx -nocerts -out msappsrv-fiddler.ekey
Note 1: The import password should be blank (just hit enter) – this assumes that when you exported the PFX file you didn’t enter a password
Note 2: When it asks for a PEM pass phrase you must enter a password or this won’t work. I use the password: secret
Decrypt the private key:
openssl rsa -in msappsrv-fiddler.ekey -out msappsrv-fiddler.ukey
Note: When it asks for the pass phrase enter the password you just used
Verify the results – the file should look similar to this:
type msappsrv-fiddler.ukey
-----BEGIN RSA PRIVATE KEY-----
MIICXgIBAAKBgQDMyzpyOm+xAR0lzc11JlXZgMQ9Parz6g/4X8Z+Ok/FaHvK4kez
(…)
/7BlxxDuLHhbytM3/Ba1A3VBjYxNqZeHkl3MJrmp2sS6cw==
-----END RSA PRIVATE KEY-----
Create a folder in the root of the C:\ drive called certs and move all the certificate, PFX, and key files to this directory.
Note: This is important - the SSL preferences in Wireshark cannot handle a space in the path. In Windows XP, the Desktop directory is located under “Documents and Settings” and so it will not work.
Open Wireshark
Click Edit, Preferences…
Click on the + box next to Protocols to open the list
Scroll down to and select SSL
For the RSA keys list, enter the following: Local (Proxy) System IP Address, SSL Port, Protocol, and Path to the unencrypted private key
In this example, the local system has an IP Address of 192.168.234.182, the SSL Port is 8888 (the proxy port for Fiddler), the protocol is http, and the path to the private key is c:\certs\msappsrv-fiddler.ukey
So in the RSA keys list I enter: 192.168.234.182,8888,http,c:\certs\msappsrv-fiddler.ukey
For the SSL debug file I use the same directory as the key: c:\certs\ssldebug.log
As soon as you click OK, Wireshark will create the ssldebug log file. If you open it up you should see a successful key load:
From c:\certs\ssldebug.log:
ssl_init keys string:
192.168.234.182,443,http,c:\certs\msappsrv-fiddler.ukey
ssl_init found host entry 192.168.234.182,443,http,c:\certs\msappsrv-fiddler.ukey
ssl_init addr '192.168.234.182' port '443' filename 'c:\certs\msappsrv-fiddler.ukey' password(only for p12 file) '(null)'
Private key imported: KeyID F6:E5:EF:CE:66:A0:D3:62:1E:7C:7C:D3:FF:14:16:99:...
ssl_init private key file c:\certs\msappsrv-fiddler.ukey successfully loaded
association_add TCP port 443 protocol http handle 02E13BF0
Start a network capture on Wireshark on the correct interface.
Note: For prolonged network captures, consider using tshark or dumpcap instead. Also consider using a capture filter to limit the traffic to only what you are interested in.
Before moving on to the client computer I would like to explain why I am using two computers – why not do everything from the same computer? In short, because I couldn’t figure out how to get it to work! In Windows, you cannot capture network traffic that goes through the loopback (local) interface – at least not with WinPcap. In other words, if I have two processes communicating on the same computer, you need to use something else to capture the traffic between them.
Check out the following articles for additional discussion around this as well as alternatives:
http://wiki.wireshark.org/CaptureSetup/Loopback
http://www.hsc.fr/ressources/articles/win_net_srv/missing_loopback.html
While there are tricks and drivers you can install to get around this, the options seem to be somewhat impractical (IMHO) for general use or involve paid software.
On the Client Computer
Open the Internet Explorer Browser
Click Tools, Internet Options
Click on Connections Tab
Setup the browser to use the Proxy System
In this case I configure the proxy as 192.168.234.182
Fiddler listens on port 8888
Copy the FiddlerRoot.cer file from the Proxy Computer to this computer. If you followed the directions, this should be in the C:\certs folder.
Open the Local Certificate Management Console that you setup earlier.
Open Certificates – Current User, Trusted Root Certification Authorities, Certificates
Right click in the Certificate Area on the right and choose All Tasks, Import…
Browse to the FiddlerRoot.cer file
For Certificate Store, make sure place all certificates in the following store is selected with Trusted Root Certification Authorities
Click Yes to the security warning:
You should now see this certificate in your Personal Trusted Root Certification Authority Store
Warning – Once you have trusted this certificate and configured your browser to use the proxy computer, any SSL traffic on this computer can be decrypted with all information such as usernames and passwords visible in on the proxy computer. Make sure this is clearly understood by any users of this system. When you are done capturing traffic for network analysis, you should remove this certificate.
Open the Internet Explorer browser and navigate to the web site to be analyzed:
The site should come up with no errors.
If you see this, something is not configured correctly:
You can verify that Fiddler is doing a man-in-the-middle analysis by looking at the certificate chain:
From Internet Explorer, click on the pad lock located on the right side of the address box and then click View certificates.
In the Certificate dialogue box, select the Certification Path:
When you start an analysis you should clear Internet Explorer’s cache:
Note: Depending on what you are looking at, you may also want to clear cookies, form data, and passwords.
You should also clear the SSL cache from Internet Explorer. In order to analyze an SSL session, the full SSL handshake must be captured. If there is an existing SSL session that is re-used, Wireshark will be unable to decrypt the session (even with the private key).
Finally, close all instances of Internet Explorer on the computer and launch a new instance for the troubleshooting session.
Now, browse to the web site in question and use it as desired noting the time when any problems occur. This is important to correlate the problem event with the corresponding traffic in Wireshark. When finished return to the proxy computer to analyze the SSL session.
Back On the Proxy Computer
Review the capture in Wireshark and verify that it successfully decrypted the SSL session.
You can filter by tcp.port==8888 to focus only on the proxied traffic.
Since the traffic is going to a non-standard port, you will need to highlight one of the frames going to Fiddler on port 8888. Right click on the frame, select Decode As… Make sure the Tab is on Transport, the port is set to 8888 and choose SSL:
In Wireshark, look for the following sequence to see if SSL decryption is working:
Working:
Warning – Once you have this setup, any SSL traffic on this computer will be decrypted with all information such as usernames and passwords visible in Fiddler. Make sure this system is only used for analyzing client SSL traffic and only where you have permission.
Setup target web server:
Open up the Internet Explorer browser and navigate to the web server that needs to be analyzed:
I will use a test web server as an example:
Now, go to the Local Certificate Management Console that you setup earlier.
You will need to hit F5 to refresh the console. After doing this, you should see a Certificates Folder under the Personal Folder for the Current User. In this folder, you should see a certificate for the web server you just went to. Notice that it is issued by Fiddler. Right click on this certificate and select All Tasks, Export…
Click Next, select Yes, Export the Private Key, disable strong protection, leave the password blank, and save the file on your desktop. I called it msappsrv-fiddler.pfx.
Open a Command Prompt
Use the following sequence to extract the private key from the PFX file you just created. In this example, I use the msappsrv-fiddler.pfx file I just created.
Note: If the openssl binary is not in your path you will need to add it or specify the full path – e.g. c:\OpenSSL-Win32\bin\openssl …
Extract the private key from the PFX file:
openssl pkcs12 -in msappsrv-fiddler.pfx -nocerts -out msappsrv-fiddler.ekey
Note 1: The import password should be blank (just hit enter) – this assumes that when you exported the PFX file you didn’t enter a password
Note 2: When it asks for a PEM pass phrase you must enter a password or this won’t work. I use the password: secret
Decrypt the private key:
openssl rsa -in msappsrv-fiddler.ekey -out msappsrv-fiddler.ukey
Note: When it asks for the pass phrase enter the password you just used
Verify the results – the file should look similar to this:
type msappsrv-fiddler.ukey
-----BEGIN RSA PRIVATE KEY-----
MIICXgIBAAKBgQDMyzpyOm+xAR0lzc11JlXZgMQ9Parz6g/4X8Z+Ok/FaHvK4kez
(…)
/7BlxxDuLHhbytM3/Ba1A3VBjYxNqZeHkl3MJrmp2sS6cw==
-----END RSA PRIVATE KEY-----
Create a folder in the root of the C:\ drive called certs and move all the certificate, PFX, and key files to this directory.
Note: This is important - the SSL preferences in Wireshark cannot handle a space in the path. In Windows XP, the Desktop directory is located under “Documents and Settings” and so it will not work.
Open Wireshark
Click Edit, Preferences…
Click on the + box next to Protocols to open the list
Scroll down to and select SSL
For the RSA keys list, enter the following: Local (Proxy) System IP Address, SSL Port, Protocol, and Path to the unencrypted private key
In this example, the local system has an IP Address of 192.168.234.182, the SSL Port is 8888 (the proxy port for Fiddler), the protocol is http, and the path to the private key is c:\certs\msappsrv-fiddler.ukey
So in the RSA keys list I enter: 192.168.234.182,8888,http,c:\certs\msappsrv-fiddler.ukey
For the SSL debug file I use the same directory as the key: c:\certs\ssldebug.log
As soon as you click OK, Wireshark will create the ssldebug log file. If you open it up you should see a successful key load:
From c:\certs\ssldebug.log:
ssl_init keys string:
192.168.234.182,443,http,c:\certs\msappsrv-fiddler.ukey
ssl_init found host entry 192.168.234.182,443,http,c:\certs\msappsrv-fiddler.ukey
ssl_init addr '192.168.234.182' port '443' filename 'c:\certs\msappsrv-fiddler.ukey' password(only for p12 file) '(null)'
Private key imported: KeyID F6:E5:EF:CE:66:A0:D3:62:1E:7C:7C:D3:FF:14:16:99:...
ssl_init private key file c:\certs\msappsrv-fiddler.ukey successfully loaded
association_add TCP port 443 protocol http handle 02E13BF0
Start a network capture on Wireshark on the correct interface.
Note: For prolonged network captures, consider using tshark or dumpcap instead. Also consider using a capture filter to limit the traffic to only what you are interested in.
Before moving on to the client computer I would like to explain why I am using two computers – why not do everything from the same computer? In short, because I couldn’t figure out how to get it to work! In Windows, you cannot capture network traffic that goes through the loopback (local) interface – at least not with WinPcap. In other words, if I have two processes communicating on the same computer, you need to use something else to capture the traffic between them.
Check out the following articles for additional discussion around this as well as alternatives:
http://wiki.wireshark.org/CaptureSetup/Loopback
http://www.hsc.fr/ressources/articles/win_net_srv/missing_loopback.html
While there are tricks and drivers you can install to get around this, the options seem to be somewhat impractical (IMHO) for general use or involve paid software.
On the Client Computer
Open the Internet Explorer Browser
Click Tools, Internet Options
Click on Connections Tab
Setup the browser to use the Proxy System
In this case I configure the proxy as 192.168.234.182
Fiddler listens on port 8888
Copy the FiddlerRoot.cer file from the Proxy Computer to this computer. If you followed the directions, this should be in the C:\certs folder.
Open the Local Certificate Management Console that you setup earlier.
Open Certificates – Current User, Trusted Root Certification Authorities, Certificates
Right click in the Certificate Area on the right and choose All Tasks, Import…
Browse to the FiddlerRoot.cer file
For Certificate Store, make sure place all certificates in the following store is selected with Trusted Root Certification Authorities
Click Yes to the security warning:
You should now see this certificate in your Personal Trusted Root Certification Authority Store
Warning – Once you have trusted this certificate and configured your browser to use the proxy computer, any SSL traffic on this computer can be decrypted with all information such as usernames and passwords visible in on the proxy computer. Make sure this is clearly understood by any users of this system. When you are done capturing traffic for network analysis, you should remove this certificate.
Open the Internet Explorer browser and navigate to the web site to be analyzed:
The site should come up with no errors.
If you see this, something is not configured correctly:
You can verify that Fiddler is doing a man-in-the-middle analysis by looking at the certificate chain:
From Internet Explorer, click on the pad lock located on the right side of the address box and then click View certificates.
In the Certificate dialogue box, select the Certification Path:
When you start an analysis you should clear Internet Explorer’s cache:
Note: Depending on what you are looking at, you may also want to clear cookies, form data, and passwords.
You should also clear the SSL cache from Internet Explorer. In order to analyze an SSL session, the full SSL handshake must be captured. If there is an existing SSL session that is re-used, Wireshark will be unable to decrypt the session (even with the private key).
Finally, close all instances of Internet Explorer on the computer and launch a new instance for the troubleshooting session.
Now, browse to the web site in question and use it as desired noting the time when any problems occur. This is important to correlate the problem event with the corresponding traffic in Wireshark. When finished return to the proxy computer to analyze the SSL session.
Back On the Proxy Computer
Review the capture in Wireshark and verify that it successfully decrypted the SSL session.
You can filter by tcp.port==8888 to focus only on the proxied traffic.
Since the traffic is going to a non-standard port, you will need to highlight one of the frames going to Fiddler on port 8888. Right click on the frame, select Decode As… Make sure the Tab is on Transport, the port is set to 8888 and choose SSL:
In Wireshark, look for the following sequence to see if SSL decryption is working:
Working:
Not working:
If it isn’t working, look at the first Client Hello frame in the capture:
Good:
Good:
Bad - won’t work:
If the first SSL Session in the capture has a Session ID, it means the client is resuming an SSL session and Wireshark won’t be able to decrypt it!
When SSL decryption is working, you should also be able to see what’s going on behind the encryption:
Before decrypting:
After decrypting:
I hope you find this useful!
Credits and References
First of all, I would like to thank Sak Blok. I could not have figured this out without his fantastic presentation on dealing with SSL in Wireshark:
http://www.cacetech.com/sharkfest.09/AU2_Blok_SSL_Troubleshooting_with_Wireshark_and_Tshark.pps
Next, one of the most knowledgeable people and instructors that I know of for Network Analysis is Laura Chappell. Her latest book on Wireshark, Wireshark Network Analysis is invaluable:
http://amzn.com/1893939995
For PKI, I found Brian Komar’s book both comprehensive and illuminating - Windows Server 2008 PKI and Certificate Security:
http://amzn.com/0735625166
Note – it also discusses general PKI and general Windows PKI including XP and up and 2003
For OpenSSL I have just googled the Web – if you have any recommendations on great books for this I would love to hear them.
For Fiddler, I only recently learned about this tool. I looks quite impressive but I am a novice at it. There are some resources on the web site – if you have any you’d recommend, please let me know.
Jim Small
jim dot small at mail dot com
Credits and References
First of all, I would like to thank Sak Blok. I could not have figured this out without his fantastic presentation on dealing with SSL in Wireshark:
http://www.cacetech.com/sharkfest.09/AU2_Blok_SSL_Troubleshooting_with_Wireshark_and_Tshark.pps
Next, one of the most knowledgeable people and instructors that I know of for Network Analysis is Laura Chappell. Her latest book on Wireshark, Wireshark Network Analysis is invaluable:
http://amzn.com/1893939995
For PKI, I found Brian Komar’s book both comprehensive and illuminating - Windows Server 2008 PKI and Certificate Security:
http://amzn.com/0735625166
Note – it also discusses general PKI and general Windows PKI including XP and up and 2003
For OpenSSL I have just googled the Web – if you have any recommendations on great books for this I would love to hear them.
For Fiddler, I only recently learned about this tool. I looks quite impressive but I am a novice at it. There are some resources on the web site – if you have any you’d recommend, please let me know.
Jim Small
jim dot small at mail dot com
