Requirements
2 Computers (Physical or Virtual Machines):
Computer 1 – Client (In this example I used Windows 7 64bit Enterprise)
Computer 2 – Proxy (In this example I used Windows XP Pro SP3)
2 Computers (Physical or Virtual Machines):
Computer 1 – Client (In this example I used Windows 7 64bit Enterprise)
Computer 2 – Proxy (In this example I used Windows XP Pro SP3)
Software:
Wireshark (and WinPcap) – Network Analysis
Fiddler – Web Debugging Proxy
OpenSSL – A Great Suite of PKI/SSL Tools
Assuming Internet Explorer 8 on both computers
Download and install the current version of Wireshark along with the included version of WinPcap:
http://www.wireshark.org/download.html
For this example, I used version 1.2.9.
Download and install the current version of Fiddler:
http://www.fiddler2.com/Fiddler2/version.asp
For this example, I used version 2.2.9.7
Download and install the current version of OpenSSL:
http://www.slproweb.com/products/Win32OpenSSL.html
I had to install the Visual C++ 2008 Redistributables to get OpenSSL to install correctly.
For this example, I used version 1.0.0a.
On Both Computers
Setup a Local Certificate Management Console:
Start, Run: mmc
From the Microsoft Management Console:
File, Add/Remove Snap-in…
Click Add…
Select Certificates and click Add
Make sure My User Account is selected and click Finish
Again, make sure Certificates is selected and click Add
This time select Computer Account and click Next
My sure Local Computer is selected and click Finish
Click Close
Click OK
You should now have a Management Console that looks like this:
Optionally, you can save this:
File, Save
Enter a name: Local Certificate Management
Click Save
On the Proxy Computer
Open up Fiddler:
Click Tools, Fiddler Options…
Optionally you can disable HTTP protocol violation warnings. My experience has been that these warnings happen often and are more annoying than useful.
Click on the HTTPS Tab:
Click on the Decrypt HTTPS traffic option
This will bring up a dialogue box to trust Fiddler’s Root Certificate – Click Yes
Note: This will allow you to decrypt SSL sessions without the client browser displaying a certificate error/warning.
Click Yes to add the Fiddler Root Certificate
Next, enable the ignore server certificate errors (unless you want to see the warnings).
Click Export Fiddler Root Certificate to Desktop
Next, click on the Connections Tab
Click Allow remote computers to connect
Note: This is necessary because if you have your browser talk to Fiddler on the same host it will use a loopback/local connection and Wireshark will not be able to see the traffic between the browser and Fiddler. In order to decrypt the SSL traffic, Wireshark must be able to see the traffic between the browser and Fiddler. I accomplish this by having the browser connect to Fiddler from a different computer (the Client computer).
Important Note: In order for Fiddler to accept incoming proxy requests from remote computers, you will need to exit out of Fiddler and then re-start it.
Warning – Once you have this setup, any SSL traffic on this computer will be decrypted with all information such as usernames and passwords visible in Fiddler. Make sure this system is only used for analyzing client SSL traffic and only where you have permission.
Setup target web server:
Open up the Internet Explorer browser and navigate to the web server that needs to be analyzed:
I will use a test web server as an example:
Now, go to the Local Certificate Management Console that you setup earlier.
You will need to hit F5 to refresh the console. After doing this, you should see a Certificates Folder under the Personal Folder for the Current User. In this folder, you should see a certificate for the web server you just went to. Notice that it is issued by Fiddler. Right click on this certificate and select All Tasks, Export…
Click Next, select Yes, Export the Private Key, disable strong protection, leave the password blank, and save the file on your desktop. I called it msappsrv-fiddler.pfx.
Open a Command Prompt
Use the following sequence to extract the private key from the PFX file you just created. In this example, I use the msappsrv-fiddler.pfx file I just created.
Note: If the openssl binary is not in your path you will need to add it or specify the full path – e.g. c:\OpenSSL-Win32\bin\openssl …
Extract the private key from the PFX file:
openssl pkcs12 -in msappsrv-fiddler.pfx -nocerts -out msappsrv-fiddler.ekey
Note 1: The import password should be blank (just hit enter) – this assumes that when you exported the PFX file you didn’t enter a password
Note 2: When it asks for a PEM pass phrase you must enter a password or this won’t work. I use the password: secret
Decrypt the private key:
openssl rsa -in msappsrv-fiddler.ekey -out msappsrv-fiddler.ukey
Note: When it asks for the pass phrase enter the password you just used
Verify the results – the file should look similar to this:
type msappsrv-fiddler.ukey
-----BEGIN RSA PRIVATE KEY-----
MIICXgIBAAKBgQDMyzpyOm+xAR0lzc11JlXZgMQ9Parz6g/4X8Z+Ok/FaHvK4kez
(…)
/7BlxxDuLHhbytM3/Ba1A3VBjYxNqZeHkl3MJrmp2sS6cw==
-----END RSA PRIVATE KEY-----
Create a folder in the root of the C:\ drive called certs and move all the certificate, PFX, and key files to this directory.
Note: This is important - the SSL preferences in Wireshark cannot handle a space in the path. In Windows XP, the Desktop directory is located under “Documents and Settings” and so it will not work.
Open Wireshark
Click Edit, Preferences…
Click on the + box next to Protocols to open the list
Scroll down to and select SSL
For the RSA keys list, enter the following: Local (Proxy) System IP Address, SSL Port, Protocol, and Path to the unencrypted private key
In this example, the local system has an IP Address of 192.168.234.182, the SSL Port is 8888 (the proxy port for Fiddler), the protocol is http, and the path to the private key is c:\certs\msappsrv-fiddler.ukey
So in the RSA keys list I enter: 192.168.234.182,8888,http,c:\certs\msappsrv-fiddler.ukey
For the SSL debug file I use the same directory as the key: c:\certs\ssldebug.log
As soon as you click OK, Wireshark will create the ssldebug log file. If you open it up you should see a successful key load:
From c:\certs\ssldebug.log:
ssl_init keys string:
192.168.234.182,443,http,c:\certs\msappsrv-fiddler.ukey
ssl_init found host entry 192.168.234.182,443,http,c:\certs\msappsrv-fiddler.ukey
ssl_init addr '192.168.234.182' port '443' filename 'c:\certs\msappsrv-fiddler.ukey' password(only for p12 file) '(null)'
Private key imported: KeyID F6:E5:EF:CE:66:A0:D3:62:1E:7C:7C:D3:FF:14:16:99:...
ssl_init private key file c:\certs\msappsrv-fiddler.ukey successfully loaded
association_add TCP port 443 protocol http handle 02E13BF0
Start a network capture on Wireshark on the correct interface.
Note: For prolonged network captures, consider using tshark or dumpcap instead. Also consider using a capture filter to limit the traffic to only what you are interested in.
Before moving on to the client computer I would like to explain why I am using two computers – why not do everything from the same computer? In short, because I couldn’t figure out how to get it to work! In Windows, you cannot capture network traffic that goes through the loopback (local) interface – at least not with WinPcap. In other words, if I have two processes communicating on the same computer, you need to use something else to capture the traffic between them.
Check out the following articles for additional discussion around this as well as alternatives:
http://wiki.wireshark.org/CaptureSetup/Loopback
http://www.hsc.fr/ressources/articles/win_net_srv/missing_loopback.html
While there are tricks and drivers you can install to get around this, the options seem to be somewhat impractical (IMHO) for general use or involve paid software.
On the Client Computer
Open the Internet Explorer Browser
Click Tools, Internet Options
Click on Connections Tab
Setup the browser to use the Proxy System
In this case I configure the proxy as 192.168.234.182
Fiddler listens on port 8888
Copy the FiddlerRoot.cer file from the Proxy Computer to this computer. If you followed the directions, this should be in the C:\certs folder.
Open the Local Certificate Management Console that you setup earlier.
Open Certificates – Current User, Trusted Root Certification Authorities, Certificates
Right click in the Certificate Area on the right and choose All Tasks, Import…
Browse to the FiddlerRoot.cer file
For Certificate Store, make sure place all certificates in the following store is selected with Trusted Root Certification Authorities
Click Yes to the security warning:
You should now see this certificate in your Personal Trusted Root Certification Authority Store
Warning – Once you have trusted this certificate and configured your browser to use the proxy computer, any SSL traffic on this computer can be decrypted with all information such as usernames and passwords visible in on the proxy computer. Make sure this is clearly understood by any users of this system. When you are done capturing traffic for network analysis, you should remove this certificate.
Open the Internet Explorer browser and navigate to the web site to be analyzed:
The site should come up with no errors.
If you see this, something is not configured correctly:
You can verify that Fiddler is doing a man-in-the-middle analysis by looking at the certificate chain:
From Internet Explorer, click on the pad lock located on the right side of the address box and then click View certificates.
In the Certificate dialogue box, select the Certification Path:
When you start an analysis you should clear Internet Explorer’s cache:
Note: Depending on what you are looking at, you may also want to clear cookies, form data, and passwords.
You should also clear the SSL cache from Internet Explorer. In order to analyze an SSL session, the full SSL handshake must be captured. If there is an existing SSL session that is re-used, Wireshark will be unable to decrypt the session (even with the private key).
Finally, close all instances of Internet Explorer on the computer and launch a new instance for the troubleshooting session.
Now, browse to the web site in question and use it as desired noting the time when any problems occur. This is important to correlate the problem event with the corresponding traffic in Wireshark. When finished return to the proxy computer to analyze the SSL session.
Back On the Proxy Computer
Review the capture in Wireshark and verify that it successfully decrypted the SSL session.
You can filter by tcp.port==8888 to focus only on the proxied traffic.
Since the traffic is going to a non-standard port, you will need to highlight one of the frames going to Fiddler on port 8888. Right click on the frame, select Decode As… Make sure the Tab is on Transport, the port is set to 8888 and choose SSL:
In Wireshark, look for the following sequence to see if SSL decryption is working:
Working:
Warning – Once you have this setup, any SSL traffic on this computer will be decrypted with all information such as usernames and passwords visible in Fiddler. Make sure this system is only used for analyzing client SSL traffic and only where you have permission.
Setup target web server:
Open up the Internet Explorer browser and navigate to the web server that needs to be analyzed:
I will use a test web server as an example:
Now, go to the Local Certificate Management Console that you setup earlier.
You will need to hit F5 to refresh the console. After doing this, you should see a Certificates Folder under the Personal Folder for the Current User. In this folder, you should see a certificate for the web server you just went to. Notice that it is issued by Fiddler. Right click on this certificate and select All Tasks, Export…
Click Next, select Yes, Export the Private Key, disable strong protection, leave the password blank, and save the file on your desktop. I called it msappsrv-fiddler.pfx.
Open a Command Prompt
Use the following sequence to extract the private key from the PFX file you just created. In this example, I use the msappsrv-fiddler.pfx file I just created.
Note: If the openssl binary is not in your path you will need to add it or specify the full path – e.g. c:\OpenSSL-Win32\bin\openssl …
Extract the private key from the PFX file:
openssl pkcs12 -in msappsrv-fiddler.pfx -nocerts -out msappsrv-fiddler.ekey
Note 1: The import password should be blank (just hit enter) – this assumes that when you exported the PFX file you didn’t enter a password
Note 2: When it asks for a PEM pass phrase you must enter a password or this won’t work. I use the password: secret
Decrypt the private key:
openssl rsa -in msappsrv-fiddler.ekey -out msappsrv-fiddler.ukey
Note: When it asks for the pass phrase enter the password you just used
Verify the results – the file should look similar to this:
type msappsrv-fiddler.ukey
-----BEGIN RSA PRIVATE KEY-----
MIICXgIBAAKBgQDMyzpyOm+xAR0lzc11JlXZgMQ9Parz6g/4X8Z+Ok/FaHvK4kez
(…)
/7BlxxDuLHhbytM3/Ba1A3VBjYxNqZeHkl3MJrmp2sS6cw==
-----END RSA PRIVATE KEY-----
Create a folder in the root of the C:\ drive called certs and move all the certificate, PFX, and key files to this directory.
Note: This is important - the SSL preferences in Wireshark cannot handle a space in the path. In Windows XP, the Desktop directory is located under “Documents and Settings” and so it will not work.
Open Wireshark
Click Edit, Preferences…
Click on the + box next to Protocols to open the list
Scroll down to and select SSL
For the RSA keys list, enter the following: Local (Proxy) System IP Address, SSL Port, Protocol, and Path to the unencrypted private key
In this example, the local system has an IP Address of 192.168.234.182, the SSL Port is 8888 (the proxy port for Fiddler), the protocol is http, and the path to the private key is c:\certs\msappsrv-fiddler.ukey
So in the RSA keys list I enter: 192.168.234.182,8888,http,c:\certs\msappsrv-fiddler.ukey
For the SSL debug file I use the same directory as the key: c:\certs\ssldebug.log
As soon as you click OK, Wireshark will create the ssldebug log file. If you open it up you should see a successful key load:
From c:\certs\ssldebug.log:
ssl_init keys string:
192.168.234.182,443,http,c:\certs\msappsrv-fiddler.ukey
ssl_init found host entry 192.168.234.182,443,http,c:\certs\msappsrv-fiddler.ukey
ssl_init addr '192.168.234.182' port '443' filename 'c:\certs\msappsrv-fiddler.ukey' password(only for p12 file) '(null)'
Private key imported: KeyID F6:E5:EF:CE:66:A0:D3:62:1E:7C:7C:D3:FF:14:16:99:...
ssl_init private key file c:\certs\msappsrv-fiddler.ukey successfully loaded
association_add TCP port 443 protocol http handle 02E13BF0
Start a network capture on Wireshark on the correct interface.
Note: For prolonged network captures, consider using tshark or dumpcap instead. Also consider using a capture filter to limit the traffic to only what you are interested in.
Before moving on to the client computer I would like to explain why I am using two computers – why not do everything from the same computer? In short, because I couldn’t figure out how to get it to work! In Windows, you cannot capture network traffic that goes through the loopback (local) interface – at least not with WinPcap. In other words, if I have two processes communicating on the same computer, you need to use something else to capture the traffic between them.
Check out the following articles for additional discussion around this as well as alternatives:
http://wiki.wireshark.org/CaptureSetup/Loopback
http://www.hsc.fr/ressources/articles/win_net_srv/missing_loopback.html
While there are tricks and drivers you can install to get around this, the options seem to be somewhat impractical (IMHO) for general use or involve paid software.
On the Client Computer
Open the Internet Explorer Browser
Click Tools, Internet Options
Click on Connections Tab
Setup the browser to use the Proxy System
In this case I configure the proxy as 192.168.234.182
Fiddler listens on port 8888
Copy the FiddlerRoot.cer file from the Proxy Computer to this computer. If you followed the directions, this should be in the C:\certs folder.
Open the Local Certificate Management Console that you setup earlier.
Open Certificates – Current User, Trusted Root Certification Authorities, Certificates
Right click in the Certificate Area on the right and choose All Tasks, Import…
Browse to the FiddlerRoot.cer file
For Certificate Store, make sure place all certificates in the following store is selected with Trusted Root Certification Authorities
Click Yes to the security warning:
You should now see this certificate in your Personal Trusted Root Certification Authority Store
Warning – Once you have trusted this certificate and configured your browser to use the proxy computer, any SSL traffic on this computer can be decrypted with all information such as usernames and passwords visible in on the proxy computer. Make sure this is clearly understood by any users of this system. When you are done capturing traffic for network analysis, you should remove this certificate.
Open the Internet Explorer browser and navigate to the web site to be analyzed:
The site should come up with no errors.
If you see this, something is not configured correctly:
You can verify that Fiddler is doing a man-in-the-middle analysis by looking at the certificate chain:
From Internet Explorer, click on the pad lock located on the right side of the address box and then click View certificates.
In the Certificate dialogue box, select the Certification Path:
When you start an analysis you should clear Internet Explorer’s cache:
Note: Depending on what you are looking at, you may also want to clear cookies, form data, and passwords.
You should also clear the SSL cache from Internet Explorer. In order to analyze an SSL session, the full SSL handshake must be captured. If there is an existing SSL session that is re-used, Wireshark will be unable to decrypt the session (even with the private key).
Finally, close all instances of Internet Explorer on the computer and launch a new instance for the troubleshooting session.
Now, browse to the web site in question and use it as desired noting the time when any problems occur. This is important to correlate the problem event with the corresponding traffic in Wireshark. When finished return to the proxy computer to analyze the SSL session.
Back On the Proxy Computer
Review the capture in Wireshark and verify that it successfully decrypted the SSL session.
You can filter by tcp.port==8888 to focus only on the proxied traffic.
Since the traffic is going to a non-standard port, you will need to highlight one of the frames going to Fiddler on port 8888. Right click on the frame, select Decode As… Make sure the Tab is on Transport, the port is set to 8888 and choose SSL:
In Wireshark, look for the following sequence to see if SSL decryption is working:
Working:
Not working:
If it isn’t working, look at the first Client Hello frame in the capture:
Good:
Good:
Bad - won’t work:
If the first SSL Session in the capture has a Session ID, it means the client is resuming an SSL session and Wireshark won’t be able to decrypt it!
When SSL decryption is working, you should also be able to see what’s going on behind the encryption:
Before decrypting:
After decrypting:
I hope you find this useful!
Credits and References
First of all, I would like to thank Sak Blok. I could not have figured this out without his fantastic presentation on dealing with SSL in Wireshark:
http://www.cacetech.com/sharkfest.09/AU2_Blok_SSL_Troubleshooting_with_Wireshark_and_Tshark.pps
Next, one of the most knowledgeable people and instructors that I know of for Network Analysis is Laura Chappell. Her latest book on Wireshark, Wireshark Network Analysis is invaluable:
http://amzn.com/1893939995
For PKI, I found Brian Komar’s book both comprehensive and illuminating - Windows Server 2008 PKI and Certificate Security:
http://amzn.com/0735625166
Note – it also discusses general PKI and general Windows PKI including XP and up and 2003
For OpenSSL I have just googled the Web – if you have any recommendations on great books for this I would love to hear them.
For Fiddler, I only recently learned about this tool. I looks quite impressive but I am a novice at it. There are some resources on the web site – if you have any you’d recommend, please let me know.
Jim Small
jim dot small at mail dot com
Credits and References
First of all, I would like to thank Sak Blok. I could not have figured this out without his fantastic presentation on dealing with SSL in Wireshark:
http://www.cacetech.com/sharkfest.09/AU2_Blok_SSL_Troubleshooting_with_Wireshark_and_Tshark.pps
Next, one of the most knowledgeable people and instructors that I know of for Network Analysis is Laura Chappell. Her latest book on Wireshark, Wireshark Network Analysis is invaluable:
http://amzn.com/1893939995
For PKI, I found Brian Komar’s book both comprehensive and illuminating - Windows Server 2008 PKI and Certificate Security:
http://amzn.com/0735625166
Note – it also discusses general PKI and general Windows PKI including XP and up and 2003
For OpenSSL I have just googled the Web – if you have any recommendations on great books for this I would love to hear them.
For Fiddler, I only recently learned about this tool. I looks quite impressive but I am a novice at it. There are some resources on the web site – if you have any you’d recommend, please let me know.
Jim Small
jim dot small at mail dot com
Interesting post and thanks for sharing. Some things in here I have not thought about
ReplyDeletebefore.Thanks for making such a cool post which is really very well written.
will be referring a lot of friends about this.Keep blogging.
Get Wireshark
Hey really good stuff!
ReplyDeleteI am using wireshark to decrypt TLS traffic. I have followed the steps mentioned in your post. However, I am unable to decrypt the dump. Also, according to the debug file, the key has been loaded successfully. I couldn't make any sense of the rest of data in the debug file. It is flooded with SSL_dissect messages.
Finally got it working. The server was using Diffie Hellman key exchange based ciphers. As it is known that the communication encrypted using these ciphers currently cannot be decrypted using wireshark.
DeleteI changes the cipher suite at the server by making changes in server.xml to exclude the ciphers based on DHE. With the dump captured after this, I was able to successfully decrypt using the above steps.
Thanks!
Same here with TLSv1 - on Client Hello the Session ID Length is 0, on Server Hello it's 32.
ReplyDeleteThe actual packets are shown as TLSv1 with "Encrypted Data".
Log shows successful key loading, but on first two SSL messages:
dissect_ssl enter frame #151 (first time)
ssl_session_init: initializing ptr 0503AF18 size 588
conversation = 0503AA7C, ssl_session = 0503AF18
record: offset = 0, reported_length_remaining = 178
dissect_ssl3_record: content_type 22 Handshake
decrypt_ssl3_record: app_data len 173, ssl state 0x00
association_find: TCP port 13550 found 00000000
packet_from_server: is from server - FALSE
decrypt_ssl3_record: using client decoder
decrypt_ssl3_record: no decoder available
dissect_ssl3_handshake iteration 1 type 1 offset 5 length 169 bytes, remaining 178
packet_from_server: is from server - FALSE
ssl_find_private_key server 69.171.187.101:443
dissect_ssl3_hnd_hello_common found CLIENT RANDOM -> state 0x01
dissect_ssl enter frame #152 (first time)
conversation = 0503AA7C, ssl_session = 0503AF18
record: offset = 0, reported_length_remaining = 1460
dissect_ssl3_record found version 0x0301(TLS 1.0) -> state 0x11
dissect_ssl3_record: content_type 22 Handshake
decrypt_ssl3_record: app_data len 81, ssl state 0x11
packet_from_server: is from server - TRUE
decrypt_ssl3_record: using server decoder
decrypt_ssl3_record: no decoder available
dissect_ssl3_handshake iteration 1 type 2 offset 5 length 77 bytes, remaining 86
dissect_ssl3_hnd_hello_common found SERVER RANDOM -> state 0x13
dissect_ssl3_hnd_srv_hello found CIPHER 0x0005 -> state 0x17
dissect_ssl3_hnd_srv_hello trying to generate keys
ssl_generate_keyring_material not enough data to generate key (0x17 required 0x37 or 0x57)
dissect_ssl3_hnd_srv_hello can't generate keyring material
record: offset = 86, reported_length_remaining = 1374
need_desegmentation: offset = 86, reported_length_remaining = 1374
Any hints?
I am simply out of words after reading your article. I want to appreciate the way you handled such a complicated subject...Network Traffic Analyzer
ReplyDeleteI am searching for that kind of blog..It is really good information shared..
ReplyDeleteIT Dienstleistungen Ulm
nice post see this as well :How to Decrypt SSL traffic using Wireshark
ReplyDeletewifi hacker app
ReplyDeletegoogle play store on pc
how to do anything
how to downgrade to windows 7
This comment has been removed by the author.
ReplyDeleteWe SVJ Technocoat are the leading Service Provider and Exporter of an extensive array of PVD Coating In Surat Service and Vapor Deposition Coating Service etc. We are a well known firm for providing excellent quality coating services across the nation and in a timely manner. Owing to our improvised business models, our professionals are offering integrated solutions for our clients
ReplyDeleteShreeja Oil Maker is a global leading manufacturer, wholesaler, supplier, exporter of various small scale machinery. Shreeja oil maker machine or Mini oil extraction machine is one of our innovative product which is able to extract 100% pure oil from the various seed. This is also known as a Oil Extraction Machine Manufacturer or mini oil Ghani. We have a stronghold in national as well as a worldwide market.
ReplyDeleteFAB VOGUE STUDIO provide all kind of Fabric which can be use for Making any designer apparel. Kota Doria Fabric We guarantee you’ll love working with us. Design Customization Available. Wholesale price available on request. Quality Assurance. COD Available.
ReplyDelete창원구출장샵 서원구출장샵 서북구출장샵
ReplyDelete출장마사지
ReplyDelete출장마사지
출장마사지
출장마사지
출장마사지
출장마사지
We are grateful to you for assisting us by giving the information regarding the project that you had. Your efforts allowed us to better understand the consumer while while saving us time. Does your business need a Bulk WhatsApp Sender service? Is it becoming hard for your business to meet your targeted audience? Absolutely, yes, or you may be looking to be advanced in the market! You have come to the right place to have the service for your business. GetItSMS lets you meet your potential customers by providing you with the best WhatsApp bulk SMS service in India. You can take the service for any purpose to approach your potential customers.
ReplyDelete