pages tagged sslENETDOWNhttp://enetdown.org//tag/ssl/ENETDOWNikiwiki2012-10-20T15:47:58Zcapturing HTTPS proxy requests with Firefoxhttp://enetdown.org//dot-plan/posts/2012/10/19/capturing_https_proxy_requests_with_firefox/zaki2012-10-20T15:47:58Z2012-10-19T22:12:52Z
<p>In one of my projects, I needed to trace HTTPS requests in order to understand
the behaviour of a web application. Since the data is encrypted, it can not be
read using the default configuration of the tools that I normally use to
inspect network data. This post details how to quickly set up an SSL proxy to
monitor the encrypted traffic.</p>
<h1 id="background">Background</h1>
<p>When debugging or reverse engineering a network protocol, it is often necessary
to look at the requests being made in order to see where they are going and what type of
parameters are being sent. Usually this is simple with a packet capture tool
such as <a href="http://www.tcpdump.org/">tcpdump</a> or
<a href="http://www.wireshark.org/">Wireshark</a> when the protocol is being sent in
plaintext; however, it is more work to capture and decode SSL packets as the purpose of
this protocol layer is to prevent the type of eavesdropping that is
accomplished in man-in-the-middle attacks. SSL works on the basis of public
certificates that are issued by trusted organisations known as certificate
authorities (CAs). The purpose of the CAs is to sign certificates so that any
clients that connect to a server that has a signed certificate can trust that they
are connecting to an entity with verified credentials; therefore, a certificate
can only be as trustworthy as the CA that signed it. Operating systems and
browsers come with a list of CA certificates that are considered
trustworthy by consensus; so, in order to run a server with verifiable SSL
communication, the owners of that server need to get their certificate signed
by one of the CAs in that list. Any traffic to and from that server will now be
accepted by the client in encrypted form.</p>
<h1 id="software">Software</h1>
<p>Trying to capture these encrypted packets would require you to have the private
key of a trusted CA; however, we can get around this by installing our own CA
certificate and using a proxy that signs certificates using that CA certificate
for every server that the client connects to. We can accomplish this by using
<a href="http://crypto.stanford.edu/ssl-mitm/">mitm-proxy</a>.</p>
<p><a href="http://enetdown.org//dot-plan/posts/2012/10/19/gfx/mitm-proxy/"><img src="http://enetdown.org//dot-plan/posts/2012/10/19/capturing_https_proxy_requests_with_firefox/400x-mitm-proxy.png" width="400" height="416" alt="SSL Man in the Middle Proxy software website" class="img" /></a></p>
<p>It is written Java and comes with a CA certificate that you can use right away
which makes it is straightforward to set up. </p>
<p>Once you download and extract the software, you have to add the fake CA
certificate into Firefox. I prefer to set up a new session of Firefox so that
the configuration will use a separate database of certificates from my usual
browsing session. You can create and start a new session using the command</p>
<pre><code>firefox -P -no-remote
</code></pre>
<p>.</p>
<h1 id="installingthecertificate">Installing the certificate</h1>
<p>When this new session starts up, add the certificate by going into the
Preferences menu of Firefox and going to the options under <code>Advanced » Encryption</code>
and selecting the <code>View Certificates</code> button.</p>
<p><a href="http://enetdown.org//dot-plan/posts/2012/10/19/gfx/preferences/"><img src="http://enetdown.org//dot-plan/posts/2012/10/19/capturing_https_proxy_requests_with_firefox/400x-preferences.png" width="400" height="366" alt="Preferences option for certificates" class="img" /></a></p>
<p>Under the <code>Authorities</code> tab, click the <code>Import</code> button and select the file
<code>FakeCA.cer</code> from the <code>mitm-proxy</code> directory.</p>
<p><a href="http://enetdown.org//dot-plan/posts/2012/10/19/gfx/add_cert0/"><img src="http://enetdown.org//dot-plan/posts/2012/10/19/capturing_https_proxy_requests_with_firefox/400x-add_cert0.png" width="400" height="277" alt="List of CA certificates" class="img" /></a></p>
<p>Once you add the certificate for identifying websites, you should see it in the
list of authorities under the item name <code>stanford</code>.</p>
<p><a href="http://enetdown.org//dot-plan/posts/2012/10/19/gfx/add_cert1/"><img src="http://enetdown.org//dot-plan/posts/2012/10/19/capturing_https_proxy_requests_with_firefox/400x-add_cert1.png" width="400" height="199" alt="Adding a fake CA for websites" class="img" /></a></p>
<p><a href="http://enetdown.org//dot-plan/posts/2012/10/19/gfx/add_cert2/"><img src="http://enetdown.org//dot-plan/posts/2012/10/19/capturing_https_proxy_requests_with_firefox/400x-add_cert2.png" width="400" height="277" alt="Check if the fake CA is added under 'stanford'" class="img" /></a></p>
<h1 id="runningtheproxy">Running the proxy</h1>
<p>You are now ready to run the proxy. A shell script called <code>run.sh</code> is contained
in the <code>mitm-proxy</code> directory and by examining it<a href="http://enetdown.org//tag/ssl/#fn:shell_script_security" id="fnref:shell_script_security" class="footnote">1</a>, you
can see that it starts a proxy on localhost:8888 using the fake CA certificate and
that it will log the HTTPS requests to <code>output.txt</code>. You need to add this proxy
to your Firefox instance by going to <code>Advanced » Network » Settings</code> and adding
the information under the SSL proxy configuration.</p>
<p><a href="http://enetdown.org//dot-plan/posts/2012/10/19/gfx/config_ssl_proxy/"><img src="http://enetdown.org//dot-plan/posts/2012/10/19/capturing_https_proxy_requests_with_firefox/400x-config_ssl_proxy.png" width="400" height="365" alt="Network configuration for the SSL proxy" class="img" /></a></p>
<p>Once you start the server, you can test it by going to
<a href="https://httpsnow.org/">HTTPSNow</a>, a website that promotes HTTPS usage for
secure browsing. Now, by running</p>
<pre><code>tail -f output.txt
</code></pre>
<p>, you can see the HTTPS requests and responses as they are sent.</p>
<p><a href="http://enetdown.org//dot-plan/posts/2012/10/19/gfx/run_mitm-proxy/"><img src="http://enetdown.org//dot-plan/posts/2012/10/19/capturing_https_proxy_requests_with_firefox/400x-run_mitm-proxy.png" width="400" height="214" alt="Start mitm-proxy using shell script" class="img" /></a></p>
<p><a href="http://enetdown.org//dot-plan/posts/2012/10/19/gfx/https_log/"><img src="http://enetdown.org//dot-plan/posts/2012/10/19/capturing_https_proxy_requests_with_firefox/400x-https_log.png" width="400" height="214" alt="Log of HTTPS output for httpsnow.org" class="img" /></a></p>
<div class="footnotes">
<hr />
<ol>
<li id="fn:shell_script_security"><p>You should examine all shell scripts you download for
security reasons. You do not want to inadvertently delete your $HOME directory!<a href="http://enetdown.org//tag/ssl/#fnref:shell_script_security" class="reversefootnote"> ↩</a></p></li>
</ol>
</div>