My GSoC Adventure

Noseyparker - 23 July

Alot of time today was spent investigating the nogotofail server daemon and if it has functionality for inspecting unencrypted versions of HTTPS content. I'm now fairly certain the tool doesn't provide this.

I started investigations on how I could add this feature. I wrote a test connection handler to experiment with events the tool provides during the TLS handshake process, as well as HTTPS requests and responses. The event model is comprehensive and most events fired when I expected.

I also looked for examples of simple Python based man-in-the-middle TLS proxies to better understand what is required. Pymiproxy looks like a good candidate - https://github.com/allfro/pymiproxy

Some recent features I've added look like goof additions to the master nogotofail repository. I built a new Debian test server on Google Compute Engine and configured the software - it's working well. Tomorrow I'll add and test these features in a fork of the repository. Hopefully pull requests will follow soon ... to be continued!

Noseyparker - 22 July

My time today was split between 2 tasks:

  • Cleaning up the two TLS connection handlers I recently wrote. In particular I added messages that reported helpful diagnostic information when running in debug mode.
  • Investigating where & how hooks could be added to intercept plain-text HTTPS traffic (before encryption/after decryption).

This investigation is a good learning exercise - I still don't fully understand the design of the nogotofail tool. I'm unsure if has hooks into the Android OS network stack, or if it simply acts as a network proxy. I suspect the later based on testing and reading project documentation.

I'll keep chipping away I feel I'm getting closer ...

Noseyparker - 20 July

Today I completed work on a new feature, detecting negotiated TLS cipher suites that don't support forward secrecy. The TLS connection API is well written and it was easier to implement this than I expected. Here is the connection handler class I wrote: https://github.com/mkenne11/nogotofail-pii/blob/dev/nogotofail/mitm/connection/handlers/connection/noforwardsecrecy.py

Ephemeral Diffie-Hellman (DHE) key exchange implements "perfect forward secrecy" - this technique creates a new public/private key pair for each TLS session, and the private keys are very (very very) difficult to derive even if the (master) certificate private key is disclosed. Without DHE key exchange, if the TLS certificates private key is ever disclosed all previously encrypted messages could be unencrypted (well that's my understanding).

From a brief discussion with the (nogotofail) project's author it doesn't seem to have hooks into Android or TLS stack/s to examine all plain-text HTTPS traffic (before encryption / after decryption). I'll do some investigation into what is needed to implement this functionality, and add this if I have time.

Note. This is an out-of-order post ... I had trouble publishing it a couple of days ago.

Noseyparker - 21 July

Most of today was spent implementing a TLS connection event handler for certificates using the SHA-1 signature algorithm whose an expiry date during or after the Google Chrome sunset period. Google and other online service providers have been pushing to phase out the SHA-1 algorithm due to weaknesses. See Chromium blog post.

The signature algorithm is used to hash the content creating a "message digest". The digest is used to provide message integrity i.e. detect tampering. The digest is also signed with the sessions private-key to produce a HMAC, allowing the receiver to verify the who signed it (using the session public key and some mathematical magic).

Noseyparker - 17 July

Today was a public holiday, so I only spent a couple hours working. I cleaned up the reporting code, to make it more efficient and to use naming consistent with terminology used in nogotofail.

If I have enough time I'd like to do some visualization of reporting data. I have used some Python libraries previously to do visualization of network data. I read about the networkx library - I've heard good things about and I browsed documentation and examples.

Noseyparker - 16 July

Spent alot of time looking at the nogotofail server and client code today. Unfortunately it doesn't appear to provide TLS proxy functionality - the ability to look at an unencrypted form of HTTPS traffic. It will be a shame if it doesn't, it would be a nice feature to look at PII an app sends to both unencrypted and encrypted domains. I have sent a question to the author and will wait to here.

While I'm waiting, I've started a new branch to tidy up, refactor and better comment recent code.

Noseyparker - 15 July

I spent today exploring ways of processing application encrypted (HTTPS) traffic. I experimented with a HTTPS listener class with request and response event methods. I could get the events to fire however I was unable to access HTTP request attributes. Tomorrow I'll look further into this and seek advice from the projects main author.

Noseyparker - 14 July

After completing the data reporting code yesterday, I moved onto the next task, adding handlers to detect PII in encrypted (HTTPS) traffic. To start with, I did some refactoring including: changing the name unencrypted PII handlers to better fit with the new encrypted handler names, moving code used by both groups of handlers into common classes. I added the base code for the encrypted PII handlers however I haven't had a chance to debug it - that's a task for tomorrow.

Noseyparker - 9 July

Most of today I spent updating the final version of the conference paper. I finished the final draft and I hopefully can submit tomorrow, after my supervisor has made her changes.

I did spend some time on the project this afternoon. I tested the application pii data report using some other log files, and came across some bugs. I am working my way through fixing them.

Noseyparker - 7 July

I made steady progress on the application PII data report today. I debugged the initial version of the code and verified the logic is more or less right. After, to verify it worked correctly I processed a couple of other of application log instances.

The main objectives I am trying to achieve with the report:

Syndicate content