This afternoon I completed testing the migration of the 2 Eclipse projects to Android Studio (mitm client and test harness). The APKs built cleanly and tested as expected on my Nexus tablet. After looking closely at the IDE project configuration files I saw some values specific to my workstation (added after migration) which I removed from Git.
Additional updates were made to the user documentation. I think it's easier to follow now, however the proof will be when someone else tries to follow the instructions.
At this point, I decided the the project was at a point where I could wrap-it-up for the Summer of Code. Here is the link to the release repo snapshot I created for the final GSoC code: https://github.com/mkenne11/nogotofail-pii/releases/tag/soc-final
This isn't the end for the project - just a milestone. There are bugs and enhancements I will continue to work through as well as new features. Unfortunately there is a lack of privacy testing tools available for mobile applications (and software in general) and hope this project can make some contribution to address this and improve the privacy awareness of developers.
I'm discussing with the parent project main author the possibility of merging PII detection and other security functions into the parent project.
Thanks so much to my supervisor (Bart Massey) for all his help, encouragement and the opportunity to be sponsored on this project! It's been such a great opportunity!! Thanks also to the nogotofail parent project author (Chad) has also been very helpful and encouraging!!
Well time to pack my bags. I've been studying in Malaysia for the last year which has been an unreal life-changing experience. Met so many great people/had so many awesome meals/and so many great experiences! Malaysia Boleh!!!
I'll be sad to leave Malaysia behind but it will be nice to get back to Australia as well. I wish everyone the best of luck in the future and all your endeavors!!
I found the code causing the intermittent bug for HTTPS responses with no header fields, and corrected the problem. This was fixed by adding error handling code to deal with the exception raised when no headers were found.
For my remaining time today I migrated the two Android Eclipse projects to Android Studio projects i.e. the nogotofail client and test-harness apps. I decided to make this change as Google has designated Android Studio as the official Android development tool, and will soon stop supporting Eclipse (Android Development tools): http://android-developers.blogspot.com/2015/06/an-update-on-eclipse-android-developer.html
The migration of the Android projects was simpler than I expected.
I spent this morning working on user documentation. More content was added to the Event Summary Page, describing how to run the report and the report format. Other small changes were made to the user documentation.
I'll read through the documentation tomorrow with fresh eyes and make any improvements needed.
This afternoon I looked at an intermittent bug which seems to occur when HTTP/HTTPS responses have no header fields. I have added debug messages but haven't identified the root cause yet.
I added code to generate JSON summary reports from the command line today. This allows a user to manually generate these reports when needed. Included are options to specify the report to generate, folder to output the report and paths for the logs to read. At the moment the "PII data report" and "Event summery report" can be generated.
See merged change for futher details: https://github.com/mkenne11/nogotofail-pii/blob/e03d03807e476ad68afa613bdf48d7b56f484358/nogotofail/mitm/report/generate_report.py
Tomorrow I will add user documentation describing how to use the report generation function. I will also explore whether this function should be auto triggered when the user stops the mitm daemon.
Today was spent tidying up code mostly - removing redundant code and debug messages.
I am also looking for the best method to trigger the PII summary reporting. Ideally I'd like it to be automatically run when the MitM daemon is stopped. Unfortunately I wasn't able to find an event in the code which captures this.
Another option is to make the summary reporting function accessible from the command line, and call it from the main application shell script method after the service is stopped.
Today's was spent improving user documentation as well as adding additional issues that need to be addressed i.e. defects and enhancements.
I also had some more problems with the Android Studio IDE and a project files being corrupted. I searched knowledge bases on the web but couldn't identify the cause. I'm considering reinstalling it.
I completed work on the test harness encrypted (HTTPS) PII test cases. The cases were easy to write, however the IDE caused drama's and stopped building the APK. After searching stackoverflow it appears the problem was caused by a corrupted project index file. I fixed the problem using the recommended method - rebuilding the project from scratch. In my case I imported the source Eclipse project into Android Studio, and applied recent changes (determined using a git-diff).
Finishing this change means I've addressed all of the main objectives in my GSoC project submission (I think).
Later in the afternoon, I created a new branch to work on user documentation improvements and start working though issues in the project issues list: https://github.com/mkenne11/nogotofail-pii/issues
I had more luck today, and quickly found the issue causing the Android test harness to crash. The application was trying to fetch and use test PII values before they were available. Improved error messaging assisted in finding this problem.
I didn't have a chance to do much coding today, the business of the last few days caught up with me. Tomorrow I will cleanup the test harness code, I hope to do some refactoring to make the application more manageable.
I spent some time improving the PII analysis user documentation I've written recently. The documentation is a good intial description of PII analysis functions and I merged this into the dev branch.
My next task is to add cases to the client test harness for testing "httpspii" scenarios. I created a new branch for this (httpspii_test_harness) and hope to start work on this tomorrow.
I presented my paper at the conference this afternoon, and think I went ok. I had good questions from the audience and plan to meet with one of the PhD students from my faculty next week who is doing a project which could overlap with mine.
Most of today was spent working on the HTTPS PII test cases in the Android test harness. I did break my no 1 rule when developing - make small changes and build and test often ... and now I am paying for it. I was over confident and felt I know the Android test app very well, and made a heap of changes before I built the APK and tested it.
I find Android applications tricky. On a few occasions the IDE has flagged no problems and the app has built cleanly, however when I've run the application in my test setup the application has crashed. On most occassions I've tracked the issue down to code which doesn't look like it should be an issue (it could be my lack of knowledge though).
I'll keep experimenting - this could be a good chance to try and test the app in the (Android Studio) IDE app emulator. The difficulty is I need to emulate the MitM app also plus setup a OpenVPN connection to the server. A another simpler option is to add better error handling and messaging.