In the past few years, and more often recently, I have received several emails asking questions about two papers that I have published back in 2012. The papers are:
Mohammed M. Alani, Neuro-Cryptanalysis of DES and Triple-DES, published in proceedings of the 19th International Conference on Neural Information Processing (ICONIP2012), Nov. 2012, Doha, Qatar.(LNCS 7667/2012, Springer, http://doi.org/jqm)
Mohammed M. Alani, Neuro-Cryptanalysis of DES, Proceedings of the World Congress on Internet Security 2012, June 10th-12th, 2012, University of Guelph, Ontario, Canada.(IEEE Xplore)
Hence, I would like to clarify a few things.
The work published in these two papers was done back in 2008-2010 based on the initial idea that I presented in my masters thesis in 2003. The first set of results I received at that time were unbelievable in comparison to other cryptanalysis techniques in terms of number of known-plaintext-ciphertext pairs, time, and processing needed. However, soon I came to know that they were not realistic.
I prepared the first version of the paper back in 2009 and sent it to the International Journal of Information Security. At that time, the EiC was Prof.Dieter Gollmann. I received a rejection letter from Prof.Gollmann only a few days later. I responded to the letter with some explanation of the points that I have apparently misrepresented in the paper. From there, we kept corresponding for about 6 months. I found that Prof.Gollmann’s support and comments were extremely useful and have shifted my thinking on how the experiment was supposed to be done. For that, I will remain thankful to him. Later on, I decided to send it to a conference instead. Seeking faster approval before someone else presents the same idea.
What is motivating me to write this post, is the fact that I have received several emails from different researchers around the world saying that they were not able to reproduce the same results. This actually got me to investigate the mater and see why these results could not be reproduced.
The first issue I faced was the loss of the original experiments data that were previously used. Since 2009, I have moved to live in 4 different countries, and apparently I have lost much of my old research data in the process of moving. Hence, I could not reproduce the same results because I did not have access to the same data. It needs to be clear to fellow researchers that it is absolutely normal to have varying degrees of success in the proposed method, as described in the second paper. Those of you who are experienced in training neural networks are aware that the success and failure can be heavily dependent on the initial weights among other variables, and not solely dependent on your data.
I have been asked several times to share the MATLAB code as well. There was no ‘code’ as in a complete program. Back then, I used the neural networks toolbox of MATLAB2008. I used direct commands like:
net = newff(input, output, … , {'logsig' 'logsig'}, 'trainscg');
train(....)
I used plaintext that was generated by a pseudo-random number generator. Then I encrypted the text using my own implementation of DES. Then, I wrote a small program to transform ciphertext into a matrix of zeros and ones so that I use this matrix for NN training in MATLAB. This transformation is done through removing the parity bit of each byte and then producing the ASCII code in bits.
I had success in about 10% of the experiments that I have done back then. I remember doing hundreds and hundreds of experiments. It is obvious that the presented results were selected from hundreds of failures. As I mentioned earlier, the starting conditions of training were a lot and in my experiments I left most of them to be handled by MATLAB. This might have reduced the time I needed to succeed in some of these experiences, but it definitely did not make the reproduction of these results any easier.
This post is not written in the intent of defending the papers, nor in arguing their accuracy. The intention behind this post is to clarify some issues and reasons why you might not be able to reproduce the exact same results.
The bottom line is that I can understand that some researchers are facing difficulties in reproducing the results due to my reliance on MATLAB in using default values, or MATLAB initialization of other important values. However, this does not nullify the importance and potential of the method presented in these two papers.