[Tails-dev] An amusing tale of indeterminism

Delete this message

Reply to this message
Author: anonym
Date:  
To: The Tails public development discussion list
Subject: [Tails-dev] An amusing tale of indeterminism
Hi,

While testing 0.14~rc1 I discovered that I2P wasn't behaving as it
should, namely that even though it connected to the I2P network just
fine, it never opened its local HTTP proxy. After many confusing tests
that didn't make sense I ended up builing a new image from the 0.14-rc1
tag using the exact same packages (i.e. the *.packages diff was empty),
and in this image *I2P* *worked* *fine*. WTF?

[Below "bad" refers to the 0.14~rc1 image (or its contents) and "good"
refers to this new image.]

This just blew my mind completely. I compared the contents of the good
and bad images' roots, and their filesystem.squashfs:s. A lot of
expectable differences showed up, but one wasn't quite as expectable:

    usr/share/i2p/lib/i2ptunnel.jar


which is part of the i2p-router package. I unpacked the .deb from our
git repo, and it turned out that the i2ptunnel.jar in the good image was
the same as the one in the .deb. So the i2ptunnel.jar in the bad image
has somehow been modified during the build process. I unpacked it and
diffed its contents with the good one, and the only difference was:

    net/i2p/i2ptunnel/I2PTunnelHTTPClient.class


That just makes too much sense, since (checking the corresponding .java
file in the I2P sources) this source file is described to:

    /**
     * Act as a mini HTTP proxy, handling various different types of
     * requests, forwarding them through I2P appropriately, and
     * displaying the reply. [...]


See attached file for the diff between decompiled versions of the good
and the bad .class file. It shows that something within the readFile()
function changed so that even the decompiler couldn't make sense out of
it (notice the "// ERROR //").

Those who were on #tails-dev yesterday may remember that I joked about
gamma rays flipping bits in my memory, and this starts to look more and
more plausible. Look at what `cmp -l` says when comparing the bad and
the good .jar file byte-by-byte:

    62050  37  33


So, only the 62050:th byte differs; it's (in base 10) 37 in the bad .jar
and 33 in the good .jar. Perhaps this made the CRC fail so the .jar file
wasn't loaded in I2P (I didn't see any warning about this in I2P's logs,
though)? Any way, notice that:

    good:  33 (base 10) = 00100001 (base 2)
    bad:   37 (base 10) = 00100101 (base 2)
                               ^------------- this bit has flipped!


Is this the prank of some mischievous cosmic ray? :) I'm unsure how
worried we should be about this happening again, and if we should
consider exploring preventative measures like only building releases on
machines using ECC memory, or similar. I nevertheless find this whole
story quite intriguing as I've never been able to track down a bit-flip
with this level of confidence before. :)

Any way, I'd rather not consider the I2P issue as a blocker for rc1
since most testing has already been done, and the interesting changes
that need to be tested for 0.14 really are outside of I2P any way. Let's
just make sure no bit-flips breaks it in the final 0.14 release. :)

Cheers!
--- I2PTunnelHTTPClient.bad.java    2012-10-11 03:32:31.502254442 +0200
+++ I2PTunnelHTTPClient.good.java    2012-10-11 03:32:22.998438194 +0200
@@ -1,6 +1,8 @@
 /*      */ package net.i2p.i2ptunnel;
 /*      */ 
+/*      */ import java.io.ByteArrayOutputStream;
 /*      */ import java.io.File;
+/*      */ import java.io.FileInputStream;
 /*      */ import java.io.IOException;
 /*      */ import java.io.InputStream;
 /*      */ import java.io.OutputStream;
@@ -839,63 +841,36 @@
 /*      */     try {
 /* 1125 */       return readFile(file); } catch (IOException ioe) {
 /*      */     }
-/* 1127 */     return backup; } 
-/*      */   // ERROR //
-/*      */   private static byte[] readFile(File file) throws IOException { // Byte code:
-/*      */     //   0: aconst_null
-/*      */     //   1: astore_1
-/*      */     //   2: sipush 512
-/*      */     //   5: newarray byte
-/*      */     //   7: astore_2
-/*      */     //   8: new 347    java/io/ByteArrayOutputStream
-/*      */     //   11: dup
-/*      */     //   12: sipush 2048
-/*      */     //   15: invokespecial 348    java/io/ByteArrayOutputStream:<init>    (I)V
-/*      */     //   18: astore_3
-/*      */     //   19: iconst_0
-/*      */     //   20: istore 4
-/*      */     //   22: new 349    java/io/FileInputStream
-/*      */     //   25: dup
-/*      */     //   26: aload_0
-/*      */     //   27: invokespecial 350    java/io/FileInputStream:<init>    (Ljava/io/File;)V
-/*      */     //   30: astore_1
-/*      */     //   31: aload_1
-/*      */     //   32: aload_2
-/*      */     //   33: invokevirtual 351    java/io/FileInputStream:read    ([B)I
-/*      */     //   36: dup
-/*      */     //   37: istore 4
-/*      */     //   39: ifle +14 -> 53
-/*      */     //   42: aload_3
-/*      */     //   43: aload_2
-/*      */     //   44: iconst_0
-/*      */     //   45: iload 4
-/*      */     //   47: invokevirtual 352    java/io/ByteArrayOutputStream:write    ([BII)V
-/*      */     //   50: goto -19 -> 31
-/*      */     //   53: aload_3
-/*      */     //   54: invokevirtual 353    java/io/ByteArrayOutputStream:toByteArray    ()[B
-/*      */     //   57: astore 5
-/*      */     //   59: jsr +14 -> 73
-/*      */     //   62: aload 5
-/*      */     //   64: areturn
-/*      */     //   65: astore 6
-/*      */     //   67: getfield 6    net/i2p/i2ptunnel/I2PTunnelHTTPClient:_context    Lnet/i2p/I2PAppContext;
-/*      */     //   70: aload 6
-/*      */     //   72: athrow
-/*      */     //   73: astore 7
-/*      */     //   75: aload_1
-/*      */     //   76: ifnull +7 -> 83
-/*      */     //   79: aload_1
-/*      */     //   80: invokevirtual 354    java/io/FileInputStream:close    ()V
-/*      */     //   83: goto +5 -> 88
-/*      */     //   86: astore 8
-/*      */     //   88: ret 7
-/*      */     //
-/*      */     // Exception table:
-/*      */     //   from    to    target    type
-/*      */     //   19    62    65    finally
-/*      */     //   65    70    65    finally
-/*      */     //   75    83    86    java/io/IOException } 
-/* 1158 */   public static void writeFooter(OutputStream out) throws IOException { out.write("<div class=\"proxyfooter\"><p><i>I2P HTTP Proxy Server<br>Generated on: ".getBytes());
+/* 1127 */     return backup;
+/*      */   }
+/*      */ 
+/*      */   private static byte[] readFile(File file) throws IOException
+/*      */   {
+/* 1132 */     FileInputStream fis = null;
+/* 1133 */     byte[] buf = new byte[512];
+/* 1134 */     ByteArrayOutputStream baos = new ByteArrayOutputStream(2048);
+/*      */     try {
+/* 1136 */       int len = 0;
+/* 1137 */       fis = new FileInputStream(file);
+/* 1138 */       while ((len = fis.read(buf)) > 0) {
+/* 1139 */         baos.write(buf, 0, len);
+/*      */       }
+/* 1141 */       return baos.toByteArray();
+/*      */     } finally {
+/*      */       try {
+/* 1144 */         if (fis != null)
+/* 1145 */           fis.close();
+/*      */       }
+/*      */       catch (IOException foo)
+/*      */       {
+/*      */       }
+/*      */     }
+/*      */   }
+/*      */ 
+/*      */   public static void writeFooter(OutputStream out)
+/*      */     throws IOException
+/*      */   {
+/* 1158 */     out.write("<div class=\"proxyfooter\"><p><i>I2P HTTP Proxy Server<br>Generated on: ".getBytes());
 /* 1159 */     out.write(new Date().toString().getBytes());
 /* 1160 */     out.write("</i></div></body></html>\n".getBytes());
 /* 1161 */     out.flush();