Merge pull request commaai#99 from gregjhogan/bit-transition-example

Script to find bits that transition from always 0 to always 1

examples/can_bit_transition.md

@@ -0,0 +1,25 @@
+# How to use can_bit_transition.py to reverse engineer a single bit field
+
+Let's say our goal is to find a brake pedal signal (caused by your foot pressing the brake pedal vs adaptive cruise control braking).
+
+The following process will allow you to quickly find bits that are always 0 during a period of time (when you know you were not pressing the brake with your foot) and always 1 in a different period of time (when you know you were pressing the brake with your foot).
+
+Open up a drive in cabana where you can find a place you used the brake pedal and another place where you did not use the brake pedal (and you can identify when you were on the brake pedal and when you were not).  You may want to go out for a drive and put something in front of the camera after you put your foot on the brake and take it away before you take your foot off the brake so you can easily identify exactly when you had your foot on the brake based on the video in cabana.  This is critical because this script needs the brake signal to always be high the entire time for one of the time frames.  A 10 second time frame worked well for me.
+
+I found a drive where I knew I was not pressing the brake between timestamp 50.0 thru 65.0 and I was pressing the brake between timestamp 69.0 thru 79.0. Determine what the timestamps are in cabana by plotting any message and putting your mouse over the plot at the location you want to discover the timestamp.  The tool tip on mouse hover has the format: timestamp: value
+
+Now download the log from cabana (Save Log button) and run the script passing in the timestamps
+(replace csv file name with cabana log you downloaded and time ranges with your own)
+```
+./can_bit_transition.py ./honda_crv_ex_2017_can-1520354796875.csv 50.0-65.0 69.0-79.0
+```
+
+The script will output bits that were always low in the first time range and always high in the second time range (and vice versa)
+```
+id 17c 0 -> 1 at byte 4 bitmask 1
+id 17c 0 -> 1 at byte 6 bitmask 32
+id 221 1 -> 0 at byte 0 bitmask 4
+id 1be 0 -> 1 at byte 0 bitmask 16
+```
+
+Now I go back to cabana and graph the above bits by searching for the message by id, double clicking on the appropriate bit, and then selecting "show plot". I already knew that message id 0x17c is both user brake and adaptive cruise control braking combined, so I plotted one of those signals along side 0x221 and 0x1be.  By replaying a drive I could see that 0x221 was not a brake signal (when high at random times that did not correspond to braking).  Next I looked at 0x1be and I found it was the brake pedal signal I was looking for (went high whenever I pressed the brake pedal and did not go high when adaptive cruise control was braking).

examples/can_bit_transition.py

@@ -0,0 +1,87 @@
+#!/usr/bin/env python
+
+import binascii
+import csv
+import sys
+
+class Message():
+  """Details about a specific message ID."""
+  def __init__(self, message_id):
+    self.message_id = message_id
+    self.ones = [0] * 8   # bit set if 1 is always seen
+    self.zeros = [0] * 8  # bit set if 0 is always seen
+
+  def printBitDiff(self, other):
+    """Prints bits that transition from always zero to always 1 and vice versa."""
+    for i in xrange(len(self.ones)):
+      zero_to_one = other.zeros[i] & self.ones[i]
+      if zero_to_one:
+        print 'id %s 0 -> 1 at byte %d bitmask %d' % (self.message_id, i, zero_to_one)
+      one_to_zero = other.ones[i] & self.zeros[i]
+      if one_to_zero:
+        print 'id %s 1 -> 0 at byte %d bitmask %d' % (self.message_id, i, one_to_zero)
+
+
+class Info():
+  """A collection of Messages."""
+
+  def __init__(self):
+    self.messages = {}  # keyed by MessageID
+
+  def load(self, filename, start, end):
+    """Given a CSV file, adds information about message IDs and their values."""
+    with open(filename, 'rb') as input:
+      reader = csv.reader(input)
+      next(reader, None)  # skip the CSV header
+      for row in reader:
+        if not len(row): continue
+        time = float(row[0])
+        bus = int(row[2])
+        if time < start or bus > 127:
+          continue
+        elif time > end:
+          break
+        if row[1].startswith('0x'):
+          message_id = row[1][2:]  # remove leading '0x'
+        else:
+          message_id = hex(int(row[1]))[2:]  # old message IDs are in decimal
+
+        if row[3].startswith('0x'):
+          data = row[3][2:]  # remove leading '0x'
+        else:
+          data = row[3]
+        new_message = False
+        if message_id not in self.messages:
+          self.messages[message_id] = Message(message_id)
+          new_message = True
+        message = self.messages[message_id]
+        bytes = bytearray.fromhex(data)
+        for i in xrange(len(bytes)):
+          ones = int(bytes[i])
+          message.ones[i] = ones if new_message else message.ones[i] & ones
+          # Inverts the data and masks it to a byte to get the zeros as ones.
+          zeros = (~int(bytes[i])) & 0xff
+          message.zeros[i] = zeros if new_message else message.zeros[i] & zeros
+
+def PrintUnique(log_file, low_range, high_range):
+  # find messages with bits that are always low
+  start, end = map(float, low_range.split('-'))
+  low = Info()
+  low.load(log_file, start, end)
+  # find messages with bits that are always high
+  start, end = map(float, high_range.split('-'))
+  high = Info()
+  high.load(log_file, start, end)
+  # print messages that go from low to high
+  found = False
+  for message_id in high.messages:
+    if message_id in low.messages:
+      high.messages[message_id].printBitDiff(low.messages[message_id])
+      found = True
+  if not found: print 'No messages that transition from always low to always high found!'
+
+if __name__ == "__main__":
+  if len(sys.argv) < 4:
+    print 'Usage:\n%s log.csv <low-start>-<low-end> <high-start>-<high-end>' % sys.argv[0]
+    sys.exit(0)
+  PrintUnique(sys.argv[1], sys.argv[2], sys.argv[3])