OP 0.5.4

Squashed commit of the following:

commit 1391e0c
Merge: 5b53cd4 a422246
Author: David Abrahams <you@example.com>
Date:   Wed Sep 26 04:52:42 2018 +0000

    Merge remote-tracking branch 'op/devel' 0.5.4 into cruise_devel

commit a422246
Author: Vehicle Researcher <user@comma.ai>
Date:   Tue Sep 25 00:13:41 2018 -0700

    openpilot v0.5.4 release

commit e5b2ec4
Merge: 97f1ee4 906b55a
Author: Vehicle Researcher <user@comma.ai>
Date:   Tue Sep 25 00:04:00 2018 -0700

    Merge opendbc subtree

commit 906b55a
Author: Vehicle Researcher <user@comma.ai>
Date:   Tue Sep 25 00:04:00 2018 -0700

    Squashed 'opendbc/' changes from af7fff8..42f8c12

    42f8c12 Adds Chinese Odyssey DBC (commaai#110)
    7b87882 Syntax and encoding fixes (commaai#111)
    f85e646 Update mercedes e350 dbc (commaai#112)

    git-subtree-dir: opendbc
    git-subtree-split: 42f8c12

commit 97f1ee4
Merge: 2d450bc 5c0935d
Author: Vehicle Researcher <user@comma.ai>
Date:   Tue Sep 25 00:03:58 2018 -0700

    Merge panda subtree

commit 5c0935d
Author: Vehicle Researcher <user@comma.ai>
Date:   Tue Sep 25 00:03:58 2018 -0700

    Squashed 'panda/' changes from f2292e4..5253ab0

    5253ab0 bump panda
    6277139 GM Volt: allowing 300 max steer
    63b7926 Hyundai safety: temporarily disabled button spam check to avoid camera faults
    53aef76 GM: passive on LKA-only cars (commaai#135)
    01b5235 bumped panda version
    1c0ffd1 Hyundai safety: simplifications
    56794d6 Hyundai Safety Auto-Detect which CAN bus Camera is on (commaai#134)

    git-subtree-dir: panda
    git-subtree-split: 5253ab0

commit 2d450bc
Author: Vasily Tarasov <vtarasov@zoneos.com>
Date:   Fri Sep 21 19:41:44 2018 -0700

    Increase Volt's steering limit (commaai#351)

    * Increase Volt's steering limit

    * Merge panda subtree

commit 96c9238
Author: Vasily Tarasov <vtarasov@zoneos.com>
Date:   Fri Sep 21 19:40:52 2018 -0700

    GM refactor (commaai#374)

commit 5b53cd4
Author: David Abrahams <you@example.com>
Date:   Fri Sep 21 03:17:44 2018 +0000

    ACC: syntax fix

commit d1d642e
Author: David Abrahams <you@example.com>
Date:   Fri Sep 21 03:15:49 2018 +0000

    ACC: Syntax fix

commit f5d3e51
Author: David Abrahams <you@example.com>
Date:   Fri Sep 21 00:15:57 2018 +0000

    ACC: If disabled, disengage traditional cruise control

commit 5ad11b2
Author: David Abrahams <you@example.com>
Date:   Thu Sep 20 23:30:29 2018 +0000

    ACC: disable on brakes or slowing below min speed

commit 14b7ead
Author: wocsor <37704329+wocsor@users.noreply.github.com>
Date:   Thu Sep 20 19:06:15 2018 -0400

    Add XSE trim to fingerprints (commaai#372)

commit 402faab
Author: Ted Slesinski <ted@slesinski.org>
Date:   Wed Sep 19 20:59:18 2018 -0400

    Docker environment breaking (commaai#369)

    * Fixes matplotlib dependency

    * Use version 2.1.2

commit d56e44b
Author: David Abrahams <you@example.com>
Date:   Wed Sep 19 18:06:44 2018 +0000

    ACC: Don't auto-engage if vRel < 0

commit d359f70
Author: David Abrahams <you@example.com>
Date:   Wed Sep 19 17:08:03 2018 +0000

    ACC: Don't autoresume if there is a slow car in front

commit de79a07
Author: Ted Slesinski <ted@slesinski.org>
Date:   Tue Sep 18 00:39:58 2018 -0400

    Add support for accord hybrid (commaai#361)

commit 4b2c137
Author: wocsor <37704329+wocsor@users.noreply.github.com>
Date:   Mon Sep 17 16:24:52 2018 -0400

    update CHR fingerprint (commaai#364)

    CHR 2016 fingerprint has more messages

Dockerfile.openpilot

@@ -2,7 +2,7 @@ FROM ubuntu:16.04
 ENV PYTHONUNBUFFERED 1
 
 RUN apt-get update && apt-get install -y build-essential clang vim screen wget bzip2 git libglib2.0-0 python-pip capnproto libcapnp-dev libzmq5-dev libffi-dev libusb-1.0-0
-RUN pip install numpy==1.11.2 scipy==0.18.1 matplotlib
+RUN pip install numpy==1.11.2 scipy==0.18.1 matplotlib==2.1.2
 
 COPY requirements_openpilot.txt /tmp/
 RUN pip install -r /tmp/requirements_openpilot.txt
@@ -17,3 +17,5 @@ COPY ./phonelibs /tmp/openpilot/phonelibs
 COPY ./pyextra /tmp/openpilot/pyextra
 
 RUN mkdir -p /tmp/openpilot/selfdrive/test/out
+RUN make -C /tmp/openpilot/selfdrive/controls/lib/longitudinal_mpc clean
+RUN make -C /tmp/openpilot/selfdrive/controls/lib/lateral_mpc clean

RELEASES.md

@@ -1,3 +1,14 @@
+Version 0.5.4 (2018-09-25)
+========================
+ * New Driving Model
+ * New Driver Monitoring Model
+ * Improve longitudinal mpc in mid-low speed braking
+ * Honda Accord hybrid support thanks to energee!
+ * Ship mpc binaries and sensibly reduce build time
+ * Calibration more stable
+ * More Hyundai and Kia cars supported thanks to emmertex!
+ * Various GM Volt improvements thanks to vntarasov!
+
 Version 0.5.3 (2018-09-03)
 ========================
  * Hyundai Santa Fe support!

cereal/car.capnp

@@ -42,7 +42,7 @@ struct CarEvent @0x9b1657f34caf3ad3 {
     speedTooLow @17;
     outOfSpace @18;
     overheat @19;
-    calibrationInProgress @20;
+    calibrationIncomplete @20;
     calibrationInvalid @21;
     controlsMismatch @22;
     pcmEnable @23;
@@ -69,6 +69,8 @@ struct CarEvent @0x9b1657f34caf3ad3 {
     promptDriverUnresponsive @44;
     driverUnresponsive @45;
     belowSteerSpeed @46;
+    calibrationProgress @47;
+    lowBattery @48;
   }
 }
 

cereal/log.capnp

@@ -276,6 +276,7 @@ struct ThermalData {
   startedTs @13 :UInt64;
 
   thermalStatus @14 :ThermalStatus;
+  chargerDisabled @17 :Bool;
 
   enum ThermalStatus {
     green @0;   # all processes run

common/dbc.py

@@ -1,7 +1,6 @@
 import re
 import os
 import struct
-import bitstring
 import sys
 import numbers
 from collections import namedtuple, defaultdict
@@ -17,6 +16,7 @@ def int_or_float(s):
   "DBCSignal", ["name", "start_bit", "size", "is_little_endian", "is_signed",
                 "factor", "offset", "tmin", "tmax", "units"])
 
+
 class dbc(object):
   def __init__(self, fn):
     self.name, _ = os.path.splitext(os.path.basename(fn))
@@ -122,6 +122,16 @@ def lookup_msg_id(self, msg_id):
       msg_id = self.msg_name_to_address[msg_id]
     return msg_id
 
+  def reverse_bytes(self, x):
+    return ((x & 0xff00000000000000) >> 56) | \
+           ((x & 0x00ff000000000000) >> 40) | \
+           ((x & 0x0000ff0000000000) >> 24) | \
+           ((x & 0x000000ff00000000) >> 8) | \
+           ((x & 0x00000000ff000000) << 8) | \
+           ((x & 0x0000000000ff0000) << 24) | \
+           ((x & 0x000000000000ff00) << 40) | \
+           ((x & 0x00000000000000ff) << 56)
+
   def encode(self, msg_id, dd):
     """Encode a CAN message using the dbc.
 
@@ -131,35 +141,40 @@ def encode(self, msg_id, dd):
     """
     msg_id = self.lookup_msg_id(msg_id)
 
-    # TODO: Stop using bitstring, which is super slow.
     msg_def = self.msgs[msg_id]
     size = msg_def[0][1]
 
-    bsf = bitstring.Bits(hex="00"*size)
+    result = 0
     for s in msg_def[1]:
       ival = dd.get(s.name)
       if ival is not None:
-        ival = (ival / s.factor) - s.offset
-        ival = int(round(ival))
 
-        # should pack this
+        b2 = s.size
         if s.is_little_endian:
-          ss = s.start_bit
+          b1 = s.start_bit
         else:
-          ss = self.bits_index[s.start_bit]
+          b1 = (s.start_bit // 8) * 8 + (-s.start_bit - 1) % 8
+        bo = 64 - (b1 + s.size)
+
+        ival = (ival / s.factor) - s.offset
+        ival = int(round(ival))
 
+        if s.is_signed and ival < 0:
+          ival = (1 << b2) + ival
 
-        if s.is_signed:
-          tbs = bitstring.Bits(int=ival, length=s.size)
-        else:
-          tbs = bitstring.Bits(uint=ival, length=s.size)
+        shift = b1 if s.is_little_endian else bo
+        mask = ((1 << b2) - 1) << shift
+        dat = (ival & ((1 << b2) - 1)) << shift
+
+        if s.is_little_endian:
+          mask = self.reverse_bytes(mask)
+          dat = self.reverse_bytes(dat)
 
-        lpad = bitstring.Bits(bin="0b"+"0"*ss)
-        rpad = bitstring.Bits(bin="0b"+"0"*(8*size-(ss+s.size)))
-        tbs = lpad+tbs+rpad
+        result &= ~mask
+        result |= dat
 
-        bsf |= tbs
-    return bsf.tobytes()
+    result = struct.pack('>Q', result)
+    return result[:size]
 
   def decode(self, x, arr=None, debug=False):
     """Decode a CAN message using the dbc.
@@ -195,55 +210,77 @@ def decode(self, x, arr=None, debug=False):
     if debug:
       print name
 
-    blen = 8*len(x[2])
-
-    st = x[2].rjust(8, '\x00')
+    st = x[2].ljust(8, '\x00')
     le, be = None, None
-    size = msg[0][1]
 
     for s in msg[1]:
       if arr is not None and s[0] not in arr:
         continue
 
-      # big or little endian?
-      #   see http://vi-firmware.openxcplatform.com/en/master/config/bit-numbering.html
-      if s[3] is False:
-        ss = self.bits_index[s[1]]
-        if be is None:
-          be = struct.unpack(">Q", st)[0]
-        x2_int = be
-        data_bit_pos = (blen - (ss + s[2]))
+      start_bit = s[1]
+      signal_size = s[2]
+      little_endian = s[3]
+      signed = s[4]
+      factor = s[5]
+      offset = s[6]
+
+      b2 = signal_size
+      if little_endian:
+        b1 = start_bit
       else:
+        b1 = (start_bit // 8) * 8 + (-start_bit - 1) % 8
+      bo = 64 - (b1 + signal_size)
+
+      if little_endian:
         if le is None:
           le = struct.unpack("<Q", st)[0]
-        x2_int = le >> (64 - 8 * size)
-        ss = s[1]
-        data_bit_pos = ss
+        shift_amount = b1
+        tmp = le
+      else:
+        if be is None:
+          be = struct.unpack(">Q", st)[0]
+        shift_amount = bo
+        tmp = be
 
-      if data_bit_pos < 0:
+      if shift_amount < 0:
         continue
-      ival = (x2_int >> data_bit_pos) & ((1 << (s[2])) - 1)
 
-      if s[4] and (ival & (1<<(s[2]-1))): # signed
-        ival -= (1<<s[2])
+      tmp = (tmp >> shift_amount) & ((1 << b2) - 1)
+      if signed and (tmp >> (b2 - 1)):
+        tmp -= (1 << b2)
 
-      # control the offset
-      ival = (ival * s[5]) + s[6]
-      #if debug:
-      #  print "%40s  %2d %2d  %7.2f %s" % (s[0], s[1], s[2], ival, s[-1])
+      tmp = tmp * factor + offset
+
+      # if debug:
+      #   print "%40s  %2d %2d  %7.2f %s" % (s[0], s[1], s[2], tmp, s[-1])
 
       if arr is None:
-        out[s[0]] = ival
+        out[s[0]] = tmp
       else:
-        out[arr.index(s[0])] = ival
+        out[arr.index(s[0])] = tmp
     return name, out
 
   def get_signals(self, msg):
     msg = self.lookup_msg_id(msg)
     return [sgs.name for sgs in self.msgs[msg][1]]
 
+
 if __name__ == "__main__":
    from opendbc import DBC_PATH
+   import numpy as np
+
+   dbc_test = dbc(os.path.join(DBC_PATH, 'toyota_prius_2017_pt_generated.dbc'))
+   msg = ('STEER_ANGLE_SENSOR', {'STEER_ANGLE': -6.0, 'STEER_RATE': 4, 'STEER_FRACTION': -0.2})
+   encoded = dbc_test.encode(*msg)
+   decoded = dbc_test.decode((0x25, 0, encoded))
+   assert decoded == msg
+
+   dbc_test = dbc(os.path.join(DBC_PATH, 'hyundai_santa_fe_2019_ccan.dbc'))
+   decoded = dbc_test.decode((0x2b0, 0, "\xfa\xfe\x00\x07\x12"))
+   assert np.isclose(decoded[1]['SAS_Angle'], -26.2)
+
+   msg = ('SAS11', {'SAS_Stat': 7.0, 'MsgCount': 0.0, 'SAS_Angle': -26.200000000000003, 'SAS_Speed': 0.0, 'CheckSum': 0.0})
+   encoded = dbc_test.encode(*msg)
+   decoded = dbc_test.decode((0x2b0, 0, encoded))
 
-   dbc_test = dbc(os.path.join(DBC_PATH, sys.argv[1]))
-   print dbc_test.get_signals(0xe4)
+   assert decoded == msg

common/filter_simple.py

@@ -0,0 +1,10 @@
+class FirstOrderFilter():
+  # first order filter
+  def __init__(self, x0, ts, dt):
+    self.k = (dt / ts) / (1. + dt / ts)
+    self.x = x0
+
+  def update(self, x):
+    self.x = (1. - self.k) * self.x + self.k * x
+
+

common/transformations/model.py

@@ -57,6 +57,8 @@
 bigmodel_frame_from_road_frame = np.dot(bigmodel_intrinsics,
   get_view_frame_from_road_frame(0, 0, 0, model_height))
 
+model_frame_from_bigmodel_frame = np.dot(model_intrinsics, np.linalg.inv(bigmodel_intrinsics))
+
 # 'camera from model camera'
 def get_model_height_transform(camera_frame_from_road_frame, height):
   camera_frame_from_road_ground = np.dot(camera_frame_from_road_frame, np.array([

opendbc/cadillac_ct6_powertrain.dbc

@@ -189,7 +189,7 @@ BO_ 1033 ASCMKeepAlive: 7 NEO
  SG_ ASCMKeepAliveAllZero : 7|56@0+ (1,0) [0|0] ""  NEO
 
 BO_ 1217 ECMEngineCoolantTemp: 8 K20_ECM
- SG_ EngineCoolantTemp : 23|8@0+ (1,-40) [0|0] "�C"  NEO
+ SG_ EngineCoolantTemp : 23|8@0+ (1,-40) [0|0] "°C"  NEO
 
 BO_ 1249 VIN_Part2: 8 K20_ECM
  SG_ VINPart2 : 7|64@0+ (1,0) [0|0] ""  NEO

opendbc/generator/honda/honda_civic_touring_2016_can.dbc

@@ -126,7 +126,7 @@ BO_ 1302 ODOMETER: 8 XXX
 
 CM_ SG_ 401 GEAR "10 = reverse, 11 = transition";
 CM_ SG_ 420 BRAKE_HOLD_RELATED "On when Brake Hold engaged";
-CM_ SG_ 450 EPB_STATE "3 "engaged" 2 "disengaging" 1 "engaging" 0 "disengaged"";
+CM_ SG_ 450 EPB_STATE "3 \"engaged\" 2 \"disengaging\" 1 \"engaging\" 0 \"disengaged\"";
 CM_ SG_ 806 REVERSE_LIGHT "Might be reverse gear selected and not the lights";
 
 VAL_ 399 STEER_STATUS 5 "fault" 4 "no_torque_alert_2" 2 "no_torque_alert_1" 0 "normal" ;

opendbc/gm_global_a_powertrain.dbc

@@ -180,7 +180,7 @@ BO_ 1033 ASCMKeepAlive: 7 NEO
 BO_ 1034 ASCM_40A: 7 K124_ASCM
 
 BO_ 1217 ECMEngineCoolantTemp: 8 K20_ECM
- SG_ EngineCoolantTemp : 23|8@0+ (1,-40) [0|0] "�C"  NEO
+ SG_ EngineCoolantTemp : 23|8@0+ (1,-40) [0|0] "°C"  NEO
 
 BO_ 1249 VIN_Part2: 8 K20_ECM
  SG_ VINPart2 : 7|64@0+ (1,0) [0|0] ""  NEO

opendbc/honda_civic_touring_2016_can_generated.dbc

@@ -355,7 +355,7 @@ BO_ 1302 ODOMETER: 8 XXX
 
 CM_ SG_ 401 GEAR "10 = reverse, 11 = transition";
 CM_ SG_ 420 BRAKE_HOLD_RELATED "On when Brake Hold engaged";
-CM_ SG_ 450 EPB_STATE "3 "engaged" 2 "disengaging" 1 "engaging" 0 "disengaged"";
+CM_ SG_ 450 EPB_STATE "3 \"engaged\" 2 \"disengaging\" 1 \"engaging\" 0 \"disengaged\"";
 CM_ SG_ 806 REVERSE_LIGHT "Might be reverse gear selected and not the lights";
 
 VAL_ 399 STEER_STATUS 5 "fault" 4 "no_torque_alert_2" 2 "no_torque_alert_1" 0 "normal" ;