Merge pull request commaai#210 from xx979xx/kegman-0.6.5

Kegman update

README.md

@@ -13,13 +13,14 @@ Here's how to flash back to v9 NEOS if you want to downgrade (it's not that bad)
 
 <b>NOTICE:</b>  Due to feedback I have turned on OTA updates.  You will receive updates automatically (after rebooting 2X) on your Eon so you don't have to reclone or git pull any longer to receive new features *MADE BETWEEN COMMA RELEASES*.  The reason why I separate the branches by release is because some releases can sometimes cause issues.  Features that I or others add, will continue to be updated when you are on the most current release.  If you DO NOT want OTA updates then create a file called "/data/no_ota_updates" and it will not perform OTA updates as long as that file exists.  
 
+<b>GM Users:</b> Execute the script, `selfdrive/car/lock_safety_model.py gm` from the root directory of this Openpilot installation. This sets the Panda safety mode to `gm` (from the default of `noOutput`), allowing OP to control the vehicle. Otherwise a "Steering Temporarily Unavailable" error is thrown when attempting to enagage OP.
 
 I will attempt to detail the changes in each of the branches here:
 
 
 <b>kegman</b> - this is the default branch which does not include Gernby's resonant feed forward steering (i.e. it's comma's default steering)
 
-<b>kegman-noAEB</b> - this branch disables the Honda Nidec AEB passthrough introduced in 0.6.4 as it gives problems and unexpectedly brakes on tight curves and oncoming traffic.
+<b>kegman-plusBoschGasPress</b> - this branch gives Honda Bosch users the ability to press the gas without disengaging OP for more "stock ACC"-like behaviour.  Remember to manually flash panda after checking out.  cd /data/openpilot/panda/board && pkill -f boardd && make - then reboot.
 
 <b>kegman-stockUI</b> - for 0.6 some people were having trouble with devUI so I separated the branches out.  
 
@@ -39,15 +40,28 @@ I will attempt to detail the changes in each of the branches here:
 
 
 List of changes and tweaks (latest changes at the top):
+- <b> New! Adjustable stopping distance, one, two, and three bar distance intervals</b>:  BE CAREFUL WITH THESE OPTIONAL SETTINGS IN kegman.json!  Add ONE_BAR_DISTANCE, TWO_BAR_DISTANCE, THREE_BAR_DISTANCE, FOUR_BAR_DISTANCE values in kegman.json to change the following distance interval.  Add STOPPING_DISTANCE to change the distance between you and the lead car at a stop.  If these values do not exist in kegman.json, they just assume default values of 0.9, 1.3, 1.8, 2.3, and 2 respectively.  Thanks to @joeljacobs for adding the optional distance intervals.
+
+- <b> New! Disable Auto Emergency Braking (Nidec Hondas)</b>:  Since 0.6.4 Comma passes through the AEB signal.  This causes unexpected emergency braking on 2 lane highways with oncoming traffic and is unsafe.  I have disabled this.  Ensure that you reflash you Panda if you get AEBs:  cd /data/openpilot/panda/board && pkill -f boardd && make  - after a successful flash, reboot.
+
 - <b> New! Toyota support</b>:  Thanks to @j4z for adding distance interval support with his Arduino solution and also helping to debug the kegman.json issues to make Kegman fork work with Toyotas!
 
 - <b> New! Added highway speed braking profile tweaks</b>.  Note that 1barHwy, 2barHwy and 3barHwy are DELTAS.  For example if One bar distance is 0.9 seconds, 1barHwy of 0.3 will add 3 seconds to the distance during braking making you brake harder.
 
 - <b> New! Added kF feedforward param to live tuner.</b>
 
-- <b> New! Enable / Disable Model based Slowdowns on turns:  On tight turns, the model will slow down the car so that you can make the turn.  Some like this, some people don't.  Set slowOnCurve = "1" to enable slowdowns on curves, or "0" (default) to disable.
+- <b> New! Enable / Disable Model based Slowdowns on turns: </b>  On tight turns, the model will slow down the car so that you can make the turn.  Some like this, some people don't.  Set slowOnCurve = "1" to enable slowdowns on curves, or "0" (default) to disable.
+
+- <b> New! Live long tuning for city speeds < 19.44 m/s (43.5 mph, 70 km/h): </b> Execute cd /data/openpilot && ./tune.sh to access live tuner on your mobile device while driving.
 
-- <b> New! Live long tuning for city speeds < 19.44 m/s </b>:</b> Execute cd /data/openpilot && ./tune.sh to access live tuner on your mobile device while driving.  
+<b>Instructions for tuning steering with live tuner:</b>
+- Kp too high = the car overshoots and undershoots center
+- Kp too low = the car doesn't turn enough
+- Ki - dampens the overshoot / undershoot of Kp and allows the car to be centered faster, allowing for a higher Kp value
+- Ki too high = it gets to center without oscillations, but it takes too long to center.  If you hit a bump or give the wheel a quick nudge, it should oscillate 3 - 5 times before coming to steady-state.  If the wheel oscillates forever (critically damped), then your Kp or Ki or both are too high.
+- SteerRatio - too high, and the car ping pongs on straights and turns, too low, and the car doesn't turn enough on curves.  If you're on a turn and the wheel is oversteering and then correcting, steerRatio is too high, and it's fighting with Kp and Ki (which you don't want) - although in the past I've been able to have an oscillating oversteering tune which could do tighter turns, but the turns werent pleasant.
+- Kf - lower this if your car oscillates and you've done everything else.  It can be lowered to 0
+- All of these parameters interact with each other so finding the balance is a bit experimental  
 
 <b>Be careful and ready to take over at any time when doing this!!!</b>  The "distance" in s is the target distance the car will try to maintain.  The default distancces are 0.9s, 1.3s, 1.8s for 1,2 and 3 bar intervals.  I manipulate this value to pass to the MPC to scale the behavior which leads to harder braking or sooner braking or softer braking.  Essentially when you are approaching a car, the distance changes depending on your approach speed.  When the lead car pulls away, the distance returns to whatever your bar setting is
 
@@ -69,29 +83,18 @@ Example:
 
 Everything inbetween -0.25 m/s and 3 m/s is interpolated, which adjusts the distance smoothly as you slow down depending on the lead car approach relative speed.  
 
-
-- <b>(Not functional in 0.6.x yet) Toggle Comma's live tuning</b>:  Comma live tunes things like alignment, steerRatio etc.  But sometimes it doesn't converge to the right value and throws lane centering off during turns.  This allows you to use /data/openpilot/.tune.sh to toggle the auto-tune to off when the car feels right so that it doesn't tune the car any further than necessary.
-
 - <b>Highway speed braking profiles</b>:  Added highway braking profiles so that you won't follow so closely at speeds > 70 kph.  This affects kegman-0.5.8-gold, kegman-0.5.11, kegman-plusGernbySteering-0.5.11, kegman-0.12, kegman-0.13-stockUI, kegman-0.6 kegman-plusGernbySteering-0.6 branches only.
 
 - <b>Live tuner for Kp and Ki</b>:  Tune your Kp and Ki values live using your cell phone by SSHing into the Eon and executing cd /data/openpilot && ./tune.sh
 
-- <b>Kill services if plugged in and Eon batt < kegman.json --> battPercOff</b>  Shutting down of the Eon never worked on Nidec vehicles because the Panda always supplies power.  When Eon senses power it just starts up again.  So I have mitigated the power drain by about 40% when it is discharging in the car.  Reminder that the Eon continues to charge cycling between battChargeMax and battChargeMin in the /data/kegman.json file.  If the car battery falls below carVoltageMinEonShutdown in the /data/kegman.json file WHILE CHARGING THE EON then charging is disabled.  As charging is disabled, the Eon battery will continue to drain until it reaches battPercOff (again in /data/kegman.json) at which point it will shut down services to conserve power.  This will not prevent the Eon from discharging completely but will cut the drainage by about 40%, buying you some more time before it goes dead.  If the Eon is in this "Power Saving" mode you will need to reboot the Eon by pressing on the power button and touching somewhere near the center of the screen (note that the screen will note work).  Also note that if you unplug the Eon and the battery is below battPercOff it will shutdown.  If you reboot the Eon while unplugged, it will give you 3 minutes until it shuts down again unless it is plugged in during this time.  If you unplugged the Eon and forgot to turn it off, it will shutdown when the battery falls below battPercOff.  Hopefully this mitigates the "dead Eon" syndrome that occurs when people have trouble powering their device back up after the battery is completely drained.
-
-For Bosch vehicles, the Eon will just simply shutdown as usual when battery falls below battPercOff.  Killing of services only occurs for Nidecs.
-
-
-
 - <b>Add @pjlao307's Dashcam Recording</b>:  Sometimes you just want to record a wicked OP run on a twisty highway to show your friends.  Sometimes you want to record a big flashing red error and complain about it and show your friends.  This does a screen video capture of the Eon screen and stores the files in /sdcard/videos on your Eon when the REC button is pressed.  Thanks to @pjlao307 and @theantihero for submitting the PR.
 
 - <b>Stop logging when space hits 18% free space</b>:  Thanks to @emmertex for this easy fix to stop the Eon from filling up while driving when free space is low.
 
 - <b>Added primitive tuning script</b>: To invoke live tuning:  (a) turn on tethering on your Eon,  (b) install JuiceSSH or similar and connect your cellphone to the wifi of the Eon using 192.168.43.1 and import the Comma private key,  (c) in JuiceSSH in the SSH session on the Eon issue cd /data/openpilot command, then ./tune.sh.  The text UI will be shown.  (d) turn "tuneGernby" to a "1"  (e) start driving and change the values to tune your steering.  It is best to have a cell phone mount in your car.  Note:  It takes 3 seconds for any changes to take effect.  
 
 - <b>Replaced dev UI</b> with @perpetuoviator dev UI with brake light icon by @berno22 - Thank you both!  NOTE:  There are lots of conveniences in this UI.  When the car is on, you have to press the top left corner to get to the Settings screen.  If you tap the lower right corner you can see the tmux session.  The brake light icon doesn't work properly with some cars (needs a fingerprint tweak I believe.  The wifi IP address and upload speed is printed on the screen.  The brake icon is so that you can see if OP is causing the brake lights to turn on and off and pissing the guy or gal off behind you. NOTE:  For GM vehicles, the brake icon indicates use of the friction brakes on the vehicle instead of the brake lights themselves.
-<b>UI touch controls:</b>
-[Acess "EON Settings" by touching the top left hand corner (left side of the Speed Limit sign)]
-[Acess "TmuxLog" by touching the bottom right hand corner]
+
 
 - <b>Added moar JSON parameters</b>:  
 
@@ -181,5 +184,3 @@ Enjoy everyone.
 <b>NOTE:</b> If you have upgraded at any time to v0.5.10, v0.6.x and you want to go back to a branch with v0.5.9 or v0.5.8, then you have to SSH into the Eon and edit the file /data/params/d/ControlsParams and rename "angle_model_bias" to "angle_offset" or your car will have Dash Errors and you'll be scratching your head for hours! 
 
 <b>Pedal Users:</b> Also note that you need to flash your Pedal to go to v0.5.10.  If you want to go back to 0.5.9 or 0.5.8 you need to flash your pedal back to 0.5.9.  Instructions are here:  https://medium.com/@jfrux/comma-pedal-updating-the-firmware-over-can-fa438a3cf910.  Also. After you flash your Pedal..  All hell will break loose on your dash.  Traction control error, Power Steering Error, Trailer Error, OMFG the sky is falling error etc.  DON'T PANIC.  Just drive around a bit and it will disappear after about 2-3 restarts of the car.  Don't rush it I believe it's time dependent as well.  Just drive as normal.  They'll go away.
-
-

common/dbc.py

@@ -147,22 +147,20 @@ def encode(self, msg_id, dd):
       ival = dd.get(s.name)
       if ival is not None:
 
-        b2 = s.size
-        if s.is_little_endian:
-          b1 = s.start_bit
-        else:
-          b1 = (s.start_bit // 8) * 8 + (-s.start_bit - 1) % 8
-        bo = 64 - (b1 + s.size)
-
         ival = (ival / s.factor) - s.offset
         ival = int(round(ival))
 
         if s.is_signed and ival < 0:
-          ival = (1 << b2) + ival
+          ival = (1 << s.size) + ival
 
-        shift = b1 if s.is_little_endian else bo
-        mask = ((1 << b2) - 1) << shift
-        dat = (ival & ((1 << b2) - 1)) << shift
+        if s.is_little_endian:
+          shift = s.start_bit
+        else:
+          b1 = (s.start_bit // 8) * 8 + (-s.start_bit - 1) % 8
+          shift = 64 - (b1 + s.size)
+
+        mask = ((1 << s.size) - 1) << shift
+        dat = (ival & ((1 << s.size) - 1)) << shift
 
         if s.is_little_endian:
           mask = self.reverse_bytes(mask)
@@ -222,30 +220,24 @@ def decode(self, x, arr=None, debug=False):
       factor = s[5]
       offset = s[6]
 
-      b2 = signal_size
-      if little_endian:
-        b1 = start_bit
-      else:
-        b1 = (start_bit // 8) * 8 + (-start_bit - 1) % 8
-      bo = 64 - (b1 + signal_size)
-
       if little_endian:
         if le is None:
           le = struct.unpack("<Q", st)[0]
-        shift_amount = b1
         tmp = le
+        shift_amount = start_bit
       else:
         if be is None:
           be = struct.unpack(">Q", st)[0]
-        shift_amount = bo
         tmp = be
+        b1 = (start_bit // 8) * 8 + (-start_bit - 1) % 8
+        shift_amount = 64 - (b1 + signal_size)
 
       if shift_amount < 0:
         continue
 
-      tmp = (tmp >> shift_amount) & ((1 << b2) - 1)
-      if signed and (tmp >> (b2 - 1)):
-        tmp -= (1 << b2)
+      tmp = (tmp >> shift_amount) & ((1 << signal_size) - 1)
+      if signed and (tmp >> (signal_size - 1)):
+        tmp -= (1 << signal_size)
 
       tmp = tmp * factor + offset
 

common/params.py

@@ -80,6 +80,7 @@ class UnknownKeyName(Exception):
   "Passive": [TxType.PERSISTENT],
   "RecordFront": [TxType.PERSISTENT],
   "ReleaseNotes": [TxType.PERSISTENT],
+  "SafetyModelLock": [TxType.PERSISTENT],
   "ShouldDoUpdate": [TxType.CLEAR_ON_MANAGER_START],
   "SpeedLimitOffset": [TxType.PERSISTENT],
   "SubscriberInfo": [TxType.PERSISTENT],

opendbc/honda_accord_touring_2016_can.dbc

@@ -177,14 +177,14 @@ BO_ 597 ROUGH_WHEEL_SPEED: 8 VSA
  SG_ SET_TO_X55_2 : 47|8@0+ (1,0) [0|255] "" NEO
  SG_ LONG_COUNTER : 55|8@0+ (1,0) [0|255] "" XXX
  SG_ COUNTER : 61|2@0+ (1,0) [0|3] "" XXX
- SG_ CHECKSUM : 59|4@1+ (1,0) [0|15] "" XXX
+ SG_ CHECKSUM : 59|4@0+ (1,0) [0|15] "" XXX
 
 BO_ 660 SCM_COMMANDS: 8 SCM
  SG_ RIGHT_BLINKER : 6|1@0+ (1,0) [0|1] "" NEO
  SG_ LEFT_BLINKER : 5|1@0+ (1,0) [0|1] "" NEO
  SG_ WIPERS_SPEED : 4|2@0+ (1,0) [0|3] "" NEO
  SG_ COUNTER : 61|2@0+ (1,0) [0|3] "" XXX
- SG_ CHECKSUM : 59|4@1+ (1,0) [0|15] "" XXX
+ SG_ CHECKSUM : 59|4@0+ (1,0) [0|15] "" XXX
 
 BO_ 661 XXX_10: 4 XXX
  SG_ COUNTER : 29|2@0+ (1,0) [0|3] "" XXX

selfdrive/boardd/boardd.cc

@@ -49,6 +49,7 @@ const uint32_t NO_IGNITION_CNT_MAX = 2 * 60 * 60 * 24 * 3;  // turn off charge a
 uint32_t no_ignition_cnt = 0;
 bool connected_once = false;
 uint8_t ignition_last = 0;
+bool safety_model_locked = false;
 
 pthread_t safety_setter_thread_handle = -1;
 pthread_t pigeon_thread_handle = -1;
@@ -74,12 +75,12 @@ void *safety_setter_thread(void *s) {
   }
   LOGW("got CarVin %s", value_vin);
 
-  pthread_mutex_lock(&usb_lock);
-
   // VIN query done, stop listening to OBDII
-  libusb_control_transfer(dev_handle, 0x40, 0xdc, (uint16_t)(cereal::CarParams::SafetyModel::NO_OUTPUT), 0, NULL, 0, TIMEOUT);
-
-  pthread_mutex_unlock(&usb_lock);
+  if (!safety_model_locked) {
+    pthread_mutex_lock(&usb_lock);
+    libusb_control_transfer(dev_handle, 0x40, 0xdc, (uint16_t)(cereal::CarParams::SafetyModel::NO_OUTPUT), 0, NULL, 0, TIMEOUT);
+    pthread_mutex_unlock(&usb_lock);
+  }
 
   char *value;
   size_t value_sz = 0;
@@ -125,6 +126,11 @@ void *safety_setter_thread(void *s) {
 bool usb_connect() {
   int err;
   unsigned char hw_query[1] = {0};
+  char *value_safety_model;
+  size_t value_safety_model_sz = 0;
+  int safety_model;
+  const int result = read_db_value(NULL, "SafetyModelLock", &value_safety_model, &value_safety_model_sz);
+
   ignition_last = 0;
 
   dev_handle = libusb_open_device_with_vid_pid(ctx, 0xbbaa, 0xddcc);
@@ -140,6 +146,16 @@ bool usb_connect() {
     libusb_control_transfer(dev_handle, 0xc0, 0xe5, 1, 0, NULL, 0, TIMEOUT);
   }
 
+  // check if safety mode is forced (needed to support gm)
+  if (value_safety_model_sz > 0) {
+    sscanf(value_safety_model, "%d", &safety_model);
+    // sanity check that we are not setting all output
+    assert(safety_model != (int)(cereal::CarParams::SafetyModel::ALL_OUTPUT));
+    safety_model_locked = true;
+    LOGW("Setting Locked Safety Model %s", value_safety_model);
+    libusb_control_transfer(dev_handle, 0x40, 0xdc, (uint16_t)(cereal::CarParams::SafetyModel(safety_model)), 0, NULL, 0, TIMEOUT);
+  }
+
   // power off ESP
   libusb_control_transfer(dev_handle, 0xc0, 0xd9, 0, 0, NULL, 0, TIMEOUT);
 
@@ -297,10 +313,11 @@ void can_health(void *s) {
     assert((result == 0) || (result == ERR_NO_VALUE));
 
     // diagnostic only is the default, needed for VIN query
-    pthread_mutex_lock(&usb_lock);
-    libusb_control_transfer(dev_handle, 0x40, 0xdc, (uint16_t)(cereal::CarParams::SafetyModel::ELM327), 0, NULL, 0, TIMEOUT);
-    pthread_mutex_unlock(&usb_lock);
-
+    if (!safety_model_locked) {
+      pthread_mutex_lock(&usb_lock);
+      libusb_control_transfer(dev_handle, 0x40, 0xdc, (uint16_t)(cereal::CarParams::SafetyModel::ELM327), 0, NULL, 0, TIMEOUT);
+      pthread_mutex_unlock(&usb_lock);
+    }
     if (safety_setter_thread_handle == -1) {
       err = pthread_create(&safety_setter_thread_handle, NULL, safety_setter_thread, NULL);
       assert(err == 0);

selfdrive/can/common.h

@@ -7,11 +7,13 @@
 
 #define ARRAYSIZE(x) (sizeof(x)/sizeof(x[0]))
 
-
 unsigned int honda_checksum(unsigned int address, uint64_t d, int l);
 unsigned int toyota_checksum(unsigned int address, uint64_t d, int l);
 unsigned int pedal_checksum(unsigned int address, uint64_t d, int l);
 
+void init_crc_lookup_tables();
+unsigned int volkswagen_crc(unsigned int address, uint64_t d, int l);
+
 struct SignalPackValue {
   const char* name;
   double value;
@@ -44,6 +46,8 @@ enum SignalType {
   TOYOTA_CHECKSUM,
   PEDAL_CHECKSUM,
   PEDAL_COUNTER,
+  VOLKSWAGEN_CHECKSUM,
+  VOLKSWAGEN_COUNTER,
 };
 
 struct Signal {

selfdrive/can/dbc_template.cc

@@ -31,6 +31,10 @@ const Signal sigs_{{address}}[] = {
       .type = SignalType::PEDAL_CHECKSUM,
       {% elif address in [512, 513] and sig.name == "COUNTER_PEDAL" %}
       .type = SignalType::PEDAL_COUNTER,
+      {% elif checksum_type == "volkswagen" and sig.name == "CHECKSUM" %}
+      .type = SignalType::VOLKSWAGEN_CHECKSUM,
+      {% elif checksum_type == "volkswagen" and sig.name == "COUNTER" %}
+      .type = SignalType::VOLKSWAGEN_COUNTER,
       {% else %}
       .type = SignalType::DEFAULT,
       {% endif %}

selfdrive/can/libdbc_py.py

@@ -41,6 +41,8 @@
   TOYOTA_CHECKSUM,
   PEDAL_CHECKSUM,
   PEDAL_COUNTER,
+  VOLKSWAGEN_CHECKSUM,
+  VOLKSWAGEN_COUNTER,
 } SignalType;
 
 typedef struct {