streaming_load just lost first 'N'

[wubx]

Summary

$cat 11.csv
NONE
NNNN
N1

curl -H "insert_sql:insert into t1 format CSV" -H "skip_header:0" -F "upload=@./11.csv" -XPUT http://wubx:wubxwubx@192.168.1.100:8000/v1/streaming_load
{"id":"4104b51a-c1d5-42f3-b054-39faa69b64c1","state":"SUCCESS","stats":{"rows":3,"bytes":39},"error":null}

MySQL [(none)]> desc t1;
+-------+---------+------+---------+-------+
| Field | Type    | Null | Default | Extra |
+-------+---------+------+---------+-------+
| c1    | VARCHAR | YES  | NULL    |       |
+-------+---------+------+---------+-------+
1 row in set (0.006 sec)
MySQL [(none)]> select * from  t1;
+------+
| c1   |
+------+
| ONE  |
| NNN  |
| 1    |
+------+
3 rows in set (0.009 sec)

[BohuTANG]

cc @youngsofun @sundy-li

[xudong963]

It's ok in my local

➜  ~ sudo curl -s -u root: -XPUT "http://localhost:8000/v1/streaming_load" -H 'insert_sql: insert into t format CSV' -H 'skip_header:0' -F "upload=@./test.csv"
Password:
{"id":"d420e070-6fa4-4d6c-a7ea-903de96e8bd9","state":"SUCCESS","stats":{"rows":3,"bytes":42},"error":null}%
➜  ~ cat test.csv
NONE
NNNN
N1

mysql> select * from t;
+------+
| c    |
+------+
| NONE |
| NNNN |
| N1   |
+------+
3 rows in set (0.03 sec)
Read 3 rows, 42.00 B in 0.003 sec., 864.65 rows/sec., 11.82 KiB/sec.

Update: when the field is nullable, I met the bug.
Btw, the bug is case-insensitive, the following case also lost the first char.

➜  ~ cat test.csv
nONE
nNNN
n1

[BohuTANG]

This is a terrible bug :/

[sundy-li]

ignore_insensitive_bytes("null") will eat the matched bytes , so 'n' is eaten.

We should use CheckpointReader to save the state.

[youngsofun]

already find a way to fix it，I will continue tomorrow morning