Problem in parsing waymo dataset

[KaiLong1]

I attempted to download the Waymo dataset from the official website, but encountered the following bug while parsing the data.

Traceback (most recent call last):
File "/home/longk/anaconda3/envs/detzero/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/longk/anaconda3/envs/detzero/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/mnt/disk8Tone/longkai/DetZero/detection/detzero_det/datasets/waymo/waymo_preprocess.py", line 241, in
create_waymo_infos(
File "/mnt/disk8Tone/longkai/DetZero/detection/detzero_det/datasets/waymo/waymo_preprocess.py", line 70, in create_waymo_infos
waymo_infos_train = get_infos_worker(
File "/mnt/disk8Tone/longkai/DetZero/detection/detzero_det/datasets/waymo/waymo_preprocess.py", line 36, in get_infos_worker
sequence_infos = list(tqdm(executor.map(process_single_sequence, sample_sequence_file_list),
File "/home/longk/anaconda3/envs/detzero/lib/python3.8/site-packages/tqdm/std.py", line 1182, in iter
for obj in iterable:
File "/home/longk/anaconda3/envs/detzero/lib/python3.8/concurrent/futures/_base.py", line 619, in result_iterator
yield fs.pop().result()
File "/home/longk/anaconda3/envs/detzero/lib/python3.8/concurrent/futures/_base.py", line 444, in result
return self.__get_result()
File "/home/longk/anaconda3/envs/detzero/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/home/longk/anaconda3/envs/detzero/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/mnt/disk8Tone/longkai/DetZero/detection/detzero_det/datasets/waymo/waymo_utils.py", line 206, in process_single_sequence_and_save
process_single_tfrecord_multiprocessing(
File "/mnt/disk8Tone/longkai/DetZero/detection/detzero_det/datasets/waymo/waymo_utils.py", line 250, in process_single_tfrecord_multiprocessing
for cnt, data in enumerate(dataset):
File "/home/longk/anaconda3/envs/detzero/lib/python3.8/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 761, in next
return self._next_internal()
File "/home/longk/anaconda3/envs/detzero/lib/python3.8/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 744, in _next_internal
ret = gen_dataset_ops.iterator_get_next(
File "/home/longk/anaconda3/envs/detzero/lib/python3.8/site-packages/tensorflow/python/ops/gen_dataset_ops.py", line 2728, in iterator_get_next
_ops.raise_from_not_ok_status(e, name)
File "/home/longk/anaconda3/envs/detzero/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 6941, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File "", line 3, in raise_from
tensorflow.python.framework.errors_impl.NotFoundError: /mnt/disk8Tone/longkai/DetZero/data/waymo/raw_data/segment-1005081002024129653_5313_150_5333_150_with_camera_labels_with_camera_labels.tfrecord; No such file or directory [Op:IteratorGetNext]

I removed '_with_camera_labels.tfrecord' from the line sequence_file_tfrecord = sequence_file[:-9] + '_with_camera_labels.tfrecord'

[YuRui-Learning]

hello，i also meet this problem，and fix by index modify in /root/DetZero/detection/detzero_det/datasets/waymo/waymo_utils.py
sequence_file_tfrecord = sequence_file[:-28] + '_with_camera_labels.tfrecord'
but i find prepare data is chucked ，do u meet this problem？like
0%| | 0/24 [00:00<?, ?it/s]

[KaiLong1]

I can parse it here, but there is an issue with the parsed data. Visualization reveals that the point cloud is messy.

And the parsing speed is very slow, far slower than the parsing data of OpenPCDET.

[KaiLong1]

I have reanalyzed some waymo data, but it is still this error. But I have no problem parsing with OpenPCDET. I haven't made any changes to the code, why did this error occur?

[L00qqq]

你好，我也遇到这个问题，通过修改 /root/DetZero/detection/detzero_det/datasets/waymo/waymo_utils.py 中的索引来修复， sequence_file_tfrecord = sequence_file[:-28] + '_with_camera_labels.tfrecord' 但是我发现准备数据被卡住了，你遇到这个问题吗？比如 0%| | 0/24 [00:00<?, ?it/s]

hello，i also meet this problem，and fix by index modify in /root/DetZero/detection/detzero_det/datasets/waymo/waymo_utils.py sequence_file_tfrecord = sequence_file[:-28] + '_with_camera_labels.tfrecord' but i find prepare data is chucked ，do u meet this problem？like 0%| | 0/24 [00:00<?, ?it/s]

Hello, I also encountered the same problem, have you solved it?

[Furchtlos001]

@KaiLong1 I also encountered the same problem, have you solved it?

[superkoma]

I have encountered this issue before, could you please provide the version of your installed waymo-open-dataset-tf, such as 2-6-0 or 2-0-0?

[zw-92]

我以前遇到过这个问题，您能否提供您安装的版本，例如 或 ？waymo-open-dataset-tf``2-6-0``2-0-0

Hello, I used waymo-open-dataset-tf-2-5-0 1.4.1, but I found that there was a duplicate part of sequence_file_tfrecord = sequence_file[:-9] + '_with_camera_labels.tfrecord' in the waymo_utils.py, so I changed it to sequence_file_tfrecord= sequence_file[:-9] + '.tfrecord', but it gets stuck again in execution, log: CPU frequency: 3187200000 Hz
0%| | 0/48 [00:00<?, ?it/s], and I limited the number of processes.