WE are hosting the IEEE ITSS Student Competition in Pedestrian Behavior Prediction 🏆.
- 20230910: All PSI data including videos, CV annotations, and Cognitive Annotation (PSI1.0 & PSI 2.0) are public for download and future exploration!
[Google Drive][PSI Homepage] 🎇⚡ - 20230830: Test data are released and a brief guidance is provided in [Test Guidance].
- 20230624: Frequently asked questions are collected as [FAQ].
- 20230606: The official [Homepage], [GitHub], and [CodaLab] (Track 1, Track 2, and Track 3) of the IEEE ITSS Student Competition are public.
The challenge has three tracks:
- 🎯 Pedestrian Intent Prediction (PIP)
- 🎯 Pedestrian Trajectory Prediction (PTP)
- 🎯 Driver Decision Prediction (DDP)
The challenge is based on the Pedestrian Situated Intent (PSI) 2.0 dataset.
Pedestrian behavior prediction poses a significant challenge for fully automated driving in urban environments, as it necessitates safe and efficient interaction between autonomous vehicles and pedestrians within dynamic and diverse surroundings. Ensuring the safety of pedestrians and autonomous vehicles alike relies heavily on accurate and reliable pedestrian behavior prediction.
🎯Track 1 (Pedestrian Intent Prediction (PIP)): to predict the intention of a pedestrian crossing a street, such as whether they intend to cross or stop.
🎯Track 2 (Pedestrian Trajectory Prediction (PTP)): to predict the future trajectory of the pedestrian, given the pedestrian’s current position and intention.
🎯Track 3 (Driver Decision Prediction (DDP)): to predict the decision of the autonomous vehicle, given the pedestrian's intent and trajectory, to ensure safe and efficient interactions.
We warmly invite participants from across the globe to join our competition. The primary participant must be a student, either at the graduate or undergraduate level. Each team is allowed to participate in multiple tracks. For further information, please visit the website provided below.
For each challenge track, cash prizes will be awarded to the top three winners as follows:
| Challenge Track | Gold 🥇 | Silver 🥈 | Bronze 🥉 |
|---|---|---|---|
| Track 1 (Pedestrian Intent Prediction (PIP)) | $ 2,000 | $ 1,000 | $ 500 |
| Track 2 (Pedestrian Trajectory Prediction (PTP)) | $ 2,000 | $ 1,000 | $ 500 |
| Track 3 (Driver Decision Prediction (DDP)) | $ 2,000 | $ 1,000 | $ 500 |
Competition:
- [Preparing Date]: until 23:59 GMT, August 30, 2023
- [Final Competition Date]: 00:00 - 23:59 UTC, August 30, 2023
- [ITSC 2023 Workshop Time]: Sep. 24 - 28, 2023, Bilbao, Spain
The winners are required to attend our ITSC Workshop either remotely or in person.
-
Download and Prepare Dataset: Please follow [Github] to prepare the PSI 2.0 dataset.
- Training set: ready ✅ 😄
- Validation set: ready ✅ 😄
- Test set: ready ✅ 😄
-
Data Splits: The PSI 2.0 dataset is splitted into Train/Val/Test split as ./splits/PSI2_split.txt.
To facilitate a quick start, we offer baselines for all challenge tracks, which serve as hints on utilizing the PSI 2.0 dataset effectively.
Track 1 (Pedestrian Intent Prediction (PIP)): after observing 15 frames, equivalent to a duration of 0.5 seconds, the objective is to predict the intention of the target pedestrian crossing the road in front of the moving vehicle as either "Not Cross" or "Cross".
Track 2 (Pedestrian Trajectory Prediction (PTP)): after observing 15 frames, which corresponds to a duration of 0.5 seconds, the task is to predict the trajectory of the target pedestrian for the subsequent time intervals of 0.5 seconds (15 frames), 1.0 second (30 frames), and 1.5 seconds (45 frames).
Track 3 (Driver Decision Prediction (DDP)): after observing 15 frames, encompassing a duration of 0.5 seconds, the goal is to predict the driving decision made by the driver. The prediction should consist of two outputs: Speed (options: 'increaseSpeed', 'decreaseSpeed', 'maintainSpeed') and Direction (options: 'goStraight', 'turnLeft', 'turnRight').
| Challenge Track | Reported Metrics | Ranking Metric | Ranking |
|---|---|---|---|
| Track 1 (Pedestrian Intent Prediction (PIP)) | F1, Recall, Precision | ||
| Track 2 (Pedestrian Trajectory Prediction (PTP)) |
ADE@1.5s FDE@1.5s |
$ADE@1.5s$ | |
| Track 3 (Driver Decision Prediction (DDP)) | speed: Acc, mAcc direction: Acc, mAcc |
Track 1 (Pedestrian Intent): The evaluation ranking score is determined based on the F1-score of the binary intention prediction.
cd ./evaluation_scripts/intention
python evaluate_results.pyTrack 2 (Pedestrian Trajectory): The evaluation ranking score for the competition is calculated as the average Average Displacement Error (ADE) of the predicted trajectory over a future time span of 1.5 seconds.
cd ./evaluation_scripts/trajectory
python evaluate_results.pyTrack 3 (Driver Decision): The evaluation ranking score is computed as the average of the mean accuracy of both the speed and direction predictions.
cd ./evaluation_scripts/driving_decision
python evaluate_results.pyPlease participate the competition and submit results via CodaLab [Track 1] [Track 2] [Track 3].
Winners will be expected to submit their code and trained weights for the validation process. Further information regarding the code submission process will be updated and provided in due course.
Track 1 (Pedestrian Intent): Example intent ground-truth/prediction JSON format:
# Filename: val/test_intent_gt/intent_pred.json
video_name: {
pedestrian_ID: {
frame_ID: {
intent: int # 0-not cross, 1-crossing
}
}
}
Track 2 (Pedestrian Trajectory): Example trajectory ground-truth/prediction JSON format:
# Filename: val/test_traj_gt/traj_pred.json
video_name: {
pedestrian_ID: {
frame_ID: {
traj/traj: {
0: [xtl, ytl, xbr, ybr],
1: [xtl, ytl, xbr, ybr],
2: [xtl, ytl, xbr, ybr],
...
}
}
}
}
Track 3 (Driver Decision): Example driving decision ground-truth/prediction JSON format:
# Filename: val/test_driving_gt/driving_pred.json
video_name: {
frame_ID: {
speed: int,
direction: int
}
}
Faculty members: Zhengming Ding, Renran Tian, Yaobin Chen
Student members: Taotao Jing, Xin Hu, Zhengming Zhang
- IEEE Intelligent Transportation Systems Society
- Technical Committee on Human-Centered AI in Transportation (HAIT)
If you have any questions, please refer to the FAQ page, or feel free to contact psi.intent.benchmark@gmail.com
[1] Chen, Tina, Taotao Jing, Renran Tian, Yaobin Chen, Joshua Domeyer, Heishiro Toyoda, Rini Sherony, and Zhengming Ding. "Psi: A pedestrian behavior dataset for socially intelligent autonomous car." arXiv preprint arXiv:2112.02604 (2021).
[2] Chen, Tina, Renran Tian, and Zhengming Ding. "Visual reasoning using graph convolutional networks for predicting pedestrian crossing intention." In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3103-3109. 2021.
[3] Jing, Taotao, Haifeng Xia, Renran Tian, Haoran Ding, Xiao Luo, Joshua Domeyer, Rini Sherony, and Zhengming Ding. "InAction: Interpretable action decision making for autonomous driving." In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXVIII, pp. 370-387. Cham: Springer Nature Switzerland, 2022.
[4] Zhang, Zhengming, Renran Tian, and Zhengming Ding. "TrEP: Transformer-based Evidential Prediction for Pedestrian Intention with Uncertainty." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 3, pp. 3589-3597. 2022.