Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[MFR21] ICCV21 masked face recognition challenge discussion #1564

Open
nttstar opened this issue Jun 4, 2021 · 99 comments
Open

[MFR21] ICCV21 masked face recognition challenge discussion #1564

nttstar opened this issue Jun 4, 2021 · 99 comments

Comments

@nttstar
Copy link
Collaborator

nttstar commented Jun 4, 2021 •
edited
Loading

How to participate:
https://github.com/deepinsight/insightface/tree/master/challenges/iccv21-mfr

Submission server:
http://iccv21-mfr.com/

Workshop homepage:
https://ibug.doc.ic.ac.uk/resources/masked-face-recognition-challenge-workshop-iccv-21/

Any issue can be left in this thread.
@nttstar nttstar pinned this issue Jun 4, 2021
@yBeOne
Copy link

yBeOne commented Jun 7, 2021

有谁测试过baseline 模型在IJBC上的准确率吗？我测试时一直报错。

@vuvko
Copy link

vuvko commented Jun 7, 2021

Hello and thank you for organizing the challenge this year!
In your rules, you prohibit the use of pre-trained models and any external data, but there is no mention of how you'd enforce this rule. Will you demand training pipelines and cooperation to reproduce the results from top teams?

@nttstar
Copy link
Collaborator Author

nttstar commented Jun 7, 2021

@vuvko Yes, top-rank players should send their solutions to organizers after submission closed.

@vuvko
Copy link

vuvko commented Jun 7, 2021

Thank you for your quick response!
Will you check the reproducibility of the training pipeline? If so, what will you do in case of a different result model (it can be hardware-dependent)?

@nttstar
Copy link
Collaborator Author

nttstar commented Jun 7, 2021

@vuvko We will strictly check the reproducibility. There should not be any hardware-dependent issue.

@anxiangsir
Copy link
Contributor

有谁测试过baseline 模型在IJBC上的准确率吗？我测试时一直报错。

Please join the QQ group or WeChat group, let’s talk in detail.

@manideep2510
Copy link

manideep2510 commented Jun 9, 2021 •
edited
Loading

Hello
Uploading the model on the submission page is very very slow. I have checked my internet connection and it is good with an upload speed greater than 25 MB/s. Upload to the submission portal is happening at <50 KB/s speed.
Can someone please help me with this?

@vuvko
Copy link

vuvko commented Jun 9, 2021

Same problem here, I cannot upload the baseline iresnet50 model due to very slow upload speed.

@manideep2510
Copy link

manideep2510 commented Jun 9, 2021 •
edited
Loading

Looks like the speed is even low now, less than 10 KB/sec is being uploaded from my system. @nttstar Can you please look into this?
Thanks in advance 🙂

@nttstar
Copy link
Collaborator Author

nttstar commented Jun 10, 2021

@manideep2510 @vuvko The submission server is located in China and it seems to have bandwidth problems for other countries. We will try to find a solution soon.

@nutsam
Copy link

nutsam commented Jun 13, 2021

Is there any preprocessing like normalization before load to model ?
iccv21-mfr only mentioned that
the input shape should equal to 3x112x112 (RGB order)

@nttstar
Copy link
Collaborator Author

nttstar commented Jun 13, 2021

@nutsam Normalization setting will be auto-detected in our onnx_helper.py, please check it.

@vuvko
Copy link

vuvko commented Jun 23, 2021

@nttstar is there any update on the uploading problem?

@nttstar
Copy link
Collaborator Author

nttstar commented Jun 23, 2021

@vuvko We're trying!

@nqkhai1706
Copy link

Hello @nttstar ,
I just submitted a model and I got: "

2021-06-24 21:39:44 ms1m ∅ ∅ ∅ FAILED ∅ -
"
The error message just said "load_onnx_failed". But as I checked my model using your onnx_helper.py, there is no error message. The output is just like in bellow, on my PC.
So, I would like to ask what error message appears if my model's inference time larger than 10 ms?

Thank you,

file_: ms1m_groupface_resnet101.onnx
use onnx-model: onnx/ms1m_groupface_resnet101/ms1m_groupface_resnet101.onnx
input-shape: ['batch_size', 3, 112, 112]
0 Shape_0
1 Constant_1
2 Gather_2
3 Conv_3
4 PRelu_4
5 MaxPool_5
6 BatchNormalization_6
7 Conv_7
max time cost exceed, given 77.5860

@nttstar
Copy link
Collaborator Author

nttstar commented Jun 24, 2021

@nqkhai1706 Have you checked with onnxruntime==1.6 ?

@nqkhai1706
Copy link

@nqkhai1706 Have you checked with onnxruntime==1.6 ?

No, my onnxruntime version: 1.8.0.
May it different on 1.6?
I will check it.
Thank you,

@nqkhai1706
Copy link

@nttstar I have checked again with onnxruntime==1.6, the result is same with onnxruntume==1.8.0.
Do you have any other idea why my model got error on your side?
Thank you,

@anxiangsir
Copy link
Contributor

@nttstar I have checked again with onnxruntime==1.6, the result is same with onnxruntume==1.8.0.
Do you have any other idea why my model got error on your side?
Thank you,

This is our error:

>>> import onnxruntime
>>> session = onnxruntime.InferenceSession("ms1m_groupface_resnet101.onnx",  None)                                                                                                                      Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/face/miniconda3/lib/python3.7/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 206, in __init__
    self._create_inference_session(providers, provider_options)
  File "/home/face/miniconda3/lib/python3.7/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 226, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidProtobuf: [ONNXRuntimeError] : 7 : INVALID_PROTOBUF : Load model from /mnt/data/models/1624534624858999668/ms1m_groupface_resnet101.onnx failed:Protobuf parsing failed.

The environment of our system is as follows:

Driver Version: 450.80.02
Cuda compilation tools, release 10.2, V10.2.89
onnx==1.8.0
onnx-simplifier==0.3.5
onnxoptimizer==0.2.5
onnxruntime-gpu==1.6.0

@zhanglaplace
Copy link

测试的时候能否知道当前的图片是否有mask

@manideep2510
Copy link

@nttstar Even though I have not submitted any model today or yesterday. The portal is still showing that

Today, you have submitted 3 times, 0 remaining.

Can you please look into this? Thanks in advance.

@nttstar
Copy link
Collaborator Author

nttstar commented Jun 30, 2021

@manideep2510 What is your username?

@nttstar
Copy link
Collaborator Author

nttstar commented Jun 30, 2021

测试的时候能否知道当前的图片是否有mask

@zhanglaplace 目前不能

@manideep2510
Copy link

@manideep2510 What is your username?

My user name is sherlock.

@anxiangsir
Copy link
Contributor

anxiangsir commented Jun 30, 2021 •
edited
Loading

@nttstar Even though I have not submitted any model today or yesterday. The portal is still showing that

Today, you have submitted 3 times, 0 remaining.

Can you please look into this? Thanks in advance.

It's probably a bug about jet lag, and we'll fix that in no time.

@JesseEisen
Copy link

@nttstar Even though I have not submitted any model today or yesterday. The portal is still showing that

Today, you have submitted 3 times, 0 remaining.

Can you please look into this? Thanks in advance.

Hi, please provide some information related to the browser to help better debug. Like which browser you use and the version of the browser.

@nttstar
Copy link
Collaborator Author

nttstar commented Jul 1, 2021

@manideep2510 Hi, @JesseEisen is helping us build the evaluation system, please let him know your detailed issue.

@manideep2510
Copy link

I am using Mozilla Firefox Version 89.02. The portal is now showing

Today, you have submitted 8 times, -5 remaining.

But I am able to upload and make submissions.

@nttstar
Copy link
Collaborator Author

nttstar commented Jul 1, 2021

@manideep2510 Can you try Chrome?

@nttstar
Copy link
Collaborator Author

nttstar commented Mar 27, 2024

I have approved all new registrations.

@Cevikalp
Copy link

Cevikalp commented Mar 30, 2024 via email

@nttstar
Copy link
Collaborator Author

nttstar commented Mar 30, 2024

@Cevikalp Sorry we don't support the model evaluation with such high feature dimension.

@Cevikalp
Copy link

Cevikalp commented Mar 30, 2024 via email

@Cevikalp
Copy link

Cevikalp commented Mar 30, 2024 via email

@juechengxi
Copy link

您好，我想根据 MFR 数据集对我的模型进行基准测试，但没有收到任何确认电子邮件，能批准一下新注册吗？

@nttstar
Copy link
Collaborator Author

nttstar commented Apr 26, 2024

@juechengxi 已全部通过

@vivianhew
Copy link

您好，我想参加 MFR-Ongoing，注册后没有收到任何确认电子邮件，能批准一下新注册吗？

@HYL-Dave
Copy link

HYL-Dave commented Jun 4, 2024

@nttstar 你好，你们提供的 vit-l 代码训练出来就超过了 1 GB，无法上传测试，请问用什么方式可以 upload 测试？

@onnx20
Copy link

onnx20 commented Jun 7, 2024

I have approved all new registrations.

@nttstar Please request registration of MFR, username: koala, Thanks.

@nttstar
Copy link
Collaborator Author

nttstar commented Jun 7, 2024

@nttstar 你好，你们提供的 vit-l 代码训练出来就超过了 1 GB，无法上传测试，请问用什么方式可以 upload 测试？

@HYL-Dave 没有办法

@nttstar
Copy link
Collaborator Author

nttstar commented Jun 7, 2024

您好，我想参加 MFR-Ongoing，注册后没有收到任何确认电子邮件，能批准一下新注册吗？

已通过

@nttstar
Copy link
Collaborator Author

nttstar commented Jun 7, 2024

I have approved all new registrations.

@nttstar Please request registration of MFR, username: koala, Thanks.

DONE

@sssssshf
Copy link

sssssshf commented Jul 5, 2024

I have approved all new registrations.

@nttstar Please request registration of MFR, username: koala, Thanks.

DONE

您好我再使用mbf作为backbone训练的人脸识别模型 送到mfr进行测试是报错如下：
image
我用r100送入评测能够得出结果，并且两份模型的转onnx代码一致 嘛烦指导一下如何解决这个问题

@HustTD
Copy link

HustTD commented Sep 5, 2024

@nttstar Can you approve my registration? username: v.thanhtm6@vinbigdata.com, Thanks.

@nttstar
Copy link
Collaborator Author

nttstar commented Sep 8, 2024

@HustTD Done

@livermoressss
Copy link

@nttstar Could you approve my registration? username: sun8878232@gmail.com. Thanks

@nttstar
Copy link
Collaborator Author

nttstar commented Sep 22, 2024

@nttstar Could you approve my registration? username: sun8878232@gmail.com. Thanks

@livermoressss done

@MSRCZW
Copy link

MSRCZW commented Sep 25, 2024

@nttstar Could you approve my registration? username: chenzw. Thanks

@nttstar
Copy link
Collaborator Author

nttstar commented Sep 26, 2024

@nttstar Could you approve my registration? username: chenzw. Thanks

@MSRCZW done

@20144115
Copy link

20144115 commented Oct 3, 2024

@nttstar Could you approve my registration? username: tom_hust. Thanks

@harikiran17
Copy link

@nttstar Could you approve my registration? username: harikiran
Thanks

@satvikkaurav
Copy link

@nttstar Could you please approve my registration? username: satvik

@nttstar
Copy link
Collaborator Author

nttstar commented Oct 9, 2024

@20144115 @harikiran17 @satvikkaurav Done

@TranThanh96
Copy link

@nttstar what is wrong with your server? http://iccv21-mfr.com/ keep spinning

@nttstar
Copy link
Collaborator Author

nttstar commented Oct 15, 2024

@TranThanh96 Restarted, you can try again.

@Eliza-and-black
Copy link

@nttstar Could you approve my registration? username: ymd. Thanks

@nttstar
Copy link
Collaborator Author

nttstar commented Nov 19, 2024

@nttstar Could you approve my registration? username: ymd. Thanks

@Eliza-and-black Done

@zhangxueyuan2017
Copy link

@nttstar Could you please approve my registration? username: xyz Thanks

@nttstar
Copy link
Collaborator Author

nttstar commented Nov 25, 2024

@nttstar Could you please approve my registration? username: xyz Thanks

@zhangxueyuan2017 Done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests