CLEVA Schema #1864
CLEVA Schema #1864
Conversation
2. Use chinese_bleu_1 in Pinyin Transliteration Task
Add Pinyin Transliteration Scenario
Add Scenarios: Instruction Following, Classical Chinese Understanding, and Sentiment Analysis
CLEVA Robustness Perturbations
Fix Python style
2. Change disinformation to fact_check
Load Prompt Setting From File
Add CLEVA Schema
Update Download links
Remove links.
| @@ -43,7 +43,15 @@ class Description(PerturbationDescription): | |||
| name: str = "chinese_typos" | |||
|
|
|||
| # For downloading resources | |||
| ASSET_URL = "https://drive.google.com/uc?id=1p5mldLpKxI-63H8YEruGJghtD1dZJI8k" | |||
| ASSET_URL = "http://39.108.215.175/assets/butter_finger" | |||
There was a problem hiding this comment.
In general, it is better to have this be a domain name due to (1) HTTPS security and (2) it's easier to lose an allocated IP address than a domain name. Is there any plan to change the URL back to a HTTPS domain name?
For instance, these assets could be hosted in a GitHub repository.
There was a problem hiding this comment.
Because of the Great Firewall, Google Drive is not accessible in China. For this reason, and to ensure our project's long-term stability, we have rented a cloud server with a two-year contract. This means that this IP address and related service will remain stable in the foreseeable future.
In the meantime, we are applying for a domain and an SSL certificate. We estimate the process should be completed within 2 weeks. Immediately after, we will update the URLs to use the new HTTPS domain name.
Please let me know how you feel about this plan. We truly value your suggestions. Thank you!
There was a problem hiding this comment.
Thanks for sharing your plans. Sounds good to me.
0928 Review Adjustment
Fix Typos: math_world_problem -> math_word_problem
| @@ -43,7 +43,15 @@ class Description(PerturbationDescription): | |||
| name: str = "chinese_typos" | |||
|
|
|||
| # For downloading resources | |||
| ASSET_URL = "https://drive.google.com/uc?id=1p5mldLpKxI-63H8YEruGJghtD1dZJI8k" | |||
| ASSET_URL = "http://39.108.215.175/assets/butter_finger" | |||
There was a problem hiding this comment.
Thanks for sharing your plans. Sounds good to me.
This PR mainly updates the corresponding entries of CLEVA in
schema.yaml. It also updates the links for accessing CLEVA data, such that users do not need to download the whole dataset if they just want to run on a few CLEVA scenarios.