Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix DBLPFetcherTest -- dblp changed the format #5582

Merged
merged 7 commits into from
Nov 10, 2019
Merged

Fix DBLPFetcherTest -- dblp changed the format #5582

merged 7 commits into from
Nov 10, 2019

Conversation

koppor
Copy link
Member

@koppor koppor commented Nov 9, 2019 •
edited
Loading

dblp changed the format of URLs - _ is now escaped.

Clean URLs in DBLPFetcher.

Additionally:

  • Style RemoveLatexCommandsFormatter
  • Add tests for RemoveLatexCommandsFormatter
  • Add support for removing single and multiple whitespaces after a command
  • Split tests in RemoveBracketsTest
  • Fix casing in CleanupUrlFormatter (to match style and current name of test class CleanUpFormatterTest)
  • Fix casing in some ...Url.. method names
  • Format CleanupUrlFormatter

@koppor koppor added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Nov 9, 2019
tobiasdiez
tobiasdiez requested changes Nov 9, 2019
View reviewed changes
Copy link
Member

@tobiasdiez tobiasdiez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new URL does not work, while the old one does. Thus, I think the fetcher needs to be changed to decode the url.

@Siedlerchr
Copy link
Member

The fetcher should not store escaped URLs. The URL field is treated as verbatim field in biblatex, so no escaping necessarily

@koppor koppor removed the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Nov 9, 2019
@koppor koppor removed the request for review from johannes-manner November 9, 2019 18:53
@koppor koppor changed the title Fix DBLPFetcherTest -- dblp changed the format [WIP] Fix DBLPFetcherTest -- dblp changed the format Nov 9, 2019
@koppor koppor force-pushed the fix-dblp-test branch 2 times, most recently from 76f3919 to 89abea3 Compare November 9, 2019 21:48
@koppor
Clean URLs in DBLPFetcher
5f7fe7e 
- Style RemoveLatexCommandsFormatter
- Add tests for RemoveLatexCommandsFormatter
- Add support for removing single and multiple whitespaces after a command
- Split tests in RemoveBracketsTest
- Fix casing in CleanupUrlFormatter (to match style and current name of test class CleanUpFormatterTest)
- Fix casing in some ...Url.. method names
- Format CleanupUrlFormatter
@koppor koppor changed the title [WIP] Fix DBLPFetcherTest -- dblp changed the format Fix DBLPFetcherTest -- dblp changed the format Nov 9, 2019
@koppor koppor added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Nov 9, 2019
@koppor
Copy link
Member Author

koppor commented Nov 9, 2019

OK, I included the CleanupUrlFormatter and adapted it accordingly. While doing that, I also added some tests. See details in the updated PR description.

tobiasdiez
tobiasdiez reviewed Nov 9, 2019
View reviewed changes
src/main/java/org/jabref/logic/formatter/bibtexfields/CleanupUrlFormatter.java Outdated
LOGGER.warn("Used unsupported character encoding", e);
}
return decodedLink;

String result = new RemoveLatexCommandsFormatter().format(decodedLink);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although somewhat of an edge case, but doesn't this lead to problems with the valid url jabref.org/test%5Ctest (which I guess would be reduced to jabref.org/test instead of the expected jabref.org/test\test?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not this LOC in particularly, but in general: Yes. I added a test case for that.

In the fetcher itself, I use the RemoveLatexCommandsFormatter directly. I did not know how to convert a LayoutFormatter to a Formatter. Thus, I "manually" updated the field in cbb3836. Moments later, I worked on LayoutFormatterBasedFormatter which wraps a LayoutFormatter in a Formatter and IMHO made the code more nice.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still unsure if we should really remove latex commands from the URL. I guess this is only an issue for the DBLP fetcher and nowhere else.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, now I got it. Reverted that in 83cbb3d

@koppor koppor merged commit de4a1ae into master Nov 10, 2019
@koppor koppor deleted the fix-dblp-test branch November 10, 2019 13:59
@koppor koppor removed the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Nov 10, 2019
@koppor koppor mentioned this pull request Nov 11, 2019
Closed
koppor pushed a commit that referenced this pull request Aug 15, 2021
Squashed 'buildres/csl/csl-styles/' changes from ec4a4c0..4703b5d
1ee45cb 
4703b5d [Frontiers in ...] Remove formatting for volume (#5580)
3a3eb0c Create journal-of-clinical-and-translational-science.csl (#5588)
cf08af8 Create clinical-pulmonary-medicine.csl (#5589)
98985b1 Create cancer-drug-resistance.csl (#5590)
0af101d Create american-journal-of-sociology.csl (#5591)
b24e1f5 Create san-francisco-estuary-and-watershed-science.csl (#5592)
6f9323c revise Manchester UP monograph style (note-bib) (#5587)
e6f982b Delete harper-adams (#5585)
1eef78f Create international-journal-of-clinical-research.csl (#5584)
f28865e OSCOLA.csl: Add publiser info to generic/working paper secondary sources (CSL type "article"). (#5541)
197ceb1 Create system-dynamics-review.csl (#5575)
1d33da2 Create microcirculation.csl (#5577)
d06f625 Create iawa-journal.csl (#5576)
4705585 Update for corrections (#5578)
c1c7852 Create praxis.csl (#5579)
e012054 Create European Review of Agricultural Economics style (#5567)
b2fa912 Create biomarkers.csl (#5582)
1b00ec0 Create journal-of-international-studies.csl (#5574)
79a5d9c Switch MLA to 9th edition (#5561)
011fef3 Update for corrections (#5573)
ff5490c Create zeitschrift-fur-geschichtsdidaktik.csl (#5569)
9093f2f Create etri-journal.csl (#5570)
c9ba757 Create radiation-protection-dosimetry.csl (#5571)
60632bc Create horttechnology.csl (#5572)
e63820b Add a new journal (magnetic resonance in medical sciences) (#5542)
b656885 Create zeitschrift-fur-papyrologie-und-epigraphik.csl (#5548)
25e1eef Update technische-universitat-dresden-linguistik.csl (#5563)
c0c0da4 Create the-american-journal-of-dermatopathology.csl (#5564)
63f7db9 Update journal-of-the-american-association-of-laboratory-animal-scien… (#5566)
ba64225 Update institut-français-darcheologie-orientale-en.csl (#5553)
32e9031 Create institut-francais-darcheologie-orientale-domaine-des-etudes-arabes-en.csl (#5551)
23e2cf5 Create institut-francais-darcheologie-orientale-domaine-des-etudes-arabes.csl (#5550)
3e1a3f3 Update institut-français-darcheologie-orientale.csl (#5552)
176997d Update veterinary-pathology.csl (#5554)
ee035c1 Update mycologia.csl (#5555)
eb4b3a6 Create international-journal-of-automotive-technology.csl (#5557)
3c4e727 Bump addressable from 2.7.0 to 2.8.0 (#5558)
ccb81c8 Create szociologiai-szemle.csl (#5547)
ed9d740 Update brain.csl to ama.csl dependent (#5546)
ab499f2 Create iran-manual-style (#5350)
6e2b13c Update and rename dependent/estuaries-and-coasts.csl to estuaries-and-coasts.csl (#5534)
8441ff9 Create fachhochschule-suedwestfalen.csl (#5459)
8439cf4 Create freie-hochschule-stuttgart.csl (#5466)
076c513 Update midwestern-baptist-theological-seminary.csl (#5420)
350b900 Update veterinary-pathology.csl (#5543)
06261c0 Create United States International Trade Commission.csl (#5508)
7ae505d Add support for blog posts (CSL: post-weblog). (#5536)
93c97ea Update tud bwl marketing style (#5538)
4308791 Create the-chinese-journal-of-international-politics.csl (#5537)
09f3362 Add hanging-indents to infoclio.ch styles (#5540)
d4dc679 Further improvements to norsk-henvisningsstandad-for-rettsvitenskapelige-tekster.csl (#5539)

git-subtree-dir: buildres/csl/csl-styles
git-subtree-split: 4703b5d
Siedlerchr pushed a commit that referenced this pull request Aug 15, 2021
@koppor
[Bot] Update CSL styles (#7985)
2bb3328 
* Squashed 'buildres/csl/csl-styles/' changes from ec4a4c0..4703b5d

4703b5d [Frontiers in ...] Remove formatting for volume (#5580)
3a3eb0c Create journal-of-clinical-and-translational-science.csl (#5588)
cf08af8 Create clinical-pulmonary-medicine.csl (#5589)
98985b1 Create cancer-drug-resistance.csl (#5590)
0af101d Create american-journal-of-sociology.csl (#5591)
b24e1f5 Create san-francisco-estuary-and-watershed-science.csl (#5592)
6f9323c revise Manchester UP monograph style (note-bib) (#5587)
e6f982b Delete harper-adams (#5585)
1eef78f Create international-journal-of-clinical-research.csl (#5584)
f28865e OSCOLA.csl: Add publiser info to generic/working paper secondary sources (CSL type "article"). (#5541)
197ceb1 Create system-dynamics-review.csl (#5575)
1d33da2 Create microcirculation.csl (#5577)
d06f625 Create iawa-journal.csl (#5576)
4705585 Update for corrections (#5578)
c1c7852 Create praxis.csl (#5579)
e012054 Create European Review of Agricultural Economics style (#5567)
b2fa912 Create biomarkers.csl (#5582)
1b00ec0 Create journal-of-international-studies.csl (#5574)
79a5d9c Switch MLA to 9th edition (#5561)
011fef3 Update for corrections (#5573)
ff5490c Create zeitschrift-fur-geschichtsdidaktik.csl (#5569)
9093f2f Create etri-journal.csl (#5570)
c9ba757 Create radiation-protection-dosimetry.csl (#5571)
60632bc Create horttechnology.csl (#5572)
e63820b Add a new journal (magnetic resonance in medical sciences) (#5542)
b656885 Create zeitschrift-fur-papyrologie-und-epigraphik.csl (#5548)
25e1eef Update technische-universitat-dresden-linguistik.csl (#5563)
c0c0da4 Create the-american-journal-of-dermatopathology.csl (#5564)
63f7db9 Update journal-of-the-american-association-of-laboratory-animal-scien… (#5566)
ba64225 Update institut-français-darcheologie-orientale-en.csl (#5553)
32e9031 Create institut-francais-darcheologie-orientale-domaine-des-etudes-arabes-en.csl (#5551)
23e2cf5 Create institut-francais-darcheologie-orientale-domaine-des-etudes-arabes.csl (#5550)
3e1a3f3 Update institut-français-darcheologie-orientale.csl (#5552)
176997d Update veterinary-pathology.csl (#5554)
ee035c1 Update mycologia.csl (#5555)
eb4b3a6 Create international-journal-of-automotive-technology.csl (#5557)
3c4e727 Bump addressable from 2.7.0 to 2.8.0 (#5558)
ccb81c8 Create szociologiai-szemle.csl (#5547)
ed9d740 Update brain.csl to ama.csl dependent (#5546)
ab499f2 Create iran-manual-style (#5350)
6e2b13c Update and rename dependent/estuaries-and-coasts.csl to estuaries-and-coasts.csl (#5534)
8441ff9 Create fachhochschule-suedwestfalen.csl (#5459)
8439cf4 Create freie-hochschule-stuttgart.csl (#5466)
076c513 Update midwestern-baptist-theological-seminary.csl (#5420)
350b900 Update veterinary-pathology.csl (#5543)
06261c0 Create United States International Trade Commission.csl (#5508)
7ae505d Add support for blog posts (CSL: post-weblog). (#5536)
93c97ea Update tud bwl marketing style (#5538)
4308791 Create the-chinese-journal-of-international-politics.csl (#5537)
09f3362 Add hanging-indents to infoclio.ch styles (#5540)
d4dc679 Further improvements to norsk-henvisningsstandad-for-rettsvitenskapelige-tekster.csl (#5539)

git-subtree-dir: buildres/csl/csl-styles
git-subtree-split: 4703b5d

* Squashed 'buildres/csl/csl-locales/' changes from ec6d62a9e7..7a507fc008

7a507fc008 Fix spelling of October in Thai locale

git-subtree-dir: buildres/csl/csl-locales
git-subtree-split: 7a507fc0084edea8376ee5345879ddc2c99829ba

Co-authored-by: github actions <jabrefmail+webfeedback@gmail.com>
koppor pushed a commit that referenced this pull request Sep 1, 2021
Squashed 'buildres/csl/csl-styles/' changes from ec4a4c0..2b37392
03cae10 
2b37392 Create betriebswirtschaftliche-forschung-und-praxis.csl (#5609)
a52989c Update universitat-zu-koln-seminar-fur-abwl-und-finanzierungslehre.csl (#5610)
47c21bf Add historioPLUS (#5611)
baf11b4 Update urad-rs-za-makroekonomske-analize-in-razvoj.csl (#5607)
636affa Create pain-medicine.csl (#5606)
1d060f9 Update sist02.csl (#5568)
54e8904 Update ucl-university-college-apa.csl (#5597)
9c6308e Creation du style cahiers-du-journalisme.csl (#5595)
7dea54b Update universite-du-quebec-a-montreal.csl (#5596)
b110d07 Update journal-of-paleontology.csl (#5594)
9cc3797 Add workaround for software style in ASA (sociology) (#5464)
459dfb7 Update american-journal-of-sociology.csl (#5604)
f73d62f Create council-of-science-editors-brackets.csl (#5593)
bacf673 Remove extra accent (#5598)
55e1d40 Update system-dynamics-review.csl (#5599)
b8262e4 Update independent style for journal-of-cataract-and-refractive-surgery.csl (#5602)
04da306 Create quaderni-ministero-dei-beni-e-delle-attivita-culturali-e-del-t… (#5600)
8144588 Update journal-of-dental-research.csl (#5601)
4703b5d [Frontiers in ...] Remove formatting for volume (#5580)
3a3eb0c Create journal-of-clinical-and-translational-science.csl (#5588)
cf08af8 Create clinical-pulmonary-medicine.csl (#5589)
98985b1 Create cancer-drug-resistance.csl (#5590)
0af101d Create american-journal-of-sociology.csl (#5591)
b24e1f5 Create san-francisco-estuary-and-watershed-science.csl (#5592)
6f9323c revise Manchester UP monograph style (note-bib) (#5587)
e6f982b Delete harper-adams (#5585)
1eef78f Create international-journal-of-clinical-research.csl (#5584)
f28865e OSCOLA.csl: Add publiser info to generic/working paper secondary sources (CSL type "article"). (#5541)
197ceb1 Create system-dynamics-review.csl (#5575)
1d33da2 Create microcirculation.csl (#5577)
d06f625 Create iawa-journal.csl (#5576)
4705585 Update for corrections (#5578)
c1c7852 Create praxis.csl (#5579)
e012054 Create European Review of Agricultural Economics style (#5567)
b2fa912 Create biomarkers.csl (#5582)
1b00ec0 Create journal-of-international-studies.csl (#5574)
79a5d9c Switch MLA to 9th edition (#5561)
011fef3 Update for corrections (#5573)
ff5490c Create zeitschrift-fur-geschichtsdidaktik.csl (#5569)
9093f2f Create etri-journal.csl (#5570)
c9ba757 Create radiation-protection-dosimetry.csl (#5571)
60632bc Create horttechnology.csl (#5572)
e63820b Add a new journal (magnetic resonance in medical sciences) (#5542)
b656885 Create zeitschrift-fur-papyrologie-und-epigraphik.csl (#5548)
25e1eef Update technische-universitat-dresden-linguistik.csl (#5563)
c0c0da4 Create the-american-journal-of-dermatopathology.csl (#5564)
63f7db9 Update journal-of-the-american-association-of-laboratory-animal-scien… (#5566)
ba64225 Update institut-français-darcheologie-orientale-en.csl (#5553)
32e9031 Create institut-francais-darcheologie-orientale-domaine-des-etudes-arabes-en.csl (#5551)
23e2cf5 Create institut-francais-darcheologie-orientale-domaine-des-etudes-arabes.csl (#5550)
3e1a3f3 Update institut-français-darcheologie-orientale.csl (#5552)
176997d Update veterinary-pathology.csl (#5554)
ee035c1 Update mycologia.csl (#5555)
eb4b3a6 Create international-journal-of-automotive-technology.csl (#5557)
3c4e727 Bump addressable from 2.7.0 to 2.8.0 (#5558)
ccb81c8 Create szociologiai-szemle.csl (#5547)
ed9d740 Update brain.csl to ama.csl dependent (#5546)
ab499f2 Create iran-manual-style (#5350)
6e2b13c Update and rename dependent/estuaries-and-coasts.csl to estuaries-and-coasts.csl (#5534)
8441ff9 Create fachhochschule-suedwestfalen.csl (#5459)
8439cf4 Create freie-hochschule-stuttgart.csl (#5466)
076c513 Update midwestern-baptist-theological-seminary.csl (#5420)
350b900 Update veterinary-pathology.csl (#5543)
06261c0 Create United States International Trade Commission.csl (#5508)
7ae505d Add support for blog posts (CSL: post-weblog). (#5536)
93c97ea Update tud bwl marketing style (#5538)
4308791 Create the-chinese-journal-of-international-politics.csl (#5537)
09f3362 Add hanging-indents to infoclio.ch styles (#5540)
d4dc679 Further improvements to norsk-henvisningsstandad-for-rettsvitenskapelige-tekster.csl (#5539)

git-subtree-dir: buildres/csl/csl-styles
git-subtree-split: 2b37392
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tobiasdiez tobiasdiez tobiasdiez approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@koppor @Siedlerchr @tobiasdiez