Saved file cannot be read

[oscargus]

Based on the discussion in #1188 I created an empty database with a single entry with a single field containing ––, which is saved as

% Encoding: UTF-8

@Article{,
  author = {\&} # 8211;& #{8211;},
}

@Comment{jabref-meta: databaseType:bibtex;}

Log messages:

WARN net.sf.jabref.importer.fileformat.BibtexParser - Could not parse entry
java.io.IOException: Error in line 4 or above: Empty text token.
This could be caused by a missing comma between two fields.

WARN net.sf.jabref.importer.fileformat.BibtexParser - Could not parse entry
java.io.IOException: Error in line 4: Expected { or ( but received }

I notice two things here, except that it doesn't work (I think one reason is that a string cannot start with a digit as I recall it, although that is probably not the reason here). First, & is escaped. Why? What if I want to have a & in my field? One reason could be in an abstract where a slightly more complicated equation or tabular might theoretically be included. Second, there should probably be a space after the second #. Although not strictly required, it would be nice for symmetry.

Suggestion: check if what JabRef believes to be a string, i.e., content between two # is a valid string label, if not make the user sort it out,

[lenhard]

Hm, I wonder why and where all this escaping and reformatting takes place on save. Didn't we decide to do no automatic reformatting of entries on save except for save actions. Critically, a closing curly bracket is entered }, but no corresponding opening bracket (Note: the first opening bracket is just to signal the beginning of the field value). Afterwards there is also a missing closing bracket.

This is not a problem in the parser, which just quits after author = {\&}. This is exactly what it should do since all content afterwards violates the BibTeX syntax. The serialization/escaping is just wrong and writes too few brackets.

[oscargus]

I do not fully agree on the details. Assume a field in JabRef which reads a#b#c, this should be serialized as {a} # b # {c} (just as #jan# is serialized as jan, i.e. without {}). So that part is still OK (apart from the missing space for symmetry after the second #) and that will indeed load correctly, I tried the example in the first post with only characters inbetween the ## and it both saved and loaded OK. Hence, I conclude that the parser is checking for a string label (b or jan or 8211;&), where there are some formal requirements on how it should look.

Now, as JabRef decided to use ## to separate field content from string labels, we can make the assumption that everything between two ## is a string label and then we put requirements on how that should look. A single #, as in #1188 is harder to actually know if it is expected or a problem.

[lenhard]

We just tested this and it is still broken (but fortunately nobody else seems to have bumped into it). @oscargus: Do you maybe want to have a go at it?

[lenhard]

I just tested this again. At the very least, the file can be read now, just that one particular entry will be ignored.

That is a step up from not reading the file at all, I would say. Essentially, what is missing here is a warning on writing the database to disk or when entering the illegal characters.