Make \cdot and \cdotp operators equivalent

[iagobaapellaniz]

It includes an enhancement discussed in #25098

(See also https://discourse.julialang.org/t/inconsistencies-in-some-methods-and-more/7788)

[fredrikekre]

Shouldn't this be done as in #19464?

[iagobaapellaniz]

Probably. I'm not an expert on how to do this.

I'll follow this thread and I'll study the PR #19464 in order to figure out how it can be done.

Thanks

[stevengj]

Yes, the right way to do this is to just add a line to https://github.com/JuliaLang/julia/blob/master/src/flisp/julia_charmap.h to make this substitution.

Also add a note to NEWS, documentation at the end of https://github.com/JuliaLang/julia/blob/master/doc/src/manual/variables.md#allowed-variable-names (along with the note about ε), and add a test.

[stevengj]

(Note also that you shouldn't open a new PR. Just revert the commit in this branch and push new commits to fix it the right way. It will all get squashed into a single commit when it is merged. That way all of the discussion will go together.)

[iagobaapellaniz]

Yes. I'll do that :-) Thanks @stevengj

[stevengj]

Maybe link https://en.wikipedia.org/wiki/Interpunct

[stevengj]

Needs a NEWS item, then should be good to go.

[iagobaapellaniz]

Not yet, I think. I don't know if I'm missing something but Julia still complains about \cdotp[TAB] with

julia> ·
ERROR: syntax: invalid character "·"

I'm pretty sure that this should be fixed in julia-parser.scm. HELP needed.

Summary: I checked a random equivalence and it works {€ -> +}, now I can sum with 2 € 2. On the other hand the error of invalid character is present at that file, but it would take me too long to learn SCHEME. We need to U+00b7 ('·': middle dot) be a valid character for Julia.

julia> €
+ (generic function with 177 methods)

Could I suggest @JeffBezanson to review this?

[stevengj]

Oh, I know the problem. The issue is that normalization of symbols only happens after they are parsed in the file, and since · is a punctuation character (Unicode category Po) it is not treated as a valid operator symbol. This doesn't happen with € because € is already a valid identifier.

The fix, at first glance, should be to add · to the list of operators at the top of src/julia-parser.scm. Just add '· to the prec-times list.

[iagobaapellaniz]

I checked this and it does not work. Now it is a valid identifier but it seems that the mapping arrives before since

julia> ·
ERROR: UndefVarError: · not defined

Maybe this is necessary, but still does not fix it. The character inserted and the displayed in the error are U+00b7.

[stevengj]

It could be that the normalization is not done to operators (as opposed to ordinary variable names), let me check.

[stevengj]

Yup, the normalization is only done to variable names, not to operators. There would need to be a call to a normalization routine at the end of read-operator.

[iagobaapellaniz]

Nice! With that I think I'm able to finish this. Thanks a lot!

[stevengj]

No, this is wrong; accum-julia-symbol reads an identifier (not an operator) from the input stream, leaving nothing else for read-operator to read.

I think you will need to export a new flisp function to normalize a symbol

[iagobaapellaniz]

I added a new commit. It works now after make cleanall and make

[iagobaapellaniz]

It works now for me after make cleanall and make

julia> ⋅   # \cdotp[TAB] or U+00b7
dot (generic function with 9 methods)
julia> ·   # \cdot[TAB] or U+22c5
dot (generic function with 9 methods)
julia> dot
dot (generic function with 9 methods)

Should I do it in another way? I'm not able to find an alternative.

[stevengj]

Ping, what this the status here?

[iagobaapellaniz]

I'm not able to write anything that really works. I think we should work on the parser, which is written in SCM/C++. At the moment I have no time to learn all this.

It seems that there is a function in the parser that only works to make equivalent the identifiers, but not the operators. Help is needed, or a new PR should be opened to solve the issue.

[JeffBezanson]

I can help with this at some point, but it's not urgent since it's a non-breaking change.

[mlhetland]

Some relevant discussion on Discourse.

[mlhetland]

A summary of the Discourse discussion based on my understanding:

• Greek ano teleia (U+0387) is a valid identifier character. This is motivated by it having the Other_ID_Continue property, as recommended by UAX notofonts/noto-fonts#31.
• Middle dot (U+00b7, \cdotp) is not a valid identifier character, despite falling under the same UAX recommendation, and being canonically equivalent to the ano teleia.
• Julia currently normalizes the (valid) ano teleia into the (invalid) middle dot.
• Ano teleia, middle dot and dot operator look quite similar in many fonts; the first two are often identical (cf. ··⋅).
• The middle dot is easy to type on many keyboards.

[stevengj]

PR #28167 removes U+0387 from the list of valid identifiers. If this PR is revived at some point, we will want to normalize both U+0387 and U+00b7 to U+22c5 (\cdot).

[stevengj]

I rebased this on master and fixed it by adding code to normalize operators as they are parsed, as I suggested above. U+0387 is also treated as equivalent. The test now passes for me.

[iagobaapellaniz]

Thanks! This will help a lot of people using a spanish keyboard layout, since U+00B7 is just typed by pressing SHIFT+3. Thanks again

[iagobaapellaniz]

Could this make its way to 1.6?

[stevengj]

Too late for 1.6, I think.

[vtjnash]

Triage didn't seem to see why this was put on the list. It seemed like nobody had objected to merging?