Sanitize core library filename

[wsxiaoys]

Hi,

I'm working on support gerbil compiling in my implementation of bazel integration of gambit (https://github.com/wsxiaoys/rules_gambit).

One restriction bazel have, is only allows specific set of filenames as sources: link

[fare]

Hi! I once co-wrote bazel support for Common Lisp. What issue are you having? Is the issue that gambit generates files with # in the name? That's a problem indeed. Maybe talk with Marc Feeley on how much this matters?

[fare]

Oh, I see it might be gerbil introducing the # in some file names... @vyzo can you comment?

[wsxiaoys]

I actually fixed the "#" issue in bazelbuild/bazel#2006 to make bazel work with gambit. (updated OP)

But gerbil have names like core::<MOP>::<MOP:1>__rt.scm.

[vyzo]

The "#" is used by convention in Gambit for macro definitions that get included from other files.

The <...>'s are coming from nested modules that get compiled into separate module files; <MOP> is the name of the nested module, that has another nested module <MOP:1>.

I guess we could sanitize the generated file names to not include invalid shell characters.

[wsxiaoys]

That should work (we also need regenerate bootstrap files, right?).

Involved characters are: <, > and :.

[vyzo]

It requires some small changes in the compiler, and then a new bootstrap generation -- but that's something that is easy to do nowadays.
We will also have to pick a different nested module name separator (that's where the :: is coming from)

[wsxiaoys]

# is safe to use as separator it has been released in bazel since 0.5.4.

[vyzo]

Can we make a list of the problematic characters so that I can have the compiler do generic mangling for all of them?

[wsxiaoys]

We could reuse bazel's sanitize target name definition: [0-9a-zA-Z_@-+,=~# .()$]+.

[vyzo]

so these are the acceptable characters in bazel?

[wsxiaoys]

For filenames yes.

It's slightly stricter for package name (e.g. , is not allowed), but it's not affecting our case.

[vyzo]

Can we add : to the list as well? it would be nice to not have to come up with a new separator.

[wsxiaoys]

: is slightly tricker, bazel used it to represent sub-packages.

For example I could run bazel run //tests/ffi:main in http://github.com/wsxiaoys/rules_gambit, where main is a package declared in tests/ffi/BUILD.

[vyzo]

ok, I guess we could use $ java-style :)

[fare]

Also, the : character won't work on macOS, a Lisp Machine, or VMS... not that it really matters... :-)

[wsxiaoys]

Regarding of a generic approach to handle special characters in module name, how about simply replace characters other than [a-zA-Z0-9_] with __asciicode__, just like gambit-scheme's?

entry point are:
compile-scm-file
core-resolve-library-module-path

Am I missing any parts?

[vyzo]

yes, that would work.

[vyzo]

So the following characters are restricted and replaced with _ by the compiler for compiled module artifacts:

:#<>&!?*;()[]{}|'`"\

Note that this has implications on library module filenames and package names; they are disallowed (but not enforced) there.

Merged in d9ff94e
Implementation in 42af642

[vyzo]

@wsxiaoys does this solve your issue?

[wsxiaoys]

@vyzo Thanks for the fix! Yes, the built file is now accepted by bazel :)