Syntax highlighting for PTX code #275
Conversation
Codecov Report
@@ Coverage Diff @@
## master #275 +/- ##
==========================================
- Coverage 86.75% 86.19% -0.56%
==========================================
Files 22 22
Lines 2023 2246 +223
==========================================
+ Hits 1755 1936 +181
- Misses 268 310 +42
Continue to review full report at Codecov.
|
maleadt
force-pushed
the
ap/highlight-ptx
branch
from
59015f8
to
bf64c20
Compare
|
Thanks! Pushed some small changes. I think we'll have to improve the tokenizer before this can work though, simple things like |
src/reflection.jl
Outdated
| istok(r"^0[dD]{hexdigit}{16}") # double-precision floating point | ||
| print_tok(tok, :number) | ||
| # TODO: function names | ||
| # TODO: labels as RHS |
There was a problem hiding this comment.
TODO: labels as RHS
What does this mean ?
There was a problem hiding this comment.
There's a distinct color for labels (foo:), but that isn't matched when the label is used as a right-hand-side value (bra foo). Generally that requires a proper parser though, but if the cases are limited we could special-case it.
|
Yes, I missed the tokeniser failures. I have added some additional regex to identify the type of token. Also, some colour changes have been reverted but we will fix that before merge. I wanted to check if the current strategy is fine. It seems easy to add support for functions and other features.
Is there any use for this besides code highlighting ? I don't think a BNF for this should be particularly difficult but then I don't have any experience in this area. There do appear to be some ANTLR to BNF converters out there. Maybe it's worth looking into one of those. Remaining TODOs:
|
Currently not, no. If we can get reasonable regex-based highlighting, it's probably overkill to consider a proper grammar (which would also incur some latency when getting compiled), but if we end up having to write down individual regular expressions for every token type it's a small step to a real grammer (where expressing those might be more compact even). But I'm just spitballing here, I don't have experience with parser generators in Julia either. |
|
I wrote some implementations and got confused as to whether we want to be minimalist here or do something closer to an actual parser with proper tokens. The Julia replacement for Pygments is Highlights.jl and they have an issue on this. The best course of action imo is to implement this there and use that here. I am going to implement a lexer there first. If that one is unsatisfactory I can resume this PR otherwise close this PR. |
|
I didn't know about Highlights.jl -- that looks to be the best place to implement PTX support, indeed. Note that you can also find some regular expression-like things in the PTX ISA manual, e.g. https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#identifiers, that may make it easier to implement a lexer. |
Removes 🐍 pygments dependency and adds highlighting similar to base.
We can create a different highlighting scheme which should be easy to do but I don't have any ideas about that.
We have a simple lexer which processes one line at a time.
In general a PTX file has
.reg .b32 r1, r2label: mov.b32 r1, %tid.x(NVIDIA PTX docs)
We can use regex to distinguish between the various tokens and do a more complex theme like:
But I would personally not prefer that.