Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Parser #808

Merged
merged 2 commits into from
Oct 21, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions components.js
Original file line number Diff line number Diff line change
Expand Up @@ -341,6 +341,11 @@ var components = {
"title": "PARI/GP",
"owner": "Golmote"
},
"parser": {
"title": "Parser",
"require": "markup",
"owner": "Golmote"
},
"pascal": {
"title": "Pascal",
"owner": "Golmote"
Expand Down
2 changes: 1 addition & 1 deletion components/prism-markup.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Prism.languages.markup = {
'doctype': /<!DOCTYPE[\w\W]+?>/,
'cdata': /<!\[CDATA\[[\w\W]*?]]>/i,
'tag': {
pattern: /<\/?[^\s>\/=.]+(?:\s+[^\s>\/=]+(?:=(?:("|')(?:\\\1|\\?(?!\1)[\w\W])*\1|[^\s'">=]+))?)*\s*\/?>/i,
pattern: /<\/?(?!\d)[^\s>\/=.$<]+(?:\s+[^\s>\/=]+(?:=(?:("|')(?:\\\1|\\?(?!\1)[\w\W])*\1|[^\s'">=]+))?)*\s*\/?>/i,
inside: {
'tag': {
pattern: /^<\/?[^\s>\/]+/i,
Expand Down
2 changes: 1 addition & 1 deletion components/prism-markup.min.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

66 changes: 66 additions & 0 deletions components/prism-parser.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
Prism.languages.parser = Prism.languages.extend('markup', {
'keyword': {
pattern: /(^|[^^])(?:\^(?:case|eval|for|if|switch|throw)\b|@(?:BASE|CLASS|GET(?:_DEFAULT)?|OPTIONS|SET_DEFAULT|USE)\b)/,
lookbehind: true
},
'variable': {
pattern: /(^|[^^])\B\$(?:\w+|(?=[.\{]))(?:(?:\.|::?)\w+)*(?:\.|::?)?/,
lookbehind: true,
inside: {
'punctuation': /\.|:+/
}
},
'function': {
pattern: /(^|[^^])\B[@^]\w+(?:(?:\.|::?)\w+)*(?:\.|::?)?/,
lookbehind: true,
inside: {
'keyword': {
pattern: /(^@)(?:GET_|SET_)/,
lookbehind: true
},
'punctuation': /\.|:+/
}
},
'escape': {
pattern: /\^(?:[$^;@()\[\]{}"':]|#[a-f\d]*)/i,
alias: 'builtin'
},
'punctuation': /[\[\](){};]/
});
Prism.languages.insertBefore('parser', 'keyword', {
'parser-comment': {
pattern: /(\s)#.*/,
lookbehind: true,
alias: 'comment'
},
'expression': {
// Allow for 3 levels of depth
pattern: /(^|[^^])\((?:[^()]|\((?:[^()]|\((?:[^()])*\))*\))*\)/,
lookbehind: true,
inside: {
'string': {
pattern: /(^|[^^])(["'])(?:(?!\2)[^^]|\^[\s\S])*\2/,
lookbehind: true
},
'keyword': Prism.languages.parser.keyword,
'variable': Prism.languages.parser.variable,
'function': Prism.languages.parser.function,
'boolean': /\b(?:true|false)\b/,
'number': /\b(?:0x[a-f\d]+|\d+\.?\d*(?:e[+-]?\d+)?)\b/i,
'escape': Prism.languages.parser.escape,
'operator': /[~+*\/\\%]|!(?:\|\|?|=)?|&&?|\|\|?|==|<[<=]?|>[>=]?|-[fd]?|\b(?:def|eq|ge|gt|in|is|le|lt|ne)\b/,
'punctuation': Prism.languages.parser.punctuation
}
}
});
Prism.languages.insertBefore('inside', 'punctuation', {
'expression': Prism.languages.parser.expression,
'keyword': Prism.languages.parser.keyword,
'variable': Prism.languages.parser.variable,
'function': Prism.languages.parser.function,
'escape': Prism.languages.parser.escape,
'parser-punctuation': {
pattern: Prism.languages.parser.punctuation,
alias: 'punctuation'
}
}, Prism.languages.parser['tag'].inside['attr-value']);
1 change: 1 addition & 0 deletions components/prism-parser.min.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

97 changes: 97 additions & 0 deletions examples/prism-parser.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
<h1>Parser</h1>
<p>To use this language, use the class "language-parser".</p>

<h2>Comments</h2>
<pre><code>$foo[bar] # Some comment</code></pre>

<h2>Variables and functions</h2>
<pre><code>@navigation[]
$sections[^table::load[sections.cfg]]
$sections.uri</code></pre>

<h2>Literals</h2>
<pre><code>$foo(3+$bar)
^switch[$sMode]{
^case[def]{$result(true)}
}
^if(in "/news/"){}</code></pre>

<h2>Escape sequences</h2>
<pre><code>^^
^"
^;</code></pre>

<h2>Embedded in markup</h2>
<pre><code>&lt;nav>
&lt;ul>
^sections.menu{
&lt;li>
&lt;a href="$sections.uri">$sections.name&lt;/a>
&lt;/li>
}
&lt;/ul>
&lt;/nav></code></pre>

<h2>Full example</h2>
<pre><code>@CLASS
MyTable

@create[uParam]
^switch[$uParam.CLASS_NAME]{
^case[string;void]{$t[^table::create{$uParam}]}
^case[table;MyTable]{$t[^table::create[$uParam]]}
^case[DEFAULT]{^throw[MyTable;Unsupported type $uParam.CLASS_NAME]}
}

# method will return value in different calling contexts
@GET[sMode]
^switch[$sMode]{
^case[table]{$result[$t]}
^case[bool]{$result($t!=0)}
^case[def]{$result(true)}
^case[expression;double]{$result($t)}
^case[DEFAULT]{^throw[MyTable;Unsupported mode '$sMode']}
}


# method will handle access to the "columns"
@GET_DEFAULT[sName]
$result[$t.$sName]


# wrappers for all existing methods are required
@count[]
^t.count[]

@menu[jCode;sSeparator]
^t.menu{$jCode}[$sSeparator]


# new functionality
@remove[iOffset;iLimit]
$iLimit(^iLimit.int(0))
$t[^t.select(^t.offset[]<$iOffset || ^t.offset[]>=$iOffset+$iLimit)]</code></pre>

<h2>Known failures</h2>
<p>There are certain edge cases where Prism will fail.
There are always such cases in every regex-based syntax highlighter.
However, Prism dares to be open and honest about them.
If a failure is listed here, it doesn’t mean it will never be fixed. This is more of a “known bugs” list, just with a certain type of bug.
</p>

<h3>Comment-like substrings</h3>
<pre><code>("fo #o")</code></pre>

<h3>Tag-like substrings</h3>
<pre><code>("fo&lt;div>o")</code></pre>

<h3>Code block starting with a comment</h3>
<pre><code># Doesn't work
# Does work</code></pre>
<pre><code> # Does work when prefixed with a space</code></pre>

<h3>Comments inside expressions break literals and operators</h3>
<pre><code>^if(
$age>=4 # not too young
&& $age<=80 # and not too old
)</code></pre>
2 changes: 1 addition & 1 deletion plugins/autoloader/prism-autoloader.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
}

// The dependencies map is built automatically with gulp
var lang_dependencies = /*languages_placeholder[*/{"javascript":"clike","actionscript":"javascript","aspnet":"markup","bison":"c","c":"clike","csharp":"clike","cpp":"c","coffeescript":"javascript","crystal":"ruby","css-extras":"css","d":"clike","dart":"clike","fsharp":"clike","glsl":"clike","go":"clike","groovy":"clike","haml":"ruby","handlebars":"markup","jade":"javascript","java":"clike","less":"css","markdown":"markup","nginx":"clike","objectivec":"c","php":"clike","php-extras":"php","processing":"clike","qore":"clike","jsx":["markup","javascript"],"ruby":"clike","sass":"css","scss":"css","scala":"java","smarty":"markup","swift":"clike","textile":"markup","twig":"markup","typescript":"javascript","wiki":"markup"}/*]*/;
var lang_dependencies = /*languages_placeholder[*/{"javascript":"clike","actionscript":"javascript","aspnet":"markup","bison":"c","c":"clike","csharp":"clike","cpp":"c","coffeescript":"javascript","crystal":"ruby","css-extras":"css","d":"clike","dart":"clike","fsharp":"clike","glsl":"clike","go":"clike","groovy":"clike","haml":"ruby","handlebars":"markup","jade":"javascript","java":"clike","less":"css","markdown":"markup","nginx":"clike","objectivec":"c","parser":"markup","php":"clike","php-extras":"php","processing":"clike","qore":"clike","jsx":["markup","javascript"],"ruby":"clike","sass":"css","scss":"css","scala":"java","smarty":"markup","swift":"clike","textile":"markup","twig":"markup","typescript":"javascript","wiki":"markup"}/*]*/;

var lang_data = {};

Expand Down
2 changes: 1 addition & 1 deletion plugins/autoloader/prism-autoloader.min.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion prism.js
Original file line number Diff line number Diff line change
Expand Up @@ -449,7 +449,7 @@ Prism.languages.markup = {
'doctype': /<!DOCTYPE[\w\W]+?>/,
'cdata': /<!\[CDATA\[[\w\W]*?]]>/i,
'tag': {
pattern: /<\/?[^\s>\/=.]+(?:\s+[^\s>\/=]+(?:=(?:("|')(?:\\\1|\\?(?!\1)[\w\W])*\1|[^\s'">=]+))?)*\s*\/?>/i,
pattern: /<\/?(?!\d)[^\s>\/=.$<]+(?:\s+[^\s>\/=]+(?:=(?:("|')(?:\\\1|\\?(?!\1)[\w\W])*\1|[^\s'">=]+))?)*\s*\/?>/i,
inside: {
'tag': {
pattern: /^<\/?[^\s>\/]+/i,
Expand Down
21 changes: 21 additions & 0 deletions tests/languages/parser/boolean_feature.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
(true)
(false)

----------------------------------------------------

[
["expression", [
["punctuation", "("],
["boolean", "true"],
["punctuation", ")"]
]],
["expression", [
["punctuation", "("],
["boolean", "false"],
["punctuation", ")"]
]]
]

----------------------------------------------------

Checks for booleans inside expressions.
68 changes: 68 additions & 0 deletions tests/languages/parser/escape_feature.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
^$
^^
^;
^@
^(
^)
^[
^]
^{
^}
^"
^'
^:
^#
^#20
^#af
^#AF

^^date::now
^$foobar

<div class="foo^^bar">

----------------------------------------------------

[
["escape", "^$"],
["escape", "^^"],
["escape", "^;"],
["escape", "^@"],
["escape", "^("],
["escape", "^)"],
["escape", "^["],
["escape", "^]"],
["escape", "^{"],
["escape", "^}"],
["escape", "^\""],
["escape", "^'"],
["escape", "^:"],
["escape", "^#"],
["escape", "^#20"],
["escape", "^#af"],
["escape", "^#AF"],

["escape", "^^"], "date::now\r\n",
["escape", "^$"], "foobar\r\n\r\n",

["tag", [
["tag", [
["punctuation", "<"],
"div"
]],
["attr-name", ["class"]],
["attr-value", [
["punctuation", "="],
["punctuation", "\""],
"foo",
["escape", "^^"],
"bar",
["punctuation", "\""]
]],
["punctuation", ">"]
]]
]

----------------------------------------------------

Checks for escapes.
58 changes: 58 additions & 0 deletions tests/languages/parser/expression_feature.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
((3-(9-2))*4)
^eval(4+2)

<div class="foo-^eval(4+2)">

----------------------------------------------------

[
["expression", [
["punctuation", "("],
["punctuation", "("],
["number", "3"],
["operator", "-"],
["punctuation", "("],
["number", "9"],
["operator", "-"],
["number", "2"],
["punctuation", ")"],
["punctuation", ")"],
["operator", "*"],
["number", "4"],
["punctuation", ")"]
]],
["keyword", "^eval"],

["expression", [
["punctuation", "("],
["number", "4"],
["operator", "+"],
["number", "2"],
["punctuation", ")"]
]],

["tag", [
["tag", [
["punctuation", "<"],
"div"
]],
["attr-name", ["class"]],
["attr-value", [
["punctuation", "="],
["punctuation", "\""],
"foo-",
["keyword", "^eval"],
["expression", [
["punctuation", "("],
["number", "4"], ["operator", "+"], ["number", "2"],
["punctuation", ")"]
]],
["punctuation", "\""]
]],
["punctuation", ">"]
]]
]

----------------------------------------------------

Checks for expressions, up to 3 levels of depth.
Loading