lexer trips up on python code-intensive sections

[sqlalchemy-bot]

Migrated issue, originally created by Michael Bayer (@zzzeek)

im assuming the condition will usually, if not always, involve quotes. such as:

${foo and "|" or "bar"}

${"some text}"}

<% print "hi %>"%>

what we have to do is take the match sections that look for ${} and <% %>, and break them down to first match the ${ or <% symbol. then a new function will be called, using a signature like this:

   def match_python_to_str(self, *strings):

used like this:

   match = self.match(r'<%')
   if match:
       (text, matched) = self.match_python_to_str('%>')

or multiple:

   (text, matched) = self.match_python_to_str('|', '}')

the function performs submatches at the current match point, matching on quoted sections, i.e. ''', """, ', ", and consuming quotes in a balanced fashion. only at the "balance" points do we also try to match one of }, | or %>. the function then returns the full range of text corresponding to where it was first called vs. where it found an appropriate end character.

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

implemented in [changeset:66]

[sqlalchemy-bot]

Changes by Michael Bayer (@zzzeek):

• changed status to closed