Code

Rewrite lexer and parser#196

Code

Merged

01mf02 merged 138 commits intomain01mf02/jaq:mainfrom faster-lexer01mf02/jaq:faster-lexerCopy head branch name to clipboardJul 22, 2024

+1,620-311

Commits

Commits on May 7, 2024

• First prototype of lexer with parcours.

  01mf02

  committed
  566e9cc

Commits on May 8, 2024

• Rewrite tokeniser without library support.

  01mf02

  committed
  cd5d735

• Bare metal string parser.

  01mf02

  committed
  32d0fcd

• Finish lexer conversion.

  01mf02

  committed
  6293c14

Commits on May 9, 2024

• New Token type for simpler and faster lexing.

  01mf02

  committed
  db2b78e

• Remove parcours dependency.

  01mf02

  committed
  6909174

• Nicer handling of words.

  01mf02

  committed
  0c671f1

• Error reporting.

  01mf02

  committed
  3bde326

• Report Unicode errors.

  01mf02

  committed
  8ac7319

• Correct lexing of incorrect string escapes, e.g. "\0".

  01mf02

  committed
  9c1a3e9

• Remove unused function.

  01mf02

  committed
  4f001de

Commits on May 10, 2024

• Make lexer an object.

  01mf02

  committed
  03df662

• Improve string handling.

  01mf02

  committed
  0f91ca2

• Document.

  01mf02

  committed
  c596dd9

• Documentation, a bit of refactoring.

  01mf02

  committed
  3928fc8

• '"' is a delimiter, too.

  01mf02

  committed
  fd85d66

• Enable new lexer!

  01mf02

  committed
  9ec392f

• Compress strings.

  01mf02

  committed
  8eb8ed6

Commits on May 13, 2024

• Removed the old lexer.

  01mf02

  committed
  0845350

Commits on May 15, 2024

• Make Punct derive Eq.

  01mf02

  committed
  0a126d1

• Format.

  01mf02

  committed
  195f694

• Work on new term parser.

  01mf02

  committed
  03ea6f8

Commits on May 16, 2024

• Variable bindings.

  01mf02

  committed
  50232d8

• Fix a typo.

  01mf02

  committed
  378539b

Commits on May 17, 2024

• Parse definitions, less verbose error reporting.

  01mf02

  committed
  bbf75f1

Commits on May 18, 2024

• Properly report next token in blocks.

  01mf02

  committed
  b31f014

Commits on May 21, 2024

• Lex ?// operator.

  01mf02

  committed
  28a4994

• Simplify definitions.

  01mf02

  committed
  fef8322

• Definitions inside terms.

  01mf02

  committed
  022d913

• Remove Main; restrict object construction.

  01mf02

  committed
  458333a

Commits on May 22, 2024

• Implement elif.

  01mf02

  committed
  5087238

• More powerful object construction.

  01mf02

  committed
  bb7452c

• String parsing with non-owned strings.

  01mf02

  committed
  e5e6d94

• Restore old lexer.

  01mf02

  committed
  da08256

• Restore bincode.

  01mf02

  committed
  9e45931

• Make jaq-parse no_std again!!!

  01mf02

  committed
  6d9271f

• Reenable jaq-std, measure performance.

  Show description for 0541ee2
  01mf02

  committed
  0541ee2

• Thanks to clippy!

  01mf02

  committed
  26ada80

• Avoid unnecessary allocation.

  01mf02

  committed
  2694dc0

• Remove unused variable.

  01mf02

  committed
  408aa88

• Correctly advance input after Unicode escape sequence.

  01mf02

  committed
  07cd6e4

• Load standard library with new parser.

  01mf02

  committed
  26dee39

• More robust Unicode escape handling.

  01mf02

  committed
  0ad3dc4

Commits on May 24, 2024

• Parse module syntax.

  01mf02

  committed
  ef0b298

Commits on May 28, 2024

• Pedantic clippy.

  01mf02

  committed
  7f7bd0a

Commits on Jun 5, 2024

• Parse module syntax.

  01mf02

  committed
  a4cbd97

Commits on Jun 19, 2024

• Parse label-break.

  01mf02

  committed
  c4e42c2

Commits on Jun 23, 2024

• Labels and definitions are atoms.

  01mf02

  committed
  a5c6b73

• For {(k): v}, v must not have commas.

  01mf02

  committed
  899a882

Commits on Jun 27, 2024

• Merge branch 'main' into faster-lexer

  01mf02

  committed
  bb62357

• Move new lexer/parser to jaq-syn.

  01mf02

  committed
  edcd731

• Start work on conversion to legacy terms.

  01mf02

  committed
  ef6ee2c

• Remove empty test.

  01mf02

  committed
  bc2b0da

• Make Def fields public.

  01mf02

  committed
  645dccf

• Convert definitions and main.

  01mf02

  committed
  3209191

• More descriptive panic.

  01mf02

  committed
  b31f89f

Commits on Jun 28, 2024

• Split away leading $ for variables.

  01mf02

  committed
  b6b515d

• Make folding operations accept only atoms for now.

  Show description for 8737856
  01mf02

  committed
  8737856

• Refactor.

  01mf02

  committed
  7374b0c

• Make Module usable.

  01mf02

  committed
  4201dd5

• Remove dependency on jaq-std!

  01mf02

  committed
  08947bd

• Simplify final parsing, make compile with MSRV.

  01mf02

  committed
  9cc0187

Commits on Jul 2, 2024

• Exclude unpermitted numbers such as 三.

  01mf02

  committed
  b444f6b

• Use more iter::from_fn.

  01mf02

  committed
  695d012

• Correct more numeric lexing.

  01mf02

  committed
  95e40a7

• Make consumed take usize instead of Chars.

  01mf02

  committed
  f75d26c

• Move conversion functions to own module.

  01mf02

  committed
  17c77fc

• Calculate (more) correct spans!

  01mf02

  committed
  a36cde2

Commits on Jul 3, 2024

• Use lex::span instead of duplicate code.

  01mf02

  committed
  3a94958

• Make span() public.

  01mf02

  committed
  cdd8806

• String representation for lex::Expect.

  01mf02

  committed
  0aefb7d

• Make Expect::Delim return only delimiter, not whole remaining input.

  01mf02

  committed
  637f58c

• Make a few lexer types abstract over S, not over 'a.

  01mf02

  committed
  95c511b

• Report lexer errors!

  01mf02

  committed
  8a4a78b

Commits on Jul 4, 2024

• Proper reporting of reports.

  01mf02

  committed
  edd9070

• Make parse error public.

  01mf02

  committed
  18fc877

• Handle unclosed quotes.

  01mf02

  committed
  4a2a15c

• Store start and end of strings.

  01mf02

  committed
  201f9ee

• Report parse errors!

  01mf02

  committed
  668bc1f

• Flatten.

  01mf02

  committed
  7a68ede

Commits on Jul 15, 2024

• Output compilation errors via new report infrastructure.

  01mf02

  committed
  5142761

• Remove chumsky and jaq_parse dependencies from jaq.

  01mf02

  committed
  2d9b3c0

• Remove unused imports.

  01mf02

  committed
  c87c678

• Warn for unimplemented functionality.

  01mf02

  committed
  c4298a5

• Document.

  01mf02

  committed
  91f6916

• Nicer conversion.

  01mf02

  committed
  fde7a83

• Document Term type.

  01mf02

  committed
  02079a5

• Format.

  01mf02

  committed
  c7b106f

• Nicer handling of leading key in path expression.

  01mf02

  committed
  2007fef

• Merge branch 'main' into faster-lexer

  01mf02

  committed
  c077331

• Remove duplicate check.

  01mf02

  committed
  bfb52f1

• More robust handling of paths starting with a key.

  01mf02

  committed
  d3f9189

• Document.

  01mf02

  committed
  2d16f1a

• Remove unused function.

  01mf02

  committed
  033ef57

Commits on Jul 16, 2024

• Unify logic for key followed by optionality.

  01mf02

  committed
  77aa7d0

• More streamlined lexing/parsing API.

  01mf02

  committed
  70b4ab4

• Document.

  01mf02

  committed
  c872e4e

• Distinguish string lifetime from token lifetime.

  01mf02

  committed
  00fd324

• Correct a few mistakes in the parser.

  01mf02

  committed
  bbf2425

• Remove invalid test.

  01mf02

  committed
  aa165a6

• Remove jaq-parse from jaq-interpret.

  01mf02

  committed
  324ea28

• Remove jaq-parse from jaq-std.

  01mf02

  committed
  4bf1db2

• Bump jaq-syn to 1.6.0.

  01mf02

  committed
  072ec5f

• Remove jaq-parse from jaq-core.

  01mf02

  committed
  a1fb0da

• Update Cargo.lock.

  01mf02

  committed
  cd8a336

• Use jaq-std in jaq again.

  01mf02

  committed
  e97afb3

• Format.

  01mf02

  committed
  bcc3d5a

• Use new parser in jaq-play.

  01mf02

  committed
  f800a82

• Remove old parser from Cargo.toml.

  01mf02

  committed
  ee34b4c

• Update Cargo.lock --- bye bye, chumsky!

  01mf02

  committed
  f483266

• Clippy.

  01mf02

  committed
  6a2c7b2

Commits on Jul 17, 2024

• Do not panic on interpolated strings in module paths.

  01mf02

  committed
  02c05cb

• Identifiers.

  01mf02

  committed
  4ec2ca3

• More permissive key syntax.

  01mf02

  committed
  fc15a22

• If you can write {key}, then you can also write .key.

  01mf02

  committed
  bdbc9a5

• Document.

  01mf02

  committed
  9968bf6

• Allow .key and {key} where key is a keyword, and disallow . key.

  01mf02

  committed
  6f2478f

• Remove def_head.

  01mf02

  committed
  abd09d5

• Clippy.

  01mf02

  committed
  f30c3fe

• Document.

  01mf02

  committed
  1b82d32

• Merge branch 'main' into faster-lexer

  01mf02

  committed
  6729198

• New test.

  01mf02

  committed
  44e5851

• Document.

  01mf02

  committed
  c379734

• Report lex errors for characters c like 💣 where c.len_utf8() != 1.

  01mf02

  committed
  b46c2b1

• Clippy.

  01mf02

  committed
  9cde4e4

• Document.

  01mf02

  committed
  2772166

Commits on Jul 22, 2024

• Document term type.

  01mf02

  committed
  1174607

• Example for parse function.

  01mf02

  committed
  2f970c0

• Report unsupported operator.

  01mf02

  committed
  88d55f0

• Document.

  01mf02

  committed
  39c3429

• Remove KEYWORDS.

  01mf02

  committed
  cd8085c

• Document expectation.

  01mf02

  committed
  33f65ab

• Document.

  01mf02

  committed
  e3df12f

• Do not attempt to support destructuring alternative operator for now.

  01mf02

  committed
  dce1814

• Correctly compute {$k}.

  01mf02

  committed
  27798d4

• Atomicity tests.

  01mf02

  committed
  7a74ea7

• Make test more meaningful.

  01mf02

  committed
  b410289

• Document.

  01mf02

  committed
  cf3fc71