Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversion to python3 and some other fixes #1

Open
wants to merge 28 commits into
base: master
Choose a base branch
from

Conversation

KOLANICH
Copy link

@KOLANICH KOLANICH commented Nov 20, 2017

Hi.
I have:

  • translated it into python 3
  • added some needed files
  • rewritten tests to use test framework.
  • remade arguments parsing using plumbum
  • stored metadata in setup.cfg

ToDos:

  • some tests fail. I guess you have made some improvements which are not tested properly
  • self-hosting. There is a problem with Coco.atg because of imports with dot.
  • rewrite testing to use invoke instead of subprocess
  • on Travis the thing doesn't work.

1. Tests fix for non-Windows OSes
2. removed obsolete stuff from setup.py
3. metadata retrieval fixes.
4. Replaced os.path with pathlib
5. Dropped python2
Moved constants into a separate enum.
Reformatted with black and autopep8. Now it uses LF line endings and tabs.
Python 3 support has been implemented, Python 2 support has been dropped.
Migrated from dumb strings to `ast`, which has greatly improved the structure of the code and allowed further improvements. The cost is that the comments and whitespaces are not preserved, since there are no nodes for them. Transforming asts into strings is done using `astor` library. Optionally (if installed) the result is postprocessed by `antiflash`.
Added types, reformatted source. Now we use tabs (a tab is now considered to be completion to 4 spaces). Enable `editorconfig` in your favorite editor if you want to see the sources right.
Implemented python code detection. Instead of custom and incomplete parsing as was done earlier we now use python parser. The first syntax error in python code is considered to be an end of python code and continuation of CoCo/R grammar.
Implemented optional typing for parameters. Typing uses Java/C++/C# syntax (type name), not python one (name: type) for compatibility
Moved the components used by every parser into `CoCoRuntime` module. It provides abstract base classes with some props that must be populated in generated code
Started eliminating global state. Stuff earlier saved into classes now is saved into instances.
Dropped frame files support. Use object oriented programming instead.
`Copyright.frame` is not used anymore (in fact it has been never used in CoCoPy, but CoCo/R manual mentions it). Instead put copyright header into `__copyright__` variable, and it will be inserted in both `Scanner` and `Parser`
Dropped generation of drivers. They were not really generated, just copied.
(Non)terminals enum values are now stored in a single shared `IntEnum` rather than in parser and lexer classes themselves.
Added a mode to use lookups from the enum instead of raw numbers in generated code. Should improve readability and simplify debugging and diffing the generated code
Now we use `setup.cfg` and `pyproject.toml` for fetching versions from git. Installation from the sources other than git is deprecated.
Refactored keywords lookup. Now they are not matched using elif ladders, instead they are stored in a map
Optimized error messages storage. Since their numbers are sequential, now they are stored in a `list` rather than a `dict`.
Dropped numeric values (that initially were not in a enum, just untidy mess) mixing node types, symbol types and state types. Now we have a dedicated class for each.
Optionally optimized keywords lookup using prefix trees using `datrie` library. Automatically enabled, if `datrie` is installed.
Moved the metadata into `pyproject.toml`.
Removed `setup.py`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants