Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Topic/genc v2: ready for general review #290

Merged
merged 77 commits into from
Feb 14, 2017
Merged

Topic/genc v2: ready for general review #290

merged 77 commits into from
Feb 14, 2017

Conversation

mantognini
Copy link
Contributor

This PR brings a lot of improvements and new features to the GenC module. It also have minor impacts on the general codebase of Leon.

What is missing for merging?

  • The documentation should be updated to highlight the supported feature set and to reflect the changes made to the I/O API.

What is in here? [Highlight]

  • A new, completely rewritten GenC phase that uses an internal pipeline of transformation over an Intermediary Representation to ease the conversion from Scala AST to C AST.
  • A better test suite for the generated C code, using many different compilers (if available), including CompCert.
  • A new TimedLeonPhase trait that provided boilerplates for timers and the --debug=timers option.
  • A new RemoveVCPhase phase that removed invariants from the AST.
  • GenC now supports:
    • inheritance (as long as the type is non-recursive),
    • generics types and functions, even if nested,
    • pattern matching (with the exception of the extractor pattern),
    • true enumeration (hierarchy of case classes without any fields is converted to a C enum),
    • Byte arithmetic.
  • Arithmetic operators and integer type promotion was re-evaluated and improved to more closely match the Scala/Java and C99 standards.
  • Unsupported fragments of Scala are now more properly reported to the user as such.
  • The I/O API was enriched to support detecting parsing error or End-Of-File as well as reading bytes of data.
  • Additionally, GenC now only converts the dependencies of the main/_main functions which allows VC to contain fragments of Scala not supported by GenC.
  • A new case study is available under testcases/genc/: Lempel–Ziv–Welch (LZW) universal lossless data compression algorithm.

What's next (independent of this PR)?

Feedback welcomed.

 - augment list of known compilers (with compcert)
 - adapt this list for the current OS
 - use advanced parameters (e.g. sanitizers)
 - run all found compilers
 - add ability to disable a given compiler on a case-by-case basis
 - ignore files generated by tests
 - improve error messages
 - report warnings and other errors from the GenC phase (and others)
Removed GenC in view of GenC v2
This new implementation of GenC uses a pipelined approach: the translation of Scala code into C99 equivalent code is split into several phases that live in the `leon.genc.phases` package. The overall process is bases on several versions of an Intermediate Representation (IR) that is defined under `leon.genc.ir`. Additionally, the C AST was partially redesigned.

This new implementation is supposed to be better than the previous version in terms of bugs but there remain a few areas for improvement:
  * The `ClassLifter` is slightly incomplete: class fields should be processed as well.
  * When converting nested generic functions, the Scala2IRPhase might not do a proper job.

Both of these points are explained in the source code; see the corresponding TODOs.
This avoids propagating failures from one test to the next.
Hope is that shutting down fsc will prevent two concurrent build to
mess with one another.
ArrayUpdated, which represents Array.updated(index, newValue) and therefore builds a new array, should not be confused by ArrayUpdate, which updates a given array element in place.
Since the introduction of the new dependency finder, many functions/types were no longer examined by GenC.
On most systems, the return value from a program is truncated to its
least significant byte.
Note: this could get supported by generating the proper function definitions and replacing the operator by a function call.
This can also prevent some clashes with C std functions.
Xlang now ensures no side effect so we remove this test from GenC.
Note: when using the private ctor of FIS/FOS, Leon will emit something like this:

[info] [ Error  ] Test.scala:93:16: Class leon.io.FileInputStream not defined?
[info]                val fis1 = FIS(Some("input.txt"))
[info]                           ^^^

and therefore identifying the call to the constructor as faulty.
Let the user know this is not 100% MISRA compliant. With stack size analysis this issue could be solved.
Except for the main function which should not be static
This representation is less efficient in terms of speed of the produced code, but requires far less small arrays and therefore significantly improve the performance of GenC and C compilers to compile the code.

Also, less properties are proven on this implementation.
 - Aliasing3 is disabled because it make verification crash (see #289)
 - Aliasing4 is disabled because it make verification produce incorrect counter-example (see #287)
 - Inheritance9 is disabled because it make xlang crash (see #288)
 - UsingConcreteCLasses is enabled because verification is easy
Because they are slow to generate, slow to compile and slow to execute,
they cannot be part of the regression suite.
Use comments to select the kernel you want to use, then recompile.
@regb regb merged commit b8f542f into master Feb 14, 2017
@mantognini mantognini deleted the topic/genc_v2 branch February 14, 2017 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants