Skip to content

Commit

Permalink
Merge pull request #18 from Tolc-Software/oss
Browse files Browse the repository at this point in the history
Oss
  • Loading branch information
srydell authored Jan 3, 2022
2 parents c3faaec + 4ccf1e8 commit 446f4f6
Show file tree
Hide file tree
Showing 10 changed files with 93 additions and 134 deletions.
18 changes: 9 additions & 9 deletions .github/workflows/macos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ name: MacO
on:
push:
branches:
- master
- main
pull_request:
branches:
- master
- main

env:
BUILD_TYPE: Release
Expand Down Expand Up @@ -68,32 +68,32 @@ jobs:
ctest --cores 4 --output-on-failure
- name: Build the release tar.xz package
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
run: cpack -G TXZ --config build/CPackConfig.cmake

- name: Setup go
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
uses: actions/setup-go@v2
with:
go-version: '1.14'

- name: Setup go cache
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
uses: actions/cache@v2
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Push master release
if: github.ref == 'refs/heads/master'
- name: Push main release
if: github.ref == 'refs/heads/main'
env:
GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}
run: |
go get -v github.com/github-release/github-release
echo "Checking if a new release is needed"
github-release release --user srydell --repo Parser --tag master-release --name "Head release" --description "This release gets updated with each commit to master" || true
github-release release --user Tolc-Software --repo Parser --tag main-release --name "Head release" --description "This release gets updated with each commit to main" || true
echo "Pushing the release"
github-release upload --user srydell --repo Parser --tag master-release --name "Parser-$(uname)-master.tar.xz" --file Parser-*.tar.xz --replace
github-release upload --user Tolc-Software --repo Parser --tag main-release --name "Parser-$(uname)-main.tar.xz" --file Parser-*.tar.xz --replace
18 changes: 9 additions & 9 deletions .github/workflows/ubuntu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ name: Ubuntu

on:
push:
branches: [ master ]
branches: [ main ]
pull_request:
branches: [ master ]
branches: [ main ]

env:
BUILD_TYPE: Release
Expand Down Expand Up @@ -66,32 +66,32 @@ jobs:
ctest --cores 4 --output-on-failure
- name: Build the release tar.xz package
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
run: cpack -G TXZ --config build/CPackConfig.cmake

- name: Setup go
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
uses: actions/setup-go@v2
with:
go-version: '1.14'

- name: Setup go cache
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
uses: actions/cache@v2
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Push master release
if: github.ref == 'refs/heads/master'
- name: Push main release
if: github.ref == 'refs/heads/main'
env:
GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}
run: |
go get -v github.com/github-release/github-release
echo "Checking if a new release is needed"
github-release release --user srydell --repo Parser --tag master-release --name "Head release" --description "This release gets updated with each commit to master" || true
github-release release --user Tolc-Software --repo Parser --tag main-release --name "Head release" --description "This release gets updated with each commit to main" || true
echo "Pushing the release"
github-release upload --user srydell --repo Parser --tag master-release --name "Parser-$(uname)-master.tar.xz" --file Parser-*.tar.xz --replace
github-release upload --user Tolc-Software --repo Parser --tag main-release --name "Parser-$(uname)-main.tar.xz" --file Parser-*.tar.xz --replace
18 changes: 9 additions & 9 deletions .github/workflows/windows-debug-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Windows-debug-deploy
on:
push:
branches:
- master
- main


jobs:
Expand All @@ -18,7 +18,7 @@ jobs:
- name: Pull down to C disk
shell: cmd
run: |
git clone https://srydell:${{ secrets.ACCESS_TOKEN }}@github.com/srydell/Parser.git C:\Repo
git clone https://Tolc-Software:${{ secrets.ACCESS_TOKEN }}@github.com/Tolc-Software/Parser.git C:\Repo
- name: Install SSH key
uses: shimataro/ssh-key-action@v2
Expand Down Expand Up @@ -68,36 +68,36 @@ jobs:
cmake --build build -j4
- name: Build the release tar.xz package
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
working-directory: C:\Repo
shell: cmd
run: |
call "%VCVARS%"
cpack -G TXZ --config build/CPackConfig.cmake
- name: Setup go
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
uses: actions/setup-go@v2
with:
go-version: '1.14'

- name: Setup go cache
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
uses: actions/cache@v2
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Push master release
if: github.ref == 'refs/heads/master'
- name: Push main release
if: github.ref == 'refs/heads/main'
working-directory: C:\Repo
env:
GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}
run: |
go get -v github.com/github-release/github-release
echo "Checking if a new release is needed"
github-release release --user srydell --repo Parser --tag master-release --name "Head release" --description "This release gets updated with each commit to master" || true
github-release release --user Tolc-Software --repo Parser --tag main-release --name "Head release" --description "This release gets updated with each commit to main" || true
echo "Pushing the release"
github-release upload --user srydell --repo Parser --tag master-release --name "Parser-Windows-master-Debug.tar.xz" --file Parser.tar.xz --replace
github-release upload --user Tolc-Software --repo Parser --tag main-release --name "Parser-Windows-main-Debug.tar.xz" --file Parser.tar.xz --replace
18 changes: 9 additions & 9 deletions .github/workflows/windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ name: Windows
on:
push:
branches:
- master
- main
pull_request:
branches:
- master
- main

jobs:
build:
Expand Down Expand Up @@ -70,35 +70,35 @@ jobs:
ctest --cores 4 --output-on-failure
- name: Build the release tar.xz package
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
shell: cmd
run: |
call "%VCVARS%"
cpack -G TXZ --config build/CPackConfig.cmake
- name: Setup go
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
uses: actions/setup-go@v2
with:
go-version: '1.14'

- name: Setup go cache
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/main'
uses: actions/cache@v2
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Push master release
if: github.ref == 'refs/heads/master'
- name: Push main release
if: github.ref == 'refs/heads/main'
env:
GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}
run: |
go get -v github.com/github-release/github-release
echo "Checking if a new release is needed"
github-release release --user srydell --repo Parser --tag master-release --name "Head release" --description "This release gets updated with each commit to master" || true
github-release release --user Tolc-Software --repo Parser --tag main-release --name "Head release" --description "This release gets updated with each commit to main" || true
echo "Pushing the release"
github-release upload --user srydell --repo Parser --tag master-release --name "Parser-Windows-master-Release.tar.xz" --file Parser.tar.xz --replace
github-release upload --user Tolc-Software --repo Parser --tag main-release --name "Parser-Windows-main-Release.tar.xz" --file Parser.tar.xz --replace
4 changes: 2 additions & 2 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ include(cmake/Sanitizers.cmake)
include(FetchContent)
FetchContent_Declare(
IRSpecification
GIT_REPOSITORY git@github.com:srydell/IntermediateRepresentation.git
GIT_TAG master)
GIT_REPOSITORY git@github.com:Tolc-Software/IntermediateRepresentation.git
GIT_TAG main)
FetchContent_MakeAvailable(IRSpecification)

FetchContent_Declare(
Expand Down
99 changes: 4 additions & 95 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
![Ubuntu](https://github.com/srydell/Parser/workflows/Ubuntu/badge.svg) ![MacOS](https://github.com/srydell/Parser/workflows/MacOS/badge.svg) ![Windows](https://github.com/srydell/Parser/actions/workflows/windows.yml/badge.svg) ![Windows-debug-deploy](https://github.com/srydell/Parser/actions/workflows/windows-debug-deploy.yml/badge.svg)
![Ubuntu](https://github.com/Tolc-Software/Parser/workflows/Ubuntu/badge.svg) ![MacOS](https://github.com/Tolc-Software/Parser/workflows/MacOS/badge.svg) ![Windows](https://github.com/Tolc-Software/Parser/actions/workflows/windows.yml/badge.svg) ![Windows-debug-deploy](https://github.com/Tolc-Software/Parser/actions/workflows/windows-debug-deploy.yml/badge.svg)

# Parser #

Expand All @@ -12,8 +12,8 @@ Use `FetchContent` in your `CMakeLists.txt` as follows:
include(FetchContent)
FetchContent_Declare(
Parser
GIT_REPOSITORY git@github.com:srydell/Parser.git
GIT_TAG master)
GIT_REPOSITORY git@github.com:Tolc-Software/Parser.git
GIT_TAG main)
# Downloads and makes the parser library available
FetchContent_MakeAvailable(Parser)
Expand All @@ -27,7 +27,7 @@ And you should get all the necessary components of the library.

## Usage ##

`Parser` only exposes one header file, `Parser/Parse.h`, which can be used as:
`Parser` exposes the header file [`Parser/Parse.h`](./include/Parser/Parse.hpp), which can be used as:

```cpp
#include <Parser/Parse.hpp>
Expand All @@ -47,94 +47,3 @@ int f() {
}
}
```
## Architecture ##
![Parser Architecture](docs/internal/Parser.png "Parser Architecture")
The architecture roughly follows the figure above. The `code` is supplied by the user, and passes through a `Frontend` and a `Consumer`. Both of these are entry points from where various clang tools can be built. Since we are mainly interested in the actual `AST` ([Abstract Syntax Tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree)) of the code, we must use the third and last entry point, the `Visitor`. The `Visitor` is a class with a set of functions, all beginning with `Visit` followed by what that function should be called on. E.g. `VisitNamespaceDecl` gets called whenever the `Visitor` encounters a `C++` `namespace` within the `AST`. The function will then take the `AST` node as input.
Whenever an appropriate `AST` node is found, it is passed on to a `Builder`, which converts it to `IRProxy`. `IRProxy` (or `Intermediate Represantation Proxy`) is continuously built by the `Visitor` functions with as much information as possible from each node. When all the `code` has been examined, the `Visitor` will run its destructor, which will be used to take all the `IRProxy` and put it through another set of builders, which will produce the final `IR`.
### Good to know ###
* The `AST` nodes (or clang `decl`s) do not live until the `Visitor` is destructed, so they cannot be a part of the `IRProxy`.
* The clang `decl`s use an inheritance scheme, where you can choose to visit more "specialized" `decl`, or even the base (simply `decl`). E.g. `FunctionDecl` covers all functions, but `CXXMethodDecl` is a class function.
### Conan library does not find headers ###
Example:
```shell
In file included from /home/simon/.conan/data/fmt/6.2.0/_/_/build/9aea1b1a46cc502b3591eafb492153938cec535f/source_subfolder/src/os.cc:13:
/home/simon/.conan/data/fmt/6.2.0/_/_/build/9aea1b1a46cc502b3591eafb492153938cec535f/source_subfolder/include/fmt/os.h:16:10: fatal error: 'cerrno' file not found
#include <cerrno>
^~~~~~~~
1 error generated.
```
This needs some debugging. To see what header directories are being searched, we need to know the compilation step; go to the build directory, in this case:
```shell
$ cd /home/simon/.conan/data/fmt/6.2.0/_/_/build/9aea1b1a46cc502b3591eafb492153938cec535f/build_subfolder/source_subfolder
```
In this case the library is built with `make`, so;
```shell
$ VERBOSE=1 make
```
The output is large, but find the offending command. In this case:
```shell
$ cd /home/simon/.conan/data/fmt/6.2.0/_/_/build/9aea1b1a46cc502b3591eafb492153938cec535f/build_subfolder/source_subfolder && /usr/bin/clang++ -I/home/simon/.conan/data/fmt/6.2.0/_/_/build/9aea1b1a46cc502b3591eafb492153938cec535f/source_subfolder/include -m64 -stdlib=libc++ -g -fPIC -std=gnu++11 -o CMakeFiles/fmt.dir/src/os.cc.o -c /home/simon/.conan/data/fmt/6.2.0/_/_/build/9aea1b1a46cc502b3591eafb492153938cec535f/source_subfolder/src/os.cc
```
Add the verbose flag `-v` at the end to get what directories are searched;
```shell
$ <build_command> -v
...
#include "..." search starts here:
#include <...> search starts here:
/home/simon/.conan/data/fmt/6.2.0/_/_/build/9aea1b1a46cc502b3591eafb492153938cec535f/source_subfolder/include
/usr/local/include
/usr/lib/clang/10.0.1/include
/usr/include
End of search list.
<previously shown error>
```
Search these directories for the missing header, hopefully you will not find it. In this case the offender is actually that we are using `-stdlib=libc++`, and it is not installed on my machine.
### Conan package is built with the wrong compiler ###
You might get the error:
```shell
CMake Error at conanbuildinfo.cmake:573 (message):
Incorrect 'clang', is not the one detected by CMake: 'GNU'
Call Stack (most recent call first):
conanbuildinfo.cmake:136 (conan_check_compiler)
CMakeLists.txt:5 (conan_basic_setup)
```
while building some dependency managed by conan. The easiest way of resolving this is setting the `CC` and `CXX` environment variables to your C and C++ compiler respectively. For example;
```shell
$ export CC=$(which clang)
$ export CXX=$(which clang++)
$ <build step>
```
### Standard library ABI with Conan ###
By default, Conan sometimes sets the default ABI to `compiler.libcxx=libstdc++` instead of `compiler.libcxx=libstdc++11`. This is to preserve backwards compatibility with older libraries, as C++11 changed the ABIof the language (for example changes in the `std::string` implementation). This is not something that we want and it should be turned off. Typically done via the conan CMake module with:
```cmake
find_conan_packages(... compiler.libcxx=libstdc++11)
```
2 changes: 1 addition & 1 deletion docs/internal/addingContainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This document describes how to add support for parsing a new type of standard library container (or anything that can contain types).

1. In the [IntermediateRepresentation repo](https://github.com/srydell/IntermediateRepresentation), start by adding an `enum` value to `IR::ContainerType` corresponding to your new container.
1. In the [IntermediateRepresentation repo](https://github.com/Tolc-Software/IntermediateRepresentation), start by adding an `enum` value to `IR::ContainerType` corresponding to your new container.
2. Add a test in the root of the `tests` directory. The naming convention is the plural version of the container name. Use another container test as a starting point. For example `lists.cpp`.
3. Change the `code` block so that the container is used.
4. Add the test to the `CMakeLists.txt` file in the `tests` directory.
Expand Down
14 changes: 14 additions & 0 deletions docs/internal/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Architecture #

![Parser Architecture](Parser.png "Parser Architecture")

The architecture roughly follows the figure above. The `code` is supplied by the user, and passes through a `Frontend` and a `Consumer`. Both of these are entry points from where various clang tools can be built. Since we are mainly interested in the actual `AST` ([Abstract Syntax Tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree)) of the code, we must use the third and last entry point, the `Visitor`. The `Visitor` is a class with a set of functions, all beginning with `Visit` followed by what that function should be called on. E.g. `VisitNamespaceDecl` gets called whenever the `Visitor` encounters a `C++` `namespace` within the `AST`. The function will then take the `AST` node as input.

Whenever an appropriate `AST` node is found, it is passed on to a `Builder`, which converts it to `IRProxy`. `IRProxy` (or `Intermediate Represantation Proxy`) is continuously built by the `Visitor` functions with as much information as possible from each node. When all the `code` has been examined, the `Visitor` will run its destructor, which will be used to take all the `IRProxy` and put it through another set of builders, which will produce the final `IR`.

## Good to know ##

* The `AST` nodes (or clang `decl`s) gets destroyed before the destructor of the `Visitor` is called, therefore they cannot be a part of the `IRProxy`.

* The clang `decl`s use an inheritance scheme, where you can choose to visit more "specialized" `decl`, or even the base (simply `decl`). E.g. `FunctionDecl` covers all functions, but `CXXMethodDecl` is a class function.

5 changes: 5 additions & 0 deletions tests/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -96,4 +96,9 @@ foreach(
LIBRARIES
fmt::fmt
TestUtil)

set(ENABLE_SANITIZER_ADDRESS TRUE)
set(ENABLE_SANITIZER_MEMORY TRUE)
enable_sanitizers(${testName})
endforeach()

Loading

0 comments on commit 446f4f6

Please sign in to comment.