Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Julia support to flatbuffers #5088

Closed
wants to merge 26 commits into from
Closed
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ set(FlatBuffers_Compiler_SRCS
src/idl_gen_general.cpp
src/idl_gen_go.cpp
src/idl_gen_js_ts.cpp
src/idl_gen_julia.cpp
src/idl_gen_php.cpp
src/idl_gen_python.cpp
src/idl_gen_lobster.cpp
Expand Down
2 changes: 2 additions & 0 deletions docs/source/Compiler.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ For any schema input files, one or more generators can be specified:

- `--rust`, `-r` : Generate Rust code.

- `--julia`: Generate Julia code.

For any data input files:

- `--binary`, `-b` : If data is contained in this file, generate a
Expand Down
2 changes: 2 additions & 0 deletions docs/source/FlatBuffers.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ sections provide a more in-depth usage guide.
own programs.
- How to [use the generated Rust code](@ref flatbuffers_guide_use_rust) in your
own programs.
- How to [use the generated Julia code](@ref flatbuffers_guide_use_julia) in your
own programs.
- [Support matrix](@ref flatbuffers_support) for platforms/languages/features.
- Some [benchmarks](@ref flatbuffers_benchmarks) showing the advantage of
using FlatBuffers.
Expand Down
57 changes: 57 additions & 0 deletions docs/source/JuliaUsage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
Use in Julia {#flatbuffers_guide_use_julia}
==============
## Before you get started
Before diving into the FlatBuffers usage in Julia, it should be noted that the
[Tutorial](@ref flatbuffers_guide_tutorial) page has a complete guide to general
FlatBuffers usage in all of the supported languages (including Julia). This
page is designed to cover the nuances of FlatBuffers usage, specific to
Julia.

You should also have read the [Building](@ref flatbuffers_guide_building)
documentation to build `flatc` and should be familiar with
[Using the schema compiler](@ref flatbuffers_guide_using_schema_compiler) and
[Writing a schema](@ref flatbuffers_guide_writing_schema).

## FlatBuffers Julia package
The `flatc` backend for Julia makes heavy use of the [FlatBuffers.jl package](https://github.com/JuliaData/FlatBuffers.jl).
Documentation for this package may be found [here](http://juliadata.github.io/FlatBuffers.jl/stable).
You can also browse the source code on the [FlatBuffers.jl GitHub page](https://github.com/JuliaData/FlatBuffers.jl).

## Testing the FlatBuffers Julia library
The code to test the Julia library can be found at [FlatBuffers.jl/test/flatc.jl](https://github.com/JuliaData/FlatBuffers.jl/tree/master/test/flatc.jl). To run the tests, run `julia test/flatc.jl` in the top-level directory.
To obtain Julia itself, go to the [Julia homepage](http://julialang.org)
for binaries or source code for your platform.

## Using the FlatBuffers Julia library
*Note: See [Tutorial](@ref flatbuffers_guide_tutorial) for a more in-depth
example of how to use FlatBuffers in Julia.*

There is support for both reading and writing FlatBuffers in Julia.
To use FlatBuffers in your own code, first generate Julia modules from your
schema with the `--julia` option to `flatc`. Then you can include both
FlatBuffers and the generated code to read or write a FlatBuffer.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.julia}
import FlatBuffers

include("MyGame/MyGame.jl")
import .MyGame.Example.Monster

monster = open("monsterdata_test.mon", "r") do f Monster(f) end
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now you can access values like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.julia}
hp = monster.hp
pos = monster.pos
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
More detailed documentation for using the FlatBuffers Julia package
may be found [here](http://juliadata.github.io/FlatBuffers.jl/stable).

## Speed
The FlatBuffers Julia package has not been thoroughly optimized for speed.
For example, instead of referring to memory in-place, values are instead
loaded with `unsafe_load`. It does however support deduplication of vtables, which
saves roughly 30% of time spent writing, when compared to writing objects
with duplicate vtables.

<br>
36 changes: 19 additions & 17 deletions docs/source/Support.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,29 +18,31 @@ In general:

NOTE: this table is a start, it needs to be extended.

Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust
------------------------------ | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ------- | ------- | ----
Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes
JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No
Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes
Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb
Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes
Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes
Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes
Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ?
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ?
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ?
Primary authors (github) | aard* | aard* | ev*/js*| rw | rw | evanw/ev* | kr* | mik* | ch* | dnfield | aard* | rw
Feature | C++ | Java | C# | Go | Python | JS | TS | C | PHP | Dart | Lobster | Rust | Julia |
------------------------------ | ------ | ------ | ------ | ------ | ------ | --------- | --------- | ------ | --- | ------- | ------- | ---- | ---- |
Codegen for all basic features | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | WiP | Yes | Yes | Yes | Yes |
JSON parsing | Yes | No | No | No | No | No | No | Yes | No | No | Yes | No | No |
Simple mutation | Yes | Yes | Yes | Yes | No | No | No | No | No | No | No | No | No |
Reflection | Yes | No | No | No | No | No | No | Basic | No | No | No | No | No |
Buffer verifier | Yes | No | No | No | No | No | No | Yes | No | No | No | No | No |
Testing: basic | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ? | Yes | Yes | Yes | Yes |
Testing: fuzz | Yes | No | No | Yes | Yes | No | No | No | ? | No | No | Yes | Yes |
Performance: | Superb | Great | Great | Great | Ok | ? | ? | Superb | ? | ? | Great | Superb | ? |
Platform: Windows | VS2010 | Yes | Yes | ? | ? | ? | Yes | VS2010 | ? | Yes | Yes | Yes | Yes |
Platform: Linux | GCC282 | Yes | ? | Yes | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes |
Platform: OS X | Xcode4 | ? | ? | ? | Yes | ? | Yes | Yes | ? | Yes | Yes | Yes | Yes |
Platform: Android | NDK10d | Yes | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | ? |
Platform: iOS | ? | ? | ? | ? | ? | ? | ? | ? | ? | Flutter | Yes | ? | ? |
Engine: Unity | ? | ? | Yes | ? | ? | ? | ? | ? | ? | ? | No | ? | ? |
Primary authors (github) | aard* | aard* | ev*/js*| rw | rw | evanw/ev* | kr* | mik* | ch* | dnfield | aard* | rw | jq*/rk* |

* aard = aardappel (previously: gwvo)
* ev = evolutional
* js = jonsimantov
* mik = mikkelfj
* ch = chobie
* kr = krojew
* jq = quinnj
* rk = rkat
rjkat marked this conversation as resolved.
Show resolved Hide resolved

<br>
Loading