Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEAT][JAVA] Implement READERS and WRITERS for Java #233

Merged
merged 14 commits into from
Sep 25, 2023
Merged

Conversation

Thespica
Copy link
Contributor

Proposed changes

As titile.

Next step: Add related documents and library referance.

Types of changes

What types of changes does your code introduce to GraphAr?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

Related issue: #72

@Thespica Thespica changed the title [JAVA] Implementing READERS and WRITERS for Java [FEAT][JAVA] Implement READERS and WRITERS for Java Sep 17, 2023
@acezen acezen self-requested a review September 20, 2023 05:46
@@ -43,7 +43,7 @@ jobs:

- name: Code Format Check
run: |
export JAVA_HOME=${JAVA_HOME_11_X64}
export JAVA_HOME=${JAVA_HOME_8_X64}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why change to JAVA 8?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't used features that higher than Java 8, and Java 8 have better compatibility with others graph systems which rely on Java 8. So maybe we should assume that environment is Java 8, and Java higher is compatible with lower version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the spark rely on java 11, so for CI unification and avoid confusion, I think 11 is better.

find_package(gar REQUIRED)

add_library(${LIBNAME} SHARED ${SOURCES})
target_link_libraries(${LIBNAME} ${CMAKE_JNI_LINKER_FLAGS} gar)
target_link_libraries(${LIBNAME} ${CMAKE_JNI_LINKER_FLAGS} Arrow::arrow_static)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe link to shared library is better? otherwise it would take a long time to compile

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On one head, linking arrow as dynamic will leads error about libprotobuf: https://github.com/Thespica/GraphAr/actions/runs/6206365474/job/16850543098#step:6:9100

On the other head, I consider linking arrow as static just copy arrow lib once, rather than recompile, which only cost little time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, this can remain and open as a issue to fix in the future.

@@ -16,6 +16,7 @@
<maven.compiler.target>8</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<fastffi.revision>0.1.2</fastffi.revision>
<arrow.version>13.0.0</arrow.version>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this have to be set? If user install other version of arrow, the program still work as expect?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maven will automatically download needed jar with specific version and manage them, so it will works.

But maybe relying arrow 12 will be better?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if work as expected, 13.0.0 is ok

* @param propertyGroup The property group that describes the property group.
* @param prefix The absolute prefix.
*/
// VertexPropertyArrowChunkReader create(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some comment here if the code has to remain?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

Copy link
Contributor

@acezen acezen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job! There're still some tiny problem to fix(see comments), and when it done, the PR if fine to me to merge to main

@acezen acezen merged commit fbe03ed into apache:main Sep 25, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants