Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OPIK-75: Add batch spans creation endpoint #205

Merged
merged 10 commits into from
Sep 13, 2024

Conversation

thiagohora
Copy link
Contributor

@thiagohora thiagohora commented Sep 10, 2024

Details

Add endpoint to batch create spans.

POST /v1/private/spans/batch

Issues

Resolves #OPIK-75

Testing

Documentation

@thiagohora thiagohora force-pushed the thiagohora/batch_spans_creation branch 2 times, most recently from 9910e26 to 05028fd Compare September 10, 2024 15:59
@thiagohora thiagohora force-pushed the thiagohora/batch_spans_creation branch from 05028fd to 3b68fd1 Compare September 10, 2024 16:08
@thiagohora thiagohora marked this pull request as ready for review September 10, 2024 16:11
@thiagohora thiagohora requested a review from a team as a code owner September 10, 2024 16:11
@thiagohora thiagohora self-assigned this Sep 10, 2024
Copy link
Collaborator

@andrescrz andrescrz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline, but we should probably:

  1. Remove the merge code all-together.
  2. Let's move compression to a separated improvement.

After that, a lot of smaller comments.

@thiagohora thiagohora force-pushed the thiagohora/batch_spans_creation branch from bee108a to e03b560 Compare September 12, 2024 10:13
Copy link
Collaborator

@andrescrz andrescrz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep removing unnecessary stuff from the PR and polish a few more details.

This is generally in the good direction.

apps/opik-backend/config.yml Outdated Show resolved Hide resolved
.toList();
}

private Mono<Project> resolveProject(String projectName) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe a method already exists in this service for this. Let's avoid the duplication if possible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is already reusing the create logic. We check if it's empty

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: looking at the latest code, this extra method isn't doing much , you can clean it up and just directly call
getOrCreateProject(WorkspaceUtils.getProjectName(projectName))

@thiagohora thiagohora force-pushed the thiagohora/batch_spans_creation branch 5 times, most recently from 3fcfbdd to c6b6d46 Compare September 13, 2024 08:51
@thiagohora thiagohora force-pushed the thiagohora/batch_spans_creation branch from c6b6d46 to 9735090 Compare September 13, 2024 08:53
@thiagohora thiagohora changed the title OPIK-75: Add bulk spans creation endpoint OPIK-75: Add batch spans creation endpoint Sep 13, 2024
Copy link
Collaborator

@andrescrz andrescrz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a few more minor comments about stylish things for the future.

Thanks for putting this together!

:name<item.index>,
:type<item.index>,
parseDateTime64BestEffort(:start_time<item.index>, 9),
if(:end_time<item.index> IS NULL, NULL, parseDateTime64BestEffort(:end_time<item.index>, 9)),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +495 to +497
public Mono<Long> batchInsert(@NonNull List<Span> spans) {

Preconditions.checkArgument(!spans.isEmpty(), "Spans list must not be empty");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: alternatively, you can remove the @NonNull annotation and do both checks in one go with CollectionUtils.isNotEmpty.

Comment on lines +255 to +257
public Mono<Long> create(@NonNull SpanBatch batch) {

Preconditions.checkArgument(!batch.spans().isEmpty(), "Batch spans must not be empty");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same.

.toList();
}

private Mono<Project> resolveProject(String projectName) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: looking at the latest code, this extra method isn't doing much , you can clean it up and just directly call
getOrCreateProject(WorkspaceUtils.getProjectName(projectName))

@thiagohora thiagohora merged commit 7b6527c into main Sep 13, 2024
2 checks passed
@thiagohora thiagohora deleted the thiagohora/batch_spans_creation branch September 13, 2024 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants