Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix gremlin job result size gt cassandra limit and lt hugegraph limit #1334

Merged
merged 3 commits into from
Jan 14, 2021
Merged

fix gremlin job result size gt cassandra limit and lt hugegraph limit #1334

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
904202b
fix gremlin job result size gt cassandra limit and lt hugegraph limit
zhoney Jan 14, 2021
7a0c7bf
improve
zhoney Jan 14, 2021
f81a3cf
improve
zhoney Jan 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions hugegraph-core/src/main/java/com/baidu/hugegraph/task/TaskCallable.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,9 @@ public abstract class TaskCallable<V> implements Callable<V> {

private static final Logger LOG = Log.logger(HugeTask.class);

private static final String ERROR_MAX_LEN = "Failed to commit changes: " +
"The max length of bytes is";
private static final String ERROR_COMMIT = "Failed to commit changes: ";
private static final String ERROR_MAX_LEN = "The max length of bytes is";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

define Set

private static final String ERROR_TOO_LARGE_BATCH = "Batch too large";

private HugeTask<V> task = null;
private HugeGraph graph = null;
Expand Down Expand Up @@ -98,7 +99,15 @@ protected void save() {
*/
LOG.error("Failed to save task with error \"{}\": {}",
e, task.asMap(false));
if (e.getMessage().contains(ERROR_MAX_LEN)) {
String message = e.getMessage();
/*
* ERROR_TOO_LARGE_BATCH occurs when using cassandra store
* and result size is in
* [batch_size_fail_threshold_in_kb, TASK_RESULT_SIZE_LIMIT)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

improve comments

*/
if (message.contains(ERROR_COMMIT) &&
(message.contains(ERROR_MAX_LEN) ||
message.contains(ERROR_TOO_LARGE_BATCH))) {
task.failSave(e);
this.graph().taskScheduler().save(task);
return;
Expand Down