-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support very large tables #2287
Conversation
…re into lab-grow-viewport
engine/table/src/main/java/io/deephaven/engine/table/impl/remote/ConstructSnapshot.java
Outdated
Show resolved
Hide resolved
engine/table/src/main/java/io/deephaven/engine/table/impl/remote/ConstructSnapshot.java
Outdated
Show resolved
Hide resolved
extensions/barrage/src/main/java/io/deephaven/extensions/barrage/table/BarrageTable.java
Outdated
Show resolved
Hide resolved
server/src/main/java/io/deephaven/server/barrage/BarrageStreamGenerator.java
Show resolved
Hide resolved
server/src/main/java/io/deephaven/server/barrage/BarrageStreamGenerator.java
Show resolved
Hide resolved
server/src/main/java/io/deephaven/server/barrage/BarrageStreamGenerator.java
Outdated
Show resolved
Hide resolved
server/src/main/java/io/deephaven/server/barrage/BarrageStreamGenerator.java
Show resolved
Hide resolved
server/src/main/java/io/deephaven/server/barrage/BarrageStreamGenerator.java
Outdated
Show resolved
Hide resolved
.../src/main/java/io/deephaven/extensions/barrage/chunk/VarBinaryChunkInputStreamGenerator.java
Outdated
Show resolved
Hide resolved
.../src/main/java/io/deephaven/extensions/barrage/chunk/VarBinaryChunkInputStreamGenerator.java
Outdated
Show resolved
Hide resolved
.../src/main/java/io/deephaven/extensions/barrage/chunk/VarBinaryChunkInputStreamGenerator.java
Outdated
Show resolved
Hide resolved
.../src/main/java/io/deephaven/extensions/barrage/chunk/VarBinaryChunkInputStreamGenerator.java
Outdated
Show resolved
Hide resolved
.../src/main/java/io/deephaven/extensions/barrage/chunk/VarBinaryChunkInputStreamGenerator.java
Outdated
Show resolved
Hide resolved
.../src/main/java/io/deephaven/extensions/barrage/chunk/VarBinaryChunkInputStreamGenerator.java
Outdated
Show resolved
Hide resolved
.../src/main/java/io/deephaven/extensions/barrage/chunk/VarBinaryChunkInputStreamGenerator.java
Outdated
Show resolved
Hide resolved
.../src/main/java/io/deephaven/extensions/barrage/chunk/VarBinaryChunkInputStreamGenerator.java
Outdated
Show resolved
Hide resolved
server/src/main/java/io/deephaven/server/barrage/BarrageStreamGenerator.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'll still need a large-table implementation for VarListChunkInputStreamGenerator (similar to the ByteStorage approach). I'm happy if that is a separate review.
@@ -45,6 +45,8 @@ | |||
|
|||
import io.deephaven.chunk.attributes.Values; | |||
|
|||
import static io.deephaven.engine.table.impl.sources.InMemoryColumnSource.TWO_DIMENSIONAL_COLUMN_SOURCE_THRESHOLD; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the sake of clarity, I think a new constant makes sense.
Adds support for snapshotting very large tables (i.e. contains more than 2^31 rows, an array or string column with accumulated length more than 2^31 values, or a column requiring marshalling with accumulated byte length more than 2^31)
Closes #2175