BarrageMessageProducer: Do Not Always Record Stream Table Rows #3857
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nbauernfeind
added
barrage
core
devinrsmith
reviewed
May 22, 2023
There was a problem hiding this comment.
I'm surprised that the UI is still slow; aren't we sending much smaller updates to the web UI?
Potentially not related, but I'm also seeing some failures when this table is reversed (I'm assuming the web UI applies a table.reverse(); I'm assuming "reverse viewport" is something else?)
2023-05-22T18:23:30.895Z | EFAULT.refreshThread | ERROR | rumentedTableListenerBase | Uncaught exception for entry= reverse(), added.size()=50025900, modified.size()=0, removed.size()=50009200, shifted.size()=0, modifiedColumnSe
t={EMPTY}:
io.deephaven.base.verify.AssertionFailure: Assertion failed: asserted update.shifted.empty().
at io.deephaven.base.verify.Assert.fail(Assert.java:86)
at io.deephaven.base.verify.Assert.assertion(Assert.java:116)
at io.deephaven.engine.table.impl.BaseTable.notifyListeners(BaseTable.java:637)
at io.deephaven.engine.table.impl.ReverseOperation.onUpdate(ReverseOperation.java:225)
at io.deephaven.engine.table.impl.ReverseOperation$2.onUpdate(ReverseOperation.java:104)
at io.deephaven.engine.table.impl.InstrumentedTableUpdateListener$Notification.lambda$run$0(InstrumentedTableUpdateListener.java:37)
at io.deephaven.engine.table.impl.InstrumentedTableListenerBase$NotificationBase.doRunInternal(InstrumentedTableListenerBase.java:283)
at io.deephaven.engine.table.impl.InstrumentedTableListenerBase$NotificationBase.doRun(InstrumentedTableListenerBase.java:261)
at io.deephaven.engine.table.impl.InstrumentedTableUpdateListener$Notification.run(InstrumentedTableUpdateListener.java:37)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.runNotification(UpdateGraphProcessor.java:1285)
at io.deephaven.engine.updategraph.UpdateGraphProcessor$QueueNotificationProcessor.doWork(UpdateGraphProcessor.java:1461)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.flushNormalNotificationsAndCompleteCycle(UpdateGraphProcessor.java:1164)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.flushNotificationsAndCompleteCycle(UpdateGraphProcessor.java:1109)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.lambda$doRefresh$11(UpdateGraphProcessor.java:1744)
at io.deephaven.util.locks.FunctionalLock.doLocked(FunctionalLock.java:33)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.doRefresh(UpdateGraphProcessor.java:1732)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.refreshAllTables(UpdateGraphProcessor.java:1719)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.refreshTablesAndFlushNotifications(UpdateGraphProcessor.java:1573)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.lambda$new$3(UpdateGraphProcessor.java:270)
at java.lang.Thread.run(Thread.java:829)
2023-05-22T18:23:30.897Z | EFAULT.refreshThread | ERROR | .AsyncClientErrorNotifier | Error in table update: io.deephaven.base.verify.AssertionFailure: Assertion failed: asserted update.shifted.empty().
at io.deephaven.base.verify.Assert.fail(Assert.java:86)
at io.deephaven.base.verify.Assert.assertion(Assert.java:116)
at io.deephaven.engine.table.impl.BaseTable.notifyListeners(BaseTable.java:637)
at io.deephaven.engine.table.impl.ReverseOperation.onUpdate(ReverseOperation.java:225)
at io.deephaven.engine.table.impl.ReverseOperation$2.onUpdate(ReverseOperation.java:104)
at io.deephaven.engine.table.impl.InstrumentedTableUpdateListener$Notification.lambda$run$0(InstrumentedTableUpdateListener.java:37)
at io.deephaven.engine.table.impl.InstrumentedTableListenerBase$NotificationBase.doRunInternal(InstrumentedTableListenerBase.java:283)
at io.deephaven.engine.table.impl.InstrumentedTableListenerBase$NotificationBase.doRun(InstrumentedTableListenerBase.java:261)
at io.deephaven.engine.table.impl.InstrumentedTableUpdateListener$Notification.run(InstrumentedTableUpdateListener.java:37)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.runNotification(UpdateGraphProcessor.java:1285)
at io.deephaven.engine.updategraph.UpdateGraphProcessor$QueueNotificationProcessor.doWork(UpdateGraphProcessor.java:1461)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.flushNormalNotificationsAndCompleteCycle(UpdateGraphProcessor.java:1164)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.flushNotificationsAndCompleteCycle(UpdateGraphProcessor.java:1109)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.lambda$doRefresh$11(UpdateGraphProcessor.java:1744)
at io.deephaven.util.locks.FunctionalLock.doLocked(FunctionalLock.java:33)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.doRefresh(UpdateGraphProcessor.java:1732)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.refreshAllTables(UpdateGraphProcessor.java:1719)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.refreshTablesAndFlushNotifications(UpdateGraphProcessor.java:1573)
at io.deephaven.engine.updategraph.UpdateGraphProcessor.lambda$new$3(UpdateGraphProcessor.java:270)
at java.lang.Thread.run(Thread.java:829)
e1b6bbc
to
07a9d6a
Compare
|
I've filed the previous stack trace as #3858 (which is reproducible on 0.24.1 release). Here's another stack trace specific to the BarrageMessageProducer I've created with your latest commit: I'm not sure if this is the same bug as reverse, or not. |
rcaudy
reviewed
May 22, 2023
server/src/main/java/io/deephaven/server/barrage/BarrageMessageProducer.java
Outdated
Show resolved
Hide resolved
This one I was able to reproduce and should be all fixed now. Things appear to work AFAICT. I'm rebasing and fixing up for the |
rcaudy
approved these changes
May 23, 2023
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tested manually using reproducer provided by Devin:
I also ran Barrage Message Round Trip Tests.
The UI performance gets hit pretty hard by the 20ns period. I'm a little surprised by this, but decreasing to 1us is sufficient to eliminate the performance issue.