block sent listener is not called for every block sent #126
Comments
|
It seems that the root cause of the issue is that when the messagequeue sends a message, the message may contain several blocks, but only one "Sent" event is emitted. go-graphsync/messagequeue/messagequeue.go Lines 202 to 207 in 11d30c6 When this event bubbles up to the responsemanager subscriber, it only includes the block data for the first block (not all blocks in the message) |
|
Hey, so this exposes a flaw in the mappable subscriber system, and provides a pretext for refactoring the interface. |
|
@acruikshank I am recommending a fix to the way MappableSubscriber currently works, as well as some interface changes that will make things more clear: These are my recommended new interfaces -- please don't take these set in stone and please feel free to improve naming. (taken from: https://github.com/ipfs/go-graphsync/blob/master/notifications/types.go) // Topic is a topic that events appear on
type Topic interface{}
// Event is a publishable event
type Event interface{}
// TopicData is data added to every message broadcast on a topic
type TopicData interface{}
// Subscriber is a subscriber that can receive events
type Subscriber interface {
OnNext(Topic, Event, ...TopicData)
OnClose(Topic, ...TopicData)
}
// TopicDataSubscriber is a subscriber that can inject data to into subscriber callbacks when events happen for a given topic
type TopicDataSubscriber interface {
Subscriber
AddTopicData(topic Topic, data TopicData)
}
// Subscribable is a stream that can be subscribed to
type Subscribable interface {
Subscribe(topic Topic, sub Subscriber) bool
Unsubscribe(sub Subscriber) bool
}
// Publisher is an publisher of events that can be subscribed to
type Publisher interface {
Close(Topic)
Publish(Topic, Event)
Shutdown()
Startup()
Subscribable
}
// EventTransform if a fucntion transforms one kind of event to another
type EventTransform func(Event) Event
// Notifee is a topic data subscriber plus a set of data you want to add to any topics subscribed to
// (used to call SubscribeWithData to inject data when events for a given topic emit)
type Notifee struct {
Data TopicData
Subscriber TopicDataSubscriber
}
// SubscribeWithData subscribes to the given subscriber on the given topic, and adds the notifiees
// custom data into the list of data injected into callbacks when events occur on that topic
func SubscribeWithData(p Subscribable, topic Topic, notifee Notifee) {
notifee.Subscriber.AddTopicData(topic, notifee.Data)
p.Subscribe(topic, notifee.Subscriber)
}-- I also recommend:
-- All of this will neccesitate changes to all of the subscribers. We will need to dispatch multiple BlockSent notifications in the subscriber for the responsemanager, which will ultimately be the solution to the root issue. |
I added
TestDataSentEventsas an integration test in go-data-transfer to watch for events emitted. It seems thatgsBlockSentHookis not called every time that a block is sent (whereasgsIncomingBlockHookis called every time a block is received).gsBlockSentHookis registered with graphsync usingRegisterBlockSentListener.The full output from the test is below. Note the output for the last two lines:
On the provider graphsync emitted "receive" events for
20439bytes.On the client graphsync emitted "queue" events for
20439bytes, but only emitted "sent" events for14295bytes.Full output:
The text was updated successfully, but these errors were encountered: