Skip to content

Commit

Permalink
fix: mark ipns pubsub router DoNotWaitForStreamingResponses
Browse files Browse the repository at this point in the history
That means if the DHT has finished searching and no one responded over pubsub *yet*, we will not spend 1 minute searching for no reason.

This also include other error handling bug fixes inside `go-libp2p-routing-helpers`.

Fixes: #9927
  • Loading branch information
Jorropo committed Jul 19, 2023
1 parent 4a5e99d commit 1c22096
Show file tree
Hide file tree
Showing 9 changed files with 42 additions and 18 deletions.
9 changes: 5 additions & 4 deletions config/routing.go
Original file line number Diff line number Diff line change
Expand Up @@ -164,10 +164,11 @@ type ComposableRouterParams struct {
}

type ConfigRouter struct {
RouterName string
Timeout Duration
IgnoreErrors bool
ExecuteAfter *OptionalDuration `json:",omitempty"`
RouterName string
Timeout Duration
IgnoreErrors bool
DoNotWaitForStreamingResponses bool
ExecuteAfter *OptionalDuration `json:",omitempty"`
}

type Method struct {
Expand Down
11 changes: 7 additions & 4 deletions core/node/libp2p/routing.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ import (
type Router struct {
routing.Routing

Priority int // less = more important
Priority int // less = more important
DoNotWaitForStreamingResponses bool
}

type p2pRouterOut struct {
Expand Down Expand Up @@ -197,8 +198,9 @@ func Routing(in p2pOnlineRoutingIn) irouting.ProvideManyRouter {
var cRouters []*routinghelpers.ParallelRouter
for _, v := range routers {
cRouters = append(cRouters, &routinghelpers.ParallelRouter{
IgnoreError: true,
Router: v.Routing,
IgnoreError: true,
DoNotWaitForStreamingResponses: v.DoNotWaitForStreamingResponses,
Router: v.Routing,

Check warning on line 203 in core/node/libp2p/routing.go

View check run for this annotation

Codecov / codecov/patch

core/node/libp2p/routing.go#L201-L203

Added lines #L201 - L203 were not covered by tests
})
}

Expand Down Expand Up @@ -244,7 +246,8 @@ func PubsubRouter(mctx helpers.MetricsCtx, lc fx.Lifecycle, in p2pPSRoutingIn) (
Namespaces: []string{"ipns"},
},
},
Priority: 100,
Priority: 100,
DoNotWaitForStreamingResponses: true,

Check warning on line 250 in core/node/libp2p/routing.go

View check run for this annotation

Codecov / codecov/patch

core/node/libp2p/routing.go#L249-L250

Added lines #L249 - L250 were not covered by tests
},
}, psRouter, nil
}
Expand Down
10 changes: 10 additions & 0 deletions docs/changelogs/v0.22.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
- [Overview](#overview)
- [🔦 Highlights](#-highlights)
- [`ipfs name publish` now supports V2 only IPNS records](#ipfs-name-publish-now-supports-v2-only-ipns-records)
- [IPNS name resolution has been fixed](#ipns-name-resolution-has-been-fixed)
- [📝 Changelog](#-changelog)
- [👨‍👩‍👧‍👦 Contributors](#-contributors)

Expand All @@ -23,6 +24,15 @@ to V2 only in the future.

**TODO**: add links to IPIP https://github.com/ipfs/specs/issues/376

#### IPNS name resolution has been fixed

IPNS name resolution used to always take 1 minute if the record was not yet cached.
Even tho the DHT query most often finish in three digits miliseconds to single digits seconds.

This has been fixed, it now takes as long as the DHT will take.

For details see [#9927](https://github.com/ipfs/kubo/issues/9927) and [#10020](https://github.com/ipfs/kubo/pull/10020).

### 📝 Changelog

### 👨‍👩‍👧‍👦 Contributors
3 changes: 3 additions & 0 deletions docs/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -1509,6 +1509,9 @@ Parallel:
- `Timeout:duration`: Local timeout. It accepts strings compatible with Go `time.ParseDuration(string)` (`10s`, `1m`, `2h`). Time will start counting when this specific router is called, and it will stop when the router returns, or we reach the specified timeout.
- `ExecuteAfter:duration`: Providing this param will delay the execution of that router at the specified time. It accepts strings compatible with Go `time.ParseDuration(string)` (`10s`, `1m`, `2h`).
- `IgnoreErrors:bool`: It will specify if that router should be ignored if an error occurred.
- `DoNotWaitForStreamingResponses:bool`: Mark a router non blocking for streamin termination.
For streaming responses (`SearchValue` used by IPNS and `FindProvidersAsync` used to locate hosts for CIDs in the network), if some answer *has been found already* and all blocking routers are complete then cancel the query on non blocking routers,
for example if you have a LanDHT and a WanDHT, you can mark the LanDHT `"DoNotWaitForStreamingResponses": true`, this will make that if the LanDHT takes longer to complete than the WanDHT, the query on the LanDHT will be canceled if WanDHT has found some content.
- `Timeout:duration`: Global timeout. It accepts strings compatible with Go `time.ParseDuration(string)` (`10s`, `1m`, `2h`).

Sequential:
Expand Down
3 changes: 2 additions & 1 deletion docs/examples/kubo-as-a-library/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ require (
require (
bazil.org/fuse v0.0.0-20200117225306-7b5117fecadc // indirect
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 // indirect
github.com/Jorropo/jsync v1.0.1 // indirect
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5 // indirect
github.com/benbjohnson/clock v1.3.5 // indirect
Expand Down Expand Up @@ -106,7 +107,7 @@ require (
github.com/libp2p/go-libp2p-pubsub v0.9.3 // indirect
github.com/libp2p/go-libp2p-pubsub-router v0.6.0 // indirect
github.com/libp2p/go-libp2p-record v0.2.0 // indirect
github.com/libp2p/go-libp2p-routing-helpers v0.7.0 // indirect
github.com/libp2p/go-libp2p-routing-helpers v0.7.1-0.20230719022107-fd78e08c072b // indirect
github.com/libp2p/go-libp2p-xor v0.1.0 // indirect
github.com/libp2p/go-mplex v0.7.0 // indirect
github.com/libp2p/go-msgio v0.3.0 // indirect
Expand Down
6 changes: 4 additions & 2 deletions docs/examples/kubo-as-a-library/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 h1:cTp8I5+VIo
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96/go.mod h1:bOvUY6CB00SOBii9/FifXqc0awNKxLFCL/+pkDPuyl8=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/Jorropo/jsync v1.0.1 h1:6HgRolFZnsdfzRUj+ImB9og1JYOxQoReSywkHOGSaUU=
github.com/Jorropo/jsync v1.0.1/go.mod h1:jCOZj3vrBCri3bSU3ErUYvevKlnbssrXeCivybS5ABQ=
github.com/OneOfOne/xxhash v1.2.2 h1:KMrpdQIwFcEqXDklaen+P1axHaj9BSKzvpUUfnHldSE=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/aead/siphash v1.0.1/go.mod h1:Nywa3cDsYNNK3gaciGTWPwHt0wlpNV15vwmswBAUSII=
Expand Down Expand Up @@ -502,8 +504,8 @@ github.com/libp2p/go-libp2p-pubsub-router v0.6.0 h1:D30iKdlqDt5ZmLEYhHELCMRj8b4s
github.com/libp2p/go-libp2p-pubsub-router v0.6.0/go.mod h1:FY/q0/RBTKsLA7l4vqC2cbRbOvyDotg8PJQ7j8FDudE=
github.com/libp2p/go-libp2p-record v0.2.0 h1:oiNUOCWno2BFuxt3my4i1frNrt7PerzB3queqa1NkQ0=
github.com/libp2p/go-libp2p-record v0.2.0/go.mod h1:I+3zMkvvg5m2OcSdoL0KPljyJyvNDFGKX7QdlpYUcwk=
github.com/libp2p/go-libp2p-routing-helpers v0.7.0 h1:sirOYVD0wGWjkDwHZvinunIpaqPLBXkcnXApVHwZFGA=
github.com/libp2p/go-libp2p-routing-helpers v0.7.0/go.mod h1:R289GUxUMzRXIbWGSuUUTPrlVJZ3Y/pPz495+qgXJX8=
github.com/libp2p/go-libp2p-routing-helpers v0.7.1-0.20230719022107-fd78e08c072b h1:dc7aec2uIUNpzlSksfi3bepZH9OiOS4tPbMzrLqzyLE=
github.com/libp2p/go-libp2p-routing-helpers v0.7.1-0.20230719022107-fd78e08c072b/go.mod h1:cHStPSRC/wgbfpb5jYdMP7zaSmc2wWcb1mkzNr6AR8o=
github.com/libp2p/go-libp2p-testing v0.12.0 h1:EPvBb4kKMWO29qP4mZGyhVzUyR25dvfUIK5WDu6iPUA=
github.com/libp2p/go-libp2p-xor v0.1.0 h1:hhQwT4uGrBcuAkUGXADuPltalOdpf9aag9kaYNT2tLA=
github.com/libp2p/go-libp2p-xor v0.1.0/go.mod h1:LSTM5yRnjGZbWNTA/hRwq2gGFrvRIbQJscoIL/u6InY=
Expand Down
3 changes: 2 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ require (
github.com/libp2p/go-libp2p-pubsub v0.9.3
github.com/libp2p/go-libp2p-pubsub-router v0.6.0
github.com/libp2p/go-libp2p-record v0.2.0
github.com/libp2p/go-libp2p-routing-helpers v0.7.0
github.com/libp2p/go-libp2p-routing-helpers v0.7.1-0.20230719022107-fd78e08c072b
github.com/libp2p/go-libp2p-testing v0.12.0
github.com/libp2p/go-socket-activation v0.1.0
github.com/mitchellh/go-homedir v1.1.0
Expand Down Expand Up @@ -91,6 +91,7 @@ require (

require (
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 // indirect
github.com/Jorropo/jsync v1.0.1 // indirect
github.com/Kubuxu/go-os-helper v0.0.1 // indirect
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5 // indirect
Expand Down
6 changes: 4 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 h1:cTp8I5+VIo
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96/go.mod h1:bOvUY6CB00SOBii9/FifXqc0awNKxLFCL/+pkDPuyl8=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/Jorropo/jsync v1.0.1 h1:6HgRolFZnsdfzRUj+ImB9og1JYOxQoReSywkHOGSaUU=
github.com/Jorropo/jsync v1.0.1/go.mod h1:jCOZj3vrBCri3bSU3ErUYvevKlnbssrXeCivybS5ABQ=
github.com/Kubuxu/go-os-helper v0.0.1 h1:EJiD2VUQyh5A9hWJLmc6iWg6yIcJ7jpBcwC8GMGXfDk=
github.com/Kubuxu/go-os-helper v0.0.1/go.mod h1:N8B+I7vPCT80IcP58r50u4+gEEcsZETFUpAzWW2ep1Y=
github.com/OneOfOne/xxhash v1.2.2 h1:KMrpdQIwFcEqXDklaen+P1axHaj9BSKzvpUUfnHldSE=
Expand Down Expand Up @@ -563,8 +565,8 @@ github.com/libp2p/go-libp2p-pubsub-router v0.6.0 h1:D30iKdlqDt5ZmLEYhHELCMRj8b4s
github.com/libp2p/go-libp2p-pubsub-router v0.6.0/go.mod h1:FY/q0/RBTKsLA7l4vqC2cbRbOvyDotg8PJQ7j8FDudE=
github.com/libp2p/go-libp2p-record v0.2.0 h1:oiNUOCWno2BFuxt3my4i1frNrt7PerzB3queqa1NkQ0=
github.com/libp2p/go-libp2p-record v0.2.0/go.mod h1:I+3zMkvvg5m2OcSdoL0KPljyJyvNDFGKX7QdlpYUcwk=
github.com/libp2p/go-libp2p-routing-helpers v0.7.0 h1:sirOYVD0wGWjkDwHZvinunIpaqPLBXkcnXApVHwZFGA=
github.com/libp2p/go-libp2p-routing-helpers v0.7.0/go.mod h1:R289GUxUMzRXIbWGSuUUTPrlVJZ3Y/pPz495+qgXJX8=
github.com/libp2p/go-libp2p-routing-helpers v0.7.1-0.20230719022107-fd78e08c072b h1:dc7aec2uIUNpzlSksfi3bepZH9OiOS4tPbMzrLqzyLE=
github.com/libp2p/go-libp2p-routing-helpers v0.7.1-0.20230719022107-fd78e08c072b/go.mod h1:cHStPSRC/wgbfpb5jYdMP7zaSmc2wWcb1mkzNr6AR8o=
github.com/libp2p/go-libp2p-testing v0.12.0 h1:EPvBb4kKMWO29qP4mZGyhVzUyR25dvfUIK5WDu6iPUA=
github.com/libp2p/go-libp2p-testing v0.12.0/go.mod h1:KcGDRXyN7sQCllucn1cOOS+Dmm7ujhfEyXQL5lvkcPg=
github.com/libp2p/go-libp2p-xor v0.1.0 h1:hhQwT4uGrBcuAkUGXADuPltalOdpf9aag9kaYNT2tLA=
Expand Down
9 changes: 5 additions & 4 deletions routing/delegated.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,10 +105,11 @@ func parse(visited map[string]bool,
}

pr = append(pr, &routinghelpers.ParallelRouter{
Router: ri,
IgnoreError: cr.IgnoreErrors,
Timeout: cr.Timeout.Duration,
ExecuteAfter: cr.ExecuteAfter.WithDefault(0),
Router: ri,
IgnoreError: cr.IgnoreErrors,
DoNotWaitForStreamingResponses: cr.DoNotWaitForStreamingResponses,
Timeout: cr.Timeout.Duration,
ExecuteAfter: cr.ExecuteAfter.WithDefault(0),
})

}
Expand Down

0 comments on commit 1c22096

Please sign in to comment.