Content Filter Plugin

[Stebalien]

Background

A frequently requested feature is the ability to filter what content a go-ipfs node will download. While go-ipfs won't download anything that hasn't been requested by the user, that user might be:

• A dweb website requesting sub-resources via the local IPFS gateway.
• A user of a public IPFS gateway.

See libp2p/specs#173 for context.

Proposal

Ideally, IPFS would come with a feature out of the box for simply subscribing to a set of whitelists/blacklists. However, different users have different needs:

• Some users just need to download a static list, updating it once in a while.
• Some users need to query a remote service.
• Some users want to fetch their list from an HTTPs endpoint.
• Some users need to read their lists from local files.

Designing a one-size-fits-all solution is difficult and likely infeasible. However, there's a simple stop-gap solution: provide a plugin interface so users can implement their own versions.

The proposal here is to introduce new plugin types for filtering requests. Specifically:

type FilterBlock interface {
  Plugin

  // FilterBlock is called before fetching a block from an
  // exchange and/or adding(?) a block to the local datastore.
  FilterBlock(context.Context, cid.Cid) error
}

type FilterPath interface {
  Plugin

  // FilterPath is called before resolving a path (IPNS, dnslink, etc.).
  FilterName(context.Context, path.Path) error
}

type FilterContent interface {
  Plugin

  // FilterContent is called before serving content via the gateway, etc.
  //
  // Note: FilterContent doesn't prevent the content from being stored.
  // If desired, the passed CID can be used to delete or blacklist the
  // content after the fact.
  FilterContent(context.Context, cid.Cid, io.ReadSeeker) (io.ReadSeeker, error)
}

Questions:

• Should this filter locally added content (ipfs add)?

Implementation

This feature requires:

• A New plugin types as described above.
• A path for hooking plugins into go-ipfs subsystems. Ideally, the plugin loader would expose a method returning a set of fx options to be passed to the go-ipfs constructor.
• A blockservice wrapper that calls FilterBlock on all requested CIDs.
• Modifications to namesys to call FilterPath at every resolution step.
• Modifications to the CoreAPI's Unixfs().Get() method to call FilterContent.

[aschmahmann]

Recent proposal for this via a builtin command #8492

[lidel]

A minimal implementation of IPIP-383 from #10161 landed in master branch and is scheduled to be released in Kubo 0.24-rc1 for feedback. More details in /docs/content-blocking.md