Sprint Sep 21

[RichardLitt]

Sprint Goals

• Finalize roadmap and ship 0.3.8

Sprint Discussions

See https://github.com/ipfs/pm#sprint-discussion-schedule

Please take notes in a separate pad, if you can, and link it here.

• main pad: https://etherpad.mozilla.org/nTqGOVynPT
• infrastructure: https://etherpad.mozilla.org/mJ1ryjJLeH

Sprint Deliverables

• Add yourself below, that way we only have people listed who are really in the sprint
• Add links to issues down here. Only add things you can finish this sprint.

@RichardLitt

• Take a crack at node-ipfs-api docs!
• Work with lgierth to make sure that set-dns-record tool works as expected.
• Streamline ipfs/support into website and community flow.
• Build a community page listing all of the repos currently in development and make sure that their internal READMEs are streamlined. Write a starlog about this once it is done.
• Close Removed the base html tag ipfs-inactive/website#14 and all baseURL website issues. Also see Links on command list are broken ipfs-inactive/website#2.
• Resolve issues on ipfs/community as much as possible. Don't go overboard.
• Fix textarea scrolling on UI on composing breaks when post is longer than screen space ipfs-inactive/starlog#26
• See if I can codify how I can help manage sprints and project management better.
• Give IPFS install another test run through for ease of use.
• Work on IPFS-textbook
• Finish job spec.
• Get information on good DNS providers; open an issue about it.
• Ask for cool websites we want to have IPFS look like.
• Cake day issues. JSON in ipfs/community for cake days and travel plans.
• Make a list of things to do for new repos: for instance, adding Contribute, Adding Readme, adding badges, Add language contribute link, add travis, add CI, add to roadmap / repo list.
• Add -contribute.md files to community list. See Added a Contribute file kubo#1734
• Review at least one node-ipfs module
• Fill out https://github.com/ipfs/ipfs/blob/master/project-directory.md

@diasdavid || @daviddias

• Open a call for volunteers + Captain.log 🚣 captain.log - IPFS JavaScript implementation 🌟 js-ipfs#30
• Fix for #4 libp2p/js-spdy-stream-muxer#5
• Organize ipfs-lx
• Complete Swarm Revisit https://github.com/diasdavid/node-ipfs-swarm/tree/revisit
• Make node-libp2p bindings
• Merge Require "go-ipfs" directly to get exe path. js-ipfsd-ctl#21 to fix bump ipfsd-ctl version, fixes #64 ipfs-inactive/js-ipfs-http-client#65

@dignifiedquire

Workable state of the electron app

• Finish current efforts and clean up [WIP] Hot reloading and redesign ipfs-desktop#43
• Test new dist version of electron app on windows and mac and find people to test on linux
• Start thinking of an improved webui (design)

@lgierth

• gateway-dmca-denylist https://github.com/ipfs/gateway-dmca-denylist
• dmca.denylists.lists.refs.ipfs.io implement access control lists for gateway kubo#1551
• [~] peer discovery using cjdns WIP: discover peers from cjdns kubo#1316
• [~] fix the pluto private key situation config: update pluto's peerID kubo#1748
• cjdns on solarnet Alexandria peering infra#94
  • update cjdns
  • update peering tools
  • allow alexandria peering with us
  • peer with ansuz' new node in london
• update infrastructure documentation
• move pollux to a protocol labs account Transfer pollux host to a Protocol Labs account infra#73
• gateway caching headers

@whyrusleeping

• finish private-blocks PR (ready for CR again) (WIP: separate private and public blocks kubo#1538)
• finish mfs extraction (mfs redo kubo#1715)
• epoll fixes (no need for multiple epoll fds libp2p/go-reuseport#8)
• integrate UDT into go-ipfs
  • go-udtwrapper fixes (cleanup code to remove race conditions jbenet/go-udtwrapper#13)
  • PR it into multiaddr-net
  • finally, PR that into go-ipfs
• add chunking algo to root unixfs file blocks
• ipfs update command
• dht command tweaks (allow for easier key input)

@amstocker

• python-ipfs-api:
  • pydoc documentation
  • make readthedocs.org page
  • make it easy to integrate python-ipfs-api into async environment
  • make filestreaming chunk large files/multiple files
  • python 3 support
  • testing framework
  • tavisci setup
• python-multiaddr
  • make testing framework
  • python 3 support
  • feature parity with go-multiaddr/node-multiaddr

@jbenet

• put ipld into go-ipfs
• meet with people @Stanford
• make libp2p go interfaces file
• node-ipfs repo module listings
• api docs designdoc

@rht

• refactor ipfs add (I have a local branch where the bulk of core/commands/add.go is moved to core/coreunix/add.go...if this is preferable?) (and optionally investigate slow add but this is not guaranteed to finish)
• fix gc test (pr has no merge conflict with 0.4.0)

[jbenet]

Food for go-ipfs #28 (comment)

[jbenet]

needed in go-ipfs@0.3.8

• ipfs update
• UDT transport
• IPNS patches

[daviddias]

@jbenet are we going full UDT or just supporting that primarily for now?

[whyrusleeping]

@diasdavid see ipfs/kubo#1738

[jbenet]

Reminder: Only add things you can finish this sprint.

[jbenet]

@rht add tasks to etherpad?

[daviddias]

SPRINT CHECKIN

• Started a 'captain.log' on node-ipfs repo, like a dev news letter for all things node-ipfs 🚣 captain.log - IPFS JavaScript implementation 🌟 js-ipfs#30 so that folks can feel welcome to chip in to the current endeavours.
• Lisbon IPFS Meetup - Got the Venue and the date (Oct 7th) and some awareness around the Lisbon community. Unfortunately meetup.com deleted our meetup (+ the Portland one) for lack of information in the description, reached out to support to get it back, waiting to hear from them.
• Swarm revisit Revisit Swarm - multitransport + upgrades - https://github.com/diasdavid/node-ipfs-swarm/issues/8 libp2p/js-libp2p-switch#10, now there is organization to add multiple transports and how to upgrade connections, following https://github.com/diasdavid/node-libp2p-swarm/issues/8
• Reviewed and merged a bunch of PRs across the board, from node-ipfs-api, npm-go-ipfs, libp2p stuff and more
• Discussed with @whyrusleeping on what is Bitswap to start implementing it on node-ipfs
• Got some notes from @jbenet on how the chunking of IPFS is done and structured
• Reviewed and posted feedback to PANDO, the PhD proposal from Erick Lavoie
• Landed the greenfield for libp2p-website https://github.com/diasdavid/libp2p-website (really happy how nice it is to use IPFS to publish static websites with simple npm scripts), continue working on the plan of ip-surge tool (that resembles https://surge.sh/)

[ghost]

Pad for next sprint (Sep 28): https://etherpad.mozilla.org/juEv7EcG5l

[whyrusleeping]

• UDT cleanup and tests
  • more cross platform makefile
• ipns patches (writing some more tests)
• iptb can use port 0
  • misc other iptb improvements
• fix api file for port zero
• ipfs update command (wrote scaffolding, still needs infra work)
• bitswap stat returns number of bytes wasted
• docs for ipfs object patch
• better input parsing for 'ipfs dht' commands
• wrote up beginnings of 0.3.8 changelog
• began discussion of udt transition steps
• bitswap discussion with @daviddias

[hackergrrl]

• rig up a crude 'ipget' prototype (https://github.com/noffle/ipget/)

Lots of learning around both the go-ipfs codebase and how to do things The Right Way(tm) with golang -- loving the unixy tooling. Getting the ipget tool working was pretty easy once @whyrusleeping showed me how to get a fresh node up. No blockers.

[ghost]

• the little gateway-dmca-denylist tool finally does something -- renders and adds a structure of notices and to-be-blocked objects https://github.com/ipfs/gateway-dmca-denylist
• ipfs.io/refs/lists/denylists/dmca should resolve to dmca.denylists.lists.refs.ipfs.io, which is a dnslink to the latest version of the denylist implement access control lists for gateway kubo#1551
• [~] peer discovery using cjdns, i'm updating the branch right now and will add the missing dialing parts WIP: discover peers from cjdns kubo#1316
• [~] fix the pluto private key situation -- we lost pluto's private key a while ago, why is mining for a new one starting in SoL[Pp] config: update pluto's peerID kubo#1748
• cjdns on solarnet Alexandria peering infra#94
  • update cjdns
  • update peering tools
  • allow alexandria peering with us
  • peer with ansuz' new node in london
• update infrastructure documentation
• move pollux to a protocol labs account Transfer pollux host to a Protocol Labs account infra#73
• gateway caching headers

edit:

• made dnslink-deploy work with @RichardLitt https://github.com/ipfs/dnslink-deploy

[dignifiedquire]

Main goal: Workable state of the electron app

• Finish current efforts and clean up [WIP] Hot reloading and redesign ipfs-desktop#43
• Test new dist version of electron app on windows and mac and find people to test on linux
• Start thinking of an improved webui (design)
• Being sick

[jbenet]

• put ipld into go-ipfs
• meet with people @Stanford
• [~] make libp2p go interfaces file (posting later today)
• [~] node-ipfs repo module listings (posting later today)
• [~] api docs designdoc (posting later today)
• reached out to golang list re golang/build https://groups.google.com/forum/#!topic/golang-nuts/kRq8Ow4qEeU
• reached out to quic list re auth + asymmetry https://groups.google.com/a/chromium.org/forum/#!topic/proto-quic/b8c39UbgmKA and https://groups.google.com/a/chromium.org/forum/#!topic/proto-quic/Gf6Hi7XJFAc
• eval CurveCP as a transport
• eval QUIC, https://github.com/devsisters/goquic, and feasibility
• eval uTP/LEDBAT, https://github.com/anacrolix/utp, https://github.com/jbenet/go-libutp
• call with ethereum swarm people
• review + feedback for http://www.ietf.org/id/draft-benfield-http2-p2p-01.txt
• figure out details re UTF-8 on paths (mailed plan9ers)
• addressed windows' users problems and triggered gobuilder manually (wtf is going on? why is it still broken?)
• met with OKFN Ireland -- Denis Parfenov, Ingo, and Zach

[RichardLitt]

• Work with lgierth to make sure that set-dns-record tool works as expected.
• Streamline ipfs/support into website and community flow.
• Close Removed the base html tag ipfs-inactive/website#14 and all baseURL website issues. Also see Links on command list are broken ipfs-inactive/website#2.
• Work on IPFS-textbook
• Get information on good DNS providers; open an issue about it.
• Ask for cool websites we want to have IPFS look like.
• Cake day issues. JSON in ipfs/community for cake days and travel plans.
• Review at least one node-ipfs module
• Fill out https://github.com/ipfs/ipfs/blob/master/project-directory.md

Worked on:

• Build a community page listing all of the repos currently in development and make sure that their internal READMEs are streamlined. Write a starlog about this once it is done.
• Make a list of things to do for new repos: for instance, adding Contribute, Adding Readme, adding badges, Add language contribute link, add travis, add CI, add to roadmap / repo list.
• Fix textarea scrolling on UI on composing breaks when post is longer than screen space ipfs-inactive/starlog#26

Not done:

• Take a crack at node-ipfs-api docs!
• Resolve issues on ipfs/community as much as possible. Don't go overboard.
• See if I can codify how I can help manage sprints and project management better.
• Give IPFS install another test run through for ease of use.
• Finish job spec.
• Add -contribute.md files to community list. See Added a Contribute file kubo#1734

[rht]

• Refactor ipfs add has to wait for 0.4.0 rebase.
• gc: disk size stuff on os x

[jbenet]

@rht is that done this week? or upcoming? you'll want to add next tasks to etherpad in #35

[jbenet]

@jbenet's sprint feedback

(sorry for misisng the timing, hopefully this compensates a bit)

@RichardLitt:

• love the dnslink-deploy tool! def was a missing piece. https://github.com/ipfs/dnslink-deploy for those interested
• +1 for ipfs/support. may want to advertise it over at https://groups.google.com/forum/#!forum/ipfs-users too.
• +1 on all the website push stuff

@diasdavid

• +1 for captain's log. great work. will do one for go-ipfs
• sorry about the meetup.com fail
• yay on swarm revisit.
• thanks for keeping on top of maintaining all the js modules

@dignifiedquire

• hope you're feeling all better!
• and nice designs

@lgierth

• good progress with denylist
• love the dnslink-deploy tool! def was a missing piece. https://github.com/ipfs/dnslink-deploy for those interested

@whyrusleeping

• yay for all the cpp UDT work. will come in very handy
• thanks for the iptb update, much better tests incoming
• 0.3.8 so close!

@rht

• the benchmark will be very useful. thanks for pressing on that

@noffle

• 👏 on ipget -- it's so nice and simple. now we can plug IPFS into important things! \o/

@amstocker

• +1 on all the testing. great to have.

[jbenet]

• @mappum - want to drop a sprint update for all the ipfs-node stuff here?
• @davidar - want to drop an update here for archives and so on?
• @chriscool - want to drop an update here re tests?

also, thanks to:

• @ianopolous finished https://github.com/ipfs/java-ipfs-api 👏 👏 (feel free to drop updates here!)
• @mildred - you pushed on very important IPLD stuff + discussion points. lmk if i'm blocking on the code (will look back)
• @willglynn thanks for implementing path support -- will take closer look this week

and thanks to lots of other people that made IPFS lots better this week! \o/ :fireworks:

[chriscool]

• Added shellquote() thanks to @ion1, see ipfs-test-lib: add shellquote() kubo#1757

Started to fix it as @ion1 suggested and to add tests, but it's not so easy to add tests because of all the quoting.

[ianopolous]

All these are in java-ipfs-api

• cleaned up api, removed MerkleNode where it was only used as a proxy for Multihash.
• ported MultiAddress, including binary encoding and all the protocols as well as varints from Go
• gave IPFS a constructor taking a MultiAddress (nice suggestion @jbenet )
• wrote blog post about it (http://ianopolous.github.io/java/IPFS/)

To Do

• Use the API in Peergos, which will probably result in further improvements
• Javadoc the API
• make sure the relation between the different types of pinning to normal add is clear to new comers
• give it an O(1) memory streaming api - ipfs.add(InputStream)
• Toying with the idea of a cross platform IPFS installer (in Java) which we would also use in Peergos

[davidar]

• IPFS-hosted markup editor thanks to @osener
• discussed getting webrecorder.io onto IPFS with @ikreymer IPFS as a backend to a web archiving ipfs-inactive/archives#28
• made a little bit of progress with the BASE data BASE ipfs-inactive/archives#3 (HELP WANTED, if anyone's interested)
• mirrored PGP pubkeys PGP public key database ipfs-inactive/archives#27
• tried to get the bookreader working with BPG (please ping me if you're an emscripten expert)
• pushed the first complete arXiv archive
• argued about IPLD some more ;)
• agreed on a standard format for archive metadata

[mappum]

• planned initial npm-bootstrapped ipfs-npm prototype
• implemented inpm with /ipfs/HASH symlink install
• filed issues on node-ipfsd-ctl for bugs that were blocking inpm (thanks for fixing stuff, @bcomnes)
• went to Tokyo ✈️ 🇯🇵

[rht]

pushed the first complete arXiv archive

Thanks a lot for this.
There is e.g. https://blog.lateral.io/2015/07/harvesting-research-arxiv/ (https://arxiv.lateral.io/), but definitely a lot more stuff can be done, etc.
(haven't dumped my thoughts yet...)

No, finished neither of the 2 checklists.

[ozanmakes]

Great initiative! Is there anything I can do to make markup.rocks easier to integrate?

[davidar]

Is there anything I can do to make markup.rocks easier to integrate?

@osener That would be great :). What I'd really like to do is replace the Dropbox open/save links with IPFS equivalents (similar to the IPFS wiki by @jamescarlyle). I've started on an IPFS Haskell API (ipfs/ipfs#4), which may be able to help with this.

Feel free to ping me (davidar) on http://chat.ipfs.io (or #ipfs on freenode) if you want to discuss this further.

[davidar]

@rht yes, definitely interested in hearing ideas for utilising the arxiv data. The ones I've thought of are:

• search engine for LaTeX equations
• publishing papers in alternative formats (html, etc)
• building a citation graph
• training a topic model (latent Dirichlet allocation / hierarchical Dirichlet processes)
• automatically extracting definitions to build a dictionary of terminology
• semantic markup of math (connecting variables in equations to their textual descriptions, disambiguating notation, etc)

Edit: see ipfs/apps#1

[jamescarlyle]

FWIW I'm planning to add a markdown parser to my ipfs-wiki, so that standard md syntax can be used (and then rendered as HTML)