Skip to content
This repository has been archived by the owner on Feb 8, 2023. It is now read-only.

Sprint: February 22 -- March 7 #93

Closed
RichardLitt opened this issue Feb 22, 2016 · 23 comments
Closed

Sprint: February 22 -- March 7 #93

RichardLitt opened this issue Feb 22, 2016 · 23 comments

Comments

@RichardLitt
Copy link
Member

Sprint February 22nd

Sprint Goals

Sprint Discussions

Schedule

Please take notes in a separate pad, if you can, and link it here.

Endeavour Lead Time (PDT - UTC/Z - CET) Pad
sync @RichardLitt 9:00pDT 17:00Z 18:00CET IRC: #ipfs on Freenode
apps on ipfs @dignifiedquire 10:30PDT 18:30Z 19:30CET https://public.etherpad-mozilla.org/p/ipfs-feb-22-apps-on-ipfs
infrastructure @lgierth 11:00PDT 19:00Z 20:00CET https://public.etherpad-mozilla.org/p/ipfs-feb-22-infrastructure
libp2p @diasdavid 11:30PDT 19:30Z 20:30CET https://public.etherpad-mozilla.org/p/ipfs-feb-22-libp2p
js-ipfs @diasdavid 12:00PDT 20:00Z 21:00CET https://public.etherpad-mozilla.org/p/ipfs-feb-22-js-ipfs
go-ipfs @whyrusleeping 12:30PDT 20:30Z 21:30CET https://public.etherpad-mozilla.org/p/ipfs-feb-22-go-ipfs

Please add the Agenda to the Pad before the endeavour sprint starts.

Sprint Deliverables

  • Add your tasks below in a comment, that way we only have people listed who are really in the sprint
  • Add links to issues down here. Only add things you can finish this sprint.
@RichardLitt RichardLitt changed the title Spring: February 22 Sprint: February 22 Feb 22, 2016
@RichardLitt RichardLitt mentioned this issue Feb 22, 2016
Closed
@dignifiedquire
Copy link
Member

Apps on IPFS

Lead: @dignifiedquire

Participants

Agenda

  • Webui
    • Contributions
    • ipfs geoip
  • Station
    • ARM
    • Stability
    • Tests
  • Dignified Hacks

Notes

  • Webui
  • Station
    • ARM
      • Blocked on some electron-packager/electron issues for proper build
    • Stability
    • Tests
  • Dignified Hacks

@RichardLitt
Copy link
Member Author

To Do this week

@daviddias
Copy link
Member

Libp2p

Lead: @diasdavid

Participants

Agenda & Notes

  • status update
    • webrtc-explorer
    • libp2p builds for IPFS
  • next steps
    • finish webrtc-explorer reboot - The Packet Switch Overlay Network over RTCDataChannels
    • libp2p-utp update to utp-native
    • libp2p-websockets for js and go (cause go doesn't have a good WebRTC implementation)
    • exo-transports spec - exo-tranports are external processes that offer transport implementations, this processes make an RPC API available through a unix socket or even other transport. This saves time and energy from developing every single transport in every language.

@hackergrrl
Copy link
Contributor

@noffle sprint goals

@daviddias
Copy link
Member

js-ipfs

Lead: @diasdavid

Participants

Agenda

@daviddias
Copy link
Member

DAVID DIAS TODO

js-ipfs

js-ipfs-api

libp2p

npm on ipfs

station

ipscend

js-ipfsd-ctl

specs

@whyrusleeping
Copy link
Member

@whyrusleeping sprint goals

  • large big huge testing framework/tests
  • pinning fix PR merged go-ipfs#2384
  • build process improved
    • force gx to use 0.4.0 gateways
    • better docs around building from source
    • go compiler version checks
    • make gx handle cross compilations better
    • makefile should warn user if not building from within GOPATH
  • try for zero random test failures on travis and circle

@nginnever
Copy link
Member

@nginnever goals

- js-ipfs - [ ] command cat (meow) - js-ipfs-data-importing (js-ipfs issue #41) - [ ] streamImporter (more questions) - [~] BufferImporter (CR) - [ ] export function - [ ] browser / mocha tests - js-peer-id - [ ] create dist files - Webui - [ ] write some tests for new components - [ ] document in contibuting.md

@whyrusleeping
Copy link
Member

go-ipfs

Lead: @whyrusleeping

Participants

Agenda

  • CI is greener, needs to be even more greenerer
  • Large Big Huge Test Things (tm)

Notes

  • generate sequence of random actions
  • build harness to run those generated actions, and halt on failure
  • check results and integrity every so often

Asks

  • fix some appveyor failures
  • provide feedback on process of installing ipfs from source
  • vendor packages in /thirdparty out with gx
    • ask @whyrusleeping for any help
    • any and all UX feedback here is appreciated

@ghost
Copy link

ghost commented Feb 24, 2016

Infrastructure

Lead: @lgierth

Participants

Agenda

  • a lot of requests for pinbot to work again. (also on 0.4)
  • gateway issues
  • moving away from docker, in part
  • metrics
    • v03x
  • dist.ipfs.io needs https

Notes

  • pinning
    • goroutine explosion and thus bitswap contention, there's a PR by @whyrusleeping fixing it
    • pinbot and pinbot-v03x
    • there is a PR open for pinbot, which starts collecting a note for each !pin invocation
  • geoip
    • merkledag and source data need pinning on both v03x and v04x
    • ipfs-station will use geoip merkledag, and ships go-ipfs 0.3.11
  • ssl for *.ipfs.io
    • the current cert only covers ipfs.io and gateway.ipfs.io
    • i fucked up the identity verification for getting a wildcard
    • having another try now :)
  • gateway issues
    • multireq was incorrectly passing on certain error responses, instead of trying the second network too
    • docker daemon sometimes crashes
    • docker sometimes loses track of container state
    • moving away from docker for now, will write down issues we had
    • next: runc and runit
  • metrics

@ghost
Copy link

ghost commented Feb 24, 2016

@dignifiedquire
Copy link
Member

  • libp2p
    • karma-peer
      • Add dynamic browser launching
  • ipfs-geoip
    • Pin all the data
    • Release 2.0.0
  • webui
    • Use new ipfs-geoip for location data
    • Add polling to the peers page
    • Files browser
    • Write captain.log (Captain.log ipfs-webui#259)
    • Finish tests for peers caching
  • dignified hacks
    • Base setup on beamp.pro
    • Decide on content for the first session: "Webui File Browser"
    • Schedule first session
    • Do first session
  • go-ipfs
    • Big Bad Tests (TM)
      • Randor - Randomized testing prototype
      • Base structure
      • Add and pin commands
      • Better randomness generators
  • PM
    • Discuss new sync format
    • Write update for weekly
  • website
    • Ship new install instructions

@Kubuxu
Copy link
Member

Kubuxu commented Feb 25, 2016

@Kubuxu TODO:

@RichardLitt RichardLitt changed the title Sprint: February 22 Sprint: February 22 -- March 7 Feb 29, 2016
@RichardLitt
Copy link
Member Author

IRC mid-fortnight hangout today. Will post updates in here.

@RichardLitt
Copy link
Member Author

Endeavour Lead Time (PDT - UTC/Z - CET) Pad
sync @RichardLitt 9:00pDT 17:00Z 18:00CET IRC: #ipfs on Freenode

@daviddias
Copy link
Member

DAVID DIAS SPRINT CHECKIN

js-ipfs

Summary: This sprint on js-ipfs was a lot about enablement and getting the next big feature on the roadmap done, that is swarm. I've focused on creating the foundation on js-ipfs for it to be easy to add extra features implemented, that is in: core, http-api or cli, by doing it myself for for a group of features, add comprehensive tests for each part and explain what I've done in the Captain.log + listing what needs to get done on issues, tagging them with difficulty and the 'help wanted' label, in case they are available to be done without anything blocking, this lead to several new contributions, thank you everyone :). On the swarm level, I've added it to js-ipfs, but started hitting some problems when it comes to interop with go-ipfs go-libp2p, to go after this issues, I entered the Go land, and went on an adventure to simplify js-libp2p-swarm, to reduce the amount of magic it was doing by default, and expose more operations, so it is easier to debug (by having more control), see the libp2p segment for more updates on that.

Tasks:

js-ipfs-api

Summary: Fixed ls and refs features on the 0.4.0 branch. CR'ed and merged the contributions by @xicombd .

libp2p

Summary: I've finished and released the webrtc-explorer 2.0.0 alpha, woot! :) On js-libp2p-ipfs, I started refactoring a lot of code, cleaning API, leveraging spdy-transport 2.0.0 to make libp2p-spdy less hacky. libp2p-utp was updated to use utp-native and libp2p-websockets was created to enable browser IPFS nodes. There is really ton of refresh and more code coverage.

specs

Summary: Standard http-api-spec reviews for @RichardLitt.

extra

Summary: On the side, after talking with @whyrusleeping, I decided to invest some time learning Go, trying to get as much as I could in the less amount of time possible (I can give some tips if someone wants to take the same path), the reason behind it is that it will improve a lot our iteration speed if both me and @whyrusleeping are comfortable with go-ipfs and js-ipfs code, specially when getting interop at the network level, which was the topic that sparked this conversation. I managed to do a contribution, ipfs repo stat, unblocking @dignifiedquire ipfs/js-ipfsd-ctl#18.

@RichardLitt
Copy link
Member Author

Summary

These past two weeks have been productive. I helped us switch up the sprint, so that we have a Monday off and that we run on a fortnightly basis. This involved a lot of talking to people. I reviewed all of the open PRs on http-api-spec, and merged a few of them - we're getting there, although there is more work to do (mostly involving me talking to @whyrusleeping). This coincided with a lot of go-ipfs PRs. We published last month's weekly, which took way more time than it ought to have, and I'm hoping that never happens again. I also udpated a few bugs and added a dependency, wihch gets PR creators, to name-your-contributors, the module I use for getting a list of community contributors for the IPFS Weekly. I gave a talk about this module at BostonJS, and got to talk about IPFS there, too.

Tasks I said I'd do

Extra stuff I did

@Kubuxu
Copy link
Member

Kubuxu commented Mar 7, 2016

@Kubuxu Update:

I wasn't super productive when it comes to IPFS (Uni started, Vulkan got released) but I was able to get something done. gx, gx-go and ipget are now available on AUR which is really nice. I worked on blag about CRDT on IPFS which would sum up my thoughts, ideas, problems and solutions. It will be something others could reference while implementing their owns. I am blocked really hard on ipfs/kubo#2405, I can flush the datastore but it will probably repeat. Also general question: how to name packaged fs-repo-migrations? Ping me on IRC with possible names.

@dignifiedquire
Copy link
Member

@dignifiedquire Sprint Checking

Success is stumbling from failure to failure with no loss of enthusiasm.
-- Winston S. Churchill

Two weeks have gone by, and I actually got everything done I set out to do
in the last sprint. karma-peer now has the ability to dynamically launch browsers,
which will help @diasdavid, and hopefully more people, to write better tests for
libp2p in the browser.
The "Big Bad Tests" for go-ipfs are on on their way with
randor, which now is able to rerun tests predictably based on stored data, so it's
easy to find and fix bugs. @whyrusleeping is already working on the first bug that
randor detected.
A big highlight for me personally was the first show of "dignified hacks" last Monday, where I
live streamed myself coding on the big webui refactoring. There were some technical
issues but I hope to do another one this week.
The overall progress on the webui is that I started on the files browser (based on new files
api in 0.4) which already lists files and is able to create directories. In addition all
the geolocation data now uses the fixed ipfs-geoip.
And last but not least, I familiarized myself with the codebase of the website, fixing some
basic performance issues (image compression, dead code) and started thinking of what the
next iteration might look like.

Big Long Lists

Things I said I would do

  • libp2p
    • karma-peer
      • Add dynamic browser launching
  • ipfs-geoip
    • Pin all the data (0.4 is done, not sure about 0.3)
    • Release 2.0.0
  • webui
    • Use new ipfs-geoip for location data
    • Add polling to the peers page
    • Started work on the files browser
    • Write captain.log (Captain.log ipfs-webui#259)
    • Finish tests for peers caching
  • dignified hacks
    • Base setup on beamp.pro
    • Decide on content for the first session: "Webui File Browser"
    • Schedule first session
    • Do first session
  • go-ipfs
    • Big Bad Tests (TM)
      • Randor - Randomized testing prototype
      • Base structure
      • Add and pin commands
      • Better randomness generators
  • PM
    • Discuss new sync format
    • Write update for weekly
  • website
    • Ship new install instructions

Things I did additionally

@nginnever
Copy link
Member

@nginnever Sprint

The past two weeks I have been studying more than I have been producing but I am getting much more comfortable with tests, the difference between using IPFS in the browser and node env, standards and practices, and so on. I wasn't able to spend as much time as I would like working on and studying our react webui project but I did tune into dignified hacks and that was helpful for a quick view of our components and data flow in the webui.

I spent some time researching a few commands in go-ipfs (cat and object get). I wrote some notes or a step-by-step analysis of what happens during those commands or what I can understand as to what's happening after seeing noffle do this with init. I spent some time writing web tests for data-importing module with dig and I started writing some export feature code to unwrap our hashes and retrieve files. Also created a distribution file for peer-id that will require node/forge rebuild to be able to work with browserify.

  • js-ipfs
    • [~] command cat (meow)
  • js-ipfs-data-importing (js-ipfs issue Sprint Oct 26 #41)
    • streamImporter (more questions)
    • [~] BufferImporter (CR)
    • export function
    • [~] browser / mocha tests
  • js-peer-id
    • create dist file
    • document node forge and inability to use browserify in readme
  • Webui
    • write some tests for new components
    • document in contibuting.md

@hackergrrl
Copy link
Contributor

I've been spending time reflecting on my own goals individually and how they connect to my work on IPFS. tl;dr I'm going to be focusing more on js-ipfs and on building p2p primitives to support apps on IPFS. My week reflects this:

I did a lot of reading on p2p subjects (bittorrent, david dias' webrtc work, gossip protocols), and produced several modules: 3 to support ipfs-pad (p2p text editor), and 2 to support p2p pubsub (potentially for haad's orbit-db). Also began spec'cing out the 'ipfs init' process for js-ipfs.

@whyrusleeping
Copy link
Member

These past couple weeks have been spent harding our testing, and fixing issues that have popped
up while doing so. I also in this last week started setting up teamcity to be used as our own
CI so we don't have to wait hours for travis to decide it wants to run our tests
teamcity also gives us awesome metrics on our tests and nice statistics on failures and
failure rates

Other:

  • worked on libp2p
    • updated some dependencies
    • made a few small tweaks that should help with address discovery
    • enabled the use of utp without a utp listener address in config (and visa versa)
  • default values for commands lib
  • introduced 'IPFS_LOW_MEM' flag, doesnt do much yet. but will be worked on
  • lots of test fixing
    • t0060 using port zero
  • repair multireq, still not perfect, but better.
  • improved the UX of building ipfs from source
    • makefile asserts GOPATH, go version
    • uses 0.4.0 gateways for fetching gx deps
    • readme updated to be more helpful
  • fixup ipfs-update to address issues @chriscool discovered
  • a bunch of gx work

@ghost
Copy link

ghost commented Mar 7, 2016

@lgierth sprint update

I just came back from a nice and productive week in Paris. I got to spend lots of time with @cjdelisle and @ansuz at @xwiki-labs, we discussed the state and future of cjdns/fc00, layed out ideas for routing improvements, and drafted spec documents for the switch and cryptoauth layers. I'll post them to the github.com/fc00/specs repo shortly, and we'll continue working on them for the rest of March. I want to start working on Go implementations of the switch and cryptoauth in April. The switch and routing layers of fc00 might be the foundation of a smarter swarm for IPFS/libp2p, so this is all very exciting.

IPFS-wise, the sprint was dominated by hunting down subtle issues. The v04x/v03x multiplexer called multireq still doesn't seem to be 100 % good. I fixed a couple bugs with tremendous help from @whyrusleeping. I've also been debugging issues with fetching big DAGs (e.g. GeoIP). The metrics dashboard at metrics.i.ipfs.io has gotten a facelift too, and will get more detailed metrics on HTTP requests and responses.

@daviddias daviddias mentioned this issue Mar 7, 2016
Merged
@jbenet jbenet removed the sprint label Mar 8, 2016
@RichardLitt RichardLitt mentioned this issue Mar 9, 2016
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@jbenet @hackergrrl @dignifiedquire @RichardLitt @daviddias @whyrusleeping @Kubuxu @nginnever