Serve Wikipedia from NYC Mesh nodes

[simplymathematics]

Hey. I work with NYC Mesh github.com/nycmeshnet. We want to serve wikipedia from every node, running ipfs. We also have a high bandwidth ixp back haul. Let's talk.

[jbenet]

@meyerscr sure, what do you need? how can we help?

[ghost]

There's two things you can do to get started:

1. we don't have wikipedia mirrored on IPFS just yet, check out Wikipedia ipfs-inactive/archives#20
2. start running IPFS within the nycmesh network with an empty bootstrap list (ipfs bootstrap rm --all) and see whether MDNS discovery works on the nycmesh network (ipfs swarm peers).

[simplymathematics]

MDNS does work via NYC Mesh because we have a VLAN even if there is not a Layer 2 connection (we use TINC if we have to 'mesh' over layer 3).

I had more technical questions about the actual deployment. For now, we're just going to add our wiki to the mesh.

Is it as simple as "ipfs add /www" since mediawiki generates links that are always internal? What about the config settings--I don't want to have to version control who is an admin. Also, will IPNS be give me an IP address to interface with so that I can send people to the ipfs wiki using a URL?

I'm pretty new to your software and there isn't anyone around with expertise, so I'm quite grateful for your help on this matter. I work with the Technology committee for Wikimedia-NYC. So, if we get this working on a small wiki, I can make a case to putting a mirror of the the entirety of wikimedia commons on our network. Since we have a datacenter, we can even use our copy as the 'official' ipfs copy, running on a building with 100Gb/s fiber lines running to London et al.

[ghost]

Oh I see, thanks for explaining, and wow that sounds great, especially for nyc mesh! Happy to help, @jbenet and I met a few other folks of NYC Mesh at d.b.a. in april :)

You can't exactly put the wikimedia application on ipfs. Data in ipfs is always static (content-addressed with hashes), and is made dynamic by updating a pointer to a new version of the data (IPNS). What you can do is

1. quickest way to useful results: make a readonly static version of it, and ipfs add -r static_page/ -- this will turn out at a couple TB I think. You can keep this mirror up-to-date and announce the IPNS path somewhere. People can fetch it through ipfs itself or use the http-to-ipfs gateway.
2. more complex, but waaay more exciting: add the raw data for all pages to IPFS, then build a javascript app which fetches data from IPFS and uses that. You can even have static, incrementally-updated search indices in IPFS.

[ghost]

And btw, if you get this working, I'd be very interested to replicate your setup within the Freifunk network in berlin.

[simplymathematics]

I think I already have the application. Outernet built a browser-side content library, called librarian. It's very late here--I'll send you the link tomorrow.

[simplymathematics]

https://librarian.outernet.is/en/files/

I think this might help?

[simplymathematics]

Oh nevermind. Y'all already have a server side content browser.