Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make execute-changes script more generally usable #21

Closed
1 of 4 tasks
flyingzumwalt opened this issue May 7, 2017 · 4 comments
Closed
1 of 4 tasks

Make execute-changes script more generally usable #21

flyingzumwalt opened this issue May 7, 2017 · 4 comments
Labels
snapshots issues related to snapshot creation and updates

Comments

@flyingzumwalt
Copy link
Contributor

flyingzumwalt commented May 7, 2017

Make the script from #18, which is in https://github.com/ipfs/distributed-wikipedia-mirror/blob/master/execute-changes.sh more generally usable. This is needed in order to fulfill #14 and in order to make it easier for other people to generate their own snapshots.

As a hacker who wants to add new wikipedia snapshots to IPFS in the language of my choice, I should be able to follow a clear set of instructions that allow me to download a zim dump, add it to IPFS, modify it with this script and then publish the resulting hash. The instructions should be clear, the configuration should be simple and it should be easy for me to set the correct IPNS hash and snapshot date based on the language version I'm adding.

Completion Requirements

  • the shell script works with minimal pre-configuration of your system
  • the shell script's documentation clearly declares what you need to do in order to run it
  • the shell script makes it easy or completely transparent to set the correct IPNS hash and snapshot date based on the current date and the language of the current snapshot
  • the readme at the root of this repo, or a page it links to, contains complete and accurate instructions for using this script when you're creating a snapshot
@flyingzumwalt
Copy link
Contributor Author

@Kubuxu cleaned up the script a bunch, making it much more general and adding command line args with help text. Still needs documentation on how to use it.

@Kubuxu
Copy link
Member

Kubuxu commented May 12, 2017

How to use the execute-changes.sh script:

  1. extract your dump as in instruction
  2. add it and note the hash as in instruction
  3. link the dump in Virtual IPFS Files API ipfs files cp /ipfs/$DUMP_HASH /wiki-zn
  4. Execute the execute-changes.sh:
bash execute-changes.sh /wiki-zn

The execute-changes has some options like:

  • --ipns $IPNS to add the ipns link of the dump
  • --search $SEARCH_CID to add search function, it needs generated search structure CID
  • --date $SOME_DATE to change the date of the snapshot creation
  • --main $MAIN_PAGE full name of article containing intro page (e.g. Main_Page.html as in english wiki) - defaults to index.htm as in many Kiwix dumps.

@lidel lidel added the snapshots issues related to snapshot creation and updates label Sep 9, 2019
@lidel
Copy link
Member

lidel commented Jan 22, 2020

Update from #60: tried to use execute-changes against unpacked wikipedia_tr_all_maxi_2019-12.zim (~4GB) but without success. JS and the directory structure changed so much, that entire execute-changes.sh needs to be redone.

@Kubuxu Kubuxu removed their assignment Jan 22, 2020
@lidel
Copy link
Member

lidel commented Feb 7, 2020

Continued in #64

@lidel lidel closed this as completed Feb 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
snapshots issues related to snapshot creation and updates
Projects
None yet
Development

No branches or pull requests

3 participants