Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement data acquisition #25

Closed
1 of 3 tasks
amis92 opened this issue Feb 8, 2022 · 3 comments
Closed
1 of 3 tasks

Implement data acquisition #25

amis92 opened this issue Feb 8, 2022 · 3 comments
Assignees

Comments

@amis92
Copy link
Member

amis92 commented Feb 8, 2022

Enable browsing BSData Gallery and downloading files from there.

For MVP here, we're aiming at:

  • downloading the Gallery JSON index: bsdata.catpkg-gallery.json form https://github.com/BSData/gallery/releases/tag/index-v1
  • showing a game system (repository) selection UI for a new roster - keep ability to select Sample Dataset
  • try to match the gamesystem ID with Gallery JSON's item (repository) when opening a roster.

BSData Gallery is a nuget.org/npm.org like Registry/Gallery that indexes gamesystems. It provides a static index asset.

Datafiles are available from GitHub data repositories via GitHub release assets. Sadly, those assets do not have CORS headers set, so that downloading them in a browser won't work.

Solutions I can see:

  1. Make Gallery "cache" the actual assets in a branch published to GitHub Pages. Content published there is CORS-enabled and can be freely downloaded in a browser.
  2. Create and use our own CORS-proxy. One thing I've researched is a Cloudflare-cors-proxy, using Cloudflare Workers.
@drobertson35
Copy link

Needed functionality:

  • browse list of game systems and repositories (CORS issue)
  • download data files (CORS issue)
  • store/manage metadata in local storage
  • (adv. feature) allow users to sync data across devices

drobertson35 added a commit that referenced this issue Feb 15, 2022
@amis92 amis92 moved this to In Progress in Phalanx Feb 15, 2022
@amis92 amis92 linked a pull request Feb 15, 2022 that will close this issue
@nstephenh
Copy link
Contributor

nstephenh commented Aug 26, 2022

I believe we can use the same proxy that godmode uses: https://cors.isomorphic-git.org/github.com/Bsdata/horus-heresy/info/refs?service=git-upload-pack
As explained by amis, we do not want to use a git proxy since we want to pull individual files from gallery, and the godmod proxy is intended only for "git clone".

@amis92
Copy link
Member Author

amis92 commented Aug 30, 2022

Cloudflare proxy is setup at https://phalanx-cors.bsdata.workers.dev

To get a GitHub URL with CORS headers, pass the full GH URL in a search parameter: https://phalanx-cors.bsdata.workers.dev/?https://github.com/thing/you/want

PS It's only enabled for the phalanx.tools origin.

@nstephenh nstephenh mentioned this issue Sep 28, 2022
3 tasks
Repository owner moved this from In Progress to Done in Phalanx Oct 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants