Upload WARC from the web UI to index

[ibnesayeed]

It will be convenient if admins can run the server and then drag and drop (or use the file selector dialog) to upload WARC files that get indexed automatically. However, this functionality should only be available to admins and should be included only after admin related controls are separated from the landing page as per #213. Uploaded WARCs can also be organized in collections as per #82.

[machawk1]

This will create an interesting dynamic between the replay system and indexer. I would be interested in this being implemented without #213 necessarily being a dependency. However, I do agree that the end-goal will be to consider the setup where the replay system and client are not co-located.

Some further tie-in with warcio and decoupling from pywb might help facilitate this functionality as well.

[ibnesayeed]

Without isolating and protecting admin functionalities from the public view, the system is only usable by one-person. For example, the publicly accessible test deployment we have, anyone can shut the IPFS daemon down in it and leave others wondering what happened in the middle of their session.

As far as this feature is concerned, I think it will be really very convenient for people to create WARC files using, say, WARCreate and immediately upload it into their IPWB system to see it replayed.

[machawk1]

@ibnesayeed Do you have ideas as to how this functionality should be displayed within the ipwb replay UI?

[ibnesayeed]

Yes, I have some UI ideas, but we can perhaps discus those offline first and draw some diagrams on whiteboard.

[machawk1]

@ibnesayeed Would you mind describing some of your UI ideas here?

[ibnesayeed]

Oh, this was something that we wanted to discuss in person, but could not recall what it was. There are a few ways we can handlel the UI, but I think it is important to first segregate admin functionality from the landing page then we can talk about where to put it.

[machawk1]

I think hiding the admin UI from client and adding the ability for clients to upload WARCs can be investigated in parallel. I am considering putting the functionality in-place in the replay backend then we can tie it in visually once we have the functional parts ready.

[ibnesayeed]

Sure, in that case we can forget about the drag and drop feature for now and simply add a file upload input anywhere to test the backend implementation.