Saving to local filesystem

[joshlk]

Hi,

Great project! I've been looking for something like this for a while. I would like to setup my own private site and add functionality so I can save and load markdown files to a local filesystem - in my own hackdmd fork (happy to merge if desired). Im a programmer but have not experience in node.js/javascript. How hard would this be? Could you please direct me to the right areas in the code for me to look at.

Thanks

[jackycute]

Hi, Thanks for loving this project!
node.js/javascript are not hard to learn but hard to be good.

Currently, all notes will store in database which is totally different from filesystem.
You can import and export markdown files, we have those features.
But you can't browse all your filesystem or markdown files in some kind of tree view right now.

Because we have to manage permission and user auth problems, the db is necessary.
But we can instead use sqlite to manage that and use filesystem to store notes.

Anyway, these will not be an easy work because these will need ORM the db which I'm working on.
And implement the filesystem tree view etc.

It's really great to hear that you want to help.
You can start from the server side will be easier.
https://github.com/hackmdio/hackmd/tree/master/lib

I might able to help after some other works and issues be done. 😄

[SISheogorath]

I'm not sure how usual and how needed this feature is.

It has a high potential to cause security issues (as always when you provide access to the server side filesystem) and I don't think that there are many users. But maybe I'm wrong, so what do you think?

[ccoenen]

(I am assuming you mean write/read from the server's disk, not from the client's disk. In case, I misunderstood, just skip my comment.)

Writing the notes out to the filesystem is not hard, In case you need pointers, there's a Pull-Request exporting to git: Pull Request #303 / the line starting the full export to disk. Importing might not be as easy, I'm not sure if you need to satisfy any other constraints other than "write to notes table".

To be quite honest, I'm also not yet convinced this requirement is very common.

[markusressel]

I'm currently using MkDocs as a private Wiki.
MkDocs is a static site generator though and they have no intention to implement an online editor for it.
There are almost no alternatives to editing the files directly on the filesystem (there is Pendulum but it has very limited capabilities compared to HackMD).

MkDocs used by many people (over 5k stars on GitHub alone) including teams - which makes the concurrent editing capabilities of HackMD a very interesting thing to have.

I would also like to contribute if my free time allows me to do so.

[ccoenen]

Hi @markusressel, this issue is quite old, and I'm not really sure someone is going to pick it up any time soon. But more to your suggestion: I don't yet fully understand the proposal. I was looking at MkDocs, and it looks like a very similar project, how would the two be combined (if that is, what you're suggesting)?

[markusressel]

MkDocs generates a static website from Markdown files.
When files change it regenerates the website (if configured to do so).
So to edit the content of the website you have to use a file editor. When the markdown file is saved the website content is updated (after a regeneration step of MkDocs).

Because the website is usually hosted on a server it can be cumbersome to edit markdown files and appropriate editors have to be installed on the system you want to edit files from. Using HackMD as a "Webeditor" with many additional functionalities (like the mentioned concurrent editing) would allow to edit the web page content without the need for local tools and as a team without the need to worry about any concurrent editing issues.

[ccoenen]

In that case, take a look at my november comment, there's a ticket mentioned with a git export, and more specifically with the point where stuff is written to disk. I believe this code should still work, or at least be a very close template for a working version.

#90 (comment)

Then, all you'd need is a cronjob (or something else) that regenerates MkDocs from that directory.

[frankgerhardt]

@markusressel if you use it as a private wiki, why not use https://wiki.js.org/ For hosting you can still generate a static site.

[davidak]

I think for editing local files a native text editor with markdown preview extension would be best.

The Atom editor is not really native (based on web technology), but has a markdown preview integrated as core plugin. SublimeText probably also has a markdown preview plugin, but that editor is not open source.

[ccoenen]

I think we're getting farther away from HackMD.

Native Text Editor + Local File System (+ by extension no collaboration) is pretty much the polar opposite of what HackMD is right now. I don't think we want to go there. There are other projects that have solved this problem (like the mentioned code editors, but also specialised things like markdownpad).

[davidak]

Why i have subscribed to this issue is that i have all my important notes in HackMD right now and want a feature to download all as single files in a compressed archive.

[ccoenen]

@davidak that is not a bad idea, but I think this ticket is not the place to get this done. In fact, to be honest, I think this ticket should probably be closed. There's at least three very different things mixed together, the original post is two years old. This is not getting anyone what they wanted.

Please feel free to create a new issue for your suggestion (download all my notes at once). I guess this is somewhat related to #314 and #653. User support is not very far developed at the moment, so please be aware that this might take a while (and, as always: contributions are welcome!)

[SISheogorath]

@davidak If you are interested in investing around 1 1/2 hour to write such a feature, I recommend you to have a look at:

https://stackoverflow.com/a/25210806

https://github.com/hackmdio/hackmd/blob/master/lib/web/userRouter.js (depending on how you think you want to implement it)

https://github.com/hackmdio/hackmd/blob/master/lib/history.js

This should provide all needed knowledge to write such a feature :) If you have questions, feel free to join the matrix (and Gitter) community ask :) Pull Requests are welcome!

[davidak]

@SISheogorath thanks. I don't will be able to do that in the next weeks. Maybe someone else wants to do that.

[pirate]

Some people use HackMD not for the collaboration, but just for the great Markdown editor & pretty publishing/export features. I myself run a few blogs using embedded published HackMD notes, but never touch the collaboration features other than to set every note to "Locked". If there were a HackMD CLI to import a markdown file to create a note, and to produce static HTML exports (just like the export right now), people could go on to build many other tools on top of HackMD. This would I think solve many requests that you guys have been getting in PRs as well, because people could write their own script solutions to things like dropbox sync, static site generation, backups, etc.

I think just even these two commands would be enough to implement tons of other fun stuff on top of HackMD:

$ hackmd import note.md
GwQwZgTCCskLQGYDsExwCwE4DGAGOm00AjAeMAEYjHCzGZA   (outputs id of created note)

$ hackmd export --format "md" GwQwZgTCCskLQGYDsExwCwE4DGAGOm00AjAeMAEYjHCzGZA > note.md
$ hackmd export --format "pdf" GwQwZgTCCskLQGYDsExwCwE4DGAGOm00AjAeMAEYjHCzGZA > note.pdf
$ hackmd export --format "html" GwQwZgTCCskLQGYDsExwCwE4DGAGOm00AjAeMAEYjHCzGZA > note.html

hackmdio/bin/hackmd:

#!/usr/bin/env node

if (process.argv[1] == 'import') {
    // ... JS code here needed to create a note
} else if (process.argv[1] == 'export') {
    // ... JS code here needed to create exports
}

How hard would it be to write a node CLI to generate these files? (using the same db as whatever hackmd server is already running in the same folder)
If someone can tell me what files I need to look in to find this code, I can probably write this myself.

(Support for continuous import/export/sync isn't needed yet, just single-shot commands to produce these files would be enough to put into cron as a backup tool or generate a static blog).

[ccoenen]

you can use curl to post a document to a url. This only works when creating that note, though.

Here's the pull request, it also includes an example how to pull this off :-)
#673

[SISheogorath]

it's oft requested is also already possible using curl or wget. Yes, someone may want to write a wrapper script either for curl in shell or completely in node.

wget -O mynote.pdf https://<yourhackmd>/<noteid | shortid>/pdf
wget -O mynote.md https://<yourhackmd>/<noteid | shortid>/markdown

If you want to be able to export as HTML directly, you can provide a PR by modifying this code:
https://github.com/hackmdio/hackmd/blob/763479bea8603b9bd18133b0534314edebc50777/lib/response.js#L187-L245

(basically write a new template that removes all the HackMD UI stuff and only leave the content or content + dependencies)

[pirate]

I think I might be able to use the publish url to get the plain html instead of adding a new route. Does this interface & code look reasonable to you guys?

./hackmd [import|export|publish] ...:

#!/usr/bin/env bash

help_str="
Usage:
    $ ./hackmd import test.md
    qhmNmwmxSmK1H2oJmkKBQQ       # returns note id on success
    
    $ ./hackmd publish qhmNmwmxSmK1H2oJmkKBQQ
    /s/S1ok9no3f                 # returns publish url
    
    $ ./hackmd export --pdf qhmNmwmxSmK1H2oJmkKBQQ my_note.pdf
    $ ./hackmd export --md qhmNmwmxSmK1H2oJmkKBQQ my_note.md
    $ ./hackmd export --html qhmNmwmxSmK1H2oJmkKBQQ my_note.html

    $ env HACKMD_SERVER='https://hackmd.example.com' ./hackmd import test.md
"
...

See my full source proposal here: #808

Update: this now exists here codimd-cli

[ccoenen]

I'm closing this ticket, while I understand that there is some need for this, I also think everyone looks at this ticket a little differently and with different needs. Special Thanks to @pirate for #808, this is a very useful thing!