Shouldn't retype build empty the target?

[DannyBen]

As I see it, the target folder is only under the responsibility of Retype. Meaning, users should never place anything manually inside it.

If this is indeed the case, I see no reason why the retype build command shouldn't erase the contents of the folder prior to rebuilding.

During development, I change my mind about file names and locations, and just running retype build leaves me with the old names as well. I ended up always doing:

$ rm -rfv docs/* ; retype build

If my assumption is incorrect, then perhaps a flag? retype build --reset?
I would prefer it being the default (and only) behavior, but can understand arguments against it.

[fabriciomurta]

Maybe a retype clean command?

And this is something we all thought about while developing retype... I believe at some point we really had it... But the thought of having Retype accidentally wipe somebody's precious files before they could backup, should they e.g. be learning Retype and point it somewhere dangerous kept us from implementing this step by default. Besides, left residue shouldn't affect the actual output. There will be "dangling" files, but they won't affect how a retype deploy works. So it simply looked the safest bet.

So personally, I wouldn't vow for "wipe-by-default". But having an explicit command to do so should be better for all cases.

I mean, this would work great and safe for somebody learning retype, and the separate command would make the experienced have an easier time refresh the output. This retype clean command is inspired of similar tools like... well, dotnet, which inspired Retype's "CLI" aspect to some extent. So in this respect, a retype rebuild could be the one to merge the two commands into one, clean+build.

[AndreyChechel]

One of the options we were considering is to persist paths of all files written by Retype to the output folder. This way Retype would be able to delete everything that it has created, not more.

[DannyBen]

Maybe a retype clean command?

Hmm. I am not a fan of this option.

And no offense to Microsoft, but I wouldn't take their command line standards as an indication to anything. On that note, for instance, it is highly unorthodox to send a clear command to the screen before showing your output - as the retype CLI is doing.

I see no point in retype clean - I can type an OS-native command for that. The only reason I wanted retype to do that, is as an integrated part in the watch-and-build cycle. It should in fact be (in my view): watch-reset-build.

If I remember correctly, mdbook is working like this.

But the thought of having Retype accidentally wipe somebody's precious files

I can understand that. So, the only thing that solves all concerns on the subject so far is:

# These are safe to run - never removes anything
$ retype build
$ retype watch

# These reset the folder prior to building
$ retype build --reset
$ retype watch --reset

One of the options we were considering is to persist paths of all files written

While this is a nice, and responsible idea, I believe this is equivalent to the Americans inventing an expensive pen that writes in space, and the Russians take a pencil... :) - if there is a simple solution, no need to add another thing in the system that can break. Then you end up with cache or some other "mystery" files in the target folder that are just there to serve some purpose that nobody remembers.

To me, simplicity is at the top of my list of things to consider before developing anything.

[geoffreymcgill]

We've gone back and forth on various options over time. Ultimately, we (mostly I) do not want Retype to be responsible for deleting ANYTHING, unless we know with absolute 100% certainty that Retype originally created that file.

The big scary scenario is someone accidentally configuring the output directory with an unintentional path, and then Retype nuking their hard drive.

A quick look at the mdBook docs and I see they have a separate mdbook clean command. Maybe that clean process happens automatically with other commands, but I don't know their API well.

Obviously, it would be trivial for us to add a similar retype clean command, or an optional argument, or just automatically handle deleting the contents of the output directory, but again, I don't want to go there just yet. My preference is to protect you from yourself. Accidents happen.

This self-cleaning feature is actually directly related to another huge new feature we are working on right now; an incremental build page dependency graph. The dependency graph refines the process of rebuilding only those pages that have been updated as well as rebuilding any dependent pages of those pages. For instance, when Page A refers a Page B and the Previous|Next buttons at the bottom of the page need to reflect the new chain.

The new dependency graph is working and it's amazing and I'm anticipating that we will use the same tech to track generated files to allow for safe deletion.

BTW, if you are using the Retype GitHub Action, the entire contents of the output are removed and regenerated with each run of the workflow, so you will never have dangler files. The danglers would only happen locally when you're developing (at least for now).

Hope this helps.

[geoffreymcgill]

Just thought of an idea, maybe we could add some meta data to each file Retype writes to the output location. Then when deleting the contents of the output directory on the next rebuild, we could just check for that meta data to confirm deletion.

A hash of the file might work well here.

Interesting...

[DannyBen]

Just thought of an idea,

You mean as part of the target file's source? Or as a separate single file that contains a list of all files, like mentioned by @AndreyChechel ?

[DannyBen]

A quick look at the mdBook docs and I see they have a separate mdbook clean command. Maybe that clean process happens automatically with other commands, but I don't know their API well.

1. Having a retype clean command is not a bad idea, but it is not a replacement for retype build cleaning the no longer needed files, in my opinion.
2. I can confirm that mdbook build removes things from the target folder once you remove them from your source folder and from the SUMMARY.md file (mdbook 0.4.9).

[geoffreymcgill]

I can confirm that mdbook build removes things from the target folder once you remove them from your source folder and from the SUMMARY.md file (mdbook 0.4.9).

Nice. Thanks for the confirmation.

I have added the topic to discuss during our Monday team meeting. We will try and come up with a solution for Retype.

[DannyBen]

... to discuss during our Monday team meeting

With all the times you and the team spent replying to me, I feel like I should be in that meeting... :)

[geoffreymcgill]

We have decided that Retype will delete files within the output location, but only files that were originally created and are being tracked by Retype. This should keep the build output folder clean.