Support for empty directories (v0.1 feedback)

[justinlittman]

In response to @awoods request for comments, here is some feedback for your consideration:

• In 3.3, describe .keep. I'd suggest based on my experience as a BagIt author, that .keep is not an optimal approach for dealing with empty directories, as it forces content creators to change content in order to be able to store it. (Suggesting its use is one of my regrets about BagIt.)
• In 3.3, why would there be empty content directories?
• In 3.4, explain how it allows for content addressable storage. --> Digest suggestions from v0.1 #273
• In 3.4, I would not make the claim about protecting against malicious actors. The protections provided by OCFL are (understandably) weak. --> Digest suggestions from v0.1 #273
• In 3.5.3, describe the significance of the user field. --> Describe significance of user field #274
• In example 5.1, why is there a V2? What doesn't V1 have a type? --> Minimal object example problem #265

Based on scrutinizing example 5.2, I find the use of file digests and filepaths to reference content inconsistent and confusing. In fixity, filepaths are used to reference content. In manifests, file digests are mapped to filepaths. In version state, file digests are again mapped to filepaths. It seems that either (1) manifests and file digests should be removed or (2) file digests should be used to reference content in the fixity section. --> #275

I look forward to this important specification moving forward.

[zimeon]

Thanks for your feedback @justinlittman !

Regarding your first two bullets -- would you do something else to support preservation of empty directories, or just not support them? I also feel awkward about the suggestion of .keep and would be just as happy to remove the sentence suggesting it from the normative specification (could stay in implementation notes).

Regarding your last bullet about example 5.1 -- that error was fixed in #265

[justinlittman]

I'd suggest explicitly encoding empty directories in the inventories.json rather than recommending .keep. Like I said, no one was that happy with that approach with BagIt and we regularly heard it as a complaint.

[ahankinson]

I think the main issue about empty directories is that there is no way to hash them, and thus no way of storing them in the manifest.

[justinlittman]

Unless you add a separate section (presumably just an array) for the empty directories, which would be in addition to the manifest.

[zimeon]

I agree with @justinlittman that we could add an extra feature to the inventory to explicitly support empty directories, the question is whether there is enough call for that. I lean toward not providing explicit support. Certainly at Cornell we have no interest in the facility, where we want to maintain filesystem details (rather than content) we will use disk image formats

[neilsjefferies]

So many other filesystem details drop off when moving between filesystems (attributes/access control etc.) that I would go far enough to say that allowing empty directories encourages a bad practice use case.

[awoods]

Speaking of use cases, it would be helpful to have use cases to inform the "empty directory" scenario.

[zimeon]

Agreement in 2018-11-21 editors' meeting that we will not add extra support for empty directories and we will explore a change of language to make the use of .keep sound less prescriptive while still including it as a possibility,

[zimeon]

Current text in Version Directories

Every file within a version's content directory MUST be referenced in the manifest section of the inventory. There MUST NOT be empty directories within a version's content directory. Directories that would otherwise be empty MAY be maintained by creating a .keep file within the directory.

perhaps keep the first two sentences and change the last sentence to:

A directory that would otherwise be empty MAY be maintained by creating file within it according to local conventions, for example by making an empty .keep file.

[ahankinson]

According to local conventions -> named according to local conventions ?

[zimeon]

2018-12-05 editors' meeting: @zimeon to do PR implementing combination of last two comments