Table "hidden" attribute flag not exported

[Indigo744]

Hello,

I identified a new small issue: when exporting a table, the "hidden" flag attribute is not exported as well. Hence when building from source, the table is created but without the flag set.

This attribute can be set on a table property (right click on it, then Property). Here in French, it says Masqué (hidden).

Here is an example before/after building from source:

I'm not sure how to tackle this issue since there is nowhere to put this information easily. I can only see a .sql and a .xml file created. There is no JSON file that could contain this property.

What do you think?

[joyfullservice]

Would that be stored with the Document Properties? I am not sure how that may differ from the table attributes property found here. I have also seen reference to deep hidden tables, so that might be another thing to consider. If we do find important metadata that is not being saved anywhere else, we could consider adding a tableName.json file to store that content.

[Indigo744]

Sadly, looping the table property is not enough: the "hidden" attribute is nowhere to be found.

Name Table1
Owner admin
UserName admin
Permissions 1048319
AllPermissions 1048319
Container Tables
DateCreated 14/11/2020 09:50:08
LastUpdated 14/11/2020 09:50:20
PublishToWeb 1
Orientation 0
OrderByOn Faux
NameMap ??      
DefaultView 2
GUID ????????
DisplayViewsOnSharePointSite 1
TotalsRow Faux
FilterOnLoad Faux
OrderByOnLoad Vrai
HideNewField Faux
BackTint 100
BackShade 100
ThemeFontIndex 1
AlternateBackThemeColorIndex 1
AlternateBackTint 100
AlternateBackShade 95
ReadOnlyWhenDisconnected Faux
DatasheetGridlinesThemeColorIndex 3
DatasheetForeThemeColorIndex 0
Description Test

As a side note, looking at this, I wonder why we do not export these properties? Most of them could be modified in the property tab when editing the table :

Going back to the hidden attribute, I think we could use the function Application.GetHiddenAttribute(type, name). But should we do this in Document properties with a special case? Or in another class?

[joyfullservice]

Let me jump in and take a look at this issue...

[joyfullservice]

Okay, after looking into this in more detail, it looks like the HiddenAttribute is kind of a unique independent attribute attached to various object types. As you noted, it is not contained in the typical properties collection, or the document properties collection, which is where I would have expected to find it.

Adding this to the documents.json file is certainly one approach we could take, as you demonstrated in your pull request. I would have a couple hesitations with this, however. First, it technically really isn't a document property, so this could be be a little counter-intuitive and blurs the lines of clean segregation based on the object model. Second, (and somewhat related) is that it requires a second custom loop to parse the HiddenAttribute section of the documents.json file, independent of the actual document properties.

The direction that I would be leaning towards is adding this property to a companion .json file for the table. We already create the .json file for linked tables an several other types of objects to handle metadata that is not included in the normal export files, so it feels like a very consistent and logical place to put it. The .json file would just contain another entry for the HiddenAttribute as shown in the screenshot below:

I am not super excited about the idea of creating piles of .json files that only contain a single attribute, but my thought would be to only add the attribute (or create the file) if the hidden flag was set to true. For the majority of use cases the hidden attribute would not be used all that much, but where it was, this would allow VCS to fully support it.

A second reason that I would lean towards the companion .json file is based on some discussions in #81 where we may in the future be adding some optional tighter integration with .git. Depending on how this is implemented, we may be using the companion .json files to store the git head version or other metadata that could be used to drastically improve performance on very large databases, and support a Merge Build concept where a handful of changed objects are integrated into an existing database rather than building the whole thing from scratch each time. There is also the possibility that we will in the future uncover other types of attributes that are not contained in the export files.

A third option to consider would be a separate hidden-attributes.json file and corresponding class that exclusively focuses on exporting and importing this stand-alone attribute. In a way that might be a bit more encapsulated than adding the HiddenAttribute logic to the classes of every type of object shown in the object browser.

Those are my initial thoughts, but I would welcome additional ideas or input from you as well!

[joyfullservice]

In regard to the additional table properties, I believe the answer there will be to change our XML table structure export to use acExportAllTableAndFieldProperties which will pick up those additional properties.

This is mainly pending some updates to the XML sanitization function to strip out elements like GUID and NameMap. I know there are very definitely some important things that are not being saved in our current export format, so that will be a helpful enhancement when someone has a chance to work on it. 😄

[Indigo744]

You are right, as usual 😺

If I sum up your proposals:

1. Add a HiddenAttribute value to a companion JSON file for every object.
2. Add a new class to export to export a new JSON root file (similar to document.json).

The issue with the first approach is that we will need to add code and JSON export for all objects that can be hidden: forms, tables, macros, reports, modules. So that could represents a lot of work since some object type are not using any JSON file.

The advantage with the second approach is that it is similar to my current PR, but inside another file. I believe implementation would be faster.

[joyfullservice]

Yes, maybe let's go with the hidden-attributes.json file and clsDbHiddenAttribute for now, and we can always revisit the topic later when we look at the .git integration down the road. 👍

[Indigo744]

Alright, I'll start working on it and provide a PR 😉

As for the other properties, we could simply remove the filtering with a blacklist approach just like above. See clsDbDocument.bas#L152

[Indigo744]

Thank you for correcting my code 😺

And I want to thank you for all your work and for being so understanding. I love this addin and it makes my life so much easier...! 😄

[joyfullservice]

Great work on the class for the hidden attributes! I made a small change to simplify the display in the json file. You are doing a great job, and I really appreciate your contributions and interest in this project! 😄

[Indigo744]

I like contributing and giving back when I have time (and when I know how!). This is open-source at its best, people from a wide horizons working together on a useful tool for them 😃 It just feels good, and I must say especially during these trying times! So thank you and be sure I'll continue to contribute if I can!