Allow defining what importer to use for a given file extension

[akien-mga]

Describe the project you are working on

Godot itself, and trying to improve the usability.

Describe the problem or limitation you are having in your project

Godot imports some assets using ResourceImporters, which recognize which file types they handle based on file extensions.

There are several cases where a given extension might be supported by several importers for different outcomes:

(Texture makes sense as the default as it's the most common use case, but if for some reason your game processes everything from Image, you might want to change that default.)

(.obj files can be imported as mesh or as scene. In theory, they could also be completely unrelated things like C/C++ compilation objects from MSVC, in which case they would be excluded with "Keep" or a .gitignore file.)

(The "fake" CSV importer is being removed in godotengine/godot#47301, but this is still a typical situation where you might want to define if you want your CSV files to be imported as translations or kept as is by default.)

Describe the feature / enhancement and how it helps to overcome the problem or limitation

There should be a way to specify, for a given extension (e.g. png, obj, csv), what should be the default ResourceImporter used to import any newly added asset with this extension.

This should likely be a project setting, and when changed, it would not modify already imported assets, but it would affect any asset imported in the future. The "Import" dock would still allow customizing the behavior and changing what importer to use for given files.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

godotengine/godot#46354 added an "Importer Defaults" editor tab in the Project Settings:

This allows customizing the behavior of ResourceImporter themselves.

I suggest splitting this dialog in two, adding a section which would allow selecting the default importer for a given file extension.
This section would populate the list of recognized extensions by querying the engine for registered importers and the extensions they handle, and for each extension, there would be a drop-down list that lets you select a valid importer OR the "Keep" option.

Quick mockup (proper implementation should likely use SectionedEditorInspector or similar):

If this enhancement will not be used often, can it be worked around with a few lines of script?

Not really, this needs to be a configuration that affects what happens even before a given file is imported when scanned by the FileSystem dock.

Is there a reason why this should be core and not an add-on in the asset library?

This is part of the core configuration of any project just like the new "Importer Defaults". It could possibly be done via an add-on that re-imports newly detected files with a chosen importer, but that would be quite hacky and inefficient.

[golddotasksquestions]

Thanks you so much for this work, importing and reimporting is still on of the most confusing parts of Godot for me. Separating the defaults into the Project settings and the individual - per file - changes is a huge improvement. The drop-down-list for filetypes equally so.

What I am still find very confusing is the naming. As a beginner and still after months and years of using the engine, I would not know the difference between a "Texture", "Texture3D", "TextureAtlas", "TextureArray", "Image" ... these all sound very similar to me still even though now after two years I at least have a vague idea about when to use what for maybe two of these.

Here are some of the questions going through my head here from when I was starting out using Godot until now:

• "Can I not use a something imported as "Texture" in 3D? Do I need to reimport is as "Texture3D" if I want to use it in 3D?"
• "Is "Texture3D" the same as "Texture", just with additional things for a special usecase in 3D I don't know about?"
• "I seem to be able to use files imported as "Textures" in 3D, why do I need "Textures3D" at all? Especially since it just changes the import panel settings according to documentation, or does it something else to my imported image I don't get? The API documentation on Texture3D is sparse to say the least. but suggests there is more going on."
• "What's the difference between an "Image" and a "Texture"?"
• "Texture sounds more like something used by a 3D artist, does this mean I need to import as "Image" since I am making a 2D game?"
• "I need an image texture for my 2D sprite, do I chose "Image" now or "Texture"? A 2D artist thinks of what is displayed on screen as an Image, a texture is the haptic description of a surface of something that is depicted, like "the rough texture of a tree bark" on an image of a tree."
• "The API documentation says "Image contains image data" which sounds like "Apple" contains Apple stuff". "Image" can be converted to Texture but why and how and when and where would I use either?"
• "Is "Image" best for 2D sprites and "Texture" best for 3D sprites and 3D mesh graphics?"
• "I know Bitmap as rasterized image data compared to vector image data. Isn't therefore every "Image" and every "Texture" not also Bitmap data in Godot?"
• "Wikipedia describes Bitmap as "bit array", how does that compare to "TextureArray"? Is a TextureArray not also a bit array?"
• "Does importing as "Bitmap" mean my image is converted to binary black and white information per pixel?"
• "I imagine a Texture Atlas or a Texture Array to be many Textures imported at once, but how are they different? Do I need to select multiple files in order for either setting to make any affect, or is this supposed to be used on single files that have multiple images in them like collages and automatically split those apart into an Array or Atlas of Images?"

The documentation does not really answer any of these questions. I'm actually more confused reading this than before, and feel less secure my chosen import type.

The difference between Import as "CSV", "CSV Translation", "Keep File (No Import)" to me is also not very clear at all:

• "What does it mean to import a CSV file "as CSV" Isn't that what a CSV file already is: a CSV file?
• "Is "CSV Translation" the same as "CSV" plus also a Translation part? This setting looks like I could also use it as regular data CSV."
• "What does it mean to "Keep" the file in comparison to the "CSV" setting, presuming "Keep File (No Import)" means the file is unchanged. Would this not mean the CSV stays a CSV, so just as in "CSV"?
• "If I only want to use the CSV as data, does this mean it's irrelevant which import setting I chose?"

"Keep File (No Import)" in general is a very unfortunate naming choice imho. Let's assume I would understand what it does (which I don't), and I would select a file in the FileSystem, then go to the Import panel to check and option called "No import" and then I click "reimport" to apply. This makes no sense to me whatsoever. I also wonder "who or what is keeping the file"?

I think a lot of this confusion can be helped with well written informative tooltips also stating example usecases and better documentation that explains the differences in more detail. Currently there are no tooltips at on any of these options.
I think double naming like "CSV" and "CSV Translation", "Texture" and "Texture3D" should be avoided (if this difference is purely in preselected settings and does not refer to classes)
I think differentiating or explicitly stating if any of these "Import As" settings are referring to specific class names, data types, or if they are just preselected options would also be of great help. Maybe this could be stated in the tooltip, and the class name would be clickable if applicable. So if one clicks on the class name in the tooltip they class reference in the script panel appears.
I also think "Keep File (No Import)" also really needs to be renamed into something more clear. Or maybe has to be a separate option next to reimport, like: "Undo Import" (with a tool tip saying something like: "removes .import files from the project and clears import metadata, but keeps raw file data in the project folder. Go to FileISystem and "delete file" to delete both import data and raw file data") - But I don't really know what it actually does, so I'm just assuming here.

[akien-mga]

@golddotasksquestions Your questions on texture import presets are valid but not really related to this proposal, they would be better discussed in an issue on the documentation repository. All those presets are different resources with different use cases - Texture3D is not a texture to use in 3D, it's a "3D texture" (i.e. a textured volume, like a stack of images), which has very specific use cases. See class reference documentation for TextureLayered (base class), Texture3D and TextureArray. They should definitely be better documented but the rule of thumb is that if you don't know that they're for, you probably don't need them. If you need them, you're using advanced rendering techniques and know what to look for (e.g. sampling texture arrays from a shader to make an object textured when you cut chunks of it, like eating a cake with a spoon :)). For images, the default import format is Texture and everything else is for advanced use cases - remember that images are data types and they're used a lot to encode data in a GPU-friendly format, not just to put 2D graphics on the screen. Beginners don't need to change those defaults, only the actual configuration of the Texture importer (filtering, repeat, etc.).

The usability can definitely be improved and adding tooltips that describe what each import preset is for is a very good idea, but only tangentially related to my proposal which is about allowing to define what preset to use by default for a given extension (and only that - it's not about changing the import system, only making the default configuration easier). A proposal to add tooltips on hover is a good idea IMO.

As for CSV, as I mention in the original issue the "CSV" import format is a hack and it has now been removed in 3.3 and 4.0. It was only added to circumvent the fact that it was not possible not to import a CSV file, which made it impossible to use CSV files for something else than translations. The new "Keep" preset prevents importing the CSV file and lets you use it as raw data.

[golddotasksquestions]

Thanks for taking the time to respond! I sorry, I must has misunderstood the purpose for proposal as usability improvement.
I was of course not writing all this to ask for better documentation (which I agree would be necessary, but I also think this should be discussed in the documentation repository)

My point was to illustrate from my own experience how beginner and intermediate users feel when confronted with these options.

My point also was to express that we really need some sort of tooltip or some other extended interface at this exact location (the import tab) in the user flow, that gives more contextual information.

You are saying "If you don't know what it means if should not concern you". But this is clearly not how this works: As a beginner I am confronted with a list of what seem equal options to me. None of these options have "(advanced)" written next to them or have any other indication at which point I am supposed to care about them, and at which point I can safely disregard them.

As a beginner, you start most likely with 2D. More often than not this 2D project is a pixel art project. This means one of the first things many Godot beginner users will be confronted with is their blurry pixel art that does not look like pixelart at all. The QAs, community ect will tell them to change the defaults and reimport without the filter enabled. Therefore my first experience is that the default options are not what I need.

This is an extremely common scenario, one of many where beginners have to struggle with the import settings and change the defaults. For you and other experienced users it may seem obvious what can or cannot be ignored. But by listing what was going through my head in the above comment, I was trying to make the point that it is not obvious at all to beginners and even slightly more intermediate users. Even user who have a background in graphics as I have.

If we had a clearer interface at this exact point in the user flow (for example with tooltips), a lot of this confusing could be lessened.

[akien-mga]

I fully agree with you, but the problem here is a matter of scope.

Yes, this proposal is about a specific usability improvement in the import workflow, but not a general proposal on improving the import workflow.

So your proposal to add tooltips to make things clearer is IMO a great idea, but it's orthogonal to my proposal - implementing my proposal doesn't affect the implementation of tooltips (or any other UX improvement that would make the import workflow clearer), nor does implementing your suggestions affect my proposal to allow defining a default import mode for a given file extension.

Proposals are best kept well-defined so that they can easily be agreed upon and implemented. My proposal doesn't change the import system, it simply lets users define what import mode should be used by default for a given extension. So instead of selecting each file to set their import mode and reimport, you'd set it upfront in the project settings.

In short, you should open a dedicated proposal for what you're suggesting (which again, I think is a good idea and a necessary UX improvement, as I agree that figuring out the import options is complex).

[golddotasksquestions]

This separate proposal already exists, I already mentioned and linked to it in my comment but here it is again: #2041

My aim was just to help those who are capable of making changes understand what goes through a beginners mind while confronted with the import options. Since those who are capable of making these changes are no longer beginners and see the world differently. As I get more experience I also tend to forget these things so I wrote them down here. I'm sorry if this was misplaced and wasted your time.

[alessandrofama]

Trying out 3.3 with the new Keep File (No Import) feature, the first thing I noticed is that I can't set this option when selecting multiple .wav files for example. Allowing to set a default importer for a specific extension would be very cool! That would solve my issue.

[golddotasksquestions]

I would also like to do that for CSV files (imho the "keep file" should be the default for CSVs, not the translation. The CSV option should not be called "CSV Translation" in the first place, but I already talked about the naming in length so I will leave it at that)

[RichardEllicott]

I think just adding one extra CSV profile ..... other than "CSV Translation"

would do this right? so then we could set this as default and it would just ignore the file

the problem is we have a menu, but only one option in the dropdown box (CSV Translation)

[akien-mga]

@RichardEllicott For CSV, you don't need an extra profile, you can use "Keep File (No Import)".

This proposal would allow to default to it for all CSV files, but until then, you can select all your CSV files and select "Keep File (No Import)", press Re-Import, and you'll get the same result that was this proposal would implement.

[RichardEllicott]

@RichardEllicott For CSV, you don't need an extra profile, you can use "Keep File (No Import)".

This proposal would allow to default to it for all CSV files, but until then, you can select all your CSV files and select "Keep File (No Import)", press Re-Import, and you'll get the same result that was this proposal would implement.

the problem is then you have to keep selecting each csv file, reimporting it, which causes Godot to require a reboot.... if i have 10 csv files.... i need to restart Godot 10 times?

you cannot change the default import profile in the "Import Defaults" menu, and if there was another setting in there, you could turn off defaulting to language file importing which is really annoying if you have existing csv files..... in the meantime i'm using a different extension, however this frustrates my ability to load in open office (which only sees csv files as the default calc import), and also means other csv repos i have to rename them all stopping me using git properly

i think it's a reasonable request to allowing changing a default setting if this bothers you

[thinkier-tim]

As an alternative method of resolving this issue, make translation files use something other than the universally-accepted human-readable flat-file record format for half a century as their extension. Source

While breaking backwards compatibility is a Bad Thing (tm), violating the principle of least surprise is also a Bad Thing (tm), and throwing errors in the console and displaying a big red "X" in the filesystem dock because someone had the audacity to add a file that any reasonable person would assume was a human-readable spreadsheet into their game's asset tree is patently ridiculous.

This issue is analogous to attempting to automatically import any .bmp file as some sort of custom "Binary Messaging Platform" file, completely disregarding that anyone who has used a computer in the last several decades would assume "filename.bmp" is an image file. You might as well decide that any .txt file is a "Texture eXtraction Tool" instead of plaintext. It's stupid on its face; If this were my project, I would be absolutely mortified to have this ticket in my reporting system.

I'm not claiming to be a programming genius, but if someone kicks the tires and the vehicle flings its drive shaft into the parking lot next door in response, the potential customer gets the feeling they should visit a different dealership.

[thinkier-tim]

@Calinou - I see you added a "confused" emoji as a reaction to my comment; What confuses you about my addition to the discussion? Was it the car analogy?

Godot assumes any CSV file in the resource tree is a translation, and continuously sprays red text in the console and the in-IDE output dock if it's not. There is no accommodation for the usage of CSV files as a portable plaintext data transfer mechanism, flying in the face of a record storage format in use since at least 1972.

The CSV file format predates the desktop PC and we're discussing how to get the engine to play nice with it. The only confusing part to me is why this is an issue in the first place.

[Zireael07]

@thinkier-tim:

There is no accommodation for the usage of CSV files as a portable plaintext data transfer mechanism

This is not true - for data CSV, you can use "no import" setting in 4.0. /master.

[RichardEllicott]

@thinkier-tim:

There is no accommodation for the usage of CSV files as a portable plaintext data transfer mechanism

This is not true - for data CSV, you can use "no import" setting in 4.0. /master.

we're not using 4.0!!!

why do i feel you're trying to miss-understand the request

to summarise:
we have to rename all our csv files.... and if we don't, we have to reboot for each and every csv file to be re-imported

can we get anyone involved in the development of Godot proper to respond? primarily why can't we turn this off.... and furthermore, why does godot have to reboot to import a csv file?

this should be a BUG.... not a feature request.... perhaps we should be filing this in bugs to be taken seriously

[willnationsdev]

For feedback on this proposal, the only thing that stands out to me is this:

This should likely be a project setting, and when changed, it would not modify already imported assets, but it would affect any asset imported in the future. The "Import" dock would still allow customizing the behavior and changing what importer to use for given files.

It seems the proposal avoids dealing with files that were already imported. What if people have already mass-imported an undesirable import setting for many files prior to modifying project settings? Do we already have a batch means of updating the import settings of individual files on a large scale so that each file doesn't have to be separately opened in the Import dock to be manually updated? If not, we might want to open a separate proposal to handle that use case. For example, you could configure an import setting and then have a "Batch Apply" button that opens a dialog with a Tree of all the files in the project for selection.

---

@RichardEllicott

we're not using 4.0!!!

Just an FYI, generally speaking, changes that are merged into master and which are deemed compatible with the latest 3.x build may be backported to it and thus available in the subsequent 3.x release. Perhaps the "no import" setting is one of such features that should be backported, if it isn't there already (I haven't dealt with Godot in some time)? And then, combined with this proposal, the CSV import problem for Godot 3.4 could be resolved?

why does godot have to reboot to import a csv file?

Not really sure about that. I haven't dealt much with the import system.

@thinkier-tim

I happen to agree that CSV spreadsheets should be a default import behavior. However, rather than making translation files use an alternative file format, a better solution might be to make translation files be the alternative import setting available for .csv files. Otherwise, the same problems you have with general CSVs would reappear for everyone trying to use translation-related CSVs (can't open in Open Office because they can't use .csv file format at all, etc.). Either that, or you make use of the same import setting for both types of files, but have the setting import typical CSVs as a CSVFile class and the translation files with locales for headings import as a derived TranslationCSVFile class.

However, all of this conversation is pretty much off-topic to the above-proposed issue and should be handled in their own separate proposal(s). The plague of CSV data management is merely one manifestation of an over-arching problem that this proposal can solve, and griping about CSVs here doesn't have any bearing on whether this proposal is a good solution for the general problem or not. From the sound of it, you guys both could use this change and more changes. So, those changes should have associated proposals.

If you have feedback about why this proposal is good or bad for dealing with the general problem or if you have suggestions on how to improve this proposal's capacity to deal with the general problem, then the feedback is valuable. Otherwise, cluttering the discussion more in this proposal is only going to frustrate core contributors who actually do the work and will be counterproductive to your goals.

[Calinou]

Perhaps the "no import" setting is one of such features that should be backported, if it isn't there already (I haven't dealt with Godot in some time)? And then, combined with this proposal, the CSV import problem for Godot 3.4 could be resolved?

The "Keep (No Import)" importer was backported to 3.x, but the issue is that you can't set the importer on multiple files at once in the FileSystem dock (or change it via the new Import Defaults editor). You need to set it on each file one by one.

[RichardEllicott]

well i have had to rename everything now in my entire projects from csv to ucsv (i suggest we could ucsv as a convention meaning "unicode csv")

i found this all very confusing as someone just springed this translation feature on us, i guess no-one agreed that this was a breaking bug for people who had csv files as part of their programs .... it triggers the editor to keep reimporting over and over again, especially when you move platforms it's just impossible

[Arecher]

This would be a welcome added functionality to the editor. The option to (for example) import .bmp files as BitMaps automatically, without having to manually reimport them (and restart the editor), could greatly improve the workflow in projects that use different file extensions for specific purposes.

[daveparadise]

This is really needed IMO. If you are working with ClickMasks on TextureButtons, you have to restart the editor every time you import a click mask, since they import as Textures. Which is reallllllllly annoying :'(

[akien-mga]

We discussed this in a proposals review meeting and confirmed that this is a good feature to implement. It can be implemented similarly to what I described initially with a new section (or subsection of the current "Importer Defaults") to configure for each extension which importer should be used by default.

[paulloz]

This would help me on a project, and it seems like it might not be too complicated, so I'd be happy to have a stab at it.

[aXu-AP]

I think that in addition to associating certain filetypes with certain import settings, they could be set by folder basis. For example, if all my sfx are .ogg files, files from sfx/*.ogg folder should not have looping on, but files from music/*.ogg should.
Maybe have a list of usual file types by default in this page, and a ➕ button to add custom filters.

[vonagam]

I support @aXu-AP idea. It is helpful not only for folder based usage but for naming conventions too. So that you can automaticall import all *-normals.png as you see fit with separate settings from simple *.png.

There is no current support for globs in godot itself, but pcre2 library does have it, so no need for additional dependencies.

Multiple patterns can match the same path so order will matter. It can be a rearrangeable table with each row having a pattern and an importer preset that corresponds to it. (With lower line having more priority, so new specific rule does not have to be dragged to the top.)

[Calinou]

There is no current support for globs in godot itself, but pcre2 library does have it, so no need for additional dependencies.

String has a match() function which supports * (wildcard) and ? (single-character wildcard) placeholders.

[paulloz]

This proposal isn't related to import settings, tough.

[aXu-AP]

I think that this should be implemented in such a way that it leaves the door open for further improvement. Having this as it's been proposed here is already good. I might write separate a proposal for further improvement then.

[vonagam]

The proposal addresses some pain point. And that is an inability to specify default import settings (in the current form only importer itself) based on pattern (in the current form only an extension, so *.extension). There is nothing stopping the proposals from being modified if there is an agreement that a modified version will be more useful.

At the very least as @aXu-AP says the potential for improvement should be taken into consideration, otherwise implementing more relevant thing later will become a breaking change.

[aXu-AP]

But would it be a breaking change though? Even if this importer settings page would be revamped, it wouldn't break any projects, so it could be included in any feature release. Is my understanding correct?

[vonagam]

Depends on how it will be represented in saved configuration. If it will be implemented first as currently presented, then later there will be need to include at least a converter code. So most likely not breaking, but there will be no need to include a converter if the format from the start was matching.

[YuriSizov]

There are several ways to preserve compatibility even if we don't plan ahead for globbing. E.g. we can have a separate property for each rule that would toggle between extensions and globs. So that's not really important to figure out right now, and it is definitely better to open a dedicated proposal for globs, because that requires more explanation and further discussion, and this feature has already been approved.