How to set 3D Search Paths Alias?

[cdwilson]

I'm trying to get the render_3d output working in the docker container, and I'm getting a bunch of warnings like the following:

WARNING:(W047) Missing 3D model for C8: `/home/root/workdir/:FCD_OSHW_PCB_Libraries:CAPC100X50X55L25.STEP` (kibot - out_base_3d.py:131)
WARNING:(W047) Missing 3D model for C14: `/home/root/workdir/:FCD_OSHW_PCB_Libraries:CAPC100X50X55L25.STEP` (kibot - out_base_3d.py:131)
WARNING:(W047) Missing 3D model for C15: `/home/root/workdir/:FCD_OSHW_PCB_Libraries:CAPC160X80X90L30.STEP` (kibot - out_base_3d.py:131)
...

I think this is happening because my footprint libraries make use of the "3D Search Paths - Alias" feature in KiCad (to avoid hard coding 3D model paths in the footprint since there is no project-local 3D-lib-table):

I don't see anything in the KiBot documentation about how to set a 3D search path alias for outputs that use the 3D models. Is this supported in any way currently?

[set-soft]

Hi @cdwilson !
You are right, this isn't supported. And KiCad itself is incomplete about it.
I think we could allow using environment variables, most probably defined in the KiBot YAML file, to replace these aliases.
What do you think?

[cdwilson]

Hi @set-soft!

And KiCad itself is incomplete about it.

Can you explain more about what you mean by this?

I think we could allow using environment variables, most probably defined in the KiBot YAML file, to replace these aliases.
What do you think?

I'm not sure that I exactly understand what you are proposing. Can you provide more details?

FYI, I believe that these 3D search path aliases are different from KiCad environment variables. They are stored in the <kicad user preferences>/3d/3Dresolver.cfg file:

Here's an example from my macOS system:

❯ cat ~/Library/Preferences/kicad/6.0/3d/3Dresolver.cfg 
#V1
"22:FCD_OSHW_PCB_Libraries","124:${KIPRJMOD}/libraries/github.com/flyingcampdesign/fcd-oshw-kicad-lib-pcb-libraries/3d_models/FCD_OSHW_PCB_Libraries.3dshapes","77:Flying Camp Design OSHW KiCad library derived from pcblibraries.com libraries"

(note the <number_of_characters>: prefix for each quoted string in the above config)

I don't think it's possible to use KiCad environment variables to achieve the same goal, but I may be wrong 🤷🏻‍♂️

Could we have KiBot create the 3Dresolver.cfg file in the docker container as part of the preflight process?

I agree that we should store the 3D aliases in the KiBot YAML file.

[set-soft]

Hi @cdwilson !

And KiCad itself is incomplete about it.

Can you explain more about what you mean by this?

What you already found: there is no mechanism to create a self contained archive using the aliases, they rely on global settings. I think the best current solution is to just avoid using them. You can always provide paths relative to the project using ${KIPRJMOD} and/or the path for 3rd party add-ons: ${KICAD6_3RD_PARTY}. In the worst case you need to add a symlink to solve the problem.

I think we could allow using environment variables, most probably defined in the KiBot YAML file, to replace these aliases.
What do you think?

I'm not sure that I exactly understand what you are proposing. Can you provide more details?

When we found an alias (module name in the format XXXX:YYYY) we could look for an environment variable named XXXX, if it exists just pretend the name is ${XXXX}/YYYY
So people can define the aliases in global.environment, as environment variables.

FYI, I believe that these 3D search path aliases are different from KiCad environment variables. They are stored in the <kicad user preferences>/3d/3Dresolver.cfg file:

Of course, but this file won't be inside a repo.

Here's an example from my macOS system:

❯ cat ~/Library/Preferences/kicad/6.0/3d/3Dresolver.cfg 
#V1
"22:FCD_OSHW_PCB_Libraries","124:${KIPRJMOD}/libraries/github.com/flyingcampdesign/fcd-oshw-kicad-lib-pcb-libraries/3d_models/FCD_OSHW_PCB_Libraries.3dshapes","77:Flying Camp Design OSHW KiCad library derived from pcblibraries.com libraries"

(note the <number_of_characters>: prefix for each quoted string in the above config)

I don't think it's possible to use KiCad environment variables to achieve the same goal, but I may be wrong 🤷🏻‍♂️

To allow it we need to rewrite the model path, but this is something KiBot already does! When a 3D model is missing KiBot tries to download it from the KiCad's repo, works quite well for standard 3D models. So changing XXXX:YYYY to ${XXXX}:YYYY isn't something really complex.

Could we have KiBot create the 3Dresolver.cfg file in the docker container as part of the preflight process?

Yes, this is another posibility, but is more complex. We must add a mechanism to define the aliases and we must touch another configuration file. KiAuto touches various configuration files, KiBot avoids it. So it involves changing the global settings and restoring them at exit, which is complex if KiBot crashes, but can be done.

BTW: KiCad maintainers are quite special, this file has yet another bizarre format. I already discussed it with them, they should choose one format, lets say JSON (I prefer YAML), and handle all files in the same format. The KiCad 6 release was an excellent moment to achieve this (it changed the schematic format from a custom one to s-expressions format, which is not really a common format), at the same time they changed various configuration files from INI format to JSON ...

I agree that we should store the 3D aliases in the KiBot YAML file.

Yes, but I don't really think they have any real use. People should simply use environment variables, and allow people defining them as text replacement variables. Aliases aren't really needed, and they aren't stored in the project.

[cdwilson]

Hi @set-soft, thanks for your detailed reply.

there is no mechanism to create a self contained archive using the aliases, they rely on global settings. I think the best current solution is to just avoid using them. You can always provide paths relative to the project using ${KIPRJMOD}

I've been thinking more about this suggestion and I think you're right. The main benefit I can see for using ${KIPRJMOD} over aliases is that it allows truly self-contained projects, without requiring other users of the project to create global aliases. (The other benefit is that it already works with KiBot, so no changes are needed on the KiBot side.)

I'm not sure that I exactly understand what you are proposing. Can you provide more details?

When we found an alias (module name in the format XXXX:YYYY) we could look for an environment variable named XXXX, if it exists just pretend the name is ${XXXX}/YYYY
So people can define the aliases in global.environment, as environment variables.

This is a clever idea. I suppose there may be a corner case where someone is already using an environment variable for a different purpose which has the same name as an alias, and there would be a name collision (probably unlikely).

Yes, but I don't really think they have any real use. People should simply use environment variables, and allow people defining them as text replacement variables. Aliases aren't really needed, and they aren't stored in the project.

Given how they are currently implemented in KiCad for 3D models (global only), I agree with this. In general, I think aliases only have real value if they can be defined globally and locally in the project. fp-lib-table & sym-lib-table--which are effectively aliases for footprints/symbols--are useful because they allow footprints to be abstractly "linked" to symbols via an alias, but leaves the actual resolution of the paths (and whether they are global or project-local) up to the end user of the libraries. The problem with the 3D models aliases in KiCad is that it's not possible to define project-local aliases in addition to the global ones.

All that said, I'm going to stop using aliases for now and switch to using ${KIPRJMOD} for my projects as you suggested. Unless you hear otherwise from me, you can assume that I no longer need anything special in KiBot regarding the 3D model aliases.

[set-soft]

Just a note: Now KiBot can use environment and text variables as aliases. It can be disabled in case of conflict.
Commit: 418b74b

[set-soft]

And now people can even define aliases in the KiBot configuration file (d01943e)