File name with non alphanumeric characters not handled correctly

[macdroid53]

Problem statement

When using File path and Read FCStd node, if the file name includes non alphanumeric characters (-, '(',')' ) file is not recognized.

If the file name includes a '-' or ')', the error (shown above the Read FCStd node):

2022-06-10 08:31:41,837 [INFO] sverchok.nodes.exchange.FCStd_read 334: FCStd read error
2022-06-10 08:31:41,837 [ERROR] sverchok.nodes.exchange.FCStd_read:337 : <class 'NameError'>

If the file name includes a '_', the error (shown in the terminal window):

File "/home/mac/.config/blender/2.93/scripts/addons/sverchok-master/nodes/exchange/FCStd_read.py", line 337, in LoadSolid
    F.closeDocument(Fname)
NameError: Unknown document 'RaspPi4random_4'

location: <unknown location>:-1

Steps to reproduce

1. Create a FreeCAD file with the described characters
2. Attempt to specify the file in the File path node connected to a Read FCStd node.

Sverchok version

v1.1.0-alpha.2

[zeffii]

okay, let's have a look

[zeffii]

that's weird, because the only file i've ever opened is called Simple_assy_001.FCStd, for testing

just to be clear here, are you saying you can use these nodes without issues read/write.etc all day long, just as long as the filenames do not contain underscores, hyphens and parenthesis characters?

is this something which has never been possible for you or don't know never tried

[zeffii]

kind of peculiar that the line numbers in the error message (334, 337) are beyond the length of the current version of that node. 243 lines) . Hinting that the error might be in the FreeCad module.

[zeffii]

the FilePath node produces the expected File Path

[macdroid53]

I'm on Debian and Manjaro.

If I remove the offending character it works fine.

Not sure it matters, but I'm am using this version of FCStd_read.py:

FCStd_read.zip

[zeffii]

so it doesn't like spaces, hyphens and parenthesis .

2022-06-10 15:37:57,326 [INFO] sverchok.nodes.exchange.FCStd_read 157: FCStd label read error
Traceback (most recent call last):
  File "C:\Users\zeffi\Desktop\scripts\addons_contrib\sverchok\nodes\exchange\FCStd_read.py", line 159, in LabelReader
    F.closeDocument(Fname)
NameError: Unknown document 'RaspPi4random_4 - (Duplicate)'
File "C:\Users\zeffi\Desktop\scripts\addons_contrib\sverchok\nodes\exchange\FCStd_read.py", line 132, in LabelReader

[zeffii]

i doubt your modified FCStd_read.py is the source of this behaviour.

[macdroid53]

i;m going to guess object labels are permitted to have spaces in FreeCAD.

from sverchok.data_structure import no_spaces

...
      labels.append( (no_space(obj.Label), obj.Label, obj.Label) )

there will probably need to be a remap to go from the nospaces(label) back to the original label for lookups based on the selected enum. This is harder to explain than to do, but the whole thing is unpleasant and brittle -- if this is indeed part of the issue

FreeCAD, indeed does allow spaces in object names. As well as commas, etc.

[zeffii]

just to be clear, there are two issues here

1. the population of enum's identifier (first column) must be [a-zA-Z_0-9] only
2. the File name issue.

fixing 1 will probably break existing .blends if the modification doesn't attempt to "upgrade" the code at runtime. Bit of a headache, but probably doable. It needs to be corrected anyway.

it could be that fixing 1. fixes 2, because the FreeCAD module while running in FreeCAD loads those files fine.

>>> import FreeCAD as F
>>> doc = F.open(r"D:\DOWNLOADS\2022\ObjectCount45\RaspPi4random_5 - Copy.FCStd")
>>> # App.setActiveDocument("RaspPi4random_5___Copy")
>>> # App.ActiveDocument=App.getDocument("RaspPi4random_5___Copy")
>>> # Gui.ActiveDocument=Gui.getDocument("RaspPi4random_5___Copy")
>>> Gui.runCommand('Std_OrthographicCamera',1)
>>> doc.FileName
'D:/DOWNLOADS/2022/ObjectCount45/RaspPi4random_5 - Copy.FCStd'
>>> 

reading the error mssage carefully suggests that an exception was encountered in this part

            for f in fc_file_list:
                try:
                    F.open(f) 
                    Fname = bpy.path.display_name_from_filepath(f)
                    F.setActiveDocument(Fname)
                    
                    for obj in F.ActiveDocument.Objects:
                        if obj.Module in obj_mask or obj.TypeId in obj_mask:
                            labels.append( (obj.Label, obj.Label, obj.Label) )
                    
                except:
                    info('FCStd label read error')
                finally:
                    F.closeDocument(Fname)

but we fail to get enough information :)

                except Exception as err:
                    info(f'FCStd label read error:\n{err}')

[zeffii]

further inside FreeCAD

>>> from pathlib import Path
>>> Path(doc.FileName).stem
'RaspPi4random_5 - Copy'
>>> Fname = Path(doc.FileName).stem
>>> F.setActiveDocument(Fname)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
Base.FreeCADError: Try to activate unknown document 'RaspPi4random_5 - Copy'
>>> 

[zeffii]

>>> doc.Name
'RaspPi4random_5___Copy'

tadaa!

[macdroid53]

Sorry, my comment about FC allowing characters like comma, or parentheses, was about the names (labels displayed in the Tree view in FC) of the solids in the file. Not the file name itself, which is the focus of this issue.
As for file names, I not sure of the complete list of characters FC allows. But, if understand, for example, Blender allows underscore, but, not minus. So, the File path node would be subject to Blender rules and the Read FCStd would be subject to FC rules? Does Read FCStd node actually do the file open? (thus making it need to follow FC rules?)
And, Read FCStd node would need to be able to deal with all the characters FC allows for object labels (the names displayed in the Tree view?

Sorry, for all the questions.

[zeffii]

all questions are welcome @macdroid53 .

0. I didn't know blender has that limitation, I'm not very adventurous with filenames.
1. F.open() is freecad, and it doesn't really care what the file is called. If you can "os-legally" name a file something, it seems to accept it.
2. File Path node is not subject to Blender's file name rules, it simply looks at your filesystem and lets you pick a file.
3. The visible name from that dropdown is just one of the elements of the enumerator list. The name member accepts a liberal character set, just the identifier element was restricted by blender devs to a limited charset

enum = [(identifier, name, description), ...]

# unfortunately the node tries to do this:

                    for obj in F.ActiveDocument.Objects:
                        if obj.Module in obj_mask or obj.TypeId in obj_mask:
                            labels.append( (obj.Label, obj.Label, obj.Label) )
                            
                            #  obj.Label  here may contain  unacceptable  characters.

4. here the FCStd read node assumes that these following will be the same "string"

>>> file_name = r"D:\DOWNLOADS\2022\ObjectCount45\RaspPi4random_5 - Copy.FCStd"
>>> import FreeCAD as F
>>> doc = F.open(file_name)
>>> doc.Name
'RaspPi4random_5___Copy'  # <--- will work
>>>
>>> from pathlib import Path
>>> Path(doc.FileName).stem
'RaspPi4random_5 - Copy'   # <--- wont work
>>>
>>> bpy.path.display_name_from_filepath(doc.FileName)      # <---- FCStd node uses this 
'RaspPi4random_5 - Copy'   # <--- also not

so... when the node tries to

                    F.open(f)   #  f = file_name
                    Fname = bpy.path.display_name_from_filepath(f)
                    F.setActiveDocument(Fname)

it throws an exception, and the exception has a finally clause, which tries to execute another thing that will cause an exception

                    F.closeDocument(Fname)

[zeffii]

this will probably work for the document name issue (the internal name doc.name)

            for f in fc_file_list:
                try:
                    temp_doc = F.open(f) 
                    Fname = temp_doc.Name or bpy.path.display_name_from_filepath(f) 

                    F.setActiveDocument(Fname)
                    
                    for obj in F.ActiveDocument.Objects:
                        if obj.Module in obj_mask or obj.TypeId in obj_mask:
                            labels.append( (obj.Label, obj.Label, obj.Label) )
                    
                except Exception as err:
                    info(f'FCStd label read error: {Fname=}')
                    info(err)
                finally:
                    F.closeDocument(Fname)

but i suspect this might work too

            for f in fc_file_list:
                try:
                    doc = F.open(f) 
                    for obj in doc.Objects:
                        if obj.Module in obj_mask or obj.TypeId in obj_mask:
                            labels.append( (obj.Label, obj.Label, obj.Label) )
                except Exception as err:
                    info(f'FCStd label read error: {f}')
                    info(err)
                finally:
                    del doc

[zeffii]

@macdroid53 #4524

[zeffii]

I think this solves the FileName issue, but the obj.label issue is still a gaping wound.

[zeffii]

and the other FCStd nodes are probably doing something similar in places.

[zeffii]

as for alternative version of the FCStd read the filename/docname issue, if you are comfortable editing the python yourself
for that file you can replace lines that create Fname, usually something like

- F.open(f)
- Fname = bpy.path.display_name_from_filepath(f)
+ doc = F.open(f)
+ Fname = doc.Name or bpy.path.display_name_from_filepath(f)

[zeffii]

it doesn't appear that using the obj.Label in enum identifier element has caused any errors, that code isn't even called until the user clicks "Select FC Part". But when it is called it's still not a problem

[macdroid53]

as for alternative version of the FCStd read the filename/docname issue, if you are comfortable editing the python yourself for that file you can replace lines that create Fname, usually something like

- F.open(f)
- Fname = bpy.path.display_name_from_filepath(f)
+ doc = F.open(f)
+ Fname = doc.Name or bpy.path.display_name_from_filepath(f)

Comfortable enough.

I see that the File path node allows multi-select. How would I use that, what does it mean to the Read_FCStd node?

[zeffii]

FilePath node, pick multiple files by holding ctrl and click the file.

according to the FCStd read node's code, it will let you

• output all objects in those files, or just your selection (singular i think..,but you can select from all objects in those files) , unless the selection mask allows you to explicitely state which objects you want to get.
• the node uses a Set collection datatype to store the objects, this should mean that there are no duplicate references to objects, but i wouldn't count on that being a useful mechanism to ensure uniqueness. Only one way to find out how effective that is :) try it.