Checking interest for Python interface to starter/engine keywords

[nawijn]

Dear All,

I would like to check whether or not there is an interest in a Python package that supports generating starter/engine input files completely from within a Python script. I have started to work on it and I am pretty happy with what I have so far:

1. Scaffolding for all ~600 keywords to support generating starter files is done;
2. Test infrastructure is done and scaffolding for ~100 keywords is done (scraped from example input files);
3. I have implemented around 30 keywords.

The syntax that is currently supported is something like the following:

N = 5
node_ids = np.arange(1, N + 1)
xc_yc_zc = np.random.random((N, 3))

starter = Starter()
starter["BEGIN"](runname="demo")
starter["TITLE"](title="This is an awesome title")
starter["NODE"](unit_id=1, node_ids=node_ids, xc_yc_zc=xc_yc_zc)
...
...
starter["END"]()
starter.write()

Since it is a lot of tedious work to implement all the keywords, I would first like to check interest and maybe get some additional ideas/feature requests.

As an example how a particular keyword is implemented, take a look at the example for "/BEAM":

@dataclass
class Beam(Keyword):
    """Describes the beam elements. Two properties (/PROP/TYPE3 (BEAM) and
    /PROP/TYPE18 (INT_BEAM)) are available for this beam element. The
    properties describing a beam element are all defined in a local beam
    coordinate system.
    """

    part_id: int
    beam_ids: List[int] | NDArray
    node_ids: List[List[int]] | NDArray

    @property
    def keyword(self):
        return f"/BEAM/{self.part_id}"

    @property
    def pre_conditions(self):
        return [
            (
                len(self.beam_ids) == len(self.node_ids),
                "Pre-condition `len(beam_ids) == len(node_ids)` is violated.",
            )
        ]

    @property
    def structure(self):
        structure = [
            ArrayOfFields(
                [
                    IntField("beam_ids", 1),
                    IntField("node_ids:0", 2),
                    IntField("node_ids:1", 3),
                    IntField("node_ids:2", 4),
                ]
            )
        ]

        return structure

Some remarks:

1. The pre_conditions property can be used to enforce consistency checks on the input values
2. The structure of a keyword is completely defined by the structure property. This particular example states that the structure consists of a sequence of lines where each line consists of 4 integer fields.
3. `IntField("beam_ids", 1) states that the first (I follow the indexing as specified in the documentation, so array indices start at 1) field holds an integer and the value should be extracted from the "beam_ids" attribute.
4. Every attribute has a type annotation, this allows for very nice editor integration.
5. As a user, you never have to deal with properly formatting a keyword.

As mentioned above, every keyword has an associated test. This test is defined in a TOML file. For example, the test corresponding to the above "/BEAM" example is as follows:

[default]
value="""\
/BEAM/1_____________________________________________________________________________________________
________21________22________32________42____________________________________________________________
________31________23________33________43____________________________________________________________
________41________24________34________44____________________________________________________________
"""
[default.params]
part_id = 1
beam_ids = [21, 31, 41]
node_ids = [[22, 32,42 ],
            [23, 33, 43],
            [24, 34, 44]]
[scrape]
value="""\
/BEAM/8_____________________________________________________________________________________________
____287769____173067____173069____140055____________________________________________________________
____287770____173065____173067____140055____________________________________________________________
____287771____173063____173065____140055____________________________________________________________
"""
[scrape.params]
part_id = 8
beam_ids = [287769, 287770, 287771]
node_ids = [[173067, 173069, 140055],
            [173065, 173067, 140055],
            [173063, 173065, 140055]]

So, each test (like [default]) has a corresponding set of parameters [default.params]. These parameters are used to instantiate the corresponding class and from this instance you can get the text representation of the keyword. This text representation is compared to the value defined in the TOML file.

When run, it looks something like the following (all tests replace whitespace by an underscore):

                                              default                                               
/ALE/MUSCL__________________________________________________________________________________________
_________________1.1_________0______________________________________________________________________

========================================== ALESolverFint ===========================================
                                              default                                               
/ALE/SOLVER/FINT____________________________________________________________________________________
_________1__________________________________________________________________________________________

=============================================== Ams ================================================
                                              default                                               
/AMS________________________________________________________________________________________________
________21__________________________________________________________________________________________

============================================== Analy ===============================================
                                              default                                               
/ANALY______________________________________________________________________________________________
_________0___________________0_________0____________________________________________________________

============================================= AnimVers =============================================
                                              default                                               
/ANIM/VERS__________________________________________________________________________________________
________44__________________________________________________________________________________________

=============================================== Bcs ================================================
                                              default                                               
/BCS/1______________________________________________________________________________________________
BCS_TITLE___________________________________________________________________________________________
___000_111________32________33______________________________________________________________________

============================================ BcsCyclic =============================================
                                              default                                               
/BCS/1______________________________________________________________________________________________
BCS_TITLE___________________________________________________________________________________________
________31________32________33______________________________________________________________________

============================================ BcsLagmul =============================================
                                              default                                               
/BCS/1______________________________________________________________________________________________
BCS_LAGMUL__________________________________________________________________________________________
___111_000________32________33______________________________________________________________________

=============================================== Beam ===============================================
                                              default                                               
/BEAM/1_____________________________________________________________________________________________
________21________22________32________42____________________________________________________________
________31________23________33________43____________________________________________________________
________41________24________34________44____________________________________________________________

                                               scrape                                               
/BEAM/8_____________________________________________________________________________________________
____287769____173067____173069____140055____________________________________________________________
____287770____173065____173067____140055____________________________________________________________
____287771____173063____173065____140055____________________________________________________________

The Python package will not replace a pre-processor, in particular when complex geometries are involved. However, I do believe it is a useful additional tool.

Some final remarks:

1. I plan to have seamless support for Pandas/Numpy and VTK. So, arrays representing nodes, cell connectivities and so on can directly be passed to the corresponding OpenRadioss keywords (like /NODE or /BRICK).
2. Current focus is on implementing the starter keywords, but it is straightforward to also support engine keywords.
3. I plan to release the package under an opensource license (probably MIT).
4. If you like the idea of generating starter/engine files in Python and want to help, that would be very welcomed by me :). As mentioned above it is pretty tedious task, but it is straightforward to do so and as an added bonus, you get very comfortable with all the keywords :).

So, what are your thoughts?

[Serince]

Hello Marco,

It is really great idea. A python script for starter makes very easy to integrate openradioss with other open source programs such as salome-meca and Freecad. It will be very tedious to write the script for each card as you said. Scraping cards from hm_cfg_files in main directory of OpenRadioss can be better. As far as I understand, this folder contains the default values of each card and the parameters in the card. I can join you if you share your codes on github.

All the best,
Serdar

[nawijn]

Hi Serdar,

I completely forgot about the config files. That sounds promising. I did a quick check and at least some of the information is available. Even semi-automatic scraping and transforming into a Python data class is already much better. I will take a better look tomorrow and commit my code to Github.

Thanks! I hope we can join forces and bring something nice into the OpenRadioss community.

Regards,

Marco

[nawijn]

I have committed what I have so far to Github. There is no packaging/installer yet. Will add that when it reaches a more mature state. Also, many of the tests/definitions only contain place holder code.

Note that I have grouped the ~600 keywords alphabetically in sets of 9 keywords per batch. This keeps the amount of code in a single module (file) to an acceptable level. Given a group of keywords contained in a single batch, there are essentially two pieces of information that needs to be defined:

1. An implementation of the dataclasses (in operato/_keywords/_starter)
2. A TOML file containing instantations of each dataclass for one or more parameters (in tests/starter/definitions)

In the tests/starter folder, you can either run python3 <batch> or pytest <batch> to run checks. I typically first start with python3 <batch> in order to check the correctness (this will not check the value parameter defined in the TOML test defnition). Afterwards, I check with pytest <batch>.

Much of the scaffolding code is automatically generated. I does however need some manual tweaks here and there. For example the "/DEF_SHELL" and "/DEF_SOLID" where not correctly represented because of the underscore in the keyword name. I manually corrected this in the dataclass names and corresponding import statements. These tweaks are however relatively minor when compared to the amount of work needed to define each and every keyword datatclass.

If you want examples, take a look at batches 001, 002 and 003. They are/should be complete. So each of the keywords defined in them has a corresponding dataclass defined and all keywords have a TOML test definition file.

If you need more information, let me know.

[nawijn]

For those who are interested. I just committed an update of the Python interface. Still a lot of work to do, but there is progress. I have added the scaffolding for the engine keywords and significantly updated the code in common.py. Starter batches 000 - 005 are complete and a few other keywords are also implemented.

I tried to use the OpenRadioss config file for the automatic generation of the Python definitions of a keyword, but I abandoned this path. So, unfortunately I am still implementing keywords by hand. If anyone wants to help, that would be highly appreciated. It is pretty straigtforward, but tedious. But if we distributed the load, the package can be completed in a couple of weeks (assuming that you can only spend a few hours a week).

If anyone wants to give it a try, let me know. In any case, I will prepare a small manual with one or more examples on how to define a keyword and commit that to the repository.

Oh, before I forget again, the repository is over here operato.

One last comment. As mentioned in the README, I am also creating a C++ based converter for the animation (and probably time history) files. I have not committed the code yet, but I plan to do so somewhere in the next few weeks. This converter has a feature rich commandline interface so you can convert a whole series of animation files in one go. In addition reads the data into VTK datastructures and as such it can support all the export options provided by VTK. I will probably separate it out in a library part and a commandline interface part so if you want to you can integrate it in your own C++ code.

[MarianBulla]

Wow ! This is fantasitc !
Thank you very much for all your effort.

[BastiLoe]

One last comment. As mentioned in the README, I am also creating a C++ based converter for the animation (and probably time history) files. I have not committed the code yet, but I plan to do so somewhere in the next few weeks. This converter has a feature rich commandline interface so you can convert a whole series of animation files in one go. In addition reads the data into VTK datastructures and as such it can support all the export options provided by VTK. I will probably separate it out in a library part and a commandline interface part so if you want to you can integrate it in your own C++ code.

Why c++ for this? I would like to have this in Python as well, alowing interactive option to load anim-files into python shells, easy result extraction ... As anim format is open in anim2vtk should be feasible.

[nawijn]

Just to keep you informed. I just committed an update. More keywords are supported (but still a lot to do) and I changed the way keywords are added to a Starter instance. If you want to take a look, the example code (still incomplete, but the parts that are in are working) is over
here

The reason why I changed the interface is that in the old way, an editor with language server support (like Visual Studio Code or Vim) cannot figure out the type of keyword you are defining and therefore isn't very helpful when adding keywords. With the updated code, you get the documentation and type information of a keyword when typing. See the image below.

If there are any requests, or comments, let me know.

[Serince]

Hello Marco,

I have started to write a hm_cfg parser for creating the keyword classes automatically by using regex library. I was not familiar with regex library to parse this kind of file, so it took time. I think in few days I will upload and create a pull request to your repository.

All the best,
Serdar

[nawijn]

Hi Serdar,

That would be awesome!!

Marco

[nawijn]

Hi Serdar,

Any progress on the hm_cfg parser? Should I continue implementing the keyword classes manually or do you think (semi-)automatic generation is feasible on a relatively short notice? I can also help you with the parser if needed.

Marco

[Serince]

Hi Marco,

It is not completed yet, but I will upload after arranging a little bit. It can be good to have an online meeting to discuss something about parsed data. My email is serince@outlook.com

All the best,
Serdar

[pmquang87]

Hello,
a pre-processor to generate input file for Open Radioss is a very good idea. About 10 years ago, I see a very good python library script for PAM-CRASH at the company that I did my apprentice. Based on that idea, I built a python library for LS-DYNA later, mainly for generating FEM model of textile reinforcement for composite. I used them for research and my Ph.D thesis. I would love to follow your projects, and may contribute, if I can. Thank you.
Regards,
Minh Quang Pham