Inquiry: Converting cFS Command and Telemetry Configurations to COSMOS Format

[rogueWookie]

Hi ~

I'm currently looking into converting each cFS module's command and telemetry configurations into COSMOS format. I've noticed that OpenSatKit provides COSMOS configurations, but it breaks down each cFS module as a target w/ extensive ERB templating. This additional complexity is posing challenges in terms of integrating it with our current setup.

I'm reaching out to inquire if anyone within the community has attempted to convert cFS command and telemetry configurations to COSMOS format. Each cFS module contains command and telemetry configurations in XML format. Perhaps there's a converter hidden somewhere in the cFS repository that I haven't come across yet? Or even better, perhaps someone has already done this?

[irowebbn]

COSMOS/OpenC3 supports converting XTCE to the COSMOS erb format: https://docs.openc3.com/docs/meta/xtce (old docs link)

[rogueWookie]

@irowebbn

I'm running w/ COSMOS 5 (OPENC3_TAG=5.14.0) and cFS tag equuleus-rc1. I tried converting the cFS/cfe/modules/sb/eds/cfe_sb.xml over to COSMOS format using their converter and received the following error.

❯ openc3.sh cli xtce_converter --import cfe_sb.xml --output ./
Processing cfe_sb.xml...
  Ignoring Unknown: <PackageFile>
  Ignoring Unknown: <Package>
  Ignoring Unknown: <DataTypeSet>
  Ignoring Unknown: <IntegerDataType>
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:394:in `xtce_process_element': undefined method `xtce_encoding=' for nil:NilClass (NoMethodError)

        @current_type.xtce_encoding = element.name
                     ^^^^^^^^^^^^^^^^
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:82:in `block (2 levels) in parse'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:740:in `xtce_recurse_element'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:743:in `block in xtce_recurse_element'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in `block in each'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `upto'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `each'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:742:in `xtce_recurse_element'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:743:in `block in xtce_recurse_element'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in `block in each'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `upto'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `each'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:742:in `xtce_recurse_element'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:743:in `block in xtce_recurse_element'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in `block in each'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `upto'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `each'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:742:in `xtce_recurse_element'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:81:in `block in parse'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in `block in each'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `upto'
	from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `each'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:80:in `parse'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:69:in `initialize'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:41:in `new'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:41:in `process'
	from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/packet_config.rb:124:in `process_file'
	from /openc3/bin/openc3cli:251:in `xtce_converter'
	from /openc3/bin/openc3cli:676:in `<main>'

Since COSMOSv5 says it can use xtce files directly I copied over the file cfe_sb.xml to my COSMOS cmd_tlm directory renaming it to cfe_sb.xtce and then rebuilt my plugin. Upon building the plugin it failed with the following output indicating a similar error as above regarding the unknown method xtce_encoding=.

❯ openc3.sh cli rake build VERSION=6.0.0
gem build openc3-cosmos-cfs-sandbox
openc3cli validate openc3-cosmos-cfs-sandbox-6.0.0.gem
  Successfully built RubyGem
  Name: openc3-cosmos-cfs-sandbox
  Version: 6.0.0
  File: openc3-cosmos-cfs-sandbox-6.0.0.gem
Installing openc3-cosmos-cfs-sandbox-6.0.0.gem
  Ignoring Unknown: <PackageFile>
  Ignoring Unknown: <Package>
  Ignoring Unknown: <DataTypeSet>
  Ignoring Unknown: <IntegerDataType>
Error: 
RuntimeError : Error processing /tmp/d20240213-11-22j7da/CFS_SANDBOX/cmd_tlm/cfe_sb.xtce:
undefined method `xtce_encoding=' for nil:NilClass
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/system/system.rb:157:in `add_target'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/system/system.rb:140:in `block in initialize'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/system/system.rb:140:in `each'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/system/system.rb:140:in `initialize'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/target_model.rb:608:in `new'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/target_model.rb:608:in `deploy'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/plugin_model.rb:238:in `block in install_phase2'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/config/config_parser.rb:217:in `parse_loop'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/config/config_parser.rb:217:in `parse_file'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/plugin_model.rb:231:in `install_phase2'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/../bin/openc3cli:313:in `validate_plugin'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/../bin/openc3cli:647:in `<main>'

plugin.txt20240213-11-g558np:9: MAP_TARGET CFS_SANDBOX
Error: Unknown keyword and parameters for Target: MAP_TARGET CFS_SANDBOX
Backtrace:
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/target_model.rb:558:in `handle_config'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/plugin_model.rb:245:in `block in install_phase2'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/config/config_parser.rb:217:in `parse_loop'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/config/config_parser.rb:217:in `parse_file'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/plugin_model.rb:231:in `install_phase2'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/../bin/openc3cli:313:in `validate_plugin'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/../bin/openc3cli:647:in `<main>'

Note

No idea what the error message "Unknown keyword and parameters for Target: MAP_TARGET CFS_SANDBOX" is implying. I've been building COSMOS plugins just fine lately with ingesting cFS sample app telemetry/commands. I see nothing wrong with my plugin.txt that would imply MAP_TARGET is incorrect. Again, I've been building/running fine prior to this. Here it is incase this is relevant ...

VARIABLE cfs_sandbox_target_name CFS_SANDBOX
VARIABLE cmd 1234
VARIABLE tlm 1235
TARGET CFS_SANDBOX <%= cfs_sandbox_target_name %>
INTERFACE <%= cfs_sandbox_target_name %>_INT udp_interface.rb host.docker.internal <%= cmd %> <%= tlm %> nil nil nil 10.0 nil
MAP_TARGET <%= cfs_sandbox_target_name %>

I also tried another file cFS/cfe/modules/time/eds/cfe_time.xml by typing openc3.sh cli xtce_converter --import cfe_time.xml --output ./ but got the same exact error about unknown method xtce_encodings=. I'm not sure if this is an issue with cFS's XML/XTCE files themselves or COSMOSv5 XTCE converter. I was hoping someone could replicate this on their end before I create a ticket on OpenC3 COSMOS GitHub issue tracker.

UPDATE

Apparently there are some features the parser just doesn't have. As such, you'll get the no method error. Moving the conversation over to their GitHub.

[zec]

EDS is not the same as XTCE. They are different formats, albeit both based on XML and with some overlap between their intended functions. The XML files in the cFE repo, including the aforementioned cfe_sb.xml, are EDS documents, not XTCE documents.

As a warning, I'll also note that COSMOS doesn't accept all valid XTCE documents. (In my work, I've run into this particular gotcha!)

[skliper]

Initial implementation of EDS to COSMOS database conversion was added to EDSLib

See: nasa/EdsLib#82