How to add a satellite to environment, but not as an Agent?

[josephTimm]

Hi,

I currently need to add a satellite to a simulation, but I do not want it to be an agent. I have looked into adding it to a custom environment; however, setting that satellite's dynamics is proving to be quite problematic and involved. Is there an easier way to add a satellite to the environment without it being an agent?

[Mark2000]

I've thought about this some for RPOD-type tasks, so I do have some thought but haven't tried implementing anything yet. I think the best move would be to add the "passive" satellite as an additional agent, and give it an action space with one custom action that's just "do nothing/follow some default behavior"; or if it's just following the satellite dynamics, just use the drift action. In the training setup, you could configure two policies using the policy_mapping_fn, one that's your actual agent policy, and the other that is trivial for the passive agent (building on the example here, but you probably wouldn't need the "Action Continuation and Concatenation" section code).

If you have any more specifics about what you're trying to implement, I'd be happy to provide better pointers.

[josephTimm]

Thanks Mark! My original implementation was more in-line with that, so I'll stick with it. Regarding the two different policy maps, my original idea was instead to filter out the "passive" satellite by sat_id in calculate_reward to prevent it from being trained. Would this implementation also work to effectively remove the passive satellite as an agent?

[Mark2000]

I think you'd also need to filter it out in the observation and action spaces and step() somehow. It might be possible to use the PettingZoo API and make that satellite dead from the start (thereby eliminating it from actions, observations, and reward, but it continues to exist in the basilisk simulation), but you'd still be forced to use the multiagent API. At that point, it might just be better to write a wrapper for the generic GeneralSatelliteTasking environment that uses the single-agent API for actions/observations/rewards for the active satellite and handles the passive satellite internally.