[DPE-4736] Improve async replication stability

[marceloneppel]

Issue

The async-replication tests sometimes fail due to the following issues:

• Sometimes, peer data is no longer accessible at _on_async_relation_broken.
• _on_async_relation_changed is sometimes executed just after the K8s pod is rescheduled, but the workload container is not accessible yet.
• During a scale up operation, sometimes the new unit doesn't receive the relation joined event (only a relation created event + the relation changed event).

Solution

• The cham checks if peer relation data is available at _on_async_relation_broken.
• The async replication changed event is deferred if the workload container is not accessible yet.
• Also, I created DPE-4737 for further investigation regarding the Juju relation created and joined events.
• Added unit tests for the changes.

DPE-4833 was created for investigation and fixing of the remaining failures.

[codecov]

Codecov Report

Attention: Patch coverage is 55.55556% with 4 lines in your changes missing coverage. Please review.

Project coverage is 70.83%. Comparing base (2827474) to head (af94eca).

Files	Patch %	Lines
src/relations/async_replication.py	55.55%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #526      +/-   ##
==========================================
+ Coverage   70.25%   70.83%   +0.57%     
==========================================
  Files          10       10              
  Lines        2797     2801       +4     
  Branches      521      522       +1     
==========================================
+ Hits         1965     1984      +19     
+ Misses        732      718      -14     
+ Partials      100       99       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[lucasgameiroborges]

Nice!