Visualize: visualization of estimated observable mapping #1409

Doresic · 2024-06-04T13:31:21Z

I implemented a visualization routine to visualize all estimated observable mappings: linear scaling+offset for relative data and splines for semiquantitative data. The visualization of estimated splines already existed, so I made the linear one and combined them into a single routine. However, I made it so that each can also be called separately.

To make the implementation easier, I had to expand some classes connected to the relative data: inner problem, inner parameter.

During the implementation, a few bugs popped up. I fixed those here as well. I'll comment on the PR so it's easier to distinguish.

The notebooks have also been updated accordingly.

Doresic · 2024-06-04T13:35:17Z

pypesto/hierarchical/base_problem.py

+        # Mask the data using the inner problem mask. This is necessary
+        # because the inner problem is aware of only the data it uses.
+        for i in range(len(data)):
+            data[i][~self.data_mask[i]] = np.nan
+


This is a bug fix. It caused an issue only if the model contained multiple non-quantitative data types (e.g. semiquantitative and relative). In that case, this check always failed, as we were not masking the data in the check in the same way it was masked in the inner problem initialization.

Doresic · 2024-06-04T13:36:42Z

pypesto/hierarchical/inner_calculator_collector.py

+
+        self.semiquant_observable_ids = None
+        self.relative_observable_ids = None
+
        self.construct_inner_calculators(
            petab_problem, model, edatas, inner_options
        )

        self.quantitative_data_mask = self._get_quantitative_data_mask(edatas)

        self._known_least_squares_safe = False
-        self.semiquant_observable_ids = None


Bugfix.
semiquant_observable_ids would be set to the right value in construct_inner_calculators and then was always overwritten to None in line 112.

pypesto/hierarchical/relative/parameter.py

Doresic · 2024-06-04T13:39:16Z

pypesto/hierarchical/relative/problem.py

@@ -66,43 +65,35 @@ def from_petab_amici(
            petab_problem, amici_model, edatas
        )

-    def check_edatas(self, edatas: list[amici.ExpData]) -> bool:


Check edatas was re-defined completely the same for no reason. It is already defined in AmiciInnerProblem.

Doresic · 2024-06-04T13:39:40Z

pypesto/hierarchical/relative/problem.py

+    def get_relative_observable_ids(self) -> list[str]:
+        """Get unique list of IDs of all relative observables with scaling and/or offset."""
+        return list(
+            {
+                x.observable_id
+                for x in self.xs.values()
+                if x.inner_parameter_type
+                in [
+                    InnerParameterType.SCALING,
+                    InnerParameterType.OFFSET,
+                ]
+            }
+        )

-        for data0, data1 in zip(self.data, data):
-            if not np.array_equal(data0, data1, equal_nan=True):
-                return False
+    def get_groups_for_xs(self, inner_parameter_type: str) -> list[int]:
+        """Get unique list of ``RelativeParameter.group`` values."""
+        groups = [x.group for x in self.get_xs_for_type(inner_parameter_type)]
+        return list(set(groups))

-        return True
+    def get_xs_for_group(self, group: int) -> list[RelativeInnerParameter]:
+        r"""Get ``RelativeParameter``\s that belong to the given group."""
+        return [x for x in self.xs.values() if x.group == group]


Ease of life class methods.

Doresic · 2024-06-04T13:40:28Z

pypesto/hierarchical/semiquantitative/problem.py

+    def get_semiquant_observable_ids(self) -> list[str]:
+        """Get the ids of semiquantitative observables."""
+        return list(
+            {
+                x.observable_id
+                for x in self.xs.values()
+                if x.inner_parameter_type == InnerParameterType.SPLINE
+            }
+        )
+


Some more ease of life

Doresic · 2024-06-04T13:40:59Z

pypesto/problem/hierarchical.py

+        self.relative_observable_ids = (
+            self.objective.calculator.relative_observable_ids
+        )


Saving which observables are relative in the pypesto problem, easier access.

Doresic · 2024-06-04T13:42:30Z

pypesto/visualize/observable_mapping.py

+    # Plot the estimated observable mapping for relative observables.
+    if n_relative_observables > 0:
+        axes = plot_linear_observable_mappings_from_pypesto_result(
+            pypesto_result=pypesto_result,
+            pypesto_problem=pypesto_problem,
+            start_index=start_index,
+            axes=axes,
+            rel_and_semiquant_obs_indices=rel_and_semiquant_obs_indices,
+        )
+
+    # Plot the estimated spline approximations for semi-quantitative observables.
+    if n_semiquant_observables > 0:
+        axes = plot_splines_from_pypesto_result(
+            pypesto_result=pypesto_result,
+            start_index=start_index,
+            axes=axes,
+            rel_and_semiquant_obs_indices=rel_and_semiquant_obs_indices,
+        )


The main routine visualize_estimated_observable_mapping calls the linear mapping and spline one if there are relative or semiquantitative observables, respectively. However, the two lower-level routines can also be called directly.

The same axes object is being passed around and plotted on.

Doresic · 2024-06-04T13:52:02Z

Boehm Example with one semiquantitative and 2 relative observables

dilpath

Looks great 👍

doc/example/relative_data.ipynb

doc/example/semiquantitative_data.ipynb

pypesto/hierarchical/relative/parameter.py

pypesto/hierarchical/relative/problem.py

pypesto/visualize/observable_mapping.py

dweindl

Thanks, looks good to me.

The only concern is the one-parameter-to-one-observable mapping.

pypesto/hierarchical/relative/parameter.py

pypesto/visualize/observable_mapping.py

codecov-commenter · 2024-06-07T09:59:36Z

Codecov Report

Attention: Patch coverage is 21.65605% with 123 lines in your changes missing coverage. Please review.

Project coverage is 83.32%. Comparing base (1a3123d) to head (07913e0).

Files	Patch %	Lines
pypesto/visualize/observable_mapping.py	4.00%	120 Missing ⚠️
pypesto/hierarchical/relative/problem.py	80.00%	2 Missing ⚠️
pypesto/hierarchical/relative/parameter.py	90.90%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1409      +/-   ##
===========================================
- Coverage    83.85%   83.32%   -0.54%     
===========================================
  Files          160      161       +1     
  Lines        13107    13226     +119     
===========================================
+ Hits         10991    11020      +29     
- Misses        2116     2206      +90

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dweindl

👍

m-philipps

Thanks for the implementation, only two suggestions but no need to address them.

m-philipps · 2024-06-12T20:51:00Z

pypesto/visualize/__init__.py

@@ -19,6 +19,13 @@
 )
 from .ensemble import ensemble_identifiability
 from .misc import process_offset_y, process_result_list, process_y_limits
+from .observable_mapping import (
+    _add_spline_mapped_simulations_to_model_fit,


Is it necessary to add that function here? It looks like it should be private

True, you're right. Don't know why it was there.
Removing it.

m-philipps · 2024-06-12T21:02:02Z

pypesto/visualize/observable_mapping.py

+    pypesto_result:
+        The pyPESTO result object from optimization.
+    pypesto_problem:
+        The pyPESTO problem.


The pypesto problem objective is used downstream and I think that we can have problems without objective so it could be nice to specify here that the pypesto_problem is expected to have an objective

True.
Added it into the docstring and added some check to check that there is an objective and that it has the correct calculator.

Implement visualize_estimated_observable_mapping

f100b9b

Doresic requested review from stephanmg, m-philipps, PaulJonasJost, dweindl and dilpath as code owners June 4, 2024 13:31

Doresic commented Jun 4, 2024

View reviewed changes

dilpath approved these changes Jun 4, 2024

View reviewed changes

dweindl reviewed Jun 5, 2024

View reviewed changes

pypesto/hierarchical/relative/parameter.py Outdated Show resolved Hide resolved

pypesto/visualize/observable_mapping.py Outdated Show resolved Hide resolved

groups -> observable_indices, Dilan and Daniel review changes

e827e06

Fix inline literal error

e9e5f52

dweindl approved these changes Jun 10, 2024

View reviewed changes

m-philipps approved these changes Jun 12, 2024

View reviewed changes

PaulJonasJost approved these changes Jun 13, 2024

View reviewed changes

Doresic and others added 3 commits June 24, 2024 17:23

Maren review changes

4888c32

Merge branch 'develop' into visualize_estimated_obs_mappings

6969d4c

Merge branch 'develop' into visualize_estimated_obs_mappings

07913e0

Doresic merged commit ba84aa6 into develop Jun 26, 2024
18 checks passed

Doresic deleted the visualize_estimated_obs_mappings branch June 26, 2024 08:59

This was referenced Jul 25, 2024

Release 0.5.3 #1432

Merged

Prepare release v0.5.3 #1434

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Visualize: visualization of estimated observable mapping #1409

Visualize: visualization of estimated observable mapping #1409

Doresic commented Jun 4, 2024 •

edited

Loading

Doresic Jun 4, 2024

Doresic Jun 4, 2024

Doresic Jun 4, 2024

Doresic Jun 4, 2024

Doresic Jun 4, 2024

Doresic Jun 4, 2024

Doresic Jun 4, 2024

Doresic commented Jun 4, 2024

dilpath left a comment

dweindl left a comment

codecov-commenter commented Jun 7, 2024 •

edited

Loading

dweindl left a comment

m-philipps left a comment •

edited

Loading

m-philipps Jun 12, 2024

Doresic Jun 24, 2024

m-philipps Jun 12, 2024

Doresic Jun 24, 2024

Visualize: visualization of estimated observable mapping #1409

Visualize: visualization of estimated observable mapping #1409

Conversation

Doresic commented Jun 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Doresic commented Jun 4, 2024

dilpath left a comment

Choose a reason for hiding this comment

dweindl left a comment

Choose a reason for hiding this comment

codecov-commenter commented Jun 7, 2024 • edited Loading

Codecov Report

dweindl left a comment

Choose a reason for hiding this comment

m-philipps left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Doresic commented Jun 4, 2024 •

edited

Loading

codecov-commenter commented Jun 7, 2024 •

edited

Loading

m-philipps left a comment •

edited

Loading