Suggesting changes to #338 #352

NicolaCourtier · 2024-06-07T15:14:19Z

Description

Demonstrating alternative changes to implement the additions in #338. The main difference is how the sigma parameter is treated in the GaussianLogLikelihood cost function. Here, it is implemented as an additional PyBOP Parameter.

Issue reference

Towards fixing #257.

Review

Before you mark your PR as ready for review, please ensure that you've considered the following:

Updated the CHANGELOG.md in reverse chronological order (newest at the top) with a concise description of the changes, including the PR number.
Noted any breaking changes, including details on how it might impact existing functionality.

Type of change

New Feature: A non-breaking change that adds new functionality.
Optimization: A code change that improves performance.
Examples: A change to existing or additional examples.
Bug Fix: A non-breaking change that addresses an issue.
Documentation: Updates to documentation or new documentation for new features.
Refactoring: Non-functional changes that improve the codebase.
Style: Non-functional changes related to code style (formatting, naming, etc).
Testing: Additional tests to improve coverage or confirm functionality.
Other: (Insert description of change)

Key checklist:

No style issues: $ pre-commit run (or $ nox -s pre-commit) (see CONTRIBUTING.md for how to set this up to run automatically when committing locally, in just two lines of code)
All unit tests pass: $ nox -s tests
The documentation builds: $ nox -s doctest

You can run integration tests, unit tests, and doctests together at once, using $ nox -s quick.

Further checks:

Code is well-commented, especially in complex or unclear areas.
Added tests that prove my fix is effective or that my feature works.
Checked that coverage remains or improves, and added tests if necessary to maintain or increase coverage.

Thank you for contributing to our project! Your efforts help us to deliver great software.

BradyPlanden · 2024-06-07T17:40:41Z

Hi @NicolaCourtier - thanks for this alternative implementation! I went down a similar path before stopping during the implementation of #338, I came to the conclusion that adding the sigma hyper into the constructed Parameters requires quite a bit of patching. Having the parameters constructed early in the life of an optimisation task and then building the x0, sigma0, and boundaries from them provides a clean separation between parameters -> model and cost -> optim. At the point of the cost and optim construction, we only need to worry about / modify x0, sigma0, and boundaries attributes, which is very easy to understand and review. I'd like to continue with the x0, sigma0, and boundaries implementation in #338, as I think it's concise and provides simplicity.

However, I'm keen to integrate your general improvements into #338. Specifically, your updates to check_sigma0, plot_2d and the CMAES check look great! Also, all of the docstrings changes I had missed 😄. If you'd like to commit those changes directly to that branch, that would be amazing, otherwise I'm happy to add them during review.

Have a great weekend!

codecov · 2024-06-07T20:22:45Z

Codecov Report

Attention: Patch coverage is 98.21429% with 4 lines in your changes missing coverage. Please review.

Project coverage is 97.53%. Comparing base (e0421a5) to head (0e75c8f).

Files with missing lines	Patch %	Lines
pybop/parameters/parameter.py	91.30%	4 Missing ⚠️

Additional details and impacted files

@@                   Coverage Diff                    @@
##           gauss-log-like-fixes     #352      +/-   ##
========================================================
+ Coverage                 97.36%   97.53%   +0.16%     
========================================================
  Files                        42       42              
  Lines                      2433     2476      +43     
========================================================
+ Hits                       2369     2415      +46     
+ Misses                       64       61       -3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

NicolaCourtier · 2024-06-10T10:27:26Z

Thanks for the comment @BradyPlanden, I'm just getting the coverage up on this branch and then it would be great to discuss the different implementations tomorrow, if you have time.

NicolaCourtier · 2024-06-13T10:11:07Z

There will be some conflicts between the changes to parameters.as_dict() in this branch and the change from parameters to inputs in #358. Merging #358 (when ready) may help to clean up the implementation in this branch.

NicolaCourtier · 2024-06-13T18:30:25Z

I have merged #359 into this one and resolved the conflicts. Merging the same into #338 will significantly reduce the number of files changed.

BradyPlanden

Thanks @NicolaCourtier, I've added a few comments for me to address in #338. Will merge now.

BradyPlanden · 2024-06-28T10:11:37Z

pybop/costs/_likelihoods.py

+        self.parameters.join(self.sigma)
+
+        if dsigma_scale is None:
+            self._dsigma_scale = sigma0


Change in #338 to self._dsgima_scale == 1.0

BradyPlanden · 2024-06-28T10:13:13Z

pybop/costs/_likelihoods.py

+        problem_inputs = self.problem.parameters.as_dict()
+        y = self.problem.evaluate(problem_inputs)


Change in #338 to:

Suggested change

problem_inputs = self.problem.parameters.as_dict()

y = self.problem.evaluate(problem_inputs)

y = self.problem.evaluate(self.problem.parameters.as_dict())

BradyPlanden · 2024-06-28T10:13:37Z

pybop/costs/_likelihoods.py

+        problem_inputs = self.problem.parameters.as_dict()
+        y, dy = self.problem.evaluateS1(problem_inputs)


change in #338 to:

Suggested change

problem_inputs = self.problem.parameters.as_dict()

y, dy = self.problem.evaluateS1(problem_inputs)

y, dy = self.problem.evaluateS1(self.problem.parameters.as_dict())

BradyPlanden · 2024-06-28T10:14:55Z

pybop/costs/_likelihoods.py

        dsigma = (
-            -self.n_time_data / sigma + sigma ** (-3.0) * np.sum(r**2, axis=1)
+            -self.n_time_data / sigma + np.sum(r**2, axis=1) / (sigma**3)


change in #338 to:

Suggested change

-self.n_time_data / sigma + np.sum(r**2, axis=1) / (sigma**3)

-self.n_time_data / sigma + np.sum(r**2, axis=1) / (sigma**3.0)

BradyPlanden · 2024-06-28T10:15:14Z

pybop/costs/_likelihoods.py

        r = np.array([self._target[signal] - y[signal] for signal in self.signal])
-        likelihood = self._evaluate(x)
-        dl = np.sum((sigma ** (-2.0) * np.sum((r * dy.T), axis=2)), axis=1)
+        dl = np.sum((np.sum((r * dy.T), axis=2) / (sigma**2)), axis=1)


change in #338 to:

Suggested change

dl = np.sum((np.sum((r * dy.T), axis=2) / (sigma**2)), axis=1)

dl = np.sum((np.sum((r * dy.T), axis=2) / (sigma**2.0)), axis=1)

BradyPlanden · 2024-06-28T10:17:52Z

pybop/costs/_likelihoods.py

+        # Compute a finite difference approximation of the gradient of the log prior
+        delta = 1e-3
+        dl_prior_approx = [
+            (
+                param.prior.logpdf(inputs[param.name] * (1 + delta))
+                - param.prior.logpdf(inputs[param.name] * (1 - delta))
+            )
+            / (2 * delta * inputs[param.name] + np.finfo(float).eps)
+            for param in self.problem.parameters
+        ]
+


Change in #340, or split the prior update for evaluate and evaluate_S1 into separate PR.

BradyPlanden · 2024-06-28T10:20:24Z

pybop/models/base_model.py

-            Input parameters for the simulation. If the input is array-like, it is converted
-            to a dictionary using the model's fitting keys. Defaults to None, indicating
-            that the default parameters should be used.
+        inputs : Inputse, optional


Update in #338:

Suggested change

inputs : Inputse, optional

inputs : Inputs, optional

BradyPlanden · 2024-06-28T10:26:30Z

tests/integration/test_spm_parameterisations.py

@@ -72,7 +72,7 @@ def spm_costs(self, model, parameters, cost_class, init_soc):
        if cost_class in [pybop.GaussianLogLikelihoodKnownSigma]:
            return cost_class(problem, sigma0=0.002)
        elif cost_class in [pybop.GaussianLogLikelihood]:
-            return cost_class(problem)
+            return cost_class(problem, sigma0=0.002 * 3)


Update in #338 to:

Suggested change

return cost_class(problem, sigma0=0.002 * 3)

return cost_class(problem, sigma0=0.002 * 3) # Initial sigma0 guess

BradyPlanden · 2024-06-28T10:41:04Z

tests/unit/test_likelihoods.py

@@ -127,7 +132,25 @@ def test_gaussian_log_likelihood(self, one_signal_problem):
        grad_result, grad_likelihood = likelihood.evaluateS1(np.array([0.5, 0.5]))
        assert isinstance(result, float)
        np.testing.assert_allclose(result, grad_result, atol=1e-5)
-        assert np.all(grad_likelihood <= 0)
+        assert grad_likelihood[0] <= 0  # TEMPORARY WORKAROUND


Update in #338

NicolaCourtier added 3 commits June 7, 2024 16:08

Add fit_keys and sigma as a Parameter

7095049

Update CMAES x0 check

7e4cc7f

Add plot2d warning if not 2 parameters

8833c00

NicolaCourtier mentioned this pull request Jun 7, 2024

Updates to likelihoods, bug fix GaussianLogLikelihood class #338

Merged

15 tasks

NicolaCourtier added 4 commits June 7, 2024 16:20

Fix integration tests' get_data

23847e4

Update exp_UKF example

22301e6

Fix case with sigma length 1

341ad54

Reset parameters not None checks

ff9ce43

NicolaCourtier and others added 3 commits June 7, 2024 21:11

Fix sigma2 in GLLKnownSigma

3641a47

Update dsigma_scale

59f09b1

style: pre-commit fixes

76371ed

NicolaCourtier added 2 commits June 10, 2024 11:07

Fix GaussianLogLikelihoodKnownSigma

88b936c

Update plot2d for wrong number of parameters

57d1768

NicolaCourtier and others added 14 commits June 10, 2024 12:20

Fix classify_and_update_parameters

fa3a70b

Add get_initial_value

d2a6b5e

Update sigma setting and tests

e3251ca

style: pre-commit fixes

52ff78d

Merge branch 'gauss-log-like-fixes' into 338b-gauss-loglikelihood

1e6d149

Update optim trace for >2 parameters

21f44a6

Add test_scipy_minimize_invalid_x0

6793c3d

Add log prior gradient

c173c69

Add optimiser.parameters, remove problem.x0

dfdc0c5

Update integration tests

3315cc0

Pass inputs instead of x

20b7822

Specify inputs as Inputs

51e8c7c

Update notebooks

2b92ea9

Add initial and true options to as_dict

9121833

NicolaCourtier added 3 commits June 12, 2024 13:09

Reset notebook versions

c6553f4

Update parameter_values to inputs

fed62f6

Update notebooks

799122a

NicolaCourtier requested a review from BradyPlanden June 13, 2024 10:05

NicolaCourtier added 4 commits June 13, 2024 11:41

Add parameters tests

07e90ad

Add quick_plot test

63dd1f4

Add test_no_optimisation_parameters

6bcb155

Add test_error_in_cost_calculation

e402e38

This was referenced Jun 13, 2024

Pass Inputs dictionary #359

Merged

Align the type of inputs passed from cost to problem and model #358

Closed

NicolaCourtier added 8 commits June 13, 2024 16:36

Add parameters.verify

2adbdce

Fix change to base_model

10df0d2

Add more base_model tests

467f1f4

Merge branch '358-passing-inputs' into 338b-gauss-loglikelihood

d6645cd

Update base_cost.py

a8ee7cb

Merge branch 'gauss-log-like-fixes' into 338b-gauss-loglikelihood

10314d7

Remove fit_keys

d3c4f1b

Update base_model.py

e6d359a

NicolaCourtier and others added 6 commits June 14, 2024 10:40

Replace store_optimised_parameters with update

85788dd

Update value() output to ndarray

17c44ef

Update likelihood inputs

6d2776a

Merge branch 'gauss-log-like-fixes' into 338b-gauss-loglikelihood

0148bec

style: pre-commit fixes

ae29165

Re-implement get item check

0e75c8f

BradyPlanden approved these changes Jun 28, 2024

View reviewed changes

BradyPlanden merged commit 5c99037 into gauss-log-like-fixes Jun 28, 2024
29 checks passed

NicolaCourtier deleted the 338b-gauss-loglikelihood branch July 5, 2024 11:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Suggesting changes to #338 #352

Suggesting changes to #338 #352

NicolaCourtier commented Jun 7, 2024

BradyPlanden commented Jun 7, 2024

codecov bot commented Jun 7, 2024 •

edited

Loading

NicolaCourtier commented Jun 10, 2024

NicolaCourtier commented Jun 13, 2024

NicolaCourtier commented Jun 13, 2024

BradyPlanden left a comment

BradyPlanden Jun 28, 2024

BradyPlanden Jun 28, 2024

BradyPlanden Jun 28, 2024

BradyPlanden Jun 28, 2024

BradyPlanden Jun 28, 2024

BradyPlanden Jun 28, 2024

BradyPlanden Jun 28, 2024

BradyPlanden Jun 28, 2024

BradyPlanden Jun 28, 2024

		problem_inputs = self.problem.parameters.as_dict()
		y = self.problem.evaluate(problem_inputs)

	problem_inputs = self.problem.parameters.as_dict()
	y = self.problem.evaluate(problem_inputs)
	y = self.problem.evaluate(self.problem.parameters.as_dict())

		problem_inputs = self.problem.parameters.as_dict()
		y, dy = self.problem.evaluateS1(problem_inputs)

	problem_inputs = self.problem.parameters.as_dict()
	y, dy = self.problem.evaluateS1(problem_inputs)
	y, dy = self.problem.evaluateS1(self.problem.parameters.as_dict())

	-self.n_time_data / sigma + np.sum(r2, axis=1) / (sigma3)
	-self.n_time_data / sigma + np.sum(r2, axis=1) / (sigma3.0)

	dl = np.sum((np.sum((r * dy.T), axis=2) / (sigma**2)), axis=1)
	dl = np.sum((np.sum((r * dy.T), axis=2) / (sigma**2.0)), axis=1)

	return cost_class(problem, sigma0=0.002 * 3)
	return cost_class(problem, sigma0=0.002 * 3) # Initial sigma0 guess

Suggesting changes to #338 #352

Suggesting changes to #338 #352

Conversation

NicolaCourtier commented Jun 7, 2024

Description

Issue reference

Review

Type of change

Key checklist:

Further checks:

BradyPlanden commented Jun 7, 2024

codecov bot commented Jun 7, 2024 • edited Loading

Codecov Report

NicolaCourtier commented Jun 10, 2024

NicolaCourtier commented Jun 13, 2024

NicolaCourtier commented Jun 13, 2024

BradyPlanden left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jun 7, 2024 •

edited

Loading