Issue273 default values for configuration settings #324

juhoinkinen · 2019-08-29T13:29:15Z

Implemented one initial approach for discussion.

In this good is that default parameters can be included only in the relevant base classes for a backend (e.g. chunksize in ChunkingBackend). The Fasttext and VW params could be included to DEFAULT_PARAMS too (with a bit of work for passing only the backend specific params to the backends).

Not-so-good is maybe the location of this, should it be in project.py instead?

Another completely different approach could be to use to use ConfigParser's defaults implementation.

codecov · 2019-08-29T13:29:48Z

Codecov Report

Merging #324 into master will decrease coverage by 0.01%.
The diff coverage is 98.98%.

@@            Coverage Diff             @@
##           master     #324      +/-   ##
==========================================
- Coverage   99.57%   99.55%   -0.02%     
==========================================
  Files          56       56              
  Lines        3040     3149     +109     
==========================================
+ Hits         3027     3135     +108     
- Misses         13       14       +1

Impacted Files	Coverage Δ
tests/test_backend_http.py	`100% <ø> (ø)`	⬆️
annif/project.py	`100% <100%> (ø)`	⬆️
annif/backend/fasttext.py	`98.76% <100%> (+0.09%)`	⬆️
annif/backend/vw_ensemble.py	`100% <100%> (ø)`	⬆️
annif/backend/dummy.py	`100% <100%> (ø)`	⬆️
tests/test_backend_tfidf.py	`100% <100%> (ø)`	⬆️
tests/test_backend_vw_multi.py	`100% <100%> (ø)`	⬆️
annif/backend/vw_multi.py	`100% <100%> (ø)`	⬆️
tests/test_backend_vw_ensemble.py	`100% <100%> (ø)`	⬆️
tests/test_backend.py	`100% <100%> (ø)`	⬆️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 81994c4...d22cd2d. Read the comment docs.

lgtm-com · 2019-08-29T14:39:58Z

This pull request introduces 1 alert when merging 4ef0b5a into 2fb6108 - view on LGTM.com

new alerts:

1 for Conflicting attributes in base classes

lgtm-com · 2019-08-30T07:13:47Z

This pull request introduces 1 alert when merging f8825ed into 2fb6108 - view on LGTM.com

new alerts:

1 for Conflicting attributes in base classes

osma · 2019-09-02T14:05:43Z

I like the way backend classes (including mixins like ChunkingBackend) declare their own default parameters, but the method for looking them up by accessing the class object seems a bit overcomplicated. I'd prefer to use standard inheritance for this.

How about letting backend classes declare a property that, when accessed, returns the list of default parameter values. The implementation of the property method can then decide whether the defaults are inherited from superclasses or not. The simple case (inherited from AnnifBackend) would just return DEFAULT_PARAMS. Something like:

class AnnifBackend:

    DEFAULT_PARAMS = {'limit': 100}

    @property
    def default_params(self):
        return self.DEFAULT_PARAMS


class ChunkingBackend:

    DEFAULT_PARAMS = {'chunksize': 2}


class FastTextBackend(ChunkingBackend, AnnifBackend):

    DEFAULT_PARAMS = {'loss', 'hs'}  # and others
    
    @property
    def default_params(self):
        # Start with AnnifBackend defaults
        params = AnnifBackend.default_params(self).copy()
        # override with ChunkingBackend defaults
        params.update(ChunkingBackend.default_params(self))
        # override with our own defaults
        params.update(self.DEFAULT_PARAMS)
        return params

Okay, maybe this isn't the simplest method either, but at least backends have full control over which default parameters they wish to inherit from upper classes.

Note that FastTextBackend already has a FASTTEXT_PARAMS field that specifies the types of parameters it supports. VWMultiBackend and VWEnsembleBackend go even further with VW_PARAMS that also specifies some default values. Their implementation of the default_params property could make use of this information.

osma · 2019-09-02T14:06:40Z

On second thought, maybe @property is not helpful here. default_params could just be a plain method.

lgtm-com · 2019-09-04T15:13:27Z

This pull request introduces 1 alert when merging bf58247 into 6ac8431 - view on LGTM.com

new alerts:

1 for Conflicting attributes in base classes

…first place

juhoinkinen · 2019-09-18T09:43:57Z

annif/backend/backend.py

+    def default_params(self):
+        return self.DEFAULT_PARAMS
+
+    def fill_params_with_defaults(self, params):


Would the params field actually be better to be implemented as a property, and then this would be decorated with @params.setter?

I like the idea of params being a property. It could be implemented in such a way that it transparently fills in default parameters before returning a dict. And it could have a setter used to set manual parameters (which would be stored in an internal field, say _params). Actually, the filling in could also be done within the setter - then it would only happen once instead of on every access. Accessing the params property would then just return self._params or perhaps a copy() to be safe.

lgtm-com · 2019-09-25T13:49:24Z

This pull request introduces 1 alert when merging 420d3d1 into e449223 - view on LGTM.com

new alerts:

1 for __init__ method calls overridden method

…settings

lgtm-com · 2019-09-26T06:52:43Z

This pull request introduces 1 alert when merging 91136a6 into e449223 - view on LGTM.com

new alerts:

1 for __init__ method calls overridden method

juhoinkinen · 2019-09-26T08:59:05Z

This pull request introduces 1 alert when merging 91136a6 into e449223 - view on LGTM.com

new alerts:
* 1 for __init__ method calls overridden method

According to this thread the alert could be ignored.

osma · 2019-09-30T12:00:53Z

Regarding the LGTM alert __init__ method calls overridden method, I think it could be solved like this:

init method doesn't touch self.params, it just sets self.config_params = config_params
params becomes a @property method like this:

    @property
    def params(self):
        params = {}
        params.update(self.default_params())
        params.update(self.config_params)
        return params

With the above implementation of params, there should be no need to call .copy() anywhere.

The benefit of starting from an empty dict and using update, instead of taking a .copy(), is that the default_params() methods in backend subclasses may return anything dict-like, it doesn't have to be an actual dict.

annif/backend/dummy.py

osma

Fine to merge

juhoinkinen · 2019-10-01T09:42:45Z

Edited Wiki pages to shortly mention the parameter defaulting:

Project configuration page
Backend: fastText (drop fastText will choose default value)

juhoinkinen added 4 commits August 28, 2019 12:09

Default project name to project_id

aa592d8

Fill missing parameters with defaults when creating backend objects

6480bd8

Switch info about using default params to debug level

7146977

Get all parent/ancestor classes instead of only direct ones

4ef0b5a

juhoinkinen added 2 commits August 30, 2019 08:32

Use type to get class and thus avoid the magic method

f52a8bc

Refine debug message and docstring

f8825ed

juhoinkinen added this to the 0.43 milestone Sep 3, 2019

Simpler & more controllable parameter defaulting

bf58247

juhoinkinen added 7 commits September 5, 2019 11:17

Add default params for fasttext and PAV

ff7969b

Debug msg for using default params to one line

089ac38

Use default params for VW multi and ensemble

0104fbc

Make fill_params_with_defaults testable; use it to set params in the …

2781087

…first place

Default params for dummy BE and a test for params filling with it

2d9a891

Tests for backend params defaulting

a215af9

Skip test when fasttext not available

0fb125c

juhoinkinen marked this pull request as ready for review September 18, 2019 09:41

juhoinkinen commented Sep 18, 2019

View reviewed changes

juhoinkinen added 3 commits September 24, 2019 08:38

Make params a property with defaults setting

c037815

From SectionProxy obj to dict to be able to use .copy()

1843332

Rename params originating from projects.cfg for clarity

63a199c

osma mentioned this pull request Sep 24, 2019

Add nn_ensemble backend #331

Merged

Defaulting simply by updating default params with config params

420d3d1

Merge branch 'master' into issue273-default-values-for-configuration-…

0df9735

…settings

Adapt to PR #322

91136a6

Make config_params a field and params a property

157ecab

osma added the enhancement label Sep 30, 2019

osma reviewed Sep 30, 2019

View reviewed changes

annif/backend/dummy.py Outdated Show resolved Hide resolved

Drop copying of default_params

d22cd2d

osma approved these changes Sep 30, 2019

View reviewed changes

juhoinkinen merged commit 244db95 into master Sep 30, 2019

juhoinkinen deleted the issue273-default-values-for-configuration-settings branch September 30, 2019 13:48

juhoinkinen mentioned this pull request Jan 28, 2020

Default values for configuration settings #273

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue273 default values for configuration settings #324

Issue273 default values for configuration settings #324

juhoinkinen commented Aug 29, 2019

codecov bot commented Aug 29, 2019 •

edited

Loading

lgtm-com bot commented Aug 29, 2019

lgtm-com bot commented Aug 30, 2019

osma commented Sep 2, 2019

osma commented Sep 2, 2019

lgtm-com bot commented Sep 4, 2019

juhoinkinen Sep 18, 2019 •

edited

Loading

osma Sep 20, 2019

lgtm-com bot commented Sep 25, 2019

lgtm-com bot commented Sep 26, 2019

juhoinkinen commented Sep 26, 2019

osma commented Sep 30, 2019

osma left a comment

juhoinkinen commented Oct 1, 2019

Issue273 default values for configuration settings #324

Issue273 default values for configuration settings #324

Conversation

juhoinkinen commented Aug 29, 2019

codecov bot commented Aug 29, 2019 • edited Loading

Codecov Report

lgtm-com bot commented Aug 29, 2019

lgtm-com bot commented Aug 30, 2019

osma commented Sep 2, 2019

osma commented Sep 2, 2019

lgtm-com bot commented Sep 4, 2019

juhoinkinen Sep 18, 2019 • edited Loading

Choose a reason for hiding this comment

osma Sep 20, 2019

Choose a reason for hiding this comment

lgtm-com bot commented Sep 25, 2019

lgtm-com bot commented Sep 26, 2019

juhoinkinen commented Sep 26, 2019

osma commented Sep 30, 2019

osma left a comment

Choose a reason for hiding this comment

juhoinkinen commented Oct 1, 2019

codecov bot commented Aug 29, 2019 •

edited

Loading

juhoinkinen Sep 18, 2019 •

edited

Loading