`options_i` command line parsing refactor #1706

jackgerrits · 2019-01-08T14:38:16Z

Add options_i interface and options_boost_po implementation for options parsing/storing/querying
Add options_serializer_i interface and options_serializer_boost_po implementation for serializing an options_i collection
Migrate 45 reductions and setup tasks to new options workflow
Default options are no longer serialized, updated several tests to accommodate this
Update loss function to advertise type rather than reading the command line argument

initialize parse_args DONE parse_diagnostics DONE parse_regressor_args parse_modules save_load_header (hard) check_interaction_settings_collision parse_feature_tweaks parse_example_tweaks parse_output_model parse_output_preds parse_reductions (large surface) parse_sources load_input_model parse_mask_regressor_args TODO clear file options parse_source (positional param) DONE enable_sources DONE parse_cache DONE all reductions

Migration progress: initialize parse_args DONE parse_diagnostics DONE parse_regressor_args parse_modules save_load_header (hard) DONE check_interaction_settings_collision DONE parse_feature_tweaks parse_example_tweaks parse_output_model parse_output_preds parse_reductions (large surface) parse_sources load_input_model parse_mask_regressor_args TODO clear file options parse_source (positional param) DONE enable_sources DONE parse_cache DONE all reductions

JohnLangford · 2019-01-15T14:39:05Z

This pull request fixes 2 alerts when merging 4dec969 into 2b64c3c - view on LGTM.com

fixed alerts:

2 for Inconsistent definition of copy constructor and assignment ('Rule of Two')

Comment posted by LGTM.com

This reverts commit 89b5959.

JohnLangford · 2019-01-15T18:43:16Z

This pull request fixes 2 alerts when merging 9b5215c into 2b64c3c - view on LGTM.com

fixed alerts:

2 for Inconsistent definition of copy constructor and assignment ('Rule of Two')

Comment posted by LGTM.com

JohnLangford · 2019-01-11T20:14:14Z

vowpalwabbit/parse_args.h

+
+// Used in parse_source
+struct input_options {
+  bool daemon;


Why put this here? It's not otherwise used in this header.

This is an attempt to encapsulate related state, because it is an option parsed as part of this block I am going to keep it in the struct

Sorry, I was ambiguous. Why have "input_options" in the header? It's defining an unused type.

JohnLangford · 2019-01-11T20:57:39Z

cs/cli/vowpalwabbit.cpp

-  { hash_function = m_vw->opts_n_args.vm["hash"].as<string>();
+  if (m_vw->options->was_supplied("hash"))
+  {
+    hash_function = m_vw->options->get_typed_option<string>("hash").value();


->get_typed_option

test/pred-sets/ref/inv_hash.cmp

vowpalwabbit/OjaNewton.cc

JohnLangford · 2019-01-11T21:03:10Z

vowpalwabbit/autolink.cc


  LEARNER::learner<autolink,example>& ret =
-    init_learner(data, as_singleline(setup_base(arg)), predict_or_learn<true>, predict_or_learn<false>);
+    init_learner(data, as_singleline(setup_base(*all.options, all)), predict_or_learn<true>, predict_or_learn<false>);


Could we pass 'options' instead of *all.options? That at least looks somewhat less silly.

JohnLangford · 2019-01-11T21:04:29Z

vowpalwabbit/global_data.h

  std::vector<std::string> dictionary_path;  // where to look for dictionaries
  std::vector<feature_dict*> namespace_dictionaries[256]; // each namespace has a list of dictionaries attached to it
  std::vector<dictionary_info> loaded_dictionaries; // which dictionaries have we loaded from a file to memory?

  void(*delete_prediction)(void*); bool audit; //should I print lots of debugging information?
  bool quiet;//Should I suppress progress-printing of updates?
+  bool help_requested;


Isn't the intent saved within the parsed options itself?

JohnLangford · 2019-01-11T21:05:06Z

vowpalwabbit/global_data.h

@@ -552,6 +559,7 @@ struct vw
  uint32_t holdout_period;
  uint32_t holdout_after;
  size_t check_holdout_every_n_passes;  // default: 1, but search might want to set it higher if you spend multiple passes learning a single policy
+  size_t early_terminate_passes;


Maybe just stick it explicitly into each reduction needing it?

vowpalwabbit/options.cc

JohnLangford · 2019-01-11T21:06:52Z

vowpalwabbit/parse_dispatch_loop.h

@@ -1,5 +1,7 @@
 #pragma once

+#include <functional>
+


Required for std::function on line 5

JohnLangford · 2019-01-15T19:37:01Z

cs/cli/vowpalwabbit.cpp

-  { hash_function = m_vw->opts_n_args.vm["hash"].as<string>();
+  if (m_vw->options->was_supplied("hash"))
+  {
+    hash_function = m_vw->options->get_typed_option<string>("hash").value();


Ping on this.

JohnLangford · 2019-01-15T21:19:18Z

This pull request fixes 2 alerts when merging dce63d8 into 2b64c3c - view on LGTM.com

fixed alerts:

2 for Inconsistent definition of copy constructor and assignment ('Rule of Two')

Comment posted by LGTM.com

JohnLangford · 2019-01-15T21:47:28Z

This pull request fixes 2 alerts when merging fa7f6d3 into 2b64c3c - view on LGTM.com

fixed alerts:

2 for Inconsistent definition of copy constructor and assignment ('Rule of Two')

Comment posted by LGTM.com

JohnLangford · 2019-01-16T13:14:40Z

This pull request fixes 2 alerts when merging 9337bed into 6dbb7e5 - view on LGTM.com

fixed alerts:

2 for Inconsistent definition of copy constructor and assignment ('Rule of Two')

Comment posted by LGTM.com

JohnLangford · 2019-01-16T15:39:33Z

I think the pointer vs. member question is the only thing left to worry about here.

jackgerrits · 2019-01-16T15:45:13Z

I replied to that, I believe we should leave it as is, with an options_i* being in the all/vw object. Most usages of the options_i object are done through a reference (in every setup function), so this covers most exposure. The problem is that there is a situation where the all object must own the options_i struct rather than just using it. This case is when the user passes in a command line string rather than an already constructed options_i object. This is the primary legacy use case. Since it must own the object in this case, we need to delete it when it's done. Storing it as a reference really hides this use case, and given it is the most likely way this will be used I think we want to optimize understanding for that.

JohnLangford · 2019-01-16T17:53:31Z

Merged, thanks :-) That's quite a monster.

arielf · 2019-01-17T18:11:49Z

Hi there. Indeed a big change. Thanks for all the effort!

I've been testing this a bit and I'm seeing another backward incompatibility break with --quiet:

$ ./vowpalwabbit/vw --quiet f20-315.tt.gz

finished run
number of examples = 0
weighted example sum = 0.000000
weighted label sum = 0.000000
average loss = n.a.
total feature number = 0
the argument ('f20-315.tt.gz') for option '--quiet' is invalid. Valid choices are 'on|off', 'yes|no', '1|0' and 'true|false'

Would it be possible to make --quiet backward-compatible in the sense that it passed naked, it defaults to =yes and doesn't cause an abort/error ?

Thanks!

jackgerrits · 2019-01-17T18:18:04Z

This isn’t restricted to quiet, but any Boolean option as we are expecting a following value. We can force Boolean options to be a switch (rather than an option that takes a value), but then it is not possible to set an option to be false. This is one of the tricky cases using a positional data option introduces. Currently Boolean options take an implicit value of true if the following token is an option, but a following value token is interpreted as a value for the Boolean option. The positional data value trips this up.

JohnLangford · 2019-01-18T13:15:33Z

Boost manages to get this to work. Why can't we?

* Initial framework commit of options refactor * Rename options -> arguments, option -> parameter * equality unit test * Working on tests * Add kept implementation * revert line ending change * Update cmake and fix build * Update arg to hold value, expose arg list and merging * Rename arguments to options * rename files arguments -> options * Add config namespace * Change back to non-duplicates and use references * Migration progress: initialize parse_args DONE parse_diagnostics DONE parse_regressor_args parse_modules save_load_header (hard) check_interaction_settings_collision parse_feature_tweaks parse_example_tweaks parse_output_model parse_output_preds parse_reductions (large surface) parse_sources load_input_model parse_mask_regressor_args TODO clear file options parse_source (positional param) DONE enable_sources DONE parse_cache DONE all reductions * Migrate model header load Migration progress: initialize parse_args DONE parse_diagnostics DONE parse_regressor_args parse_modules save_load_header (hard) DONE check_interaction_settings_collision DONE parse_feature_tweaks parse_example_tweaks parse_output_model parse_output_preds parse_reductions (large surface) parse_sources load_input_model parse_mask_regressor_args TODO clear file options parse_source (positional param) DONE enable_sources DONE parse_cache DONE all reductions * Migrate parse_feature_tweaks * migrate parse_example_tweaks * migrate parse_output_model and parse_output_preds * Change to using reference instead of pointer for options object * Migrate marginal reduction * Change all signatures to include options_i and vw * migrate gd * Migrate kernel svm and fix keep arg in marginal * migrate ftrl reduction * migrate svrg reduction * migrate sender * migrate gd_mf * work in progress * Update loss_function to be able to dynamically identify type * work in progress * Finish migrating reductions, and search tasks * fix compile issues in vw_core * working through test fixes * fix bugs * Fix bug, update test model file * Fix tests, update cppcli wrapper, update c# unit tests, fix c# versions, remove parser_helper * Fix help printout, unregistered options check * Fix options leaked object * remove width of print for older boost version * update usage of options in python and remove program_options reference in Java * add move constructor, fix onethread unregistered issue and fix false negative unrecognized args * try not using move constructors * move to a unique ptr to deal with gcc4.9 * fix save resume * trigger ci * implement positional parameter for data and revert test changes * properly handle invalid args for Java * fix writing if incorrect cb_type to model * fix --opt=val style parsing from model file args * change handling of defaults for cb_type, change insert behavior for empty value string * Fix merge conflict, kvm loss function check is fixed in this PR * revert unwanted changes * revert Resource.rc removal * revert adding include * fix kernel_svn -> kernel_svm * Use const& for strings and larger objects in options * Fix lgtm alert about copy assignment operator * add back <algorithm> for min/max * using namespace VW::config; * fix java dependencies, update cpprestsdk build * Revert "fix java dependencies, update cpprestsdk build" This reverts commit 89b5959. * return reference from, rename make function * remove options.cc * use given reference for setup_base * remove fields from all

* Update vowpal_wabbit submodule * Pulls in these fixes - fac3c06 ("Fix crash when empty multi_ex is supplied for --cb_explore_adf (VowpalWabbit#1679)") - f085178 ("bugfix: cb_adf is not including some examples in stats calculation (VowpalWabbit#1686)") - c9d9af7 ("Fix memory leak in cb_explore_adf (VowpalWabbit#1698)") - 1e8742b ("Fix some VW initialization memory leaks (VowpalWabbit#1697)") - 75423e3 ("Fix best constant and best constant's loss calculation when using ksvm (VowpalWabbit#1704)") - 2b64c3c ("Use std::exp instead of exp free function (VowpalWabbit#1709)") - 6b7b160 ("Fix unused params warnings plus incomplete struct init (done to default values). (VowpalWabbit#1710)") - 2e1602d ("Compile fixes (VowpalWabbit#1713)") - b50fb59 ("Make cbify reduction respect is_learn parameter (VowpalWabbit#1722)") This also includes the option parsing changes - b8ab396 ("`options_i` command line parsing refactor (VowpalWabbit#1706)" - 5b020c4 ("Ataymano/mac options types fix (VowpalWabbit#1718)") There are other small or non-rl-consumable changes * Enable injection of msbuild path in Windows command-line build * Update vowpal_wabbit submodule * This pulls in the additional fix made to address the bug discovered in the previous attempt to update this. Fixed in VowpalWabbit#1742 * Updates safe_vw.cc to get rid of a no-longer-necessary cleanup step

jackgerrits added 30 commits December 17, 2018 11:37

Initial framework commit of options refactor

dbf5dee

Rename options -> arguments, option -> parameter

45627b0

equality unit test

2e4d6fb

Working on tests

aa35d8a

Add kept implementation

f20f73d

revert line ending change

8b3dacc

Update cmake and fix build

b12c3a9

Update arg to hold value, expose arg list and merging

a53364f

Rename arguments to options

f317655

rename files arguments -> options

196af06

Add config namespace

c548106

Change back to non-duplicates and use references

f2d8da8

Migrate parse_feature_tweaks

19f6087

migrate parse_example_tweaks

5e9951e

migrate parse_output_model and parse_output_preds

ddb0222

Change to using reference instead of pointer for options object

bd42a7e

Migrate marginal reduction

212f57e

Change all signatures to include options_i and vw

201f34a

migrate gd

de0408f

Migrate kernel svm and fix keep arg in marginal

3be311e

migrate ftrl reduction

020325c

migrate svrg reduction

c4ebae5

migrate sender

7982425

migrate gd_mf

969ddd7

work in progress

3c88834

Update loss_function to be able to dynamically identify type

1225797

work in progress

34f967a

Finish migrating reductions, and search tasks

8defa86

ataymano approved these changes Jan 15, 2019

View reviewed changes

jackgerrits added 2 commits January 15, 2019 13:13

fix java dependencies, update cpprestsdk build

89b5959

Revert "fix java dependencies, update cpprestsdk build"

9b5215c

This reverts commit 89b5959.

JohnLangford reviewed Jan 15, 2019

View reviewed changes

jackgerrits added 3 commits January 15, 2019 15:41

return reference from, rename make function

6d9cae3

remove options.cc

f3b022b

use given reference for setup_base

dce63d8

remove fields from all

fa7f6d3

Merge branch 'master' into jagerrit/migrate_options

9337bed

merge commit

2e32eb7

JohnLangford merged commit b8ab396 into VowpalWabbit:master Jan 16, 2019

jackgerrits mentioned this pull request Jan 18, 2019

Change to using a bool_switch for bool options #1717

Merged

This was referenced Jan 18, 2019

Issue with --save_resume option #1672

Closed

--save_resume bug prevents out-of-sample predictions #1573

Closed

lokitoth mentioned this pull request Jan 30, 2019

Update vowpal_wabbit submodule VowpalWabbit/reinforcement_learning#37

Merged

jackgerrits deleted the jagerrit/migrate_options branch April 5, 2019 16:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`options_i` command line parsing refactor #1706

`options_i` command line parsing refactor #1706

jackgerrits commented Jan 8, 2019 •

edited

Loading

JohnLangford commented Jan 15, 2019

JohnLangford commented Jan 15, 2019

JohnLangford Jan 11, 2019

jackgerrits Jan 15, 2019

JohnLangford Jan 15, 2019

JohnLangford Jan 11, 2019

JohnLangford Jan 11, 2019

JohnLangford Jan 11, 2019

JohnLangford Jan 11, 2019

JohnLangford Jan 11, 2019

jackgerrits Jan 15, 2019

JohnLangford Jan 15, 2019

JohnLangford commented Jan 15, 2019

JohnLangford commented Jan 15, 2019

JohnLangford commented Jan 16, 2019

JohnLangford commented Jan 16, 2019

jackgerrits commented Jan 16, 2019

JohnLangford commented Jan 16, 2019

arielf commented Jan 17, 2019

jackgerrits commented Jan 17, 2019

JohnLangford commented Jan 18, 2019

options_i command line parsing refactor #1706

options_i command line parsing refactor #1706

Conversation

jackgerrits commented Jan 8, 2019 • edited Loading

JohnLangford commented Jan 15, 2019

JohnLangford commented Jan 15, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JohnLangford commented Jan 15, 2019

JohnLangford commented Jan 15, 2019

JohnLangford commented Jan 16, 2019

JohnLangford commented Jan 16, 2019

jackgerrits commented Jan 16, 2019

JohnLangford commented Jan 16, 2019

arielf commented Jan 17, 2019

jackgerrits commented Jan 17, 2019

JohnLangford commented Jan 18, 2019

`options_i` command line parsing refactor #1706

`options_i` command line parsing refactor #1706

jackgerrits commented Jan 8, 2019 •

edited

Loading