Ataymano/c wrapper fix2 #1859

ataymano · 2019-05-02T13:54:01Z

c wrapper:

missing feature space construction functions
Add label deprecation comments
parse_mask usage unification

jackgerrits · 2019-05-02T16:27:39Z

vowpalwabbit/parse_example.cc

@@ -69,6 +69,7 @@ class TC_parser
  bool* spelling_features;
  v_array<char> spelling;
  uint32_t hash_seed;
+  uint64_t parse_mask;


Would you be able to explain what the parse_mask is for?

JohnLangford · 2019-06-03T13:14:35Z

vowpalwabbit/vwdll.h

@@ -120,7 +126,9 @@ extern "C"
  VW_DLL_MEMBER float VW_CALLING_CONV VW_Learn(VW_HANDLE handle, VW_EXAMPLE e);
  VW_DLL_MEMBER float VW_CALLING_CONV VW_Predict(VW_HANDLE handle, VW_EXAMPLE e);
  VW_DLL_MEMBER float VW_CALLING_CONV VW_PredictCostSensitive(VW_HANDLE handle, VW_EXAMPLE e);
+  //deprecated. Please use either VW_ReadExample for parsing, or VW_ImportExample for example construction


Not deprecated, right?

JohnLangford · 2019-06-03T13:15:30Z

vowpalwabbit/vwdll.cpp

 { vw * pointer = static_cast<vw*>(handle);
  VW::primitive_feature_space * f = reinterpret_cast<VW::primitive_feature_space*>( features );
  return static_cast<VW_EXAMPLE>(VW::import_example(*pointer, label, f, len));
 }

+VW_DLL_MEMBER VW_FEATURE_SPACE VW_CALLING_CONV VW_InitializeFeatureSpaces(size_t len)
+{


Why are these needed precisely?

In other words, what is the missing functionality amongst existing interfaces?

With existing API the only way to "construct" example - VwExample ImportExample(FeatureSpace, Label);
But the only way to get feature space - FeatureSpace ExportExample(VwExample).
So I do not see any way to construct example without introducing functions required for initializing, adding features / namespaces etc to feature space. Please correct me if I am wrong.

I guess the question is: What is the api? Is it vwdll.cpp? Or is it vw.h? To the extent that vwdll.cpp requires additional functions exposing vw.h that seems fine. To the extent that vw.h is missing things, I'd like to understand more deeply. Is vw.h missing things? If not, can these extra interface functions just call the vw.h interface?

My understanding that cpp api is vw.h while c one is vwdll.
My previous comment was about vwdll.
vw.h is not missing anything - at least allows to create example, since it is exposing full primitive_feature_space implementation:

vowpal_wabbit/vowpalwabbit/vw.h

Line 54 in 9206466

struct primitive_feature_space // just a helper definition.

So given current vw.h code, direct manipulations with primitive_feature space is technically "using vw.h interface".
vw.h can be refactored to hide its feature_space implementation details behind the interfaces, but still looks like corresponding flat c methods have to be added to vwdll.

Good. So, can we make sure the added vwdll functions use functions in vw.h? Currently, it seems they are not, so this is sort-of creating an alternate interface.

Sure, let me try.

Sent the update draft, please take a look.

JohnLangford · 2019-06-03T13:17:37Z

This one is breaking on a python test related to lrq which needs to be updated.

I'm also hazy on a detail: why do we need the extra functions in the interface?

It does seem plausible we could easily get this one done quickly.

This reverts commit f24bb6d.

This reverts commit 36bf448.

This reverts commit b9c334a.

JohnLangford · 2019-06-04T19:37:10Z

We need to update this test: https://github.com/VowpalWabbit/vowpal_wabbit/blob/master/python/tests/test_sklearn_vw.py#L144 as well, given the implied changes t lrq.

JohnLangford · 2019-06-04T19:39:06Z

And it looks like a C# test is failing, although the reason why is unclear... https://ci.appveyor.com/project/JohnLangford/vowpal-wabbit/builds/25040837#L1329

ataymano · 2019-06-04T20:03:47Z

And it looks like a C# test is failing, although the reason why is unclear... https://ci.appveyor.com/project/JohnLangford/vowpal-wabbit/builds/25040837#L1329

Yes, I am looking into this - was not able to reproduce locally so far

lokitoth · 2019-06-04T20:13:07Z

And it looks like a C# test is failing, although the reason why is unclear... https://ci.appveyor.com/project/JohnLangford/vowpal-wabbit/builds/25040837#L1329

Yes, I am looking into this - was not able to reproduce locally so far

I have seen some flakiness out of this test in the past.

…/vowpal_wabbit into ataymano/c_wrapper_fix2

* tag '8.7.0': (354 commits) Update version to 8.7.0 (VowpalWabbit#1926) Fix misconfiguration (VowpalWabbit#1925) Ataymano ataymano/warnings fixes (VowpalWabbit#1924) Update new version script (VowpalWabbit#1922) Run clang-format over codebase (VowpalWabbit#1921) change semantics of lambda (VowpalWabbit#1920) Bremen79 fix save ftrl (VowpalWabbit#1919) fix for daemon race condition (VowpalWabbit#1918) Revert "Fix to save the state of FTRL models (VowpalWabbit#1912)" (VowpalWabbit#1916) fix static library build (VowpalWabbit#1913) more warnings (VowpalWabbit#1915) Fix to save the state of FTRL models (VowpalWabbit#1912) remove warnings (VowpalWabbit#1911) fix closing invalid file descriptor with memory_io_buf (VowpalWabbit#1910) Optional exception (VowpalWabbit#1906) Contextual Memory Tree (VowpalWabbit#1799) Coin betting (VowpalWabbit#1903) Ataymano/c wrapper fix2 (VowpalWabbit#1859) Use Appveyor MSBuildLogger (VowpalWabbit#1904) fix for no label confidence (VowpalWabbit#1901) ...

* releases: (354 commits) Update version to 8.7.0 (VowpalWabbit#1926) Fix misconfiguration (VowpalWabbit#1925) Ataymano ataymano/warnings fixes (VowpalWabbit#1924) Update new version script (VowpalWabbit#1922) Run clang-format over codebase (VowpalWabbit#1921) change semantics of lambda (VowpalWabbit#1920) Bremen79 fix save ftrl (VowpalWabbit#1919) fix for daemon race condition (VowpalWabbit#1918) Revert "Fix to save the state of FTRL models (VowpalWabbit#1912)" (VowpalWabbit#1916) fix static library build (VowpalWabbit#1913) more warnings (VowpalWabbit#1915) Fix to save the state of FTRL models (VowpalWabbit#1912) remove warnings (VowpalWabbit#1911) fix closing invalid file descriptor with memory_io_buf (VowpalWabbit#1910) Optional exception (VowpalWabbit#1906) Contextual Memory Tree (VowpalWabbit#1799) Coin betting (VowpalWabbit#1903) Ataymano/c wrapper fix2 (VowpalWabbit#1859) Use Appveyor MSBuildLogger (VowpalWabbit#1904) fix for no label confidence (VowpalWabbit#1901) ...

* dfsg: (354 commits) Update version to 8.7.0 (VowpalWabbit#1926) Fix misconfiguration (VowpalWabbit#1925) Ataymano ataymano/warnings fixes (VowpalWabbit#1924) Update new version script (VowpalWabbit#1922) Run clang-format over codebase (VowpalWabbit#1921) change semantics of lambda (VowpalWabbit#1920) Bremen79 fix save ftrl (VowpalWabbit#1919) fix for daemon race condition (VowpalWabbit#1918) Revert "Fix to save the state of FTRL models (VowpalWabbit#1912)" (VowpalWabbit#1916) fix static library build (VowpalWabbit#1913) more warnings (VowpalWabbit#1915) Fix to save the state of FTRL models (VowpalWabbit#1912) remove warnings (VowpalWabbit#1911) fix closing invalid file descriptor with memory_io_buf (VowpalWabbit#1910) Optional exception (VowpalWabbit#1906) Contextual Memory Tree (VowpalWabbit#1799) Coin betting (VowpalWabbit#1903) Ataymano/c wrapper fix2 (VowpalWabbit#1859) Use Appveyor MSBuildLogger (VowpalWabbit#1904) fix for no label confidence (VowpalWabbit#1901) ...

ataymano added 5 commits April 29, 2019 09:09

vwdll example construction update. first version

3921e94

cleanup

9aaa1a7

parse_mask usage unification

c0f7090

wrapper test

8bf8ba9

deprecation comment

a02b6a8

jackgerrits reviewed May 2, 2019

View reviewed changes

jackgerrits approved these changes May 2, 2019

View reviewed changes

test fix

2f8e1dc

ataymano and others added 3 commits May 15, 2019 19:15

Merge branch 'master' into ataymano/c_wrapper_fix2

78eff1a

travis unit test build fix

2f582f3

unit test 151 fix

dfe5345

JohnLangford reviewed Jun 3, 2019

View reviewed changes

ataymano@microsoft.com and others added 8 commits June 4, 2019 13:15

vw.h example construction helpers first version

b9c334a

get rid of initializers

36bf448

get rid of initializers 2

f24bb6d

Revert "get rid of initializers 2"

9c01912

This reverts commit f24bb6d.

Revert "get rid of initializers"

8eb9868

This reverts commit 36bf448.

Revert "vw.h example construction helpers first version"

4f6bf78

This reverts commit b9c334a.

adding init / set features methods to vw.h

26fbaec

Merge branch 'master' into ataymano/c_wrapper_fix2

d72a03c

ataymano@microsoft.com and others added 3 commits June 4, 2019 17:05

fix lrq test

a05a80b

Merge branch 'ataymano/c_wrapper_fix2' of https://github.com/ataymano…

8b950a0

…/vowpal_wabbit into ataymano/c_wrapper_fix2

Merge branch 'master' into ataymano/c_wrapper_fix2

b83a541

JohnLangford merged commit 599f4d3 into VowpalWabbit:master Jun 4, 2019

jackgerrits mentioned this pull request Aug 22, 2019

Logistic regression C app stopped working between releases 8.5.0 and 8.6.0 #1780

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ataymano/c wrapper fix2 #1859

Ataymano/c wrapper fix2 #1859

ataymano commented May 2, 2019

jackgerrits May 2, 2019

JohnLangford Jun 3, 2019

JohnLangford Jun 3, 2019

JohnLangford Jun 3, 2019

ataymano Jun 3, 2019

JohnLangford Jun 3, 2019

ataymano Jun 3, 2019 •

edited

Loading

JohnLangford Jun 3, 2019

ataymano Jun 3, 2019

ataymano Jun 4, 2019

JohnLangford commented Jun 3, 2019

JohnLangford commented Jun 4, 2019

JohnLangford commented Jun 4, 2019

ataymano commented Jun 4, 2019

lokitoth commented Jun 4, 2019

Ataymano/c wrapper fix2 #1859

Ataymano/c wrapper fix2 #1859

Conversation

ataymano commented May 2, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ataymano Jun 3, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JohnLangford commented Jun 3, 2019

JohnLangford commented Jun 4, 2019

JohnLangford commented Jun 4, 2019

ataymano commented Jun 4, 2019

lokitoth commented Jun 4, 2019

ataymano Jun 3, 2019 •

edited

Loading