-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ataymano/c wrapper fix2 #1859
Ataymano/c wrapper fix2 #1859
Conversation
@@ -69,6 +69,7 @@ class TC_parser | |||
bool* spelling_features; | |||
v_array<char> spelling; | |||
uint32_t hash_seed; | |||
uint64_t parse_mask; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you be able to explain what the parse_mask is for?
@@ -120,7 +126,9 @@ extern "C" | |||
VW_DLL_MEMBER float VW_CALLING_CONV VW_Learn(VW_HANDLE handle, VW_EXAMPLE e); | |||
VW_DLL_MEMBER float VW_CALLING_CONV VW_Predict(VW_HANDLE handle, VW_EXAMPLE e); | |||
VW_DLL_MEMBER float VW_CALLING_CONV VW_PredictCostSensitive(VW_HANDLE handle, VW_EXAMPLE e); | |||
//deprecated. Please use either VW_ReadExample for parsing, or VW_ImportExample for example construction |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not deprecated, right?
{ vw * pointer = static_cast<vw*>(handle); | ||
VW::primitive_feature_space * f = reinterpret_cast<VW::primitive_feature_space*>( features ); | ||
return static_cast<VW_EXAMPLE>(VW::import_example(*pointer, label, f, len)); | ||
} | ||
|
||
VW_DLL_MEMBER VW_FEATURE_SPACE VW_CALLING_CONV VW_InitializeFeatureSpaces(size_t len) | ||
{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are these needed precisely?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In other words, what is the missing functionality amongst existing interfaces?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With existing API the only way to "construct" example - VwExample ImportExample(FeatureSpace, Label);
But the only way to get feature space - FeatureSpace ExportExample(VwExample).
So I do not see any way to construct example without introducing functions required for initializing, adding features / namespaces etc to feature space. Please correct me if I am wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess the question is: What is the api? Is it vwdll.cpp? Or is it vw.h? To the extent that vwdll.cpp requires additional functions exposing vw.h that seems fine. To the extent that vw.h is missing things, I'd like to understand more deeply. Is vw.h missing things? If not, can these extra interface functions just call the vw.h interface?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My understanding that cpp api is vw.h while c one is vwdll.
My previous comment was about vwdll.
vw.h is not missing anything - at least allows to create example, since it is exposing full primitive_feature_space implementation:
vowpal_wabbit/vowpalwabbit/vw.h
Line 54 in 9206466
struct primitive_feature_space // just a helper definition. |
So given current vw.h code, direct manipulations with primitive_feature space is technically "using vw.h interface".
vw.h can be refactored to hide its feature_space implementation details behind the interfaces, but still looks like corresponding flat c methods have to be added to vwdll.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good. So, can we make sure the added vwdll functions use functions in vw.h? Currently, it seems they are not, so this is sort-of creating an alternate interface.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, let me try.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sent the update draft, please take a look.
This one is breaking on a python test related to lrq which needs to be updated. I'm also hazy on a detail: why do we need the extra functions in the interface? It does seem plausible we could easily get this one done quickly. |
We need to update this test: https://github.com/VowpalWabbit/vowpal_wabbit/blob/master/python/tests/test_sklearn_vw.py#L144 as well, given the implied changes t lrq. |
And it looks like a C# test is failing, although the reason why is unclear... https://ci.appveyor.com/project/JohnLangford/vowpal-wabbit/builds/25040837#L1329 |
Yes, I am looking into this - was not able to reproduce locally so far |
I have seen some flakiness out of this test in the past. |
* tag '8.7.0': (354 commits) Update version to 8.7.0 (VowpalWabbit#1926) Fix misconfiguration (VowpalWabbit#1925) Ataymano ataymano/warnings fixes (VowpalWabbit#1924) Update new version script (VowpalWabbit#1922) Run clang-format over codebase (VowpalWabbit#1921) change semantics of lambda (VowpalWabbit#1920) Bremen79 fix save ftrl (VowpalWabbit#1919) fix for daemon race condition (VowpalWabbit#1918) Revert "Fix to save the state of FTRL models (VowpalWabbit#1912)" (VowpalWabbit#1916) fix static library build (VowpalWabbit#1913) more warnings (VowpalWabbit#1915) Fix to save the state of FTRL models (VowpalWabbit#1912) remove warnings (VowpalWabbit#1911) fix closing invalid file descriptor with memory_io_buf (VowpalWabbit#1910) Optional exception (VowpalWabbit#1906) Contextual Memory Tree (VowpalWabbit#1799) Coin betting (VowpalWabbit#1903) Ataymano/c wrapper fix2 (VowpalWabbit#1859) Use Appveyor MSBuildLogger (VowpalWabbit#1904) fix for no label confidence (VowpalWabbit#1901) ...
* releases: (354 commits) Update version to 8.7.0 (VowpalWabbit#1926) Fix misconfiguration (VowpalWabbit#1925) Ataymano ataymano/warnings fixes (VowpalWabbit#1924) Update new version script (VowpalWabbit#1922) Run clang-format over codebase (VowpalWabbit#1921) change semantics of lambda (VowpalWabbit#1920) Bremen79 fix save ftrl (VowpalWabbit#1919) fix for daemon race condition (VowpalWabbit#1918) Revert "Fix to save the state of FTRL models (VowpalWabbit#1912)" (VowpalWabbit#1916) fix static library build (VowpalWabbit#1913) more warnings (VowpalWabbit#1915) Fix to save the state of FTRL models (VowpalWabbit#1912) remove warnings (VowpalWabbit#1911) fix closing invalid file descriptor with memory_io_buf (VowpalWabbit#1910) Optional exception (VowpalWabbit#1906) Contextual Memory Tree (VowpalWabbit#1799) Coin betting (VowpalWabbit#1903) Ataymano/c wrapper fix2 (VowpalWabbit#1859) Use Appveyor MSBuildLogger (VowpalWabbit#1904) fix for no label confidence (VowpalWabbit#1901) ...
* dfsg: (354 commits) Update version to 8.7.0 (VowpalWabbit#1926) Fix misconfiguration (VowpalWabbit#1925) Ataymano ataymano/warnings fixes (VowpalWabbit#1924) Update new version script (VowpalWabbit#1922) Run clang-format over codebase (VowpalWabbit#1921) change semantics of lambda (VowpalWabbit#1920) Bremen79 fix save ftrl (VowpalWabbit#1919) fix for daemon race condition (VowpalWabbit#1918) Revert "Fix to save the state of FTRL models (VowpalWabbit#1912)" (VowpalWabbit#1916) fix static library build (VowpalWabbit#1913) more warnings (VowpalWabbit#1915) Fix to save the state of FTRL models (VowpalWabbit#1912) remove warnings (VowpalWabbit#1911) fix closing invalid file descriptor with memory_io_buf (VowpalWabbit#1910) Optional exception (VowpalWabbit#1906) Contextual Memory Tree (VowpalWabbit#1799) Coin betting (VowpalWabbit#1903) Ataymano/c wrapper fix2 (VowpalWabbit#1859) Use Appveyor MSBuildLogger (VowpalWabbit#1904) fix for no label confidence (VowpalWabbit#1901) ...
c wrapper: