Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement basic in-memory store and change handling. #165

Merged
merged 40 commits into from
Jul 12, 2023
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
95bab24
chore: Implement architecture diagram for data store.
kinyoklion Jul 6, 2023
a4b2c8c
Updates types for get methods on IDataStore.
kinyoklion Jul 6, 2023
d368988
Update collection types.
kinyoklion Jul 6, 2023
48bfc55
Merge branch 'server-side' into rlamb/data-store-arch
kinyoklion Jul 6, 2023
758e825
feat: Implement basic store.
kinyoklion Jul 6, 2023
6914703
Overloads for upsert and add extra const qualifiers.
kinyoklion Jul 6, 2023
c74f508
Merge branch 'rlamb/data-store-arch' of github.com:launchdarkly/cpp-s…
kinyoklion Jul 6, 2023
cd1ec6f
Merge branch 'rlamb/data-store-arch' into rlamb/basic-store-implement…
kinyoklion Jul 6, 2023
e82dc04
Basic data store and data source update sink interfaces.
kinyoklion Jul 6, 2023
54499d6
Start adding change notifier.
kinyoklion Jul 7, 2023
1df05f9
Merge branch 'server-side' into rlamb/basic-store-implementation
kinyoklion Jul 7, 2023
5221784
Add empty implementation file.
kinyoklion Jul 7, 2023
e9734bc
Start adding memory store
kinyoklion Jul 7, 2023
b53b004
Basic memory store functionaliy.
kinyoklion Jul 7, 2023
1d07b00
Add memory store tests.
kinyoklion Jul 7, 2023
58d284f
Add tests for dependency tracker.
kinyoklion Jul 10, 2023
f6b9558
Split implementation and cleanup clang issues.
kinyoklion Jul 10, 2023
d4a2a9d
Refactoring and continue implementation of the data store updater.
kinyoklion Jul 10, 2023
091bcb8
Code cleanup and more tests.
kinyoklion Jul 10, 2023
f3e0a5d
Add a test for using an old value after upsert.
kinyoklion Jul 10, 2023
0bd9846
Merge branch 'server-side' into rlamb/basic-store-implementation
kinyoklion Jul 10, 2023
2fcca51
Cleanup cmakelists.txt.
kinyoklion Jul 10, 2023
eedb155
Merge branch 'rlamb/basic-store-implementation' of github.com:launchd…
kinyoklion Jul 10, 2023
f2f4fe6
Add explainer comment for event handler.
kinyoklion Jul 10, 2023
2fb9abb
Linting
kinyoklion Jul 10, 2023
b33ae7f
Handle added and deleted. Add new test for update, add, delete, and u…
kinyoklion Jul 11, 2023
e2d205a
Reformatting.
kinyoklion Jul 11, 2023
0561288
Const
kinyoklion Jul 11, 2023
e530857
Update libs/server-sdk/src/data_store/data_store.hpp
kinyoklion Jul 11, 2023
e62258b
Merge branch 'rlamb/basic-store-implementation' of github.com:launchd…
kinyoklion Jul 11, 2023
057355e
Aliases and descriptors
kinyoklion Jul 11, 2023
a3614f7
More PR feedback updates.
kinyoklion Jul 11, 2023
e3fadc0
Fallthrough
kinyoklion Jul 11, 2023
b8fb098
Remove comment.
kinyoklion Jul 11, 2023
fe27583
More const and lint
kinyoklion Jul 11, 2023
7eb0023
Tidy
kinyoklion Jul 11, 2023
5cef5f3
More kind usage.
kinyoklion Jul 11, 2023
6618806
Fix namespace comment.
kinyoklion Jul 11, 2023
662dbd9
Add some more const references.
kinyoklion Jul 12, 2023
b812099
Merge branch 'server-side' into rlamb/basic-store-implementation
kinyoklion Jul 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions libs/client-sdk/src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,8 @@ add_library(${LIBNAME}
data_sources/data_source_status_manager.cpp
event_processor/event_processor.cpp
event_processor/null_event_processor.cpp
boost_signal_connection.cpp
kinyoklion marked this conversation as resolved.
Show resolved Hide resolved
client_impl.cpp
client.cpp
boost_signal_connection.hpp
client_impl.hpp
data_sources/data_source.hpp
data_sources/data_source_event_handler.hpp
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
#include <utility>

#include <launchdarkly/connection.hpp>
#include <launchdarkly/signals/boost_signal_connection.hpp>

#include "../boost_signal_connection.hpp"
#include "data_source_status_manager.hpp"

namespace launchdarkly::client_side::data_sources {
Expand Down Expand Up @@ -104,15 +104,16 @@ DataSourceStatus DataSourceStatusManager::Status() const {
std::unique_ptr<IConnection> DataSourceStatusManager::OnDataSourceStatusChange(
std::function<void(data_sources::DataSourceStatus)> handler) {
std::lock_guard lock{status_mutex_};
return std::make_unique< ::launchdarkly::client_side::SignalConnection>(
return std::make_unique<
::launchdarkly::internal::signals::SignalConnection>(
data_source_status_signal_.connect(handler));
}

std::unique_ptr<IConnection>
DataSourceStatusManager::OnDataSourceStatusChangeEx(
std::function<bool(data_sources::DataSourceStatus)> handler) {
std::lock_guard lock{status_mutex_};
return std::make_unique< ::launchdarkly::client_side::SignalConnection>(
return std::make_unique<launchdarkly::internal::signals::SignalConnection>(
data_source_status_signal_.connect_extended(
[handler](boost::signals2::connection const& conn,
data_sources::DataSourceStatus status) {
Expand Down
7 changes: 4 additions & 3 deletions libs/client-sdk/src/flag_manager/flag_updater.cpp
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#include <utility>

#include "../boost_signal_connection.hpp"
#include <launchdarkly/signals/boost_signal_connection.hpp>

#include "flag_updater.hpp"

namespace launchdarkly::client_side::flag_manager {
Expand Down Expand Up @@ -74,7 +75,7 @@ void FlagUpdater::DispatchEvent(FlagValueChangeEvent event) {
auto handler = signals_.find(event.FlagName());
if (handler != signals_.end()) {
if (handler->second.empty()) {
// Empty, remove it from the map so it doesn't count toward
// Empty, remove it from the map, so it doesn't count toward
// future calculations.
signals_.erase(event.FlagName());
} else {
Expand Down Expand Up @@ -123,7 +124,7 @@ std::unique_ptr<IConnection> FlagUpdater::OnFlagChange(
std::string const& key,
std::function<void(std::shared_ptr<FlagValueChangeEvent>)> handler) {
std::lock_guard lock{signal_mutex_};
return std::make_unique< ::launchdarkly::client_side::SignalConnection>(
return std::make_unique< launchdarkly::internal::signals::SignalConnection>(
signals_[key].connect(handler));
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

#include <launchdarkly/connection.hpp>

namespace launchdarkly::client_side {
namespace launchdarkly::internal::signals {

class SignalConnection : public IConnection {
public:
Expand Down
4 changes: 3 additions & 1 deletion libs/internal/src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ file(GLOB HEADER_LIST CONFIGURE_DEPENDS
"${LaunchDarklyInternalSdk_SOURCE_DIR}/include/launchdarkly/network/*.hpp"
"${LaunchDarklyInternalSdk_SOURCE_DIR}/include/launchdarkly/serialization/*.hpp"
"${LaunchDarklyInternalSdk_SOURCE_DIR}/include/launchdarkly/serialization/events/*.hpp"
"${LaunchDarklyInternalSdk_SOURCE_DIR}/include/launchdarkly/signals/*.hpp"
)

# Automatic library: static or dynamic based on user config.
Expand Down Expand Up @@ -38,7 +39,8 @@ add_library(${LIBNAME} OBJECT
serialization/json_rule_clause.cpp
serialization/json_flag.cpp
encoding/base_64.cpp
encoding/sha_256.cpp)
encoding/sha_256.cpp
signals/boost_signal_connection.cpp)

add_library(launchdarkly::internal ALIAS ${LIBNAME})

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#include "boost_signal_connection.hpp"
#include <launchdarkly/signals/boost_signal_connection.hpp>

namespace launchdarkly::client_side {
namespace launchdarkly::internal::signals {

SignalConnection::SignalConnection(boost::signals2::connection connection)
: connection_(std::move(connection)) {}
Expand All @@ -9,4 +9,4 @@ void SignalConnection::Disconnect() {
connection_.disconnect();
}

} // namespace launchdarkly::client_side
} // namespace launchdarkly::internal::signals
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#pragma once

#include <launchdarkly/connection.hpp>

#include <functional>
#include <memory>
#include <set>
#include <string>

namespace launchdarkly::server_side {

/**
* Interface to allow listening for flag changes. Notification events should
* be distributed after the store has been updated.
*/
class IChangeNotifier {
public:
using ChangeSet = std::set<std::string>;
using ChangeHandler = std::function<void(std::shared_ptr<ChangeSet>)>;

/**
* Listen for changes to flag configuration. The change handler will be called
* with a set of affected flag keys. Changes include flags whose dependencies
* (either other flags, or segments) changed.
*
* @param signal The handler for the changes.
* @return A connection which can be used to stop listening.
*/
virtual std::unique_ptr<IConnection> OnFlagChange(ChangeHandler handler) = 0;

virtual ~IChangeNotifier() = default;
IChangeNotifier(IChangeNotifier const& item) = delete;
IChangeNotifier(IChangeNotifier&& item) = delete;
IChangeNotifier& operator=(IChangeNotifier const&) = delete;
IChangeNotifier& operator=(IChangeNotifier&&) = delete;

protected:
IChangeNotifier() = default;
};

} // namespace launchdarkly::server_side
10 changes: 8 additions & 2 deletions libs/server-sdk/src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,20 @@ file(GLOB HEADER_LIST CONFIGURE_DEPENDS
# Automatic library: static or dynamic based on user config.

add_library(${LIBNAME}
${HEADER_LIST})
${HEADER_LIST}
data_source/data_source_update_sink.hpp
data_store/data_store.hpp
data_store/data_store_updater.hpp
data_store/data_store_updater.cpp
data_store/memory_store.cpp
data_store/dependency_tracker.hpp data_store/dependency_tracker.cpp)

if (MSVC OR (NOT BUILD_SHARED_LIBS))
target_link_libraries(${LIBNAME}
PUBLIC launchdarkly::common
PRIVATE Boost::headers Boost::json Boost::url launchdarkly::sse launchdarkly::internal foxy)
else ()
# The default static lib builds, for linux, are positition independent.
# The default static lib builds, for linux, are position independent.
# So they do not link into a shared object without issues. So, when
# building shared objects do not link the static libraries and instead
# use the "src.hpp" files for required libraries.
Expand Down
32 changes: 32 additions & 0 deletions libs/server-sdk/src/data_source/data_source_update_sink.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#pragma once

#include <launchdarkly/data_model/flag.hpp>
#include <launchdarkly/data_model/item_descriptor.hpp>
#include <launchdarkly/data_model/sdk_data_set.hpp>
#include <launchdarkly/data_model/segment.hpp>

namespace launchdarkly::server_side::data_source {
/**
* Interface for handling updates from LaunchDarkly.
*/
class IDataSourceUpdateSink {
public:
using FlagDescriptor = launchdarkly::data_model::ItemDescriptor<
launchdarkly::data_model::Flag>;
using SegmentDescriptor = launchdarkly::data_model::ItemDescriptor<
launchdarkly::data_model::Segment>;

virtual void Init(launchdarkly::data_model::SDKDataSet data_set) = 0;
virtual void Upsert(std::string key, FlagDescriptor flag) = 0;
virtual void Upsert(std::string key, SegmentDescriptor segment) = 0;

IDataSourceUpdateSink(IDataSourceUpdateSink const& item) = delete;
IDataSourceUpdateSink(IDataSourceUpdateSink&& item) = delete;
IDataSourceUpdateSink& operator=(IDataSourceUpdateSink const&) = delete;
IDataSourceUpdateSink& operator=(IDataSourceUpdateSink&&) = delete;
virtual ~IDataSourceUpdateSink() = default;

protected:
IDataSourceUpdateSink() = default;
};
} // namespace launchdarkly::server_side::data_source
7 changes: 7 additions & 0 deletions libs/server-sdk/src/data_store/data_kind.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#pragma once

#include <cstddef>

namespace launchdarkly::server_side::data_store {
enum class DataKind : std::size_t { kFlag = 0, kSegment = 1, kKindCount = 2 };
} // namespace launchdarkly::server_side::data_store
85 changes: 85 additions & 0 deletions libs/server-sdk/src/data_store/data_store.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
#pragma once

#include <launchdarkly/data_model/flag.hpp>
#include <launchdarkly/data_model/item_descriptor.hpp>
#include <launchdarkly/data_model/segment.hpp>

#include <memory>
#include <string>
#include <unordered_map>

namespace launchdarkly::server_side::data_store {

/**
* Interface for readonly access to SDK data.
*
* The data store is what the client uses to store feature flag data that has
kinyoklion marked this conversation as resolved.
Show resolved Hide resolved
* been received from LaunchDarkly.
*/
class IDataStore {
public:
using FlagDescriptor = launchdarkly::data_model::ItemDescriptor<
launchdarkly::data_model::Flag>;
using SegmentDescriptor = launchdarkly::data_model::ItemDescriptor<
kinyoklion marked this conversation as resolved.
Show resolved Hide resolved
launchdarkly::data_model::Segment>;

/**
* Get a flag from the store.
*
* @param key The key for the flag.
* @return Returns a shared_ptr to the FlagDescriptor, or a nullptr if there
* is not such flag, or the flag was deleted.
kinyoklion marked this conversation as resolved.
Show resolved Hide resolved
*/
[[nodiscard]] virtual std::shared_ptr<FlagDescriptor> GetFlag(
std::string key) const = 0;
kinyoklion marked this conversation as resolved.
Show resolved Hide resolved

/**
* Get a segment from the store.
*
* @param key The key for the segment.
* @return Returns a shared_ptr to the SegmentDescriptor, or a nullptr if
* there is no such segment, or the segment was deleted.
*/
[[nodiscard]] virtual std::shared_ptr<SegmentDescriptor> GetSegment(
std::string key) const = 0;
kinyoklion marked this conversation as resolved.
Show resolved Hide resolved

/**
* Get all of the flags.
*
* @return Returns an unordered map of FlagDescriptors.
*/
[[nodiscard]] virtual std::unordered_map<std::string, std::shared_ptr<FlagDescriptor>> AllFlags()
const = 0;

/**
* Get all of the segments.
*
* @return Returns an unordered map of SegmentDescriptors.
*/
[[nodiscard]] virtual std::unordered_map<std::string, std::shared_ptr<SegmentDescriptor>>
AllSegments() const = 0;

/**
* Check if the store is initialized.
*
* @return Returns true if the store is initialized.
*/
[[nodiscard]] virtual bool Initialized() const = 0;

/**
* Get a description of the store.
* @return Returns a string containing a description of the store.
*/
[[nodiscard]] virtual std::string const& Description() const = 0;

IDataStore(IDataStore const& item) = delete;
IDataStore(IDataStore&& item) = delete;
IDataStore& operator=(IDataStore const&) = delete;
IDataStore& operator=(IDataStore&&) = delete;
virtual ~IDataStore() = default;

protected:
IDataStore() = default;
};

} // namespace launchdarkly::server_side::data_store
75 changes: 75 additions & 0 deletions libs/server-sdk/src/data_store/data_store_updater.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#include "data_store_updater.hpp"

#include <launchdarkly/signals/boost_signal_connection.hpp>
#include <utility>

namespace launchdarkly::server_side::data_store {

std::unique_ptr<IConnection> DataStoreUpdater::OnFlagChange(
launchdarkly::server_side::IChangeNotifier::ChangeHandler handler) {
std::lock_guard lock{signal_mutex_};

return std::make_unique<launchdarkly::internal::signals::SignalConnection>(
signals_.connect(handler));
}

void DataStoreUpdater::Init(launchdarkly::data_model::SDKDataSet data_set) {
std::optional<DependencySet> change_notifications;
if (HasListeners()) {
auto updated_items = DependencySet();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ambiguous reading if this is a function or constructor. Maybe just

DepdendencySet updated_items;

?


CalculateChanges(DataKind::kFlag, store_->AllFlags(), data_set.flags,
updated_items);
CalculateChanges(DataKind::kSegment, store_->AllSegments(),
data_set.segments, updated_items);
change_notifications = updated_items;
}

dependency_tracker_.Clear();
for (auto const& flag : data_set.flags) {
dependency_tracker_.UpdateDependencies(flag.first, flag.second);
}
for (auto const& segment : data_set.segments) {
dependency_tracker_.UpdateDependencies(segment.first, segment.second);
}
// Data will move into the store, so we want to update dependencies before
// it is moved.
sink_->Init(data_set);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the comment meant to literally mean "moved" into the store i.e. std::move? (because that would seem to be a good use of move here, if the compiler can't figure it out.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a move here as well, but the comment refers to the actual descriptors.
std::make_shared<IDataStore::FlagDescriptor>(std::move(flag));
In that we move the flag out of this descriptor and into the descriptor in the actual store.

// After updating the sunk let listeners know of changes.
if (change_notifications) {
NotifyChanges(*change_notifications);
}
}

void DataStoreUpdater::Upsert(std::string key,
launchdarkly::server_side::data_source::
IDataSourceUpdateSink::FlagDescriptor flag) {
UpsertCommon(DataKind::kFlag, key, store_->GetFlag(key), flag);
Copy link
Contributor

@cwaldren-ld cwaldren-ld Jul 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I might've made some code suggestions that were wrong (std::string to std::string const&), since I'm realizing you might've had a strategy.

for all these functions that take key by value, they could be const&. But maybe deep down the key is being moved into a map or whatever?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the plan, but I think it got lost in the layers.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may just make more things const and let it copy, otherwise it is really easy to use the key twice in one function and it to be invalid. Downside of the way C++ does moves.

}

void DataStoreUpdater::Upsert(
std::string key,
launchdarkly::server_side::data_source::IDataSourceUpdateSink::
SegmentDescriptor segment) {
UpsertCommon(DataKind::kSegment, key, store_->GetSegment(key), segment);
}

bool DataStoreUpdater::HasListeners() const {
std::lock_guard lock{signal_mutex_};
return !signals_.empty();
}

void DataStoreUpdater::NotifyChanges(DependencySet changes) {
std::lock_guard lock{signal_mutex_};
auto flag_changes = changes.SetForKind(DataKind::kFlag);
// Only emit an event if there are changes.
if (!flag_changes.empty()) {
signals_(std::make_shared<ChangeSet>(std::move(flag_changes)));
}
}

DataStoreUpdater::DataStoreUpdater(std::shared_ptr<IDataSourceUpdateSink> sink,
std::shared_ptr<IDataStore> store)
: sink_(std::move(sink)), store_(std::move(store)) {}

} // namespace launchdarkly::server_side::data_store
Loading