Skip to content

Commit

Permalink
Update Readme to match latest destination changes and Downgrade Condu…
Browse files Browse the repository at this point in the history 
…it Version (#148)

* Update Readme to match latest destination changes

* reword destination behavior

* downgrade conduit sdk to 0.11.0
  • Loading branch information
@anna-cross
anna-cross authored Nov 8, 2024
1 parent 5423a24 commit 0bbb854
Show file tree
Hide file tree
Showing 3 changed files with 135 additions and 87 deletions.
61 changes: 14 additions & 47 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
## General

The Salesforce plugin is one of [Conduit](https://github.com/ConduitIO/conduit) plugins.
It currently provides only source Salesforce connector, allowing for receiving Salesforce changes events in a Conduit pipeline.

## How to build it

Expand All @@ -13,13 +12,17 @@ Run `make`.

The Source connector subscribes to Salesforce platform events and queries published events in real time.

## Destination

The Destination connector publishes incoming records to Salesforce platform events in real time.

### How it works

This section describes the technical details of how the connector works.

#### PubSub API

Conduit Source Connector uses [PubSub API](https://developer.salesforce.com/docs/platform/pub-sub-api/overview) for subscribing to event data.
Conduit Source and Destination Connector uses [PubSub API](https://developer.salesforce.com/docs/platform/pub-sub-api/overview) for subscribing to event data.

##### Platform Events

Expand Down Expand Up @@ -95,17 +98,17 @@ While the connector is operational, it may be necessary to reconnect. The server

### Configuration Options

With the above set up followed, you can begin configuring the source connector. Refer to the table below on which values to set.
With the above set up followed, you can begin configuring the source or destination connector. Refer to the table below on which values to set.

#### Source
Additionally, on the destination connector, verify that the source record’s data fields align with those of the event it’s being published to. You can include or exclude fields from the data as necessary using Conduit Processors.

#### Source

| name | description | required | default |
|-------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|---------|
| `oauthEndpoint` | Authorization service based on Organization’s Domain Name (e.g.: https://MyDomainName.my.salesforce.com ) | | `true` |
| `clientId` | OAuth Client ID (Consumer Key) | `true` | |
| `clientSecret` | OAuth Client Secret (Consumer Secret) | `true` | |
| `username` | Username. | `false` | |
| ~~`topicName`~~ | Event topic name for your event (e.g: /event/Accepted_Quote__e) **Deprecated: use `topicNames` instead** | `false` ||
| `topicsNames` | One or multiple comma separated topic names the source will subscribe to (e.g */event/Test__e, /event/Test2__e*). | `true` ||
| `retryCount` | Number of times the connector will retry is the connection to a topic breaks. | `false` | `10` |
Expand All @@ -131,48 +134,12 @@ The following data is included:

| name | description | required | default |
|-------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|---------|
| `environment` | Authorization service based on Organization’s Domain Name (e.g.: https://MyDomainName.my.salesforce.com -> `MyDomainName`) or `sandbox` for test environment. | `true` | |
| `clientId` | OAuth Client ID (Consumer Key). | `true` | |
| `clientSecret` | OAuth Client Secret (Consumer Secret). | `true` | |
| `username` | Username. | `true` | |
| `password` | Password. | `true` | |
| `securityToken` | Security token as described here: https://help.salesforce.com/s/articleView?id=sf.user_security_token.htm&type=5. | `false` | |
| `pushTopicsNames` | The comma-separated list of names of the Push Topics to listen to. All values will be prefixed with `/topic/`. All values will have white spaces trimmed. Any empty value is skipped. All Topics have to exist for connector to start working. | `true` | |
| `keyField` | The name of the Response's field that should be used as a Payload's Key. Empty value will set it to `nil`. | `false` | `"Id"` |

## Destination

There are a couple of steps that need to be done to start working with Salesforce connector as destination.

1. Log in into Your Salesforce account, e.g. https://my-demo-app.my.salesforce.com. The environment is `my-demo-app`.
2. First, if not already done, You need to create connected app and enable OAuth: [Enable OAuth Settings for API Integration](https://help.salesforce.com/s/articleView?id=sf.connected_app_create_api_integration.htm&type=5).

The callback URL is required, but not relevant for this connector, so you can put anything there.

Successfully configured app example can be seen below:

![Connected App example](docs/connected_app.png)
3. Copy **Consumer Key** and **Consumer Secret**. If You need these values once again You can always find them in _Setup -> Apps -> App Manager_, find app on the list and choose _View_ option.
![View OAuth tokens](docs/view_oauth_tokens.png)
4. You may need to configure **Security Token** for Your account. For more details follow instructions: [Reset Your Security Token](https://help.salesforce.com/s/articleView?id=sf.user_security_token.htm&type=5).
5. When all credentials are done, next You need to [create push topics](https://developer.salesforce.com/docs/atlas.en-us.api_streaming.meta/api_streaming/code_sample_interactive_vfp_create_pushtopic.htm).
Configure topics to Your needs specifying query and notifications behaviour for them.
6. Once done, You can begin with configuring the connector:
1. Use Step 1 environment value as `environment` config, e.g. `my-demo-app`.
2. Use Step 3 **Consumer Key** value as `clientId` config.
3. Use Step 3 **Consumer Secret** value as `clientSecret` config.
4. Use Step 1 username and password credentials as values for `username` and `password` config.
5. If required for Your account, use Step 4 **Security Token** value as `securityToken` config.
6. Use Step 5 Topic Names as a comma-separated values for `pushTopicsNames` config, e.g. `TaskUpdates,ContactUpdates`.
7. Optionally, configure `keyField`. The field's with this name value will be used as a Record Key, which may be utilized by other connectors.

Example:

When Push Topic query is: `SELECT Id, Name FROM Task` and `keyField` is set to **Id**,

Then for event with value `Id=123, Name=Create summary` Record's Key will be set to `123`.

Later, this may be utilized by other connectors, e.g. [Elasticsearch connector](https://github.com/conduitio-labs/conduit-connector-elasticsearch) will create Document with ID of Record's Key when available.
| `oauthEndpoint` | Authorization service based on Organization’s Domain Name (e.g.: https://MyDomainName.my.salesforce.com ) | | `true` |
| `clientId` | OAuth Client ID (Consumer Key) | `true` | |
| `clientSecret` | OAuth Client Secret (Consumer Secret) | `true` | |
| `topicName` | Event topic name for your event (e.g: /event/Accepted_Quote__e) | `true` ||
| `retryCount` | Number of times the connector will retry is the connection to a topic breaks. | `false` | `10` |
| `insecureSkipVerify` | Disables certificate validation. | `false` | `false` |


## Testing
Expand Down
80 changes: 40 additions & 40 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.23.2

require (
github.com/conduitio/conduit-commons v0.4.0
github.com/conduitio/conduit-connector-sdk v0.11.1
github.com/conduitio/conduit-connector-sdk v0.11.0
github.com/go-errors/errors v1.5.1
github.com/golangci/golangci-lint v1.61.0
github.com/linkedin/goavro/v2 v2.13.0
Expand All @@ -16,7 +16,7 @@ require (

require (
dario.cat/mergo v1.0.1 // indirect
github.com/hamba/avro/v2 v2.26.0 // indirect
github.com/hamba/avro/v2 v2.27.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
Expand All @@ -34,10 +34,10 @@ require (
4d63.com/gocheckcompilerdirectives v1.2.1 // indirect
4d63.com/gochecknoglobals v0.2.1 // indirect
github.com/4meepo/tagalign v1.3.4 // indirect
github.com/Abirdcfly/dupword v0.1.1 // indirect
github.com/Antonboom/errname v0.1.13 // indirect
github.com/Antonboom/nilnil v0.1.9 // indirect
github.com/Antonboom/testifylint v1.4.3 // indirect
github.com/Abirdcfly/dupword v0.1.3 // indirect
github.com/Antonboom/errname v1.0.0 // indirect
github.com/Antonboom/nilnil v1.0.0 // indirect
github.com/Antonboom/testifylint v1.5.0 // indirect
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c // indirect
github.com/Crocmagnon/fatcontext v0.5.2 // indirect
github.com/Djarvur/go-err113 v0.0.0-20210108212216-aea10b59be24 // indirect
Expand All @@ -46,48 +46,48 @@ require (
github.com/Masterminds/semver/v3 v3.3.0 // indirect
github.com/Masterminds/sprig/v3 v3.3.0 // indirect
github.com/OpenPeeDeeP/depguard/v2 v2.2.0 // indirect
github.com/alecthomas/go-check-sumtype v0.1.4 // indirect
github.com/alexkohler/nakedret/v2 v2.0.4 // indirect
github.com/alecthomas/go-check-sumtype v0.2.0 // indirect
github.com/alexkohler/nakedret/v2 v2.0.5 // indirect
github.com/alexkohler/prealloc v1.0.0 // indirect
github.com/alingse/asasalint v0.0.11 // indirect
github.com/ashanbrown/forbidigo v1.6.0 // indirect
github.com/ashanbrown/makezero v1.1.1 // indirect
github.com/avast/retry-go/v4 v4.6.0
github.com/beorn7/perks v1.0.1 // indirect
github.com/bkielbasa/cyclop v1.2.1 // indirect
github.com/bkielbasa/cyclop v1.2.3 // indirect
github.com/blizzy78/varnamelen v0.8.0 // indirect
github.com/bombsimon/wsl/v4 v4.4.1 // indirect
github.com/breml/bidichk v0.2.7 // indirect
github.com/breml/errchkjson v0.3.6 // indirect
github.com/breml/bidichk v0.3.2 // indirect
github.com/breml/errchkjson v0.4.0 // indirect
github.com/butuzov/ireturn v0.3.0 // indirect
github.com/butuzov/mirror v1.2.0 // indirect
github.com/catenacyber/perfsprint v0.7.1 // indirect
github.com/ccojocar/zxcvbn-go v1.0.2 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/charithe/durationcheck v0.0.10 // indirect
github.com/chavacava/garif v0.1.0 // indirect
github.com/ckaznocha/intrange v0.2.0 // indirect
github.com/ckaznocha/intrange v0.2.1 // indirect
github.com/conduitio/conduit-connector-protocol v0.8.0 // indirect
github.com/curioswitch/go-reassign v0.2.0 // indirect
github.com/daixiang0/gci v0.13.5
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/denis-tingaikin/go-header v0.5.0 // indirect
github.com/ettle/strcase v0.2.0 // indirect
github.com/fatih/color v1.17.0 // indirect
github.com/fatih/color v1.18.0 // indirect
github.com/fatih/structtag v1.2.0 // indirect
github.com/firefart/nonamedreturns v1.0.5 // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/fsnotify/fsnotify v1.8.0 // indirect
github.com/fzipp/gocyclo v0.6.0 // indirect
github.com/ghostiam/protogetter v0.3.6 // indirect
github.com/go-critic/go-critic v0.11.4 // indirect
github.com/ghostiam/protogetter v0.3.8 // indirect
github.com/go-critic/go-critic v0.11.5 // indirect
github.com/go-toolsmith/astcast v1.1.0 // indirect
github.com/go-toolsmith/astcopy v1.1.0 // indirect
github.com/go-toolsmith/astequal v1.2.0 // indirect
github.com/go-toolsmith/astfmt v1.1.0 // indirect
github.com/go-toolsmith/astp v1.1.0 // indirect
github.com/go-toolsmith/strparse v1.1.0 // indirect
github.com/go-toolsmith/typep v1.1.0 // indirect
github.com/go-viper/mapstructure/v2 v2.1.0 // indirect
github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
github.com/go-xmlfmt/xmlfmt v1.1.2 // indirect
github.com/gobwas/glob v0.2.3 // indirect
github.com/goccy/go-json v0.10.3 // indirect
Expand All @@ -109,10 +109,10 @@ require (
github.com/gostaticanalysis/forcetypeassert v0.1.0 // indirect
github.com/gostaticanalysis/nilerr v0.1.1 // indirect
github.com/hashicorp/go-hclog v1.6.3 // indirect
github.com/hashicorp/go-plugin v1.6.1 // indirect
github.com/hashicorp/go-plugin v1.6.2 // indirect
github.com/hashicorp/go-version v1.7.0 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/hashicorp/yamux v0.1.1 // indirect
github.com/hashicorp/yamux v0.1.2 // indirect
github.com/hexops/gotextdiff v1.0.3 // indirect
github.com/huandu/xstrings v1.5.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
Expand All @@ -123,7 +123,7 @@ require (
github.com/jpillora/backoff v1.0.0 // indirect
github.com/julz/importas v0.1.0 // indirect
github.com/karamaru-alpha/copyloopvar v1.1.0 // indirect
github.com/kisielk/errcheck v1.7.0 // indirect
github.com/kisielk/errcheck v1.8.0 // indirect
github.com/kkHAIKE/contextcheck v1.1.5 // indirect
github.com/kulti/thelper v0.6.3 // indirect
github.com/kunwardeep/paralleltest v1.0.10 // indirect
Expand All @@ -142,7 +142,7 @@ require (
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/mgechev/revive v1.3.9 // indirect
github.com/mgechev/revive v1.5.0 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
Expand All @@ -152,15 +152,15 @@ require (
github.com/nakabonne/nestif v0.3.1 // indirect
github.com/nishanths/exhaustive v0.12.0 // indirect
github.com/nishanths/predeclared v0.2.2 // indirect
github.com/nunnatsa/ginkgolinter v0.16.2 // indirect
github.com/nunnatsa/ginkgolinter v0.18.0 // indirect
github.com/oklog/run v1.1.0 // indirect
github.com/olekukonko/tablewriter v0.0.5 // indirect
github.com/pelletier/go-toml/v2 v2.2.3 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/polyfloyd/go-errorlint v1.6.0 // indirect
github.com/prometheus/client_golang v1.20.3 // indirect
github.com/prometheus/client_golang v1.20.5 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.59.1 // indirect
github.com/prometheus/common v0.60.1 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/quasilyte/go-ruleguard v0.4.3-0.20240823090925-0fe6f58b47b1 // indirect
github.com/quasilyte/go-ruleguard/dsl v0.3.22 // indirect
Expand All @@ -174,13 +174,13 @@ require (
github.com/santhosh-tekuri/jsonschema/v5 v5.3.1 // indirect
github.com/sashamelentyev/interfacebloat v1.1.0 // indirect
github.com/sashamelentyev/usestdlibvars v1.27.0 // indirect
github.com/securego/gosec/v2 v2.21.2 // indirect
github.com/securego/gosec/v2 v2.21.4 // indirect
github.com/shazow/go-diff v0.0.0-20160112020656-b6b7b6733b8c // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/sivchari/containedctx v1.0.3 // indirect
github.com/sivchari/tenv v1.10.0 // indirect
github.com/sonatard/noctx v0.0.2 // indirect
github.com/sivchari/tenv v1.12.1 // indirect
github.com/sonatard/noctx v0.1.0 // indirect
github.com/sourcegraph/go-diff v0.7.0 // indirect
github.com/spf13/afero v1.11.0 // indirect
github.com/spf13/cast v1.7.0 // indirect
Expand All @@ -193,8 +193,8 @@ require (
github.com/subosito/gotenv v1.6.0 // indirect
github.com/tdakkota/asciicheck v0.2.0 // indirect
github.com/tetafro/godot v1.4.18 // indirect
github.com/timakin/bodyclose v0.0.0-20240125160201-f835fa56326a // indirect
github.com/timonwong/loggercheck v0.9.4 // indirect
github.com/timakin/bodyclose v0.0.0-20241017074824-adbc21e6bf36 // indirect
github.com/timonwong/loggercheck v0.10.1 // indirect
github.com/tomarrell/wrapcheck/v2 v2.9.0 // indirect
github.com/tommy-muehle/go-mnd/v2 v2.5.1 // indirect
github.com/ultraware/funlen v0.1.0 // indirect
Expand All @@ -207,24 +207,24 @@ require (
gitlab.com/bosi/decorder v0.4.2 // indirect
go-simpler.org/musttag v0.12.2 // indirect
go-simpler.org/sloglint v0.7.2 // indirect
go.uber.org/automaxprocs v1.5.3 // indirect
go.uber.org/automaxprocs v1.6.0 // indirect
go.uber.org/goleak v1.3.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
golang.org/x/crypto v0.28.0 // indirect
golang.org/x/exp v0.0.0-20241004190924-225e2abe05e6 // indirect
golang.org/x/exp/typeparams v0.0.0-20240909161429-701f63a606c0 // indirect
golang.org/x/mod v0.21.0 // indirect
golang.org/x/sync v0.8.0 // indirect
golang.org/x/sys v0.26.0 // indirect
golang.org/x/text v0.19.0 // indirect
golang.org/x/time v0.7.0 // indirect
golang.org/x/crypto v0.29.0 // indirect
golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c // indirect
golang.org/x/exp/typeparams v0.0.0-20241009180824-f66d83c29e7c // indirect
golang.org/x/mod v0.22.0 // indirect
golang.org/x/sync v0.9.0 // indirect
golang.org/x/sys v0.27.0 // indirect
golang.org/x/text v0.20.0 // indirect
golang.org/x/time v0.8.0 // indirect
golang.org/x/tools v0.26.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240930140551-af27646dc61f // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20241104194629-dd2ea8efbc28 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
honnef.co/go/tools v0.5.1 // indirect
mvdan.cc/gofumpt v0.7.0
mvdan.cc/unparam v0.0.0-20240528143540-8a5130ca722f // indirect
mvdan.cc/unparam v0.0.0-20240917084806-57a3b4290ba3 // indirect
)
Loading

0 comments on commit 0bbb854

Please sign in to comment.