Implement baton-workato Connector #1

MarcusGoldschmidt · 2024-12-27T20:04:42Z

No description provided.

ggreer · 2025-01-06T19:15:16Z

.golangci.yml

@@ -29,7 +29,7 @@ linters-settings:
    rules:
      - name: atomic
      - name: line-length-limit
-        arguments: [ 200 ]
+        arguments: [ 300 ]


I don't have strong opinions about max line length, but it'd be nice to keep this the same in every repo.

Some description from privileges have more then 250 lenght

ggreer · 2025-01-06T19:27:29Z

pkg/connector/client/helpers.go

+		ctx,
+		method,
+		urlAddress,
+		uhttp.WithHeader(AuthHeaderName, fmt.Sprintf("Bearer %s", c.apiKey)),


If you bump the baton-sdk version to whatever the latest is (v0.2.61 I think), you could use WithBearerToken here.

ggreer · 2025-01-06T19:28:42Z

pkg/connector/client/helpers.go

+
+	switch method {
+	case http.MethodGet:
+		resp, err = c.httpClient.Do(req, uhttp.WithResponse(&res))


Would it be simpler to just check if res is not nil, and if so add uhttp.WithResponse(&res) to the DoOptions? Other methods could potentially return useful json responses.

Sure, we can do

var options []uhttp.DoOption if res != nil { options = append(options, uhttp.WithResponse(&res)) } resp, err = c.httpClient.Do(req, options...) if resp != nil { defer resp.Body.Close() }

ggreer · 2025-01-06T19:34:22Z

pkg/connector/client/helpers.go

+		}
+	}
+
+	if resp != nil {


Whatever error we're returning should have a grpc status code that tells syncer whether or not to retry. Otherwise a single error could cause a sync to fail.
That means that whatever errors are returned by this function should wrap errors instead of creating new errors and discarding the original error. I think getError() should also have that behavior.

See https://github.com/ConductorOne/baton-sdk/blob/main/pkg/sync/syncer.go#L213 for how syncer handles retrying.

Thanks, I found that we already have a mapper for each error on https://github.com/ConductorOne/baton-sdk/blob/main/pkg/uhttp/wrapper.go#L361

The issue is that getError sometimes throws away the original error, and on line 83 we throw away the original error and make a new one without a grpc status, so we won't retry in some cases.

ggreer · 2025-01-06T19:37:46Z

pkg/connector/collaborator.go

+		resource.WithUserProfile(profile),
+		resource.WithStatus(userStatus),
+		resource.WithEmail(collaborator.Email, true),
+		resource.WithUserLogin(collaborator.Email),


It looks like you could use a couple other user traits here to bubble up more info in the conductorone UI. I know we have WithCreatedAt() and WithAccountType(). If last login time exists on the collaborator struct, you could also use WithLastLogin().

ggreer · 2025-01-06T19:39:04Z

README.md

+2. Create an API KEY https://app.workato.com/members/api/clients.
+3. Run it.
+
+Obs: if you have a basic account, you can ignore the subusers using ```.


Does this triple backtick mess up the formatting of the readme?

ggreer · 2025-01-06T19:41:15Z

cmd/baton-workato/conf/config.go

+
+	WorkatoDataCenterFiekd = field.StringField(
+		"workato-data-center",
+		field.WithDescription("Your workato data center (us, eu, jp, sg, au) default is 'us' see more on https://docs.workato.com/workato-api.html#base-url"),


I guess we don't have a field type that is "one of these values". Maybe we should add that to baton-sdk at some point.

ggreer · 2025-01-06T19:42:06Z

pkg/connector/client/client.go

+		return nil, err
+	}
+
+	uhtppClient, err := uhttp.NewBaseHttpClientWithContext(ctx, httpClient)


Nitpick: uhtppClient should probably be uhttpClient.

ggreer · 2025-01-06T19:46:00Z

pkg/connector/collaborator.go

+// List returns all the users from the database as resource objects.
+// Users include a UserTrait because they are the 'shape' of a standard user.
+func (o *collaboratorBuilder) List(ctx context.Context, parentResourceID *v2.ResourceId, pToken *pagination.Token) ([]*v2.Resource, string, annotations.Annotations, error) {
+	collaborators, err := o.client.GetCollaborators(ctx)


Will this only return up to 500 collaborators? If so, it should return a next token so that syncer calls List() again.

List collaborators does not have pagination on workato API

https://docs.workato.com/workato-api/team.html#all-users-get

Ah, I saw pageLimit: 500, and thought that was being used everywhere. Nevermind. :)

ggreer · 2025-01-06T19:47:51Z

pkg/connector/resource_types.go

-	DisplayName: "User",
+var collaboratorResourceType = &v2.ResourceType{
+	Id:          "collaborator",
+	DisplayName: "Collaborator",
 	Traits:      []v2.ResourceType_Trait{v2.ResourceType_TRAIT_USER},


You can add the SkipEntitlementsAndGrants annotation to the user resource type to speed up syncing. See https://github.com/ConductorOne/baton-bitbucket-datacenter/blob/1ce05111f8d152eac56e64024ebf54ca83d5eaff/pkg/connector/resource_types.go#L15 for an example.

ggreer · 2025-01-08T18:48:14Z

pkg/connector/client/colaborator.go

+	}
+
+	if len(response.Data) != 1 {
+		return nil, fmt.Errorf("baton-workato: expected 1 collaborator, got %d", len(response.Data))


Ideally this would return a grpc status with codes.NotFound so that if we fail to fetch grants for a resource, baton-sdk's syncer will log a warning and continue. A generic error will cause the sync to fail. See https://github.com/ConductorOne/baton-sdk/blob/d102c4f59bd342ca17224dbec24d9e1fa138c9c7/pkg/sync/syncer.go#L264 for how this is handled in syncer.

Sure, we could replace with

status.Errorf(codes.NotFound, "baton-workato: expected 1 collaborator, got %d", len(response.Data))

ggreer · 2025-01-08T18:50:56Z

pkg/connector/client/helpers.go

+		}
+	}
+
+	if resp != nil {


The issue is that getError sometimes throws away the original error, and on line 83 we throw away the original error and make a new one without a grpc status, so we won't retry in some cases.

ggreer · 2025-01-08T18:52:26Z

pkg/connector/client/helpers.go

+
+	resp, err = c.httpClient.Do(req, options...)
+	if resp != nil {
+		defer resp.Body.Close()


This can go in the if resp != nil on line 76.

Actually, it might be simpler if we check if resp == nil and return err in that case, then all this code can be un-indented.

ggreer · 2025-01-08T18:54:23Z

pkg/connector/collaborator.go

+// List returns all the users from the database as resource objects.
+// Users include a UserTrait because they are the 'shape' of a standard user.
+func (o *collaboratorBuilder) List(ctx context.Context, parentResourceID *v2.ResourceId, pToken *pagination.Token) ([]*v2.Resource, string, annotations.Annotations, error) {
+	collaborators, err := o.client.GetCollaborators(ctx)


Ah, I saw pageLimit: 500, and thought that was being used everywhere. Nevermind. :)

MarcusGoldschmidt added 15 commits December 27, 2024 16:03

feat: collaborator List

1ce230d

feat: implement privileges

d3aac57

feat: list folder project

d98ebe9

feat: add recursive list for folders

5f6fb88

update capabilities

949a254

add pagination for folders

623897d

feat: add grants for folder

cd6e53e

feat: add grants for roles

706e4ec

refactor: pagination

151c4fc

fix: lint

d5fcf7d

fix pagination add role grants

7dc65cc

feat: add env for field

d81ded3

feat: provisioningV2 roles to user

3e8ca25

remove usuless test

3bfa617

feat: add CI

0914f6a

MarcusGoldschmidt marked this pull request as ready for review January 6, 2025 15:21

MarcusGoldschmidt added 4 commits January 6, 2025 11:23

fix: ci

6d4e48d

fix: linter

47ef35b

fix: add workato env for ci

e4c6a7e

fix: provisioning and ci

9a8eef4

ggreer reviewed Jan 6, 2025

View reviewed changes

PR requested changes

c6a835b

MarcusGoldschmidt requested a review from ggreer January 6, 2025 22:37

ggreer approved these changes Jan 8, 2025

View reviewed changes

MarcusGoldschmidt added 2 commits January 9, 2025 06:02

fix: status code for http client

0f89a65

Fix linter

cca67dc

ggreer approved these changes Jan 17, 2025

View reviewed changes

ggreer merged commit dbec6de into main Jan 17, 2025
3 checks passed

ggreer deleted the goldschmidt/workato-connector branch January 17, 2025 00:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement baton-workato Connector #1

Implement baton-workato Connector #1

MarcusGoldschmidt commented Dec 27, 2024

ggreer Jan 6, 2025

MarcusGoldschmidt Jan 6, 2025

ggreer Jan 6, 2025

ggreer Jan 6, 2025

MarcusGoldschmidt Jan 6, 2025

ggreer Jan 6, 2025

MarcusGoldschmidt Jan 6, 2025

ggreer Jan 8, 2025

ggreer Jan 6, 2025

ggreer Jan 6, 2025

ggreer Jan 6, 2025

ggreer Jan 6, 2025

ggreer Jan 6, 2025

MarcusGoldschmidt Jan 6, 2025

ggreer Jan 8, 2025

ggreer Jan 6, 2025

MarcusGoldschmidt Jan 6, 2025

ggreer Jan 8, 2025

MarcusGoldschmidt Jan 9, 2025

ggreer Jan 8, 2025

ggreer Jan 8, 2025

MarcusGoldschmidt Jan 9, 2025

ggreer Jan 8, 2025

Implement baton-workato Connector #1

Implement baton-workato Connector #1

Conversation

MarcusGoldschmidt commented Dec 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment