Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflow Management - Initial Methods #1003

Merged
merged 33 commits into from
Jan 31, 2023
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
c2a41d9
Initial work for workflows DotNET SDK
RyanLettieri Jan 3, 2023
8cd0f0e
Addressing review comments from authoring portion of workflow SDK
RyanLettieri Jan 12, 2023
b4498cc
Merge branch 'master' into master
RyanLettieri Jan 13, 2023
94cea4d
Moving workflows test into E2E directory
RyanLettieri Jan 17, 2023
ba0d6b8
Adding back in workflow options
RyanLettieri Jan 17, 2023
9d2db44
Adding health check and updating itests for workflows
RyanLettieri Jan 17, 2023
3fa3012
Updating go version
RyanLettieri Jan 17, 2023
211224d
Adding in grpc endpoint for workflows test
RyanLettieri Jan 17, 2023
01f257c
Using http endpoint in workflows test
RyanLettieri Jan 17, 2023
6688012
Using http endpoint in workflows test
RyanLettieri Jan 17, 2023
68e1450
Updating itest for testing with local dapr
RyanLettieri Jan 17, 2023
f005729
Updating ref used for testing
RyanLettieri Jan 17, 2023
3ba809f
Addressing more review comments and adding in serialization for input
RyanLettieri Jan 18, 2023
51e4349
Addressing more review comments for workflow management SDK
RyanLettieri Jan 19, 2023
aaf7dcc
Using a record for getworkflow and using UTC time
RyanLettieri Jan 20, 2023
a704c02
Merge branch 'master' into master
RyanLettieri Jan 24, 2023
e49ef1b
Formatting cleanup
RyanLettieri Jan 25, 2023
cff1801
adding in a sleep command to wait for engine to start
RyanLettieri Jan 25, 2023
dc309c3
Merge branch 'master' into master
RyanLettieri Jan 25, 2023
e086a4a
Updating workflows SDK now that built in engine is present
RyanLettieri Jan 25, 2023
bd8f2ea
Testing new changes to workflow engine
RyanLettieri Jan 26, 2023
770f96e
Updating comment
RyanLettieri Jan 26, 2023
989d7c4
Merging in master
RyanLettieri Jan 26, 2023
fba845f
Updating checks for workflows E2E test
RyanLettieri Jan 26, 2023
f713701
adding sleep to wf activity so terminate can be called
RyanLettieri Jan 26, 2023
55856e2
Removing temp changes to itests file
RyanLettieri Jan 27, 2023
b737575
Adding in commit ref from dapr dapr
RyanLettieri Jan 27, 2023
43b2035
Adding in commit ref from dapr dapr
RyanLettieri Jan 27, 2023
aedece7
Blank commit to restart git run
RyanLettieri Jan 27, 2023
654b5e4
Addressing review comments
RyanLettieri Jan 27, 2023
49a0c94
Adding in more verbose error logging
RyanLettieri Jan 27, 2023
09e6bbf
Fixing string interpolation in workflows test
RyanLettieri Jan 30, 2023
938e349
Fixing string interpolation
RyanLettieri Jan 30, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/itests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,15 @@ jobs:
prefix: 'net6'
env:
NUPKG_OUTDIR: bin/Release/nugets
GOVER: 1.17
GOVER: 1.19
GOOS: linux
GOARCH: amd64
GOPROXY: https://proxy.golang.org
DAPR_CLI_VER: 1.8.0
DAPR_RUNTIME_VER: 1.8.0
DAPR_CLI_VER: 1.9.1
DAPR_RUNTIME_VER: 1.9.5
DAPR_INSTALL_URL: https://raw.githubusercontent.com/dapr/cli/3dacfb672d55f1436c249057aaebbe597e1066f3/install/install.sh
DAPR_CLI_REF: ''
DAPR_REF: ''
DAPR_REF: '68159926dea9694f12a132c80935df0da92ae7bf' #TODO, REMOVE ONCE TESTING IS DONE
steps:
- name: Set up Dapr CLI
run: wget -q ${{ env.DAPR_INSTALL_URL }} -O - | /bin/bash -s ${{ env.DAPR_CLI_VER }}
Expand All @@ -69,7 +69,7 @@ jobs:
uses: actions/checkout@v2
if: env.DAPR_REF != ''
with:
repository: dapr/dapr
repository: RyanLettieri/dapr #TODO, REMOVE ONCE TESTING IS DONE
ref: ${{ env.DAPR_REF }}
path: dapr
- name: Build and override dapr cli with referenced commit.
Expand Down
47 changes: 47 additions & 0 deletions src/Dapr.Client/DaprClient.cs
Original file line number Diff line number Diff line change
Expand Up @@ -941,6 +941,53 @@ public abstract Task<UnlockResponse> Unlock(
string lockOwner,
CancellationToken cancellationToken = default);

/// <summary>
/// Attempt to start the given workflow with response indicating success.
/// </summary>
/// <param name="instanceID">Identifier of the specific run.</param>
/// <param name="workflowComponent">The component to interface with.</param>
/// <param name="workflowType">Name of the workflow to run.</param>
/// <param name="workflowOptions">The list of options that are potentially needed to start a workflow.</param>
/// <param name="input">The input input for the given workflow.</param>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we've had this conversation, so forgive me. But is there a reason we went with Object instead of just a Generic? Is it just because we don't have a good means of using the generic so we get nothing from the type checking side of the world?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, there's no benefit to using a generic here. We don't really care what the type of the input is. We just need to be able to serialize it to JSON.

/// <param name="cancellationToken">A <see cref="CancellationToken" /> that can be used to cancel the operation.</param>
/// <returns>A <see cref="Task"/> containing a <see cref="WorkflowReference"/></returns>
[Obsolete("This API is currently not stable as it is in the Alpha stage. This attribute will be removed once it is stable.")]
public abstract Task<WorkflowReference> StartWorkflow(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normal convention in .NET is to suffix all method names that return Task or Task<T> with Async. However, I see this isn't being done consistently in the DaprClient class. I'd love to get the opinion of a .NET SDK maintainer on whether these new API names should be suffixed with Async.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I we have a "Deprecation" warning to the Workflow Authoring SDK too?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tmacam my initial thinking is that this isn't necessary in the WF authoring SDK because it's a new SDK, whereas Ryan's changes are to an existing SDK. However, we may want @halspang to weigh in on this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the point of the attribute here is to make people aware they're using a new feature. If they're using a new SDK, I think that's part of the implication.

Also, since we're here, I would like to have the Async in the method name (and the one below). I know we aren't consistent but we should follow standards where we can and be sad about our lack of consistency elsewhere.

string instanceID,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: In normal C# coding conventions, this would actually be instanceId (lowercase "d") instead of instanceID. We don't need to worry about this now, but might be good to consider for the next release.

string workflowComponent,
string workflowType,
Copy link
Contributor

@cgillum cgillum Jan 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based some feedback I got from Mark, I suggest we rename workflowType to workflowName.

Suggested change
string workflowType,
string workflowName,

Object input,
IReadOnlyDictionary<string, string> workflowOptions = default,
CancellationToken cancellationToken = default);

/// <summary>
/// Attempt to get information about the given workflow.
/// </summary>
/// <param name="instanceID">Identifier of the specific run.</param>
/// <param name="workflowComponent">The component to interface with.</param>
/// <param name="workflowType">Name of the workflow to run.</param>
/// <param name="cancellationToken">A <see cref="CancellationToken" /> that can be used to cancel the operation.</param>
/// <returns>A <see cref="Task"/> containing a <see cref="GetWorkflowResponse"/></returns>
[Obsolete("This API is currently not stable as it is in the Alpha stage. This attribute will be removed once it is stable.")]
public abstract Task<GetWorkflowResponse> GetWorkflow(
string instanceID,
string workflowComponent,
string workflowType,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little awkward that I didn't notice this until now, but we don't actually need the workflowType parameter at all for GetWorkflow. I suppose we can keep it in for now since it's a little late to be making significant changes, but we should remove it in the next release. Assuming we agree that we should keep this for now, I suggest we rename it here to workflowName to be consistent with my feedback on StartWorkflow.

CancellationToken cancellationToken = default);

/// <summary>
/// Attempt to get terminate the given workflow.
/// </summary>
/// <param name="instanceID">Identifier of the specific run.</param>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// <param name="instanceID">Identifier of the specific run.</param>
/// <param name="instanceID">The unique ID of the target workflow instance.</param>

/// <param name="workflowComponent">The component to interface with.</param>
/// <param name="cancellationToken">A <see cref="CancellationToken" /> that can be used to cancel the operation.</param>
/// <returns>A <see cref="Task" /> that will complete when the operation has completed. If the wrapped value is true the operation suceeded.</returns>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's currently ambiguous as to what "operation" refers to here. For example, developers reading this might incorrectly assume that this task completes when the workflow is terminated. However, that's not actually the case. Terminate operations are normally async and aren't guaranteed to terminate the workflow right away. I suggest we say that this task completes when the terminate operation has been scheduled.

[Obsolete("This API is currently not stable as it is in the Alpha stage. This attribute will be removed once it is stable.")]
public abstract Task TerminateWorkflow(
string instanceID,
string workflowComponent,
CancellationToken cancellationToken = default);

/// <inheritdoc />
public void Dispose()
{
Expand Down
112 changes: 112 additions & 0 deletions src/Dapr.Client/DaprClientGrpc.cs
Original file line number Diff line number Diff line change
Expand Up @@ -1374,6 +1374,118 @@ public async override Task<UnlockResponse> Unlock(

#endregion


#region Workflow API
/// <inheritdoc/>
[Obsolete]
public async override Task<WorkflowReference> StartWorkflow(
string instanceId,
string workflowComponent,
string workflowType,
Object input,
IReadOnlyDictionary<string, string> workflowOptions = default,
CancellationToken cancellationToken = default)
{
ArgumentVerifier.ThrowIfNullOrEmpty(instanceId, nameof(instanceId));
ArgumentVerifier.ThrowIfNullOrEmpty(workflowComponent, nameof(workflowComponent));
ArgumentVerifier.ThrowIfNullOrEmpty(workflowType, nameof(workflowType));
ArgumentVerifier.ThrowIfNull(workflowOptions, nameof(workflowOptions));
ArgumentVerifier.ThrowIfNull(input, nameof(input));

var request = new Autogenerated.StartWorkflowRequest()
{
InstanceId = instanceId,
WorkflowComponent = workflowComponent,
WorkflowName = workflowType,
Input = (ByteString)input
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yikes, this is not what I had in mind when I suggested allowing object instead of ByteString. What you have here seems very problematic because it will result in InvalidCastException whenever a user specifies an input that can't be cast directly into ByteString. What I was actually proposing is that we accept any object, JSON-serialize it, and then convert the serialized input JSON into whatever the protobuf contract requires (ByteString, in this case).

};

if (workflowOptions?.Count > 0)
{
foreach (var item in workflowOptions)
Comment on lines +1493 to +1495
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need this if here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need it for the null check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is here to check against the possibility of workflowOptions being Null.

{
request.Options[item.Key] = item.Value;
}
}

try
{
var options = CreateCallOptions(headers: null, cancellationToken);
var response = await client.StartWorkflowAlpha1Async(request, options);
return new WorkflowReference(response.InstanceId);

}
catch (RpcException ex)
{
throw new DaprException("Start Workflow operation failed: the Dapr endpoint indicated a failure. See InnerException for details.", ex);
}
}

/// <inheritdoc/>
[Obsolete]
public async override Task<GetWorkflowResponse> GetWorkflow(
string instanceId,
string workflowComponent,
string workflowType,
CancellationToken cancellationToken = default)
{
ArgumentVerifier.ThrowIfNullOrEmpty(instanceId, nameof(instanceId));
ArgumentVerifier.ThrowIfNullOrEmpty(workflowComponent, nameof(workflowComponent));
ArgumentVerifier.ThrowIfNullOrEmpty(workflowType, nameof(workflowType));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this specific argument verifier since we don't actually use the workflowType value for anything.


var request = new Autogenerated.GetWorkflowRequest()
{
InstanceId = instanceId,
WorkflowComponent = workflowComponent,
WorkflowType = workflowType
};

try
{
var options = CreateCallOptions(headers: null, cancellationToken);
var response = await client.GetWorkflowAlpha1Async(request, options);
return new GetWorkflowResponse(response.InstanceId, response.StartTime, response.Metadata);
}
catch (RpcException ex)
{
throw new DaprException("Get workflow operation failed: the Dapr endpoint indicated a failure. See InnerException for details.", ex);
}

}


/// <inheritdoc/>
[Obsolete]
public async override Task TerminateWorkflow(
string instanceID,
string workflowComponent,
CancellationToken cancellationToken = default)
{
ArgumentVerifier.ThrowIfNullOrEmpty(instanceID, nameof(instanceID));
ArgumentVerifier.ThrowIfNullOrEmpty(workflowComponent, nameof(workflowComponent));

var request = new Autogenerated.TerminateWorkflowRequest()
{
InstanceId = instanceID,
WorkflowComponent = workflowComponent
};

var options = CreateCallOptions(headers: null, cancellationToken);
Autogenerated.TerminateWorkflowResponse response = new Autogenerated.TerminateWorkflowResponse(); // RRL Do we need a response?
try
{
response = await client.TerminateWorkflowAlpha1Async(request, options);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We aren't returning or consuming response. Why is it needed?

}
catch (RpcException ex)
{
throw new DaprException("Terminate workflow operation failed: the Dapr endpoint indicated a failure. See InnerException for details.", ex);
}

}

#endregion


#region Dapr Sidecar Methods

/// <inheritdoc/>
Expand Down
56 changes: 56 additions & 0 deletions src/Dapr.Client/GetWorkflowResponse.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// ------------------------------------------------------------------------
// Copyright 2021 The Dapr Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ------------------------------------------------------------------------

using System;
using System.Collections.Generic;

namespace Dapr.Client
{
/// <summary>
/// Represents the response from invoking a binding.
/// </summary>
public sealed class GetWorkflowResponse
{
/// <summary>
/// Initializes a new <see cref="GetWorkflowResponse" />.`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra character at the end:

Suggested change
/// Initializes a new <see cref="GetWorkflowResponse" />.`
/// Initializes a new <see cref="GetWorkflowResponse" />.

/// </summary>
/// <param name="instanceId">The instance ID assocated with this response.</param>
/// <param name="startTime">The time at which the workflow started executing.</param>
/// <param name="metadata">The response metadata.</param>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the whitespace needs to be fixed here too.

public GetWorkflowResponse(string instanceId, Int64 startTime, IReadOnlyDictionary<string, string> metadata)
Copy link
Contributor

@tmacam tmacam Jan 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the class be a record? Aside construction, we never really overwrite any of its fields, right? I guess we would lose the ability to verify those arguments if this was a record, correct?

I know this can be done (https://stackoverflow.com/questions/68915019/how-to-check-if-parameters-are-null-when-using-positional-record-constructors-an) but I wonder if it would make the code any better. Again, C# is not my forté.

{
ArgumentVerifier.ThrowIfNull(instanceId, nameof(instanceId));
ArgumentVerifier.ThrowIfNull(startTime, nameof(startTime));
ArgumentVerifier.ThrowIfNull(metadata, nameof(metadata));

this.InstanceId = instanceId;
this.StartTime = startTime;
this.Metadata = metadata;
}

/// <summary>
/// Gets the workflow instance ID assocated with this response.
/// </summary>
public string InstanceId { set; get; }

/// <summary>
/// Gets the time that the workflow started.
/// </summary>
public Int64 StartTime { set; get; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be DateTime (or DateTimeOffset)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The proto definition of start_time is an int64 so StartTime in this scenario was created as an Int64

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but it's not very friendly to present a .NET developer with an integer value when a much more friendly types are available. I think we really need to use DateTime or DateTimeOffset here, and do the conversion from int64 internally.


/// <summary>
/// Gets the response metadata from the associated workflow. This includes information such as start time and status of workflow.
/// </summary>
public IReadOnlyDictionary<string, string> Metadata { set; get; }
}
}
Loading