Exec agent Che IDE client #3032

dkuleshov · 2016-11-09T14:50:51Z

What does this PR do?

ExecAgentCommandManager - new exec agent Che IDE client
Improvements/fixes/changes to JsonRPC protocol
Improvements/fixes/changes to websocket protocol

What issues does this PR fix or reference?

This is an adaptation of a new exec agent implementation located in exec-agent branch, the work is done under #1947 issue.

Changes to existing protocols

Websocket protocol

Websocket implementation on IDE side now support custom endpoints. This is done by calling improved initializer:
org.eclipse.che.ide.websocket.ng.impl.WebSocketInitializer#initialize(String endpointId, String url)
where

endpointId is a high level websocket connection identifier (e.g. "exec-agent", "ws-agent", etc.)
url is a URL of websocket endpoint the connection is established to.
After a connection is initialized and established an endpoint identifier can be used to choose where to send a message via a transmitter, an instance of org.eclipse.che.ide.websocket.ng.WebSocketMessageTransmitter#transmit(String endpointId, String message)

Multi-protocol support is considered redundant and removed, now websocket supports only json rpc protocol transfers, though core routines for addition of support of new protocols are still present.
Websocket messages are now plain text messages thus parsing and analyzing is moved to a higher level consumers (e.g. json rpc service dispatchers).

JsonRPC protocol

Added automation on different levels, requests and responses are now parsed automatically and the business objects are now passed to actual receivers and transmitters.
As client-side websocket implementation supports multiple endpoints json rpc receivers for client side now requires a new way of binding in a dependency injection container. This is now done like this:

GinMapBinder.newMapBinder(binder(), String.class, RequestHandler.class)
                    .addBinding("exec-agent" + '@' + "connected")
                    .to(ConnectedEventHandler.class);

where

exec-agent is an endpoint identifier
connected is a method name in terms of JsonRpc 2.0 protocol.
At the same time for server-side registration procedure nothing is changed.

Changed API of request and notification receivers. At the moment it is done by extending of org.eclipse.che.ide.jsonrpc.RequestHandler class for client-side and
org.eclipse.che.api.core.jsonrpc.RequestHandler for server-side. The request handlers are now responsible for handling all kind of incoming request and notification. Handling is done by overriding of a corresponding method. Classes have two generic type parameters <P, R> that corresponds for request/notification parameters and response result class types. Note that for handling requests the implementor isn't responsible now for response transmission because it is done automatically by taking the return value of a corresponding method.
Changed API of request and notification transmitters. This is now done by calling instances of
org.eclipse.che.api.core.jsonrpc.RequestTransmitter and org.eclipse.che.ide.jsonrpc.RequestTransmitter for server side and client side correspondingly. Now request transmission methods return javascript promise wrappers for client-side and completable futures for server-side. That allows to dynamically handle response while previously was done by static binding of a corresponding response receiver in a dependency injection container.

New exec agent client capabilities and use-cases

This is the core feature of this pull request, main task is to add a handy client to address process related calls to an exec agent through json rpc protocol.
Original json rpc exec agent API:

The client is added as an implementation of org.eclipse.che.ide.api.machine.ExecAgentCommandManager and provide next methods

    /**
     * Call exec agent to start a process with specified command parameters
     *
     * @param command command
     * @return exec agent promise with appropriate dto
     */
    ExecAgentPromise<ProcessStartResponseDto> startProcess(Command command);

    /**
     * Call exec agent to kill a process with specified identifier
     *
     * @param pid process identifier
     * @return promise with appropriate dto
     */
    Promise<ProcessKillResponseDto> killProcess(int pid);

    /**
     * Call for a subscription to events related to a specified process after defined timestamp
     * represented by a corresponding string (RFC3339Nano e.g. "2016-07-26T09:36:44.920890113+03:00").
     *
     * @param pid process identifier
     * @param eventTypes event types (e.g. stderr, stdout)
     * @param after after timestamp
     * @return exec agent promise with appropriate dto
     */
    ExecAgentPromise<ProcessSubscribeResponseDto> subscribe(int pid, List<String> eventTypes, String after);

    /**
     * Call for a cancellation of a subscription to events related to a specific process after defined
     * timestamp represented by a corresponding string (RFC3339Nano e.g. "2016-07-26T09:36:44.920890113+03:00").
     *
     * @param pid process identifier
     * @param eventTypes event types (e.g. stderr, stdout)
     * @param after after timestamp
     * @return promise with appropriate dto
     */
    Promise<ProcessUnSubscribeResponseDto> unsubscribe(int pid, List<String> eventTypes, String after);

    /**
     * Call for an update of a subscription to events related to a specific process.
     *
     * @param pid process identifier
     * @param eventTypes event types (e.g. stderr, stdout)
     * @return promise with appropriate dto
     */
    Promise<UpdateSubscriptionResponseDto> updateSubscription(int pid, List<String> eventTypes);

    /**
     * Call for a report on proess logs of a specific process.
     *
     * @param pid process identifier
     * @param from string represented timestamp the beginning of a time
     *             segment (RFC3339Nano e.g. "2016-07-26T09:36:44.920890113+03:00")
     * @param till string represented timestamp the ending of a time
     *             segment (RFC3339Nano e.g. "2016-07-26T09:36:44.920890113+03:00")
     * @param limit the limit of logs in result, the default value is 50
     * @param skip the logs to skip, default value is 0
     * @return promise with appropriate dto
     */
    Promise<List<GetProcessLogsResponseDto>> getProcessLogs(int pid, String from, String till, int limit, int skip);

    /**
     * Call for a process info
     *
     * @param pid process identifier
     * @return promise with appropriate dto
     */
    Promise<GetProcessResponseDto> getProcess(int pid);

    /**
     * Call for a processes info
     *
     * @param all defines if include already stopped processes, true for all process, 
     * false for running
     * @return promise with appropriate dto
     */
    Promise<List<GetProcessesResponseDto>> getProcesses(boolean all);

As you can see this API requires support of optional parameters and it will be added soon but now it is quite enough for all actual workflows and probably the most simple and basic one should be the following:

We ask exec agent to start a process -> request
Exec agent starts a process and provide process start related information <- response
Exec agent automatically subscribe us for all process related events (start, stdout, stderr, finish, etc.)
Each time an event happens we receive a message from an exec agent <- notification
In order to react on event we can register event operations by using either org.eclipse.che.ide.api.machine.ExecAgentEventManager injecting it or org.eclipse.che.ide.api.machine.execagent.ExecAgentPromise that is retuned by corresponding methods (startProcess, subscribe).
We ask exec agent to stop a process -> request
Exec agent stops a process and provide process stop related information <- response

A good example of how to run a command using the new client is a piece of org.eclipse.che.ide.extension.machine.client.actions.RunExecAgentCommandsAction source code:

Command command = new CommandImpl("Test command", value, "Custom");
jsonRpcExecAgentCommandManager.startProcess(command)
                  .thenIfProcessStartedEvent(new ProcessStartedEventOperation(console))
                  .thenIfProcessDiedEvent(new ProcessDiedEventOperation(console))
                  .thenIfProcessStdOutEvent(new ProcessStdOutEventOperation(console))
                  .thenIfProcessStdErrEvent(new ProcessStdErrEventOperation(console));

where

"Test command" is the name of the command
value is the command line
"Custom" is the type of the command
Process*EventOperation operation that will be performed on project (started, died, stdout, stderr ) event

Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

…, test fixes Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

codenvy-ci · 2016-11-09T15:37:54Z

Build success. https://ci.codenvycorp.com/job/che-pullrequests-build/967/

codenvy-ci · 2016-11-09T17:01:03Z

Build success. https://ci.codenvycorp.com/job/che-pullrequests-build/969/

azatsarynnyy · 2016-11-10T07:41:55Z

...ore-api-core/src/main/java/org/eclipse/che/api/core/jsonrpc/impl/NotificationDispatcher.java

+     * Dispatches json rpc notification received from endpoint identified by a
+     * high level identifier and represented as a json object.
+     *
+     * @param endpointId high level endpoint identifier


javadoc isn't formatted with che code style.
Please, review all classes in org.eclipse.che.api.core.jsonrpc.impl package because another ones have the same problem.

azatsarynnyy · 2016-11-10T07:44:18Z

...i-core/src/main/java/org/eclipse/che/api/core/jsonrpc/impl/WebSocketToJsonRpcDispatcher.java

+import javax.inject.Singleton;
+
+/**
+ * Dispatches messages received from web socket endpoint throughout json rcp


json rcp -> json rpc

azatsarynnyy · 2016-11-10T07:46:01Z

...commons-gwt/src/main/java/org/eclipse/che/ide/jsonrpc/impl/WebSocketToJsonRpcDispatcher.java

+import javax.inject.Singleton;
+
+/**
+ * Dispatches messages received from web socket endpoint throughout json rcp


json rcp -> json rpc

azatsarynnyy · 2016-11-10T07:52:18Z

...e-core-ide-api/src/main/java/org/eclipse/che/ide/api/machine/execagent/ExecAgentPromise.java

+
+    /**
+     * Register an operation which will be performed when a process generates
+     * process standard error event.


Doc says standard error event but method name is thenIfProcessStdOutEvent.
I think that doc should say standard output event.
No?

You're absolutely right

azatsarynnyy · 2016-11-10T07:52:41Z

...e-core-ide-api/src/main/java/org/eclipse/che/ide/api/machine/execagent/ExecAgentPromise.java

+
+    /**
+     * Register an operation which will be performed when a process generates
+     * process standard output event.


Doc says standard output event but method name is thenIfProcessStdErrEvent.
I think that doc should say standard error event.
No?

azatsarynnyy · 2016-11-10T07:55:08Z

.../main/java/org/eclipse/che/api/machine/shared/dto/execagent/event/ProcessStdOutEventDto.java

+ * Copyright (c) 2012-2016 Codenvy, S.A.
+ * All rights reserved. This program and the accompanying materials
+ * are made available under the terms of the Eclipse Public License v1.0
+ * which accompanies this distribution, and is available at


I think that one of these license headers is unnecessary )
And please review all new DTOs in this PR. Because really a lot of them have duplicated license headers.

azatsarynnyy · 2016-11-10T08:07:30Z

.../src/main/java/org/eclipse/che/api/machine/shared/dto/execagent/ProcessStartResponseDto.java

+ * @author Dmitry Kuleshov
+ */
+@DTO
+public interface ProcessStartResponseDto extends DtoWithPidDto{


Line isn't formatted. There is no space before {

azatsarynnyy · 2016-11-10T08:07:42Z

...d/src/main/java/org/eclipse/che/api/machine/shared/dto/execagent/ProcessKillResponseDto.java

-     */
-    void transmit(JsonRpcRequest request);
+@DTO
+public interface ProcessKillResponseDto extends DtoWithPidDto{


Line isn't formatted. There is no space before {

azatsarynnyy · 2016-11-10T08:21:53Z

ide/commons-gwt/src/main/java/org/eclipse/che/ide/jsonrpc/impl/NotificationDispatcher.java

+     * Dispatches json rpc notification received from endpoint identified by a
+     * high level identifier and represented as a json object.
+     *
+     * @param endpointId high level endpoint identifier


javadoc isn't formatted with che code style.
Please, review all classes in org.eclipse.che.ide.jsonrpc.impl package because several of them have the same problem.

…for endusers Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

codenvy-ci · 2016-11-17T13:06:28Z

Build # 1050 - FAILED

Please check console output at https://ci.codenvycorp.com/job/che-pullrequests-build/1050/ to view the results.

Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

codenvy-ci · 2016-11-18T19:18:00Z

Build # 1071 - FAILED

Please check console output at https://ci.codenvycorp.com/job/che-pullrequests-build/1071/ to view the results.

codenvy-ci · 2016-12-02T00:03:00Z

Build # 1193 - FAILED

Please check console output at https://ci.codenvycorp.com/job/che-pullrequests-build/1193/ to view the results.

codenvy-ci · 2016-12-06T16:52:04Z

Build # 1232 - FAILED

Please check console output at https://ci.codenvycorp.com/job/che-pullrequests-build/1232/ to view the results.

codenvy-ci · 2016-12-08T14:46:45Z

Build # 1262 - FAILED

Please check console output at https://ci.codenvycorp.com/job/che-pullrequests-build/1262/ to view the results.

TylerJewell · 2017-01-07T17:32:27Z

@dkuleshov - now that we have tagged 5.0.0 - can you resolve these conflicts and be ready to merge this? I think it's good for us to clean up our backlog of PRs, before moving onto bigger projects.

dkuleshov · 2017-01-13T15:34:06Z

This is merged to master under another pull request

voievodin and others added 24 commits August 17, 2016 22:22

Add exec-agent

66dd1a7

fixup! Add exec-agent

eb26d41

fixup! fixup! Add exec-agent

9be9ad1

Merged with master

6ae9828

Add 'subscribed' event

c98a30c

Fix filed name

50ff013

Make Transmitter to be a struct

564db99

Implement jsonrpc2.0 protocol

e230999

Merge branch 'master' into exec-agent

781a8a3

Merge branch 'master' into exec-agent

a0bcf31

Test cleanup

696eb25

Test RestoreSubscriber

f139f31

fix internal components structure

30c8270

fixup! fix internal components structure

4cba052

Merge branch 'master' into exec-agent

a89f72f

fixup! Merge branch 'master' into exec-agent

f1802e1

Merge branch 'master' into exec-agent

5d55bcf

⬆️ to 5.0.0-M7-SNAPSHOT

f019709

Merge branch 'master' into exec-agent

b315c07

Rename artifact

c5a35da

Merge branch 'master' into exec-agent

ea3797a

Add exec-agent link to dev-machine

f9390cf

che#1947: initial commit

52517c3

Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

che#1947: json rpc exec agent command manager improvements, bug fixes…

5902b96

…, test fixes Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

dkuleshov added the status/in-progress This issue has been taken by an engineer and is under active development. label Nov 9, 2016

dkuleshov self-assigned this Nov 9, 2016

che#1947: fixed docs

19de613

Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

azatsarynnyy reviewed Nov 10, 2016

View reviewed changes

Dmitry Kuleshov and others added 5 commits November 16, 2016 13:47

che#1947: moved endpoint id into receivers methods to provide access …

8648028

…for endusers Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

⬆️ docs

87eec43

Merge branch 'master' into exec-agent

bc02a64

⬆️ version to 5.0.0-M8

dd39ab7

merge 'exec-agent' branch

d6a6e69

Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

che:1947: fixed sample

ad6fe70

Signed-off-by: Dmitry Kuleshov <dkuleshov@codenvy.com>

dkuleshov mentioned this pull request Nov 22, 2016

Json RPC and websocket architecture reorganization #3143

Merged

vparfonov added sprint/next team/ide status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. and removed status/in-progress This issue has been taken by an engineer and is under active development. labels Nov 24, 2016

voievodin force-pushed the exec-agent branch 2 times, most recently from 19a4927 to a34f616 Compare December 1, 2016 16:55

voievodin force-pushed the exec-agent branch from 68917c2 to c8f1977 Compare December 6, 2016 11:49

voievodin force-pushed the exec-agent branch 2 times, most recently from a79f9ba to 6fba10b Compare December 8, 2016 13:36

vparfonov added sprint/current and removed sprint/next labels Dec 8, 2016

dkuleshov closed this Jan 13, 2017

dkuleshov deleted the exec-agent-adapt branch January 13, 2017 15:34

dkuleshov mentioned this pull request Feb 17, 2017

Json RPC client-side to server-side port #3627

Merged

JamesDrummond mentioned this pull request Feb 17, 2017

5.3.0 Milestone Overview #4030

Closed

9 tasks

benoitf removed the status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. label Nov 2, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exec agent Che IDE client #3032

Exec agent Che IDE client #3032

dkuleshov commented Nov 9, 2016 •

edited

Loading

codenvy-ci commented Nov 9, 2016

codenvy-ci commented Nov 9, 2016

azatsarynnyy Nov 10, 2016

azatsarynnyy Nov 10, 2016

azatsarynnyy Nov 10, 2016

azatsarynnyy Nov 10, 2016

dkuleshov Nov 10, 2016

azatsarynnyy Nov 10, 2016

dkuleshov Nov 10, 2016

azatsarynnyy Nov 10, 2016

azatsarynnyy Nov 10, 2016

azatsarynnyy Nov 10, 2016

azatsarynnyy Nov 10, 2016

codenvy-ci commented Nov 17, 2016

codenvy-ci commented Nov 18, 2016

codenvy-ci commented Dec 2, 2016

codenvy-ci commented Dec 6, 2016

codenvy-ci commented Dec 8, 2016

TylerJewell commented Jan 7, 2017

dkuleshov commented Jan 13, 2017

Exec agent Che IDE client #3032

Exec agent Che IDE client #3032

Conversation

dkuleshov commented Nov 9, 2016 • edited Loading

What does this PR do?

What issues does this PR fix or reference?

Changes to existing protocols

Websocket protocol

JsonRPC protocol

New exec agent client capabilities and use-cases

codenvy-ci commented Nov 9, 2016

codenvy-ci commented Nov 9, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codenvy-ci commented Nov 17, 2016

codenvy-ci commented Nov 18, 2016

codenvy-ci commented Dec 2, 2016

codenvy-ci commented Dec 6, 2016

codenvy-ci commented Dec 8, 2016

TylerJewell commented Jan 7, 2017

dkuleshov commented Jan 13, 2017

dkuleshov commented Nov 9, 2016 •

edited

Loading