feat: support installing the agent with TAR installer and the install…

… subcommand (#378)

* first version of feature file

* fix: typos

* fix: proper BDD sequencing

* chore: remove non-gherkin line

* chore: refine gherkin tags

* chore: reuse existing step for uninstalling/restarting the agent

* chore: export kibana base URL from client

* feat: add support for centos/debian with TAR.GZ installer

* chore: rename variable

* feat: support for a TAR installer

We refactored the installer install actions to delegate the execution to
the installer instead of running it in the tests.

We also added the new steps for the commands feature.

* feat: create a local cache for downloaded binaries

* fixup. new tar installer

* feat: support mulltiple installer types at feature level

* chore: create token before running the agent

* chore: move enroll method to the installer

* chore: move installFn to preInstalllFn

* chore: proper life cycle for install and enroll

The life cycle (preInstall > install > enroll > postInstall) depends on
the installer:

- TAR leverages preInstall phase to extract the TAR file and create the
proper layout

- RPM and DEB leverage install phase to use package managers to install
the artifact
- TAR leverages install phase to actually install and enroll the agent

- RPM and DEP needs an extra enrollment call
- TAR defines an enroll method to be used outside the life cycle

- RPM and DEB leverages postInstall phase to enable and run the service
using systemd

* chore: add missing examples in feature file

* chore: enroll ooutside of the install life cycle

* chore: use existing step

* chore: match existing scenario for installing an agent

* chore: move to fleet feature file as an example

* chore: move stop scenario to fleet feature file as an example

* chore: move restart scenario as a Fleet example

* chore: move unenroll scenario as a Fleet example

* chore: move reenroll scenario as Fleet example

* fix: use proper log path for TAR installer

* fix: create the tokens in the Default policy

* fix: remove the proper service name

* chore: use tar installer in endpoint feature

* fix: update Elastic Endpoint integration title

* feat: implement the step for the empty working dir folder

* fix: proper container name

* chore: capture specific error for installing agent with revoked token

* chore: use tar installer for fleet

* chore: join process steps

* chore: support for restarting the agent

* chore: move subcommand scenarios to fleet ones

* chore: simplify getting aggent hash and logs

* fix: agents are offline after uninstall

* fix: update desired status too

* fix: include inactive status in check

* chore: increase timeout for agent status checks

Co-authored-by: Eric Davis <eric.davis@elastic.co>

cli/services/kibana.go

@@ -115,6 +115,11 @@ func (k *KibanaClient) DeleteIntegrationFromPolicy(packageConfigID string) (stri
 	return body, err
 }
 
+// GetBaseURL retrieves the base URl where Kibana is listening
+func (k *KibanaClient) GetBaseURL() string {
+	return k.baseURL
+}
+
 // GetIntegration sends a GET request to fetch an integration by name and version
 func (k *KibanaClient) GetIntegration(packageName string, version string) (string, error) {
 	k.withURL(fmt.Sprintf(ingestManagerIntegrationURL, packageName, version))

cli/services/kibana_test.go

@@ -10,6 +10,13 @@ import (
 	"github.com/stretchr/testify/assert"
 )
 
+func TestGetBaseURL(t *testing.T) {
+	client := NewKibanaClient()
+	assert.NotNil(t, client)
+
+	assert.Equal(t, "http://localhost:5601", client.GetBaseURL())
+}
+
 func TestNewClient(t *testing.T) {
 	client := NewKibanaClient()
 

e2e/_suites/fleet/features/agent_endpoint_integration.feature

@@ -4,35 +4,35 @@ Feature: Agent Endpoint Integration
 
 @deploy-endpoint-with-agent
 Scenario: Adding the Endpoint Integration to an Agent makes the host to show in Security App
-  Given a "centos" agent is deployed to Fleet
+  Given a "centos" agent is deployed to Fleet with "tar" installer
     And the agent is listed in Fleet as "online"
-  When the "Elastic Endpoint Security" integration is "added" in the policy
-  Then the "Elastic Endpoint Security" datasource is shown in the policy as added
+  When the "Endpoint Security" integration is "added" in the policy
+  Then the "Endpoint Security" datasource is shown in the policy as added
     And the host name is shown in the Administration view in the Security App as "online"
 
 @endpoint-policy-check
 Scenario: Deploying an Endpoint makes policies to appear in the Security App
-  When an Endpoint is successfully deployed with a "centos" Agent
+  When an Endpoint is successfully deployed with a "centos" Agent using "tar" installer
   Then the policy response will be shown in the Security App
 
 @set-policy-and-check-changes
 Scenario: Changing an Agent policy is reflected in the Security App
-  Given an Endpoint is successfully deployed with a "centos" Agent
+  Given an Endpoint is successfully deployed with a "centos" Agent using "tar" installer
   When the policy is updated to have "malware" in "detect" mode
   Then the policy will reflect the change in the Security App
 
 @deploy-endpoint-then-unenroll-agent
 Scenario: Un-enrolling Elastic Agent stops Elastic Endpoint
-  Given an Endpoint is successfully deployed with a "centos" Agent
+  Given an Endpoint is successfully deployed with a "centos" Agent using "tar" installer
   When the agent is un-enrolled
   Then the agent is listed in Fleet as "inactive"
     And the host name is not shown in the Administration view in the Security App
     And the "elastic-endpoint" process is in the "stopped" state on the host
 
 @deploy-endpoint-then-remove-it-from-policy
 Scenario: Removing Endpoint from Agent policy stops the connected Endpoint
-  Given an Endpoint is successfully deployed with a "centos" Agent
-  When the "Elastic Endpoint Security" integration is "removed" in the policy
+  Given an Endpoint is successfully deployed with a "centos" Agent using "tar" installer
+  When the "Endpoint Security" integration is "removed" in the policy
   Then the agent is listed in Fleet as "online"
     But the host name is not shown in the Administration view in the Security App
     And the "elastic-endpoint" process is in the "stopped" state on the host

e2e/_suites/fleet/features/fleet_mode_agent.feature

@@ -4,70 +4,98 @@ Feature: Fleet Mode Agent
 
 @enroll
 Scenario Outline: Deploying the <os> agent
-  When a "<os>" agent is deployed to Fleet
+  When a "<os>" agent is deployed to Fleet with "<installer>" installer
   Then the "elastic-agent" process is in the "started" state on the host
     And the "filebeat" process is in the "started" state on the host
     And the "metricbeat" process is in the "started" state on the host
     And the agent is listed in Fleet as "online"
     And system package dashboards are listed in Fleet
 Examples:
-| os     |
-| centos |
-| debian |
+| os     | installer |
+| centos | tar       |
+| debian | tar       |
 
 @stop-agent
 Scenario Outline: Stopping the <os> agent stops backend processes
-  Given a "<os>" agent is deployed to Fleet
+  Given a "<os>" agent is deployed to Fleet with "<installer>" installer
   When the "elastic-agent" process is "stopped" on the host
   Then the "filebeat" process is in the "stopped" state on the host
     And the "metricbeat" process is in the "stopped" state on the host
 Examples:
-| os     |
-| centos |
-| debian |
+| os     | installer |
+| centos | tar       |
+| debian | tar       |
+
+@restart-agent
+Scenario Outline: Restarting the installed <os> agent
+  Given a "<os>" agent is deployed to Fleet with "<installer>" installer
+  When the "elastic-agent" process is "restarted" on the host
+    And the "filebeat" process is in the "started" state on the host
+    And the "metricbeat" process is in the "started" state on the host
+    And the agent is listed in Fleet as "online"
+Examples:
+| os     | installer |
+| centos | tar       |
+| debian | tar       |
 
 @restart-host
 Scenario Outline: Restarting the <os> host with persistent agent restarts backend processes
-  Given a "<os>" agent is deployed to Fleet
+  Given a "<os>" agent is deployed to Fleet with "<installer>" installer
   When the host is restarted
   Then the "elastic-agent" process is in the "started" state on the host
     And the "filebeat" process is in the "started" state on the host
     And the "metricbeat" process is in the "started" state on the host
 Examples:
-| os     |
-| centos |
-| debian |
+| os     | installer |
+| centos | tar       |
+| debian | tar       |
 
 @unenroll
 Scenario Outline: Un-enrolling the <os> agent
-  Given a "<os>" agent is deployed to Fleet
+  Given a "<os>" agent is deployed to Fleet with "<installer>" installer
   When the agent is un-enrolled
   Then the "elastic-agent" process is in the "started" state on the host
     And the agent is listed in Fleet as "inactive"
+    And the "filebeat" process is in the "stopped" state on the host
+    And the "metricbeat" process is in the "stopped" state on the host
 Examples:
-| os     |
-| centos |
-| debian |
+| os     | installer |
+| centos | tar       |
+| debian | tar       |
 
 @reenroll
 Scenario Outline: Re-enrolling the <os> agent
-  Given a "<os>" agent is deployed to Fleet
+  Given a "<os>" agent is deployed to Fleet with "<installer>" installer
     And the agent is un-enrolled
     And the "elastic-agent" process is "stopped" on the host
   When the agent is re-enrolled on the host
     And the "elastic-agent" process is "started" on the host
   Then the agent is listed in Fleet as "online"
 Examples:
-| os     |
-| centos |
-| debian |
+| os     | installer |
+| centos | tar       |
+| debian | tar       |
 
 @revoke-token
 Scenario Outline: Revoking the enrollment token for the <os> agent
-  Given a "<os>" agent is deployed to Fleet
+  Given a "<os>" agent is deployed to Fleet with "<installer>" installer
   When the enrollment token is revoked
   Then an attempt to enroll a new agent fails
 Examples:
-| os     |
-| centos |
-| debian |
+| os     | installer |
+| centos | tar       |
+| debian | tar       |
+
+@uninstall-host
+Scenario Outline: Un-installing the installed <os> agent
+  Given a "<os>" agent is deployed to Fleet with "<installer>" installer
+  When the "elastic-agent" process is "uninstalled" on the host
+  Then the "elastic-agent" process is in the "stopped" state on the host
+    And the "filebeat" process is in the "stopped" state on the host
+    And the "metricbeat" process is in the "stopped" state on the host
+    And the file system Agent folder is empty
+    And the agent is listed in Fleet as "offline"
+Examples:
+| os     | installer |
+| centos | tar       |
+| debian | tar       |