feat(cli): User Management and Credentials sharing (n8n-io#3602)

* 🎉 starting feature development

* ✨ sharing/unsharing a credential (n8n-io#3601)

* 🎉 initial design

* ✨ sharing/unsharing of credentials

* ✅ add tests for EE credentials controller

* 💪 implement review comments

* 🛠 refactor agent creation and credential role locking

* 👕 linting adjustments (n8n-io#3691)

* 👕 Adjust rule `naming-convention`

* 👕 Fix `naming-convention` config value

* 👕 Disregard casing for EE-prefixed vars

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>

* 🛠 refactor authAgents in tests (n8n-io#3725)

* 🛠 refactor authAgent

* 🛠 refactor authAgent

* 🛠 refactor authAgent

* 🛠 refactor authAgent

* 👕 fix ts issue

* 🐘 add migration for mysql and postgres + add AuthAgent type

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>

* ⚡️ refactor existing credentials routes (n8n-io#3672)

* 🎉 initial design

* ✨ sharing/unsharing of credentials

* ♻️ split credential update route into controller and service

* 🔥 remove credentials test that is no longer applicable

* ♻️ split credential creation route into controller and service

* ♻️ split single credential get

* ♻️ split delete credentials route

* ♻️ split get all credentials route

* 🔥 remove unused imports in credentials contoller

* 🔥 remove console.log

* :refactor: changes to credentials controller and service from review

 - removed credentials from service function names
 - made relations list optional
 - put allowGlobalOwner in options objects
 - check length of relations array so join doesn't happen if empty
 - update some comments to further explain rationale
 - remove unneeded `Object.assign`
 - remove non-null assertion from test

* ♻️ move filtered credentials selected fields to variable

* ♻️ remove unneeded merges in credentials service

Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com>

* ✅ fix test

* 🐛 fix imports

* 👕 fix lint issue

* User Management: switch over to decorators to define routes (n8n-io#3827)

* Add permissions details to credentials for User Management (n8n-io#3863)

* ⚡ Open `GET /users`

* ⚡ Add permissions to cred service

* 🚚 Rename method

* ⚡ Refactor cred controller

* 🧪 Adjust test

* ✏️ Improve comment

* ✏️ Improve another comment

* ⚡ Account for multiple sharings

* 🐛 Fix access when user is editor

* 📘 Expand interface

* 📘 Relocate types

* 📘 Exempt cred entity with service-injected fields

* 📘 Adjust interface

* ♻️ Add permissions only in `GET /credentials`

* 🧪 Add expectations for `ownedBy`

* 🧪 Add sharing details test

* 🧪 Make `ownedBy` checks more granular

* 📘 Adjust interface

* 🚚 Rename cred getter

* ♻️ Refactor cred getter

* 🧪 Expand tests

* ♻️ Refactor to use guard

* 👕 Remove unneeded lint exception

* 🔥 Remove unneeded relation

* 🚚 Move relation to `GET /credentials/:id`

* 📘 Consolidate typings

* 🎨 Add multiline for readability

* 🔥 Remove unneeded type

* ✏️ Clarity comment

* ✏️ Make comments consistent

* 👕 Add exception to fix build

* 👕 Add more lint exceptions to fix build

* 🐛 Check for non-owner

* 📘 Improve typings

* 🧪 Temporarily skip tests

* 🔥 Remove `@ts-ignore`

* 👕 Move lint exceptions

* ♻️ Refactor cred service and controller

* ⚡ Simplify check

* ✏️ adjust naming to experimental

* ⚡️ add credentialsSharing flag to settings

* 🛠 add helper to check if UM is also enabled as dependency for CredentialsSharing

* 👕 fix lint error

* 🐘 change name of credential role

* 🚧 WIP batch sharing

* 🚧 WIP use put for sharing

* ✅ add tests for batch sharing, 🛠 implement review suggestions

* ✅ expand credential sharing tests for User Management (n8n-io#3931)

* 🧪 Expand cred sharing tests

* ⚡ Add recently added flags

* ✅ fix and adjust tests for /credentials

Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>

* ✨ User management v2 Front End (n8n-io#3795)

* feat: Added responsive generic page view layout.

* feat: Added empty state.

* feat: Added credentials view empty state.

* test: Added unit tests for N8nActionBox

* feat: Added credentials list initial design.

* feat: Added credential actions. Started working on filters.

* feat: Updated InfoTip markup, added tests and changed stories to typescript.

* feat: Added credentials filtering by type. Added support for apply/reset filters.

* feat: Added credential sharing user select and user list. Added paywall component.

* feat: Updated credentials view permissions.

* feat: Added support for temporary sharing config for unsaved credentials.

* test: Fixed broken snapshots.

* feat: Added overflow styles to page-view-layout list.

* feat: Handled sharee specific views.

* feat: Integration between FE and BE to support real-world credential sharing scenario.

* feat: Added front end permissions table.

* feat: Refactored credential sharing flow. Updated design elements.

* feat: Added margin and padding auto spacer utilities.

* feat: Rehauled permissions to support instanceOwner role and action inheritance.

* feat: Updated credentials view to apply filters automatically.

* feat: Removed apply filters button and added active button state.

* test: Updated component snapshots.

* refactor: Renamed ResourceSharee to ResourceReader.

* feat: Credential sharing error handling, permissions improvement.

* feat: Updated permissions and error handling.

* chore: Removed console.log.

* 🛠 refactor enabling of credentialsSharing

* feat: Removed owner menu selector from credentials when sharing is disabled.

* refactor: Moved EE features into ee store module file.

* 🛠 add sharing info to GET credentials/:id

* fix: Fixed initial credential data loading for sharing.

* chore: Removed console.log.

* 🐛 owner can fetch any credential

* 🛠 refactor users test

* 👕 fix build type issue

* fix: Removed owner tag when credential sharing is disabled. Fixed small reactivity issue.

* chore: Removed console.log.

* 🚧 separate fetching credentials between EE and open

* fix: Fixed empty dropdown in users list.

* fix: Fixed error message and initialization when credential gets unshared.

* ✅ add tests for fetching single credential

* Revert decorators based controllers

* ⚡️ adjust credentials test route to also allow testing for sharees (n8n-io#3999)

* ⚡️ pull data if user is sharee

* fix: Removed sharedWith and ownedBy from credentialData on testing credentials.

Co-authored-by: Alex Grozav <alex@grozav.com>

* 📈 add BE analytics

* 💪 improve credential test

* ⚡️ adjust tracking properties

* ⚡️ removed roles from tracking

* 🐛 fix build by removing imports

* 🐛 fix missed merge conflict

* feat: User management P2 Front End bug bash and improvements (n8n-io#4014)

* fix: Fixed type select size after reopening dropdown.

* fix: Fixed template cards.

* fix: Fixed card content size and copy input.

* fix: Fixed horizontal overflow.

* fix: Hiding el-tags scrollbar in select.

* fix: Added fallback credential icon. Added oAuth credential owner check.

* feat: Added disabled state to user select.

* feat: Added fallback scenario for non-existent credential types.

* feat: Adjusted credentials empty state to show that there are shared credentials.

* fix: Fixed time title.

* feat: Added actionable empty state when shared credentials are present.

* fix: Made action box x padding smaller

* feat: Repositioned owner tag for credential card.

* feat: Updated message box styling to use n8n css variables.

* feat: Added confirmation for deleting sharee.

* fix: Fixed deleted credential types. Fixed select in dropdown bug.

* fix: Various code improvements. Addressed PR review comments.

* fix: Fixed credential deletion errors.

* fix: Various code quality improvements.

* feat: N8N-4531 update cloud coming soon features (n8n-io#4025)

* feat: Showing different upcoming feature messages and format for cloud.

* fix: Changed url format.

* fix: Updated how cloud deployment is determined.

* feat: N8N-4527 implementing credential sharing FE telemetry (n8n-io#4023)

* feat: Added credential sharing telemetry.

* chore: Renamed computed function for consistency.

* refactor: Simplified subview telemetry sending.

* fix: Changed to callDebounced() helper.

* 📧 update email text

* fix: Adjusted feature coming soon margin.

* chore: Fixed type and line height for delete sharee confirmation modal.

* refactor(editor-ui): Update telemetry (n8n-io#4040)

* 🔥 Remove `identify` from BE

* ⚡ Add `versionCli`

* ⚡ Add node creator ignore input

* ⚡ Move obfuscators to editor-ui

* ⚡ Refactor `ph-no-capture`

* ⚡ Pass `user_id` to manual exec props

* 🚚 Relocate class in `SettingsApiView`

* ⚡ Add `userId` to BE PH `identify` call

* ⏪ Revert "⚡ Add `userId` to BE PH `identify` call"

This reverts commit 895aaa4.

* Revert ":rewind: Revert ":zap: Add `userId` to BE PH `identify` call""

This reverts commit b86a098.

* 🐛 Fix `Promise` handling in `track()` call

* ⏪ Restore `Db.collections` call

* ⚡ Set up PH payload to mirror RS

* 🔥 Remove excess `userId`

* 📘 Remove `userId` from interface

* 🔥 Remove unused ref and method

* fix: Fixed bug causing instanceOwner to become credential owner on update. (n8n-io#4079)

* 🐛 fix test for credential shared with member

* 👕 fix lint issues

* delete conflicting migration. this data is already seeded in CreateUserManagement

* feat: Expand obfuscation to User Management credential sharing (n8n-io#4070)

:zap: Expand obfuscation

* feat: Added credential sharing infotip for instance owner.

* bring back the migration. add a check to avoid conflicts on inserts

* fix(cli): use a non-env config flag to detect of enterprise features are enabled (n8n-io#4105)

* chore: Changed ampersand to and in translation.

* refactor(telemetry): Obfuscate code and JSON editors (n8n-io#4118)

:zap: Obfuscate code and JSON editors

* feat(editor): improve design and functionality of coming soon features (n8n-io#4116)

* feat: Improved coming soon feature design and functionality.

* style: Removed empty line.

* chore: Removed unused translation.

* fix: fix telemetry for credential creates and updates (n8n-io#4125)

fix telemetry for credential creates and updates

* feat: Display errors due to missing credentials in the correct node (n8n-io#4124)

feat: Display errors due to invalid credentials in the correct node when missing permissions

* fix: remove duplicate header for coming soon features in cloud deployment

* telemetry: fix the payload for `User viewed credential tab`

* telemetry: add credential_id to 'User selected credential from node modal'

* feat: update empty states for coming soon features

* Update ActionBox.spec.ts.snap

* replace UserSharingsDetails with a subset of User properties

* rename the CreateCredentialsEditorRole to CreateCredentialsUserRole

* move IUser to the workflow package

* use IUser in the frontend as well

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
Co-authored-by: Valya <68596159+valya@users.noreply.github.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <netroy@users.noreply.github.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
Co-authored-by: Omar Ajoue <krynble@gmail.com>

packages/cli/commands/worker.ts

@@ -41,6 +41,7 @@ import {
 	checkPermissionsForExecution,
 	getWorkflowOwner,
 } from '../src/UserManagement/UserManagementHelper';
+import { generateFailedExecutionFromError } from '../src/WorkflowHelpers';
 
 export class Worker extends Command {
 	static description = '\nStarts a n8n worker';
@@ -183,8 +184,6 @@ export class Worker extends Command {
 			settings: currentExecutionDb.workflowData.settings,
 		});
 
-		await checkPermissionsForExecution(workflow, workflowOwner.id);
-
 		const additionalData = await WorkflowExecuteAdditionalData.getBase(
 			workflowOwner.id,
 			undefined,
@@ -197,6 +196,20 @@ export class Worker extends Command {
 			{ retryOf: currentExecutionDb.retryOf as string },
 		);
 
+		try {
+			await checkPermissionsForExecution(workflow, workflowOwner.id);
+		} catch (error) {
+			const failedExecution = generateFailedExecutionFromError(
+				currentExecutionDb.mode,
+				error,
+				error.node,
+			);
+			await additionalData.hooks.executeHookFunctions('workflowExecuteAfter', [failedExecution]);
+			return {
+				success: true,
+			};
+		}
+
 		additionalData.hooks.hookFunctions.sendResponse = [
 			async (response: IExecuteResponsePromiseData): Promise<void> => {
 				const progress: Queue.WebhookResponse = {

packages/cli/config/schema.ts

@@ -875,6 +875,15 @@ export const schema = {
 		},
 	},
 
+	enterprise: {
+		features: {
+			sharing: {
+				format: Boolean,
+				default: false,
+			},
+		},
+	},
+
 	hiringBanner: {
 		enabled: {
 			doc: 'Whether hiring banner in browser console is enabled.',

packages/cli/src/CredentialsHelper.ts

@@ -805,7 +805,7 @@ export async function getCredentialWithoutUser(
 	return credential;
 }
 
-export function createCredentiasFromCredentialsEntity(
+export function createCredentialsFromCredentialsEntity(
 	credential: CredentialsEntity,
 	encrypt = false,
 ): Credentials {

packages/cli/src/Interfaces.ts

@@ -28,16 +28,17 @@ import { Repository } from 'typeorm';
 
 import { ChildProcess } from 'child_process';
 import { Url } from 'url';
-import { Request } from 'express';
-import { WorkflowEntity } from './databases/entities/WorkflowEntity';
-import { TagEntity } from './databases/entities/TagEntity';
-import { Role } from './databases/entities/Role';
-import { User } from './databases/entities/User';
-import { SharedCredentials } from './databases/entities/SharedCredentials';
-import { SharedWorkflow } from './databases/entities/SharedWorkflow';
-import { Settings } from './databases/entities/Settings';
-import { InstalledPackages } from './databases/entities/InstalledPackages';
-import { InstalledNodes } from './databases/entities/InstalledNodes';
+
+import type { Request } from 'express';
+import type { InstalledNodes } from './databases/entities/InstalledNodes';
+import type { InstalledPackages } from './databases/entities/InstalledPackages';
+import type { Role } from './databases/entities/Role';
+import type { Settings } from './databases/entities/Settings';
+import type { SharedCredentials } from './databases/entities/SharedCredentials';
+import type { SharedWorkflow } from './databases/entities/SharedWorkflow';
+import type { TagEntity } from './databases/entities/TagEntity';
+import type { User } from './databases/entities/User';
+import type { WorkflowEntity } from './databases/entities/WorkflowEntity';
 
 export interface IActivationError {
 	time: number;
@@ -153,6 +154,7 @@ export interface ICredentialsBase {
 export interface ICredentialsDb extends ICredentialsBase, ICredentialsEncrypted {
 	id: number | string;
 	name: string;
+	shared?: SharedCredentials[];
 }
 
 export interface ICredentialsResponse extends ICredentialsDb {
@@ -506,6 +508,9 @@ export interface IN8nUISettings {
 		type: string;
 	};
 	isNpmAvailable: boolean;
+	enterprise: {
+		sharing: boolean;
+	};
 }
 
 export interface IPersonalizationSurveyAnswers {

packages/cli/src/InternalHooks.ts

@@ -1,5 +1,5 @@
 /* eslint-disable import/no-cycle */
-import { get as pslGet } from 'psl';
+import { snakeCase } from 'change-case';
 import { BinaryDataManager } from 'n8n-core';
 import {
 	INodesGraphResult,
@@ -8,23 +8,28 @@ import {
 	ITelemetryTrackProperties,
 	TelemetryHelpers,
 } from 'n8n-workflow';
-import { snakeCase } from 'change-case';
+import { get as pslGet } from 'psl';
 import {
 	IDiagnosticInfo,
 	IInternalHooksClass,
 	ITelemetryUserDeletionData,
 	IWorkflowBase,
 	IWorkflowDb,
 } from '.';
-import { Telemetry } from './telemetry';
 import { IExecutionTrackProperties } from './Interfaces';
+import { Telemetry } from './telemetry';
 
 export class InternalHooksClass implements IInternalHooksClass {
 	private versionCli: string;
 
 	private nodeTypes: INodeTypes;
 
-	constructor(private telemetry: Telemetry, versionCli: string, nodeTypes: INodeTypes) {
+	constructor(
+		private telemetry: Telemetry,
+		private instanceId: string,
+		versionCli: string,
+		nodeTypes: INodeTypes,
+	) {
 		this.versionCli = versionCli;
 		this.nodeTypes = nodeTypes;
 	}
@@ -187,6 +192,7 @@ export class InternalHooksClass implements IInternalHooksClass {
 				}
 
 				const manualExecEventProperties: ITelemetryTrackProperties = {
+					user_id: userId,
 					workflow_id: workflow.id.toString(),
 					status: properties.success ? 'success' : 'failed',
 					error_message: properties.error_message as string,
@@ -398,6 +404,34 @@ export class InternalHooksClass implements IInternalHooksClass {
 		);
 	}
 
+	/**
+	 * Credentials
+	 */
+
+	async onUserCreatedCredentials(userCreatedCredentialsData: {
+		credential_type: string;
+		credential_id: string;
+		public_api: boolean;
+	}): Promise<void> {
+		return this.telemetry.track('User created credentials', {
+			...userCreatedCredentialsData,
+			instance_id: this.instanceId,
+		});
+	}
+
+	async onUserSharedCredentials(userSharedCredentialsData: {
+		credential_type: string;
+		credential_id: string;
+		user_id_sharer: string;
+		user_ids_sharees_added: string[];
+		sharees_removed: number | null;
+	}): Promise<void> {
+		return this.telemetry.track('User updated cred sharing', {
+			...userSharedCredentialsData,
+			instance_id: this.instanceId,
+		});
+	}
+
 	/**
 	 * Community nodes backend telemetry events
 	 */

packages/cli/src/InternalHooksManager.ts

@@ -18,6 +18,7 @@ export class InternalHooksManager {
 		if (!this.internalHooksInstance) {
 			this.internalHooksInstance = new InternalHooksClass(
 				new Telemetry(instanceId, versionCli),
+				instanceId,
 				versionCli,
 				nodeTypes,
 			);

packages/cli/src/ResponseHelper.ts

@@ -148,9 +148,11 @@ const isUniqueConstraintError = (error: Error) =>
  * @returns
  */
 
-export function send(processFunction: (req: Request, res: Response) => Promise<any>, raw = false) {
-	// eslint-disable-next-line consistent-return
-	return async (req: Request, res: Response) => {
+export function send<T, R extends Request, S extends Response>(
+	processFunction: (req: R, res: S) => Promise<T>,
+	raw = false,
+) {
+	return async (req: R, res: S) => {
 		try {
 			const data = await processFunction(req, res);
 

packages/cli/src/Server.ts

@@ -29,20 +29,19 @@
 /* eslint-disable import/no-dynamic-require */
 /* eslint-disable no-await-in-loop */
 
-import express from 'express';
-import { readFileSync, promises } from 'fs';
 import { exec as callbackExec } from 'child_process';
-import _ from 'lodash';
+import { promises, readFileSync } from 'fs';
+import os from 'os';
 import { dirname as pathDirname, join as pathJoin, resolve as pathResolve } from 'path';
-import { FindManyOptions, getConnectionManager, In } from 'typeorm';
-import bodyParser from 'body-parser';
+import { createHmac } from 'crypto';
+import { promisify } from 'util';
 import cookieParser from 'cookie-parser';
-import history from 'connect-history-api-fallback';
-import os from 'os';
+import express from 'express';
+import _ from 'lodash';
+import { FindManyOptions, getConnectionManager, In } from 'typeorm';
 // eslint-disable-next-line import/no-extraneous-dependencies
-import clientOAuth1, { RequestOptions } from 'oauth-1.0a';
 import axios, { AxiosRequestConfig } from 'axios';
-import { createHmac } from 'crypto';
+import clientOAuth1, { RequestOptions } from 'oauth-1.0a';
 // IMPORTANT! Do not switch to anther bcrypt library unless really necessary and
 // tested with all possible systems like Windows, Alpine on ARM, FreeBSD, ...
 import { compare } from 'bcryptjs';
@@ -70,8 +69,41 @@ import jwks from 'jwks-rsa';
 import timezones from 'google-timezones-json';
 import parseUrl from 'parseurl';
 import promClient, { Registry } from 'prom-client';
-import { promisify } from 'util';
+import history from 'connect-history-api-fallback';
+import bodyParser from 'body-parser';
+import config from '../config';
 import * as Queue from './Queue';
+
+import { InternalHooksManager } from './InternalHooksManager';
+import { getCredentialTranslationPath } from './TranslationHelpers';
+import { WEBHOOK_METHODS } from './WebhookHelpers';
+import { getSharedWorkflowIds, whereClause } from './WorkflowHelpers';
+
+import { nodesController } from './api/nodes.api';
+import { workflowsController } from './api/workflows.api';
+import { AUTH_COOKIE_NAME, RESPONSE_ERROR_MESSAGES } from './constants';
+import { credentialsController } from './credentials/credentials.controller';
+import { oauth2CredentialController } from './credentials/oauth2Credential.api';
+import type {
+	ExecutionRequest,
+	NodeParameterOptionsRequest,
+	OAuthRequest,
+	WorkflowRequest,
+} from './requests';
+import { userManagementRouter } from './UserManagement';
+import { resolveJwt } from './UserManagement/auth/jwt';
+
+import { executionsController } from './api/executions.api';
+import { nodeTypesController } from './api/nodeTypes.api';
+import { tagsController } from './api/tags.api';
+import { isCredentialsSharingEnabled } from './credentials/helpers';
+import { loadPublicApiVersions } from './PublicApi';
+import * as telemetryScripts from './telemetry/scripts';
+import {
+	getInstanceBaseUrl,
+	isEmailSetUp,
+	isUserManagementEnabled,
+} from './UserManagement/UserManagementHelper';
 import {
 	ActiveExecutions,
 	ActiveWorkflowRunner,
@@ -82,6 +114,8 @@ import {
 	Db,
 	ExternalHooks,
 	GenericHelpers,
+	getCredentialForUser,
+	getCredentialWithoutUser,
 	ICredentialsDb,
 	ICredentialsOverwrite,
 	ICustomRequest,
@@ -101,41 +135,8 @@ import {
 	WebhookHelpers,
 	WebhookServer,
 	WorkflowExecuteAdditionalData,
-	getCredentialForUser,
-	getCredentialWithoutUser,
 } from '.';
 
-import config from '../config';
-
-import { InternalHooksManager } from './InternalHooksManager';
-import { getSharedWorkflowIds, whereClause } from './WorkflowHelpers';
-import { getCredentialTranslationPath, getNodeTranslationPath } from './TranslationHelpers';
-import { WEBHOOK_METHODS } from './WebhookHelpers';
-
-import { userManagementRouter } from './UserManagement';
-import { resolveJwt } from './UserManagement/auth/jwt';
-import type {
-	ExecutionRequest,
-	NodeParameterOptionsRequest,
-	OAuthRequest,
-	WorkflowRequest,
-} from './requests';
-import { AUTH_COOKIE_NAME, RESPONSE_ERROR_MESSAGES } from './constants';
-import { credentialsController } from './api/credentials.api';
-import { executionsController } from './api/executions.api';
-import { workflowsController } from './api/workflows.api';
-import { nodesController } from './api/nodes.api';
-import { oauth2CredentialController } from './api/oauth2Credential.api';
-import { tagsController } from './api/tags.api';
-import {
-	getInstanceBaseUrl,
-	isEmailSetUp,
-	isUserManagementEnabled,
-} from './UserManagement/UserManagementHelper';
-import { loadPublicApiVersions } from './PublicApi';
-import * as telemetryScripts from './telemetry/scripts';
-import { nodeTypesController } from './api/nodeTypes.api';
-
 require('body-parser-xml')(bodyParser);
 
 const exec = promisify(callbackExec);
@@ -310,6 +311,9 @@ class App {
 				type: config.getEnv('deployment.type'),
 			},
 			isNpmAvailable: false,
+			enterprise: {
+				sharing: false,
+			},
 		};
 	}
 
@@ -336,6 +340,11 @@ class App {
 				config.getEnv('userManagement.skipInstanceOwnerSetup') === false,
 		});
 
+		// refresh enterprise status
+		Object.assign(this.frontendSettings.enterprise, {
+			sharing: isCredentialsSharingEnabled(),
+		});
+
 		if (config.get('nodes.packagesMissing').length > 0) {
 			this.frontendSettings.missingPackages = true;
 		}