Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create credentials namespace and add tests #2831

Merged
merged 39 commits into from
Feb 23, 2022
Merged

Create credentials namespace and add tests #2831

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
e06e147
:test_tube: Fix failing test
ivov Feb 16, 2022
013aab0
:blue_book: Improve `createAgent` signature
ivov Feb 16, 2022
2af92ab
:truck: Fix `LoggerProxy` import
ivov Feb 16, 2022
491ac27
:sparkles: Create credentials endpoints namespace
ivov Feb 16, 2022
011ed90
:test_tube: Set up initial tests
ivov Feb 16, 2022
8ca92f4
:zap: Add validation to model
ivov Feb 16, 2022
a21ff8c
:zap: Adjust validation
ivov Feb 16, 2022
217c149
:test_tube: Add test
ivov Feb 16, 2022
e73afad
:truck: Sort creds endpoints
ivov Feb 16, 2022
4a42471
:pencil2: Plan out pending tests
ivov Feb 16, 2022
73ae460
:test_tube: Add deletion tests
ivov Feb 16, 2022
2624e11
:test_tube: Add patch tests
ivov Feb 16, 2022
937de2f
:test_tube: Add get cred tests
ivov Feb 16, 2022
1f5af23
:truck: Hoist import
ivov Feb 16, 2022
ea1b429
:twisted_rightwards_arrows: Merge parent branch
ivov Feb 16, 2022
4786f67
:pencil2: Make test descriptions consistent
ivov Feb 16, 2022
bac584f
:pencil2: Adjust description
ivov Feb 16, 2022
4012671
:test_tube: Add missing test
ivov Feb 16, 2022
69c587a
:pencil2: Make get descriptions consistent
ivov Feb 17, 2022
06aeb49
:rewind: Undo line break
ivov Feb 17, 2022
b002eed
:zap: Refactor to simplify `saveCredential`
ivov Feb 17, 2022
eac3b42
:test_tube: Add non-owned tests for owner
ivov Feb 17, 2022
a0b43c8
:pencil2: Improve naming
ivov Feb 17, 2022
7dc103e
:pencil2: Add clarifying comments
ivov Feb 17, 2022
80162a3
:truck: Improve imports
ivov Feb 17, 2022
390cdb8
:zap: Initialize config file
ivov Feb 17, 2022
7682701
:fire: Remove unneeded import
ivov Feb 17, 2022
24e2aee
:truck: Rename dir
ivov Feb 19, 2022
7aa60c9
:zap: Adjust deletion call
ivov Feb 19, 2022
2821265
:zap: Adjust error code
ivov Feb 19, 2022
f096c19
:pencil2: Touch up comment
ivov Feb 19, 2022
5e3e023
:zap: Optimize fetching with `@RelationId`
ivov Feb 19, 2022
e99c60f
:test_tube: Add expectations
ivov Feb 19, 2022
13c39f8
:zap: Simplify mock calls
ivov Feb 19, 2022
8a8fe69
:blue_book: Set deep readonly to object constants
ivov Feb 21, 2022
b405c33
:fire: Remove unused param and encryption key
ivov Feb 22, 2022
5f1f00b
:zap: Add more `@RelationId` calls in models
ivov Feb 22, 2022
ab70b94
:twisted_rightwards_arrows: Merge parent branch
ivov Feb 22, 2022
3104fc5
:rewind: Restore
ivov Feb 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
345 changes: 2 additions & 343 deletions packages/cli/src/Server.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,6 @@ import {
IDataObject,
INodeCredentials,
INodeCredentialsDetails,
INodeCredentialTestRequest,
INodeCredentialTestResult,
INodeParameters,
INodePropertyOptions,
INodeType,
Expand Down Expand Up @@ -112,10 +110,7 @@ import {
GenericHelpers,
ICredentialsDb,
ICredentialsOverwrite,
ICredentialsResponse,
ICustomRequest,
IExecutionDeleteFilter,
IExecutionFlatted,
IExecutionFlattedDb,
IExecutionFlattedResponse,
IExecutionPushResponse,
Expand All @@ -130,7 +125,6 @@ import {
ITagWithCountDb,
IWorkflowExecutionDataProcess,
IWorkflowResponse,
IPersonalizationSurveyAnswers,
NodeTypes,
Push,
ResponseHelper,
Expand Down Expand Up @@ -169,8 +163,8 @@ import type {
import { DEFAULT_EXECUTIONS_GET_ALL_LIMIT, validateEntity } from './GenericHelpers';
import { ExecutionEntity } from './databases/entities/ExecutionEntity';
import { SharedWorkflow } from './databases/entities/SharedWorkflow';
import { SharedCredentials } from './databases/entities/SharedCredentials';
import { RESPONSE_ERROR_MESSAGES } from './constants';
import { credentialsEndpoints } from './endpoints/credentials.endpoints';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would prefer sth like this:

Suggested change
import { credentialsEndpoints } from './endpoints/credentials.endpoints';
import { credentialsEndpoints } from './api/namespaces/credentials';

Copy link
Contributor Author

@ivov ivov Feb 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed as suggested, but please consider:

  • File searchability. This is a monorepo where we already have four different credentials.ts files that match on file search - even searching for credentials name (file plus part of new dirname) actually brings up CredentialsSelectModal.vue first. On the other hand, using a dotted suffix like .endpoints brings up the file already on credentials.e, followed also by the relevant tests credentials.endpoints.test.ts, with all other similar but irrelevant credentials files getting filtered out. We could use .api instead of .endpoints to keep it concise, ideally mirrored in the tests file.
  • Consistency. I think this bundle of endpoints does not constitute a namespace (e.g. thinking of wildcard * imports, TS types grouped in namespaces, etc.) because in Server.ts we are not referencing them as <namespace>.<entity>. These endpoints are not namespaced, just split off into a separate file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also let's move UM endpoints to this dir as well? In this PR or in another.


require('body-parser-xml')(bodyParser);

Expand Down Expand Up @@ -1557,342 +1551,7 @@ class App {
// Credentials
// ----------------------------------------

this.app.get(
`/${this.restEndpoint}/credentials/new`,
ResponseHelper.send(async (req: WorkflowRequest.NewName): Promise<{ name: string }> => {
const requestedName =
req.query.name && req.query.name !== '' ? req.query.name : this.defaultCredentialsName;

return await GenericHelpers.generateUniqueName(requestedName, 'credentials');
}),
);

// Deletes a specific credential
this.app.delete(
`/${this.restEndpoint}/credentials/:id`,
ResponseHelper.send(async (req: CredentialRequest.Delete) => {
const { id: credentialId } = req.params;

const shared = await Db.collections.SharedCredentials!.findOne({
relations: ['credentials'],
where: whereClause({
user: req.user,
entityType: 'credentials',
entityId: credentialId,
}),
});

if (!shared) {
throw new ResponseHelper.ResponseError(
`Credential with ID "${credentialId}" could not be found to be deleted.`,
undefined,
404,
);
}

await this.externalHooks.run('credentials.delete', [credentialId]);

await Db.collections.Credentials!.delete(credentialId);

return true;
}),
);

// Creates new credentials
this.app.post(
`/${this.restEndpoint}/credentials`,
ResponseHelper.send(async (req: CredentialRequest.Create) => {
delete req.body.id; // delete if sent

const newCredential = new CredentialsEntity();

Object.assign(newCredential, req.body);

await validateEntity(newCredential);

// Add the added date for node access permissions
for (const nodeAccess of newCredential.nodesAccess) {
nodeAccess.date = this.getCurrentDate();
}

const encryptionKey = await UserSettings.getEncryptionKey();

if (!encryptionKey) {
throw new Error(RESPONSE_ERROR_MESSAGES.NO_ENCRYPTION_KEY);
}

// Encrypt the data
const coreCredential = new Credentials(
{ id: null, name: newCredential.name },
newCredential.type,
newCredential.nodesAccess,
);

// @ts-ignore
coreCredential.setData(newCredential.data, encryptionKey);

const encryptedData = coreCredential.getDataToSave() as ICredentialsDb;

Object.assign(newCredential, encryptedData);

await this.externalHooks.run('credentials.create', [encryptedData]);

const role = await Db.collections.Role!.findOneOrFail({
name: 'owner',
scope: 'credential',
});

const savedCredential = await getConnection().transaction(async (transactionManager) => {
const savedCredential = await transactionManager.save<CredentialsEntity>(newCredential);

savedCredential.data = newCredential.data;

const newSharedCredential = new SharedCredentials();

Object.assign(newSharedCredential, {
role,
user: req.user,
credentials: savedCredential,
});

await transactionManager.save<SharedCredentials>(newSharedCredential);

return savedCredential;
});

const { id, ...rest } = savedCredential;

return { id: id.toString(), ...rest };
}),
);

// Test credentials
this.app.post(
`/${this.restEndpoint}/credentials-test`,
ResponseHelper.send(
async (req: express.Request, res: express.Response): Promise<INodeCredentialTestResult> => {
const incomingData = req.body as INodeCredentialTestRequest;

const encryptionKey = await UserSettings.getEncryptionKey();
if (encryptionKey === undefined) {
return {
status: 'Error',
message: 'No encryption key got found to decrypt the credentials!',
};
}

const credentialsHelper = new CredentialsHelper(encryptionKey);

const credentialType = incomingData.credentials.type;
return credentialsHelper.testCredentials(
credentialType,
incomingData.credentials,
incomingData.nodeToTestWith,
);
},
),
);

// Updates existing credentials
this.app.patch(
`/${this.restEndpoint}/credentials/:id`,
ResponseHelper.send(async (req: CredentialRequest.Update): Promise<ICredentialsResponse> => {
const { id: credentialId } = req.params;

const updateData = new CredentialsEntity();
Object.assign(updateData, req.body);

const shared = await Db.collections.SharedCredentials!.findOne({
relations: ['credentials'],
where: whereClause({
user: req.user,
entityType: 'credentials',
entityId: credentialId,
}),
});

if (!shared) {
throw new ResponseHelper.ResponseError(
`Credential with ID "${credentialId}" could not be found to be updated.`,
undefined,
404,
);
}

const { credentials: credential } = shared;

// Add the date for newly added node access permissions
for (const nodeAccess of updateData.nodesAccess) {
if (!nodeAccess.date) {
nodeAccess.date = this.getCurrentDate();
}
}

const encryptionKey = await UserSettings.getEncryptionKey();

if (!encryptionKey) {
throw new Error(RESPONSE_ERROR_MESSAGES.NO_ENCRYPTION_KEY);
}

const coreCredential = new Credentials(
{ id: credential.id.toString(), name: credential.name },
credential.type,
credential.nodesAccess,
credential.data,
);

const decryptedData = coreCredential.getData(encryptionKey);

// Do not overwrite the oauth data else data like the access or refresh token would get lost
// everytime anybody changes anything on the credentials even if it is just the name.
if (decryptedData.oauthTokenData) {
// @ts-ignore
updateData.data.oauthTokenData = decryptedData.oauthTokenData;
}

// Encrypt the data
const credentials = new Credentials(
{ id: credentialId, name: updateData.name },
updateData.type,
updateData.nodesAccess,
);

// @ts-ignore
credentials.setData(updateData.data, encryptionKey);

const newCredentialData = credentials.getDataToSave() as ICredentialsDb;

// Add special database related data
newCredentialData.updatedAt = this.getCurrentDate();

await this.externalHooks.run('credentials.update', [newCredentialData]);

// Update the credentials in DB
await Db.collections.Credentials!.update(credentialId, newCredentialData);

// We sadly get nothing back from "update". Neither if it updated a record
// nor the new value. So query now the hopefully updated entry.
const responseData = await Db.collections.Credentials!.findOne(credentialId);

if (responseData === undefined) {
throw new ResponseHelper.ResponseError(
`Credential with ID "${credentialId}" could not be found to be updated.`,
undefined,
400,
);
}

// Remove the encrypted data as it is not needed in the frontend
const { id, data, ...rest } = responseData;

return {
id: id.toString(),
...rest,
};
}),
);

// Returns specific credentials
this.app.get(
`/${this.restEndpoint}/credentials/:id`,
ResponseHelper.send(async (req: CredentialRequest.Get) => {
const { id: credentialId } = req.params;

const shared = await Db.collections.SharedCredentials!.findOne({
relations: ['credentials'],
where: whereClause({
user: req.user,
entityType: 'credentials',
entityId: credentialId,
}),
});

if (!shared) return {};

const { credentials: credential } = shared;

if (req.query.includeData !== 'true') {
const { data, id, ...rest } = credential;

return {
id: id.toString(),
...rest,
};
}

const { data, id, ...rest } = credential;

const encryptionKey = await UserSettings.getEncryptionKey();

if (!encryptionKey) {
throw new Error(RESPONSE_ERROR_MESSAGES.NO_ENCRYPTION_KEY);
}

const coreCredential = new Credentials(
{ id: credential.id.toString(), name: credential.name },
credential.type,
credential.nodesAccess,
credential.data,
);

return {
id: id.toString(),
data: coreCredential.getData(encryptionKey),
...rest,
};
}),
);

// Returns all the saved credentials
this.app.get(
`/${this.restEndpoint}/credentials`,
ResponseHelper.send(
async (req: CredentialRequest.GetAll): Promise<ICredentialsResponse[]> => {
let credentials: ICredentialsDb[] = [];

const filter: Record<string, string> = req.query.filter
? JSON.parse(req.query.filter)
: {};

if (req.user.globalRole.name === 'owner') {
credentials = await Db.collections.Credentials!.find({
select: ['id', 'name', 'type', 'nodesAccess', 'createdAt', 'updatedAt'],
where: filter,
});
} else {
const shared = await Db.collections.SharedCredentials!.find({
relations: ['credentials'],
where: whereClause({
user: req.user,
entityType: 'credentials',
}),
});

if (!shared.length) return [];

credentials = await Db.collections.Credentials!.find({
select: ['id', 'name', 'type', 'nodesAccess', 'createdAt', 'updatedAt'],
where: {
id: In(shared.map(({ credentials }) => credentials.id)),
...filter,
},
});
}

let encryptionKey;

if (req.query.includeData === 'true') {
encryptionKey = await UserSettings.getEncryptionKey();

if (!encryptionKey) {
throw new Error(RESPONSE_ERROR_MESSAGES.NO_ENCRYPTION_KEY);
}
}

return credentials.map(({ id, ...rest }) => ({ id: id.toString(), ...rest }));
},
),
);
credentialsEndpoints.apply(this);

// ----------------------------------------
// Credential-Types
Expand Down
4 changes: 3 additions & 1 deletion packages/cli/src/UserManagement/Interfaces.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/* eslint-disable import/no-cycle */
import { Application } from 'express';
import { JwtFromRequestFunction } from 'passport-jwt';
import { IPersonalizationSurveyAnswers } from '../Interfaces';
import type { IExternalHooksClass, IPersonalizationSurveyAnswers } from '../Interfaces';

export interface JwtToken {
token: string;
Expand Down Expand Up @@ -32,4 +32,6 @@ export interface PublicUser {
export interface N8nApp {
app: Application;
restEndpoint: string;
externalHooks: IExternalHooksClass;
defaultCredentialsName: string;
}
Loading