Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor(core): Decouple server started event from internal hooks (no-changelog) #10221

Merged
merged 1 commit into from
Jul 29, 2024
Merged

refactor(core): Decouple server started event from internal hooks (no-changelog) #10221

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 0 additions & 78 deletions packages/cli/src/InternalHooks.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
import { Service } from 'typedi';
import { snakeCase } from 'change-case';
import os from 'node:os';
import { get as pslGet } from 'psl';
import { GlobalConfig } from '@n8n/config';
import type {
ExecutionStatus,
INodesGraphResult,
Expand All @@ -11,27 +9,23 @@ import type {
IWorkflowBase,
} from 'n8n-workflow';
import { TelemetryHelpers } from 'n8n-workflow';
import { InstanceSettings } from 'n8n-core';

import config from '@/config';
import { N8N_VERSION } from '@/constants';
import type { AuthProviderType } from '@db/entities/AuthIdentity';
import type { User } from '@db/entities/User';
import { SharedWorkflowRepository } from '@db/repositories/sharedWorkflow.repository';
import { WorkflowRepository } from '@db/repositories/workflow.repository';
import { determineFinalExecutionStatus } from '@/executionLifecycleHooks/shared/sharedHookFunctions';
import type {
ITelemetryUserDeletionData,
IWorkflowDb,
IExecutionTrackProperties,
} from '@/Interfaces';
import { License } from '@/License';
import { WorkflowStatisticsService } from '@/services/workflow-statistics.service';
import { NodeTypes } from '@/NodeTypes';
import { Telemetry } from '@/telemetry';
import type { Project } from '@db/entities/Project';
import { ProjectRelationRepository } from './databases/repositories/projectRelation.repository';
import { SharedCredentialsRepository } from './databases/repositories/sharedCredentials.repository';
import { MessageEventBus } from './eventbus/MessageEventBus/MessageEventBus';

/**
Expand All @@ -42,16 +36,11 @@ import { MessageEventBus } from './eventbus/MessageEventBus/MessageEventBus';
@Service()
export class InternalHooks {
constructor(
private readonly globalConfig: GlobalConfig,
private readonly telemetry: Telemetry,
private readonly nodeTypes: NodeTypes,
private readonly sharedWorkflowRepository: SharedWorkflowRepository,
private readonly workflowRepository: WorkflowRepository,
workflowStatisticsService: WorkflowStatisticsService,
private readonly instanceSettings: InstanceSettings,
private readonly license: License,
private readonly projectRelationRepository: ProjectRelationRepository,
private readonly sharedCredentialsRepository: SharedCredentialsRepository,
private readonly _eventBus: MessageEventBus, // needed until we decouple telemetry
) {
workflowStatisticsService.on(
Expand All @@ -68,73 +57,6 @@ export class InternalHooks {
await this.telemetry.init();
}

async onServerStarted(): Promise<unknown[]> {
const cpus = os.cpus();
const binaryDataConfig = config.getEnv('binaryDataManager');

const isS3Selected = config.getEnv('binaryDataManager.mode') === 's3';
const isS3Available = config.getEnv('binaryDataManager.availableModes').includes('s3');
const isS3Licensed = this.license.isBinaryDataS3Licensed();
const authenticationMethod = config.getEnv('userManagement.authenticationMethod');

const info = {
version_cli: N8N_VERSION,
db_type: this.globalConfig.database.type,
n8n_version_notifications_enabled: this.globalConfig.versionNotifications.enabled,
n8n_disable_production_main_process: config.getEnv(
'endpoints.disableProductionWebhooksOnMainProcess',
),
system_info: {
os: {
type: os.type(),
version: os.version(),
},
memory: os.totalmem() / 1024,
cpus: {
count: cpus.length,
model: cpus[0].model,
speed: cpus[0].speed,
},
},
execution_variables: {
executions_mode: config.getEnv('executions.mode'),
executions_timeout: config.getEnv('executions.timeout'),
executions_timeout_max: config.getEnv('executions.maxTimeout'),
executions_data_save_on_error: config.getEnv('executions.saveDataOnError'),
executions_data_save_on_success: config.getEnv('executions.saveDataOnSuccess'),
executions_data_save_on_progress: config.getEnv('executions.saveExecutionProgress'),
executions_data_save_manual_executions: config.getEnv(
'executions.saveDataManualExecutions',
),
executions_data_prune: config.getEnv('executions.pruneData'),
executions_data_max_age: config.getEnv('executions.pruneDataMaxAge'),
},
n8n_deployment_type: config.getEnv('deployment.type'),
n8n_binary_data_mode: binaryDataConfig.mode,
smtp_set_up: this.globalConfig.userManagement.emails.mode === 'smtp',
ldap_allowed: authenticationMethod === 'ldap',
saml_enabled: authenticationMethod === 'saml',
license_plan_name: this.license.getPlanName(),
license_tenant_id: config.getEnv('license.tenantId'),
binary_data_s3: isS3Available && isS3Selected && isS3Licensed,
multi_main_setup_enabled: config.getEnv('multiMainSetup.enabled'),
};

const firstWorkflow = await this.workflowRepository.findOne({
select: ['createdAt'],
order: { createdAt: 'ASC' },
where: {},
});

return await Promise.all([
this.telemetry.identify(info),
this.telemetry.track('Instance started', {
...info,
earliest_workflow_created: firstWorkflow?.createdAt,
}),
]);
}

async onFrontendSettingsAPI(pushRef?: string): Promise<void> {
return await this.telemetry.track('Session started', { session_id: pushRef });
}
Expand Down
5 changes: 3 additions & 2 deletions packages/cli/src/Server.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ import { isLdapEnabled } from '@/Ldap/helpers.ee';
import { AbstractServer } from '@/AbstractServer';
import { PostHogClient } from '@/posthog';
import { MessageEventBus } from '@/eventbus/MessageEventBus/MessageEventBus';
import { InternalHooks } from '@/InternalHooks';
import { handleMfaDisable, isMfaFeatureEnabled } from '@/Mfa/helpers';
import type { FrontendService } from '@/services/frontend.service';
import { OrchestrationService } from '@/services/orchestration.service';
Expand Down Expand Up @@ -66,6 +65,7 @@ import '@/ExternalSecrets/ExternalSecrets.controller.ee';
import '@/license/license.controller';
import '@/workflows/workflowHistory/workflowHistory.controller.ee';
import '@/workflows/workflows.controller';
import { EventService } from './eventbus/event.service';

const exec = promisify(callbackExec);

Expand All @@ -82,6 +82,7 @@ export class Server extends AbstractServer {
private readonly orchestrationService: OrchestrationService,
private readonly postHogClient: PostHogClient,
private readonly globalConfig: GlobalConfig,
private readonly eventService: EventService,
) {
super('main');

Expand All @@ -106,7 +107,7 @@ export class Server extends AbstractServer {
void this.loadNodesAndCredentials.setupHotReload();
}

void Container.get(InternalHooks).onServerStarted();
this.eventService.emit('server-started');
}

private async registerAdditionalControllers() {
Expand Down
2 changes: 2 additions & 0 deletions packages/cli/src/eventbus/event.types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ export type UserLike = {
* Events sent by `EventService` and forwarded by relays, e.g. `AuditEventRelay` and `TelemetryEventRelay`.
*/
export type Event = {
'server-started': {};

'workflow-created': {
user: UserLike;
workflow: IWorkflowBase;
Expand Down
77 changes: 77 additions & 0 deletions packages/cli/src/telemetry/telemetry-event-relay.service.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,20 @@ import { EventService } from '@/eventbus/event.service';
import type { Event } from '@/eventbus/event.types';
import { Telemetry } from '.';
import config from '@/config';
import os from 'node:os';
import { License } from '@/License';
import { GlobalConfig } from '@n8n/config';
import { N8N_VERSION } from '@/constants';
import { WorkflowRepository } from '@/databases/repositories/workflow.repository';

@Service()
export class TelemetryEventRelay {
constructor(
private readonly eventService: EventService,
private readonly telemetry: Telemetry,
private readonly license: License,
private readonly globalConfig: GlobalConfig,
private readonly workflowRepository: WorkflowRepository,
) {}

async init() {
Expand All @@ -20,6 +28,8 @@ export class TelemetryEventRelay {
}

private setupHandlers() {
this.eventService.on('server-started', async () => await this.serverStarted());

this.eventService.on('team-project-updated', (event) => this.teamProjectUpdated(event));
this.eventService.on('team-project-deleted', (event) => this.teamProjectDeleted(event));
this.eventService.on('team-project-created', (event) => this.teamProjectCreated(event));
Expand Down Expand Up @@ -420,4 +430,71 @@ export class TelemetryEventRelay {
private loginFailedDueToLdapDisabled({ userId }: Event['login-failed-due-to-ldap-disabled']) {
void this.telemetry.track('User login failed since ldap disabled', { user_ud: userId });
}

private async serverStarted() {
const cpus = os.cpus();
const binaryDataConfig = config.getEnv('binaryDataManager');

const isS3Selected = config.getEnv('binaryDataManager.mode') === 's3';
const isS3Available = config.getEnv('binaryDataManager.availableModes').includes('s3');
const isS3Licensed = this.license.isBinaryDataS3Licensed();
const authenticationMethod = config.getEnv('userManagement.authenticationMethod');

const info = {
version_cli: N8N_VERSION,
db_type: this.globalConfig.database.type,
n8n_version_notifications_enabled: this.globalConfig.versionNotifications.enabled,
n8n_disable_production_main_process: config.getEnv(
'endpoints.disableProductionWebhooksOnMainProcess',
),
system_info: {
os: {
type: os.type(),
version: os.version(),
},
memory: os.totalmem() / 1024,
cpus: {
count: cpus.length,
model: cpus[0].model,
speed: cpus[0].speed,
},
},
execution_variables: {
executions_mode: config.getEnv('executions.mode'),
executions_timeout: config.getEnv('executions.timeout'),
executions_timeout_max: config.getEnv('executions.maxTimeout'),
executions_data_save_on_error: config.getEnv('executions.saveDataOnError'),
executions_data_save_on_success: config.getEnv('executions.saveDataOnSuccess'),
executions_data_save_on_progress: config.getEnv('executions.saveExecutionProgress'),
executions_data_save_manual_executions: config.getEnv(
'executions.saveDataManualExecutions',
),
executions_data_prune: config.getEnv('executions.pruneData'),
executions_data_max_age: config.getEnv('executions.pruneDataMaxAge'),
},
n8n_deployment_type: config.getEnv('deployment.type'),
n8n_binary_data_mode: binaryDataConfig.mode,
smtp_set_up: this.globalConfig.userManagement.emails.mode === 'smtp',
ldap_allowed: authenticationMethod === 'ldap',
saml_enabled: authenticationMethod === 'saml',
license_plan_name: this.license.getPlanName(),
license_tenant_id: config.getEnv('license.tenantId'),
binary_data_s3: isS3Available && isS3Selected && isS3Licensed,
multi_main_setup_enabled: config.getEnv('multiMainSetup.enabled'),
};

const firstWorkflow = await this.workflowRepository.findOne({
select: ['createdAt'],
order: { createdAt: 'ASC' },
where: {},
});
Comment on lines +486 to +490
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially tried to pass this as an event arg from Server but importing the workflow repository there leads to ServiceNotFoundError: Service with "MaybeConstructable<DataSource>" identifier was not found in the container. Register it before usage via explicitly calling the "Container.set" function or using the "@Service()" decorator.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving this in Server meant that this query would have run even if telemetry was disabled. This file is the correct place for this query.


void Promise.all([
this.telemetry.identify(info),
this.telemetry.track('Instance started', {
...info,
earliest_workflow_created: firstWorkflow?.createdAt,
}),
]);
}
}
97 changes: 0 additions & 97 deletions packages/cli/test/unit/InternalHooks.test.ts
View file
Open in desktop
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking to bring this back later, else let me know if we should rewrite it now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's fine to do this later.

This file was deleted.

Loading