Thr 18 dev feature h5p editor (#4525)

* THR-1 h5p editor: temporary file storage (#4147) * Created H5P microservice * Add kubernetes files for h5p microservice * Added @lumieducation/h5p-server package * Don't use mock auth for testing * Fix api tests to use new TestApiClient * WIP: H5P editor: temp file storage * WIP: H5P editor: added data classes * resolve merge conflict leftovers * h5p editor: temp file storage implementation completed * h5p editor: temp file storage tests added * h5p editor: temp file storage moved * h5p editor: temp storage files moved * Update temporary-file-storage.ts Create temporary file folder --------- Co-authored-by: Marvin Rode <marvin.rode@capgemini.com> Co-authored-by: Marvin Rode (Cap) <127723478+marode-cap@users.noreply.github.com> Co-authored-by: Andre Blome <andre-david.blome@capgemini.com> Co-authored-by: Stephan Krause <101647440+SteKrause@users.noreply.github.com> * THR-2 local content storage (#4140) * Created H5P microservice * Add kubernetes files for h5p microservice * Added @lumieducation/h5p-server package * add ContentStorage with interface IContentStorage * add addContent() * implement addFile(), contentExists(), deleteContent() * implement deleteFile(), fileExists() * add getFileStats(), getFileStream() * add getMetadata(), getParameters(), getUsage() * add getUserPermissions(), listContent() * add listFiles(), sanitizeFilename() * refactor sanitizeFileneame * add user check for getMetadata and getParameters * fix existsOrCreateDir * fix deleteContent and fileExists * fix fileExists and contentExists, refactor methods * add unit tests for contentStorage * delete comments in addFile * fix addFile test and change descriptions * fix getUsage * fix getUsage test * delete TODO * add error message to contentExists + tests * add test for empty contentId at fileEsists * remove unused private sanitize method * adjust errorhandling + add tests for error cases * add tests and fix error cases * replace math.random * change regex in checkfilename * refactor tests and content storage implementation * change any values to unknown * fix create content id * refactor and fix exist boolean in createContentId * fix test can not createContentId * create folder if content will be created * Allow files in subdirectories --------- Co-authored-by: Marvin Rode <marvin.rode@capgemini.com> Co-authored-by: Marvin Rode (Cap) <127723478+marode-cap@users.noreply.github.com> * THR-3 local library storage (#4133) * Implemented library storage * Tests for librray storage * full test coverage * Moved files --------- Co-authored-by: Stephan Krause <101647440+SteKrause@users.noreply.github.com> * h5p editor: added deployment which uses existing template (#4136) Co-authored-by: Andre Blome <andre-david.blome@capgemini.com> Co-authored-by: Stephan Krause <101647440+SteKrause@users.noreply.github.com> * Thr 5 h5p implementation endpoints (#4169) * Created H5P microservice * Add kubernetes files for h5p microservice * Added @lumieducation/h5p-server package * Don't use mock auth for testing * add ContentStorage with interface IContentStorage * Fix api tests to use new TestApiClient * WIP: H5P editor: temp file storage * WIP: H5P editor: added data classes * add addContent() * implement addFile(), contentExists(), deleteContent() * implement deleteFile(), fileExists() * add getFileStats(), getFileStream() * add getMetadata(), getParameters(), getUsage() * add getUserPermissions(), listContent() * add listFiles(), sanitizeFilename() * refactor sanitizeFileneame * Implemented library storage * add user check for getMetadata and getParameters * fix existsOrCreateDir * h5p editor: added deployment which uses existing template * fix deleteContent and fileExists * Tests for librray storage * resolve merge conflict leftovers * fix fileExists and contentExists, refactor methods * add unit tests for contentStorage * delete comments in addFile * fix addFile test and change descriptions * fix getUsage * fix getUsage test * delete TODO * full test coverage * Moved files * Implemented library storage * Tests for librray storage * full test coverage * Moved files * add error message to contentExists + tests * add test for empty contentId at fileEsists * remove unused private sanitize method * adjust errorhandling + add tests for error cases * add tests and fix error cases * replace math.random * change regex in checkfilename * delete unused code * change regex in checkfilename * refactor tests and content storage implementation * change any values to unknown * fix create content id * Fix issues from merging main into THR-2 * refactor and fix exist boolean in createContentId * fix test can not createContentId * Stubbed endpoints * Editor and Player service * Added H5P Usecase * h5p editor: temp file storage implementation completed * h5p editor: temp file storage tests added * h5p editor: temp file storage moved * h5p editor: temp storage files moved * File streaming * Post Ajax endpoint * Merge branch 'THR-8-h5p-api-endpoints' of github.com:hpi-schul-cloud/schulcloud-server into THR-5-h5p-implementation-endpoints * Remove @nestjs/serve-static dependency * Editor working more reliably * Disable saving of userstate * Organized code * More organization * Tests for getAjax usecase * add tests for h5p controller * Tests and organization * add import of FilesStorageAMQPModule to h5p-editor module * Switched over to storage implementations * Organized modules * change stringPath of createNewEditor * Testing for files in UC * Testing content parameters in UC * change AjaxPostBodyParams to undefined if empty * change save api endpoint * Fixed file streaming * Fixed h5p uploading * Update temporary-file-storage.ts Create temporary file folder * merge save and create endpoints to one get and post endpoint * create folder if content will be created * adjust paths at api tests * rename usecase to getH5pEditor * Use real user * Allow files in subdirectories * Testing for h5p files * API tests for internal AJAX endpoint * Test range requests for storages * add unit-tests to usecases * add api tests for h5p endpoints * add api test save or create * adjust create or save api test --------- Co-authored-by: Marvin Rode <marvin.rode@capgemini.com> Co-authored-by: Marvin Rode (Cap) <127723478+marode-cap@users.noreply.github.com> Co-authored-by: Andre Blome <andre-david.blome@capgemini.com> * THR-7 ContentStorage: store data on S3 via S3ClientAdapter (#4198) * Add "List" and "Head" Methods to S3ClientAdapter * H5P Content Storage now uses S3 Bucket and database * Added database repo for H5P metadata * Thr 20 create UI elements (#4190) * Necessary changes to support H5P on the client * h5p editor: library storage s3 implementation * h5p editor: library storage updated; unit tests; bugfixes * change to correct Buckets / Key names * add config keys * change default value for env var S3_REGION * Fix API Validation * THR-25 S3 implementation of temporary file storage (#4204) * Created H5P microservice * Add kubernetes files for h5p microservice * Added @lumieducation/h5p-server package * Don't use mock auth for testing * Fix api tests to use new TestApiClient * WIP: H5P editor: temp file storage * WIP: H5P editor: added data classes * resolve merge conflict leftovers * h5p editor: temp file storage implementation completed * h5p editor: temp file storage tests added * h5p editor: temp file storage moved * h5p editor: temp storage files moved * h5p editor: temp file storage changed to use s3 * h5p editor: s3 config and adapter injection handled * h5p editor: temp file storage fixes * h5p editor: temp file storage: repo method renamed to findByUserAndFilename() * change to correct Buckets names * change to correct Bucket name * initial commit * Merge remote-tracking branch 'origin/THR-18-dev-feature-h5p-editor' into THR-25-temp-storage-s3 * Update configs * Update module and fix all tests * Fix module config * Fix Library Entity from THR-18 * Updated Tests * Remove console output * Remove config from development.json * Remove unused config file * Organized all files for H5P module --------- Co-authored-by: Marvin Rode <marvin.rode@capgemini.com> Co-authored-by: Marvin Rode (Cap) <127723478+marode-cap@users.noreply.github.com> Co-authored-by: Andre Blome <andre-david.blome@capgemini.com> Co-authored-by: Majed Mak <132336669+MajedAlaitwniCap@users.noreply.github.com> Co-authored-by: Majed Aitwni <majed.alaitwni@capgemini.com> * Thr 27 h5p language parameter (#4283) * adjust controller and uc for language params * refactor h5p-editor service * use userService in getUserLanguage() * use env available languages * Refactor merging * resolve bugs in uc tests * change h5p-editor dto * delete await from editor + player declarations --------- Co-authored-by: Marvin Rode <marvin.rode@capgemini.com> Co-authored-by: Marvin Rode (Cap) <127723478+marode-cap@users.noreply.github.com> Co-authored-by: Andre Blome <andre-david.blome@capgemini.com> Co-authored-by: Majed Aitwni <majed.alaitwni@capgemini.com> * THR-42 fixes to library storage (#4332) * Library storage THR-34 hotfixes * Fix small bugs in LibraryStorageService * Update Tests * resolved PR comments * delete h5p-editor static files * Remove deployment from main * remove static server * add LibraryStorage to uc tests * expect LanguageType in editor uc test * add missing language param at api tests * add field name to response type IGetFileResponse * change library entity constructor * refactor library entity constructor * add list of files (#4349) * add list of files * change s3 command type * fix tests * move recursive part to private methode * fix limits --------- Co-authored-by: Marvin Rode (Cap) <127723478+marode-cap@users.noreply.github.com> * code Smells corrector * THR-6 H5P editor authorization (#4364) * Preparing entities for authorization * Add entities * save and create tests * get player tests * get editor tests * files uc tests * last tests for changed UC * better error message * Document custom transform pipe * Fix PR comments * BaseEntityWithTimestamps for entities * Address more PR comments * pr comments * pr comments * Better file handling * Missing file * fix changes from s3 module * h5p editor: change api and static files urls * refactor library tests and imports * refactor imports of contentStorage and temporary file storage * refactor h5p content tests * delete unused imports * refactor h5p api tests * refactor imports in module * delete blank line * restructure api tests h5p-save-create * h5p editor: temporarily dont map errors * refactor tests regarding authorization * fix save-create api tests * quick fixes service tests * fix test in contentStorage service * refactor api editor files tests * set default parameter rangeStart = 0 * minor code structure adjustments * refactor: only one return statement * fix commit dd41731: redefine type of returnValue * resolve type error for response stream * revert minor code changes * review comments controller * Revert "review comments controller" * Fix s3 client mocks * Fix tests * fix file storage service test * Fix integration tests * add empty line before return * refactor: await below ifelse block * refactor: set default for rangeStart * Merge remote-tracking branch 'origin/main' into THR-18-dev-feature-h5p-editor * Fix imports * Remove import from files-storage * do not overwrite input parameter * change Error Type to NotAcceptable * create own h5pfile dto * fix remaining review comments in temporary-file-storage.service * refactor h5peditor/ h5pplayer service to provider Hint: npm ci * add test for new Repo Method * fix unit test for Temporary-file-storage * fix test Module imports * fix review comments: return method and error types * create seperate h5p error mapper - unit tests still WIP * update unit tests for uc.getAjax & uc.postAjax * resolve ts expect error & Test corrector * resolve rest review comments h5p.uc.ts * remove unused import * move PostBodyParamsTransformPipe to extra file * rename TemporaryFile to BaseEntityWithTimestamp * add private method and use cause error * refactor from Service to H5PAjaxEndpointProvider * use explicit return type * use cause error types * initial commit * create library.repo.spec test setup and first test * create tests for library.repo * revert * revert zu main * last review comments * fix library.repo.spec code coverage * Add library.entity.spec * Add test for existsOne to increase CodeCoverage * add h5p-content mapper Test * Fix eslint * fix Test * fix saveFile mockImplementation * fix unnecessary type assertion * import Correction * add tests libraryStorage.service * remove uneccessary * Remove uneccessary import * correction and add test for h5p mapper * remove uneccessary imports * remove unused import * test structure * add tests for library.entity * WIP | create tests post.body.params.transform-pipe * add tests for post.body.params.transform-pipe * remove unused parameter * Create Test for H5PContentMetadata * refactor Method * define method return type * Add test for temporary-file-storage.service * remove unused import * added H5p File Response interface * create tests h5p-content.entity * add GetLibraryFile dto * rename entity file * Create own GetLibraryFile Dto for UC * Revert post.body.params.transform-pipe & logging * bugfix post.body.params.transform-pipe * add return undefined * quick fix: transformed can be undefined * adapt userMcok to updated ICurrentUser Interface * Restructure AjaxPostBodyParamsTransformPipe * quick fix nest-cli.json * revert wrong Changes * Comment out end-to-end tests in push * Update push.yml * Update push.yml: add legacy e2e test again --------- Co-authored-by: André Blome <43345275+ssmid@users.noreply.github.com> Co-authored-by: Marvin Rode <marvin.rode@capgemini.com> Co-authored-by: Marvin Rode (Cap) <127723478+marode-cap@users.noreply.github.com> Co-authored-by: Andre Blome <andre-david.blome@capgemini.com> Co-authored-by: Stephan Krause <101647440+SteKrause@users.noreply.github.com> Co-authored-by: Majed Mak <132336669+MajedAlaitwniCap@users.noreply.github.com> Co-authored-by: stekrause <stephan.krause.ext@capgemini.com> Co-authored-by: Majed Aitwni <majed.alaitwni@capgemini.com> Co-authored-by: Sergej Hoffmann <97111299+SevenWaysDP@users.noreply.github.com> Co-authored-by: SevenWaysDP <sergej.hoffmann@dataport.de>
hpi-schul-cloud · Nov 8, 2023 · 301324f · 301324f
1 parent aff2019
commit 301324f
Show file tree

Hide file tree

Showing 75 changed files with 8,951 additions and 171 deletions.
diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml
@@ -172,7 +172,7 @@ jobs:
         uses: github/codeql-action/upload-sarif@v2
         with:
           sarif_file: 'trivy-results.sarif'
-
+  
   end-to-end-tests:
     needs:
       - build_and_push

diff --git a/ansible/roles/schulcloud-server-h5p/tasks/main.yml b/ansible/roles/schulcloud-server-h5p/tasks/main.yml
@@ -1,4 +1,4 @@
-  - name: H5pEditorService
+  - name: H5PEditorProvider
     kubernetes.core.k8s:
       kubeconfig: ~/.kube/config
       namespace: "{{ NAMESPACE }}"

diff --git a/apps/server/src/apps/h5p-editor.app.ts b/apps/server/src/apps/h5p-editor.app.ts
@@ -19,6 +19,7 @@ async function bootstrap() {
 	const nestExpress = express();
 
 	const nestExpressAdapter = new ExpressAdapter(nestExpress);
+
 	const nestApp = await NestFactory.create(H5PEditorModule, nestExpressAdapter);
 	// WinstonLogger
 	nestApp.useLogger(await nestApp.resolve(LegacyLogger));

diff --git a/apps/server/src/modules/h5p-editor/controller/api-test/h5p-editor-ajax.api.spec.ts b/apps/server/src/modules/h5p-editor/controller/api-test/h5p-editor-ajax.api.spec.ts
@@ -0,0 +1,178 @@
+import { DeepMocked, createMock } from '@golevelup/ts-jest';
+import { H5PAjaxEndpoint } from '@lumieducation/h5p-server';
+import { EntityManager } from '@mikro-orm/core';
+import { HttpStatus, INestApplication } from '@nestjs/common';
+import { Test } from '@nestjs/testing';
+import { S3ClientAdapter } from '@shared/infra/s3-client';
+import { TestApiClient, UserAndAccountTestFactory } from '@shared/testing';
+import { H5PEditorTestModule } from '../../h5p-editor-test.module';
+import { H5P_CONTENT_S3_CONNECTION, H5P_LIBRARIES_S3_CONNECTION } from '../../h5p-editor.config';
+
+describe('H5PEditor Controller (api)', () => {
+	let app: INestApplication;
+	let em: EntityManager;
+	let testApiClient: TestApiClient;
+
+	let ajaxEndpoint: DeepMocked<H5PAjaxEndpoint>;
+
+	beforeAll(async () => {
+		const module = await Test.createTestingModule({
+			imports: [H5PEditorTestModule],
+		})
+			.overrideProvider(H5P_CONTENT_S3_CONNECTION)
+			.useValue(createMock<S3ClientAdapter>())
+			.overrideProvider(H5P_LIBRARIES_S3_CONNECTION)
+			.useValue(createMock<S3ClientAdapter>())
+			.overrideProvider(H5PAjaxEndpoint)
+			.useValue(createMock<H5PAjaxEndpoint>())
+			.compile();
+
+		app = module.createNestApplication();
+		await app.init();
+		em = app.get(EntityManager);
+		ajaxEndpoint = app.get(H5PAjaxEndpoint);
+		testApiClient = new TestApiClient(app, 'h5p-editor');
+	});
+
+	afterEach(() => {
+		jest.resetAllMocks();
+	});
+
+	afterAll(async () => {
+		await app.close();
+	});
+
+	describe('when calling AJAX GET', () => {
+		describe('when user not exists', () => {
+			it('should respond with unauthorized exception', async () => {
+				const response = await testApiClient.get('ajax');
+
+				expect(response.statusCode).toEqual(HttpStatus.UNAUTHORIZED);
+				expect(response.body).toEqual({
+					type: 'UNAUTHORIZED',
+					title: 'Unauthorized',
+					message: 'Unauthorized',
+					code: 401,
+				});
+			});
+		});
+
+		describe('when user is logged in', () => {
+			const createStudent = () => UserAndAccountTestFactory.buildStudent();
+
+			const setup = async () => {
+				const { studentAccount, studentUser } = createStudent();
+
+				await em.persistAndFlush([studentAccount, studentUser]);
+				em.clear();
+
+				const loggedInClient = await testApiClient.login(studentAccount);
+
+				return { loggedInClient, studentUser };
+			};
+
+			it('should call H5PAjaxEndpoint', async () => {
+				const {
+					loggedInClient,
+					studentUser: { id },
+				} = await setup();
+
+				const dummyResponse = {
+					apiVersion: { major: 1, minor: 1 },
+					details: [],
+					libraries: [],
+					outdated: false,
+					recentlyUsed: [],
+					user: 'DummyUser',
+				};
+
+				ajaxEndpoint.getAjax.mockResolvedValueOnce(dummyResponse);
+
+				const response = await loggedInClient.get(`ajax?action=content-type-cache`);
+
+				expect(response.statusCode).toEqual(HttpStatus.OK);
+				expect(response.body).toEqual(dummyResponse);
+				expect(ajaxEndpoint.getAjax).toHaveBeenCalledWith(
+					'content-type-cache',
+					undefined, // MachineName
+					undefined, // MajorVersion
+					undefined, // MinorVersion
+					'de', // Language
+					expect.objectContaining({ id })
+				);
+			});
+		});
+
+		describe('when calling AJAX POST', () => {
+			describe('when user not exists', () => {
+				it('should respond with unauthorized exception', async () => {
+					const response = await testApiClient.post('ajax');
+
+					expect(response.statusCode).toEqual(HttpStatus.UNAUTHORIZED);
+					expect(response.body).toEqual({
+						type: 'UNAUTHORIZED',
+						title: 'Unauthorized',
+						message: 'Unauthorized',
+						code: 401,
+					});
+				});
+			});
+
+			describe('when user is logged in', () => {
+				const createStudent = () => UserAndAccountTestFactory.buildStudent();
+
+				const setup = async () => {
+					const { studentAccount, studentUser } = createStudent();
+
+					await em.persistAndFlush([studentAccount, studentUser]);
+					em.clear();
+
+					const loggedInClient = await testApiClient.login(studentAccount);
+
+					return { loggedInClient, studentUser };
+				};
+
+				it('should call H5PAjaxEndpoint', async () => {
+					const {
+						loggedInClient,
+						studentUser: { id },
+					} = await setup();
+
+					const dummyResponse = [
+						{
+							majorVersion: 1,
+							minorVersion: 2,
+							metadataSettings: {},
+							name: 'Dummy Library',
+							restricted: false,
+							runnable: true,
+							title: 'Dummy Library',
+							tutorialUrl: '',
+							uberName: 'dummyLibrary-1.1',
+						},
+					];
+
+					const dummyBody = { contentId: 'id', field: 'field', libraries: ['dummyLibrary-1.0'], libraryParameters: '' };
+
+					ajaxEndpoint.postAjax.mockResolvedValueOnce(dummyResponse);
+
+					const response = await loggedInClient.post(`ajax?action=libraries`, dummyBody);
+
+					expect(response.statusCode).toEqual(HttpStatus.CREATED);
+					expect(response.body).toEqual(dummyResponse);
+					expect(ajaxEndpoint.postAjax).toHaveBeenCalledWith(
+						'libraries',
+						dummyBody,
+						'de',
+						expect.objectContaining({ id }),
+						undefined,
+						undefined,
+						undefined,
+						undefined,
+						undefined
+					);
+				});
+			});
+		});
+	});
+});
diff --git a/apps/server/src/modules/h5p-editor/controller/api-test/h5p-editor-delete.api.spec.ts b/apps/server/src/modules/h5p-editor/controller/api-test/h5p-editor-delete.api.spec.ts
@@ -0,0 +1,106 @@
+import { DeepMocked, createMock } from '@golevelup/ts-jest/lib/mocks';
+import { EntityManager, ObjectId } from '@mikro-orm/mongodb';
+import { ExecutionContext, INestApplication } from '@nestjs/common';
+import { Test } from '@nestjs/testing';
+import { Permission } from '@shared/domain';
+import { S3ClientAdapter } from '@shared/infra/s3-client';
+import { cleanupCollections, mapUserToCurrentUser, roleFactory, schoolFactory, userFactory } from '@shared/testing';
+import { ICurrentUser } from '@src/modules/authentication';
+import { JwtAuthGuard } from '@src/modules/authentication/guard/jwt-auth.guard';
+import { Request } from 'express';
+import request from 'supertest';
+import { H5PEditorTestModule } from '../../h5p-editor-test.module';
+import { H5P_CONTENT_S3_CONNECTION, H5P_LIBRARIES_S3_CONNECTION } from '../../h5p-editor.config';
+import { H5PEditorUc } from '../../uc/h5p.uc';
+
+class API {
+	constructor(private app: INestApplication) {
+		this.app = app;
+	}
+
+	async deleteH5pContent(contentId: string) {
+		return request(this.app.getHttpServer()).post(`/h5p-editor/delete/${contentId}`);
+	}
+}
+
+const setup = () => {
+	const contentId = new ObjectId(0).toString();
+	const notExistingContentId = new ObjectId(1).toString();
+	const badContentId = '';
+
+	return { contentId, notExistingContentId, badContentId };
+};
+
+describe('H5PEditor Controller (api)', () => {
+	let app: INestApplication;
+	let api: API;
+	let em: EntityManager;
+	let currentUser: ICurrentUser;
+	let h5PEditorUc: DeepMocked<H5PEditorUc>;
+
+	beforeAll(async () => {
+		const module = await Test.createTestingModule({
+			imports: [H5PEditorTestModule],
+		})
+			.overrideGuard(JwtAuthGuard)
+			.useValue({
+				canActivate(context: ExecutionContext) {
+					const req: Request = context.switchToHttp().getRequest();
+					req.user = currentUser;
+					return true;
+				},
+			})
+			.overrideProvider(H5P_CONTENT_S3_CONNECTION)
+			.useValue(createMock<S3ClientAdapter>())
+			.overrideProvider(H5P_LIBRARIES_S3_CONNECTION)
+			.useValue(createMock<S3ClientAdapter>())
+			.overrideProvider(H5PEditorUc)
+			.useValue(createMock<H5PEditorUc>())
+			.compile();
+
+		app = module.createNestApplication();
+		await app.init();
+		h5PEditorUc = module.get(H5PEditorUc);
+
+		api = new API(app);
+		em = module.get(EntityManager);
+	});
+
+	afterAll(async () => {
+		await app.close();
+	});
+
+	describe('delete h5p content', () => {
+		beforeEach(async () => {
+			await cleanupCollections(em);
+			const school = schoolFactory.build();
+			const roles = roleFactory.buildList(1, {
+				permissions: [Permission.FILESTORAGE_CREATE, Permission.FILESTORAGE_VIEW],
+			});
+			const user = userFactory.build({ school, roles });
+
+			await em.persistAndFlush([user, school]);
+			em.clear();
+
+			currentUser = mapUserToCurrentUser(user);
+		});
+		describe('with valid request params', () => {
+			it('should return 200 status', async () => {
+				const { contentId } = setup();
+
+				h5PEditorUc.deleteH5pContent.mockResolvedValueOnce(true);
+				const response = await api.deleteH5pContent(contentId);
+				expect(response.status).toEqual(201);
+			});
+		});
+		describe('with bad request params', () => {
+			it('should return 500 status', async () => {
+				const { notExistingContentId } = setup();
+
+				h5PEditorUc.deleteH5pContent.mockRejectedValueOnce(new Error('Could not delete H5P content'));
+				const response = await api.deleteH5pContent(notExistingContentId);
+				expect(response.status).toEqual(500);
+			});
+		});
+	});
+});