Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build(deps): bump express-fileupload from 1.2.1 to 1.4.0 #1890

Merged
merged 1 commit into from
Sep 20, 2022
Merged

build(deps): bump express-fileupload from 1.2.1 to 1.4.0 #1890

merged 1 commit into from
Sep 20, 2022

Conversation

halfwhole
Copy link
Collaborator

Problem

Dicer 0.3.0 is out of date, as seen on our Snyk dashboard

Link to Notion issue

Solution

Ran npm update express-fileupload to bump it to 1.4.0, which removes the dependency on dicer and resolves the issue

Tested file uploads and it works well on dev environment

@halfwhole halfwhole closed this Sep 20, 2022
@halfwhole halfwhole force-pushed the build/deps/upgrade-express-fileupload branch from 0b3de18 to 4eff871 Compare September 20, 2022 05:44
@halfwhole halfwhole reopened this Sep 20, 2022
@halfwhole halfwhole merged commit 81bbb6e into develop Sep 20, 2022
@halfwhole halfwhole deleted the build/deps/upgrade-express-fileupload branch September 20, 2022 07:48
gweiying pushed a commit that referenced this pull request Oct 6, 2022
@halfwhole @gweiying
build(deps): bump express-fileupload from 1.2.1 to 1.4.0 (#1890)
d536bf7
jimvae added a commit that referenced this pull request Oct 16, 2022
@jimvae @thanhdatle @halfwhole
feat(bulk-backend): Job and JobItem DB model, repository and service (#…
77c294a 
…2007)

* Feat/link audit to include tags (#1950)

* feat: added tags table and many to many rel to urls

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: merge link-audit-to-include-tags with develop

* feat: link-audit-to-include-tags

* feat: search for tags in lower cap (#1973)

* feat: search for tags in lower cap

* feat: search for tags in lower cap

* feat(tagging): add tags to link audit history (#1974)

* chore(datadog): refactor custom metric names (#1965)

* feat(tagging): add frontend tagging on create new link form (#1919)

* feat(tagging): add frontend tagging on create new link form

* chore: refactor autocomplete and address comments

* chore: refactor text field height and add disable submit checks

* chore: refactor max tag length in helper error message

* chore(tagging): refactor max number of tags per link

* chore(tagging): update tag validation tests for underscores

* feat(tagging): add dropdown for tags on user page search bar (#1934)

* feat(tagging): add dropdown for tags on user page search bar

* feat(tagging): add link icon on search dropdown

* feat(tagging): refactor link icon and add tag icon on search dropdown

* feat(tagging): revert link icon to original size

* build(deps): bump winston from 3.3.3 to 3.8.1 (#1889)

* build(deps): bump express-fileupload from 1.2.1 to 1.4.0 (#1890)

* feat(tagging): add frontend API integrations on create new link form (#1971)

* feat(tagging): add frontend tagging on create new link form

* chore: refactor autocomplete and address comments

* chore: refactor text field height and add disable submit checks

* chore: refactor max tag length in helper error message

* chore(tagging): refactor max number of tags per link

* chore(tagging): update tag validation tests for underscores

* feat(tagging): add frontend API integrations for create new link form

* chore(tagging): update tag validation tests for underscores

* chore(tagging): refactor tags to use redux instead of useState

* Feat/add tag to link audit frontend (#1975)

* feat: added TagList component by using mui/Chip to show tags in Link Audit

* feat: added TagList component by using mui/Chip to show tags in Link Audit

* feat: added TagList component by using mui/Chip to show tags in Link Audit

* feat(tagging): add editing of link tags (#1976)

* feat(tagging): add frontend tagging on create new link form

* chore: refactor autocomplete and address comments

* chore: refactor text field height and add disable submit checks

* chore: refactor max tag length in helper error message

* chore(tagging): refactor max number of tags per link

* chore(tagging): update tag validation tests for underscores

* feat(tagging): add frontend API integrations for create new link form

* chore(tagging): update tag validation tests for underscores

* chore(tagging): refactor tags to use redux instead of useState

* feat(tagging): add editing of link tags

* feat(tagging): use manual save button for editing link tags

* fix: return successful response when no URLs are found (#1979)

* fix(tagging): reset tags state after link creation (#1982)

* feat(tagging): add tags to links on user page (#1972)

* feat(tagging): add tags to links on user page

* feat(tagging): apply search when clicking on tags

* chore(tagging): remove duplicate identifier for tags

* fix: revise findUrlsForUser to return urls in correct order (#1981)

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: fix urlMapper to correctly check for empty tagStrings before parsing (#1989)

* chore: remove disallowed file extensions (#1985)

* Fix/link tagging for file upload (#1986)

* fix: add tags when creating new link from file

* chore: update sentry message

* fix: wrap backend errors in json messages (#1991)

* fix(tagging): serialize single tags for file uploads (#1990)

* fix(tagging): serialize single tags for file uploads

* chore(tagging): add comment for preprocessing file tags

* feat: add job and job item models

* feat: add jobmanagementservice

* chore: fix conflicts

* chore: rename JobStatusEnum to JobItemStatusEnum

* chore: remove JobItemStatusEnum - Ready

* chore: replaced isSuccess to getJobStatus in JobManagementService

Co-authored-by: thanhdatle <thanhdat.toby@gmail.com>
Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>
gweiying added a commit that referenced this pull request Dec 5, 2022
@gweiying @halfwhole
@jimvae @thanhdatle
Feat/bulk/all (#2020)
8870ac3 
* feat: bulk frontend modal (#1958)

* chore: create new CreateTypeButton for CreateLinkForm

* chore: swap isFile state boolean to createType enum

* chore: add CreateType BULK button

* feat: add BulkUpload form

* chore: fix button disabling check

* chore: add download sample csv functionality

* fix: bugs

* chore: add user actions for bulk create

* fix: misspelling and children nesting

* chore: remove isFile from LinkFormStyles

* chore: try to fix the css

* chore: refactor FileInputField component (#1960)

* chore: refactor FileInputField component

* fix: remove unused styles

* feat: connect bulk csv upload to backend

* fix: tag serialization and deserialization on FormData

* fix: renamed button

* chore: add e2e tests for bulk creation (#2022)

* chore: add e2e tests for bulk creation

* chore: clean up unused code

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: add small wait time to stop test flakiness

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* feat(bulk-backend): Job and JobItem DB model, repository and service (#2007)

* Feat/link audit to include tags (#1950)

* feat: added tags table and many to many rel to urls

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: merge link-audit-to-include-tags with develop

* feat: link-audit-to-include-tags

* feat: search for tags in lower cap (#1973)

* feat: search for tags in lower cap

* feat: search for tags in lower cap

* feat(tagging): add tags to link audit history (#1974)

* chore(datadog): refactor custom metric names (#1965)

* feat(tagging): add frontend tagging on create new link form (#1919)

* feat(tagging): add frontend tagging on create new link form

* chore: refactor autocomplete and address comments

* chore: refactor text field height and add disable submit checks

* chore: refactor max tag length in helper error message

* chore(tagging): refactor max number of tags per link

* chore(tagging): update tag validation tests for underscores

* feat(tagging): add dropdown for tags on user page search bar (#1934)

* feat(tagging): add dropdown for tags on user page search bar

* feat(tagging): add link icon on search dropdown

* feat(tagging): refactor link icon and add tag icon on search dropdown

* feat(tagging): revert link icon to original size

* build(deps): bump winston from 3.3.3 to 3.8.1 (#1889)

* build(deps): bump express-fileupload from 1.2.1 to 1.4.0 (#1890)

* feat(tagging): add frontend API integrations on create new link form (#1971)

* feat(tagging): add frontend tagging on create new link form

* chore: refactor autocomplete and address comments

* chore: refactor text field height and add disable submit checks

* chore: refactor max tag length in helper error message

* chore(tagging): refactor max number of tags per link

* chore(tagging): update tag validation tests for underscores

* feat(tagging): add frontend API integrations for create new link form

* chore(tagging): update tag validation tests for underscores

* chore(tagging): refactor tags to use redux instead of useState

* Feat/add tag to link audit frontend (#1975)

* feat: added TagList component by using mui/Chip to show tags in Link Audit

* feat: added TagList component by using mui/Chip to show tags in Link Audit

* feat: added TagList component by using mui/Chip to show tags in Link Audit

* feat(tagging): add editing of link tags (#1976)

* feat(tagging): add frontend tagging on create new link form

* chore: refactor autocomplete and address comments

* chore: refactor text field height and add disable submit checks

* chore: refactor max tag length in helper error message

* chore(tagging): refactor max number of tags per link

* chore(tagging): update tag validation tests for underscores

* feat(tagging): add frontend API integrations for create new link form

* chore(tagging): update tag validation tests for underscores

* chore(tagging): refactor tags to use redux instead of useState

* feat(tagging): add editing of link tags

* feat(tagging): use manual save button for editing link tags

* fix: return successful response when no URLs are found (#1979)

* fix(tagging): reset tags state after link creation (#1982)

* feat(tagging): add tags to links on user page (#1972)

* feat(tagging): add tags to links on user page

* feat(tagging): apply search when clicking on tags

* chore(tagging): remove duplicate identifier for tags

* fix: revise findUrlsForUser to return urls in correct order (#1981)

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: fix urlMapper to correctly check for empty tagStrings before parsing (#1989)

* chore: remove disallowed file extensions (#1985)

* Fix/link tagging for file upload (#1986)

* fix: add tags when creating new link from file

* chore: update sentry message

* fix: wrap backend errors in json messages (#1991)

* fix(tagging): serialize single tags for file uploads (#1990)

* fix(tagging): serialize single tags for file uploads

* chore(tagging): add comment for preprocessing file tags

* feat: add job and job item models

* feat: add jobmanagementservice

* chore: fix conflicts

* chore: rename JobStatusEnum to JobItemStatusEnum

* chore: remove JobItemStatusEnum - Ready

* chore: replaced isSuccess to getJobStatus in JobManagementService

Co-authored-by: thanhdatle <thanhdat.toby@gmail.com>
Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: add error message for invalid date string and csv header

* Feat/bulk/frontend/status bar (#2040)

* chore: add icons

* feat: add StatusBarComponent

* chore: fix alignment

* chore: add actions and reducers

* chore: fix JSX linting error with eslint

* chore: attach actions and reducers to StatusBar component

* chore: display QR code inprogress after bulk upload (#2041)

* fix: combine class styles

* chore: add docs inline for disabling eslint no-undef

* Feat/bulk/frontend/status bar button (#2047)

* chore: refactor existing DownloadButton to separate component from logic

* chore: add CloseButton as a reusable component

* chore: add DownloadBulkButton to StatusBar

* chore: styling

* chore: rename icon

* Feat/bulk/all lambda (#2033)

* feat: add lambda deployment and sample code

* feat: add qrcode and zip lambda code

* feat: add createCsv, uploadToS3 and sendsqsMessage logic

* feat: update handler logic

* feat: stream files to s3 bulk-qrcode-generation

* chore: add assets

* chore: add sns publish method

* chore: refactor code

* chore: temp remove unnecessary ci build commadns

* chore: fix qrcode url

* chore: import services instead of functions

* Revert "chore: temp remove unnecessary ci build commadns"

This reverts commit f4474a19e25a82aa0205fdc5bdb7c7ef8562824a.

* chore: set memorySize, ephemeralStorageSize, timeout on serverless

* fix: clean up dir on err

* chore: use archiver directory false

* chore: remove streamToS3 from exports

* chore: set domain in env variables

Co-authored-by: Alexis <gweiying@gmail.com>

* Feat/bulk/all cleanup unused (#2082)

* chore: remove JobItemType from JobItem

* chore: remove JobItemStatusEnum Ready

* chore: refactor Job module to group files and remove mappers

Co-authored-by: Jim Engay <e0425954@u.nus.edu>

* Feat/bulk/all create job (#2083)

* feat: add sqs client and service

* feat: create job and job items and sendSqsmessage in bulk controller

* chore: move qrCode job creation into JobController

* chore: generalize sqs message sending

* chore: add sqs timeout

* chore: fix missing region

* chore: create qrBatchSize and sqsRegion env variable

* chore: fix error logging and type

* chore: fix bulk tests

* chore: fix lambda params

* chore: fix env variable documentation

Co-authored-by: Jim Engay <e0425954@u.nus.edu>

* Feat/bulk/all update job (#2085)

* feat: add sqs client and service

* feat: create job and job items and sendSqsmessage in bulk controller

* chore: move qrCode job creation into JobController

* chore: generalize sqs message sending

* chore: add sqs timeout

* chore: fix missing region

* chore: create qrBatchSize and sqsRegion env variable

* chore: fix error logging and type

* chore: fix bulk tests

* chore: fix lambda params

* chore: add JobItemId to job_items model for easy update

* chore: refactor logic to update jobItemStatus

* feat: expose endpoint to update jobItem

* chore: use env var for lambda hash value

* chore: use http post request to send completion callback

* chore: fix validation

* chore: add tests

* chore: fix error catching for lambda

* chore: add status to job table

* chore: add logic to update job after job item callback

* chore: fix tests

* chore: fix tests

* fix: call next to pass control

* fix: typo in docs

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: refactor job status computation logic

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: fix lint errors after merging suggestions

* chore: attach only jobId to req body in updateJobItem

* chore: change enum from Failed to Failure

* chore: add unique constraint on jobItemId

Co-authored-by: Jim Engay <e0425954@u.nus.edu>
Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* Feat/bulk/all get job server (#2092)

* feat: add sqs client and service

* feat: create job and job items and sendSqsmessage in bulk controller

* chore: move qrCode job creation into JobController

* chore: generalize sqs message sending

* chore: add sqs timeout

* chore: fix missing region

* chore: create qrBatchSize and sqsRegion env variable

* chore: fix error logging and type

* chore: fix bulk tests

* chore: fix lambda params

* chore: add JobItemId to job_items model for easy update

* chore: refactor logic to update jobItemStatus

* feat: expose endpoint to update jobItem

* chore: use env var for lambda hash value

* chore: use http post request to send completion callback

* chore: fix validation

* chore: add tests

* chore: fix error catching for lambda

* chore: add status to job table

* chore: add logic to update job after job item callback

* chore: fix tests

* chore: fix tests

* fix: call next to pass control

* fix: typo in docs

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: refactor job status computation logic

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: fix lint errors after merging suggestions

* chore: attach only jobId to req body in updateJobItem

* chore: change enum from Failed to Failure

* fix: add env variable to feature flag job creation

* chore: add repository methods to retrieve jobs

* chore: add service methods for long polling and retrieving job information

* chore: add endpoints for users to retrieve their latest job and updated job status

* chore: add tests

* fix: add env variables to docker-compose

* chore: add env variable for bulk download bucket

* chore: fix tests

* chore: fix typos in test

* chore: fix tests

* chore: change job status endpoint to get

* chore: rename jobItemIds to jobItemUrls

* chore: update error statuses

* chore: fix lint errors

Co-authored-by: Jim Engay <e0425954@u.nus.edu>
Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* Feat/bulk/all get job client (#2093)

* feat: add sqs client and service

* feat: create job and job items and sendSqsmessage in bulk controller

* chore: move qrCode job creation into JobController

* chore: generalize sqs message sending

* chore: add sqs timeout

* chore: fix missing region

* chore: create qrBatchSize and sqsRegion env variable

* chore: fix error logging and type

* chore: fix bulk tests

* chore: fix lambda params

* chore: add JobItemId to job_items model for easy update

* chore: refactor logic to update jobItemStatus

* feat: expose endpoint to update jobItem

* chore: use env var for lambda hash value

* chore: use http post request to send completion callback

* chore: fix validation

* chore: add tests

* chore: fix error catching for lambda

* chore: add status to job table

* chore: add logic to update job after job item callback

* chore: fix tests

* chore: fix tests

* fix: call next to pass control

* fix: add env variable to feature flag job creation

* chore: add repository methods to retrieve jobs

* chore: add service methods for long polling and retrieving job information

* chore: add endpoints for users to retrieve their latest job and updated job status

* chore: add tests

* fix: add env variables to docker-compose

* chore: add env variable for bulk download bucket

* fix: env variable naming in docker-compose

* chore: fix tests

* chore: fix tests

* fix: status bar should appear even if no urls

* chore: pass download urls to Download QR code button

* chore: add callback variable to actions and reducers

* chore: remove redundant bulkQRCodesStarted action

* chore: add client-side logic to retrieve user job

* chore: add component logic for status bar

* chore: fix end to end tests

* fix: uncomment sqs.sendService function

* chore: update api call on client for job update

Co-authored-by: Jim Engay <e0425954@u.nus.edu>

* Feat/bulk/all email send (#2099)

* chore: make sender domain optional

* chore: add email sending for job completion

* chore: fix local mailDev logging for non-OTP emails

* chore: fix types

* chore: fix tests

* Fix/bulk/all errors (#2102)

* chore: add dogstatsd variables

* fix: add promise to papaparse and refactor bulk validation

* chore: fix bulk upload error styling and handling

* chore: fix tests

* chore: fix types

* chore: add enums to shared folder

* chore: add email sending logic into email service

* chore: fix copy (#2103)

* chore: fix copy

* chore: update copy

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* Feat/bulk/all api guard (#2104)

* chore: refactor guard to use api key access

* chore: remove unused env variables and files

* chore: add tests

* chore: remove admin default email

* chore: fix email copy (#2107)

* chore: get header from shared folder

* chore: create shared types for bulk download file names

* chore: remove duplicate REPLICA_URI

* chore: add unauthorized error message

Co-authored-by: Jim Engay <e0425954@u.nus.edu>
Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>
Co-authored-by: Jim Vincent Andes Engay <65295129+jimvae@users.noreply.github.com>
Co-authored-by: thanhdatle <thanhdat.toby@gmail.com>
gweiying added a commit that referenced this pull request Dec 6, 2022
@gweiying @halfwhole
@jimvae @thanhdatle
Feat/bulk/all (#2020)
f3944d2 
* feat: bulk frontend modal (#1958)

* chore: create new CreateTypeButton for CreateLinkForm

* chore: swap isFile state boolean to createType enum

* chore: add CreateType BULK button

* feat: add BulkUpload form

* chore: fix button disabling check

* chore: add download sample csv functionality

* fix: bugs

* chore: add user actions for bulk create

* fix: misspelling and children nesting

* chore: remove isFile from LinkFormStyles

* chore: try to fix the css

* chore: refactor FileInputField component (#1960)

* chore: refactor FileInputField component

* fix: remove unused styles

* feat: connect bulk csv upload to backend

* fix: tag serialization and deserialization on FormData

* fix: renamed button

* chore: add e2e tests for bulk creation (#2022)

* chore: add e2e tests for bulk creation

* chore: clean up unused code

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: add small wait time to stop test flakiness

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* feat(bulk-backend): Job and JobItem DB model, repository and service (#2007)

* Feat/link audit to include tags (#1950)

* feat: added tags table and many to many rel to urls

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create tag API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: create url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: update url API to include tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: search-link-with-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: link-audit-to-include-tags

* feat: merge link-audit-to-include-tags with develop

* feat: link-audit-to-include-tags

* feat: search for tags in lower cap (#1973)

* feat: search for tags in lower cap

* feat: search for tags in lower cap

* feat(tagging): add tags to link audit history (#1974)

* chore(datadog): refactor custom metric names (#1965)

* feat(tagging): add frontend tagging on create new link form (#1919)

* feat(tagging): add frontend tagging on create new link form

* chore: refactor autocomplete and address comments

* chore: refactor text field height and add disable submit checks

* chore: refactor max tag length in helper error message

* chore(tagging): refactor max number of tags per link

* chore(tagging): update tag validation tests for underscores

* feat(tagging): add dropdown for tags on user page search bar (#1934)

* feat(tagging): add dropdown for tags on user page search bar

* feat(tagging): add link icon on search dropdown

* feat(tagging): refactor link icon and add tag icon on search dropdown

* feat(tagging): revert link icon to original size

* build(deps): bump winston from 3.3.3 to 3.8.1 (#1889)

* build(deps): bump express-fileupload from 1.2.1 to 1.4.0 (#1890)

* feat(tagging): add frontend API integrations on create new link form (#1971)

* feat(tagging): add frontend tagging on create new link form

* chore: refactor autocomplete and address comments

* chore: refactor text field height and add disable submit checks

* chore: refactor max tag length in helper error message

* chore(tagging): refactor max number of tags per link

* chore(tagging): update tag validation tests for underscores

* feat(tagging): add frontend API integrations for create new link form

* chore(tagging): update tag validation tests for underscores

* chore(tagging): refactor tags to use redux instead of useState

* Feat/add tag to link audit frontend (#1975)

* feat: added TagList component by using mui/Chip to show tags in Link Audit

* feat: added TagList component by using mui/Chip to show tags in Link Audit

* feat: added TagList component by using mui/Chip to show tags in Link Audit

* feat(tagging): add editing of link tags (#1976)

* feat(tagging): add frontend tagging on create new link form

* chore: refactor autocomplete and address comments

* chore: refactor text field height and add disable submit checks

* chore: refactor max tag length in helper error message

* chore(tagging): refactor max number of tags per link

* chore(tagging): update tag validation tests for underscores

* feat(tagging): add frontend API integrations for create new link form

* chore(tagging): update tag validation tests for underscores

* chore(tagging): refactor tags to use redux instead of useState

* feat(tagging): add editing of link tags

* feat(tagging): use manual save button for editing link tags

* fix: return successful response when no URLs are found (#1979)

* fix(tagging): reset tags state after link creation (#1982)

* feat(tagging): add tags to links on user page (#1972)

* feat(tagging): add tags to links on user page

* feat(tagging): apply search when clicking on tags

* chore(tagging): remove duplicate identifier for tags

* fix: revise findUrlsForUser to return urls in correct order (#1981)

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: revise findUrlsForUser to return urls in correct order

* fix: fix urlMapper to correctly check for empty tagStrings before parsing (#1989)

* chore: remove disallowed file extensions (#1985)

* Fix/link tagging for file upload (#1986)

* fix: add tags when creating new link from file

* chore: update sentry message

* fix: wrap backend errors in json messages (#1991)

* fix(tagging): serialize single tags for file uploads (#1990)

* fix(tagging): serialize single tags for file uploads

* chore(tagging): add comment for preprocessing file tags

* feat: add job and job item models

* feat: add jobmanagementservice

* chore: fix conflicts

* chore: rename JobStatusEnum to JobItemStatusEnum

* chore: remove JobItemStatusEnum - Ready

* chore: replaced isSuccess to getJobStatus in JobManagementService

Co-authored-by: thanhdatle <thanhdat.toby@gmail.com>
Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: add error message for invalid date string and csv header

* Feat/bulk/frontend/status bar (#2040)

* chore: add icons

* feat: add StatusBarComponent

* chore: fix alignment

* chore: add actions and reducers

* chore: fix JSX linting error with eslint

* chore: attach actions and reducers to StatusBar component

* chore: display QR code inprogress after bulk upload (#2041)

* fix: combine class styles

* chore: add docs inline for disabling eslint no-undef

* Feat/bulk/frontend/status bar button (#2047)

* chore: refactor existing DownloadButton to separate component from logic

* chore: add CloseButton as a reusable component

* chore: add DownloadBulkButton to StatusBar

* chore: styling

* chore: rename icon

* Feat/bulk/all lambda (#2033)

* feat: add lambda deployment and sample code

* feat: add qrcode and zip lambda code

* feat: add createCsv, uploadToS3 and sendsqsMessage logic

* feat: update handler logic

* feat: stream files to s3 bulk-qrcode-generation

* chore: add assets

* chore: add sns publish method

* chore: refactor code

* chore: temp remove unnecessary ci build commadns

* chore: fix qrcode url

* chore: import services instead of functions

* Revert "chore: temp remove unnecessary ci build commadns"

This reverts commit f4474a19e25a82aa0205fdc5bdb7c7ef8562824a.

* chore: set memorySize, ephemeralStorageSize, timeout on serverless

* fix: clean up dir on err

* chore: use archiver directory false

* chore: remove streamToS3 from exports

* chore: set domain in env variables

Co-authored-by: Alexis <gweiying@gmail.com>

* Feat/bulk/all cleanup unused (#2082)

* chore: remove JobItemType from JobItem

* chore: remove JobItemStatusEnum Ready

* chore: refactor Job module to group files and remove mappers

Co-authored-by: Jim Engay <e0425954@u.nus.edu>

* Feat/bulk/all create job (#2083)

* feat: add sqs client and service

* feat: create job and job items and sendSqsmessage in bulk controller

* chore: move qrCode job creation into JobController

* chore: generalize sqs message sending

* chore: add sqs timeout

* chore: fix missing region

* chore: create qrBatchSize and sqsRegion env variable

* chore: fix error logging and type

* chore: fix bulk tests

* chore: fix lambda params

* chore: fix env variable documentation

Co-authored-by: Jim Engay <e0425954@u.nus.edu>

* Feat/bulk/all update job (#2085)

* feat: add sqs client and service

* feat: create job and job items and sendSqsmessage in bulk controller

* chore: move qrCode job creation into JobController

* chore: generalize sqs message sending

* chore: add sqs timeout

* chore: fix missing region

* chore: create qrBatchSize and sqsRegion env variable

* chore: fix error logging and type

* chore: fix bulk tests

* chore: fix lambda params

* chore: add JobItemId to job_items model for easy update

* chore: refactor logic to update jobItemStatus

* feat: expose endpoint to update jobItem

* chore: use env var for lambda hash value

* chore: use http post request to send completion callback

* chore: fix validation

* chore: add tests

* chore: fix error catching for lambda

* chore: add status to job table

* chore: add logic to update job after job item callback

* chore: fix tests

* chore: fix tests

* fix: call next to pass control

* fix: typo in docs

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: refactor job status computation logic

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: fix lint errors after merging suggestions

* chore: attach only jobId to req body in updateJobItem

* chore: change enum from Failed to Failure

* chore: add unique constraint on jobItemId

Co-authored-by: Jim Engay <e0425954@u.nus.edu>
Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* Feat/bulk/all get job server (#2092)

* feat: add sqs client and service

* feat: create job and job items and sendSqsmessage in bulk controller

* chore: move qrCode job creation into JobController

* chore: generalize sqs message sending

* chore: add sqs timeout

* chore: fix missing region

* chore: create qrBatchSize and sqsRegion env variable

* chore: fix error logging and type

* chore: fix bulk tests

* chore: fix lambda params

* chore: add JobItemId to job_items model for easy update

* chore: refactor logic to update jobItemStatus

* feat: expose endpoint to update jobItem

* chore: use env var for lambda hash value

* chore: use http post request to send completion callback

* chore: fix validation

* chore: add tests

* chore: fix error catching for lambda

* chore: add status to job table

* chore: add logic to update job after job item callback

* chore: fix tests

* chore: fix tests

* fix: call next to pass control

* fix: typo in docs

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: refactor job status computation logic

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* chore: fix lint errors after merging suggestions

* chore: attach only jobId to req body in updateJobItem

* chore: change enum from Failed to Failure

* fix: add env variable to feature flag job creation

* chore: add repository methods to retrieve jobs

* chore: add service methods for long polling and retrieving job information

* chore: add endpoints for users to retrieve their latest job and updated job status

* chore: add tests

* fix: add env variables to docker-compose

* chore: add env variable for bulk download bucket

* chore: fix tests

* chore: fix typos in test

* chore: fix tests

* chore: change job status endpoint to get

* chore: rename jobItemIds to jobItemUrls

* chore: update error statuses

* chore: fix lint errors

Co-authored-by: Jim Engay <e0425954@u.nus.edu>
Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* Feat/bulk/all get job client (#2093)

* feat: add sqs client and service

* feat: create job and job items and sendSqsmessage in bulk controller

* chore: move qrCode job creation into JobController

* chore: generalize sqs message sending

* chore: add sqs timeout

* chore: fix missing region

* chore: create qrBatchSize and sqsRegion env variable

* chore: fix error logging and type

* chore: fix bulk tests

* chore: fix lambda params

* chore: add JobItemId to job_items model for easy update

* chore: refactor logic to update jobItemStatus

* feat: expose endpoint to update jobItem

* chore: use env var for lambda hash value

* chore: use http post request to send completion callback

* chore: fix validation

* chore: add tests

* chore: fix error catching for lambda

* chore: add status to job table

* chore: add logic to update job after job item callback

* chore: fix tests

* chore: fix tests

* fix: call next to pass control

* fix: add env variable to feature flag job creation

* chore: add repository methods to retrieve jobs

* chore: add service methods for long polling and retrieving job information

* chore: add endpoints for users to retrieve their latest job and updated job status

* chore: add tests

* fix: add env variables to docker-compose

* chore: add env variable for bulk download bucket

* fix: env variable naming in docker-compose

* chore: fix tests

* chore: fix tests

* fix: status bar should appear even if no urls

* chore: pass download urls to Download QR code button

* chore: add callback variable to actions and reducers

* chore: remove redundant bulkQRCodesStarted action

* chore: add client-side logic to retrieve user job

* chore: add component logic for status bar

* chore: fix end to end tests

* fix: uncomment sqs.sendService function

* chore: update api call on client for job update

Co-authored-by: Jim Engay <e0425954@u.nus.edu>

* Feat/bulk/all email send (#2099)

* chore: make sender domain optional

* chore: add email sending for job completion

* chore: fix local mailDev logging for non-OTP emails

* chore: fix types

* chore: fix tests

* Fix/bulk/all errors (#2102)

* chore: add dogstatsd variables

* fix: add promise to papaparse and refactor bulk validation

* chore: fix bulk upload error styling and handling

* chore: fix tests

* chore: fix types

* chore: add enums to shared folder

* chore: add email sending logic into email service

* chore: fix copy (#2103)

* chore: fix copy

* chore: update copy

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>

* Feat/bulk/all api guard (#2104)

* chore: refactor guard to use api key access

* chore: remove unused env variables and files

* chore: add tests

* chore: remove admin default email

* chore: fix email copy (#2107)

* chore: get header from shared folder

* chore: create shared types for bulk download file names

* chore: remove duplicate REPLICA_URI

* chore: add unauthorized error message

Co-authored-by: Jim Engay <e0425954@u.nus.edu>
Co-authored-by: halfwhole <41856541+halfwhole@users.noreply.github.com>
Co-authored-by: Jim Vincent Andes Engay <65295129+jimvae@users.noreply.github.com>
Co-authored-by: thanhdatle <thanhdat.toby@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thanhdatle thanhdatle thanhdatle approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@halfwhole @thanhdatle