Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(qr): replace jsdom and d3 with cheerio #1286

Merged
merged 2 commits into from
Mar 11, 2021

Conversation

orbitalsqwib
Copy link
Contributor

Problem

Currently, the jsdom and d3 libraries have been imported to generate HTML for the QRCodeService module.
However, we only use a small subset of the functionalities contained within these libraries.

Solution

Replaced jsdom and d3 libraries with cheerio to reduce client app memory footprint.
Closes #1284.

New dependencies:

  • cheerio : Lightweight markup parser that can traverse and manipulate HTML.

New dev dependencies:

  • @types/cheerio : Supplies types for cheerio

@liangyuanruo
Copy link
Contributor

Do you have any metrics to back up this PR?

@orbitalsqwib
Copy link
Contributor Author

orbitalsqwib commented Mar 9, 2021

As of now i haven’t evaluated the metrics, but potentially they would be the application load speed and the qr code generation speed

@orbitalsqwib
Copy link
Contributor Author

orbitalsqwib commented Mar 10, 2021

Update: I realised I can't evaluate the application load speed as the files are stored locally on my device, however, I checked the bundle sizes for cheerio, jsdom and d3 on Bundlephobia and found that cheerio's bundle footprint is about 323.8kB minified, while jsdom and d3 are 2.9MB and 278.8kB minified respectively.

After gzipping, cheerio is at 88.5kB while jsdom and d3 are 726kB and 86.7kB respectively. Hence, there should theoretically be a 89% reduction in the bundle footprint for this code segment specifically.

@liangyuanruo
Copy link
Contributor

Hence, there should theoretically be an 89% reduction in the bundle footprint for this code segment specifically.

Actually, there is zero change in bundle size because the QR code service is on the server.

@LoneRifle LoneRifle changed the title refactor(QRCodeService): replaced jsdom and d3 libraries with cheerio refactor(QRCodeService): replace jsdom and d3 libraries with cheerio Mar 10, 2021
@LoneRifle
Copy link
Contributor

@liangyuanruo another reason to replace jsdom and d3 is that of dependency management - cheerio is lighter and has fewer dependencies than jsdom+d3 (7 vs 23 + 30), and despite having a limited API relative to d3, is still suitable enough for our usecase of generating SVGs. Given that we only use a tiny subset of the incumbents (ie, jsdom to create a DOM and d3.select to manipulate the DOM), it makes sense to have the libraries replaced, so that generally, we minimise the number of dependency updates that we may get but not need, particularly from d3.

@liangyuanruo
Copy link
Contributor

Still doesn't sound like a very strong reason to change battle-tested server-side code, but if you insist... I see that we have unit tests in qrcode.test.ts so I don't suppose this will break anything.

@liangyuanruo
Copy link
Contributor

liangyuanruo commented Mar 10, 2021

btw can help to remove the unnecessary typecast in the unit tests in qrcode.test.ts? I believve @yong-jie has also encountered potentially unsafe typecasts previously introduced in the codebase - we should strive to avoid these when we can.

const buffer = (await qrCodeService.createGoQrCode(
        testUrl,
        ImageFormat.PNG,
      )) as Buffer

@LoneRifle
Copy link
Contributor

btw can help to remove the unnecessary typecast in the unit tests in qrcode.test.ts?

const buffer = (await qrCodeService.createGoQrCode(
        testUrl,
        ImageFormat.PNG,
      )) as Buffer

@orbitalsqwib can get to that once he's subjected the codebase to k6 testing

@liangyuanruo
Copy link
Contributor

sure thing, thanks!

Copy link
Contributor

@LoneRifle LoneRifle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See this and the other request to remove the test

src/server/modules/qr/services/QrCodeService.ts Outdated Show resolved Hide resolved
@orbitalsqwib
Copy link
Contributor Author

QR code generation speed metric:

100 virtual users
100 iterations each

Before refactor:
Run 1: 1m28.4s
Run 2: 1m24.7s
Run 3: 1m25.0s
Avg runtime: 1m26s

After refactor:
Run 1: 1m33.0s
Run 2: 1m25.6s
Run 3: 1m22.2s
Avg runtime: 1m26.9s

Memory Heap usage metric:

100 virtual users
100 iterations each

Before refactor:
Total heap allocated: 48.6 MB

After refactor:
Total heap allocated: 33.8 MB

Noted that the profiler for before refactor ran 2 minutes longer than the after refactor due to the total runtime for before refactor being longer, thus the after refactor's heap allocation may be underestimated.

@orbitalsqwib orbitalsqwib force-pushed the refactor/qr-code-service branch from 7d9fc60 to 9d7e710 Compare March 10, 2021 07:28
@LoneRifle LoneRifle force-pushed the refactor/qr-code-service branch from 077d2e7 to 482683e Compare March 10, 2021 08:54
@LoneRifle LoneRifle force-pushed the refactor/qr-code-service branch from 482683e to 47954da Compare March 11, 2021 01:42
@LoneRifle LoneRifle changed the title refactor(QRCodeService): replace jsdom and d3 libraries with cheerio refactor(qr): replace jsdom and d3 with cheerio Mar 11, 2021
@LoneRifle LoneRifle merged commit 4dfbe44 into develop Mar 11, 2021
@LoneRifle LoneRifle deleted the refactor/qr-code-service branch March 11, 2021 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replace jsdom and d3 libraries with leaner libraries such as cheeriojs
3 participants