bug: privateKeyToAccount throws error in tests

[70nyIT]

Is there an existing issue for this?

• I have searched the existing issues

Package Version

1.10.14

Current Behavior

I'm creating a React plugin, to be later published on NPM, and I'm doing that with a test-driven approach. I'm encountering issues when using the function privateKeyToAccount(); inside my test, and when the function is called I get the following error

   expected Uint8Array, got object

      42 |     const viewingPrivateKey = generatePrivateKey();
      43 |     console.log(viewingPrivateKey);
    > 44 |     const viewingAccount = privateKeyToAccount(viewingPrivateKey);
         |                                               ^
      45 |     const viewingPublicKey = viewingAccount.publicKey;
      46 |     const spendingPrivateKey = generatePrivateKey();
      47 |     const spendingAccount = privateKeyToAccount(spendingPrivateKey);

      at toBytes (node_modules/@noble/hashes/src/utils.ts:105:25)
      at hashC (node_modules/@noble/hashes/src/utils.ts:177:63)
      at keccak256 (node_modules/viem/utils/hash/keccak256.ts:19:27)
      at checksumAddress (node_modules/viem/utils/address/getAddress.ts:13:25)
      at publicKeyToAddress (node_modules/viem/accounts/utils/publicKeyToAddress.ts:16:25)
      at privateKeyToAccount (node_modules/viem/accounts/privateKeyToAccount.ts:20:37)
      at Object.<anonymous> (test/core/class/Client.test.ts:44:47)

Expected Behavior

To ingest correctly the private key and return an object of type Account.

Steps To Reproduce

Create a projen application with the following .projenrc.ts

import { typescript } from 'projen';
import { NpmAccess, TypeScriptJsxMode } from 'projen/lib/javascript';

const project = new typescript.TypeScriptProject({
  defaultReleaseBranch: 'main',
  name: 'sdk',
  projenrcTs: true,
  releaseToNpm: true,

  // Npm publishing
  npmAccess: NpmAccess.PUBLIC,
  packageName: '@client/sdk',
  peerDeps: [
    'react@^16.8.0 || ^17.0.0 || ^18.0.0',
    'react-dom@^16.8.0 || ^17.0.0 || ^18.0.0',
  ],

  // Github repo
  gitignore: [
    '.idea',
  ],

  /* Dependencies */
  devDeps: [
    'jest-environment-jsdom@29.7.0',
    'react@^16.8.0',
    'react-dom@^16.8.0',
    '@types/react@^16.8.0',
    '@types/react-dom@^16.8.0',
    '@testing-library/jest-dom@^6.1.3',
    '@testing-library/react@^12.1.5',
    '@types/jest@^29.5.5',
  ],
  deps: [
    'viem@^1.10.14',
  ],

  /* Extending tsconfig to accept .tsx files and JSX */
  tsconfig: {
    compilerOptions: {
      jsx: TypeScriptJsxMode.REACT,
    },
    include: ['src/**/*.tsx'],
  },
  tsconfigDev: {
    compilerOptions: {
      jsx: TypeScriptJsxMode.REACT,
      esModuleInterop: true,
    },
    // need to set also the `test` folder, otherwise errors are raised when releasing the package with the github CI/CD
    include: ['src/**/*.tsx', 'test/**/*.tsx'],
  },

  /* Jest configuration extension */
  jestOptions: {
    jestConfig: {
      setupFilesAfterEnv: ['<rootDir>/jest.setup.ts'],
      testEnvironment: 'jsdom',
      moduleNameMapper: {
        // point to the `react/jsx-runtime` bundled inside react NPM package without dealing with webpack or using create-react-app
        'react/jsx-runtime': '<rootDir>/node_modules/react',
        // same for `react-dom/client` as per `react/jsx-runtime`
        'react-dom/client': '<rootDir>/node_modules/react-dom',
      },
    },
  },
});

project.synth();

The jest.setup.ts file mentioned has the following structure

import { TextEncoder } from 'util';

global.TextEncoder = TextEncoder;

then edit also the default test provided by projen with the following content

import { generatePrivateKey, privateKeyToAccount } from 'viem/accounts';

describe('Manual Key setup', () => {
  it('Pass generated keys and expect to be corrected', async () => {
    const viewingPrivateKey = generatePrivateKey();
    console.log(viewingPrivateKey);
    const viewingAccount = privateKeyToAccount(viewingPrivateKey);
    const viewingPublicKey = viewingAccount.publicKey;
    const spendingPrivateKey = generatePrivateKey();
    const spendingAccount = privateKeyToAccount(spendingPrivateKey);
    const spendingPublicKey = spendingAccount.publicKey;
    // other parts of the test here - not related to bug
  });
});

Link to Minimal Reproducible Example (StackBlitz, CodeSandbox, GitHub repo etc.)

No response

Anything else?

No response

[tmm]

Mind packaging this up into a runnable example that reproduces the issue? Without this, it's less likely someone picks it up.

[jxom]

I'd assume this is something that needs tweaking in your test environment, and less so a bug in viem. Moving to discussion.

[70nyIT]

Thanks for the prompt answer. I was able to reproduce the bug here
https://replit.com/@70nyIT/SimulateViemBug?v=1

Just login and run npx projen test

[70nyIT]

first time using replit with projen, so if you need to be explicitly invited to run command line commands, just drop your replit username and 'll add you

[70nyIT]

I've debugged more today, and I can see the problem down to node_modules/@noble/hashes/src/utils.ts, specifically at the function export function toBytes(data: Input): Uint8Array

When that function calls if (!u8a(data)) throw new Error(expected Uint8Array, got ${typeof data});, the u8a function asserts that it is not a Uint8Array, even if with the debugger I can see it is a Uint8Array.

Specifically this error does NOT happen when calling the toBytes with a Uint8Array made of 64 or 46 elements, while instead he error is raise if the toBytes function is called with a Uint8Array made of 40 elements. I guess that the issue is not connected to the lenght of the array, but that arrays of 40s are generated by a parent function that converts hex strings to Uint8Array.

I suspect the issue starts in my jest.setup.js config file I had to add as it was not finding the TextEncoder . In that file I do

import { TextEncoder } from 'util';

// required by viem library to run properly - see import inside .projenrc.ts
global.TextEncoder = TextEncoder;

This is the encoder used inside node_modules/viem/utils/encoding/toBytes.ts, specifically in the stringToBytes function of that file. Since this file is part of your library, what do you suggest me to set an global encoder, so that tests can run?
My take is that encoder generates a different (while similar in content) Uint8Array, that throws the error.

[70nyIT]

I have good news, as I changed it to text-encoder and it's working. I know it's a library no more maintained by its author, so if you have any suggestions of different libraries I could use, I'd be happy to take.

This is how my jest.setup.js looks now

import { TextEncoder, TextDecoder } from 'text-encoding';

global.TextEncoder = TextEncoder;
global.TextDecoder = TextDecoder;

What I find stange is that I'm running jest with testEnvironment: 'jsdom', but there's no TextEncoder inside

[70nyIT]

As an extra reference, here's the point - jestjs/jest#13227

Looks like this is a jest issue, not easy to test. Just curious: did you have issue when testing stringToBytes in you CI/CD? How did you pass TextEncoder in the jsdom?

[bjfresh]

Ran into something similar and tracked back to test environment, specifically: vitest-dev/vitest#4043 (comment)

TextEncoder (and TextDecoder) is a Node API and Uint8Array is a jsdom API. The Uint8Array that both produce are not comparable, even while structurally ident (I guess some prototype stuff differs which breaks toEqual).

For me switching from jsdom to node in my vitest config fixed the issue.