Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

node-compat: npm:oracledb(thin mode) #20924

Closed
chibat opened this issue Oct 17, 2023 · 25 comments · Fixed by #20925 or #25598
Closed

node-compat: npm:oracledb(thin mode) #20924

chibat opened this issue Oct 17, 2023 · 25 comments · Fixed by #20925 or #25598
Labels
bug Something isn't working correctly node API Related to various "node:*" modules APIs node compat

Comments

@chibat
Copy link
Contributor

chibat commented Oct 17, 2023

Node compat doesn’t work for npm:oracledb(thin mode).
The steps to reproduce are summarized below.

test repository: https://github.com/chibat/oracledb-test

$ git clone https://github.com/chibat/oracledb-test.git
$ cd oracledb-test

start oracle database

$ docker run --name test -p 1521:1521 -e ORACLE_RANDOM_PASSWORD="y" -v ${PWD}/init_scripts:/container-entrypoint-initdb.d gvenzl/oracle-xe

main.js

// @deno-types="npm:@types/oracledb"
import oracledb from "npm:oracledb@6.2.0";

const conn = await oracledb.getConnection({connectString: "localhost:1521/XEPDB1", user: "test", password: "test"});
const result = await conn.execute("select table_name from tabs");
console.log(result);
conn.close();

Deno

$ cd deno
$ deno --version
deno 1.37.2 (release, x86_64-unknown-linux-gnu)
v8 11.8.172.13
typescript 5.2.2
$ deno task start
Task start deno run --allow-sys --allow-env --allow-net main.ts
error: Uncaught (in promise) TypeError: The "string" argument must be of type string or an instance of Buffer or ArrayBuffer. Received undefined
    at Function.byteLength (ext:deno_node/internal/buffer.mjs:359:11)
    at WritePacket.writeKeyValue (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/packet.js:514:32)
    at AuthMessage.encode (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/messages/auth.js:186:11)
    at Protocol._encodeMessage (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/protocol.js:108:20)
    at Protocol._processMessage (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/protocol.js:151:18)
    at ThinConnectionImpl.connect (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/connection.js:687:30)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async Object.getConnection (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/oracledb.js:642:3)
    at async file:///home/chiba/oracledb-test/deno/main.ts:4:14

Node

$ cd node
$ npm install
$ node index.mjs
{
  metaData: [
    {
      name: 'TABLE_NAME',
      dbType: [DbType DB_TYPE_VARCHAR],
      nullable: false,
      byteSize: 128,
      dbTypeName: 'VARCHAR2',
      fetchType: [DbType DB_TYPE_VARCHAR]
    }
  ],
  rows: [
    [ 'REGIONS' ],
    [ 'COUNTRIES' ],
    [ 'CITIES' ],
    [ 'CURRENCIES' ],
    [ 'CURRENCIES_COUNTRIES' ]
  ]
}

Bun

$ bun index.mjs
{
  metaData: [
    {
      name: "TABLE_NAME",
      dbType: [DbType DB_TYPE_VARCHAR],
      nullable: false,
      byteSize: 128,
      dbTypeName: "VARCHAR2",
      fetchType: [DbType DB_TYPE_VARCHAR]
    }
  ],
  rows: [
    [ "REGIONS" ], [ "COUNTRIES" ], [ "CITIES" ], [ "CURRENCIES" ], [ "CURRENCIES_COUNTRIES" ]
  ]
}

related issue: oracle/node-oracledb#1258

@chibat
Copy link
Contributor Author

chibat commented Oct 17, 2023

I have debugged it and found that process.argv0 is handled differently.

$ node
> process.argv0
'node'

$ deno
> import process from "node:process"
undefined
> process.argv0
undefined

$ bash -c 'exec -a customArgv0 node'
> process.argv0
'customArgv0'

$ bash -c 'exec -a customArgv0 deno'
> import process from "node:process"
undefined
> process.argv0
undefined

@ry ry added bug Something isn't working correctly node compat labels Oct 17, 2023
@chibat
Copy link
Contributor Author

chibat commented Oct 17, 2023

Perhaps support for aes-256-cbc, aes-192-cbc will also be required.

// TODO(kt3k): add more algorithms Aes192Cbc, Aes256Cbc, etc.

I bypassed the process.argv0 issue and tested it.

error: Uncaught (in promise) TypeError: Unknown cipher
    at new Decipheriv (ext:deno_node/internal/crypto/cipher.ts:140:13)
    at Object.createDecipheriv (node:crypto:28:10)
    at EncryptDecrypt._decrypt (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/encryptDecrypt.js:45:29)
    at EncryptDecrypt.updateVerifierData (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/encryptDecrypt.js:142:34)
    at AuthMessage.encode (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/messages/auth.js:211:14)
    at Protocol._encodeMessage (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/protocol.js:108:20)
    at Protocol._processMessage (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/protocol.js:151:18)
    at ThinConnectionImpl.connect (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/connection.js:700:30)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async Object.getConnection (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/oracledb.js:642:3)

@chibat
Copy link
Contributor Author

chibat commented Oct 18, 2023

@bartlomieju @kt3k
Thank you for fixing.
I did a test with a canary release.

$ deno task start
Task start deno run --allow-sys --allow-env --allow-net --reload main.ts
error: Uncaught (in promise) TypeError: Unknown cipher
    at new Decipheriv (ext:deno_node/internal/crypto/cipher.ts:140:13)
    at Object.createDecipheriv (node:crypto:28:10)
    at EncryptDecrypt._decrypt (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/encryptDecrypt.js:45:29)
    at EncryptDecrypt.updateVerifierData (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/encryptDecrypt.js:141:34)
    at AuthMessage.encode (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/messages/auth.js:211:14)
    at Protocol._encodeMessage (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/protocol.js:108:20)
    at Protocol._processMessage (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/protocol/protocol.js:151:18)
    at ThinConnectionImpl.connect (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/thin/connection.js:700:30)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async Object.getConnection (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.2.0/lib/oracledb.js:642:3)

Perhaps support for aes-256-cbc, aes-192-cbc will also be required.

// TODO(kt3k): add more algorithms Aes192Cbc, Aes256Cbc, etc.

Also, regarding the fix of process.argv0.
The result when explicitly specifying the process name is different from Node.js.

Node

$ bash -c 'exec -a customArgv0 node'
> process.argv0
'customArgv0'

Deno

$ deno --version
deno 1.37.2+5095af7 (canary, x86_64-unknown-linux-gnu)
v8 11.8.172.13
typescript 5.2.2
$ bash -c 'exec -a customArgv0 deno'
> import process from "node:process"
undefined
> process.argv0
"deno"

@bartlomieju
Copy link
Member

@chibat can you please open a separate issue for this problem?

@chibat
Copy link
Contributor Author

chibat commented Oct 18, 2023

@bartlomieju
I have created the Issue.
#20934

@chibat
Copy link
Contributor Author

chibat commented Oct 20, 2023

I have debugged this issue.
I understood that it depends on the following Issue.
#18455

  • crypto.Decipheriv.prototype.setAutoPadding

#21804

@chibat chibat changed the title node-compat: oracledb node-compat: npm:oracledb Oct 21, 2023
@baqsoft
Copy link

baqsoft commented Nov 27, 2023

@chibat As a workaround for missing "aes-256-cbc" it is possible to override functions "createCipheriv" and "createDecipheriv" with ones from https://github.com/browserify/browserify-aes
Just create a patched module:

import browserify from 'https://esm.sh/browserify-aes';
import crypto from 'node:crypto';

crypto.createDecipheriv = browserify.createDecipheriv;
crypto.createCipheriv = browserify.createCipheriv;

export default crypto;

and configure Deno's importMap to use it specifically for oracledb's scope:

{
  ....
  "scopes": {
      "https://esm.sh/v135/oracledb@6.2.0/": {
        "node:crypto": "./polyfills/crypto.ts"
       }
    }
}

@kt3k
Copy link
Member

kt3k commented Nov 27, 2023

Let's reopen as this doesn't look resolved

@kt3k kt3k reopened this Nov 27, 2023
@r-dahlstedt-linux-guy
Copy link

There is something that is going on when you hit release 1.39.0 where after connecting to the database when a query is issued it just hangs. All other versions from 1.37.x and 1.38.x all work perfectly fine with the exact same script. The actual Node.js package did just change to a new release (6.3.0) on December 21st but even when using the previous release (6.2.0) from October 11th which again worked until 1.39.0 the same hanging issue occurs. When running the bulk of the same script, the only difference being is the way the package is imported, Node.js has no similar issue with either version of the package up through release 21.5.0.

@r-dahlstedt-linux-guy
Copy link

I just tried again using 1.39.4 and its still just hangs. As soon as I switch back to 1.38.5 it works. Something substantially had to change when import npm modules such that when executing methods in the module, perhaps through those that are called through await (i.e. getConnection which is where mine hangs) they seemingly go into never never land.

@bartlomieju
Copy link
Member

@r-dahlstedt-linux-guy are you using a TLS connection?

@r-dahlstedt-linux-guy
Copy link

No

@chibat
Copy link
Contributor Author

chibat commented Feb 1, 2024

$ deno --version
deno 1.40.3 (release, x86_64-unknown-linux-gnu)
v8 12.1.285.6
typescript 5.3.3
$ deno task start
Task start deno run --allow-sys --allow-env --allow-net main.js
error: Uncaught (in promise) Error: Not implemented: crypto.Decipheriv.prototype.setAutoPadding
    at notImplemented (ext:deno_node/_utils.ts:9:9)
    at Decipheriv.setAutoPadding (ext:deno_node/internal/crypto/cipher.ts:163:5)
    at EncryptDecrypt._decrypt (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.3.0/lib/thin/protocol/encryptDecrypt.js:46:14)
    at EncryptDecrypt.updateVerifierData (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.3.0/lib/thin/protocol/encryptDecrypt.js:141:34)
    at AuthMessage.encode (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.3.0/lib/thin/protocol/messages/auth.js:210:14)
    at Protocol._encodeMessage (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.3.0/lib/thin/protocol/protocol.js:108:20)
    at Protocol._processMessage (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.3.0/lib/thin/protocol/protocol.js:151:18)
    at ThinConnectionImpl.connect (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.3.0/lib/thin/connection.js:703:30)
    at eventLoopTick (ext:core/01_core.js:64:7)
    at async Object.getConnection (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.3.0/lib/oracledb.js:653:3)

We moved forward.

@bartlomieju bartlomieju added the node API Related to various "node:*" modules APIs label Mar 4, 2024
@chibat chibat changed the title node-compat: npm:oracledb node-compat: npm:oracledb(thin mode) Jul 14, 2024
@salva-dev
Copy link

Today I get the same error as @chibat .
Does anyone know when this issue is expected to be resolved?
Is there any workarround while it is being resolved? I love deno and I am trying to test it in my company but we use oracle as database for almost everything.

Thank you so much for making deno, it's great, and adding oracle support would be a great improvement.

@chibat
Copy link
Contributor Author

chibat commented Jul 22, 2024

@salva-dev
I am hoping that once the following issue, PR, is resolved, this issue will also be resolved.
#21804, #22228

I confirm that the following comment workaround works well.
#20924 (comment)

I believe thick mode, which uses a native Oracle client, will also work.
https://node-oracledb.readthedocs.io/en/latest/user_guide/initialization.html#enabling-node-oracledb-thick-mode
But we would be happier if it worked in thin mode.

@salva-dev
Copy link

salva-dev commented Jul 24, 2024

@chibat , thanks a lot. The workaround works for me!!.
I look forward to the integration of the PR with enthusiasm.
Deno is great and I think it is present and future of js in server. I am testing it in my company where they let me, for cli's it is a marvel and now with the oracle support I will use it to do many backend services.
Thanks again.

littledivy added a commit that referenced this issue Aug 8, 2024
Co-Authored-By: Luca Casonato <hello@lcas.dev>

Fixes #21804
Ref #20924

---------

Signed-off-by: Divy Srivastava <dj.srivastava23@gmail.com>
Co-authored-by: Luca Casonato <hello@lcas.dev>
@chibat
Copy link
Contributor Author

chibat commented Aug 17, 2024

I have confirmed canary release.

$ deno --version
deno 1.46.0-rc.0+b6cdb31 (canary, release, x86_64-unknown-linux-gnu)
v8 12.8.374.6-rusty
typescript 5.5.2
$ date -u
Sat Aug 17 02:45:57 UTC 2024

$ deno task start
Task start deno task thin
Task thin deno run --allow-sys --allow-env --allow-net main.js
error: Uncaught (in promise) Error: ORA-01017: invalid username/password; logon denied
Help: https://docs.oracle.com/error-help/db/ora-01017/
    at Protocol._processMessage (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.6.0/lib/thin/protocol/protocol.js:181:17)
    at eventLoopTick (ext:core/01_core.js:174:7)
    at async ThinConnectionImpl.connect (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.6.0/lib/thin/connection.js:805:9)
    at async Object.getConnection (file:///home/chiba/.cache/deno/npm/registry.npmjs.org/oracledb/6.6.0/lib/oracledb.js:651:3)
    at async file:///home/chiba/data/git/github.com/chibat/oracledb-test/deno/main.js:8:14

There seems to be a difference in size when decrypted.

https://github.com/oracle/node-oracledb/blob/77c4610899fb2de92db1aeb15ac051541f6f3713/lib/thin/protocol/encryptDecrypt.js#L47

decrypted.length

  • Node: 32
  • Deno: 16

@chibat
Copy link
Contributor Author

chibat commented Sep 12, 2024

I have excerpted the oracledb code to check the behavior.

test.mjs

import { Buffer } from 'node:buffer';
import crypto from 'node:crypto';

const algorithm = 'aes-256-cbc';
const key = Buffer.from([0x84,0xdc,0xdd,0x96,0x49,0x68,0x73,0x4f,0xdf,0x0d,0xe4,0xa2,0xcb,0xa4,0x71,0xc2,0xe0,0xa7,0x53,0x93,0x0b,0x84,0x1c,0x01,0x4b,0x1e,0x77,0xf4,0x56,0xb5,0x79,0x7b]);
const val = Buffer.from([0xfe,0xab,0xbd,0xf6,0x6e,0x2c,0x71,0xcc,0x78,0x0d,0x0c,0xd2,0x76,0x5d,0xcc,0xe2,0x83,0xe8,0xae,0x7e,0x58,0xfc,0xc1,0xa9,0xac,0xaf,0xc6,0x78,0x58,0x1e,0x0e,0x06]);
const iv = Buffer.alloc(16, 0);
const decipher = crypto.createDecipheriv(algorithm, key, iv);
decipher.setAutoPadding(false);
let decrypted = decipher.update(val);
console.log(JSON.stringify(decrypted));
$ node test.mjs
{"type":"Buffer","data":[237,44,144,143,38,87,27,248,229,13,96,183,127,185,194,95,149,185,51,181,145,17,84,60,111,172,65,173,107,71,230,129]}

$ bun test.mjs
{"type":"Buffer","data":[237,44,144,143,38,87,27,248,229,13,96,183,127,185,194,95,149,185,51,181,145,17,84,60,111,172,65,173,107,71,230,129]}

$ deno test.mjs
{"type":"Buffer","data":[237,44,144,143,38,87,27,248,229,13,96,183,127,185,194,95]}

$ deno --version
deno 2.0.0-rc.2+c906510 (canary, release, x86_64-unknown-linux-gnu)
v8 12.9.202.13-rusty
typescript 5.5.2

kt3k added a commit that referenced this issue Sep 12, 2024
This change fixes Decipheriv behavior when autoPadding disabled and enabled.

By this change, the example given in
#20924 (comment)
works in the same way as Node.

closes #20924
@chibat
Copy link
Contributor Author

chibat commented Sep 12, 2024

I was able to connect to the Oracle DB.
Thank you all.

$ deno --version
deno 2.0.0-rc.2+e2875ae (canary, release, x86_64-unknown-linux-gnu)
v8 12.9.202.13-rusty
typescript 5.5.2

$ deno task thin
Task thin deno run --allow-sys --allow-env --allow-net main.js
{
  metaData: [
    {
      name: "TABLE_NAME",
      dbType: [DbType DB_TYPE_VARCHAR],
      nullable: false,
      isJson: false,
      isOson: false,
      byteSize: 128,
      dbTypeName: "VARCHAR2",
      fetchType: [DbType DB_TYPE_VARCHAR]
    }
  ],
  rows: [
    [ "CITIES" ],
    [ "CURRENCIES" ],
    [ "CURRENCIES_COUNTRIES" ],
    [ "REGIONS" ],
    [ "COUNTRIES" ]
  ]
}

@vegarringdal
Copy link

I just tried to connect to oracle cloud using wallet without luck, just said connecting.
if I try nodejs it works
Anyone managed to get it working ?

deno run --allow-sys --allow-env --allow-net --allow-read main.js

using popos and deno 2.0.6

main.js

import oracledb from 'oracledb'


async function connectToOracle() {
  let connection;

  try {
    console.log("connecting")
    
    // Establish connection to the Oracle Database
    connection = await oracledb.getConnection({
      user: 'ADMIN', 
      password: 'removed... ',
      connectString: 'removed... oraclecloud.com',
      walletLocation: "/home/xxxxx/Github/oracletest/wallet",
      walletPassword: 'removed... '
    });

    console.log('Successfully connected to Oracle Database!');

    // Example Query: Select all records from a sample table
    const result = await connection.execute(
      `SELECT 1 form dual` // Adjust as needed
    );

    console.log('Query Results:', result.rows);

  } catch (err) {
    console.error('Error connecting to Oracle Database:', err);
  } finally {
    if (connection) {
      try {
        await connection.close();
        console.log('Connection closed.');
      } catch (err) {
        console.error('Error closing connection:', err);
      }
    }
  }
}

connectToOracle();

@chibat
Copy link
Contributor Author

chibat commented Nov 18, 2024

@vegarringdal
If you are getting error messages can you share them?

@vegarringdal
Copy link

Sure
image

@chibat
Copy link
Contributor Author

chibat commented Nov 20, 2024

What happens if the following environment variable are set before execution?

export DENO_CERT=/home/developer/Github/oracletest/ewallet.pem

@vegarringdal
Copy link

image

same script in nodejs
image

@vegarringdal
Copy link

Connection string, just incase its doing something
(description= (retry_count=1)(retry_delay=0)(address=(protocol=tcps)(port=1522)(host=adb.uk-cardiff-1.oraclecloud.com))(connect_data=(service_name=XXXXXXXXX_maindata_high.adb.oraclecloud.com))(security=(ssl_server_dn_match=yes)))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working correctly node API Related to various "node:*" modules APIs node compat
Projects
None yet
8 participants