Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

question: defaultMetadataStorage is no more available in 0.3.2 #563

Open
Arnaud-Dev-Nodejs opened this issue Jan 19, 2021 · 36 comments
Open
Labels
type: question Questions about the usage of the library.

Comments

@Arnaud-Dev-Nodejs
Copy link

Description

In the previous version it was possible to access to the defaultMetadataStorage which was useful to create custom transformer.

for example :
Minimal code-snippet showcasing the problem

import { TransformOptions } from 'class-transformer';
import { defaultMetadataStorage } from 'class-transformer/storage/'; <== cannot find module anymore

export function doSplit(separator: string | RegExp = ' ') {
  return (value: any) => {
    return value !== undefined && value !== null
      ? String(value).split(separator)
      : [];
  };
}
export function Split(
  separator: string | RegExp,
  options: TransformOptions = {},
): PropertyDecorator {
  const transformFn = doSplit(separator);

  return function (target: any, propertyName: string | symbol): void {
    defaultMetadataStorage.addTransformMetadata({
      target: target.constructor,
      propertyName: propertyName as string,
      transformFn,
      options,
    });
  };
}

Expected behavior

Get the defaultMetadataStorage !

Actual behavior

"Cannot find module 'class-transformer/storage' or its corresponding type declarations."

@Arnaud-Dev-Nodejs Arnaud-Dev-Nodejs added status: needs triage Issues which needs to be reproduced to be verified report. type: fix Issues describing a broken feature. labels Jan 19, 2021
@NoNameProvided
Copy link
Member

You should be able to reach it from class-transformer/MetadataStorage.ts, but keep in mind that this is an internal class, and can break anytime.

@NoNameProvided NoNameProvided added type: question Questions about the usage of the library. and removed status: needs triage Issues which needs to be reproduced to be verified report. type: fix Issues describing a broken feature. labels Jan 19, 2021
@NoNameProvided NoNameProvided changed the title fix: defaultMetadataStorage is no more available in 0.3.2 question: defaultMetadataStorage is no more available in 0.3.2 Jan 19, 2021
@igoreso
Copy link

igoreso commented Jan 27, 2021

Hi, we used defaultMetadataStorage to extract all applied validations so we could construct our OpenAPI schema on the fly. I understand that this is a "private" class of sorts, but we're ready to adapt to any changes, as there is no other way for us to automate OpenAPI generation.

edit 1:
Strangely, source code doesn't seem to be changed, but imports don't work the same way. Could it be because of changes in tsconfig.json?

edit 2:
Found a workaround, @Arnaud-Dev-Nodejs you may want to try this.
import { defaultMetadataStorage } from 'class-transformer/cjs/storage';

@mlakmal
Copy link

mlakmal commented Jan 27, 2021

Looks like new release changed the npm library definitions to use cjs and other formats to support library package formats. Looks like this is also breaking the package class-validate-jsonschema since it’s also referencing defaultMetadataStorage...

@NoNameProvided
Copy link
Member

Strangely, source code doesn't seem to be changed, but imports don't work the same way. Could it be because of changes in tsconfig.json?

Yes, we move to the universal package format for each TypeStack package. However, this should not break your workflow as your build tool should be able to understand this as we reference each entry point properly.

"main": "./cjs/index.js",
"module": "./esm5/index.js",
"es2015": "./esm2015/index.js",
"typings": "./types/index.d.ts",

Can I ask what build tools you use @igoreso?

@PhilippMolitor
Copy link

Trying @igoreso 's workaround produces this error:

Could not find a declaration file for module 'class-transformer/cjs/storage'. './node_modules/class-transformer/cjs/storage.js' implicitly has an 'any' type.
  Try `npm i --save-dev @types/class-transformer` if it exists or add a new declaration (.d.ts) file containing `declare module 'class-transformer/cjs/storage';`ts(7016)

I am also not able to see why the "storage" module has vanished from the typescript detection. maybe only the index is not exposed, and defaultMetadataStorage should just be re-exported from the index module? maybe named so it wont break with future releases?

@NoNameProvided
Copy link
Member

NoNameProvided commented Feb 20, 2021

maybe only the index is not exposed, and defaultMetadataStorage should just be re-exported from the index module?

It won't be re-exported, because it's an internal part of the lib and we don't support its API officially. I won't actively limit the access to it, but it will be never included in the barrel export to indicate that is not an official API.

so it wont break with future releases?

It's an internal part of the lib, it will break with future releases.

@NoNameProvided
Copy link
Member

NoNameProvided commented Feb 20, 2021

What we can do here is update the project config if needed to allow importing it directly with every build tool from its direct path. For that, I will need what build tool cannot find it.

@NoNameProvided NoNameProvided added the status: awaiting answer Awaiting answer from the author of issue or PR. label Feb 22, 2021
@Arnaud-Dev-Nodejs
Copy link
Author

Nestjs :)

In fact, nobody needs to access your internal interface. The only thing we want is the ability to add dynamic Transformer.

@tmtron
Copy link
Contributor

tmtron commented Feb 22, 2021

Nestjs :)

In fact, nobody needs to access your internal interface. The only thing we want is the ability to add dynamic Transformer.

That is misleading.
I'd argue that everybody who has an advanced use-case absolutely needs access to the internal interface because there is no other way to do that.

E.g. we need access to the column-mapping somehow.
Of course, we would also prefer a public, stable and documented way to do this (instead of accessing internal private parts, that can change without notice between releases).
But we do understand that this is an advanced use-case and we don't expect the library authors to invest lots of effort, just to cover exotic use-cases.

@nolazybits
Copy link

same problem here, updating to 0.3.2 breaks the import.
using require syntax instead of imports works though.
Node version 14

// import { defaultMetadataStorage as classTransformerDefaultMetadataStorage, } from "class-transformer/storage";
// eslint-disable-next-line @typescript-eslint/no-var-requires, @typescript-eslint/no-require-imports
const classTransformerDefaultMetadataStorage = require(`class-transformer/storage`);

@epiphone
Copy link

You can also import the ES module as import { defaultMetadataStorage } from 'class-transformer/esm5/storage' and then provide types separately with a .d.ts declaration like

declare module 'class-transformer/esm5/storage' {
  import type { MetadataStorage } from 'class-transformer/types/MetadataStorage';

  export const defaultMetadataStorage: MetadataStorage;
}

This is a bit of a hassle though: exposing defaultMetadataStorage or another API for accessing metadata objects would make it much easier to develop libraries that integrate with class-transformer. Note that metadata object types are already part of the public API.

@nolazybits
Copy link

Just wondering @NoNameProvided why you wouldn't want to expose it?
There are needs it seems from this thread.

One I can contribute to is we are testing our Models to make sure the dto (domain transfer object) don't change
Here is the class to do those validation

/* eslint-disable @typescript-eslint/ban-types */
import { ExcludeOptions, ExposeOptions } from "class-transformer";
import { ExcludeMetadata, ExposeMetadata, TypeMetadata } from "class-transformer";
// import { defaultMetadataStorage as classTransformerDefaultMetadataStorage } from "class-transformer/types/storage";
const classTransformerDefaultMetadataStorage = require(`class-transformer/storage`);
import { getMetadataStorage, ValidationOptions, ValidationTypes } from "class-validator";
import { ConstraintMetadata } from "class-validator/types/metadata/ConstraintMetadata";
import { ValidationMetadata } from "class-validator/types/metadata/ValidationMetadata";

export class ValidationConstraintUtils
{
    public static TestExposeMetadata(target: Function, propertyName?: string, options: ExposeOptions = {}): void
    {
        // eslint-disable-next-line @typescript-eslint/ban-ts-comment
        // @ts-ignore
        const exposeMetadata: ExposeMetadata = classTransformerDefaultMetadataStorage.findExposeMetadata(target, propertyName);
        expect(exposeMetadata).toBeDefined();
        expect(exposeMetadata.options).toEqual(options);
    }

    public static TestNotExposeMetadata(target: Function, propertyName?: string, options: ExcludeOptions = {}): void
    {
        // eslint-disable-next-line @typescript-eslint/ban-ts-comment
        // @ts-ignore
        const exposeMetadata: ExposeMetadata | undefined = classTransformerDefaultMetadataStorage.findExposeMetadata(target, propertyName);
        expect(exposeMetadata).toBeUndefined();
    }

    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
    // @ts-ignore
    public static TestExcludeMetadata(target: Function, propertyName?: string, options: ExcludeMetadata = {}): void
    {
        // eslint-disable-next-line @typescript-eslint/ban-ts-comment
        // @ts-ignore
        const excludeMetadata: ExcludeMetadata = classTransformerDefaultMetadataStorage.findExcludeMetadata(target, propertyName);
        expect(excludeMetadata).toBeDefined();
        expect(excludeMetadata.options).toEqual(options);
    }

    public static TestTypeMetadata(target: Function, propertyName: string, type: Function): void
    {
        const typeMetadata: TypeMetadata = classTransformerDefaultMetadataStorage.findTypeMetadata(target, propertyName);
        expect(typeMetadata).toBeDefined();
        expect(typeMetadata.typeFunction()).toBe(type);
    }

    public static TestValidation(target: Function, propertyName: string, validationNames: ValidationTypes[]): void
    {
        const validationChecks: Array<Partial<ValidationMetadata>> = [];
        for (const validationName of validationNames)
        {
            validationChecks.push(expect.objectContaining({
                propertyName: propertyName,
                type: validationName
            }));
        }

        const validation: ValidationMetadata[] = getMetadataStorage().getTargetValidationMetadatas(target, ``, true, false).filter((value: ValidationMetadata) => value.propertyName === propertyName && value.type !== `customValidation`);
        expect(validation).toEqual(
            expect.arrayContaining(validationChecks)
        );
    }

    // eslint-disable-next-line @typescript-eslint/no-explicit-any
    public static TestConstraints(target: Function, propertyName: string, constraints: Array<{ name: string; options?: any[]; validationOptions?: ValidationOptions }>): void
    {
        const validationMetadatas: ValidationMetadata[] = getMetadataStorage().getTargetValidationMetadatas(target, ``, true, false)
            .filter((value: ValidationMetadata) => value.propertyName === propertyName && value.type === `customValidation`);

        // eslint-disable-next-line @typescript-eslint/no-explicit-any
        const constraintMetadatas: ConstraintMetadata[] = (getMetadataStorage() as any).constraintMetadatas
            .filter((constraint: ConstraintMetadata) => validationMetadatas.find((validation: ValidationMetadata) => validation.constraintCls === constraint.target));

        const constraintsCheck: Array<Partial<ConstraintMetadata>> = [];
        for (const constraint of constraints)
        {
            const constraintMetadata: ConstraintMetadata | undefined = constraintMetadatas.find((c: ConstraintMetadata) => c.name === constraint.name);
            expect(constraintMetadata).toBeDefined();

            const validationMetadata: ValidationMetadata | undefined = validationMetadatas.find((v: ValidationMetadata) => v.constraintCls === constraintMetadata?.target);
            expect(validationMetadata).toBeDefined();

            if (constraint.validationOptions)
            {
                expect(validationMetadata).toEqual(
                    expect.objectContaining(constraint.validationOptions)
                );
            }

            constraintsCheck.push(expect.objectContaining({name: constraint.name}));

            if (constraint.options)
            {
                expect(validationMetadata?.constraints.length).toBe(constraint.options.length);
                for (let i = 0; i < constraint.options.length; i++)
                {
                    expect(validationMetadata?.constraints[i]).toEqual(constraint.options[i]);
                }
            }
        }

        expect(constraintMetadatas).toEqual(
            expect.arrayContaining(constraintsCheck)
        );
    }
}

and here are test using it

    test(`paramOptions`, async() =>
    {
        ValidationConstraintUtils.TestExposeMetadata(target, `paramOptions`, {name: `param_options`});
        ValidationConstraintUtils.TestValidation(target, `paramOptions`, [ValidationTypes.CONDITIONAL_VALIDATION, ValidationTypes.NESTED_VALIDATION]);
    });

@nolazybits
Copy link

acutally importing it as I shown above failed today.
Another way was this
import { defaultMetadataStorage as classTransformerDefaultMetadataStorage } from "class-transformer/cjs/storage";
with aliasing, otherwise it doesn't get picked...
This is super weird...

TS 4.1.2
Node 14
Using ts-node latest

@NoNameProvided
Copy link
Member

Yeah, this is a tooling bug with the new format, I am kind of out of depth here, so I will ask about this on SO and hope someone knows whats up.

@NoNameProvided NoNameProvided removed the status: awaiting answer Awaiting answer from the author of issue or PR. label Mar 24, 2021
@whimzyLive
Copy link

@NoNameProvided is there any plan to make this API public so that other libraries can more easily and reliably interact with class-transformer?
My use case is, I need to access user-defined type for nested objects on models, and the class-transformer is already storing/ requiring users to annotate this type, so I was hoping to reuse type metadata information that is already collected by the class-transformer.
Also, Let me know if this is ever going to happen.

@kay-schecker
Copy link

Also voting to have defaultMetadataStorage as public API. It's very very useful to build advanced features on top of class-transformer. So please add support <3

@DennisKuhn
Copy link

I would also greatly appreciate to have the metadata defaultMetadataStorage as public API.

@xkguq007
Copy link

any update in here? we really need to use storage for implementing @nestjs/mapped-types in browser

hjf added a commit to hjf/mapped-types that referenced this issue Oct 1, 2021

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fixes error related to typestack/class-transformer#563 that makes it not be able to build --webpack.
@xerxes235
Copy link

I would also greatly appreciate having defaultMetadataStorage as a public API.

@ghost
Copy link

ghost commented Mar 4, 2022

Any update on this? Without exposing the storage or at least exposing methods to retrieve the metadata based on a target any additional tooling won't be able to benefit from the metadata already generated by the decorators provided by class-transformer.

@nosachamos
Copy link

nosachamos commented Dec 9, 2022

I support making this public - we need access to it in order to get the metadata. Please make this public, guys. It's really not a big deal.

When I attempt to import {defaultMetadataStorage} from 'class-transformer/esm5/storage' I get

SyntaxError: Cannot use import statement outside a module

@natejgardner
Copy link

Any updates? Would be good to prioritize this issue as it prevents class transformer from being used in new projects.

@aqibgatoo
Copy link

Kindly make defaultMetadataStorage public.

@krzysiek3d
Copy link

Importing it like this in my ts file worked for Me :)

// @ts-nocheck
import { defaultMetadataStorage } from 'node_modules/class-transformer/esm5/storage.js';

class-transformer v. 0.5.1
TS 4.8.2
Node 18.12.1

@ptheofan
Copy link

I also cannot load it, the import mentioned didn't work for me. Any luck making it public?

@zingmane
Copy link

In a deno project it worked for me only via commonjs import 🤷‍♂️

// @ts-ignore
import { defaultMetadataStorage } from "npm:class-transformer@0.5.1/cjs/storage.js";

@codemile
Copy link

codemile commented May 29, 2023

This works for me in version 0.14.0

import {getMetadataStorage} from 'class-validator';

While I've read the entire thread above. It wasn't clear the import problem had been fixed and this issue is still open.

Also, while I appreciate this is an internal feature of the library. It's not mentioned in the README file, and if you're like me going to Google or going to ChatGPT for a solution of "how do I get the metadata at run-time" then there is a lot of public information pointing to this function as a feature of the library.

It would be nice, if a footnote could be added to the README explaining how to import the method with a stern warning that it's not an official part of the API and subject to change. That would be good enough for most use cases.

@TechQuery
Copy link

@codemile Thanks your discovery, but your solution is incompatible with class-validator-jsonschema, such as:

error TS2740: Type 'MetadataStorage' is missing the following properties from type 'MetadataStorage': _typeMetadatas, _transformMetadatas, _exposeMetadatas, _excludeMetadatas, and 20 more.

14             classTransformerMetadataStorage: getMetadataStorage(),
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  node_modules/.pnpm/class-validator-jsonschema@5.0.0_class-transformer@0.5.1_class-validator@0.14.0/node_modules/class-validator-jsonschema/build/options.d.ts:6:5
    6     classTransformerMetadataStorage?: ClassTransformerMetadataStorage;
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    The expected type comes from property 'classTransformerMetadataStorage' which is declared here on type 'Partial<IOptions>'

@ronskyi
Copy link

ronskyi commented Nov 24, 2023

Any chance metadata will be exported? My use case is to get an @Expose() metadata during serializing validation errors. Thnx.

@taiwanhua
Copy link

taiwanhua commented Jan 20, 2024

Hi , i'm not sure which is correct or not, i try :

import { MetadataStorage } from "class-transformer/types/MetadataStorage";

...
 const schemas = validationMetadatasToSchemas({
      // classTransformerMetadataStorage: defaultMetadataStorage,
      classTransformerMetadataStorage: new MetadataStorage(),
      refPointerPrefix: "#/components/schemas/",
    });

I read this and found it just new MetadataStorage() and export

@cojack
Copy link
Contributor

cojack commented Jun 24, 2024

This is so dumb that this was excluded that I can not even imagine.

@elliot-sabitov
Copy link

It looks like there is an open PR already here #1715, can this be moved forward? 🙏

@Javimtib92
Copy link

We would also benefit from this PR being merged 🙏

@elliot-sabitov
Copy link

For anyone that needs it, the intermediate solution right now for us has been to use:

import { defaultMetadataStorage } from 'class-transformer/cjs/storage.js';

but this may require additional configuration and it would be much better if we can just do:

import { defaultMetadataStorage } from 'class-transformer';

@arnaugomez
Copy link

It would be great to have this PR merged, as the defaultMetadataStorage API is useful for libraries that interoperate with the class-transformer library, such as the one we have been developing: https://github.com/mrmilu/schema_data_loader

@Javimtib92
Copy link

Meanwhile, as a hacky workaround I created a post install script to modify the files installed by npm to expose storage.

Note: the patch covers our use case but doesn't care about the bundled umd file for example. If you use this patch feel free to adapt it as needed.

/**
 * @author Javier Muñoz Tous <javimtib92@gmail.com>
 *
 * class-transformer version 0.5.1 patch
 *
 * This patch aims to address an inconvenience introduced in class-transformer version 0.3.2 making storage module no longer
 * exposed as a public module.
 *
 * While we wait for this PR to be merged [feat(export): export defaultMetadataStorage](https://github.com/typestack/class-transformer/pull/1715)
 * we decided to introduce this patch as a quick workaround to avoid importing the customjs version of the class-transformer files in vite.
 *
 */

import fs from "fs";
import path from "path";
import { fileURLToPath } from "url";

const __dirname = getDirname();

(() => {
  const nodeModulesDir = path.join(findRootWithPackageJson(), "node_modules");

  const classTransformerDir = path.join(nodeModulesDir, "class-transformer");

  const classTransformerPackageFile = getPackageJson(classTransformerDir);

  if (classTransformerPackageFile.version !== "0.5.1") {
    console.warn("Warning: class-transformer-0.5.1.patch is being applied to a different version. Please verify if this patch is still necessary.");
  }

  const items = fs.readdirSync(classTransformerDir);

  // Iterate over the items and filter only directories
  for (const item of items) {
    const itemPath = path.join(classTransformerDir, item);

    if (!fs.statSync(itemPath).isDirectory()) {
      continue;
    }

    const subItems = fs.readdirSync(itemPath);

    for (const subItem of subItems) {
      const subItemPath = path.join(itemPath, subItem);

      // Check if the subItem is a file and does not have the .map extension
      if (fs.statSync(subItemPath).isFile() && subItem.startsWith("index") && path.extname(subItem) !== ".map" && path.extname(subItem) !== ".ts") {
        let fileContent = fs.readFileSync(subItemPath, "utf-8");
        let modifiedContent = fileContent;

        const comment = `/**\n * This file has been modified by patch "class-transformer-0.5.1.patch.js.\n * The patch introduces changes to append storage exports in 'cjs', 'esm5', and 'esm2025' module types.\n */\n\n`;

        if (!fileContent.startsWith(comment)) {
          modifiedContent = comment + modifiedContent;
        } else {
          // If patch was already applied to this file we exit early
          console.error(`Patch class-transformer-0.5.1.patch is already applied. Skipping...`);

          return;
        }

        switch (item) {
          case "cjs": {
            const searchLine = '__exportStar(require("./enums"), exports);';
            const appendLine = '__exportStar(require("./storage"), exports);';

            if (fileContent.includes(searchLine)) {
              modifiedContent = modifiedContent.replace(searchLine, `${searchLine}\n${appendLine}`);
            }

            break;
          }
          case "esm5":
          case "esm2015": {
            const searchLine = "export * from './enums';";
            const appendLine = "export * from './storage';";

            if (fileContent.includes(searchLine)) {
              modifiedContent = modifiedContent.replace(searchLine, `${searchLine}\n${appendLine}`);
            }
            break;
          }
        }

        if (modifiedContent.length !== fileContent.length) {
          fs.writeFileSync(subItemPath, modifiedContent, "utf-8");
        }
      }
    }
  }

  console.log(`\x1b[32m Applied class-transformer-0.5.1.patch  \x1b[0m`);
})();

function getDirname() {
  try {
    return __dirname;
  } catch {
    const __filename = fileURLToPath(import.meta.url);

    const __dirname = path.dirname(__filename);

    return __dirname;
  }
}

function getPackageJson(currentDir = __dirname) {
  const packageJsonPath = path.join(currentDir, "package.json");

  if (fs.existsSync(packageJsonPath)) {
    const content = fs.readFileSync(packageJsonPath, "utf-8");
    const packageJson = JSON.parse(content);
    return packageJson;
  } else {
    throw new Error("class-transformer package not found");
  }
}

/**
 * Search the neareast directory that contains a package.json file.
 * @param {string} currentDir
 * @returns
 */
function findRootWithPackageJson(currentDir = __dirname) {
  const packagePath = path.join(currentDir, "package.json");

  if (fs.existsSync(packagePath)) {
    return currentDir;
  }

  const parentDir = path.resolve(currentDir, "..");
  if (parentDir === currentDir) {
    throw new Error("package.json not found in any parent directory");
  }

  return findRootWithPackageJson(parentDir);
}

You can add this script to the post install script like so:

"scripts": {
    "postinstall": "node scripts/patches/class-transformer-0.5.1.patch.js",
},

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: question Questions about the usage of the library.
Development

No branches or pull requests