Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HUDI-3633] Allow non-string values to be set in TypedProperties #5045

Merged
merged 2 commits into from
Mar 15, 2022

Conversation

codope
Copy link
Member

@codope codope commented Mar 15, 2022

What is the purpose of the pull request

TypedProperties invokes Properties#stringPropertynames() which expects that key and values should be strings. It causes keys with non-string values to be not found. See #5026 . This PR fixes that behavior.

Brief change log

(for example:)

  • Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request

(Please pick either of the following options)

This pull request is a trivial rework / code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end.
  • Added HoodieClientWriteTest to verify the change.
  • Manually verified the change by running a job locally.

Committer checklist

  • Has a corresponding JIRA in PR title & commit

  • Commit message is descriptive of the change

  • CI is green

  • Necessary doc changes done or have another open PR

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@wangxianghu wangxianghu self-assigned this Mar 15, 2022
@wangxianghu
Copy link
Contributor

wangxianghu commented Mar 15, 2022

hi @codope This patch solved issue #5026
but if the user use

typedProperties.put("key", 123);

the kv still lost

may be we should override java.util.Properties#getProperty(java.lang.String) to ignore the string type check
the getProperty method in java.util.Properties:

    public String getProperty(String key) {
        Object oval = super.get(key);
        String sval = (oval instanceof String) ? (String)oval : null;
        return ((sval == null) && (defaults != null)) ? defaults.getProperty(key) : sval;
    }

@codope
Copy link
Member Author

codope commented Mar 15, 2022

may be we should override java.util.Properties#getProperty(java.lang.String) to ignore the string type check
the getProperty method in java.util.Properties

We could do that. I wonder why was this instanceof String check added?

@wangxianghu
Copy link
Contributor

may be we should override java.util.Properties#getProperty(java.lang.String) to ignore the string type check
the getProperty method in java.util.Properties

We could do that. I wonder why was this instanceof String check added?

The return type is String, use instanceof String check may avoid class cast exception

@codope
Copy link
Member Author

codope commented Mar 15, 2022

Still the TypedProperties constructor takes a Properties argument and calling defaults.getProperty won't have the desired effect.

@nsivabalan nsivabalan added the priority:major degraded perf; unable to move forward; potential bugs label Mar 15, 2022
Copy link
Contributor

@wangxianghu wangxianghu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wangxianghu
Copy link
Contributor

@hudi-bot run azure

@hudi-bot
Copy link

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@wangxianghu wangxianghu merged commit d514570 into apache:master Mar 15, 2022
vingov pushed a commit to vingov/hudi that referenced this pull request Apr 3, 2022
…che#5045)

* [HUDI-3633] Allow non-string values to be set in TypedProperties

* Override getProperty to ignore instanceof string check
stayrascal pushed a commit to stayrascal/hudi that referenced this pull request Apr 12, 2022
…che#5045)

* [HUDI-3633] Allow non-string values to be set in TypedProperties

* Override getProperty to ignore instanceof string check
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority:major degraded perf; unable to move forward; potential bugs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[SUPPORT] TypedProperties can not set non-string values using its original type
4 participants