Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[generator] Fix NRE by cloning method when copying from base type to derived type. #1080

Merged
merged 1 commit into from
Feb 15, 2023

Conversation

jpobst
Copy link
Contributor

@jpobst jpobst commented Feb 2, 2023

Context: generator crash while binding Google Material 1.6.1

Imagine:

  • A public class that inherits from a package-private class
  • The package-private class exposes a method we would like to expose on the public class
public class MyClass extends BaseClass<Object> 
{
}

class BaseClass<V>
{
  public void DoThing (V obj) { }
}

We will not bind BaseClass because it is package-private, but we would like for its methods to be available for users. In 4ec5d4e, we supported this by copying members from the base class to the public derived class.

Or at least, that's what the commit says it does. It does not actually create a copy of the Method, it simply adds the existing Method object to the public derived class, meaning that the same Method object is referenced by both types.

In general, this is fine. However consider the case above.

DoThing (V obj) is copied to MyClass, which does not have the generic type argument V. When validation runs on the copied method, this is detected, a warning is emitted, the Method is marked as IsValid=false and is removed from the public class.

public class MyClass extends BaseClass<Object> 
{
  public void DoThing (V obj) { }   // What is V?
}
warning BG8800: Unknown parameter type 'V' for member 'MyClass.DoThing(V)'.

This is also fine and expected.

However this Method object is still referenced by the BaseClass model in its Methods collection. This version should be valid, however it has been marked as IsValid=false and the parameter ISymbol has been set to null.

After validation, class Methods collections should only contain IsValid=true Methods, but this is no longer the case. When later logic attempts to use this Method it hits an NRE trying to access the parameter type. An example NRE is below, but guarding the NRE will simply move it to another location.

System.NullReferenceException
  Message=Object reference not set to an instance of an object.
  StackTrace:
   at MonoDroid.Generation.Parameter.get_JniType() in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\Parameter.cs:line 103
   at MonoDroid.Generation.ParameterList.get_JniSignature() in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\ParameterList.cs:line 203
   at MonoDroid.Generation.Method.get_JniSignature() in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\Method.cs:line 210
   at MonoDroid.Generation.GenBase.ContainsMethod(Method method, Boolean check_ifaces, Boolean check_base_ifaces) in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\GenBase.cs:line 225
   at MonoDroid.Generation.GenBase.ContainsMethod(Method method, Boolean check_ifaces) in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\GenBase.cs:line 208
   at MonoDroid.Generation.GenBase.FixupMethodOverrides(CodeGenerationOptions opt) in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\GenBase.cs:line 341

The fix is "simple": do what the original change says it was doing and make a "copy" of the Method object using a new Clone () method. With this fix, the copied method will be invalidated and removed without corrupting the original object instance.

As seen in the unit test changes, the XPath comments are updated to point to the public type rather than the original package-private type. While this isn't "correct", neither was the previous XPath specification. After discussion, we have decided that the needed change is to not add XPath comments on "created" methods (that is, API that is created by generator and is not part of the Java public API). However, the cost of doing this change is higher than we currently wish to spend, so we will live with this minor issue for now.

@jpobst
Copy link
Contributor Author

jpobst commented Feb 13, 2023

Test XA bump to ensure no regressions:
dotnet/android#7773

@jonpryor
Copy link
Member

Draft commit message:

Context: 4ec5d4e9d1c6483f41bb84791268d8434ffb5269
Context: `generator` crash while binding Google Material 1.6.1

Imagine:

 1. A `public` class that inherits from a *package-private* class, and
 2. The *package-private* class contains `public` or `protected`
    members which should be exposed on (1).

For example:

	/* package-private*/ class BaseClass<V> {
	    public void doThing(V value) {}
	}

	public class MyClass extends BaseClass<Object> {
	}

We won't bind `BaseClass<V>` as it is *package-private*, but we want
(need) it's publicly accessible members to be available to users of
the `public` type `MyClass`; that is, *in Java*,
`MyClass.doThing(Object)` will be an accessible method, and our
binding of `MyClass` should likewise have a `MyClass.DoThing(Object)`
method binding.

In 4ec5d4e9 we supported this scenario by copying members from the
*package-private* base class into the derived class.

Or at least, that's what the commit says it does.  It does not actually
create a *copy* of the `Method`; rather, it simply adds the existing
`Method` instance to the public derived class, meaning that the same
`Method` instance is referenced by both the base & derived types!

In general, this is fine.  However consider `BaseClass<V>`, above:
the derived type `MyClass` doesn't have a generic type argument `V`,
and thus "copying" `BaseClass<V>.doThing(V)` as-is into `MyClass`
is a non-sequitur; when we later `Validate()` the `Method` instance,
this incongruity is detected and a warning is emitted:

	warning BG8800: Unknown parameter type 'V' for member 'MyClass.DoThing(V)'.

As part of validation, the `Method` instance is also marked as
invalid, by setting the `MethodBase.IsValid` property to `false`, and
the instance is removed from `GenBase.Methods` for `MyClass`.

Unfortunately that's not the end of it, because the `Method` instance
is shared and thus is also in the `GenBase.Methods` collection for
`BaseClass<V>`.  This instance is expected/assumed to be valid, but
was invalidated by `MyClass` validation.

`generator` assumes that after validation, everything within
`GenBase.Methods` is still valid.  This assumption is no longer true
for `BaseClass<V>`.  The result is that when we later attempt to
process `BaseClass<V>`, things blow up when trying to use the parameter
types on `BaseClass<V>.doThing(V)`:

	System.NullReferenceException: Object reference not set to an instance of an object.
	   at MonoDroid.Generation.Parameter.get_JniType() in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\Parameter.cs:line 103
	   at MonoDroid.Generation.ParameterList.get_JniSignature() in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\ParameterList.cs:line 203
	   at MonoDroid.Generation.Method.get_JniSignature() in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\Method.cs:line 210
	   at MonoDroid.Generation.GenBase.ContainsMethod(Method method, Boolean check_ifaces, Boolean check_base_ifaces) in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\GenBase.cs:line 225
	   at MonoDroid.Generation.GenBase.ContainsMethod(Method method, Boolean check_ifaces) in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\GenBase.cs:line 208
	   at MonoDroid.Generation.GenBase.FixupMethodOverrides(CodeGenerationOptions opt) in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\GenBase.cs:line 341


The fix is "simple": do what 4ec5d4e9 said it was doing and make a
*copy* of the `Method` instance using a new `Clone()` method.  With
this fix, the copied method will be invalidated and removed without
corrupting the original `Method` instance.

*Note*: As seen in the unit test changes, the XPath comments are
updated to point to the public type rather than the original package-
private type.  While this isn't "correct", neither was the previous
XPath specification.  After discussion, we have decided that the
needed change is to not add XPath comments on "created" methods (that
is, API that is created by `generator` and is not part of the Java
public API).  However, the cost of doing this change is higher than we
currently wish to spend, so we will live with this minor issue for now.

clone.GenericArguments.Add (ga.Clone ());

foreach (var p in Parameters)
clone.Parameters.Add (p.Clone ());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While writing the draft commit message, my thoughts turn to BaseClass<V>.doThing(V): wouldn't this still have a parameter type of V? How is that "resolved" via declaringType.TypeParameters?

Rephrased, I would expect that the clone instance will have clone.Parameters.Count==1, and clone.Parameters[0] equivalent to:

new Parameter (
        name: "value",
        type: "java.lang.Object",
        managedType: "Java.Lang.Object"
        isEnumified: false
);

but I don't see this happening. Should this be happening?

Or is this instead handled by new lines 128-130 (above), wherein we added this.GenericArguments to clone.GenericArguments. Though I'm not certain about that either, as shouldn't GenericArguments be empty as BaseClass<V>.doThing(V) isn't itself generic…

var base_class = gens.Single (g => g.Name == "ViewOffsetBehavior");

// Method got removed
Assert.AreEqual (0, public_class.Methods.Count);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this asserting that the method got removed?

OK, this test is asserting that we no longer NRE, that the BG8800 warning is still emitted (implicitly), that public_class has no methods, and base_class.Methods is still sane.

But… shouldn't the "final" fix be a situation in which:

  1. public_class.Methods.Count is also 1, and
  2. public_class.Methods[0].IsValid is true, and
  3. public_class.Methods[0].Parameters[0].JniType is java/lang/Object?

Upon review, it feels like this PR fixes the NRE, but doesn't resolve the underlying scenario allowing publicly accessible members from package-private types to be bound in the derived type.

Is that possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, this PR only fixes the NRE. It does not try to fix the underlying issue of correctly supporting generics when copying methods from a package-private class. I do not believe that is a topic we want to tackle any time soon.

@jonpryor jonpryor merged commit 9e0a469 into main Feb 15, 2023
@jonpryor jonpryor deleted the clone-fix branch February 15, 2023 02:15
jonpryor pushed a commit to dotnet/android that referenced this pull request Feb 15, 2023
Changes: dotnet/java-interop@8a1ae57...9e0a469

  * dotnet/java-interop@9e0a4690: [generator] deep clone methods to avoid NREs (dotnet/java-interop#1080)
  * dotnet/java-interop@5fa7ac45: [java-source-utils] Fix lgtm java/path-injection-local (dotnet/java-interop#1079)
  * dotnet/java-interop@120d8a71: [generator] Add more [ObsoleteOSPlatform] to prevent CA1422 (dotnet/java-interop#1078)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
jonpryor pushed a commit that referenced this pull request Mar 30, 2023
…1089)

Context: #1080
Context: 9e0a469

When building an updated
`com.google.firebase.firebase-components.csproj` in
xamarin/GooglePlayServicesComponents using a *post*- 9e0a469
`generator`, the build hits an NullReferenceException when attempting
to `Method.Clone (…)` a method with generic arguments:

	error BG0000: System.NullReferenceException: Object reference not set to an instance of an object.
	   at MonoDroid.Generation.Method.Clone(GenBase declaringType) in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\Method.cs:line 131
	   at MonoDroid.Generation.Method.Clone(GenBase declaringType) in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\Method.cs:line 131
	   at MonoDroid.Generation.ClassGen.FixupAccessModifiers(CodeGenerationOptions opt) in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\ClassGen.cs:line 67
	   at MonoDroid.Generation.ClassGen.FixupAccessModifiers(CodeGenerationOptions opt) in C:\code\xamarin-android\external\Java.Interop\tools\generator\Java.Interop.Tools.Generator.ObjectModel\ClassGen.cs:line 67
	   at Xamarin.Android.Binder.CodeGenerator.Validate(List`1 gens, CodeGenerationOptions opt, CodeGeneratorContext context) in C:\code\xamarin-android\external\Java.Interop\tools\generator\CodeGenerator.cs:line 300
	   at Xamarin.Android.Binder.CodeGenerator.Validate(List`1 gens, CodeGenerationOptions opt, CodeGeneratorContext context) in C:\code\xamarin-android\external\Java.Interop\tools\generator\CodeGenerator.cs:line 300
	   at Xamarin.Android.Binder.CodeGenerator.Run(CodeGeneratorOptions options, DirectoryAssemblyResolver resolver) in C:\code\xamarin-android\external\Java.Interop\tools\generator\CodeGenerator.cs:line 208
	   at Xamarin.Android.Binder.CodeGenerator.Run(CodeGeneratorOptions options, DirectoryAssemblyResolver resolver) in C:\code\xamarin-android\external\Java.Interop\tools\generator\CodeGenerator.cs:line 208
	   at Xamarin.Android.Binder.CodeGenerator.Run(CodeGeneratorOptions options) in C:\code\xamarin-android\external\Java.Interop\tools\generator\CodeGenerator.cs:line 50
	   at Xamarin.Android.Binder.CodeGenerator.Run(CodeGeneratorOptions options) in C:\code\xamarin-android\external\Java.Interop\tools\generator\CodeGenerator.cs:line 50
	   at Xamarin.Android.Binder.CodeGenerator.Main(String[] args) in C:\code\xamarin-android\external\Java.Interop\tools\generator\CodeGenerator.cs:line 33
	   at Xamarin.Android.Binder.CodeGenerator.Main(String[] args) in C:\code\xamarin-android\external\Java.Interop\tools\generator\CodeGenerator.cs:line 33

This is because the `MethodBase.GenericArguments` collection is `null`
unless it has items.  Thus we need to create it before populating it
with items in `Method.Clone(GenBase)`.
@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants