Skip to content

Commit

Permalink
[Java.Interop.Tools.Generator] Create a v2 version of map.csv (#646)
Browse files Browse the repository at this point in the history
For ages, `generator --enumfields` has supported a `map.csv` format,
which was used within xamarin-android to "enumify" `Mono.Android.dll`.

This format was entirely undocumented.

Add `Documentation/EnumMappingFile.md` and document the format.

The previous `map.csv` line is 5-6 "columns", comma-delimited:

  * API level the field was added in
  * C# namespace and type of the C# enum to create.
  * C# enum member name to create.
  * C# enum member value
  * JNI field to convert to the C# enum member, in the format
    `{JNI type reference}.{Java field name}`.
  * Flags?  If (1) present, and (2) has the value `flags`, the
    created C# enum has the `[Flags]` attribute.

Add support for a new 7-8 column line format, also documented in
`EnumMappingFile.md`:

  * Action: `E`num, `A`dd, `R`emove, `I`gnore, `?` (decide later)
  * API level
  * JNI field to convert to the C# enum member, in the format
    `{JNI type reference}.{Java field name}`.
  * C# enum member value (which is the Java field value)
  * C# namespace and type of C# enum to create
  * C# enum member name to create.
  * What to do with the Java field: `remove` it or `keep` it.
    Replaces the previous `- ENTER TRANSIENT MODE -` "command".
  * Flags?  If (1) present, and (2) has the value `flags`, the
    created C# enum has the `[Flags]` attribute.

Note that the column ordering for the 7-8 column format differs from
the previous 5-6 column format.  This change in ordering is done for
improved "enumification workflow" support: future tooling will
generate the 7-8 column output *without* knowing what the C# enum type
and members will be.  Placing the JNI information "leftmost" allows
the output to be "sensibly grouped" by tooling:

	?,0,com/example/Class.FIELD,0,,,remove,

Support for parsing `map.csv` has been moved out of `generator.exe`
and into the new `Java.Interop.Tools.Generator.dll` assembly, in
`src/Java.Interop.Tools.Generator`.  This new assembly will also be
used by a forthcoming `generator-utilities` app.
  • Loading branch information
jpobst authored May 19, 2020
1 parent 1708d8a commit b00e644
Show file tree
Hide file tree
Showing 18 changed files with 1,041 additions and 58 deletions.
98 changes: 98 additions & 0 deletions Documentation/EnumMappingFile.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Enumeration Mapping File Documentation

## Background

In order to help with binding enumification (the process of converting groups
of Java constant int fields into C# enums), a file can be created that defines
a map between the fields and the enums to be created.

There is a CSV format and an XML format of this file. In practice, users are
guided to the XML format via our [documentation][0] and templates. The CSV format
is mainly used internally for `Mono.Android.dll`, as it converts thousands of
constants into hundreds of enums.

## CSV Format

The basic format since the beginning of Xamarin contains up to 6 fields:

* **API Level** - This is generally only used by `Mono.Android.dll` to denote
the Android level the constant was introduced in. For other uses this
is generally `0`.
* **Enum Type** - C# namespace and type of the enum to create. For example:
`Android.Views.WindowProgress`.
* **Enum Member** - C# name of the enum to create. For example:
`Start`
* **Enum Value** - The value of the enum. For example: `0`.
* **JNI Signature** - The JNI signature of the Java constant to convert. For example:
`android/view/Window.PROGRESS_START`.
* **Flags** - If this field contains `flags` the enum will be created with the
`[Flags]` attribute. (Any member will `flags` will make the whole enum `[Flags]`.)

Full example:
```
10,Android.Views.WindowProgress,Start,android/view/Window.PROGRESS_START,0,flags
```

---
**NOTE**

Our CSV files also allow comments using `//`. Lines beginning with this
sequence are ignored.

---

### Transient Mode

By default, Java constants referenced in this format are kept. However the
file can contain a line like this at any point:
```
- ENTER TRANSIENT MODE -
```

Any v1 constants referenced *AFTER* this line will be removed from the bindings.
(This will not affect v2 constants.)

## CSV Format v2

Over time we have found some limitations to the format, such as being able
to specify if the Java constant field should be removed. Additionally, since the
format only specifies constants that *SHOULD* be mapped, we cannot use this
to track constants that we have examined and determined *SHOULD NOT* be mapped.
This has led to various blacklists and tooling of varying success to prevent
us from needing to continually re-audit those constants.

There is now a "v2" version of defining constants. This is a line-level change
and you can mix "v1" and "v2" lines in the same file for backwards compatibility,
but for consistency it's probably better to stick to one style.

A "v2" line contains up to 8 fields:

* **Action** - The action to perform. This is what denotes a "v2" line, if the first
character is not one of the following it will be treated as "v1".
* `E` - Create a C# enum from a Java constant
* `A` - Create a C# enum not mapped to a Java constant
* `R` - Remove a Java constant but do not create a C# enum
* `I` - Explicitly ignore this Java constant
* `?` - Unknown, an explicit action has not been decided yet, will be ignored
* **API Level** - This is generally only used by `Mono.Android.dll` to denote
the Android level the constant was introduced in. For other uses this
is generally `0`.
* **JNI Signature** - The JNI signature of the Java constant to convert. For example:
`android/view/Window.PROGRESS_START`.
* **Enum Value** - The value of the enum. For example: `0`.
* **Enum Type** - C# namespace and type of the enum to create. For example:
`Android.Views.WindowProgress`.
* **Enum Member** - C# name of the enum to create. For example:
`Start`
* **Field Action** - Action to take on the Java constant. (This replaces Transient mode.)
* `remove` - Remove the Java constant
* `keep` - Keeps the Java constant
* **Flags** - If this field contains `flags` the enum will be created with the
`[Flags]` attribute. (Any member will `flags` will make the whole enum `[Flags]`.)

Full example:
```
E,10,android/view/Window.PROGRESS_START,0,Android.Views.WindowProgress,Start,remove,flags
```

[0]: https://docs.microsoft.com/en-us/xamarin/android/platform/binding-java-library/customizing-bindings/java-bindings-metadata#enumfieldsxml-and-enummethodsxml
14 changes: 14 additions & 0 deletions Java.Interop.sln
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,10 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Java.Interop.Tools.JavaSour
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "param-name-importer", "tools\param-name-importer\param-name-importer.csproj", "{0E3AF6C1-7638-464D-9174-485D494499DC}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Java.Interop.Tools.Generator", "src\Java.Interop.Tools.Generator\Java.Interop.Tools.Generator.csproj", "{C2FD2F12-DE3B-4FB9-A0D3-FA3EF597DD04}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Java.Interop.Tools.Generator-Tests", "tests\Java.Interop.Tools.Generator-Tests\Java.Interop.Tools.Generator-Tests.csproj", "{7F4828AB-3908-458C-B09F-33C74A1368F9}"
EndProject
Global
GlobalSection(SharedMSBuildProjectFiles) = preSolution
src\Java.Interop.NamingCustomAttributes\Java.Interop.NamingCustomAttributes.projitems*{58b564a1-570d-4da2-b02d-25bddb1a9f4f}*SharedItemsImports = 5
Expand Down Expand Up @@ -239,6 +243,14 @@ Global
{0E3AF6C1-7638-464D-9174-485D494499DC}.Debug|Any CPU.Build.0 = Debug|Any CPU
{0E3AF6C1-7638-464D-9174-485D494499DC}.Release|Any CPU.ActiveCfg = Release|Any CPU
{0E3AF6C1-7638-464D-9174-485D494499DC}.Release|Any CPU.Build.0 = Release|Any CPU
{C2FD2F12-DE3B-4FB9-A0D3-FA3EF597DD04}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C2FD2F12-DE3B-4FB9-A0D3-FA3EF597DD04}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C2FD2F12-DE3B-4FB9-A0D3-FA3EF597DD04}.Release|Any CPU.ActiveCfg = Release|Any CPU
{C2FD2F12-DE3B-4FB9-A0D3-FA3EF597DD04}.Release|Any CPU.Build.0 = Release|Any CPU
{7F4828AB-3908-458C-B09F-33C74A1368F9}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{7F4828AB-3908-458C-B09F-33C74A1368F9}.Debug|Any CPU.Build.0 = Debug|Any CPU
{7F4828AB-3908-458C-B09F-33C74A1368F9}.Release|Any CPU.ActiveCfg = Release|Any CPU
{7F4828AB-3908-458C-B09F-33C74A1368F9}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -280,6 +292,8 @@ Global
{E34BCFA0-CAA4-412C-AA1C-75DB8D67D157} = {172B608B-E6F3-41CC-9949-203A76BA247C}
{093B5E94-7FB7-499F-9C11-30944BAFEE25} = {271C9F30-F679-4793-942B-0D9527CB3E2F}
{0E3AF6C1-7638-464D-9174-485D494499DC} = {C8F58966-94BF-407F-914A-8654F8B8AE3B}
{C2FD2F12-DE3B-4FB9-A0D3-FA3EF597DD04} = {0998E45F-8BCE-4791-A944-962CD54E2D80}
{7F4828AB-3908-458C-B09F-33C74A1368F9} = {271C9F30-F679-4793-942B-0D9527CB3E2F}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {29204E0C-382A-49A0-A814-AD7FBF9774A5}
Expand Down
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ TESTS = \
bin/Test$(CONFIGURATION)/logcat-parse-Tests.dll \
bin/Test$(CONFIGURATION)/generator-Tests.dll \
bin/Test$(CONFIGURATION)/Xamarin.Android.Tools.ApiXmlAdjuster-Tests.dll \
bin/Test$(CONFIGURATION)/Xamarin.Android.Tools.Bytecode-Tests.dll
bin/Test$(CONFIGURATION)/Xamarin.Android.Tools.Bytecode-Tests.dll \
bin/Test$(CONFIGURATION)/Java.Interop.Tools.Generator-Tests.dll

PTESTS = \
bin/Test$(CONFIGURATION)/Java.Interop-PerformanceTests.dll
Expand Down
9 changes: 8 additions & 1 deletion build-tools/automation/azure-pipelines.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ jobs:
inputs:
solution: build-tools/scripts/RunNUnitTests.targets
configuration: $(Build.Configuration)
msbuildArguments: /p:TestAssembly="bin\Test$(Build.Configuration)\generator-Tests.dll;bin\Test$(Build.Configuration)\Java.Interop.Tools.JavaCallableWrappers-Tests.dll;bin\Test$(Build.Configuration)\logcat-parse-Tests.dll;bin\Test$(Build.Configuration)\Xamarin.Android.Tools.ApiXmlAdjuster-Tests.dll;bin\Test$(Build.Configuration)\Xamarin.Android.Tools.Bytecode-Tests.dll"
msbuildArguments: /p:TestAssembly="bin\Test$(Build.Configuration)\generator-Tests.dll;bin\Test$(Build.Configuration)\Java.Interop.Tools.JavaCallableWrappers-Tests.dll;bin\Test$(Build.Configuration)\logcat-parse-Tests.dll;bin\Test$(Build.Configuration)\Xamarin.Android.Tools.ApiXmlAdjuster-Tests.dll;bin\Test$(Build.Configuration)\Xamarin.Android.Tools.Bytecode-Tests.dll;bin\Test$(Build.Configuration)\Java.Interop.Tools.Generator-Tests.dll"
condition: succeededOrFailed()

- task: PublishTestResults@2
Expand Down Expand Up @@ -95,6 +95,13 @@ jobs:
projects: Java.Interop.sln
arguments: '-c $(Build.Configuration)'

- task: DotNetCoreCLI@2
displayName: 'Tests: Java.Interop.Tools.Generator'
inputs:
command: test
arguments: bin\Test$(Build.Configuration)\Java.Interop.Tools.Generator-Tests.dll
continueOnError: true

- task: DotNetCoreCLI@2
displayName: 'Tests: generator'
inputs:
Expand Down
11 changes: 11 additions & 0 deletions src/Java.Interop.Tools.Generator/Enumification/ConstantAction.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
namespace Java.Interop.Tools.Generator.Enumification
{
public enum ConstantAction
{
None,
Ignore,
Enumify,
Add,
Remove
}
}
Loading

0 comments on commit b00e644

Please sign in to comment.