[android] android gradle project for ops

[IvanKobzarev]

PR introduces android gradle target that contains prebuilt torchvision ops library for android with android test application example how to use it.

The user workflow:

1. To add gradle dependency 'org.pytorch:torchvision_ops:0.0.1-SNAPSHOT'
2. Add loading of the library on java side:

    NativeLoader.loadLibrary("pytorch_jni");
    NativeLoader.loadLibrary("torchvision");

Content layout:

android/ops
android torchvision ops library setup, that uses nightly pytorch android dependencies with prebuilit libraries to link,
android/ops/CMakeLists.txt for linking libtorchvision.so

android/ops/CMakeLists.txt adds directive -DMOBILE which is used to strip python dependencies and backward ops that can be stripped from compilation for android.
Compilation includes only sources of ops, without image, video processing

   ../../torchvision/csrc/cpu/*.h
    ../../torchvision/csrc/cpu/*.cpp
    ../../torchvision/csrc/*.h
    ../../torchvision/csrc/*.cpp

CMakeLists.txt setup repeats the setup for android native apps that use prebuilt pytorch android binaries: https://pytorch.org/tutorials/recipes/android_native_app_with_custom_op.html

android/gradle_scripts
gradle scripts that repeat main pytorch setup in https://github.com/pytorch/pytorch/tree/master/android/gradle

android/test_app
example of android application that uses the result library of android/ops, contains several flavors to build application with dependency on local android/ops folder or with maven artifact 'org.pytorch:torchvision_ops:0.0.1-SNAPSHOT'

Testing

1. Generate faster rcnn pretrained model for test application using android/test_app/make_assets.py that will produce pt file in android/app/src/main/assets/frcnn.pt that will be packaged inside application

Connect android device that adb devices shows it

Install application that feeds camera output and draws the result bounding boxes on the screen;

cd $VISION_ROOT
gradle -p android test_app:installFrcnnCameraDebug

3. Run application on the device, the name is 'TV_FCRNN'

Publishing

1. Snapshot artifacts

edit android/gradle.properties to change VERSION_NAME=0.0.1-SNAPSHOT

gradle -p android ops:uploadArchives

uploads snapshot 'org.pytorch:torchvision_ops:0.0.1-SNAPSHOT' to sonatype repository, that can be added to the app:

...
repositories {
  maven {
    url "https://oss.sonatype.org/content/repositories/snapshots"
  }
}
...
dependencies {
	implementation 'org.pytorch:torchvision_ops:0.0.1-SNAPSHOT'
}

2. Release artifacts

gradle -p android ops:bintrayUpload

Will upload maven artifact to bintray that will be synced with jcenter (adding new artifact may need additional registration on bintray) that will be used in gradle as:

...
repositories {
  maven {
    jcenter()
  }
}
...
dependencies {
	implementation 'org.pytorch:torchvision_ops:0.0.1'
}

At the moment snapshot artifact 'org.pytorch:torchvision_ops:0.0.1-SNAPSHOT' is already published on sonatype repo: https://oss.sonatype.org/#nexus-search;quick~org.pytorch

https://oss.sonatype.org/service/local/repositories/snapshots/content/org/pytorch/torchvision_ops/

[ezyang]

In private communication, @fmassa reached out to me to see how we could avoid slapping mobile ifdefs everywhere, and was also interested if some of the restructuring in #2998 could help making this happen.

For the benefit of @fmassa and @datumbox, let me just describe some of the high level constraints that feed into the C++ library design process.

• On mobile, libraries with a lot of operators need a way to selectively build only a few operators (if there aren't too many operators, it's often OK to eat the cost of just always including everything; but as you increase the amount of operators selectivity is inevitable). What complicates this is that in fbcode, we want only one version of any given library, even though different apps have different operator requirements. So what ends up happening is we end up rebuilding the library multiple times per app, and we want to reduce the amount of recompilation work we have to do each time.
• As mentioned previously, on fbcode, we only want one version of a library. So that means CUDA support is most easily done by having separate libraries, rather than ifdef conditional for CUDA support. This doesn't apply to mobile though, as mobile build is entirely separate; ifdefs are OK. There's also some hypothetical payoff on the open source side, where hypothetically we could do a CUDA build, and then just delete the CUDA library, to get the CPU version (rather than having to build twice), although we haven't actually managed to cash in on this payoff.

I can tell you how PyTorch main library handles these things, and also mention why torchvision situation might be different.

• In PyTorch, we have a selective build macro https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/core/op_registration/op_whitelist.h used in https://github.com/pytorch/pytorch/blob/9aaf7fb39829e4e69a85b6a9e44a39131a1dd17f/torch/library.h#L341 which lets you easily control whether or not any given operator should be registered or not depending on the setting of a preprocessor macro. It's actually pretty similar in spirit to what Ivan has done here, but it's just more flexible since you can pick and choose which operators to include. Since torchvision is much smaller than torch, per-operator registration is not as pressing
• PyTorch library is split into CPU and CUDA libraries. In xplat, we also have an operator registration target as well. As of today, the operator registration target is what gets recompiled on a per app basis; but @iseeyuan has been working on making it so that the entire CPU/CUDA library can be recompiled per app, so we don't need a separate operator registration file.

OK, so what should you do here? TBH, the mobile ifdefs are probably a fine stopgap for now. The important thing is knowing that these don't stop you from moving to a more flexible solution later; the ifdefs are pretty mild and easily to remove later. If y'all in torchvision are planning to add a lot of operators in the near future, this matter is more pressing, but this is OK for bootstrapping.

[iseeyuan]

In private communication, @fmassa reached out to me to see how we could avoid slapping mobile ifdefs everywhere, and was also interested if some of the restructuring in #2998 could help making this happen.

For the benefit of @fmassa and @datumbox, let me just describe some of the high level constraints that feed into the C++ library design process.

• On mobile, libraries with a lot of operators need a way to selectively build only a few operators (if there aren't too many operators, it's often OK to eat the cost of just always including everything; but as you increase the amount of operators selectivity is inevitable). What complicates this is that in fbcode, we want only one version of any given library, even though different apps have different operator requirements. So what ends up happening is we end up rebuilding the library multiple times per app, and we want to reduce the amount of recompilation work we have to do each time.
• As mentioned previously, on fbcode, we only want one version of a library. So that means CUDA support is most easily done by having separate libraries, rather than ifdef conditional for CUDA support. This doesn't apply to mobile though, as mobile build is entirely separate; ifdefs are OK. There's also some hypothetical payoff on the open source side, where hypothetically we could do a CUDA build, and then just delete the CUDA library, to get the CPU version (rather than having to build twice), although we haven't actually managed to cash in on this payoff.

I can tell you how PyTorch main library handles these things, and also mention why torchvision situation might be different.

• In PyTorch, we have a selective build macro https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/core/op_registration/op_whitelist.h used in https://github.com/pytorch/pytorch/blob/9aaf7fb39829e4e69a85b6a9e44a39131a1dd17f/torch/library.h#L341 which lets you easily control whether or not any given operator should be registered or not depending on the setting of a preprocessor macro. It's actually pretty similar in spirit to what Ivan has done here, but it's just more flexible since you can pick and choose which operators to include. Since torchvision is much smaller than torch, per-operator registration is not as pressing
• PyTorch library is split into CPU and CUDA libraries. In xplat, we also have an operator registration target as well. As of today, the operator registration target is what gets recompiled on a per app basis; but @iseeyuan has been working on making it so that the entire CPU/CUDA library can be recompiled per app, so we don't need a separate operator registration file.

OK, so what should you do here? TBH, the mobile ifdefs are probably a fine stopgap for now. The important thing is knowing that these don't stop you from moving to a more flexible solution later; the ifdefs are pretty mild and easily to remove later. If y'all in torchvision are planning to add a lot of operators in the near future, this matter is more pressing, but this is OK for bootstrapping.

Awesome summary! @dreiss also brought up the topic of using selective build to replace those macros. As discussed offline, it might be similar to maskrcnn ops that we've already done. @fmassa please let us know if you want to have more details on per-app selective build!

[IvanKobzarev]

@ezyang , Thanks a lot for your detailed answer. I read the way how Selective build works.

My main intention to use ifdefs (apart excluding dependency on Python) was to save some binary size, not including backward ops.

I checked with and without it - the difference is only 60K. I think, for the first iteration, that amount of savings does not worth to have ifdefs in so many places and have a big divergence desktop-mobile :) I have updated PR removing them.

@fmassa Does torchvision plan to add a lot more new ops that size can grow significantly and there can be the case that we do not want to include all of them to mobile?

In that case we can repeat torch selective build approach and use fixed list of ops for torchvision mobile.

[codecov]

Codecov Report

Merging #2897 (f1ede53) into master (bf211da) will decrease coverage by 1.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2897      +/-   ##
==========================================
- Coverage   73.78%   72.75%   -1.03%     
==========================================
  Files         103       99       -4     
  Lines        9507     8979     -528     
  Branches     1524     1431      -93     
==========================================
- Hits         7015     6533     -482     
+ Misses       2019     1999      -20     
+ Partials      473      447      -26     

Impacted Files	Coverage Δ
torchvision/models/mobilenet.py	83.72% <0.00%> (-16.28%)	⬇️
torchvision/models/quantization/mobilenet.py	87.75% <0.00%> (-12.25%)	⬇️
torchvision/transforms/functional_tensor.py	73.49% <0.00%> (-5.56%)	⬇️
torchvision/ops/_register_onnx_ops.py	48.78% <0.00%> (-4.35%)	⬇️
torchvision/datasets/utils.py	64.63% <0.00%> (-3.00%)	⬇️
torchvision/datasets/ucf101.py	89.58% <0.00%> (-1.91%)	⬇️
torchvision/transforms/transforms.py	82.07% <0.00%> (-1.62%)	⬇️
torchvision/datasets/video_utils.py	68.22% <0.00%> (-1.05%)	⬇️
torchvision/transforms/functional.py	80.83% <0.00%> (-0.65%)	⬇️
torchvision/datasets/folder.py	85.54% <0.00%> (-0.51%)	⬇️
... and 31 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bf211da...1e2e5e5. Read the comment docs.

[datumbox]

FYI there are currently discussions for changing the structure of the library. If we do that, we will need to change this part.

[datumbox]

@IvanKobzarev We changed the structure. Have a look on the latest master.

[fmassa]

I have one question, otherwise looks great to me, thanks a lot @IvanKobzarev !

[fmassa]

do we need to change this file to match frcnn_mnetv3.pt?

[datumbox]

@IvanKobzarev @fmassa awesome work, great to see this merged!

[datumbox]

@IvanKobzarev @fmassa Why do we use two duration vars initialized and estimated on exactly the same point? This seems unnecessary. Perhaps the intention was for moduleForwardDuration to estimate the duration of the mModule.forward() call? Or perhaps the intention of analysisDuration is to measure just the unpacking of the response?