Make Keras 2 available for TensorFlow 2.16.1

[ethanbb]

Description

This provides a way to continue using Keras 2 with TensorFlow 2.16.1 (see #1355) by following the instructions here: https://github.com/tensorflow/tensorflow/releases/tag/v2.16.1 While sticking with 2.15.0 should still work for most people, I had an annoying CUDA situation which was only fixed by upgrading, so I wanted to see if I could make 2.16 work.

It's somewhat hacky (especially in nn_models.py), but since the long-term goal is to move away from TensorFlow anyway, I thought that was OK. No worries if you don't want to merge this, though.

It requires a matching version of tf-keras to be installed alongside tensorflow, but this package is only available for 2.15 and 2.16, so it won't work on Windows where tensorflow 2.10 is the maximum. So there would have to be some OS-dependent logic in the environment specification; I didn't touch any of that and I think it might be only possible for the conda recipe (i.e. not in environment.yml).

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)

Has your PR been tested?

Ran caimanmanager test and caimanmanager demotest.

[pgunn]

We have someone working this summer on swapping from tensorflow to pytorch; for now I'm going to hold on merging this, but there's a chance we may need it so let's leave it open as a PR until we know if it's needed. Thanks for putting in the time to find a solution.

[hmaarrfk]

hmm i hit this issue when trying to enabled Python 3.12 at conda-forge. Just wanted to resurface this.

[hmaarrfk]

I went an other way with my attempt, but it failed at the tests.

From cbcf7bbe395c3f0b4ef237c1d799e4739a1f922b Mon Sep 17 00:00:00 2001
From: Mark Harfouche <mark.harfouche@gmail.com>
Date: Sun, 15 Sep 2024 19:57:20 -0400
Subject: [PATCH 4/4] Use tf_keras instead of tensorflow.keras

See release notes from: https://github.com/tensorflow/tensorflow/releases/tag/v2.16.1
---
 caiman/components_evaluation.py               |  2 +-
 caiman/source_extraction/cnmf/online_cnmf.py  |  4 ++--
 caiman/source_extraction/volpy/mrcnn/model.py | 12 +++++------
 caiman/tests/test_tensorflow.py               |  3 ++-
 caiman/utils/nn_models.py                     | 20 +++++++++----------
 5 files changed, 21 insertions(+), 20 deletions(-)

diff --git a/caiman/components_evaluation.py b/caiman/components_evaluation.py
index 53315d02..ba8e8311 100644
--- a/caiman/components_evaluation.py
+++ b/caiman/components_evaluation.py
@@ -275,7 +275,7 @@ def evaluate_components_CNN(A,
     try:
         import tensorflow as tf
         os.environ["KERAS_BACKEND"] = "tensorflow"
-        from tensorflow.keras.models import model_from_json
+        from tf_keras.models import model_from_json
         use_keras = True
         logger.info('Using Keras')
     except (ModuleNotFoundError):
diff --git a/caiman/source_extraction/cnmf/online_cnmf.py b/caiman/source_extraction/cnmf/online_cnmf.py
index 4ed17662..9d6cf82f 100644
--- a/caiman/source_extraction/cnmf/online_cnmf.py
+++ b/caiman/source_extraction/cnmf/online_cnmf.py
@@ -268,7 +268,7 @@ class OnACID(object):

         if self.is1p:
             estim = self.estimates
-            d1, d2 = estim.dims
+            d1, d2 = estim.dims
             estim.Yres_buf -= estim.b0
             if ssub_B == 1:
                 estim.Atb = estim.Ab.T.dot(estim.W.dot(estim.b0) - estim.b0)
@@ -323,7 +323,7 @@ class OnACID(object):
             self.tf_out = None
         else:
             try:
-                from tensorflow.keras.models import model_from_json
+                from tf_keras.models import model_from_json
                 logger.info('Using Keras')
                 use_keras = True
             except(ModuleNotFoundError):
diff --git a/caiman/source_extraction/volpy/mrcnn/model.py b/caiman/source_extraction/volpy/mrcnn/model.py
index c3f95ed7..4fae4cfe 100644
--- a/caiman/source_extraction/volpy/mrcnn/model.py
+++ b/caiman/source_extraction/volpy/mrcnn/model.py
@@ -15,13 +15,13 @@ from collections import OrderedDict
 import multiprocessing
 import numpy as np
 import tensorflow as tf
-import tensorflow.keras as keras
-import tensorflow.keras.backend as K
-import tensorflow.keras.layers as KL
-import tensorflow.keras.layers as KE
-import tensorflow.keras.utils as KU
+import tf_keras as keras
+import tf_keras.backend as K
+import tf_keras.layers as KL
+import tf_keras.layers as KE
+import tf_keras.utils as KU
 from tensorflow.python.eager import context
-import tensorflow.keras.models as KM
+import tf_keras.models as KM

 from ..mrcnn import utils

diff --git a/caiman/tests/test_tensorflow.py b/caiman/tests/test_tensorflow.py
index f4e022c7..8589f5c1 100644
--- a/caiman/tests/test_tensorflow.py
+++ b/caiman/tests/test_tensorflow.py
@@ -12,7 +12,8 @@ from caiman.utils.utils import load_graph

 try:
     os.environ["KERAS_BACKEND"] = "tensorflow"
-    from tensorflow.keras.models import model_from_json
+    import tensorflow as tf
+    from tf_keras.models import model_from_json
     use_keras = True
 except(ModuleNotFoundError):
     import tensorflow as tf
diff --git a/caiman/utils/nn_models.py b/caiman/utils/nn_models.py
index dcf5eff4..683d0e79 100644
--- a/caiman/utils/nn_models.py
+++ b/caiman/utils/nn_models.py
@@ -7,13 +7,13 @@ one photon data using a "ring-CNN" background model.

 import numpy as np
 import os
-from tensorflow.keras.layers import Input, Dense, Reshape, Layer, Activation
-from tensorflow.keras.models import Model
-from tensorflow.keras.optimizers import Adam
-from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, LearningRateScheduler
-from tensorflow.keras import backend as K
-from tensorflow.keras.initializers import Constant, RandomUniform
-from tensorflow.keras.utils import Sequence
+from tf_keras.layers import Input, Dense, Reshape, Layer, Activation
+from tf_keras.models import Model
+from tf_keras.optimizers import Adam
+from tf_keras.callbacks import ModelCheckpoint, EarlyStopping, LearningRateScheduler
+from tf_keras import backend as K
+from tf_keras.initializers import Constant, RandomUniform
+from tf_keras.utils import Sequence
 import time

 import caiman.base.movies
@@ -93,7 +93,7 @@ class Masked_Conv2D(Layer):
             add a bias term to each convolution kernel

     Returns:
-        Masked_Conv2D: tensorflow.keras.layer
+        Masked_Conv2D: tf_keras.layer
             A trainable layer implementing the convolution with a ring
     """
     def __init__(self, output_dim=1, kernel_size=(5,5), strides=(1,1),
@@ -202,7 +202,7 @@ def masked_constraint(R):


 class Hadamard(Layer):
-    """ Creates a tensorflow.keras multiplicative layer that performs
+    """ Creates a tf_keras multiplicative layer that performs
     pointwise multiplication with a set of learnable weights.

     Args:
@@ -231,7 +231,7 @@ class Hadamard(Layer):


 class Additive(Layer):
-    """ Creates a tensorflow.keras additive layer that performs
+    """ Creates a tf_keras additive layer that performs
     pointwise addition with a set of learnable weights.

     Args:
--
2.46.0

[ethanbb]

@hmaarrfk this approach should work if you have tensorflow 2.15 or newer installed, I believe. You have to install the corresponding version of tf-keras (conda should find the right version automatically).

The reason I did it this way is that tf-keras is only available for 2.15 or newer, and we still need to support older versions of tensorflow (Windows is stuck at 2.10).

[pgunn]

Manuel's efforts to port all of caiman's tensorflow code to pytorch are not that far off from being ready to land, although this may pose some (different) packaging problems for Windows; apparently conda-forge doesn't have any Windows builds of pytorch.

These GPU acceleration libraries are no end of trouble for cross-platform projects. Trying to figure out how to navigate all of this.