Fix Bidirectional Regularization #10012
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Override the Wrapper `get_updates_for` and `get_losses_for` methods so that contributions from both the forward and backward layers are included in Bidirectional.
fchollet
reviewed
Apr 23, 2018
| @@ -487,12 +487,33 @@ def updates(self): | |||
| return self.forward_layer.updates + self.backward_layer.updates | |||
| return [] | |||
|
|
|||
| def get_updates_for(self, inputs=None): | |||
There was a problem hiding this comment.
Why not
forward_updates = self.forward_layer.get_updates_for(inputs)
backward_updates = self.backward_layer.get_updates_for(inputs)
return super(Wrapper, self).get_updates_for(inputs) + forward_updates + backward_updatesIndeed, the inner layers are called with the parent's inputs argument directly.
Remove unneeded inner input calculations.
fchollet
reviewed
Apr 23, 2018
| @property | ||
| def losses(self): | ||
| if hasattr(self.forward_layer, 'losses'): | ||
| return self.forward_layer.losses + self.backward_layer.losses | ||
| return [] | ||
|
|
||
| def get_losses_for(self, inputs=None): |
There was a problem hiding this comment.
This one should follow the same logic as above, as well.
added 3 commits
April 23, 2018 16:23
Test Bidirectional updates and losses.
fchollet
reviewed
Apr 24, 2018
keras/layers/wrappers.py
Outdated
| @property | ||
| def losses(self): | ||
| if hasattr(self.forward_layer, 'losses'): | ||
| return self.forward_layer.losses + self.backward_layer.losses | ||
| return [] | ||
|
|
||
| def get_losses_for(self, inputs=None): | ||
| forward_losses = self.forward_layer.get_losses_for(None) |
There was a problem hiding this comment.
That should be inputs, not None (same below).
| layer.backward_layer.add_update(0, inputs=x) | ||
| layer.backward_layer.add_update(1, inputs=None) | ||
| assert len(layer.updates) == 4 | ||
| assert len(layer.get_updates_for(None)) == 2 |
There was a problem hiding this comment.
This test and the one above should also cover the case of counting conditional updates/losses.
dschwertfeger
added a commit
to dschwertfeger/keras
that referenced
this pull request
Apr 25, 2018
…ack-embeddings-from-layer-outputs * upstream/master: (34 commits) Refactor training part of `engine` module. (keras-team#10029) Fix Bidirectional Regularization (keras-team#10012) Refactor topological part of `engine` module (keras-team#10023) Add `separable_conv2d` for Theano (keras-team#10003) fixing typos (keras-team#10016) Prepare 2.1.6 release. Add tutorial link in docs. Sync docstring of preprocess_input with _preprocess_numpy_input and _preprocess_symbolic_input in imagenet_utils.py (keras-team#10006) Immigrate reference operations to a separate module (keras-team#9948) Chenta/cntk bn (keras-team#9952) Make conv_invalid_use and pooling_invalid_use efficient (keras-team#9944) Exclude multi-gpu utils when reporting coverages (keras-team#9942) Allow shift_range to be 1-D array-like or int (keras-team#8869) Fix PEP8 Added an error message for undefined shape on NASNet. (keras-team#9891) fix TimeSeriesGenerator glitch (keras-team#9899) Updated for TF 1.7 (keras-team#9937) Add support for `constants` in Bidirectional wrapper (keras-team#9260) Added note to manually install h5py where needed (keras-team#9830) Fix stateful metrics when passing dict to compile (keras-team#9894) ...
Vijayabhaskar96
pushed a commit
to Vijayabhaskar96/keras
that referenced
this pull request
May 3, 2018
* Fix Bidirectional Regularization Override the Wrapper `get_updates_for` and `get_losses_for` methods so that contributions from both the forward and backward layers are included in Bidirectional. * Use Parent for Calculating Inner Inputs Remove unneeded inner input calculations. * Simplify Bidirectional Losses * Add Bidirectional Unit Tests Test Bidirectional updates and losses. * Remove Trailing Whitespace * Fix Bidirectional Loss Inputs * Add Tests for Conditional Updates/Losses * Remove Whitespace
4 tasks
sandeep-krishnamurthy
pushed a commit
to awslabs/keras-apache-mxnet
that referenced
this pull request
Jun 15, 2018
* Fix Bidirectional Regularization Override the Wrapper `get_updates_for` and `get_losses_for` methods so that contributions from both the forward and backward layers are included in Bidirectional. * Use Parent for Calculating Inner Inputs Remove unneeded inner input calculations. * Simplify Bidirectional Losses * Add Bidirectional Unit Tests Test Bidirectional updates and losses. * Remove Trailing Whitespace * Fix Bidirectional Loss Inputs * Add Tests for Conditional Updates/Losses * Remove Whitespace
sandeep-krishnamurthy
added a commit
to awslabs/keras-apache-mxnet
that referenced
this pull request
Jun 21, 2018
* Document that "same" is inconsistent across backends with strides!=1 (#9629) * Document that `"same"` is inconsistent across backends with `strides` != 1 * Use "[here](...)" * Add h5py to dependencies * import `pydot`, improve error messages about `pydot` and GraphViz, bump to `pydot >= 1.2.4` (#9904) * REL: bump to `pydot >= 1.2.4` in `extras_require` * MAI: import pydot (as required in `extras_require`) * MAI: refine error messages for `pydot` and GraphViz distinguish between absence of `pydot` and failure to find the executables of GraphViz in the $PATH. * DEV: ignore `.pytest_cache` * Added note to manually install h5py where needed (#9830) * Added notes to manually install h5py if needed * Added FAQ entry on h5py * deleted redundant remark about h5py * updated FAQ to reflect dependency change * fixed comment format to pass failing test * removed new trailing whitespaces * improved docstring format * reverted callbacks.py * fixed links in model.py * updated faq.py * link pointing to FAQ * Added an error message for undefined shape on NASNet. (#9891) * Added an error message for undefined shape on NASNet. * Forgot that the message should be present only when loading imagenet weights. * Changed the message. * Fix PEP8 * Make conv_invalid_use and pooling_invalid_use efficient (#9944) * Chenta/cntk bn (#9952) * fix cntk static learning phase issue; add a test * fix code style;add more comments * add boolean support * fix code style issue * fixing typos (#10016) * Add `separable_conv2d` for Theano (#10003) * Add `separable_conv2d` for Theano * Enable the layer test for `separable_conv2d` * Fix the strides for 1x1 conv * Refactor topological part of `engine` module (#10023) * Refactor topological part of Keras engine. * Fix imports * Fix merge mixup. * Fix Bidirectional Regularization (#10012) * Fix Bidirectional Regularization Override the Wrapper `get_updates_for` and `get_losses_for` methods so that contributions from both the forward and backward layers are included in Bidirectional. * Use Parent for Calculating Inner Inputs Remove unneeded inner input calculations. * Simplify Bidirectional Losses * Add Bidirectional Unit Tests Test Bidirectional updates and losses. * Remove Trailing Whitespace * Fix Bidirectional Loss Inputs * Add Tests for Conditional Updates/Losses * Remove Whitespace * Refactor training part of `engine` module. (#10029) * Refactor topological part of Keras engine. * Fix imports * Fix merge mixup. * Refactor training part of the Keras engine. * Fix unit tests. * Fix `int_shape` of Theano and Refactor associated lines (#10030) * Add exceptions for `batch_dot` (#10020) * Enable Xception to work on Theano and CNTK (#10024) * Enable Xception to work on Theano and CNTK * Fix different predictions over all the backends * Add support for passthrough arguments to NumpyArrayIterator (#10035) * Add support for second output to NumpyArrayIterator * Enable Model subclassing API (#10046) * Refactor topological part of Keras engine. * Fix imports * Fix merge mixup. * Refactor training part of the Keras engine. * Fix unit tests. * Refactor Network to prepare support for Model subclassing * Finish enabling Model subclassing. * Add `long_description` field in setup.py. * Remove unused import * Increase pytest duration from 10 mins to 20 mins (#10072) * [RELNOTES] Simplify implementation of Sequential and remove legacy Merge support (#10077) * Refactor topological part of Keras engine. * Fix imports * Fix merge mixup. * Refactor training part of the Keras engine. * Fix unit tests. * Refactor Network to prepare support for Model subclassing * Finish enabling Model subclassing. * Simplify Sequential implementation. RELNOTES: This breaks weight loading and model loading/saving for models from Keras 0.* that used Merge layers. The Merge layer has been deprecated for 2 years. * [RELNOTES] Remove support for Keras 0.* Merge layer and associated functionality, which was scheduled for 08/2017. * fix typo (#10078) * Add documentation to several activation functions (#10066) * Add documentation to several activation functions * Fix style issues. * Small style fixes. * [RELNOTES] Introduce `preprocessing.image.save_img` and remove deprecated imsave method in neural style transfer example (#9996) * imsave method in scipy.misc package is deprecated - now using imageio * updated save_img method to use array_to_img method. also updated the neural style transfer example to use the new save_img method * forgot to commit changes - updated save_img * added file_format and **kwargs parameter to save_img and updated docstring * removed space at the end of a line in save_img method. updated instances of imsave in example scripts with save_img method. * added kwargs to docstring. removed additional whitespace lines * removed trailing whitespace * Extensive style fixes in `preprocessing.image`. * [RELNOTES] Allow loading external backends (#10034) * load external backend * safety++ * Excpetion -> ValueError * Revert TF version to 1.7 on Travis CI. (#10101) * New Callback: EarlyBaselineStopping (#10061) * Initial Commit * Continued changes * Alpha version * Added support to make sure that previous epochs, which may pass the baseline test, are kept in variable history. * Code formatting * Code formatting * Code formatting to address CI errors * Code formatting for CI errors * Initial unit test code * Adjust for epoch being zero-based * Code formatting * Unit tests added * Code formatting * Code formatting * Factorized the unit test * Code formatting * Refactored to be part of EarlyStopping and modified unit tests * Code formatting * Adds MobiletNetV2 to applications (#10047) * [RELNOTES] Allow symbolic tensors to be fed to models (with TF backend) (#10087) * Allow symbolic tensors to be fed to models (with TF backend) * Skip tensor feeding tests when TF>=1.8 not available. * [RELNOTES] Fix EarlyStopping API * Fix shape mismatch in `rnn()` of tensorflow_backend (#10038) * Fix shape mismatch in `rnn()` of tensorflow_backend Fix the shape mismatch in `rnn()` with `not unroll` and `mask is not None` of the Tensorflow backend. Problem: If the rnn cell has any (recurrent) `states[i]` whose shape is different from that of the `output` (`states[0]`), there will raise an `ValueError` when updating that state. Reason: This is because the `tiled_mask_t` is not updated correctly for each `states[i]` but instead is simply copied from that of the `output` (`states[0]`). Solution: Tile the `mask_t` with the correct shape of each `states[i]`. Notice that in a similar situation with `unroll is True`, the `tiled_mask_t` is handled correctly. * add unit_test for rnn() with states whose shape is different from that of the output. * Revert "add unit_test for rnn() with states whose shape is different from that of the output." This reverts commit f1df2a58ff635bbf698444e3d7403785a92dfed1. * Simplify the unit_test for rnn with additional states * [RELNOTES] Default `Flatten` layer’s `data_format` argument to `None`. * #10080 Convert CuDNN weights in nested Model. (#10081) * #10080 Convert CuDNN weights in nested Model. - similar problem to nesting in Bidirectional (#8860) - quick fix: just copy/paste. I'll refactor it later - also convert biases from H5 Dataset to np.array for reshape() (#9662) * Refactor keras.engine.saving.preprocess_weights_for_loading(). - less duplication - extracted conversion methods for Bidirectional/Model/Sequential * Format docstrings to follow the project code style. * Move tests of CuDNN RNN weight conversion to a more proper place. - move from cudnn_recurrent_test.py to test_model_saving.py - adjust imports to be consistent * Make better tests for CuDNN RNN weight conversion. - add nested models (#10080) - various model types: add Model, Sequential (not only Sequential) - convert weights on model, not layer (not test nesting in models) - test just by save/load model weights instead of calling preprocess_weights_for_loading() directly * Check GPU support via tf in pytest skipif with short-circuit evaluation. It seems that multiple skipif decorators get evaluated eagerly, not lazily. Thus it fails on theano and cntk backends. I don't understand why the tests didn't fail until now. Maybe some change in pytest? * Refactor (deduplicate) skipif decorators for TensorFlow and GPU. At least multiple occurrences within a test module. Now we can't import from `tests/` and `keras.utils.test_utils` don't se pytest. Otherwise we can define the marker only once at all. * Fix PEP8 (whitespace). * Add `separable_conv1d` (#10125) * Add `separable_conv1d` for Theano * Add `separable_conv1d` for CNTK * Fix preprocess_input not working with int arrays (#10134) * Make MobileNet tests random (#10132) * Sample weighted ImageDataGenerator (#10092) * Add support for sample_weight in ImageDataGenerator.flow * Added test for sample weighted imagedatagen * clarified docs + PEP8 * pep8 - blank line * sample_weight argument after shuffle * RNN docstring fixes; pass `maximum_iteration` argument to `while_loop` in TF. * Use `embedding_lookup` for `gather` in TF backend (enables partitioned variables). * Add `data_format` argument to Conv1D; docstring style fixes. * Further fixes. * Fix initializers (#9963) * fix VarianceScaling * Correct standard initializer testing * Add test for truncated normal in backend * Use correct theano function for truncated_normal * Fix sloppiness in correction * fix PEP * improve style initialiser fix * fix line length * Use NCW/NWC for conv1d data format in TF backend. * Model loading: do not return inside `try` block. * Rely on `Flatten` layer to do data_format conversion in ResNet50, VGG16, VGG19. * Remove potentially misleading warning messages in `applications`. * Fix typo. * Remove useless super delegation. * Remove batch_size when doing stepwise fit (#10144) `batch_size` is always `None` , because `steps_per_epoch` and `batch_size` are mutually exclusive. Passing batch_size seems irrelevant as `None` is its default value in `test_loop` * Add exceptions for `fit_loop` (#10145) * Style fixes (#10148) * Fix ConvLSTM2D unit_forget_bias initializer * Avoid a warning by using 'x = x * y' instead of 'x *= y' (#10158) * Fix undefined behaviour: preprocess_input copying/not copying the input arrays (#10153) * Add copy option for image preprocessing * Fix unnecessary import * Fix style * fix test error * Make modifications in-place instead * vae examples fixes (#10062) * vae examples fixes * white spaces around op * comments and docstring fixes * comments and docstring fixes * comments and docstring fixes * fixes on docs * docs and spacing * Merge 2 functions together in applications MobileNetV2 (#10163) * merge 2 functions together _inverted_res_block() and _first_inverted_res_block() are nearly the same. Merged them into one function 1. skip "Expand" part for 0 block 2. made layers names similar to original TF graph names * added "mobl_" to prefix, pep8 fix * remove dropout parameter * concise prefix name * Remove named argument from schedule function (#10178) * Add missing doc (#10188) * Best to set self.built=True at the end of build() (#10191) When using Keras on TensorFlow (in graph mode), via `tf.keras`, then calling `self.add_weight()` fails if `self.built==True`, so it is best to encourage users to set `self.built=True` at the *end* of their `build()` method, rather than just "somewhere". * Clean up preprocessing of `depthwise_kernel` for Theano (#10189) * Allow dynamic backends in _need_convert_kernel (#10111) * Fix _need_convert_kernel for external backends * Don't assume external backend won't support NASNet * Update from review comments * fix TensorBoard callback with unit test (#10173) * Allow project specific config files in Keras (#10183) * Allow project specific config files in Keras * Clearer code, updated comment * Update cifar10_resnet.py (#10199) change the comments on line 133. * Adds height_shift_range to preprocessing.image doc and adds support for indentation in auto-generated doc (#10194) * Update README.me * Mobilenetv2 explanation (#10174) * i add mobilenetv2 to the table * dd explanation for mobilenetv2 * add explanation for mibilenetv2 * Make multi_gpu_model serializable. * Skip tests for multi gpu * Fix slice namespace collision in backend. * Skip multi_gpu tests for other backends than TF * Replace np.ceil() with a faster operation (#10184) * Replace ceil() with faster a operation Replacing np.ceil() with faster operations, as suggested in #8488 * Revert "Replace ceil() with faster a operation" This reverts commit 74fe60b445377b4f1be1d0a36b4777d7ed4f1c1b. * Replace np.ceil() with a faster operation * Fix syntax error * Remove trailing whitespace * Explicitly mention using a Sequence object in val (#10223) * Grammatical error - "inputs channels" rather than "input". (#10217) * Fix duplicated argname: num_gpus === parts (#10228) Signed-off-by: CUI Wei <ghostplant@qq.com> * Fix NASNet (#10209) * Fix NASNet * Update weight files * In-place split to avoid inter-device duplication (#10230) New Benchmark by in-place split: >> keras.application.Resnet50 224x224x3 (NCWH; NVidia Tesla P100 x 4) input_shape = 3x224x224, batch_size = 96 x 4: 392(images/sec) => 417(images/sec) input_shape = 3x299x299, batch_size = 64 x 4: 229(images/sec) => 244(images/sec) input_shape = 3x224x224, batch_size = 8 x 4: 148(images/sec) => 163(images/sec) >> keras.application.InceptionV3 (NCWH; NVidia Tesla P100 x 4) input_shape = 3x224x224, batch_size = 128 x 4: 488(images/sec) => 526(images/sec) input_shape = 3x299x299, batch_size = 96 x 4: 270(images/sec) => 294(images/sec) input_shape = 3x224x224, batch_size = 8 x 4: 146(images/sec) => 158(images/sec) Signed-off-by: CUI Wei <ghostplant@qq.com> * Increase test coverages by adding invalid CNTK usecases (#10236) * Remove Sequential.model deprecation warning (#10256) * Remove Sequential.model deprecation warning * Remove dead line of code * Increase test coverages by factorizing CNTK pads (#10259) * Refactor ImageDataGenerator (#10130) * Create get_random_transform and refactor * Fix style and add tests * Add more tests * Fix documentation error * Fix documentation style issue * add apply_affine_transform * document transformation dictionary * Doc style fix * Remove deprecated model.model from engine/saving (#10275) * Typo in docstring for softplus (#10277) Softplus docstring missing a parenthesis. * Make Dot documentation inline with Concatenate (#10271) Doc expects a list containing 2 tensors. * Fixes automatic doc generation problem with nested lists. Adds a new test (#10212) * Fixes automatic doc generation problem with indented lists. Adds a new test * Some style fixes on doc automatic generation files * Fixes a bad space in convolutional_recurrent.py * Changes the test_doc_auto_generation in order to include a doc string taken from the codebase. Allows text lines following nested lists * Use count_params function for non_trainable_count. (#10280) * load_weights will fail if shape mismatch (#10266) Fix for #10265 * Adds to and alphabetizes documentation of Layer base class. (#10282) * Alphabetizes and adds to layers doc. * Responding to @cais comments * fix spacing. Remove in(out)bound_nodes * Non training Batch Norm operator has bad performance for it running into tensorflow's non fused batch norm API (#10207) * When use tensorflow as backend, let batch norm run into fused batch norm as much as possible, which has better performance. fix issue: http://github.com/keras-team/keras/issues/10058 * In Tensorflow backend, let batch norm call to FusedBatchNorm only NHWC format, also gamma and beta are not None. Test result: test env: with Tensorflow(commit a543d9471047ca3f6881c87105fcbe2cdff9207d Date: Thu May 10 17:43:30 2018, local build), python3.4, centos7.4 test cases: "pytest ./tests/keras/layers/normalization_test.py" <all passed> "pytest ./tests" <keep same result as without this commit's modification on BN> * fix code sytle. * 1. Add axis parameter in backend's batch_normalization functions. 2. Refine the batch_normalization function in tensorflow backend, Let's it call to fused batch norm as much as possible. Thanks the coments from fchollet. * Trigger * 1. add default value -1 for parameter axis in batch_normalization function in backend. 2. fix some code style. Thanks the comments from fchollet. * Handle capitalised extensions in list_pictures (#10220) #10219 * Typo fix (#10293) * Fix doc (#10308) * Fix naming convention * Add missing doc * Fix typo * Improve docstrings of applications (#10310) * Add pooling options in MobileNetV2 (#10313) * Add pooling option * Add pooling test * Fix doc (#10327) Fixed doc * Handle `mask` in `TimeDistributed` wrapper. (#10242) * equip TimeDistributed with mask and unspecified input length * fix bugs in theano. add test on timedistributed + masking * skip tests on cntk with multiple unspecified time lengths. * move static shape inference to theano_backend, add docstring, etc. * fix format * Split `applications` and `preprocessing` modules. (#10339) * Split `applications` and `preprocessing` modules. * Fix dependencies. * Move tests for applications (#10341) * Improve the docstring of Conv3DTranspose (#10342) * Add depth as third dimension in docstring of Conv3DTranspose in convolutional.py in keras.layers * Reduce tests for applications (#10346) * Reduce tests for applications * Make selection over all models random * Add an advanced activation layer for ReLU (#10322) The max_value argument can not be used in a layer, except custom layer or Lambda. Hence, similarly to LeakyReLU or for example Softmax, this PR adds a layer for ReLU, enabling also a capped ReLU to be used. * FIX: Tensorboard callback only supports logging Embeddings layer weights (#7766) * Embed layer-outputs rather than layer-weights in TensorBoard callback * Update docstring and allow multiple inputs * Fix tests * Renaming * Set learning phase * Compute embeddings in batches * Pass embedding data explicitly * Actually process embeddings in batches * Allow multiple inputs and validate input data * Add example * Delete utils.py * Revert uncorrectly resolved merge-conflict * Minor renaming * Add comment clarifying the design choice * Fix HDF5Matrix issue when working in conjunction with TimeSeriesGenerator (#10334) * Fix issue when working in conjunction with TimeSeriesGenerator The TimeSeriesGenerator class uses xrange through six which caused an IndexError * Add test * Add corresponding test * Fix for python3 * Simplified code * Fix indent * Fix test * Supporting channels_first data format with crossentropy losses (#9715) * Add error message when calling `summary` on unbuilt subclassed models. * Prepare 2.2.0 release. * Fix a version number (#10361) * Update to Keras Applications 1.0.2 (fixes NASNet issue). * Add tests for inputs set dynamically (#10367) * CuDNN RNN layers nested in TimeDistributed are not converted when loading (#10357) * Add a unit test for CuDNNGRU conversion with TimeDistributed. * Extract duplicated function convert_model() to _convert_model_weights(). * #10356 Convert weights of CuDNN/plain RNN nested in TimeDistributed. Same case as for Bidirectional, except that in TimeDistributed there's only one nested layer instead of two. * Style fix * Update docs for 2.2.0. * Add spatial dropout and 3D global pooling to docs (#10373) * spatial dropout in docs * 3d global pooling in docs * Doc update (#10376) A couple of variables are "used" in two examples without being defined. For consistency with other examples where auxiliary dimensions are defined, I think it would be better to explicitly assign them a value. I just used made up values, feel free to change to whatever makes more sense! * Preserve input shape data when serializing deferred-build Sequential models. * Add MXNet Backend (#59) * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - roywei@ * Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance * Conv3d() operator implementation for Keras2.0 using MXNet backend (#40) * conv3d implementation for keras2.0 as MXNet backend * conv3d implementation/testing for keras2.0 using MXNet backend * keeping -n option in pytest.ini file * fixed comments given by Sandeep * Add Conv1D support for MXNet backend (#44) * Add Conv1D support for MXNet backend * Fix CR comments * Conv2d transpose (#47) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * enable test case in topology * Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48) * Fix conv kernel shape bug for TF backend. (#50) * Add support for keras multi_gpu_model() API with MXNet backend (#49) * Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine * Fix typo * Add SAME padding mode support for pooling operator. (#51) * Add rnn() operator for MXNet backend with unrolling and masking feature (#46) * Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators. * Modified comments * Added comments to a method * Enable categorical crossentropy testcases and made minor changes * Modified message * nit * Added detail description of handling variable length input in RNN * Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn() * Adamax and NAdam optimizer for MXNet backend (#54) * Add Adamax optimizer for MXNet backend * Fix lr and adamax params * Add Nadam optimizer for mxnet backend * Add Conv3d transpose (#52) * conv3d tranpose, enabled test case * update kernel shape * replace conv2d_transpse conv3d_transpose with convnd_transpose * update value errors with MXNet Backend info, fix typo * add check for conv3d transpose only supports gpu with cudnn * update context check * diable conv3d transpose test * fix typo in comment * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - roywei@ * Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance * Conv3d() operator implementation for Keras2.0 using MXNet backend (#40) * conv3d implementation for keras2.0 as MXNet backend * conv3d implementation/testing for keras2.0 using MXNet backend * keeping -n option in pytest.ini file * fixed comments given by Sandeep * Add Conv1D support for MXNet backend (#44) * Add Conv1D support for MXNet backend * Fix CR comments * Conv2d transpose (#47) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * enable test case in topology * Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48) * Fix conv kernel shape bug for TF backend. (#50) * Add support for keras multi_gpu_model() API with MXNet backend (#49) * Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine * Fix typo * Add SAME padding mode support for pooling operator. (#51) * Add rnn() operator for MXNet backend with unrolling and masking feature (#46) * Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators. * Modified comments * Added comments to a method * Enable categorical crossentropy testcases and made minor changes * Modified message * nit * Added detail description of handling variable length input in RNN * Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn() * Adamax and NAdam optimizer for MXNet backend (#54) * Add Adamax optimizer for MXNet backend * Fix lr and adamax params * Add Nadam optimizer for mxnet backend * Add Conv3d transpose (#52) * conv3d tranpose, enabled test case * update kernel shape * replace conv2d_transpse conv3d_transpose with convnd_transpose * update value errors with MXNet Backend info, fix typo * add check for conv3d transpose only supports gpu with cudnn * update context check * diable conv3d transpose test * fix typo in comment * Rebase to latest Keras - April 3, 2018 * Add build badges * Fix multi_gpu API bug for CPU. Fix PEP. (#64) * Fix multi_gpu API bug for CPU. Fix PEP. * fix embedding layer bug (#61) * fix embedding bug * addressed comments, enabled more test cases * add keras test * reduce line length * fix style, add blank lines * Benchmark (#55) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * add benchmark scripts for resnet and imagenet data * combine scripts * fix args * fix num of gpus * update log * multi_gpu_model only support tf * add benchamrk scripts for synthetic data * update read me and scripts * add mxnet traing result table * update on readme * add cifar10 dataset and enable various resnet layers * fix compile for mxnet multiple gpu * update callbacks * update synthetic data script, add credits * undo new line * update readme, addressed pr comments * update readme * benchmark scripts style fix (#66) * style fix * remove unused import, fix line too long * adrressed pr comments * Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend (#65) * Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend * Modified comments * Addressed review comments and made the API more generic accross backends * Removed shape check * Modified comments * Added edge cases * moved helper method as nested * Added RNN benchmark scripts (#69) * Added RNN benchmark scripts * Fixed new line in bash script * Removed different backend code and modified comments * Removed spacing * Automated the wikiText2 download script * Added dataset_util functionality to have more flexible code * Added minor comments * modified minor comments * Fixed the multi-gpu context (#68) * Update benchmark result (#70) * update benchmark result * update result * simplify folder structure * add image result * add note * add note * Add MXNet Backend (#59) * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - roywei@ * Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance * Conv3d() operator implementation for Keras2.0 using MXNet backend (#40) * conv3d implementation for keras2.0 as MXNet backend * conv3d implementation/testing for keras2.0 using MXNet backend * keeping -n option in pytest.ini file * fixed comments given by Sandeep * Add Conv1D support for MXNet backend (#44) * Add Conv1D support for MXNet backend * Fix CR comments * Conv2d transpose (#47) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * enable test case in topology * Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48) * Fix conv kernel shape bug for TF backend. (#50) * Add support for keras multi_gpu_model() API with MXNet backend (#49) * Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine * Fix typo * Add SAME padding mode support for pooling operator. (#51) * Add rnn() operator for MXNet backend with unrolling and masking feature (#46) * Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators. * Modified comments * Added comments to a method * Enable categorical crossentropy testcases and made minor changes * Modified message * nit * Added detail description of handling variable length input in RNN * Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn() * Adamax and NAdam optimizer for MXNet backend (#54) * Add Adamax optimizer for MXNet backend * Fix lr and adamax params * Add Nadam optimizer for mxnet backend * Add Conv3d transpose (#52) * conv3d tranpose, enabled test case * update kernel shape * replace conv2d_transpse conv3d_transpose with convnd_transpose * update value errors with MXNet Backend info, fix typo * add check for conv3d transpose only supports gpu with cudnn * update context check * diable conv3d transpose test * fix typo in comment * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - roywei@ * Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance * Conv3d() operator implementation for Keras2.0 using MXNet backend (#40) * conv3d implementation for keras2.0 as MXNet backend * conv3d implementation/testing for keras2.0 using MXNet backend * keeping -n option in pytest.ini file * fixed comments given by Sandeep * Add Conv1D support for MXNet backend (#44) * Add Conv1D support for MXNet backend * Fix CR comments * Conv2d transpose (#47) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * enable test case in topology * Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48) * Fix conv kernel shape bug for TF backend. (#50) * Add support for keras multi_gpu_model() API with MXNet backend (#49) * Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine * Fix typo * Add SAME padding mode support for pooling operator. (#51) * Add rnn() operator for MXNet backend with unrolling and masking feature (#46) * Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators. * Modified comments * Added comments to a method * Enable categorical crossentropy testcases and made minor changes * Modified message * nit * Added detail description of handling variable length input in RNN * Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn() * Adamax and NAdam optimizer for MXNet backend (#54) * Add Adamax optimizer for MXNet backend * Fix lr and adamax params * Add Nadam optimizer for mxnet backend * Add Conv3d transpose (#52) * conv3d tranpose, enabled test case * update kernel shape * replace conv2d_transpse conv3d_transpose with convnd_transpose * update value errors with MXNet Backend info, fix typo * add check for conv3d transpose only supports gpu with cudnn * update context check * diable conv3d transpose test * fix typo in comment * Rebase to latest Keras - April 3, 2018 * Add build badges * Fix multi_gpu API bug for CPU. Fix PEP. (#64) * Fix multi_gpu API bug for CPU. Fix PEP. * fix embedding layer bug (#61) * fix embedding bug * addressed comments, enabled more test cases * add keras test * reduce line length * fix style, add blank lines * Benchmark (#55) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * add benchmark scripts for resnet and imagenet data * combine scripts * fix args * fix num of gpus * update log * multi_gpu_model only support tf * add benchamrk scripts for synthetic data * update read me and scripts * add mxnet traing result table * update on readme * add cifar10 dataset and enable various resnet layers * fix compile for mxnet multiple gpu * update callbacks * update synthetic data script, add credits * undo new line * update readme, addressed pr comments * update readme * benchmark scripts style fix (#66) * style fix * remove unused import, fix line too long * adrressed pr comments * Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend (#65) * Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend * Modified comments * Addressed review comments and made the API more generic accross backends * Removed shape check * Modified comments * Added edge cases * moved helper method as nested * Added RNN benchmark scripts (#69) * Added RNN benchmark scripts * Fixed new line in bash script * Removed different backend code and modified comments * Removed spacing * Automated the wikiText2 download script * Added dataset_util functionality to have more flexible code * Added minor comments * modified minor comments * Fixed the multi-gpu context (#68) * Update benchmark result (#70) * update benchmark result * update result * simplify folder structure * add image result * add note * add note * rebase to latest Keras - April 20, 2018, fix bug and unit tests * Added detailed RNN results (#73) * Added detailed RNN results * Modified table content and added CUDA version * fix keras examples (#72) * fix auto encoder examples * update other examples * fix style and add ctc not implemented error * Added Detailed RNN results (#77) * Modified RNN benchmark document * Added minor comments * fixed broken image link * Added API to extract metrics from a test and also added epoch parameter (#78) * Add mxnet backend tutorial documents (#76) * add performance tips document * update warning * add docs from wiki * add initial multi gpu doc, simplified installation doc, fix benchmark doc typo * update install steps * add multi_gpu_model tutorial * Support exporting model as MXNet model (sym, params). (#80) * Support exporting model as MXNet model (sym, params). * Return data_names and data_shapes * add unit tests for mxnet model save API * Add test with LSTM layer for mxnet model save API * Add support for functional Model graphs in save_mxnet_model API * Add additional logging for cnn benchmarks (#89) * add extra logging * add logging for cnn synthetic * fix log name * fix file name * Log RNN benchmark results (#90) * Make benchmark result logging available in RNN scripts * Make log file name consistent across CNN and RNN benchmarks * fix pytest errors (#93) * Cherry pick keras-team/keras 2.1.6 missing 3 commits into awslabs/keras-apache-mxnet (#96) * update multi_gpu api in benchmark scripts (#95) * update multi_gpu * update logging * fix logging * fix logging * fix speed format * remove learning rate log * Revamp keras-mxnet docs (#82) * Update main README and move mxnet_backend_docs under docs * revisit installation mxnet backend docs * revisit multi_gpu_training mxnet backend docs * revisit performance_guide mxnet backend docs * revisit using rnn with mxnet backend in mxnet backend docs * add save_mxnet_model tutorials in mxnet backend docs * Fixing review comments from aaron * Resolve CR comments on save_mxnet_model tutorial * Fix broken links, update tutorial links in the mxnet_backend code * revamp benchmark results readme * Benchmark results README page revamp * Add library versions * Remove too detailed benchmark results. Summarize in README * Get back detailed results document * Remove experiemental RNN benchmarks from README * addressed review comments on benchmark results * Set latest stable dependency of h5py to avoid warnings * Rebase to latest Keras April 20 2018 (#71) * Improve tests by designating dtype of sample data (#9834) * Document that "same" is inconsistent across backends with strides!=1 (#9629) * Document that `"same"` is inconsistent across backends with `strides` != 1 * Use "[here](...)" * #9642 Add kwarg and documentation for dilation_rate to SeparableConvs (#9844) * Add kwarg and documentation for dilation_rate to SeparableConvs * Fix pep8 complaint I forgot to check the style before committing. Pep8 was complaining about a missing whitespace after comma, now it's fixed. * fit/evaluate_generator supporting native tensors (#9816) Currently, `fit/evaluate_generator` don't support this case without this fix. But framework-native data tensors are already supported by `_fit_loop` and `_test_loop`. Signed-off-by: CUI Wei <ghostplant@qq.com> * Add h5py to dependencies * Fixed typo. (#9866) * Fix image_ocr.py example ValueError (#9869) * Fixed the NASNet issue. (#9865) * Fixed the NASNet issue. * Nasnet doesn't require flatten. * Updated documentation accordingly. * Removed generate dropout ones from recurrent. (#9892) * Removed generate dropout ones from recurrent. * Fixed index issue. * Fix `in_test_phase` of CNTK and Add its tests (#9902) * Fix dtype designation for `variable` of CNTK and Add its tests (#9903) * import `pydot`, improve error messages about `pydot` and GraphViz, bump to `pydot >= 1.2.4` (#9904) * REL: bump to `pydot >= 1.2.4` in `extras_require` * MAI: import pydot (as required in `extras_require`) * MAI: refine error messages for `pydot` and GraphViz distinguish between absence of `pydot` and failure to find the executables of GraphViz in the $PATH. * DEV: ignore `.pytest_cache` * Fix documentation of flow_from_directory() (#9910) The way the documentation is parsed for the Keras website made some lines of the documentation beginning with "Default:" look funny. Also changed the documentation of return value to be clear that it always returns a batch of images. * ModelCheckpoint: print previous best (#9911) * multi_gpu_model supporting legacy/fullCPU/fullGPU (#9638) Signed-off-by: CUI Wei <ghostplant@qq.com> * Fix `batch_dot` of Theano when `axes=0` (#9920) * Fix `batch_dot` of CNTK when `axes=None` (#9921) * Fix `batch_dot` of TensorFlow when `axes=None` (#9922) * Fix stateful metrics when passing dict to compile (#9894) * Added note to manually install h5py where needed (#9830) * Added notes to manually install h5py if needed * Added FAQ entry on h5py * deleted redundant remark about h5py * updated FAQ to reflect dependency change * fixed comment format to pass failing test * removed new trailing whitespaces * improved docstring format * reverted callbacks.py * fixed links in model.py * updated faq.py * link pointing to FAQ * Add support for `constants` in Bidirectional wrapper (#9260) * Add support fot `constants` in Bidirectional wrapper * Add more tests for Bidirectional wrapper * Fix `compute_mask` for Birectional with return_state=True Fix `compute_mask` to properly support `return_state` introduced in Birectional with #8977 * Add test for Bidirectional with unknown timestamps * Skip test for CNTK for unknown timestamps with Bidirectional * avoid override the input constant when need broadcast sequential axis on rnn's constant * Move _standardize_args to recurrent, remove duplication * Fix for Birectional when multiple masks are passed * Updated for TF 1.7 (#9937) * fix TimeSeriesGenerator glitch (#9899) * Added an error message for undefined shape on NASNet. (#9891) * Added an error message for undefined shape on NASNet. * Forgot that the message should be present only when loading imagenet weights. * Changed the message. * Fix PEP8 * Allow shift_range to be 1-D array-like or int (#8869) * Allow shift_range to be 1-D array-like or int * Add docstrings * Fix conflict resolution merge minor disaster * remove stray line from merge * Remove extra "tabs" * Exclude multi-gpu utils when reporting coverages (#9942) * Make conv_invalid_use and pooling_invalid_use efficient (#9944) * Chenta/cntk bn (#9952) * fix cntk static learning phase issue; add a test * fix code style;add more comments * add boolean support * fix code style issue * Immigrate reference operations to a separate module (#9948) * Add MXNet Backend (#59) * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - royw…
sandeep-krishnamurthy
added a commit
to awslabs/keras-apache-mxnet
that referenced
this pull request
Aug 14, 2018
* Document that "same" is inconsistent across backends with strides!=1 (#9629) * Document that `"same"` is inconsistent across backends with `strides` != 1 * Use "[here](...)" * Add h5py to dependencies * import `pydot`, improve error messages about `pydot` and GraphViz, bump to `pydot >= 1.2.4` (#9904) * REL: bump to `pydot >= 1.2.4` in `extras_require` * MAI: import pydot (as required in `extras_require`) * MAI: refine error messages for `pydot` and GraphViz distinguish between absence of `pydot` and failure to find the executables of GraphViz in the $PATH. * DEV: ignore `.pytest_cache` * Added note to manually install h5py where needed (#9830) * Added notes to manually install h5py if needed * Added FAQ entry on h5py * deleted redundant remark about h5py * updated FAQ to reflect dependency change * fixed comment format to pass failing test * removed new trailing whitespaces * improved docstring format * reverted callbacks.py * fixed links in model.py * updated faq.py * link pointing to FAQ * Added an error message for undefined shape on NASNet. (#9891) * Added an error message for undefined shape on NASNet. * Forgot that the message should be present only when loading imagenet weights. * Changed the message. * Fix PEP8 * Make conv_invalid_use and pooling_invalid_use efficient (#9944) * Chenta/cntk bn (#9952) * fix cntk static learning phase issue; add a test * fix code style;add more comments * add boolean support * fix code style issue * fixing typos (#10016) * Add `separable_conv2d` for Theano (#10003) * Add `separable_conv2d` for Theano * Enable the layer test for `separable_conv2d` * Fix the strides for 1x1 conv * Refactor topological part of `engine` module (#10023) * Refactor topological part of Keras engine. * Fix imports * Fix merge mixup. * Fix Bidirectional Regularization (#10012) * Fix Bidirectional Regularization Override the Wrapper `get_updates_for` and `get_losses_for` methods so that contributions from both the forward and backward layers are included in Bidirectional. * Use Parent for Calculating Inner Inputs Remove unneeded inner input calculations. * Simplify Bidirectional Losses * Add Bidirectional Unit Tests Test Bidirectional updates and losses. * Remove Trailing Whitespace * Fix Bidirectional Loss Inputs * Add Tests for Conditional Updates/Losses * Remove Whitespace * Refactor training part of `engine` module. (#10029) * Refactor topological part of Keras engine. * Fix imports * Fix merge mixup. * Refactor training part of the Keras engine. * Fix unit tests. * Fix `int_shape` of Theano and Refactor associated lines (#10030) * Add exceptions for `batch_dot` (#10020) * Enable Xception to work on Theano and CNTK (#10024) * Enable Xception to work on Theano and CNTK * Fix different predictions over all the backends * Add support for passthrough arguments to NumpyArrayIterator (#10035) * Add support for second output to NumpyArrayIterator * Enable Model subclassing API (#10046) * Refactor topological part of Keras engine. * Fix imports * Fix merge mixup. * Refactor training part of the Keras engine. * Fix unit tests. * Refactor Network to prepare support for Model subclassing * Finish enabling Model subclassing. * Add `long_description` field in setup.py. * Remove unused import * Increase pytest duration from 10 mins to 20 mins (#10072) * [RELNOTES] Simplify implementation of Sequential and remove legacy Merge support (#10077) * Refactor topological part of Keras engine. * Fix imports * Fix merge mixup. * Refactor training part of the Keras engine. * Fix unit tests. * Refactor Network to prepare support for Model subclassing * Finish enabling Model subclassing. * Simplify Sequential implementation. RELNOTES: This breaks weight loading and model loading/saving for models from Keras 0.* that used Merge layers. The Merge layer has been deprecated for 2 years. * [RELNOTES] Remove support for Keras 0.* Merge layer and associated functionality, which was scheduled for 08/2017. * fix typo (#10078) * Add documentation to several activation functions (#10066) * Add documentation to several activation functions * Fix style issues. * Small style fixes. * [RELNOTES] Introduce `preprocessing.image.save_img` and remove deprecated imsave method in neural style transfer example (#9996) * imsave method in scipy.misc package is deprecated - now using imageio * updated save_img method to use array_to_img method. also updated the neural style transfer example to use the new save_img method * forgot to commit changes - updated save_img * added file_format and **kwargs parameter to save_img and updated docstring * removed space at the end of a line in save_img method. updated instances of imsave in example scripts with save_img method. * added kwargs to docstring. removed additional whitespace lines * removed trailing whitespace * Extensive style fixes in `preprocessing.image`. * [RELNOTES] Allow loading external backends (#10034) * load external backend * safety++ * Excpetion -> ValueError * Revert TF version to 1.7 on Travis CI. (#10101) * New Callback: EarlyBaselineStopping (#10061) * Initial Commit * Continued changes * Alpha version * Added support to make sure that previous epochs, which may pass the baseline test, are kept in variable history. * Code formatting * Code formatting * Code formatting to address CI errors * Code formatting for CI errors * Initial unit test code * Adjust for epoch being zero-based * Code formatting * Unit tests added * Code formatting * Code formatting * Factorized the unit test * Code formatting * Refactored to be part of EarlyStopping and modified unit tests * Code formatting * Adds MobiletNetV2 to applications (#10047) * [RELNOTES] Allow symbolic tensors to be fed to models (with TF backend) (#10087) * Allow symbolic tensors to be fed to models (with TF backend) * Skip tensor feeding tests when TF>=1.8 not available. * [RELNOTES] Fix EarlyStopping API * Fix shape mismatch in `rnn()` of tensorflow_backend (#10038) * Fix shape mismatch in `rnn()` of tensorflow_backend Fix the shape mismatch in `rnn()` with `not unroll` and `mask is not None` of the Tensorflow backend. Problem: If the rnn cell has any (recurrent) `states[i]` whose shape is different from that of the `output` (`states[0]`), there will raise an `ValueError` when updating that state. Reason: This is because the `tiled_mask_t` is not updated correctly for each `states[i]` but instead is simply copied from that of the `output` (`states[0]`). Solution: Tile the `mask_t` with the correct shape of each `states[i]`. Notice that in a similar situation with `unroll is True`, the `tiled_mask_t` is handled correctly. * add unit_test for rnn() with states whose shape is different from that of the output. * Revert "add unit_test for rnn() with states whose shape is different from that of the output." This reverts commit f1df2a58ff635bbf698444e3d7403785a92dfed1. * Simplify the unit_test for rnn with additional states * [RELNOTES] Default `Flatten` layer’s `data_format` argument to `None`. * #10080 Convert CuDNN weights in nested Model. (#10081) * #10080 Convert CuDNN weights in nested Model. - similar problem to nesting in Bidirectional (#8860) - quick fix: just copy/paste. I'll refactor it later - also convert biases from H5 Dataset to np.array for reshape() (#9662) * Refactor keras.engine.saving.preprocess_weights_for_loading(). - less duplication - extracted conversion methods for Bidirectional/Model/Sequential * Format docstrings to follow the project code style. * Move tests of CuDNN RNN weight conversion to a more proper place. - move from cudnn_recurrent_test.py to test_model_saving.py - adjust imports to be consistent * Make better tests for CuDNN RNN weight conversion. - add nested models (#10080) - various model types: add Model, Sequential (not only Sequential) - convert weights on model, not layer (not test nesting in models) - test just by save/load model weights instead of calling preprocess_weights_for_loading() directly * Check GPU support via tf in pytest skipif with short-circuit evaluation. It seems that multiple skipif decorators get evaluated eagerly, not lazily. Thus it fails on theano and cntk backends. I don't understand why the tests didn't fail until now. Maybe some change in pytest? * Refactor (deduplicate) skipif decorators for TensorFlow and GPU. At least multiple occurrences within a test module. Now we can't import from `tests/` and `keras.utils.test_utils` don't se pytest. Otherwise we can define the marker only once at all. * Fix PEP8 (whitespace). * Add `separable_conv1d` (#10125) * Add `separable_conv1d` for Theano * Add `separable_conv1d` for CNTK * Fix preprocess_input not working with int arrays (#10134) * Make MobileNet tests random (#10132) * Sample weighted ImageDataGenerator (#10092) * Add support for sample_weight in ImageDataGenerator.flow * Added test for sample weighted imagedatagen * clarified docs + PEP8 * pep8 - blank line * sample_weight argument after shuffle * RNN docstring fixes; pass `maximum_iteration` argument to `while_loop` in TF. * Use `embedding_lookup` for `gather` in TF backend (enables partitioned variables). * Add `data_format` argument to Conv1D; docstring style fixes. * Further fixes. * Fix initializers (#9963) * fix VarianceScaling * Correct standard initializer testing * Add test for truncated normal in backend * Use correct theano function for truncated_normal * Fix sloppiness in correction * fix PEP * improve style initialiser fix * fix line length * Use NCW/NWC for conv1d data format in TF backend. * Model loading: do not return inside `try` block. * Rely on `Flatten` layer to do data_format conversion in ResNet50, VGG16, VGG19. * Remove potentially misleading warning messages in `applications`. * Fix typo. * Remove useless super delegation. * Remove batch_size when doing stepwise fit (#10144) `batch_size` is always `None` , because `steps_per_epoch` and `batch_size` are mutually exclusive. Passing batch_size seems irrelevant as `None` is its default value in `test_loop` * Add exceptions for `fit_loop` (#10145) * Style fixes (#10148) * Fix ConvLSTM2D unit_forget_bias initializer * Avoid a warning by using 'x = x * y' instead of 'x *= y' (#10158) * Fix undefined behaviour: preprocess_input copying/not copying the input arrays (#10153) * Add copy option for image preprocessing * Fix unnecessary import * Fix style * fix test error * Make modifications in-place instead * vae examples fixes (#10062) * vae examples fixes * white spaces around op * comments and docstring fixes * comments and docstring fixes * comments and docstring fixes * fixes on docs * docs and spacing * Merge 2 functions together in applications MobileNetV2 (#10163) * merge 2 functions together _inverted_res_block() and _first_inverted_res_block() are nearly the same. Merged them into one function 1. skip "Expand" part for 0 block 2. made layers names similar to original TF graph names * added "mobl_" to prefix, pep8 fix * remove dropout parameter * concise prefix name * Remove named argument from schedule function (#10178) * Add missing doc (#10188) * Best to set self.built=True at the end of build() (#10191) When using Keras on TensorFlow (in graph mode), via `tf.keras`, then calling `self.add_weight()` fails if `self.built==True`, so it is best to encourage users to set `self.built=True` at the *end* of their `build()` method, rather than just "somewhere". * Clean up preprocessing of `depthwise_kernel` for Theano (#10189) * Allow dynamic backends in _need_convert_kernel (#10111) * Fix _need_convert_kernel for external backends * Don't assume external backend won't support NASNet * Update from review comments * fix TensorBoard callback with unit test (#10173) * Allow project specific config files in Keras (#10183) * Allow project specific config files in Keras * Clearer code, updated comment * Update cifar10_resnet.py (#10199) change the comments on line 133. * Adds height_shift_range to preprocessing.image doc and adds support for indentation in auto-generated doc (#10194) * Update README.me * Mobilenetv2 explanation (#10174) * i add mobilenetv2 to the table * dd explanation for mobilenetv2 * add explanation for mibilenetv2 * Make multi_gpu_model serializable. * Skip tests for multi gpu * Fix slice namespace collision in backend. * Skip multi_gpu tests for other backends than TF * Replace np.ceil() with a faster operation (#10184) * Replace ceil() with faster a operation Replacing np.ceil() with faster operations, as suggested in #8488 * Revert "Replace ceil() with faster a operation" This reverts commit 74fe60b445377b4f1be1d0a36b4777d7ed4f1c1b. * Replace np.ceil() with a faster operation * Fix syntax error * Remove trailing whitespace * Explicitly mention using a Sequence object in val (#10223) * Grammatical error - "inputs channels" rather than "input". (#10217) * Fix duplicated argname: num_gpus === parts (#10228) Signed-off-by: CUI Wei <ghostplant@qq.com> * Fix NASNet (#10209) * Fix NASNet * Update weight files * In-place split to avoid inter-device duplication (#10230) New Benchmark by in-place split: >> keras.application.Resnet50 224x224x3 (NCWH; NVidia Tesla P100 x 4) input_shape = 3x224x224, batch_size = 96 x 4: 392(images/sec) => 417(images/sec) input_shape = 3x299x299, batch_size = 64 x 4: 229(images/sec) => 244(images/sec) input_shape = 3x224x224, batch_size = 8 x 4: 148(images/sec) => 163(images/sec) >> keras.application.InceptionV3 (NCWH; NVidia Tesla P100 x 4) input_shape = 3x224x224, batch_size = 128 x 4: 488(images/sec) => 526(images/sec) input_shape = 3x299x299, batch_size = 96 x 4: 270(images/sec) => 294(images/sec) input_shape = 3x224x224, batch_size = 8 x 4: 146(images/sec) => 158(images/sec) Signed-off-by: CUI Wei <ghostplant@qq.com> * Increase test coverages by adding invalid CNTK usecases (#10236) * Remove Sequential.model deprecation warning (#10256) * Remove Sequential.model deprecation warning * Remove dead line of code * Increase test coverages by factorizing CNTK pads (#10259) * Refactor ImageDataGenerator (#10130) * Create get_random_transform and refactor * Fix style and add tests * Add more tests * Fix documentation error * Fix documentation style issue * add apply_affine_transform * document transformation dictionary * Doc style fix * Remove deprecated model.model from engine/saving (#10275) * Typo in docstring for softplus (#10277) Softplus docstring missing a parenthesis. * Make Dot documentation inline with Concatenate (#10271) Doc expects a list containing 2 tensors. * Fixes automatic doc generation problem with nested lists. Adds a new test (#10212) * Fixes automatic doc generation problem with indented lists. Adds a new test * Some style fixes on doc automatic generation files * Fixes a bad space in convolutional_recurrent.py * Changes the test_doc_auto_generation in order to include a doc string taken from the codebase. Allows text lines following nested lists * Use count_params function for non_trainable_count. (#10280) * load_weights will fail if shape mismatch (#10266) Fix for #10265 * Adds to and alphabetizes documentation of Layer base class. (#10282) * Alphabetizes and adds to layers doc. * Responding to @cais comments * fix spacing. Remove in(out)bound_nodes * Non training Batch Norm operator has bad performance for it running into tensorflow's non fused batch norm API (#10207) * When use tensorflow as backend, let batch norm run into fused batch norm as much as possible, which has better performance. fix issue: http://github.com/keras-team/keras/issues/10058 * In Tensorflow backend, let batch norm call to FusedBatchNorm only NHWC format, also gamma and beta are not None. Test result: test env: with Tensorflow(commit a543d9471047ca3f6881c87105fcbe2cdff9207d Date: Thu May 10 17:43:30 2018, local build), python3.4, centos7.4 test cases: "pytest ./tests/keras/layers/normalization_test.py" <all passed> "pytest ./tests" <keep same result as without this commit's modification on BN> * fix code sytle. * 1. Add axis parameter in backend's batch_normalization functions. 2. Refine the batch_normalization function in tensorflow backend, Let's it call to fused batch norm as much as possible. Thanks the coments from fchollet. * Trigger * 1. add default value -1 for parameter axis in batch_normalization function in backend. 2. fix some code style. Thanks the comments from fchollet. * Handle capitalised extensions in list_pictures (#10220) #10219 * Typo fix (#10293) * Fix doc (#10308) * Fix naming convention * Add missing doc * Fix typo * Improve docstrings of applications (#10310) * Add pooling options in MobileNetV2 (#10313) * Add pooling option * Add pooling test * Fix doc (#10327) Fixed doc * Handle `mask` in `TimeDistributed` wrapper. (#10242) * equip TimeDistributed with mask and unspecified input length * fix bugs in theano. add test on timedistributed + masking * skip tests on cntk with multiple unspecified time lengths. * move static shape inference to theano_backend, add docstring, etc. * fix format * Split `applications` and `preprocessing` modules. (#10339) * Split `applications` and `preprocessing` modules. * Fix dependencies. * Move tests for applications (#10341) * Improve the docstring of Conv3DTranspose (#10342) * Add depth as third dimension in docstring of Conv3DTranspose in convolutional.py in keras.layers * Reduce tests for applications (#10346) * Reduce tests for applications * Make selection over all models random * Add an advanced activation layer for ReLU (#10322) The max_value argument can not be used in a layer, except custom layer or Lambda. Hence, similarly to LeakyReLU or for example Softmax, this PR adds a layer for ReLU, enabling also a capped ReLU to be used. * FIX: Tensorboard callback only supports logging Embeddings layer weights (#7766) * Embed layer-outputs rather than layer-weights in TensorBoard callback * Update docstring and allow multiple inputs * Fix tests * Renaming * Set learning phase * Compute embeddings in batches * Pass embedding data explicitly * Actually process embeddings in batches * Allow multiple inputs and validate input data * Add example * Delete utils.py * Revert uncorrectly resolved merge-conflict * Minor renaming * Add comment clarifying the design choice * Fix HDF5Matrix issue when working in conjunction with TimeSeriesGenerator (#10334) * Fix issue when working in conjunction with TimeSeriesGenerator The TimeSeriesGenerator class uses xrange through six which caused an IndexError * Add test * Add corresponding test * Fix for python3 * Simplified code * Fix indent * Fix test * Supporting channels_first data format with crossentropy losses (#9715) * Add error message when calling `summary` on unbuilt subclassed models. * Prepare 2.2.0 release. * Fix a version number (#10361) * Update to Keras Applications 1.0.2 (fixes NASNet issue). * Add tests for inputs set dynamically (#10367) * CuDNN RNN layers nested in TimeDistributed are not converted when loading (#10357) * Add a unit test for CuDNNGRU conversion with TimeDistributed. * Extract duplicated function convert_model() to _convert_model_weights(). * #10356 Convert weights of CuDNN/plain RNN nested in TimeDistributed. Same case as for Bidirectional, except that in TimeDistributed there's only one nested layer instead of two. * Style fix * Update docs for 2.2.0. * Add spatial dropout and 3D global pooling to docs (#10373) * spatial dropout in docs * 3d global pooling in docs * Doc update (#10376) A couple of variables are "used" in two examples without being defined. For consistency with other examples where auxiliary dimensions are defined, I think it would be better to explicitly assign them a value. I just used made up values, feel free to change to whatever makes more sense! * Preserve input shape data when serializing deferred-build Sequential models. * Add MXNet Backend (#59) * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - roywei@ * Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance * Conv3d() operator implementation for Keras2.0 using MXNet backend (#40) * conv3d implementation for keras2.0 as MXNet backend * conv3d implementation/testing for keras2.0 using MXNet backend * keeping -n option in pytest.ini file * fixed comments given by Sandeep * Add Conv1D support for MXNet backend (#44) * Add Conv1D support for MXNet backend * Fix CR comments * Conv2d transpose (#47) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * enable test case in topology * Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48) * Fix conv kernel shape bug for TF backend. (#50) * Add support for keras multi_gpu_model() API with MXNet backend (#49) * Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine * Fix typo * Add SAME padding mode support for pooling operator. (#51) * Add rnn() operator for MXNet backend with unrolling and masking feature (#46) * Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators. * Modified comments * Added comments to a method * Enable categorical crossentropy testcases and made minor changes * Modified message * nit * Added detail description of handling variable length input in RNN * Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn() * Adamax and NAdam optimizer for MXNet backend (#54) * Add Adamax optimizer for MXNet backend * Fix lr and adamax params * Add Nadam optimizer for mxnet backend * Add Conv3d transpose (#52) * conv3d tranpose, enabled test case * update kernel shape * replace conv2d_transpse conv3d_transpose with convnd_transpose * update value errors with MXNet Backend info, fix typo * add check for conv3d transpose only supports gpu with cudnn * update context check * diable conv3d transpose test * fix typo in comment * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - roywei@ * Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance * Conv3d() operator implementation for Keras2.0 using MXNet backend (#40) * conv3d implementation for keras2.0 as MXNet backend * conv3d implementation/testing for keras2.0 using MXNet backend * keeping -n option in pytest.ini file * fixed comments given by Sandeep * Add Conv1D support for MXNet backend (#44) * Add Conv1D support for MXNet backend * Fix CR comments * Conv2d transpose (#47) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * enable test case in topology * Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48) * Fix conv kernel shape bug for TF backend. (#50) * Add support for keras multi_gpu_model() API with MXNet backend (#49) * Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine * Fix typo * Add SAME padding mode support for pooling operator. (#51) * Add rnn() operator for MXNet backend with unrolling and masking feature (#46) * Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators. * Modified comments * Added comments to a method * Enable categorical crossentropy testcases and made minor changes * Modified message * nit * Added detail description of handling variable length input in RNN * Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn() * Adamax and NAdam optimizer for MXNet backend (#54) * Add Adamax optimizer for MXNet backend * Fix lr and adamax params * Add Nadam optimizer for mxnet backend * Add Conv3d transpose (#52) * conv3d tranpose, enabled test case * update kernel shape * replace conv2d_transpse conv3d_transpose with convnd_transpose * update value errors with MXNet Backend info, fix typo * add check for conv3d transpose only supports gpu with cudnn * update context check * diable conv3d transpose test * fix typo in comment * Rebase to latest Keras - April 3, 2018 * Add build badges * Fix multi_gpu API bug for CPU. Fix PEP. (#64) * Fix multi_gpu API bug for CPU. Fix PEP. * fix embedding layer bug (#61) * fix embedding bug * addressed comments, enabled more test cases * add keras test * reduce line length * fix style, add blank lines * Benchmark (#55) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * add benchmark scripts for resnet and imagenet data * combine scripts * fix args * fix num of gpus * update log * multi_gpu_model only support tf * add benchamrk scripts for synthetic data * update read me and scripts * add mxnet traing result table * update on readme * add cifar10 dataset and enable various resnet layers * fix compile for mxnet multiple gpu * update callbacks * update synthetic data script, add credits * undo new line * update readme, addressed pr comments * update readme * benchmark scripts style fix (#66) * style fix * remove unused import, fix line too long * adrressed pr comments * Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend (#65) * Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend * Modified comments * Addressed review comments and made the API more generic accross backends * Removed shape check * Modified comments * Added edge cases * moved helper method as nested * Added RNN benchmark scripts (#69) * Added RNN benchmark scripts * Fixed new line in bash script * Removed different backend code and modified comments * Removed spacing * Automated the wikiText2 download script * Added dataset_util functionality to have more flexible code * Added minor comments * modified minor comments * Fixed the multi-gpu context (#68) * Update benchmark result (#70) * update benchmark result * update result * simplify folder structure * add image result * add note * add note * Add MXNet Backend (#59) * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - roywei@ * Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance * Conv3d() operator implementation for Keras2.0 using MXNet backend (#40) * conv3d implementation for keras2.0 as MXNet backend * conv3d implementation/testing for keras2.0 using MXNet backend * keeping -n option in pytest.ini file * fixed comments given by Sandeep * Add Conv1D support for MXNet backend (#44) * Add Conv1D support for MXNet backend * Fix CR comments * Conv2d transpose (#47) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * enable test case in topology * Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48) * Fix conv kernel shape bug for TF backend. (#50) * Add support for keras multi_gpu_model() API with MXNet backend (#49) * Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine * Fix typo * Add SAME padding mode support for pooling operator. (#51) * Add rnn() operator for MXNet backend with unrolling and masking feature (#46) * Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators. * Modified comments * Added comments to a method * Enable categorical crossentropy testcases and made minor changes * Modified message * nit * Added detail description of handling variable length input in RNN * Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn() * Adamax and NAdam optimizer for MXNet backend (#54) * Add Adamax optimizer for MXNet backend * Fix lr and adamax params * Add Nadam optimizer for mxnet backend * Add Conv3d transpose (#52) * conv3d tranpose, enabled test case * update kernel shape * replace conv2d_transpse conv3d_transpose with convnd_transpose * update value errors with MXNet Backend info, fix typo * add check for conv3d transpose only supports gpu with cudnn * update context check * diable conv3d transpose test * fix typo in comment * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - roywei@ * Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance * Conv3d() operator implementation for Keras2.0 using MXNet backend (#40) * conv3d implementation for keras2.0 as MXNet backend * conv3d implementation/testing for keras2.0 using MXNet backend * keeping -n option in pytest.ini file * fixed comments given by Sandeep * Add Conv1D support for MXNet backend (#44) * Add Conv1D support for MXNet backend * Fix CR comments * Conv2d transpose (#47) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * enable test case in topology * Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48) * Fix conv kernel shape bug for TF backend. (#50) * Add support for keras multi_gpu_model() API with MXNet backend (#49) * Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine * Fix typo * Add SAME padding mode support for pooling operator. (#51) * Add rnn() operator for MXNet backend with unrolling and masking feature (#46) * Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators. * Modified comments * Added comments to a method * Enable categorical crossentropy testcases and made minor changes * Modified message * nit * Added detail description of handling variable length input in RNN * Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn() * Adamax and NAdam optimizer for MXNet backend (#54) * Add Adamax optimizer for MXNet backend * Fix lr and adamax params * Add Nadam optimizer for mxnet backend * Add Conv3d transpose (#52) * conv3d tranpose, enabled test case * update kernel shape * replace conv2d_transpse conv3d_transpose with convnd_transpose * update value errors with MXNet Backend info, fix typo * add check for conv3d transpose only supports gpu with cudnn * update context check * diable conv3d transpose test * fix typo in comment * Rebase to latest Keras - April 3, 2018 * Add build badges * Fix multi_gpu API bug for CPU. Fix PEP. (#64) * Fix multi_gpu API bug for CPU. Fix PEP. * fix embedding layer bug (#61) * fix embedding bug * addressed comments, enabled more test cases * add keras test * reduce line length * fix style, add blank lines * Benchmark (#55) * add conv2d_transpose * conv2d transpose for both channels, enabled test case * add detailed comments and examples, fix style issue * add benchmark scripts for resnet and imagenet data * combine scripts * fix args * fix num of gpus * update log * multi_gpu_model only support tf * add benchamrk scripts for synthetic data * update read me and scripts * add mxnet traing result table * update on readme * add cifar10 dataset and enable various resnet layers * fix compile for mxnet multiple gpu * update callbacks * update synthetic data script, add credits * undo new line * update readme, addressed pr comments * update readme * benchmark scripts style fix (#66) * style fix * remove unused import, fix line too long * adrressed pr comments * Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend (#65) * Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend * Modified comments * Addressed review comments and made the API more generic accross backends * Removed shape check * Modified comments * Added edge cases * moved helper method as nested * Added RNN benchmark scripts (#69) * Added RNN benchmark scripts * Fixed new line in bash script * Removed different backend code and modified comments * Removed spacing * Automated the wikiText2 download script * Added dataset_util functionality to have more flexible code * Added minor comments * modified minor comments * Fixed the multi-gpu context (#68) * Update benchmark result (#70) * update benchmark result * update result * simplify folder structure * add image result * add note * add note * rebase to latest Keras - April 20, 2018, fix bug and unit tests * Added detailed RNN results (#73) * Added detailed RNN results * Modified table content and added CUDA version * fix keras examples (#72) * fix auto encoder examples * update other examples * fix style and add ctc not implemented error * Added Detailed RNN results (#77) * Modified RNN benchmark document * Added minor comments * fixed broken image link * Added API to extract metrics from a test and also added epoch parameter (#78) * Add mxnet backend tutorial documents (#76) * add performance tips document * update warning * add docs from wiki * add initial multi gpu doc, simplified installation doc, fix benchmark doc typo * update install steps * add multi_gpu_model tutorial * Support exporting model as MXNet model (sym, params). (#80) * Support exporting model as MXNet model (sym, params). * Return data_names and data_shapes * add unit tests for mxnet model save API * Add test with LSTM layer for mxnet model save API * Add support for functional Model graphs in save_mxnet_model API * Add additional logging for cnn benchmarks (#89) * add extra logging * add logging for cnn synthetic * fix log name * fix file name * Log RNN benchmark results (#90) * Make benchmark result logging available in RNN scripts * Make log file name consistent across CNN and RNN benchmarks * fix pytest errors (#93) * Cherry pick keras-team/keras 2.1.6 missing 3 commits into awslabs/keras-apache-mxnet (#96) * update multi_gpu api in benchmark scripts (#95) * update multi_gpu * update logging * fix logging * fix logging * fix speed format * remove learning rate log * Revamp keras-mxnet docs (#82) * Update main README and move mxnet_backend_docs under docs * revisit installation mxnet backend docs * revisit multi_gpu_training mxnet backend docs * revisit performance_guide mxnet backend docs * revisit using rnn with mxnet backend in mxnet backend docs * add save_mxnet_model tutorials in mxnet backend docs * Fixing review comments from aaron * Resolve CR comments on save_mxnet_model tutorial * Fix broken links, update tutorial links in the mxnet_backend code * revamp benchmark results readme * Benchmark results README page revamp * Add library versions * Remove too detailed benchmark results. Summarize in README * Get back detailed results document * Remove experiemental RNN benchmarks from README * addressed review comments on benchmark results * Set latest stable dependency of h5py to avoid warnings * Rebase to latest Keras April 20 2018 (#71) * Improve tests by designating dtype of sample data (#9834) * Document that "same" is inconsistent across backends with strides!=1 (#9629) * Document that `"same"` is inconsistent across backends with `strides` != 1 * Use "[here](...)" * #9642 Add kwarg and documentation for dilation_rate to SeparableConvs (#9844) * Add kwarg and documentation for dilation_rate to SeparableConvs * Fix pep8 complaint I forgot to check the style before committing. Pep8 was complaining about a missing whitespace after comma, now it's fixed. * fit/evaluate_generator supporting native tensors (#9816) Currently, `fit/evaluate_generator` don't support this case without this fix. But framework-native data tensors are already supported by `_fit_loop` and `_test_loop`. Signed-off-by: CUI Wei <ghostplant@qq.com> * Add h5py to dependencies * Fixed typo. (#9866) * Fix image_ocr.py example ValueError (#9869) * Fixed the NASNet issue. (#9865) * Fixed the NASNet issue. * Nasnet doesn't require flatten. * Updated documentation accordingly. * Removed generate dropout ones from recurrent. (#9892) * Removed generate dropout ones from recurrent. * Fixed index issue. * Fix `in_test_phase` of CNTK and Add its tests (#9902) * Fix dtype designation for `variable` of CNTK and Add its tests (#9903) * import `pydot`, improve error messages about `pydot` and GraphViz, bump to `pydot >= 1.2.4` (#9904) * REL: bump to `pydot >= 1.2.4` in `extras_require` * MAI: import pydot (as required in `extras_require`) * MAI: refine error messages for `pydot` and GraphViz distinguish between absence of `pydot` and failure to find the executables of GraphViz in the $PATH. * DEV: ignore `.pytest_cache` * Fix documentation of flow_from_directory() (#9910) The way the documentation is parsed for the Keras website made some lines of the documentation beginning with "Default:" look funny. Also changed the documentation of return value to be clear that it always returns a batch of images. * ModelCheckpoint: print previous best (#9911) * multi_gpu_model supporting legacy/fullCPU/fullGPU (#9638) Signed-off-by: CUI Wei <ghostplant@qq.com> * Fix `batch_dot` of Theano when `axes=0` (#9920) * Fix `batch_dot` of CNTK when `axes=None` (#9921) * Fix `batch_dot` of TensorFlow when `axes=None` (#9922) * Fix stateful metrics when passing dict to compile (#9894) * Added note to manually install h5py where needed (#9830) * Added notes to manually install h5py if needed * Added FAQ entry on h5py * deleted redundant remark about h5py * updated FAQ to reflect dependency change * fixed comment format to pass failing test * removed new trailing whitespaces * improved docstring format * reverted callbacks.py * fixed links in model.py * updated faq.py * link pointing to FAQ * Add support for `constants` in Bidirectional wrapper (#9260) * Add support fot `constants` in Bidirectional wrapper * Add more tests for Bidirectional wrapper * Fix `compute_mask` for Birectional with return_state=True Fix `compute_mask` to properly support `return_state` introduced in Birectional with #8977 * Add test for Bidirectional with unknown timestamps * Skip test for CNTK for unknown timestamps with Bidirectional * avoid override the input constant when need broadcast sequential axis on rnn's constant * Move _standardize_args to recurrent, remove duplication * Fix for Birectional when multiple masks are passed * Updated for TF 1.7 (#9937) * fix TimeSeriesGenerator glitch (#9899) * Added an error message for undefined shape on NASNet. (#9891) * Added an error message for undefined shape on NASNet. * Forgot that the message should be present only when loading imagenet weights. * Changed the message. * Fix PEP8 * Allow shift_range to be 1-D array-like or int (#8869) * Allow shift_range to be 1-D array-like or int * Add docstrings * Fix conflict resolution merge minor disaster * remove stray line from merge * Remove extra "tabs" * Exclude multi-gpu utils when reporting coverages (#9942) * Make conv_invalid_use and pooling_invalid_use efficient (#9944) * Chenta/cntk bn (#9952) * fix cntk static learning phase issue; add a test * fix code style;add more comments * add boolean support * fix code style issue * Immigrate reference operations to a separate module (#9948) * Add MXNet Backend (#59) * Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1) * add activation functions * add activation functions * fix some legacy * fix some legacy * cross entropy * cross entropy * fix name scoping introduced in 2.0 * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * remove the logic for hacking RNN * add pooling with utils * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * mxnet_backend graph fix, layer support (#3) * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * Add dropout, l2_normalization, random_normal/uniform/binomial (#2) * remove the logic for hacking RNN * add pooling with utils * add activation functions * fix some legacy * cross entropy * fix name scoping introduced in 2.0 * remove the logic for hacking RNN * add pooling with utils * minor * lint and name scope fix * fix access protected var * fix add neighbor, removed __eq__ in KerasSymbol * fix eval function, unittest for placeholder and variable * add unittests * fix bug * fix bug * fix * add some temporary fixes in mxnet backend. undo change to the pytest.ini * Keras function not working is a known issue, add skip in the test * fix random_uniform/constant * fix legacy randomize methods * Fix MXNet backend operator bugs. Enabled Keras backend tests * add bias * Add Amazon copyrights to License (#6) * fix * fix * fix backend for mlp * fix context management, add optimizers * minor change * undo changes on example * fix eval * minor cleanup * fix some property usage * fixing AlphaDroupout, not finished yet * add mx model instantiate * modifies training model construct logic, fix some tests. fix reshape layer. * minor fix * fix bias_add * more fix on Dense and bias_add * In progress commit * fix comment * small fix * remove pytest.skip in conv3d. But it failed with theano backend in my workspace though. * Add conv2d and in_topk operator for mxnet backend (#11) * Skip BatchDot tests for Theano backend. (#12) * BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14) * Fix Conv2d shape issues and enable Conv2D UTs * Remove redundant mxnet only unit tests * Adding batch_dot, remove deconv, code comments and cleanup * Remove buggy conv1d implementation * Fix CR comments. Fix lint check issues * Move mxnet specific code from keras engine to mxnet_backend. (#15) * Move MXNet optimizers from keras optimizers to mxnet backend (#16) * Fix bug in reshape. Minor rename to avoid local conflicts * Bug fixes and enable/skip all Keras tests for mxnet backend (#21) * test results - 374 passed, 235 skipped in 114.44 seconds * fix/skip keras tests - tests/integration_tests, tests/keras/applications * fix/skip keras tests - tests/keras/engine/test_topology * fix/skip keras tests - tests/keras/engine/test_training * fix/skip keras tests - tests/keras/legacy/ * fix/skip keras tests - tests/keras/preprocessing * fix/skip keras tests - tests/keras/utils/ * Fix CR comments * Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test * Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization * Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test * Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving * Fix mxnet backend multi-gpu training (#31) Fixing bug for mxnet backend to use multiple gpus. * Fix performance issue - Batchnormalization, Conv operator (#35) * Fix default axis for batchnorm layer for channels_first data_format * Performance improvement by avoiding kernel transpose in conv operation for channels_first format * Fix model - architecture, weights and both, load and save. (#36) * Prepare initial version of mxnet related documentation in keras (#38) * Skip failing unit tests for unsupported functionality in mxnet backend * Fix pep tests reported by CI * Use pytest module skip, revert kernel_shape logic * remove data_format param from bias_add API * Allow Predict() without compile for mxnet backend and enable tests. contributor - royw…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes the issue raised in #7514.