Code

Port IDEFICS to tensorflow#26870

Code

Merged

Rocketknight1 merged 119 commits intohuggingface:mainhuggingface/transformers:mainfrom a8nova:add_tf_ideficsa8nova/transformers:add_tf_ideficsCopy head branch name to clipboardMay 13, 2024

+3,392-49

Commits

Commits on May 11, 2024

• Initial commit

  a8nova

  committed
  61a1519

• Just a copy of modeling_idefics.py that will be ported to TF

  a8nova

  committed
  a976da1

• - Prepend TF to the name of all classes

  Show description for 1873605
  a8nova

  committed
  1873605

• Add TF imports

  a8nova

  committed
  c7b8dbe

• Add autotranslated files

  

  Rocketknight1

  authored and

  a8nova

  committed
  9060913

• Add TF classes to model_tf_auto.py

  a8nova

  committed
  fccfbb0

• Add the TF classes in model_doc

  a8nova

  committed
  87dd0f9

• include auto-translated code

  a8nova

  committed
  3c2309d

• Adopted from auto-translated version

  a8nova

  committed
  4eaf3f3

• Add a forgotten super().build

  a8nova

  committed
  a8fabec

• Add test code for TF version.

  a8nova

  committed
  2115f97

• Fix indentation and load pytorch weights for now

  a8nova

  committed
  3a41a10

• Some fixes. Many tests are still failing but some are passing now.

  Show description for 411c02f
  a8nova

  committed
  411c02f

• Add ALL_LAYERNORM_LAYERS to match pytorch

  a8nova

  committed
  5da7317

• Revert "Add ALL_LAYERNORM_LAYERS to match pytorch"

  Show description for 6e356c5
  a8nova

  committed
  6e356c5

• Fix freeze_relevant_params()

  a8nova

  committed
  bdc06fe

• Some more fixes

  a8nova

  committed
  3643fe8

• Fix test_attention_outputs

  a8nova

  committed
  a8b4b4a

• Add tf stuff to processing_idefics.py

  Show description for 5d9e29f
  a8nova

  committed
  5d9e29f

• Pass return_tensors to image processing code and fix test

  a8nova

  committed
  e5aef04

• Pass return_tensors to the image processor __init__

  a8nova

  committed
  f468187

• Fix several test cases

  Show description for 74fbec8
  a8nova

  committed
  74fbec8

• Some more fixes forgotten in last commit

  a8nova

  committed
  1840c19

• Fix processing code and vision_tf.py

  a8nova

  committed
  e05549e

• Fix perceiver bug

  a8nova

  committed
  39ed34f

• Import from

  a8nova

  committed
  10a54a1

• Auto-add build() methods + style pass

  

  Rocketknight1

  authored and

  a8nova

  committed
  e058b2c

• Fix build() errors due to None being passed as shape to some layers

  a8nova

  committed
  5ba6381

• Change name in TFIdeficsForVisionText2Text to attribute in IdeficsForVisionText2Text

  a8nova

  committed
  443a276

• Fix pytorch weights load for tf2

  Show description for d066a76
  a8nova

  committed
  d066a76

• Attempt to fix CI

  a8nova

  committed
  0059e3e

• Add back accidently removed line

  a8nova

  committed
  4b153f5

• Remove torch-specific stuff from the TF test file

  

  Rocketknight1

  authored and

  a8nova

  committed
  f4ef81e

• make fix-copies, make style, remove autotranslated files

  

  Rocketknight1

  authored and

  a8nova

  committed
  f446d48

• Fixes to imports/docstrings

  

  Rocketknight1

  authored and

  a8nova

  committed
  2bdd087

• Let's try the from future import in desperation

  

  Rocketknight1

  authored and

  a8nova

  committed
  bf1bbaf

• Fix the core random_attention_mask fn to match the torch/flax behaviour

  

  Rocketknight1

  authored and

  a8nova

  committed
  3ba416b

• Clean random_attention_mask up correctly

  

  Rocketknight1

  authored and

  a8nova

  committed
  576699f

• Remove torch-only test

  

  Rocketknight1

  authored and

  a8nova

  committed
  a25b241

• Fix loss shape, couple of nits

  

  Rocketknight1

  authored and

  a8nova

  committed
  e1caf35

• make style

  

  Rocketknight1

  authored and

  a8nova

  committed
  3b1ea02

• Don't test for OOB embeddings because IDEFICS uses those deliberately

  

  Rocketknight1

  authored and

  a8nova

  committed
  c8dd00c

• Fix loss computation to handle masking

  

  Rocketknight1

  authored and

  a8nova

  committed
  bf16b5e

• Fix test failures when flattening

  a8nova

  committed
  5709929

• Fix some test failures

  Show description for 7bd30ea
  a8nova

  committed
  7bd30ea

• Add a proper stateless scaled_dot_product_attention

  

  Rocketknight1

  authored and

  a8nova

  committed
  a2178ec

• make style

  

  Rocketknight1

  authored and

  a8nova

  committed
  f195048

• Adding missing attribute from the PyTorch version

  

  Rocketknight1

  authored and

  a8nova

  committed
  5de955a

• Small cleanups to decoupledlinearlayer in case that helps

  

  Rocketknight1

  authored and

  a8nova

  committed
  3b95a14

• Pass epsilon to LayerNormalization

  a8nova

  committed
  2d7199a

• Attemp to fix pytorch weight cross-loading for TFIdeficsEmbedding

  a8nova

  committed
  34e866d

• Fix a bug in TFIdeficsGatedCrossAttentionLayer

  a8nova

  committed
  1aba914

• Patching up build() methods

  

  Rocketknight1

  authored and

  a8nova

  committed
  18ae095

• Constant self.inv_freq

  

  Rocketknight1

  authored and

  a8nova

  committed
  16a3274

• Constant self.inv_freq

  

  Rocketknight1

  authored and

  a8nova

  committed
  1fb31d6

• First working version

  Show description for 3ae9fcc
  a8nova

  committed
  3ae9fcc

• Fix some test failures

  a8nova

  committed
  0fd2639

• remove print statement

  a8nova

  committed
  e767798

• Fix return_tensors

  a8nova

  committed
  b2da9c2

• Fix CI test failure check_code_quality

  a8nova

  committed
  29f102c

• Attempt to fix CI failures by running make fixup

  Show description for fdc4d2a
  a8nova

  committed
  fdc4d2a

• Attempt to fix tests_pr_documentation_tests

  a8nova

  committed
  7a374b0

• Fix a test failure in test_image_processing_idefics.py

  a8nova

  committed
  8e9c5b5

• Fix test test_pt_tf_model_equivalence

  a8nova

  committed
  ac96b55

• Fix a few failures

  a8nova

  committed
  834b37f

• Tiny fix

  a8nova

  committed
  3393ccb

• Some minor fixes

  a8nova

  committed
  d075840

• Remove a duplicate test

  a8nova

  committed
  bb23c7c

• Override a few test failures for IDEFICS

  Show description for 92f78c8
  a8nova

  committed
  92f78c8

• Fix processing_idefics.py after rebase

  a8nova

  committed
  447fb88

• Guard import keras with is_tf_available

  a8nova

  committed
  9c548a1

• fix check code quality

  a8nova

  committed
  100592b

• fix check code quality

  a8nova

  committed
  7f72162

• Minor fixes

  a8nova

  committed
  f4913ef

• Skip test_save_load temporarily

  Show description for 596e06d
  a8nova

  committed
  596e06d

• Run ruff format tests src utils

  a8nova

  committed
  ac9e72c

• Fix last failing test, test_compile_tf_model

  a8nova

  committed
  77a779f

• Add fixes for vision_tf.py

  Show description for e99fa82
  a8nova

  committed
  e99fa82

• Minor fixes

  a8nova

  committed
  c45a780

• Replace "<<<" with "<<" for doc tests

  Show description for 0e59b95
  a8nova

  committed
  0e59b95

• Make code more readable

  a8nova

  committed
  81232e9

• Fix bug after code review

  Show description for 8ddb167
  a8nova

  committed
  8ddb167

• Fix after code review

  Show description for 3259268
  a8nova

  committed
  3259268

• Keep PyTorch as the default return_tensors

  a8nova

  committed
  67bd686

• Fixes to modeling_tf after code review

  a8nova

  committed
  5502db4

• Fixes from code review

  Show description for 4b6084f
  a8nova

  committed
  4b6084f

• Run ruff

  a8nova

  committed
  1fbae25

• Undo a change

  a8nova

  committed
  601100d

• Refactor processing code after Matt's suggestion

  a8nova

  committed
  4483691

• Remove TODO's that aren't needed anymore

  a8nova

  committed
  832b2cd

• For pytorch, Use original pytorch processing code from main

  Show description for ecbb417
  a8nova

  committed
  ecbb417

• Update tests/models/idefics/test_modeling_idefics.py

  

  Rocketknight1

  authored and

  a8nova

  committed
  c01e5a0

• Update tests/models/idefics/test_modeling_tf_idefics.py

  

  Rocketknight1

  authored and

  a8nova

  committed
  29490c3

• Add missing imports for is_pt_tf_cross_test

  

  Rocketknight1

  authored and

  a8nova

  committed
  c2c097d

• [DO NOT MERGE]: This is a commit for debugging and will be reverted

  Show description for 6179fe8
  a8nova

  committed
  6179fe8

• Revert "[DO NOT MERGE]: This is a commit for debugging and will be reverted"

  Show description for c7ddd5b
  a8nova

  committed
  c7ddd5b

• [DO NOT MERGE]: This commit is for debugging a CI failure and will be reverted

  a8nova

  committed
  c6bcbd9

• [DO NOT MERGE]: This commit is for debugging a CI failure and will be reverted

  a8nova

  committed
  a93bbe8

• Revert "[DO NOT MERGE]: This commit is for debugging a CI failure and will be reverted"

  Show description for 2e75279
  a8nova

  committed
  2e75279

• Revert "[DO NOT MERGE]: This commit is for debugging a CI failure and will be reverted"

  Show description for e5e6200
  a8nova

  committed
  e5e6200

• Don't skip test_save_load

  Show description for b51b2f1
  a8nova

  committed
  b51b2f1

• Debugging commit, will be reverted

  a8nova

  committed
  19c7cc2

• Revert "Debugging commit, will be reverted"

  Show description for 63d44e5
  a8nova

  committed
  63d44e5

• Override test_save_load and push model to save

  Show description for fd76004
  a8nova

  committed
  fd76004

• pass my repo_id

  a8nova

  committed
  8af771b

• add endpoint

  a8nova

  committed
  23878f1

• Pass a temp (write) token just for this CI

  a8nova

  committed
  f11e065

• Undo last few commits, still pushing to hub for model debugging

  Show description for 8963dba
  a8nova

  committed
  8963dba

• Add logging to modeling tf utils, will be reverted just for debugging

  a8nova

  committed
  779ccda

• Debugging, will revert

  a8nova

  committed
  5dadf2e

• Revert "Debugging, will revert"

  Show description for f13bded
  a8nova

  committed
  f13bded

• Revert "Add logging to modeling tf utils, will be reverted just for debugging"

  Show description for bda9fd8
  a8nova

  committed
  bda9fd8

• Remove test_save_load

  Show description for c2b3da8
  a8nova

  committed
  c2b3da8

• Run make fix-copies

  a8nova

  committed
  f492a0c

• Run ruff format tests src utils

  a8nova

  committed
  6ef4954

Commits on May 12, 2024

• Debugging commit, will be reverted

  a8nova

  committed
  4f27ec0

• Run ruff, also trigger CI run

  a8nova

  committed
  c09ba32

• Run ruff again

  a8nova

  committed
  8c8a879

• Undo debugging commit

  a8nova

  committed
  59c5f56