Add conditional random fields #314

Squadrick · 2019-06-21T08:29:44Z

~~WIP till I can resolve some of the failing test cases.~~
Closes #22

Ref: tf.contrib.crf

Squadrick · 2019-06-24T13:46:21Z

The test cases were failing due to not masking the inputs to the RNN.

Squadrick · 2019-06-24T13:48:46Z

Ping @seanpmorgan @facaiy

seanpmorgan

Thanks for submitting this! Some initial comments, and then I'll get around to a more thorough review this week.

One thing that would be nice after this merges would be an example colab notebook as shown in the old README (but we could probably show more):
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/crf/README.md

tensorflow_addons/text/crf_ops.py

tensorflow_addons/text/BUILD

tensorflow_addons/text/crf_ops.py

facaiy · 2019-06-25T05:28:12Z

I don't know it. @qlzh727 Qianli, are you a good person to look at this?

cc @WindQAQ @howl-anderson @NLP-ZY who might be interested.

Squadrick · 2019-06-25T06:21:36Z

@seanpmorgan I'm making a colab notebook with usage right now, I'll open a separate PR once this merges.

howl-anderson · 2019-06-25T06:22:20Z

Since TF 2.0 encourages users using Keras API, implement CRF functions in the form of Keras layer will be a great feature. keras-contrib already has a CRF layer (https://github.com/keras-team/keras-contrib/blob/master/keras_contrib/layers/crf.py) which is pretty widely used in Keras community. We'd better consistent with it.

howl-anderson · 2019-06-25T07:14:31Z

I am working on implementing the CRF Keras layer based on Squadrick's solution. When it's ready I will post a new PR.

Squadrick · 2019-06-25T07:30:41Z

@howl-anderson Will you using any of the functions implemented here? Similar to how tf.keras.layers internally wraps tf.nn or will you doing the computation from scratch.

If it's the latter, I'll close this PR and we can proceed with our approach. If it's the latter, I'll keep this open.

P.S.: Implement your CRF layer under tfa.layers.

Squadrick · 2019-06-25T07:35:47Z

IMO, it's best to include both. The raw functions under tfa.text for those who want low level control, and an end-to-end layer under tfa.layers for easy integration with the Keras API. What do you think? Standardising the naming scheme used in both will be an important aspect.

What do y'all think? Any thoughts on this? @seanpmorgan @WindQAQ @facaiy

howl-anderson · 2019-06-25T08:38:38Z

@Squadrick CRF layer will be based on your functions (internally wraps your functions). This is a friendly feature for Keras users. low-level functions (which compatible with tf.contrib) for CRF are still very useful for advanced users and it's very important to users who migrate they code from 1.x to 2.0 and don't want to change their code on a large scale.

qlzh727

Haven't looked deep into the logic yet. Probably need to have some one compare the original implementation again this PR.

tensorflow_addons/text/crf.py

howl-anderson · 2019-06-26T03:16:38Z

Haven't looked deep into the logic yet. Probably need to have some one compare the original implementation again this PR.

@qlzh727 I have some experiences with the original implementation of CRF in the tf.contrib and familiar with how CRF works, I can check/compare this PR with the original one.

Squadrick · 2019-07-05T10:45:46Z

@qlzh727 @howl-anderson @seanpmorgan Modified the CRF code to use tf.scan for decode backward and forward alpha calculation. This makes the code slightly faster as well.

decode_forward still uses an RNN cell, but I can rewrite it to use a tf.while_loop, but it ends up being far more complicated then just using an RNN cell.

Also, transition_params will be not tracked by Keras RNN.

howl-anderson · 2019-07-07T07:27:26Z

@Squadrick Thanks for your update

Squadrick · 2019-07-12T18:16:51Z

@qlzh727 @howl-anderson Any feedback on this?

howl-anderson · 2019-07-13T06:39:39Z

@howl-anderson Don't worry, I am still working on it.

gaojie-shumei · 2019-07-18T02:39:23Z

you said the crf is in the tfa.text, but I not find it, are you sure that merge is successful??

* Rename crf_ops* -> crf* * The RNN cells inherit `AbstractRNNCell` instead of `Layer` * Remove used `training` variable * Add docstring for RNN Cells

Squadrick · 2019-07-31T21:17:42Z

@howl-anderson @guillaumekln @seanpmorgan Made the changes, this should be good to merge.

howl-anderson · 2019-08-01T05:05:30Z

@Squadrick Nice job!

seanpmorgan · 2019-08-01T13:25:33Z

Looks great @Squadrick Thanks! Going to leave this open just for a bit to test #387

seanpmorgan · 2019-08-01T15:22:18Z

Assuming this is some edge case because the sanity checks are working on other PRs. Merging. Thanks @Squadrick !

Squadrick · 2019-08-01T15:58:23Z

Haha, not sure what caused the sanity check to fail. Finally got this long running PR closed.

Thanks for the reviews, guys.

lingvisa · 2019-10-18T17:06:30Z

@howl-anderson did you have this implementation in the tf-addons in TF 2.0? like this one: https://github.com/keras-team/keras-contrib/blob/master/keras_contrib/layers/crf.py

seanpmorgan · 2019-10-18T17:57:52Z

Hi @lingvisa , we do have support for CRF, but as it stands today they are only TF functions:
https://github.com/tensorflow/addons/blob/master/tensorflow_addons/text/crf.py

We have an open PR for wrapping these around a layer, please feel free to review and let us know if it meets your expectations. At a first glance I think the keras-contrib version is much simplier and probably benefits from not have CRF functions carried over from tf.contrib. I'll make a note in that PR to consider dropping CRF functions and adopting the keras-contrib version more or less as is

Squadrick requested review from facaiy and seanpmorgan as code owners June 21, 2019 08:29

googlebot added the cla: yes label Jun 21, 2019

Squadrick added text wip Work in-progress kokoro:force-run and removed wip Work in-progress labels Jun 21, 2019

Squadrick changed the title ~~[WIP] Add conditional random fields~~ Add conditional random fields Jun 24, 2019

kokoro-team removed the kokoro:force-run label Jun 24, 2019

seanpmorgan reviewed Jun 25, 2019

View reviewed changes

tensorflow_addons/text/crf_ops.py Outdated Show resolved Hide resolved

tensorflow_addons/text/crf_ops.py Outdated Show resolved Hide resolved

tensorflow_addons/text/BUILD Outdated Show resolved Hide resolved

tensorflow_addons/text/crf_ops.py Outdated Show resolved Hide resolved

facaiy removed their request for review June 25, 2019 05:28

qlzh727 requested changes Jun 25, 2019

View reviewed changes

Squadrick closed this Jul 4, 2019

Squadrick reopened this Jul 4, 2019

seanpmorgan mentioned this pull request Jul 6, 2019

Request for example: CRF #337

Closed

Squadrick added 11 commits August 1, 2019 02:10

Port CRF from tf.contrib to tfa.text

0ef5a9e

Format using make code-format

51bebe8

Add tf.function to all the CRF functions

6909369

RNN call masks computation based on seq len

426e129

Rename files and minor fixes

d43d1ac

* Rename crf_ops* -> crf* * The RNN cells inherit `AbstractRNNCell` instead of `Layer` * Remove used `training` variable * Add docstring for RNN Cells

code format

69cc6ca

Remove unnecessary code and params

aaddd73

Replace RNN Cells with tf.scan

0b781c9

Remove @tf.function wrappers

ee052ce

Add missing docstrings

7a0435b

reformat

205ebd1

Squadrick force-pushed the crf branch from d3f1986 to 205ebd1 Compare July 31, 2019 21:04

seanpmorgan added the kokoro:force-run label Jul 31, 2019

kokoro-team removed the kokoro:force-run label Jul 31, 2019

seanpmorgan added the kokoro:force-run label Jul 31, 2019

kokoro-team removed the kokoro:force-run label Jul 31, 2019

seanpmorgan added the kokoro:force-run label Aug 1, 2019

kokoro-team removed the kokoro:force-run label Aug 1, 2019

seanpmorgan approved these changes Aug 1, 2019

View reviewed changes

seanpmorgan merged commit 47e6877 into tensorflow:master Aug 1, 2019

Squadrick deleted the crf branch August 1, 2019 19:04

gabrieldemarmiesse mentioned this pull request Oct 18, 2019

Migration of keras-team/keras-contrib to tensorflow/addon keras-team/keras-contrib#519

Open

jaspersjsun mentioned this pull request Jul 16, 2020

CRF layer v3.0 continued #1999

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add conditional random fields #314

Add conditional random fields #314

Squadrick commented Jun 21, 2019 •

edited

Loading

Squadrick commented Jun 24, 2019

Squadrick commented Jun 24, 2019

seanpmorgan left a comment

facaiy commented Jun 25, 2019

Squadrick commented Jun 25, 2019

howl-anderson commented Jun 25, 2019

howl-anderson commented Jun 25, 2019

Squadrick commented Jun 25, 2019

Squadrick commented Jun 25, 2019

howl-anderson commented Jun 25, 2019

qlzh727 left a comment

howl-anderson commented Jun 26, 2019

Squadrick commented Jul 5, 2019 •

edited

Loading

howl-anderson commented Jul 7, 2019

Squadrick commented Jul 12, 2019

howl-anderson commented Jul 13, 2019

gaojie-shumei commented Jul 18, 2019

Squadrick commented Jul 31, 2019

howl-anderson commented Aug 1, 2019

seanpmorgan commented Aug 1, 2019

seanpmorgan commented Aug 1, 2019

Squadrick commented Aug 1, 2019

lingvisa commented Oct 18, 2019

seanpmorgan commented Oct 18, 2019

Add conditional random fields #314

Add conditional random fields #314

Conversation

Squadrick commented Jun 21, 2019 • edited Loading

Squadrick commented Jun 24, 2019

Squadrick commented Jun 24, 2019

seanpmorgan left a comment

Choose a reason for hiding this comment

facaiy commented Jun 25, 2019

Squadrick commented Jun 25, 2019

howl-anderson commented Jun 25, 2019

howl-anderson commented Jun 25, 2019

Squadrick commented Jun 25, 2019

Squadrick commented Jun 25, 2019

howl-anderson commented Jun 25, 2019

qlzh727 left a comment

Choose a reason for hiding this comment

howl-anderson commented Jun 26, 2019

Squadrick commented Jul 5, 2019 • edited Loading

howl-anderson commented Jul 7, 2019

Squadrick commented Jul 12, 2019

howl-anderson commented Jul 13, 2019

gaojie-shumei commented Jul 18, 2019

Squadrick commented Jul 31, 2019

howl-anderson commented Aug 1, 2019

seanpmorgan commented Aug 1, 2019

seanpmorgan commented Aug 1, 2019

Squadrick commented Aug 1, 2019

lingvisa commented Oct 18, 2019

seanpmorgan commented Oct 18, 2019

Squadrick commented Jun 21, 2019 •

edited

Loading

Squadrick commented Jul 5, 2019 •

edited

Loading