Skip to content

Latest commit

 

History

History
86 lines (63 loc) · 3.33 KB

README.md

File metadata and controls

86 lines (63 loc) · 3.33 KB
Raw

A TensorFlow Implementation of NADST

It is Tensorflow version NADST training and test code repository. and not official repository Original NADST code here pytorch version. so I develop Tensorflow version.

I'm make model and test operation some difference result. and I plain continue update for code annotation.

I develop environment using python poetry

Requirements

If you not poetry framework, first install poetry

pip install poetry

environment setup -> TF_NADST Folder

poetry install
poetry shell

Dataset Download

MultiWOZ benchmark, including both version 2.0 (Link) and 2.1 (Link). Download the data and unzip into the root directory of the repo e.g. TF_NADST/data2.0 and TF_NADST/data2.1.

Scripts

I created scripts/run.sh to prepare evaluation code, train models, generate dialogue states, and evaluating the generated states with automatic metrics. You can directly run this file which includes example parameter setting:

If you run, download my pretraining code end you change -save_path= argument e.g. -save_path=save/pretraing_nadst/[downloaded model].

Pytorch Original NADST Experiment Result

No Gate Joint Acc Slot Acc F1
Use predicted fertility/no gate 48.25% 97.24% 0.8858
Use oracle fertility/no gate 70.64% 94.58% 0.9886
Gate Joint Acc Slot Acc F1
Use predicted fertility/gate 48.25% 97.24% 0.8858
Use oracle fertility/gate 70.64% 98.86% 0.9459

Our Tensorflow NADST Experiment Result

No Gate Joint Acc Slot Acc F1
Use predicted fertility/no gate 44.14% 96.88% 0.8520
Use oracle fertility/no gate 60.13% 90.19% 0.9810
Gate Joint Acc Slot Acc F1
Use predicted fertility/gate 42.14% 96.70% 0.8456
Use oracle fertility/gate 58.23% 90.04% 0.9794

Training

  • Run No Gate Training
python train.py -save_path=save/nadst -path=temp -d=256 -h_attn=16 -bsz=32 -wu=20000 -dr=0.2 -dv=2.1 -fert_dec_N=3 -state_dec_N=3 -gate=0
  • Run Gate Training
python train.py -save_path=save/nadst -path=temp -d=256 -h_attn=16 -bsz=32 -wu=20000 -dr=0.2 -dv=2.1 -fert_dec_N=3 -state_dec_N=3 -gate=1

Test

  • Run No Gate Test
python test.py  -save_path=save/nadst -path=temp -d=256 -h_attn=16 -bsz=32 -wu=20000 -dr=0.2 -dv='2.1' -fert_dec_N=3 -state_dec_N=3 -ep=1 -gate=0
  • Run Gate Test
python test.py  -save_path=save/nadst -path=temp -d=256 -h_attn=16 -bsz=32 -wu=20000 -dr=0.2 -dv='2.1' -fert_dec_N=3 -state_dec_N=3 -ep=1 -gate=1

Notes

  • Tensorflow 2.0 version code