-
Notifications
You must be signed in to change notification settings - Fork 13
/
wordbyword_attention_dropout0.2.log
11548 lines (11539 loc) · 520 KB
/
wordbyword_attention_dropout0.2.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
(root)junfeng@hadoop1:~/reasoning_attention$ python snli_reasoning_attention.py
Loading data ...
Loading train ...
550152
550149
549364
Loading dev ...
10000
10000
9842
Loading test ...
10000
10000
9824
num_epochs: 20
k: 100
batch_size: 30
display_frequency: 100
save_frequency: 1000
load previous: False
attention: True
word_by_word: True
Building network ...
unchanged_W.shape: (34283, 300)
oov_in_train_W.shape: (9166, 300)
apply dropout rate 0.2 to decoder
apply dropout mask id 139654683757200 to embedding matrix ...
dropout rate is 0.2
input var is hypo_var
apply dropout mask id 139654683757200 to embedding matrix ...
dropout rate is 0.2
input var is premise_var
Computing updates ...
Compiling functions ...
Training ...
train_df.shape: (549364, 4)
dev_df.shape: (9842, 4)
test_df.shape: (9824, 4)
Starting training...
Seen 3000 samples, time used: 70.550s
current training loss: 1.058450
current training accuracy: 0.454333
Seen 6000 samples, time used: 142.724s
current training loss: 1.028129
current training accuracy: 0.488833
Seen 9000 samples, time used: 215.414s
current training loss: 1.012946
current training accuracy: 0.503000
Seen 12000 samples, time used: 285.631s
current training loss: 1.002247
current training accuracy: 0.513500
Seen 15000 samples, time used: 358.950s
current training loss: 0.990062
current training accuracy: 0.527667
Seen 18000 samples, time used: 431.633s
current training loss: 0.981677
current training accuracy: 0.537278
Seen 21000 samples, time used: 504.303s
current training loss: 0.973090
current training accuracy: 0.544619
Seen 24000 samples, time used: 580.315s
current training loss: 0.967607
current training accuracy: 0.549458
Seen 27000 samples, time used: 654.325s
current training loss: 0.960573
current training accuracy: 0.555630
Seen 30000 samples, time used: 728.366s
current training loss: 0.957582
current training accuracy: 0.558967
saving to ..., time used 728.366s
Seen 33000 samples, time used: 801.570s
current training loss: 0.954111
current training accuracy: 0.560636
Seen 36000 samples, time used: 875.266s
current training loss: 0.950953
current training accuracy: 0.563944
Seen 39000 samples, time used: 947.427s
current training loss: 0.948558
current training accuracy: 0.566077
Seen 42000 samples, time used: 1021.095s
current training loss: 0.945443
current training accuracy: 0.567857
Seen 45000 samples, time used: 1095.418s
current training loss: 0.941933
current training accuracy: 0.570733
Seen 48000 samples, time used: 1169.017s
current training loss: 0.938352
current training accuracy: 0.573167
Seen 51000 samples, time used: 1243.517s
current training loss: 0.935981
current training accuracy: 0.574314
Seen 54000 samples, time used: 1317.343s
current training loss: 0.932651
current training accuracy: 0.576222
Seen 57000 samples, time used: 1393.935s
current training loss: 0.929052
current training accuracy: 0.578333
Seen 60000 samples, time used: 1469.266s
current training loss: 0.926869
current training accuracy: 0.580050
saving to ..., time used 740.329s
Seen 63000 samples, time used: 1543.862s
current training loss: 0.924585
current training accuracy: 0.581397
Seen 66000 samples, time used: 1618.405s
current training loss: 0.923107
current training accuracy: 0.582591
Seen 69000 samples, time used: 1694.256s
current training loss: 0.920630
current training accuracy: 0.583957
Seen 72000 samples, time used: 1768.998s
current training loss: 0.917708
current training accuracy: 0.585611
Seen 75000 samples, time used: 1845.098s
current training loss: 0.915333
current training accuracy: 0.587027
Seen 78000 samples, time used: 1920.224s
current training loss: 0.913331
current training accuracy: 0.588218
Seen 81000 samples, time used: 1996.728s
current training loss: 0.911064
current training accuracy: 0.589802
Seen 84000 samples, time used: 2073.960s
current training loss: 0.909034
current training accuracy: 0.591155
Seen 87000 samples, time used: 2149.168s
current training loss: 0.907857
current training accuracy: 0.591931
Seen 90000 samples, time used: 2223.754s
current training loss: 0.906369
current training accuracy: 0.592789
saving to ..., time used 753.934s
Seen 93000 samples, time used: 2300.643s
current training loss: 0.904891
current training accuracy: 0.594097
Seen 96000 samples, time used: 2374.767s
current training loss: 0.903432
current training accuracy: 0.595104
Seen 99000 samples, time used: 2451.684s
current training loss: 0.902138
current training accuracy: 0.595980
Seen 102000 samples, time used: 2527.388s
current training loss: 0.900729
current training accuracy: 0.597324
Seen 105000 samples, time used: 2603.019s
current training loss: 0.898827
current training accuracy: 0.598705
Seen 108000 samples, time used: 2678.314s
current training loss: 0.897753
current training accuracy: 0.599333
Seen 111000 samples, time used: 2754.624s
current training loss: 0.896715
current training accuracy: 0.599847
Seen 114000 samples, time used: 2830.629s
current training loss: 0.895394
current training accuracy: 0.600465
Seen 117000 samples, time used: 2908.376s
current training loss: 0.893256
current training accuracy: 0.601701
Seen 120000 samples, time used: 2985.160s
current training loss: 0.892140
current training accuracy: 0.602333
saving to ..., time used 760.917s
Seen 123000 samples, time used: 3061.921s
current training loss: 0.890923
current training accuracy: 0.603179
Seen 126000 samples, time used: 3139.882s
current training loss: 0.889364
current training accuracy: 0.604389
Seen 129000 samples, time used: 3216.813s
current training loss: 0.888290
current training accuracy: 0.605155
Seen 132000 samples, time used: 3293.700s
current training loss: 0.887103
current training accuracy: 0.605932
Seen 135000 samples, time used: 3370.489s
current training loss: 0.886399
current training accuracy: 0.606452
Seen 138000 samples, time used: 3447.364s
current training loss: 0.885536
current training accuracy: 0.606942
Seen 141000 samples, time used: 3523.391s
current training loss: 0.884477
current training accuracy: 0.607731
Seen 144000 samples, time used: 3598.818s
current training loss: 0.883690
current training accuracy: 0.608354
Seen 147000 samples, time used: 3675.130s
current training loss: 0.882003
current training accuracy: 0.609633
Seen 150000 samples, time used: 3750.648s
current training loss: 0.880902
current training accuracy: 0.610360
saving to ..., time used 764.986s
Seen 153000 samples, time used: 3828.365s
current training loss: 0.879758
current training accuracy: 0.611046
Seen 156000 samples, time used: 3904.438s
current training loss: 0.878782
current training accuracy: 0.611712
Seen 159000 samples, time used: 3981.737s
current training loss: 0.877662
current training accuracy: 0.612453
Seen 162000 samples, time used: 4056.862s
current training loss: 0.876627
current training accuracy: 0.613327
Seen 165000 samples, time used: 4132.630s
current training loss: 0.875626
current training accuracy: 0.613933
Seen 168000 samples, time used: 4210.021s
current training loss: 0.874510
current training accuracy: 0.614607
Seen 171000 samples, time used: 4286.664s
current training loss: 0.873347
current training accuracy: 0.615228
Seen 174000 samples, time used: 4363.062s
current training loss: 0.871881
current training accuracy: 0.616155
Seen 177000 samples, time used: 4440.542s
current training loss: 0.870728
current training accuracy: 0.617062
Seen 180000 samples, time used: 4516.914s
current training loss: 0.869758
current training accuracy: 0.617683
saving to ..., time used 765.767s
Seen 183000 samples, time used: 4592.880s
current training loss: 0.869033
current training accuracy: 0.618158
Seen 186000 samples, time used: 4671.211s
current training loss: 0.868205
current training accuracy: 0.618710
Seen 189000 samples, time used: 4748.834s
current training loss: 0.867292
current training accuracy: 0.619323
Seen 192000 samples, time used: 4826.439s
current training loss: 0.866717
current training accuracy: 0.619828
Seen 195000 samples, time used: 4904.512s
current training loss: 0.865819
current training accuracy: 0.620472
Seen 198000 samples, time used: 4981.086s
current training loss: 0.864718
current training accuracy: 0.621106
Seen 201000 samples, time used: 5057.278s
current training loss: 0.863890
current training accuracy: 0.621542
Seen 204000 samples, time used: 5132.647s
current training loss: 0.863093
current training accuracy: 0.621902
Seen 207000 samples, time used: 5209.594s
current training loss: 0.862247
current training accuracy: 0.622406
Seen 210000 samples, time used: 5286.110s
current training loss: 0.861501
current training accuracy: 0.622929
saving to ..., time used 768.582s
Seen 213000 samples, time used: 5362.617s
current training loss: 0.860481
current training accuracy: 0.623535
Seen 216000 samples, time used: 5440.070s
current training loss: 0.859284
current training accuracy: 0.624176
Seen 219000 samples, time used: 5518.801s
current training loss: 0.858481
current training accuracy: 0.624612
Seen 222000 samples, time used: 5595.570s
current training loss: 0.857809
current training accuracy: 0.625063
Seen 225000 samples, time used: 5672.717s
current training loss: 0.856960
current training accuracy: 0.625600
Seen 228000 samples, time used: 5750.543s
current training loss: 0.855970
current training accuracy: 0.626193
Seen 231000 samples, time used: 5827.890s
current training loss: 0.855197
current training accuracy: 0.626645
Seen 234000 samples, time used: 5906.121s
current training loss: 0.854365
current training accuracy: 0.627179
Seen 237000 samples, time used: 5982.841s
current training loss: 0.853599
current training accuracy: 0.627738
Seen 240000 samples, time used: 6059.667s
current training loss: 0.852604
current training accuracy: 0.628479
saving to ..., time used 773.038s
Seen 243000 samples, time used: 6137.219s
current training loss: 0.851878
current training accuracy: 0.628992
Seen 246000 samples, time used: 6214.856s
current training loss: 0.851155
current training accuracy: 0.629423
Seen 249000 samples, time used: 6292.779s
current training loss: 0.850244
current training accuracy: 0.630096
Seen 252000 samples, time used: 6370.032s
current training loss: 0.849409
current training accuracy: 0.630722
Seen 255000 samples, time used: 6448.344s
current training loss: 0.848375
current training accuracy: 0.631388
Seen 258000 samples, time used: 6525.827s
current training loss: 0.847384
current training accuracy: 0.631996
Seen 261000 samples, time used: 6602.790s
current training loss: 0.846526
current training accuracy: 0.632513
Seen 264000 samples, time used: 6678.194s
current training loss: 0.845677
current training accuracy: 0.633080
Seen 267000 samples, time used: 6754.856s
current training loss: 0.844715
current training accuracy: 0.633622
Seen 270000 samples, time used: 6832.567s
current training loss: 0.843931
current training accuracy: 0.634178
saving to ..., time used 772.379s
Seen 273000 samples, time used: 6911.022s
current training loss: 0.843138
current training accuracy: 0.634758
Seen 276000 samples, time used: 6989.312s
current training loss: 0.842343
current training accuracy: 0.635297
Seen 279000 samples, time used: 7066.756s
current training loss: 0.841583
current training accuracy: 0.635735
Seen 282000 samples, time used: 7144.451s
current training loss: 0.840777
current training accuracy: 0.636202
Seen 285000 samples, time used: 7222.840s
current training loss: 0.839991
current training accuracy: 0.636632
Seen 288000 samples, time used: 7300.611s
current training loss: 0.839102
current training accuracy: 0.637208
Seen 291000 samples, time used: 7378.999s
current training loss: 0.838377
current training accuracy: 0.637684
Seen 294000 samples, time used: 7455.955s
current training loss: 0.837620
current training accuracy: 0.638214
Seen 297000 samples, time used: 7534.368s
current training loss: 0.836800
current training accuracy: 0.638771
Seen 300000 samples, time used: 7612.794s
current training loss: 0.836119
current training accuracy: 0.639300
saving to ..., time used 779.672s
Seen 303000 samples, time used: 7690.596s
current training loss: 0.835410
current training accuracy: 0.639776
Seen 306000 samples, time used: 7771.854s
current training loss: 0.834700
current training accuracy: 0.640219
Seen 309000 samples, time used: 7850.173s
current training loss: 0.833848
current training accuracy: 0.640796
Seen 312000 samples, time used: 7930.185s
current training loss: 0.833122
current training accuracy: 0.641247
Seen 315000 samples, time used: 8010.226s
current training loss: 0.832480
current training accuracy: 0.641575
Seen 318000 samples, time used: 8092.514s
current training loss: 0.831846
current training accuracy: 0.641934
Seen 321000 samples, time used: 8171.449s
current training loss: 0.831251
current training accuracy: 0.642299
Seen 324000 samples, time used: 8252.354s
current training loss: 0.830513
current training accuracy: 0.642719
Seen 327000 samples, time used: 8331.348s
current training loss: 0.829876
current training accuracy: 0.643095
Seen 330000 samples, time used: 8411.129s
current training loss: 0.829133
current training accuracy: 0.643530
saving to ..., time used 797.739s
Seen 333000 samples, time used: 8490.465s
current training loss: 0.828562
current training accuracy: 0.643928
Seen 336000 samples, time used: 8571.024s
current training loss: 0.827978
current training accuracy: 0.644396
Seen 339000 samples, time used: 8650.506s
current training loss: 0.827388
current training accuracy: 0.644776
Seen 342000 samples, time used: 8729.620s
current training loss: 0.826704
current training accuracy: 0.645272
Seen 345000 samples, time used: 8811.936s
current training loss: 0.825998
current training accuracy: 0.645788
Seen 348000 samples, time used: 8891.614s
current training loss: 0.825274
current training accuracy: 0.646244
Seen 351000 samples, time used: 8971.370s
current training loss: 0.824640
current training accuracy: 0.646621
Seen 354000 samples, time used: 9050.490s
current training loss: 0.824112
current training accuracy: 0.646977
Seen 357000 samples, time used: 9132.854s
current training loss: 0.823348
current training accuracy: 0.647392
Seen 360000 samples, time used: 9210.575s
current training loss: 0.822844
current training accuracy: 0.647756
saving to ..., time used 798.905s
Seen 363000 samples, time used: 9288.534s
current training loss: 0.822372
current training accuracy: 0.648141
Seen 366000 samples, time used: 9367.652s
current training loss: 0.821746
current training accuracy: 0.648549
Seen 369000 samples, time used: 9446.224s
current training loss: 0.821116
current training accuracy: 0.648959
Seen 372000 samples, time used: 9525.181s
current training loss: 0.820527
current training accuracy: 0.649272
Seen 375000 samples, time used: 9603.285s
current training loss: 0.819984
current training accuracy: 0.649584
Seen 378000 samples, time used: 9683.882s
current training loss: 0.819474
current training accuracy: 0.649844
Seen 381000 samples, time used: 9762.385s
current training loss: 0.818987
current training accuracy: 0.650220
Seen 384000 samples, time used: 9839.191s
current training loss: 0.818296
current training accuracy: 0.650635
Seen 387000 samples, time used: 9918.167s
current training loss: 0.817921
current training accuracy: 0.650904
Seen 390000 samples, time used: 9995.818s
current training loss: 0.817336
current training accuracy: 0.651323
saving to ..., time used 784.710s
Seen 393000 samples, time used: 10074.369s
current training loss: 0.816814
current training accuracy: 0.651555
Seen 396000 samples, time used: 10154.144s
current training loss: 0.816189
current training accuracy: 0.651942
Seen 399000 samples, time used: 10234.721s
current training loss: 0.815564
current training accuracy: 0.652303
Seen 402000 samples, time used: 10313.530s
current training loss: 0.814815
current training accuracy: 0.652769
Seen 405000 samples, time used: 10392.349s
current training loss: 0.814213
current training accuracy: 0.653195
Seen 408000 samples, time used: 10471.245s
current training loss: 0.813462
current training accuracy: 0.653662
Seen 411000 samples, time used: 10549.904s
current training loss: 0.812966
current training accuracy: 0.653978
Seen 414000 samples, time used: 10626.159s
current training loss: 0.812332
current training accuracy: 0.654382
Seen 417000 samples, time used: 10703.324s
current training loss: 0.811765
current training accuracy: 0.654717
Seen 420000 samples, time used: 10781.662s
current training loss: 0.811159
current training accuracy: 0.655071
saving to ..., time used 785.329s
Seen 423000 samples, time used: 10862.916s
current training loss: 0.810501
current training accuracy: 0.655553
Seen 426000 samples, time used: 10942.944s
current training loss: 0.810101
current training accuracy: 0.655812
Seen 429000 samples, time used: 11024.157s
current training loss: 0.809608
current training accuracy: 0.656161
Seen 432000 samples, time used: 11104.469s
current training loss: 0.809338
current training accuracy: 0.656384
Seen 435000 samples, time used: 11185.540s
current training loss: 0.808774
current training accuracy: 0.656736
Seen 438000 samples, time used: 11268.074s
current training loss: 0.808411
current training accuracy: 0.657055
Seen 441000 samples, time used: 11348.420s
current training loss: 0.807850
current training accuracy: 0.657472
Seen 444000 samples, time used: 11430.465s
current training loss: 0.807281
current training accuracy: 0.657813
Seen 447000 samples, time used: 11506.953s
current training loss: 0.806842
current training accuracy: 0.658076
Seen 450000 samples, time used: 11584.854s
current training loss: 0.806289
current training accuracy: 0.658420
saving to ..., time used 802.709s
Seen 453000 samples, time used: 11663.175s
current training loss: 0.805796
current training accuracy: 0.658689
Seen 456000 samples, time used: 11741.102s
current training loss: 0.805197
current training accuracy: 0.659059
Seen 459000 samples, time used: 11819.311s
current training loss: 0.804648
current training accuracy: 0.659392
Seen 462000 samples, time used: 11896.642s
current training loss: 0.804072
current training accuracy: 0.659758
Seen 465000 samples, time used: 11974.515s
current training loss: 0.803673
current training accuracy: 0.660077
Seen 468000 samples, time used: 12053.498s
current training loss: 0.803240
current training accuracy: 0.660374
Seen 471000 samples, time used: 12131.072s
current training loss: 0.802600
current training accuracy: 0.660754
Seen 474000 samples, time used: 12208.327s
current training loss: 0.801929
current training accuracy: 0.661175
Seen 477000 samples, time used: 12287.774s
current training loss: 0.801393
current training accuracy: 0.661497
Seen 480000 samples, time used: 12365.440s
current training loss: 0.800831
current training accuracy: 0.661856
saving to ..., time used 780.088s
Seen 483000 samples, time used: 12444.851s
current training loss: 0.800363
current training accuracy: 0.662168
Seen 486000 samples, time used: 12523.230s
current training loss: 0.799814
current training accuracy: 0.662529
Seen 489000 samples, time used: 12602.246s
current training loss: 0.799402
current training accuracy: 0.662800
Seen 492000 samples, time used: 12682.109s
current training loss: 0.798880
current training accuracy: 0.663136
Seen 495000 samples, time used: 12763.496s
current training loss: 0.798289
current training accuracy: 0.663503
Seen 498000 samples, time used: 12846.349s
current training loss: 0.797708
current training accuracy: 0.663821
Seen 501000 samples, time used: 12923.748s
current training loss: 0.797193
current training accuracy: 0.664096
Seen 504000 samples, time used: 13001.622s
current training loss: 0.796659
current training accuracy: 0.664389
Seen 507000 samples, time used: 13081.048s
current training loss: 0.796165
current training accuracy: 0.664679
Seen 510000 samples, time used: 13160.527s
current training loss: 0.795756
current training accuracy: 0.664900
saving to ..., time used 794.591s
Seen 513000 samples, time used: 13242.257s
current training loss: 0.795261
current training accuracy: 0.665232
Seen 516000 samples, time used: 13320.550s
current training loss: 0.794617
current training accuracy: 0.665593
Seen 519000 samples, time used: 13400.905s
current training loss: 0.794118
current training accuracy: 0.665940
Seen 522000 samples, time used: 13480.283s
current training loss: 0.793723
current training accuracy: 0.666161
Seen 525000 samples, time used: 13559.521s
current training loss: 0.793136
current training accuracy: 0.666550
Seen 528000 samples, time used: 13638.899s
current training loss: 0.792717
current training accuracy: 0.666758
Seen 531000 samples, time used: 13717.249s
current training loss: 0.792280
current training accuracy: 0.667000
Seen 534000 samples, time used: 13796.649s
current training loss: 0.791761
current training accuracy: 0.667272
Seen 537000 samples, time used: 13875.270s
current training loss: 0.791341
current training accuracy: 0.667508
Seen 540000 samples, time used: 13956.190s
current training loss: 0.790862
current training accuracy: 0.667804
saving to ..., time used 795.099s
Seen 543000 samples, time used: 14035.616s
current training loss: 0.790385
current training accuracy: 0.668125
Seen 546000 samples, time used: 14114.374s
current training loss: 0.790070
current training accuracy: 0.668361
Seen 549000 samples, time used: 14193.427s
current training loss: 0.789635
current training accuracy: 0.668650
Epoch 1 of 20 took 14225.506s
training loss: 0.789572
training accuracy: 66.87 %
validation loss: 0.656260
validation accuracy: 72.55 %
test loss: 0.650373
test accuracy: 73.16 %
saving to ./snli/word_by_word_model_epoch1.npz
Seen 3000 samples, time used: 79.568s
current training loss: 0.679066
current training accuracy: 0.739000
Seen 6000 samples, time used: 159.031s
current training loss: 0.675284
current training accuracy: 0.744333
Seen 9000 samples, time used: 236.402s
current training loss: 0.677945
current training accuracy: 0.741000
Seen 12000 samples, time used: 315.357s
current training loss: 0.681474
current training accuracy: 0.739000
Seen 15000 samples, time used: 393.435s
current training loss: 0.682879
current training accuracy: 0.736467
Seen 18000 samples, time used: 470.952s
current training loss: 0.683248
current training accuracy: 0.737444
Seen 21000 samples, time used: 550.337s
current training loss: 0.681935
current training accuracy: 0.736048
Seen 24000 samples, time used: 628.854s
current training loss: 0.680633
current training accuracy: 0.736042
Seen 27000 samples, time used: 708.434s
current training loss: 0.682093
current training accuracy: 0.735481
Seen 30000 samples, time used: 787.643s
current training loss: 0.682300
current training accuracy: 0.735133
saving to ..., time used 787.643s
Seen 33000 samples, time used: 870.331s
current training loss: 0.681044
current training accuracy: 0.735576
Seen 36000 samples, time used: 952.542s
current training loss: 0.681258
current training accuracy: 0.735472
Seen 39000 samples, time used: 1034.914s
current training loss: 0.682185
current training accuracy: 0.735051
Seen 42000 samples, time used: 1114.677s
current training loss: 0.681508
current training accuracy: 0.735381
Seen 45000 samples, time used: 1194.146s
current training loss: 0.680435
current training accuracy: 0.735156
Seen 48000 samples, time used: 1275.580s
current training loss: 0.680002
current training accuracy: 0.735271
Seen 51000 samples, time used: 1356.721s
current training loss: 0.679898
current training accuracy: 0.735686
Seen 54000 samples, time used: 1437.873s
current training loss: 0.680299
current training accuracy: 0.735222
Seen 57000 samples, time used: 1517.596s
current training loss: 0.679055
current training accuracy: 0.735702
Seen 60000 samples, time used: 1599.790s
current training loss: 0.679128
current training accuracy: 0.735517
saving to ..., time used 811.651s
Seen 63000 samples, time used: 1679.250s
current training loss: 0.679130
current training accuracy: 0.735333
Seen 66000 samples, time used: 1755.508s
current training loss: 0.678999
current training accuracy: 0.735167
Seen 69000 samples, time used: 1835.625s
current training loss: 0.678349
current training accuracy: 0.735290
Seen 72000 samples, time used: 1916.657s
current training loss: 0.677087
current training accuracy: 0.735931
Seen 75000 samples, time used: 1995.637s
current training loss: 0.676646
current training accuracy: 0.735973
Seen 78000 samples, time used: 2074.104s
current training loss: 0.676273
current training accuracy: 0.736167
Seen 81000 samples, time used: 2154.345s
current training loss: 0.675894
current training accuracy: 0.736136
Seen 84000 samples, time used: 2232.449s
current training loss: 0.676019
current training accuracy: 0.736119
Seen 87000 samples, time used: 2310.719s
current training loss: 0.675159
current training accuracy: 0.736391
Seen 90000 samples, time used: 2389.234s
current training loss: 0.675206
current training accuracy: 0.736567
saving to ..., time used 788.962s
Seen 93000 samples, time used: 2471.337s
current training loss: 0.675138
current training accuracy: 0.736570
Seen 96000 samples, time used: 2552.531s
current training loss: 0.674903
current training accuracy: 0.736563
Seen 99000 samples, time used: 2632.130s
current training loss: 0.674511
current training accuracy: 0.736859
Seen 102000 samples, time used: 2713.731s
current training loss: 0.673197
current training accuracy: 0.737343
Seen 105000 samples, time used: 2795.655s
current training loss: 0.673563
current training accuracy: 0.737410
Seen 108000 samples, time used: 2877.251s
current training loss: 0.674193
current training accuracy: 0.736991
Seen 111000 samples, time used: 2955.307s
current training loss: 0.674506
current training accuracy: 0.736892
Seen 114000 samples, time used: 3036.044s
current training loss: 0.673844
current training accuracy: 0.737211
Seen 117000 samples, time used: 3116.423s
current training loss: 0.674136
current training accuracy: 0.736991
Seen 120000 samples, time used: 3198.656s
current training loss: 0.674111
current training accuracy: 0.736917
saving to ..., time used 808.864s
Seen 123000 samples, time used: 3278.335s
current training loss: 0.674009
current training accuracy: 0.736894
Seen 126000 samples, time used: 3358.683s
current training loss: 0.673946
current training accuracy: 0.737024
Seen 129000 samples, time used: 3438.598s
current training loss: 0.674182
current training accuracy: 0.737240
Seen 132000 samples, time used: 3518.821s
current training loss: 0.674068
current training accuracy: 0.737432
Seen 135000 samples, time used: 3597.767s
current training loss: 0.673723
current training accuracy: 0.737585
Seen 138000 samples, time used: 3677.113s
current training loss: 0.674091
current training accuracy: 0.737384
Seen 141000 samples, time used: 3756.337s
current training loss: 0.673969
current training accuracy: 0.737113
Seen 144000 samples, time used: 3836.030s
current training loss: 0.673851
current training accuracy: 0.737201
Seen 147000 samples, time used: 3916.255s
current training loss: 0.674049
current training accuracy: 0.737000
Seen 150000 samples, time used: 3994.687s
current training loss: 0.673459
current training accuracy: 0.737233
saving to ..., time used 795.478s
Seen 153000 samples, time used: 4074.245s
current training loss: 0.673184
current training accuracy: 0.737301
Seen 156000 samples, time used: 4152.980s
current training loss: 0.672790
current training accuracy: 0.737462
Seen 159000 samples, time used: 4233.650s
current training loss: 0.671863
current training accuracy: 0.737887
Seen 162000 samples, time used: 4311.858s
current training loss: 0.671619
current training accuracy: 0.738117
Seen 165000 samples, time used: 4391.394s
current training loss: 0.671677
current training accuracy: 0.738079
Seen 168000 samples, time used: 4471.372s
current training loss: 0.671735
current training accuracy: 0.737946
Seen 171000 samples, time used: 4548.626s
current training loss: 0.671749
current training accuracy: 0.737842
Seen 174000 samples, time used: 4628.015s
current training loss: 0.671822
current training accuracy: 0.737805
Seen 177000 samples, time used: 4708.444s
current training loss: 0.671405
current training accuracy: 0.737977
Seen 180000 samples, time used: 4788.299s
current training loss: 0.671739
current training accuracy: 0.737872
saving to ..., time used 793.143s
Seen 183000 samples, time used: 4867.838s
current training loss: 0.671602
current training accuracy: 0.737831
Seen 186000 samples, time used: 4946.710s
current training loss: 0.671485
current training accuracy: 0.737844
Seen 189000 samples, time used: 5027.273s
current training loss: 0.671437
current training accuracy: 0.737952
Seen 192000 samples, time used: 5105.786s
current training loss: 0.671488
current training accuracy: 0.737823
Seen 195000 samples, time used: 5185.052s
current training loss: 0.671580
current training accuracy: 0.737728
Seen 198000 samples, time used: 5264.344s
current training loss: 0.671214
current training accuracy: 0.737722
Seen 201000 samples, time used: 5342.224s
current training loss: 0.670890
current training accuracy: 0.737985
Seen 204000 samples, time used: 5420.703s
current training loss: 0.670605
current training accuracy: 0.738113
Seen 207000 samples, time used: 5500.098s
current training loss: 0.670796
current training accuracy: 0.737884
Seen 210000 samples, time used: 5578.501s
current training loss: 0.670460
current training accuracy: 0.737895
saving to ..., time used 789.663s
Seen 213000 samples, time used: 5658.742s
current training loss: 0.670216
current training accuracy: 0.738019
Seen 216000 samples, time used: 5737.475s
current training loss: 0.669811
current training accuracy: 0.738306
Seen 219000 samples, time used: 5816.029s
current training loss: 0.669517
current training accuracy: 0.738475
Seen 222000 samples, time used: 5895.554s
current training loss: 0.669805
current training accuracy: 0.738383
Seen 225000 samples, time used: 5972.856s
current training loss: 0.669620
current training accuracy: 0.738520
Seen 228000 samples, time used: 6053.021s
current training loss: 0.669600
current training accuracy: 0.738583
Seen 231000 samples, time used: 6131.128s
current training loss: 0.669327
current training accuracy: 0.738771
Seen 234000 samples, time used: 6211.280s
current training loss: 0.669020
current training accuracy: 0.738868
Seen 237000 samples, time used: 6290.698s
current training loss: 0.668841
current training accuracy: 0.738958
Seen 240000 samples, time used: 6370.327s
current training loss: 0.668817
current training accuracy: 0.738900
saving to ..., time used 790.844s
Seen 243000 samples, time used: 6449.738s
current training loss: 0.668865
current training accuracy: 0.738819
Seen 246000 samples, time used: 6528.201s
current training loss: 0.668542
current training accuracy: 0.738980
Seen 249000 samples, time used: 6606.957s
current training loss: 0.668114
current training accuracy: 0.739221
Seen 252000 samples, time used: 6685.405s
current training loss: 0.667811
current training accuracy: 0.739325
Seen 255000 samples, time used: 6765.476s
current training loss: 0.667699
current training accuracy: 0.739451
Seen 258000 samples, time used: 6844.847s
current training loss: 0.667585
current training accuracy: 0.739578
Seen 261000 samples, time used: 6924.023s
current training loss: 0.667282
current training accuracy: 0.739670
Seen 264000 samples, time used: 7003.112s
current training loss: 0.666964
current training accuracy: 0.739811
Seen 267000 samples, time used: 7081.713s
current training loss: 0.666729
current training accuracy: 0.739798
Seen 270000 samples, time used: 7160.041s
current training loss: 0.666610
current training accuracy: 0.739889
saving to ..., time used 789.149s
Seen 273000 samples, time used: 7240.037s
current training loss: 0.666388
current training accuracy: 0.739978
Seen 276000 samples, time used: 7320.695s
current training loss: 0.666294
current training accuracy: 0.740029
Seen 279000 samples, time used: 7401.538s
current training loss: 0.665855
current training accuracy: 0.740297
Seen 282000 samples, time used: 7479.971s
current training loss: 0.665652
current training accuracy: 0.740383
Seen 285000 samples, time used: 7560.335s
current training loss: 0.665551
current training accuracy: 0.740418
Seen 288000 samples, time used: 7638.032s
current training loss: 0.665180
current training accuracy: 0.740604
Seen 291000 samples, time used: 7716.945s
current training loss: 0.665218
current training accuracy: 0.740680
Seen 294000 samples, time used: 7794.173s
current training loss: 0.664837
current training accuracy: 0.740918
Seen 297000 samples, time used: 7874.521s
current training loss: 0.664435
current training accuracy: 0.741121
Seen 300000 samples, time used: 7953.223s
current training loss: 0.664275
current training accuracy: 0.741173
saving to ..., time used 792.624s
Seen 303000 samples, time used: 8033.258s
current training loss: 0.664025
current training accuracy: 0.741363
Seen 306000 samples, time used: 8112.643s
current training loss: 0.663911
current training accuracy: 0.741389
Seen 309000 samples, time used: 8192.786s
current training loss: 0.663691
current training accuracy: 0.741476
Seen 312000 samples, time used: 8271.426s
current training loss: 0.663433
current training accuracy: 0.741593
Seen 315000 samples, time used: 8350.894s
current training loss: 0.663261
current training accuracy: 0.741781
Seen 318000 samples, time used: 8431.060s
current training loss: 0.663058
current training accuracy: 0.741855
Seen 321000 samples, time used: 8510.702s
current training loss: 0.663136
current training accuracy: 0.741798
Seen 324000 samples, time used: 8588.399s
current training loss: 0.662876
current training accuracy: 0.741880
Seen 327000 samples, time used: 8668.020s
current training loss: 0.662757
current training accuracy: 0.742034
Seen 330000 samples, time used: 8747.261s
current training loss: 0.662413
current training accuracy: 0.742161
saving to ..., time used 793.505s
Seen 333000 samples, time used: 8827.066s
current training loss: 0.662232
current training accuracy: 0.742201
Seen 336000 samples, time used: 8905.729s
current training loss: 0.661969
current training accuracy: 0.742345
Seen 339000 samples, time used: 8986.041s
current training loss: 0.661742
current training accuracy: 0.742496
Seen 342000 samples, time used: 9064.561s
current training loss: 0.661345
current training accuracy: 0.742643
Seen 345000 samples, time used: 9143.159s
current training loss: 0.661188
current training accuracy: 0.742768
Seen 348000 samples, time used: 9221.981s
current training loss: 0.660852
current training accuracy: 0.742836
Seen 351000 samples, time used: 9301.851s
current training loss: 0.660750
current training accuracy: 0.742929
Seen 354000 samples, time used: 9381.739s
current training loss: 0.660419
current training accuracy: 0.743048
Seen 357000 samples, time used: 9460.131s
current training loss: 0.660370
current training accuracy: 0.743073
Seen 360000 samples, time used: 9537.823s
current training loss: 0.660338
current training accuracy: 0.743108
saving to ..., time used 789.962s
Seen 363000 samples, time used: 9618.025s
current training loss: 0.660220
current training accuracy: 0.743248
Seen 366000 samples, time used: 9698.343s
current training loss: 0.660106
current training accuracy: 0.743205
Seen 369000 samples, time used: 9778.935s
current training loss: 0.659805
current training accuracy: 0.743336
Seen 372000 samples, time used: 9859.925s
current training loss: 0.659593
current training accuracy: 0.743444
Seen 375000 samples, time used: 9940.859s
current training loss: 0.659508