Skip to content

Commit

Permalink
Merge pull request #109 from CDU-data-science-team/107_v6_framework
Browse files Browse the repository at this point in the history 
107 v6 framework
  • Loading branch information
@yiwen-h
yiwen-h authored Jun 13, 2023
2 parents e078e12 + 71d7bd8 commit 60a1f29
Show file tree
Hide file tree
Showing 7 changed files with 279 additions and 162 deletions.
66 changes: 0 additions & 66 deletions current_best_multilabel/bert_additional_features.txt
View file

This file was deleted.

123 changes: 57 additions & 66 deletions current_best_multilabel/bert_minorcats.txt → ...ent_best_multilabel/v6_framework/bert.txt
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@

*****************
Random state seed for train test split is: 982
Random state seed for train test split is: 42


Model: "BERT_MultiLabel"
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
Expand All @@ -28,79 +28,70 @@ ________________________________________________________________________________
concatenate (Concatenate) (None, 778) 0 ['pooled_output[0][0]',
'dense[0][0]']

output (Dense) (None, 53) 41287 ['concatenate[0][0]']
output (Dense) (None, 44) 34276 ['concatenate[0][0]']

==================================================================================================
Total params: 66,404,207
Trainable params: 66,404,207
Total params: 66,397,196
Trainable params: 66,397,196
Non-trainable params: 0
__________________________________________________________________________________________________


Training time: 8:13:44
Training time: 9:11:31

exact_accuracy: 0.5399855907780979
hamming_loss: 0.016142406611929748
macro_jaccard_score: 0.29789067724914653
exact_accuracy: 0.5655571635311143
hamming_loss: 0.018300223654782264
macro_jaccard_score: 0.43808067429481296

Classification report:
precision recall f1-score support

Gratitude/ good experience 0.88 0.67 0.76 502
Negative experience 0.13 0.34 0.19 35
Not assigned 0.37 0.40 0.38 83
Organisation & efficiency 0.50 0.71 0.59 84
Funding & use of financial resources 0.44 0.67 0.53 6
Collecting patients feedback 0.27 0.27 0.27 11
Non-specific praise for staff 0.69 0.62 0.66 224
Non-specific dissatisfaction with staff 0.00 0.00 0.00 8
Staff manner & personal attributes 0.93 0.82 0.87 844
Number & deployment of staff 0.91 0.61 0.73 33
Staff responsiveness 0.33 0.40 0.37 89
Staff continuity 0.40 0.38 0.39 16
Competence & training 0.41 0.36 0.38 83
Unspecified communication 1.00 0.52 0.68 25
Staff listening, understanding & involving patients 0.77 0.62 0.69 208
Information directly from staff during care 0.85 0.69 0.76 201
Information provision & guidance 0.52 0.57 0.54 49
Being kept informed, clarity & consistency of information 0.48 0.16 0.24 80
Service involvement with family/ carers 0.18 0.25 0.21 12
Patient contact with family/ carers 0.80 0.44 0.57 9
Contacting services 0.53 0.59 0.55 70
Appointment arrangements 0.64 0.66 0.65 172
Appointment method 0.52 0.52 0.52 21
Timeliness of care 0.69 0.63 0.66 258
Supplying & understanding medication 0.86 0.72 0.78 43
Pain management 0.17 0.12 0.14 8
Diagnosis & triage 0.38 0.26 0.31 35
Referals & continuity of care 0.47 0.20 0.28 40
Length of stay/ duration of care 0.00 0.00 0.00 5
Admission 0.00 0.00 0.00 2
Discharge 0.75 0.19 0.30 16
Care plans 0.00 0.00 0.00 10
Patient records 0.00 0.00 0.00 6
Impact of treatment/ care 0.39 0.21 0.27 76
Links with non-NHS organisations 0.00 0.00 0.00 7
Cleanliness, tidiness & infection control 0.90 0.68 0.78 41
Sensory experience of environment 0.60 0.50 0.55 18
Comfort of environment 0.80 0.20 0.32 20
Atmosphere of ward/ environment 0.38 0.14 0.21 21
Privacy 0.00 0.00 0.00 2
Safety & security 1.00 0.07 0.13 14
Provision of medical equipment 0.50 0.23 0.32 13
Food & drink provision & facilities 0.91 0.92 0.91 63
Service location 0.71 0.21 0.32 24
Transport to/ from services 0.62 0.43 0.51 47
Parking 0.90 0.75 0.82 12
Activities & access to fresh air 0.77 0.38 0.51 26
Electronic entertainment 0.60 0.38 0.46 8
Feeling safe 0.50 0.22 0.31 9
Patient appearance & grooming 0.00 0.00 0.00 2
Equality, Diversity & Inclusion 0.00 0.00 0.00 8
Mental Health Act 0.33 0.12 0.18 8
Labelling not possible 1.00 1.00 1.00 224
Not assigned 0.70 0.45 0.55 89
Organisation & efficiency 0.49 0.82 0.61 88
Funding & use of financial resources 0.53 0.89 0.67 19
Staff manner & personal attributes 0.91 0.88 0.90 1120
Number & deployment of staff 0.60 0.85 0.70 59
Staff responsiveness 0.41 0.52 0.46 109
Staff continuity 0.38 0.12 0.19 24
Competence & training 0.74 0.51 0.61 111
Unspecified communication 0.76 0.76 0.76 29
Staff listening, understanding & involving patients 0.61 0.74 0.67 261
Information directly from staff during care 0.79 0.79 0.79 245
Information provision & guidance 0.52 0.52 0.52 64
Being kept informed, clarity & consistency of information 0.59 0.49 0.53 119
Contacting services 0.75 0.58 0.66 79
Appointment arrangements 0.56 0.77 0.65 203
Appointment method 0.66 0.62 0.64 34
Timeliness of care 0.72 0.70 0.71 380
Pain management 0.78 0.44 0.56 16
Diagnosis & triage 0.50 0.20 0.28 46
Referals & continuity of care 0.28 0.28 0.28 57
Discharge 0.70 0.26 0.38 27
Care plans 1.00 0.10 0.18 10
Patient records 0.71 0.36 0.48 14
Cleanliness, tidiness & infection control 0.88 0.71 0.79 65
Provision of medical equipment 1.00 0.18 0.31 22
Service location 0.79 0.43 0.56 35
Transport to/ from services 0.62 0.62 0.62 60
Parking 1.00 1.00 1.00 17
Electronic entertainment 0.88 0.64 0.74 11
Feeling safe 0.67 0.86 0.75 14
Patient appearance & grooming 1.00 0.40 0.57 5
Mental Health Act 0.67 0.20 0.31 10
Equality, Diversity & Inclusion 0.58 0.38 0.46 29
Admission 1.00 0.11 0.20 18
Labelling not possible 1.00 1.00 1.00 274
Environment & Facilities 0.72 0.65 0.68 117
Supplying & understanding medication 0.71 0.65 0.68 49
Activities & access to fresh air 0.89 0.62 0.73 63
Food & drink provision & facilities 0.89 0.91 0.90 96
Sensory experience 0.66 0.66 0.66 41
Impact of treatment/ care 0.48 0.33 0.39 90
Interaction with family/ carers 0.64 0.27 0.38 33
Negative experience & dissatisfaction 0.12 0.45 0.19 38
Positive experience & gratitude 0.88 0.68 0.77 875

micro avg 0.73 0.63 0.68 3931
macro avg 0.51 0.37 0.41 3931
weighted avg 0.74 0.63 0.67 3931
samples avg 0.74 0.69 0.69 3931
micro avg 0.74 0.71 0.72 5165
macro avg 0.70 0.55 0.58 5165
weighted avg 0.77 0.71 0.73 5165
samples avg 0.76 0.75 0.74 5165
Loading

0 comments on commit 60a1f29

Please sign in to comment.