-
Notifications
You must be signed in to change notification settings - Fork 0
/
exp4_analysis_supplementary.Rmd
329 lines (259 loc) · 8.86 KB
/
exp4_analysis_supplementary.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
---
title: 'Experiment 4: Supplementary Analyses'
date: "`r Sys.Date()`"
output:
github_document:
toc: true
toc_depth: 3
pdf_document:
toc: true
toc_depth: 3
editor_options:
markdown:
wrap: 72
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(tidyverse)
options(dplyr.summarise.inform = FALSE)
library(magrittr)
library(lme4)
library(lmerTest)
library(buildmer)
library(insight)
library(broom.mixed)
library(kableExtra)
```
# Setup
Variable names:
- Experiment: exp4\_
- Data (\_d\_)
- d = main df
- noOther = just *male* and *female* responses
- Models (\_m\_)
- noOther = effect of Conditions (Last vs First+Full) and Name
Gender Rating, only on *male* and *female* responses
- FF = dummy coded with First + Full Name conditions as 0, Last
Name condition as 1
- L = dummy coded with Last Name condition as 0, First + Full Name
conditions as 1
- quad = quadratic effect of Name Gender
- subjGender = participant gender
- recenter= center name gender rating by scale (at 4)
Load data and select columns used in model. See data/exp4_data_about.txt
for more details.
```{r load-data}
exp4_d <- read.csv("../data/exp4_data.csv",
stringsAsFactors = TRUE) %>%
rename("Participant" = "SubjID", "Item" = "Name") %>%
select(
Participant, Condition, SubjGenderMale,
GenderRating, Item, Male, Female, Other
)
str(exp4_d)
```
Center gender rating for names: Original scale from 1 to 7, with 1 as
most masculine and 7 as most feminine. Mean-centered with higher still
as more feminine.
```{r center-gender-rating}
exp4_d %<>% mutate(GenderRatingCentered = scale(GenderRating, scale = FALSE))
```
Set contrasts for name conditions, now weighted to account for uneven
sample sizes. This uses Scott Fraundorf's function for weighted
contrasts. (The psycholing package version doesn't support doing 2v1
comparisons, only 1v1.) Condition1 is Last vs First+Full. Condition2 is
First vs Full.
```{r contrast-coding}
source("centerfactor.R")
contrasts(exp4_d$Condition) <- centerfactor(exp4_d$Condition,
c("last", "first"))
contrasts(exp4_d$Condition)
```
# Without *Other* Responses
The first supplementary analysis tests if excluding *other* responses
(2.99% of total responses) affects the pattern of results.
```{r count-other}
sum(exp4_d$Other)
sum(exp4_d$Other) / length(exp4_d$Other)
```
Exclude *other* responses.
```{r subset-other}
exp4_d_noOther <- exp4_d %>% filter(Other == 0)
```
Effect of Name Condition (first name, last name, full name) and first
name Gender Rating on likelihood of a *female* response, as opposed to a
*male* response, with *other* responses excluded. Participant and Item
are again included as random intercepts, with items defined as the
unique first, last and first + last name combinations.
```{r model-other}
exp4_m_noOther <- glmer(
Female ~ Condition * GenderRatingCentered + (1 | Participant) + (1 | Item),
data = exp4_d_noOther, family = binomial
)
summary(exp4_m_noOther)
```
Compared to the main model:
- Intercept and Condition2:GenderRatingCentered (difference between
Last Name and First+Full name conditions) potentially smaller
differences
- Condition2 now trending
## Odds Ratios: Intercept
```{r OR-intercept-other}
exp(get_intercept(exp4_m_noOther))
exp(-get_intercept(exp4_m_noOther))
```
0.84x less likely to recall as female overall (or: 1.18x more likely to
recall as male overall), p\<.05
## Odds Ratios: Last vs First+Full
```{r OR-L-FF-other}
exp4_m_noOther %>%
tidy() %>%
filter(term == "Condition1") %>%
pull(estimate) %>%
exp()
```
1.14x more likely to recall as female in First + Full compared to Last,
p\<.05
## Odds Ratios: Last Only
Dummy code with Last Name as 0, so that intercept is the Last Name
condition only.
```{r dummy-code-L-other}
exp4_d_noOther %<>% mutate(Condition_Last = case_when(
Condition == "first" ~ 1,
Condition == "full" ~ 1,
Condition == "last" ~ 0
))
exp4_d_noOther$Condition_Last %<>% as.factor()
```
```{r model-L-other}
exp4_m_noOther_L <- glmer(
Female ~ Condition_Last + (1 | Participant) + (1 | Item),
data = exp4_d_noOther, family = binomial
)
summary(exp4_m_noOther_L)
```
```{r OR-L-other}
exp(get_intercept(exp4_m_noOther_L))
exp(-get_intercept(exp4_m_noOther_L))
```
0.76x times less likely to recall as female in the Last Name condition
(or: 1.31x more likely to recall as male in the Last Name condition),
p=.17
## Odds Ratios: First and Full Only
Dummy code with First and Full Name as 0, so that intercept is average
for these two conditions.
```{r dummy-code-FF-other}
exp4_d_noOther %<>% mutate(Condition_FF = case_when(
Condition == "first" ~ 0,
Condition == "full" ~ 0,
Condition == "last" ~ 1
))
exp4_d_noOther$Condition_FF %<>% as.factor()
```
```{r model-FF-other}
exp4_m_noOther_FF <- glmer(
Female ~ Condition_FF + (1 | Participant) + (1 | Item),
data = exp4_d_noOther, family = binomial
)
summary(exp4_m_noOther_FF)
```
```{r OR-FF-other}
exp(get_intercept(exp4_m_noOther_FF))
exp(-get_intercept(exp4_m_noOther_FF))
```
0.89x less likely to recall as female in First and Full Name conditions
(or: 1.12x more likely to recall as male in First and Full Name
conditions), p=0.56
# Quadratic Name Gender Rating
The second supplementary analysis tested the effect of squared name
gender rating, such that larger values meant names with stronger gender
associations (masc or fem), and smaller values meant names with weaker
gender associations.
```{r model-quad}
exp4_d %<>% mutate(GenderRatingSquared = GenderRatingCentered^2)
exp4_m_quad <- glmer(
Female ~ Condition * GenderRatingCentered + Condition * GenderRatingSquared +
(1 | Participant) + (1 | Item),
data = exp4_d, family = binomial
)
summary(exp4_m_quad)
```
- Condition (F v F) \* Quadratic Gender Rating interaction, but n.s.
after correction for multiple comparisons, so not making a big deal
of it
# Participant Gender
## Setup/Data Summary
The third supplementary analysis looks at participant gender: if male
participants show a larger bias to recall the character as male than
non-male participants.
Participants entered their gender in a free-response box.
```{r count-subj-gender}
exp4_d %>%
group_by(SubjGenderMale) %>%
summarise(total = n_distinct(Participant)) %>%
kable()
```
For this analysis, we exclude participants who did not respond (N=91).
Because there are not enough participants to create 3 groups, we compare
male to non-male participants. Male (N=602) and transgender male (N=1)
are coded as 1, and female (N=555), nonbinary (N=3), and transgender
female (N=1) are coded as 0.
Summary of responses by condition and participant gender:
```{r means-subj-gender}
exp4_d_subjGender <- exp4_d %>%
filter(!is.na(SubjGenderMale)) %>%
mutate(ResponseAll = case_when(
Male == 1 ~ "Male",
Female == 1 ~ "Female",
Other == 1 ~ "Other"
))
exp4_d_subjGender %>%
group_by(SubjGenderMale) %>%
summarise(total = n_distinct(Participant)) %>%
kable()
```
Participant gender is mean centered effects coded, comparing non-male
participants to male participants.
```{r contrasts-subj-gender}
exp4_d_subjGender$SubjGenderMale %<>% as.factor()
contrasts(exp4_d_subjGender$SubjGenderMale) <- cbind("NM_M" = c(-.5, .5))
contrasts(exp4_d_subjGender$SubjGenderMale)
```
## Model: Condition \* Name Gender \* Participant Gender
Effects of Name Condition (first name, full name), the first name's
Gender Rating (centered, positive=more feminine), and Participant Gender
(non-male vs. male) on the likelihood of a *female* response as opposed
to *male* or *other* responses.
```{r model-subj-gender}
exp4_m_subjGender <- glmer(
Female ~ Condition * GenderRatingCentered * SubjGenderMale +
(1 | Participant) + (1 | Item),
data = exp4_d_subjGender, family = binomial
)
summary(exp4_m_subjGender)
```
- Male participants less likely to recall character as female than
non-male participants overall.
- No other interactions with participant gender significant.
# Gender Rating Centering
The first name gender ratings aren't perfectly centered, partially
because mostly-feminine/somewhat-masculine names are much less common
than mostly-masculine/somewhat-feminine names.
```{r gender-rating-mean-center}
mean(exp4_d$GenderRating, na.rm = TRUE)
```
Does it make a difference if we center it on 4, the mean of the scale,
instead of 4.21, the mean of the items?
```{r gender-rating-abs-center}
exp4_d %<>% mutate(GenderRating4 = GenderRating - 4)
```
```{r model-gender-rating-recenter}
exp4_m_recenter <- glmer(
Female ~ Condition * GenderRating4 + (1 | Participant) + (1 | Item),
data = exp4_d, family = binomial
)
summary(exp4_m_recenter)
```
Here, the beta estimate for the intercept has a larger absolute value
(-0.41 vs -0.26), and the beta estimates for the condition effects is
slightly different (0.10 vs 0.13; 0.09 vs 0.07).