-
Notifications
You must be signed in to change notification settings - Fork 260
/
chapter_13_old.html
577 lines (458 loc) · 19.9 KB
/
chapter_13_old.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
<!DOCTYPE html>
<html lang="" xml:lang="">
<head>
<title>chapter_13.knit</title>
<meta charset="utf-8" />
<meta name="author" content="" />
<script src="libs/header-attrs-2.21/header-attrs.js"></script>
<link href="libs/remark-css-0.0.1/default.css" rel="stylesheet" />
<link rel="stylesheet" href="css/Font_Style.css" type="text/css" />
</head>
<body>
<textarea id="source">
class: center, middle
<span style="font-size: 60px;">第十三章</span> <br>
<span style="font-size: 50px;">效应量和元分析 <br>
<br>
<br>
<span style="font-size: 30px;">胡传鹏</span> <br>
<span style="font-size: 30px;">2023/05/29</span> <br>
---
# <h1 lang="zh-CN">什么是效应量</h1>
**效应量**: 研究者感兴趣的任何效应的量(Cumming, 2012, *Psych Sci*)。<br>
<br>
标准化的效应量可以分为三大家族：<br>
- *d*-family（difference family）：如Cohen's *d*, Hedges' *g*
- *r*-family（correlation family）：如Pearson r, `\(R^2\)`, `\(\eta^2\)`, `\(\omega^2\)`, & *f*
- *OR*-family（categorical family）：如odds ratio (OR), risk ratio (RR)
---
# <h1 lang="zh-CN">什么是效应量</h1>
.center[<img src="./picture/chp13/power1.png" width="40%">]
- Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. *Frontiers in Psychology, 4*, 863.
---
# <h1 lang="zh-CN">什么是效应量</h1>
.center[<img src="./picture/chp13/power.png" width="90%">]
- Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. *Frontiers in Psychology, 4*, 863.
---
# <h1 lang="zh-CN">效应量与置信区间</h1>
SPSS: 偏eta squared ( `\(\eta^{2}_{p}\)` )
<br>
JASP: Cohen's *d*, `\(\eta^{2}_{p}\)`, `\(\eta^{2}_{G}\)`
<br>
Lakens (2013): 基于 excel 的计算程序 <br>
G\*power (Faul, et al., 2009) 可用于计算效应量, 但与SPSS输出的偏eta squared不相同, 使用时需要转换 (Lakens, 2013)
<br>
- Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G\*Power 3.1: Tests for correlation and regression analyses. *Behavior Research Methods, 41*(4), 1149–1160.
- Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. *Frontiers in Psychology, 4*, 863.
---
# <h1 lang="zh-CN">效应量与置信区间</h1>
<br>
## 点估计与区间估计
<br>
**基于估计的统计**被认为是一种对虚无假设的补充, 但实际上反映了推断统计的不同取向, 最重要的是从二分的思维方式转向量化的思维方式 (Cumming, 2012) <br>
在效应量的解读中, 没有固定的标准来判断一个效应量是大还是小。<br>
Richard, Bond, & Stokes-Zoota (2003) 对100多年来社会心理学研究中的元分析进行分析, 发现社会心理学研究中的平均效应量大约为*r* = 0.21, Cohen's *d* = 0.45。<br>
总的来说, 对效应量以及其置信区间的解读需要根据自己研究的实际情况来进行, 没有像NHST 中那样简单的二分标准(Cumming, 2012)。<br>
- Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York: Routledge.
- Richard, F. D., Bond, C. F., Jr., & Stokes-Zoota, J. J. (2003). One hundred years of social psychology quantitatively described. *Review of General Psychology, 7*(4), 331–363.
---
# <h1 lang="zh-CN">效应量与置信区间</h1>
<br>
## 如何解读效应量
<br>
.center[<img src="./picture/chp13/Avoid_Cohen.png" width="100%">]
https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(19)30297-9
---
# <h1 lang="zh-CN">效应量与置信区间</h1>
.center[<img src="./picture/chp13/CI_dance.jpg" width="100%">]
<br>
https://rpsychologist.com/d3/ci/
---
# <h1 lang="zh-CN">*t*检验中的*d*</h1>
<br>
## 基本公式: 独立样本*t*-test [Cohen's *d* for *sample*]
<br>
`$$Cohen's \ d_s = \frac{X_1 - X_2}{\sqrt{SD_{pool}}} = \frac{X_1 - X_2}{\sqrt{\frac{(n_1 -1)SD_1^2 + (n_2-1)SD_2^2)}{n_1+n2-2}}}$$`
## Hedges's *g*:
$$Hedges's \ g_s = Cohen's \ d_s \times (1 - \frac{3}{4(n_1 + n_2) - 9}) $$
---
# <h1 lang="zh-CN">*t*检验中的*d*</h1>
<br>
## 配对样本*t*-test: Cohen's `\(d_{rm}\)`
<br>
$$Cohen's \ d_{rm} = \frac{M_{diff}}{\sqrt{SD_1^2 + SD_2^2 -2 \times r \times SD_1 \times SD_2}} \times \sqrt{2(1-r)} $$
## 配对样本*t*-test: Cohen's `\(d_{av}\)`
$$Cohen's \ d_{av} = \sqrt{ \frac {M_{diff}} {(SD_1 + SD_2)/2}} $$
---
# <h1 lang="zh-CN">计算Cohen's *d* </h1>
<br>
载入数据
```r
rm(list = ls())
if (!requireNamespace("pacman", quietly = TRUE)) {
install.packages("pacman") } # # 检查是否已安装 pacman, 如果未安装，则安装包
pacman::p_load("tidyverse", "easystats") # 使用p_load来载入需要的包
df.mt.raw <- read.csv('./data/match/match_raw.csv', # load data:
header = T, sep=",", stringsAsFactors = FALSE)
```
---
# <h1 lang="zh-CN">计算Cohen's *d* </h1>
<br>
清理数据
```r
# from chapter 11, Chunk 3
df.mt.rt.subj <- df.mt.raw %>%
dplyr::filter(ACC == 1 & RT > 0.2) %>%
tidyr::extract(Shape, into = c("Valence", "Identity"),
regex = "(moral|immoral)(Self|Other)", remove = FALSE) %>%
dplyr::mutate(Valence = case_when(Valence == "moral" ~ "Good",
Valence == "immoral" ~ "Bad"),
RT_ms = RT * 1000) %>%
dplyr::mutate(Valence = factor(Valence, levels = c("Good", "Bad")),
Identity = factor(Identity, levels = c("Self", "Other"))) %>%
dplyr::group_by(Sub, Match, Identity, Valence) %>%
dplyr::summarise(RT_mean = mean(RT_ms)) %>%
dplyr::ungroup()
```
```
## `summarise()` has grouped output by 'Sub', 'Match', 'Identity'. You can
## override using the `.groups` argument.
```
```r
head(df.mt.rt.subj, 5)
```
```
## # A tibble: 5 × 5
## Sub Match Identity Valence RT_mean
## <int> <chr> <fct> <fct> <dbl>
## 1 7302 match Self Good 694.
## 2 7302 match Self Bad 702.
## 3 7302 match Other Good 598.
## 4 7302 match Other Bad 666.
## 5 7302 mismatch Self Good 755.
```
---
# <h1 lang="zh-CN">计算Cohen's *d* </h1>
<br>
## 感兴趣的效应量: `Match`条件下`好我`与`好人`的平均反应时间差异
<br>
## 计算所需要的统计量：
- 每种条件下的组水平平均反应时间
- 每种条件下的组水平反应时间的SD
- 两种条件下个体水平平均反应时间的相关系数
---
# <h1 lang="zh-CN">计算Cohen's *d* </h1>
$$Cohen's \ d_{rm} = \frac{M_{diff}}{\sqrt{SD_1^2 + SD_2^2 -2 \times r \times SD_1 \times SD_2}} \times \sqrt{2(1-r)} $$
```r
# from chapter 11, Chunk 3
df.mt.rt.subj.effect <- df.mt.rt.subj %>%
dplyr::filter(Match == "match" & Valence == "Good") %>%
dplyr::group_by(Identity) %>%
dplyr::summarise(mean = mean(RT_mean),
sd = sd(RT_mean))
df.mt.rt.subj.effect.wide <- df.mt.rt.subj %>%
dplyr::filter(Match == "match" & Valence == "Good") %>%
tidyr::pivot_wider(names_from = "Identity", values_from = "RT_mean")
corr_est <- cor(df.mt.rt.subj.effect.wide$Self, df.mt.rt.subj.effect.wide$Other)
```
```r
Cohens_d_manu <- ((df.mt.rt.subj.effect$mean[1] - df.mt.rt.subj.effect$mean[2])/sqrt(df.mt.rt.subj.effect$sd[1]**2 + df.mt.rt.subj.effect$sd[2]**2 - 2*corr_est*df.mt.rt.subj.effect$sd[1]*df.mt.rt.subj.effect$sd[2]))*sqrt(2*(1-corr_est))
```
Cohen's *d* = -0.568
---
# <h1 lang="zh-CN">计算Cohen's *d* </h1>
<br>
## Calculate using an existing package:
```r
SelfOther_diff <- t.test(df.mt.rt.subj.effect.wide$Self, df.mt.rt.subj.effect.wide$Other, paired = TRUE)
effectsize::effectsize(SelfOther_diff, paired = TRUE)
```
```
## Cohen's d | 95% CI
## --------------------------
## -0.48 | [-0.79, -0.16]
```
## Be cautious about the underlying formula!!!
---
# <h1 lang="zh-CN">元分析(meta-analysis)</h1>
<br>
## 一个研究的效应是一次采样而做出的估计
<br>
## 理论上，对多次采样的估计量进行综合，可以得到更加准确地估计
<br>
## 元分析是对效应量进行综合的统计方法 <br>
## 元分析需要： <br>
- 效应量，例如相关系数*r*
- 效应量的估计误差，其计算需要样本量*n*。
- 样本量越大，取样误差越小，得到的效应量置信区间（95% CI）也就越精确。
---
# <h1 lang="zh-CN">元分析</h1>
传统的元分析模型通过效应值大小的变异的倒数对效应值进行加权(例如，标准误平方)来估计平均效应值。<br>
固定效应模型： 不考虑研究间变异，其结果在理论上不能推广到包括在模型中的研究。
随机效应模型： 考虑研究间变异，其结果在理论上可以推广到包括在模型中的研究。
<img src="./picture/chp13/weightingForm.png" width="100%">
---
# <h1 lang="zh-CN">元分析</h1>
元分析作为一种文章类型 <br>
<img src="./picture/chp13/trend.png" width="60%"> <br>
- Quintana, D. S. (2015). From pre-registration to publication: a non-technical primer for conducting a meta-analysis to synthesize correlational data. *Frontiers in psychology*, 1549.
---
# <h1 lang="zh-CN">元分析是层级模型的特例</h1>
H(L)M: Hierarchical (Linear) Modeling (Linear mixed model, multi-level model) <br>
元分析是只有level2的HLM <br>
<img src="./picture/chp13/fe.png" width="100%">
---
# <h1 lang="zh-CN">元分析是层级模型的特例</h1>
HLM: Hierarchical Linear Modeling <br>
Level 1: 效应量及其误差代替 <br>
<img src="./picture/chp13/re.png" width="100%">
---
# <h1 lang="zh-CN">元分析作为一种研究类型</h1>
- 问题驱动（即元分析用于回答什么问题）；
- 流程相对清楚，有“套路”可循；
- 可以回答重要的问题；
- 细节繁琐，耗时耗力，可重复性和规范性值得注意(刘宇等, 2021, doi: 10.1360/SSV-2021-0009)；
- 口碑可能不佳 (Ioannidis, 2016, doi: 10.1111/1468-0009.12210)；
- 国内一些领域的重要评选中不纳入考虑；
---
# <h1 lang="zh-CN">元分析与R</h1>
- 生态丰富
- 工具完善: http://cran.nexr.com/web/views/MetaAnalysis.html
- 见 Polanin, Hennessy, & Tanner-Smith. (2017)的综述 (doi: 10.3102/1076998616674)
- 心理学研究中最常用包: metafor, meta
- SEM与元分析结合：[metaSEM](https://cran.r-project.org/web/packages/metaSEM/index.html)
- 公开的电子书：https://bookdown.org/MathiasHarrer/Doing_Meta_Analysis_in_R/
---
# <h1 lang="zh-CN">元分析：核心过程演示</h1>
<br>
*注意：本演示代码可以进一步优化*
## 将44人数据分成两部分
```r
subjs <- unique(df.mt.rt.subj$Sub)
set.seed(1234)
subj_ls1 <- sample(subjs, 21)
df.mt.rt.subj.ls1 <- df.mt.rt.subj %>%
dplyr::filter(Sub %in% subj_ls1)
df.mt.rt.subj.ls2 <- df.mt.rt.subj %>%
dplyr::filter(!(Sub %in% subj_ls1))
```
---
# <h1 lang="zh-CN">元分析：核心过程演示</h1>
<br>
*注意：本演示代码可以进一步优化*
## 整理第一组的数据
```r
## effect size of group 1
df.mt.rt.subj.effect.ls1 <- df.mt.rt.subj.ls1 %>%
dplyr::filter(Match == "match" & Valence == "Good") %>%
dplyr::group_by(Identity) %>%
dplyr::summarise(mean = mean(RT_mean),
sd = sd(RT_mean)) %>%
dplyr::ungroup() %>%
tidyr::pivot_wider(names_from = Identity,
values_from = c(mean, sd))
colnames(df.mt.rt.subj.effect.ls1) <- c("Self_RT_M_mean","Other_RT_M_mean",
"Self_RT_M_sd", "Other_RT_M_sd")
df.mt.rt.subj.effect.ls1.wide <- df.mt.rt.subj.ls1 %>%
dplyr::filter(Match == "match" & Valence == "Good") %>%
tidyr::pivot_wider(names_from = "Identity", values_from = "RT_mean")
corr_est.ls1 <- cor(df.mt.rt.subj.effect.ls1.wide$Self, df.mt.rt.subj.effect.ls1.wide$Other)
df.mt.rt.subj.effect.ls1$Sample_size <- length(unique(df.mt.rt.subj.ls1$Sub))
df.mt.rt.subj.effect.ls1$ri <- corr_est.ls1
```
---
# <h1 lang="zh-CN">元分析：核心过程演示</h1>
<br>
*注意：本演示代码可以进一步优化*
## 整理第二组的数据
```r
## effect size of group 2
df.mt.rt.subj.effect.ls2 <- df.mt.rt.subj.ls2 %>%
dplyr::filter(Match == "match" & Valence == "Good") %>%
dplyr::group_by(Identity) %>%
dplyr::summarise(mean = mean(RT_mean),
sd = sd(RT_mean)) %>%
dplyr::ungroup() %>%
tidyr::pivot_wider(names_from = Identity,
values_from = c(mean, sd))
colnames(df.mt.rt.subj.effect.ls2) <- c("Self_RT_M_mean","Other_RT_M_mean",
"Self_RT_M_sd", "Other_RT_M_sd")
df.mt.rt.subj.effect.ls2.wide <- df.mt.rt.subj.ls2 %>%
dplyr::filter(Match == "match" & Valence == "Good") %>%
tidyr::pivot_wider(names_from = "Identity", values_from = "RT_mean")
corr_est.ls2 <- cor(df.mt.rt.subj.effect.ls2.wide$Self, df.mt.rt.subj.effect.ls2.wide$Other)
df.mt.rt.subj.effect.ls2$Sample_size <- length(unique(df.mt.rt.subj.ls2$Sub))
df.mt.rt.subj.effect.ls2$ri <- corr_est.ls2
```
---
# <h1 lang="zh-CN">元分析：核心过程演示</h1>
<br>
## 合并数据，计算效应量和效应量的误差
```r
# and nrow with 1
df.mt.meta <- rbind(df.mt.rt.subj.effect.ls1, df.mt.rt.subj.effect.ls2)
df.es <- metafor::escalc(
measure = "SMCRH",
#standardized mean change using raw score standardization with heteroscedastic population variances at the two measurement occasions (Bonett, 2008)
m1i = Self_RT_M_mean,
m2i = Other_RT_M_mean,
sd1i = Self_RT_M_sd,
sd2i = Other_RT_M_sd,
ni = Sample_size,
ri = ri,
data = df.mt.meta
) %>%
dplyr::mutate(unique_ID = c("study1a", "study1b"))
```
---
# <h1 lang="zh-CN">使用随机效应模型进行效应量综合</h1>
```r
# 随机效果模型
rma1 <- metafor::rma(yi, vi, data = df.es)
```
---
# <h1 lang="zh-CN">森林图表格</h1>
```r
metafor::forest(rma1)
```
<!-- -->
---
# <h1 lang="zh-CN">元分析小结</h1>
- 本质上是一种效应量综述的**统计方法**
- 不同领域的看法/地位不一定，心理学相对规范性不足（与医学相比）
- 用途广泛，不仅是进行大规模的元分析进行发表论文
- mini-meta-analysis or single-paper meta-analysis在实证研究中非常有用
---
</textarea>
<style data-target="print-only">@media screen {.remark-slide-container{display:block;}.remark-slide-scaler{box-shadow:none;}}</style>
<script src="https://remarkjs.com/downloads/remark-latest.min.js"></script>
<script>var slideshow = remark.create({
"highlightStyle": "github",
"highlightLines": true,
"countIncrementalSlides": false
});
if (window.HTMLWidgets) slideshow.on('afterShowSlide', function (slide) {
window.dispatchEvent(new Event('resize'));
});
(function(d) {
var s = d.createElement("style"), r = d.querySelector(".remark-slide-scaler");
if (!r) return;
s.type = "text/css"; s.innerHTML = "@page {size: " + r.style.width + " " + r.style.height +"; }";
d.head.appendChild(s);
})(document);
(function(d) {
var el = d.getElementsByClassName("remark-slides-area");
if (!el) return;
var slide, slides = slideshow.getSlides(), els = el[0].children;
for (var i = 1; i < slides.length; i++) {
slide = slides[i];
if (slide.properties.continued === "true" || slide.properties.count === "false") {
els[i - 1].className += ' has-continuation';
}
}
var s = d.createElement("style");
s.type = "text/css"; s.innerHTML = "@media print { .has-continuation { display: none; } }";
d.head.appendChild(s);
})(document);
// delete the temporary CSS (for displaying all slides initially) when the user
// starts to view slides
(function() {
var deleted = false;
slideshow.on('beforeShowSlide', function(slide) {
if (deleted) return;
var sheets = document.styleSheets, node;
for (var i = 0; i < sheets.length; i++) {
node = sheets[i].ownerNode;
if (node.dataset["target"] !== "print-only") continue;
node.parentNode.removeChild(node);
}
deleted = true;
});
})();
// add `data-at-shortcutkeys` attribute to <body> to resolve conflicts with JAWS
// screen reader (see PR #262)
(function(d) {
let res = {};
d.querySelectorAll('.remark-help-content table tr').forEach(tr => {
const t = tr.querySelector('td:nth-child(2)').innerText;
tr.querySelectorAll('td:first-child .key').forEach(key => {
const k = key.innerText;
if (/^[a-z]$/.test(k)) res[k] = t; // must be a single letter (key)
});
});
d.body.setAttribute('data-at-shortcutkeys', JSON.stringify(res));
})(document);
(function() {
"use strict"
// Replace <script> tags in slides area to make them executable
var scripts = document.querySelectorAll(
'.remark-slides-area .remark-slide-container script'
);
if (!scripts.length) return;
for (var i = 0; i < scripts.length; i++) {
var s = document.createElement('script');
var code = document.createTextNode(scripts[i].textContent);
s.appendChild(code);
var scriptAttrs = scripts[i].attributes;
for (var j = 0; j < scriptAttrs.length; j++) {
s.setAttribute(scriptAttrs[j].name, scriptAttrs[j].value);
}
scripts[i].parentElement.replaceChild(s, scripts[i]);
}
})();
(function() {
var links = document.getElementsByTagName('a');
for (var i = 0; i < links.length; i++) {
if (/^(https?:)?\/\//.test(links[i].getAttribute('href'))) {
links[i].target = '_blank';
}
}
})();
// adds .remark-code-has-line-highlighted class to <pre> parent elements
// of code chunks containing highlighted lines with class .remark-code-line-highlighted
(function(d) {
const hlines = d.querySelectorAll('.remark-code-line-highlighted');
const preParents = [];
const findPreParent = function(line, p = 0) {
if (p > 1) return null; // traverse up no further than grandparent
const el = line.parentElement;
return el.tagName === "PRE" ? el : findPreParent(el, ++p);
};
for (let line of hlines) {
let pre = findPreParent(line);
if (pre && !preParents.includes(pre)) preParents.push(pre);
}
preParents.forEach(p => p.classList.add("remark-code-has-line-highlighted"));
})(document);</script>
<script>
slideshow._releaseMath = function(el) {
var i, text, code, codes = el.getElementsByTagName('code');
for (i = 0; i < codes.length;) {
code = codes[i];
if (code.parentNode.tagName !== 'PRE' && code.childElementCount === 0) {
text = code.textContent;
if (/^\\\((.|\s)+\\\)$/.test(text) || /^\\\[(.|\s)+\\\]$/.test(text) ||
/^\$\$(.|\s)+\$\$$/.test(text) ||
/^\\begin\{([^}]+)\}(.|\s)+\\end\{[^}]+\}$/.test(text)) {
code.outerHTML = code.innerHTML; // remove <code></code>
continue;
}
}
i++;
}
};
slideshow._releaseMath(document);
</script>
<!-- dynamically load mathjax for compatibility with self-contained -->
<script>
(function () {
var script = document.createElement('script');
script.type = 'text/javascript';
script.src = 'https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-MML-AM_CHTML';
if (location.protocol !== 'file:' && /^https?:/.test(script.src))
script.src = script.src.replace(/^https?:/, '');
document.getElementsByTagName('head')[0].appendChild(script);
})();
</script>
</body>
</html>