chapter10.lyx

#LyX 2.3 created this file. For more info see http://www.lyx.org/
\lyxformat 544
\begin_document
\begin_header
\save_transient_properties true
\origin unavailable
\textclass amsbook
\use_default_options true
\begin_modules
theorems-ams
eqs-within-sections
figs-within-sections
\end_modules
\maintain_unincluded_children false
\language english
\language_package default
\inputencoding auto
\fontencoding global
\font_roman "default" "default"
\font_sans "default" "default"
\font_typewriter "default" "default"
\font_math "auto" "auto"
\font_default_family default
\use_non_tex_fonts false
\font_sc false
\font_osf false
\font_sf_scale 100 100
\font_tt_scale 100 100
\use_microtype false
\use_dash_ligatures true
\graphics default
\default_output_format default
\output_sync 0
\bibtex_command default
\index_command default
\paperfontsize default
\spacing single
\use_hyperref false
\papersize default
\use_geometry false
\use_package amsmath 1
\use_package amssymb 1
\use_package cancel 1
\use_package esint 1
\use_package mathdots 1
\use_package mathtools 1
\use_package mhchem 1
\use_package stackrel 1
\use_package stmaryrd 1
\use_package undertilde 1
\cite_engine basic
\cite_engine_type default
\biblio_style plain
\use_bibtopic false
\use_indices false
\paperorientation portrait
\suppress_date false
\justification true
\use_refstyle 1
\use_minted 0
\index Index
\shortcut idx
\color #008000
\end_index
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\paragraph_indentation default
\is_math_indent 0
\math_numbering_side default
\quotes_style english
\dynamic_quotes 0
\papercolumns 1
\papersides 1
\paperpagestyle default
\tracking_changes false
\output_changes false
\html_math_output 0
\html_css_as_file 0
\html_be_strict false
\end_header

\begin_body

\begin_layout Part*
Chapter 10: Categorical Data And Nonparametric Methods
\end_layout

\begin_layout Chapter*
10.1 Tests of Goodness-of-Fit
\end_layout

\begin_layout Standard
In some problems, we have one specific distribution in mind for the data
 we will observe.
 If that one distribution is not appropriate, we do not necessarily have
 a parametric family of alternative distributions in mind.
 In these cases and others, we can still test the null hypothesis that the
 data come from the one specific distribution against the alternative hypothesis
 that the data do not come from that distribution.
\end_layout

\begin_layout Standard
In Chapters 7,8,9, we have assumed that the observations that are available
 to the statisticaian come from dsitributionfor which the exact form is
 known, even though the values of some parameters are unknown.
 In othe words, we have assumed that the observations come from a certain
 parametric family of distributions, and a statistic inference must be made
 about the values of the parameters defining that family.
\end_layout

\begin_layout Standard
In many of the problems to be discussed in this chapter, we shall not assume
 that the the available observations come from a particular parametric family
 of distributions.
 Rather, we shall study inferences that can be made about the distribution
 from which the observations come, without making special assumptions about
 the form of that distribution.
 As one example, we might assmue that the observations form a random sample
 from a continuous distribution, without specifying the form of this distributio
n any further, and we might then investigate the 
\series bold
possibity
\series default
 that this distribution is a normal distribution.
 As a third example, we might be interested in investigating the possibility
 that two independent random samples actually come from the same distribution,
 and we might assume only that both distributions from which the samples
 are taken are continuous.
\end_layout

\begin_layout Standard
Problems in which the possible distributions of the observations are not
 restricted to a specific parametric family are called 
\series bold
nonparametric problems
\series default
, and the statistical methods that are appicable in such problems are called
 
\series bold
nonparameretric methods
\series default
.
\end_layout

\begin_layout Section*
Categorical Data
\end_layout

\begin_layout Standard
In this section and the next four sections, we shall consider statistical
 problems based on data such that each observation can be classified as
 belonging to one of a finite number of possible categories or types.
 Observations of this type are called 
\series bold
categorical data.
 
\series default
Since there are only a finite number of possible categories in these problems,
 and since we are interested in making inferences about the probabilities
 of these categories, these problems actually involve just a finite number
 of parameters.
 However, as we shall see, methods based on categorical data can be usefully
 applied in both parametric and nonparametric problems.
\end_layout

\begin_layout Section*
The 
\begin_inset Formula $\chi^{2}$
\end_inset

 Test
\end_layout

\begin_layout Standard
Suppose that a larfe population consists of items of 
\begin_inset Formula $k$
\end_inset

 different types, and let 
\begin_inset Formula $p_{i}$
\end_inset

 denote the probability that an item selected at random will be of type
 
\begin_inset Formula $i$
\end_inset

 (
\begin_inset Formula $i=1,...,k$
\end_inset

).
 Example 
\begin_inset Formula $10.1.2$
\end_inset

 is of this type with 
\begin_inset Formula $k=4$
\end_inset

.
 Of course, 
\begin_inset Formula $p_{i}\ge0$
\end_inset

 for 
\begin_inset Formula $i=1,...,k$
\end_inset

 and 
\begin_inset Formula $\sum_{i=1}^{k}p_{i}^{0}=1$
\end_inset

, and suppose that the following hypotheses are to be tested:
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{0}:p_{i}=p_{i}^{0}\text{ for }i=1,...,k,
\]

\end_inset


\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{1}:p_{i}=p_{i}^{0}\text{ for at least one value of }i\text{ (10.1.1)}
\]

\end_inset


\end_layout

\begin_layout Standard
We shall assume that a random sample of size 
\begin_inset Formula $n$
\end_inset

 is to be taken from the given population.
 That is, 
\begin_inset Formula $n$
\end_inset

 independent observations are to be taken, and there is probability 
\begin_inset Formula $p_{i}$
\end_inset

 that each observations will be of type 
\begin_inset Formula $i$
\end_inset

 
\begin_inset Formula $(i=1,...,k)$
\end_inset

.
 On the basis of these 
\begin_inset Formula $n$
\end_inset

 observations, the hypotheses (10.1.1) are to be tested.
 
\end_layout

\begin_layout Standard
For 
\begin_inset Formula $i=1,...,k$
\end_inset

, we shall let 
\begin_inset Formula $N_{i}$
\end_inset

 denote the number of observations in the random sample that are of type
 
\begin_inset Formula $i$
\end_inset

.
 Thus, 
\begin_inset Formula $N_{1},..,N_{k}$
\end_inset

 are nonnegative integers such that 
\begin_inset Formula $\sum_{i=1}^{k}N_{i}=n$
\end_inset

.
 Indeed, (
\begin_inset Formula $N_{1},...,N_{n})$
\end_inset

 has the multinomial distribution (see Sec, 5.9)
\end_layout

\begin_layout Theorem*
10.1.1, 
\begin_inset Formula $\chi^{2}$
\end_inset

 Statistic.
 The following statistic
\end_layout

\begin_layout Theorem*
\begin_inset Formula 
\[
Q=\sum_{i=1}^{k}\frac{(N_{i}-np_{i}^{0})^{2}}{np_{i}^{0}}\text{ (10.1.2)}
\]

\end_inset


\end_layout

\begin_layout Theorem*
has the property that if 
\begin_inset Formula $H_{0}$
\end_inset

is true and the sample size 
\begin_inset Formula $n\rightarrow\infty$
\end_inset

, then 
\begin_inset Formula $Q$
\end_inset

 converges in distribution to the 
\begin_inset Formula $\chi^{2}$
\end_inset

 distribution with 
\begin_inset Formula $k-1$
\end_inset

 degrees of freedom.
 (see Definition 6.3.1)
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Standard
Theorem 10.1.1 says that if 
\begin_inset Formula $H_{0}$
\end_inset

 is true and the sample size 
\begin_inset Formula $n$
\end_inset

 is large, the distribution of 
\begin_inset Formula $Q$
\end_inset

 will be approximately the 
\begin_inset Formula $\chi^{2}$
\end_inset

 distribution with 
\begin_inset Formula $k-1$
\end_inset

 degrees of freedom.
 The discussion that we have presented indicates that 
\begin_inset Formula $H_{0}$
\end_inset

 should be rejected when 
\begin_inset Formula $Q\ge c$
\end_inset

, where 
\begin_inset Formula $c$
\end_inset

 is an appropriate constant.
 If it is desired to carry out the test at the level of significance 
\begin_inset Formula $\alpha_{0}$
\end_inset

, then 
\begin_inset Formula $c$
\end_inset

 should be chosem to be the 
\begin_inset Formula $1-\alpha_{0}$
\end_inset

 quantile of 
\begin_inset Formula $\chi^{2}$
\end_inset

 distribution with 
\begin_inset Formula $k-1$
\end_inset

 degrees of freedom.
 This test is called the 
\begin_inset Formula $\chi^{2}$
\end_inset

 
\series bold
test of goodness-of-fit.
\end_layout

\begin_layout Paragraph*
Note: General form of 
\begin_inset Formula $\chi^{2}$
\end_inset

 test statistic.
 
\series medium
The form of the statistic 
\begin_inset Formula $Q$
\end_inset

 in (10.1.2) is common to all 
\begin_inset Formula $\chi^{2}$
\end_inset

 tests including those that will be introduced later in this chapter.
 The form is a sum of terms, each of which is the square of different between
 an observed count and an expected count divided by the expected count:
 
\begin_inset Formula $\sum\text{(observed-expected)}^{2}/\text{expected }$
\end_inset

.
 The expected counts are computed under the assumption that the null hypothesis
 is true.
\end_layout

\begin_layout Standard
Whenever the value of each expected count, 
\begin_inset Formula $np_{i}^{0}\ge5$
\end_inset

 for 
\begin_inset Formula $i=1,...,k$
\end_inset

, and the approximation should still be satisfactory if 
\begin_inset Formula $np_{i}^{0}\ge1.5$
\end_inset

 for 
\begin_inset Formula $i=1,...,k$
\end_inset

.
\end_layout

\begin_layout Section*
Testing Hypotheses about a Continuous Distribution
\end_layout

\begin_layout Section*
Likelihood Ratio Tests for Proportions
\end_layout

\begin_layout Chapter*
10.3.
 Contingency Tables
\end_layout

\begin_layout Standard
When each observation in our sample is a bivariate discrete random vector
 (a pair of discrete random variables), then there is a simple way to test
 the hypothesis that the two random variables are independent.
 The test is another form of 
\begin_inset Formula $\chi^{2}$
\end_inset

 test like the ones used earlier in this chapter.
\end_layout

\begin_layout Section*
Independence in Contingency Tables 
\end_layout

\begin_layout Definition*
10.3.1 Contingency Tables.
 A table in which each observation is clssified in two or more ways is called
 a contingency table.
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Standard
In Table 10.12, only two classifications are considered for each student,
 namely, the curriculum in which he is enrolled and the candidate he prefers.
 Such a table is called a two-way contingency table.
\end_layout

\begin_layout Standard
Furthermore, we shall let 
\begin_inset Formula $N_{i+}$
\end_inset

 denote the total number of individuals classified in the 
\begin_inset Formula $i$
\end_inset

th row and 
\begin_inset Formula $N_{+j}$
\end_inset

 denote the total number of individuals classified in the 
\begin_inset Formula $j$
\end_inset

th column.
\end_layout

\begin_layout Standard
Thus, 
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
N_{i+}=\sum_{j=1}^{C}N_{ij}\text{ and }N_{+j}=\sum_{i=1}^{R}N_{ij}\text{ (10.3.1)}
\]

\end_inset


\end_layout

\begin_layout Standard
Also, 
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
\sum_{i=1}^{R}\sum_{j=1}^{C}N_{ij}=\sum_{i=1}^{R}N_{i+}=\sum_{j=1}^{C}N_{+j}=n\text{ (10.3.2)}
\]

\end_inset


\end_layout

\begin_layout Standard
On the basis of these observations, the following hypotheses are to be tested:
 
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{0}:p_{ij}=p_{i+}p_{+j}\text{ for }i=1,...,R\text{ and }j=1,...,C\text{ (10.3.3)}
\]

\end_inset


\end_layout

\begin_layout Section*
\begin_inset Formula 
\[
H_{1}:\text{The hypothesis }H_{0}\text{ is not true}
\]

\end_inset

The 
\begin_inset Formula $\chi^{2}$
\end_inset

 Test of Independence 
\end_layout

\begin_layout Standard
The 
\begin_inset Formula $\chi^{2}$
\end_inset

 tests described in Sec.10.2 can be applied to the problem of testing the
 hypotheses (10.3.3).
 Each individual in the population from which the sample is taken must belong
 in one the RC cells of the contingency table.
 Under the null hypothesis 
\begin_inset Formula $H_{0}$
\end_inset

, the unknown probabilities 
\begin_inset Formula $p_{ij}$
\end_inset

 of these cells have been expressed as functions of the unknown parameters
 
\begin_inset Formula $p_{i+}$
\end_inset

 and 
\begin_inset Formula $p_{+j}$
\end_inset

.
 Since 
\begin_inset Formula $\sum_{i=1}^{R}p_{i+}=1$
\end_inset

 and 
\begin_inset Formula $\sum_{j=1}^{C}p_{+j}=1$
\end_inset

, the actual number of unknown parameters to be estimated when 
\begin_inset Formula $H_{0}$
\end_inset

 is true is 
\begin_inset Formula $s=(R-1)+(C-1)$
\end_inset

, or 
\begin_inset Formula $s=R+C-2$
\end_inset

.
\end_layout

\begin_layout Standard
For 
\begin_inset Formula $i=1,..,R$
\end_inset

 and 
\begin_inset Formula $j=1,...,C$
\end_inset

 , let 
\begin_inset Formula $\hat{E_{ij}}$
\end_inset

 denote the M.L.E., when 
\begin_inset Formula $H_{0}$
\end_inset

 is true of the expected number of observations that will be classified
 in the 
\begin_inset Formula $i$
\end_inset

th row and the 
\begin_inset Formula $j$
\end_inset

th column of the table.
 In this problem, the statistic 
\begin_inset Formula $Q$
\end_inset

 defined by Eq.
 (10.2.4) will have the following form :
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
Q=\sum_{i=1}^{R}\sum_{j-1}^{C}\frac{(N_{ij}-\hat{E}_{ij})^{2}}{\hat{E}_{ij}}\text{ (10.3.4)}
\]

\end_inset


\end_layout

\begin_layout Standard
Furthermore, since the contingency table contains RC cells, and since 
\begin_inset Formula $s=R+C-2$
\end_inset

 parameters are to be estimated when 
\begin_inset Formula $H_{0}$
\end_inset

 is true, it follows that when 
\begin_inset Formula $H_{0}$
\end_inset

 is true and 
\begin_inset Formula $n\rightarrow\infty$
\end_inset

, the c.d.f.
 of 
\begin_inset Formula $Q$
\end_inset

 converges to the c.d.f.
 of the 
\begin_inset Formula $\chi^{2}$
\end_inset

 distribution for which the number of degrees of freedom is 
\begin_inset Formula $RC-1-s=(R-1)(C-1)$
\end_inset


\end_layout

\begin_layout Standard
Next, we shall consider the form of the estimator 
\begin_inset Formula $\hat{E}_{ij}$
\end_inset

.
 The expected number of observations in the 
\begin_inset Formula $i$
\end_inset

th row and the 
\begin_inset Formula $j$
\end_inset

th column is simply 
\begin_inset Formula $np_{ij}$
\end_inset

.
 When 
\begin_inset Formula $H_{0}$
\end_inset

 is true, 
\begin_inset Formula $p_{ij}=p_{i+}p_{+j}$
\end_inset

.
 Therefore, if 
\begin_inset Formula $\hat{p}_{i+}$
\end_inset

 and 
\begin_inset Formula $\hat{p}_{+j}$
\end_inset

 then it follows that 
\begin_inset Formula $\hat{E}_{ij}=n\hat{p}_{i+}\hat{p}_{+j}$
\end_inset

.
 
\begin_inset Formula $\hat{p}_{i+}=N_{i+}/n$
\end_inset

.
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
\hat{E}_{ij}=n(\frac{N_{i+}}{n})(\frac{N_{+j}}{n})=\frac{N_{i+}N_{+j}}{n}\text{ (10.3.5)}.
\]

\end_inset


\end_layout

\begin_layout Standard
If we substitute this value of 
\begin_inset Formula $\hat{E}_{ij}$
\end_inset

 into Eq.
 (10.3.4), we can calculate the value of 
\begin_inset Formula $Q$
\end_inset

 from the observed values of 
\begin_inset Formula $N_{ij}.$
\end_inset

The null hypothesis 
\begin_inset Formula $H_{0}$
\end_inset

 should be rejected if 
\begin_inset Formula $Q\geq c$
\end_inset

, where 
\begin_inset Formula $c$
\end_inset

 is an appropriately chosen constant.
 When 
\begin_inset Formula $H_{0}$
\end_inset

 is true, and the sample size 
\begin_inset Formula $n$
\end_inset

 is large, the distribution of 
\begin_inset Formula $Q$
\end_inset

 will be approximately 
\begin_inset Formula $\chi^{2}$
\end_inset

 distribution with 
\begin_inset Formula $(R-1)(C-1)$
\end_inset

 degrees of freedom.
\end_layout

\begin_layout Section*
Summary
\end_layout

\begin_layout Standard
We learned how to test the null hypothesis that two discrete random variables
 are independent based on a random sample of 
\begin_inset Formula $n$
\end_inset

 pairs.
 First, form a contingency table of the counts for every pair of possible
 observed values.
 Then, estimate the two marginal distribution of the two random variables.
 Under the null hypothesis that the random variables are independent, the
 expected count for value 
\begin_inset Formula $i$
\end_inset

 of the first variable and value 
\begin_inset Formula $j$
\end_inset

 of the second variable in 
\begin_inset Formula $n$
\end_inset

 times the product the product of the two estimated marginal probabilites.
 We then form the 
\begin_inset Formula $\chi^{2}$
\end_inset

 statistic 
\begin_inset Formula $Q$
\end_inset

 by summing (observed-expected)
\begin_inset Formula $^{2}/$
\end_inset

 expected over all the cells in the contingency table.
 The degrees of freedom is 
\begin_inset Formula $(R-1)(C-1)$
\end_inset

, where 
\begin_inset Formula $R$
\end_inset

 is the number of rows in the table and 
\begin_inset Formula $C$
\end_inset

 is the number of columns.
\end_layout

\begin_layout Chapter*
10.4.
 Tests of Homogeneity
\end_layout

\begin_layout Standard
Imagine that we select subjects from several different populations, and
 that we observe a discrete random variable for each subject.
 We might be interested in whether or not the distribution of that discrete
 random variable is the same in each population.
 There is a 
\begin_inset Formula $\chi^{2}$
\end_inset

 test of this hypothesis that is very similar to the 
\begin_inset Formula $\chi^{2}$
\end_inset

 test of independence.
\end_layout

\begin_layout Section*
Samples from Several Populations
\end_layout

\begin_layout Standard
In general, we shall consider a problem in which random samples are taken
 from 
\begin_inset Formula $R$
\end_inset

 different populations, and each observation in each sample can be classified
 as one of 
\begin_inset Formula $C$
\end_inset

 different types.
 Thus, the data obtained from the 
\begin_inset Formula $R$
\end_inset

 samples,can be represented in an 
\begin_inset Formula $RxC$
\end_inset

 table.
 For 
\begin_inset Formula $i=1,...,R$
\end_inset

, and 
\begin_inset Formula $j=1,..,C$
\end_inset

, we shall let 
\begin_inset Formula $p_{ij}$
\end_inset

 denote the probability that an observation chosen at random from the 
\begin_inset Formula $i$
\end_inset

tj population will be of type 
\begin_inset Formula $j$
\end_inset

.
 Thus,
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
\sum_{j=1}^{C}p_{ij}=1\text{ for }i=1,...,R
\]

\end_inset


\end_layout

\begin_layout Standard
The hypotheses to be tested are as follows:
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{0}:p_{1j}=p_{2j}=...=P_{Rj}\text{ for }j=1,...,C
\]

\end_inset


\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{1}:\text{ The hypothesis }H_{0}\text{ is not true(10.4.1)}
\]

\end_inset


\end_layout

\begin_layout Standard
If the null hypothesis in (10.4.1) were true, then combining the 
\begin_inset Formula $R$
\end_inset

 populations would produce one homogeneous population with regard to the
 distribution of the random random variables we are studying.
 For this reason, a test of the hypotheses (10.4.1) is called a test of homogeneit
y of the 
\begin_inset Formula $R$
\end_inset

 distributions.
\end_layout

\begin_layout Section*
The 
\begin_inset Formula $\chi^{2}$
\end_inset

 Test of Homogeneity
\end_layout

\begin_layout Standard
We shall now develop a test procedure for the hypotheses (10.4.1).
 Suppose for the moment that the probabilities 
\begin_inset Formula $p_{ij}$
\end_inset

 are known, and consider the following statistic calculated from the observation
s in the 
\begin_inset Formula $i$
\end_inset

th random sample:
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
\sum_{j=1}^{C}\frac{(N_{ij}-N_{i+}p_{ij})^{2}}{N_{i+}p_{ij}}
\]

\end_inset


\end_layout

\begin_layout Standard
This statistic is just the standard 
\begin_inset Formula $\chi^{2}$
\end_inset

 statistic, introduced in Eq.
 (10.1.2), for the random sample of 
\begin_inset Formula $N_{i+}$
\end_inset

 observations from the 
\begin_inset Formula $ith$
\end_inset

 population.
 Therefore, when the sample size 
\begin_inset Formula $N_{i+}$
\end_inset

 is large, the distribution of this statistic will be approximately the
 
\begin_inset Formula $\chi^{2}$
\end_inset

 distribution with 
\begin_inset Formula $C-1$
\end_inset

 degrees of freedom.
\end_layout

\begin_layout Standard
If we now sum this statistic over the 
\begin_inset Formula $R$
\end_inset

 different samples, we obtain the following statistic:
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
\sum_{i=1}^{R}\sum_{j=1}^{C}\frac{(N_{ij}-N_{i+}p_{ij})^{2}}{N_{i+}p_{ij}}\text{ (10.4.2)}
\]

\end_inset


\end_layout

\begin_layout Standard
Since the observations in the 
\begin_inset Formula $R$
\end_inset

 samples are drawn independently, the distribution of the statistic (10.4.2)
 will be the distribution of the sum of 
\begin_inset Formula $R$
\end_inset

 independent random variables, each of which has approximately the 
\begin_inset Formula $\chi^{2}$
\end_inset

 distribution with 
\begin_inset Formula $C-1$
\end_inset

 degrees of freedom.
 Hence, the distribution of the statistic (10.4.2) will be approximately the
 
\begin_inset Formula $\chi^{2}$
\end_inset

 distribution with 
\begin_inset Formula $R(C-1)$
\end_inset

 degrees of freedom.
\end_layout

\begin_layout Standard
MLE of 
\begin_inset Formula $p_{ij}=\hat{p}_{ij}=N_{+j}/n$
\end_inset

.
 We obtain the statistic
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
Q=\sum_{i=1}^{R}\sum_{j=1}^{C}\frac{(N_{ij}-\hat{E}_{ij})^{2}}{\hat{E}_{ij}}\text{ (10.4.3)}
\]

\end_inset


\end_layout

\begin_layout Standard
where
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
\hat{E}_{ij}=\frac{N_{i+}N_{+j}}{n}\text{ (10.4.4)}
\]

\end_inset


\end_layout

\begin_layout Standard
It can be seen that Eqs (10.4.3) and (10.4.4) are precisely the same as Eqs
 (10.3.4) and (10.3.5).
 Thus the statistic 
\begin_inset Formula $Q$
\end_inset

 to be used for the test of homogeneity in this section is precisely the
 same as the statistic 
\begin_inset Formula $Q$
\end_inset

 to be used for the test of independence in Sec.
 10.3.
 We shall now show that the number of degrees of freedom is also precisely
 the same for the test of homogeneity as for the test of independence.
\end_layout

\begin_layout Standard
Because the distributios of the 
\begin_inset Formula $R$
\end_inset

 populations are alike when 
\begin_inset Formula $H_{0}$
\end_inset

 is true, and because 
\begin_inset Formula $\sum_{j=1}^{C}p_{ij}=1$
\end_inset

 for this common distribution, we have estimated 
\begin_inset Formula $C-1$
\end_inset

 parameters in this problem.
 Therefore, the statistic 
\begin_inset Formula $Q$
\end_inset

 will have approximately the 
\begin_inset Formula $\chi^{2}$
\end_inset

 distribution with 
\begin_inset Formula $R(C-1)-(C-1)=(R-1)(C-1)$
\end_inset

 degrees of freedom.
 this number is the same as that found in Sec.
 10.3.
\end_layout

\begin_layout Section*
Comparing Two or More Proportions
\end_layout

\begin_layout Section*
Correlated 2x2 tables
\end_layout

\begin_layout Section*
Summary
\end_layout

\begin_layout Standard
When we sample discrete random variables from several populations, we might
 be interested in the null hypothesis that the distribution of the random
 variables is the same in all populations.
 We can perform a 
\begin_inset Formula $\chi^{2}$
\end_inset

 test of this null hypothesis as follows: Create a new variable with values
 equal to the names of the different populations.
 Next, pretend as if each observation consists of the original discrete
 random variable together with the new 
\begin_inset Quotes eld
\end_inset

population name
\begin_inset Quotes erd
\end_inset

 variable.
 Finally, compute the 
\begin_inset Formula $\chi^{2}$
\end_inset

 test statistic 
\begin_inset Formula $Q$
\end_inset

 from Sec.
 10.3.
 with the same degrees of freedom.
 For the type of data considered in this section, the 
\begin_inset Quotes eld
\end_inset

population name
\begin_inset Quotes erd
\end_inset

 for each observation is known before sampling begin, and hence it is not
 a random variable.
 Whether the population name is known ahead of time or is observed as part
 of the sampled data (as in Sec.
 10.3), the mechanics of the 
\begin_inset Formula $\chi^{2}$
\end_inset

 test are the same.
\end_layout

\begin_layout Paragraph*
Note: 
\series medium
If testing based on divide (interval with normal or discrete with poisson(
 10.2.6)), we apply theorem 10.2.2 for testing with p-value k-1 and k-s-1 
\begin_inset Formula $\chi^{2}$
\end_inset

 .
 
\series default
Every parametric testing will be applying 10.2.2 -> Wrong
\end_layout

\begin_layout Standard

\series bold
Note: 
\series default
If MLE based on 
\begin_inset Formula $N_{1},...,N_{k}$
\end_inset

 then testing based on 
\begin_inset Formula $\chi_{k-s-1}^{2}$
\end_inset

.
 But if M.L.E based on sample : 
\begin_inset Formula $X_{1},...,X_{n}$
\end_inset

 ( 
\begin_inset Formula $\hat{\mu},\hat{\sigma}$
\end_inset

), we need to testing between (
\begin_inset Formula $\chi_{k-s-1}^{2}$
\end_inset

, 
\begin_inset Formula $\chi_{k-1}^{2}$
\end_inset

) (For example exercise 5 in Sec.
 10.2 - this is based on 
\begin_inset Formula $X_{i}$
\end_inset

 to compute MLE of parameter of Poisson distribution.
\end_layout

\begin_layout Chapter*
10.6.
 Kolmogorov-Smirnov Test
\end_layout

\begin_layout Standard
In Sec.
 10.1, we used the 
\begin_inset Formula $\chi^{2}$
\end_inset

 test to test the null hypothesis that a random sample came from a particular
 continuous distribution against the alternative hypothesis that the sample
 did not come from that distribution.
 A more suitable test for these hypotheses is introduced in this section.
 This test can also be extended to test the null hypothesis that two independent
 samples from the same distribution against the alternative hypothesis that
 they came from two different distributions.
\end_layout

\begin_layout Section*
The Sample Distribution Function
\end_layout

\begin_layout Standard
The first step in trying to answer in Example 10.6.1 is to construct an estimator
 of the distribution of the random sample that not rely on the assumption
 that the distribtuion was normal.
 Suppose that the random variables 
\begin_inset Formula $X_{1},..,X_{n}$
\end_inset

 form a random sample from some continuous distribution, and let 
\begin_inset Formula $x_{1},...,x_{n}$
\end_inset

 denote the observed values of 
\begin_inset Formula $X_{1},....,X_{n}$
\end_inset

.
 Sice the observations come from the conitinuous distributon, there is probabili
ty 
\begin_inset Formula $0$
\end_inset

 that any two of the observed values 
\begin_inset Formula $x_{1},...,x_{n}$
\end_inset

 will be equal.
\end_layout

\begin_layout Definition*
10.6.1.
 Sample (Empirical) Distribution Function.
 Let 
\begin_inset Formula $x_{1},..,x_{n}$
\end_inset

 be the observed values of a random sample 
\begin_inset Formula $X_{1},....,X_{n}$
\end_inset

.
 For each number 
\begin_inset Formula $x$
\end_inset

 
\begin_inset Formula $(-\infty<x<\infty)$
\end_inset

, define the value 
\begin_inset Formula $F_{n}(x)$
\end_inset

 as the proportion of observed values in the sample that are less than or
 equal to 
\begin_inset Formula $x$
\end_inset

.
 In other words, if exactly 
\begin_inset Formula $k$
\end_inset

 of the observed values in the sample are less than or equal to 
\begin_inset Formula $x$
\end_inset

, then 
\begin_inset Formula $F_{n}(x)=k/n$
\end_inset

.
 The function 
\begin_inset Formula $F_{n}(x)$
\end_inset

 defined in this way is called sample distribution function, or simply the
 sample c.d.f.
 Sometimes 
\begin_inset Formula $F_{n}(x)$
\end_inset

 is called the empirical c.d.f.
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Standard
Now let 
\begin_inset Formula $F(x)$
\end_inset

 denote the c.d.f of the distribution from which the random sample 
\begin_inset Formula $X_{1},...,X_{n}$
\end_inset

 was drawn.
 For each given number 
\begin_inset Formula $x$
\end_inset

 (
\begin_inset Formula $-\infty<x<\infty)$
\end_inset

, the probability that any particular observation 
\begin_inset Formula $X_{i}$
\end_inset

 will be less than or equal to 
\begin_inset Formula $x$
\end_inset

 is 
\begin_inset Formula $F(x)$
\end_inset

 , therefore, it follows from the law of large numbers that as 
\begin_inset Formula $n\rightarrow\infty$
\end_inset

, the proportion 
\begin_inset Formula $F_{n}(x)$
\end_inset

 of observation in the sample that are less than or equal 
\begin_inset Formula $x$
\end_inset

 will converge in the probability to 
\begin_inset Formula $F(x)$
\end_inset

.
 In symbols,
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
F_{n}(x)\rightarrow_{p}F(x)\text{ for }-\infty<x<\infty\text{ (10.6.1)}
\]

\end_inset


\end_layout

\begin_layout Standard
The relation (10.6.1) expresses the fact that at each point 
\begin_inset Formula $x$
\end_inset

, the sample c.d.f 
\begin_inset Formula $F_{n}(x)$
\end_inset

 will converge to the actual c.d.f 
\begin_inset Formula $F(x)$
\end_inset

 of the distribution from which the random sample was taken.
\end_layout

\begin_layout Theorem*
10.6.1.
 Glivenko-Cantelli Lemma.
 Let 
\begin_inset Formula $F_{n}$
\end_inset

 be the sample c.d.f.
 from an i.i.f.
 sample 
\begin_inset Formula $X_{1},...,X_{n}$
\end_inset

 from from the c.d.f 
\begin_inset Formula $F$
\end_inset

.
 Define
\end_layout

\begin_layout Theorem*
\begin_inset Formula 
\[
D_{n}=\sup_{-\infty<x<\infty}|F_{n}(x)-F(x)|\text{ (10.6.2)}.
\]

\end_inset


\end_layout

\begin_layout Theorem*
Then 
\begin_inset Formula $D_{n}\rightarrow^{p}0$
\end_inset

.
\end_layout

\begin_layout Theorem*
A value of 
\begin_inset Formula $D_{n}$
\end_inset

 is illustrated in Fig.
 10.3.
 Before the values of 
\begin_inset Formula $X_{1},...,X_{n}$
\end_inset

 have been observed, the value of 
\begin_inset Formula $D_{n}$
\end_inset

 is a random variable.
\end_layout

\begin_layout Theorem*
\begin_inset Graphics
	filename 10.6.D_n.png

\end_inset


\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Standard
Theorem 10.6.1 implies that when the sample size 
\begin_inset Formula $n$
\end_inset

 is large, the sample c.d.f.
 
\begin_inset Formula $F_{n}(x)$
\end_inset

 is quite likely to be close to the c.d.f.
 
\begin_inset Formula $F(x)$
\end_inset

 over the entire real line.
 In this sense, when the c.d.f.
 
\begin_inset Formula $F(x)$
\end_inset

 is unknown, the sample c.d.f 
\begin_inset Formula $F_{x}(x)$
\end_inset

 can be considered to be an estimator of 
\begin_inset Formula $F(x)$
\end_inset

.
 In another sense, however, 
\begin_inset Formula $F_{n}(x)$
\end_inset

 is not a very reasonable estimator of 
\begin_inset Formula $F(x)$
\end_inset

.
 As authors explained earlier, 
\begin_inset Formula $F_{n}(x)$
\end_inset

 will be the c.d.f.
 of a discrete distribution that is concentrated on 
\begin_inset Formula $n$
\end_inset

 points, whereas we are assuming in this section that the unknown c.d.f.
 
\begin_inset Formula $F(x)$
\end_inset

 is the c.d.f.
 of a continuous distribution.
 Some type of smoothed version of 
\begin_inset Formula $F_{n}(x)$
\end_inset

, from which the jumps have been removed, might yield a reasonable estimator
 of 
\begin_inset Formula $F(x)$
\end_inset

, but we shall not purse this topic further here.
\end_layout

\begin_layout Section*
The Kolmogorov-Smirnov Test of a Simple Hypothesis
\end_layout

\begin_layout Standard
Suppose now that we wish to test the simple null hypothesis that the unknown
 c.d.f.
 
\begin_inset Formula $F(x)$
\end_inset

 is actually a particular continuos c.d.f.
 
\begin_inset Formula $F^{*}(x)$
\end_inset

 against the general alternative that the actual c.d.f.
 is not 
\begin_inset Formula $F^{*}(x)$
\end_inset

.
 In other words, syppose that we wish to test the following hypotheses:
 
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{0}:F(x)=F^{*}(x)\text{\text{ for }-\ensuremath{\infty<x<\infty}}
\]

\end_inset


\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{1}:\text{ the hypothesis }H_{0}\text{ is not true (10.6.3)}
\]

\end_inset

This problem is a nonparametric problem because the unknown distribution
 from which the random sample is taken might be any continuous distribution.
\end_layout

\begin_layout Standard
In Sec.
 10.1, we described how the 
\begin_inset Formula $\chi^{2}$
\end_inset

 test of goodness-of-fit can be used to test hypothesis having he form (10.6.3).
 That test, however, requires grouping the observations into a finite number
 of intervals in arbitrary manner.
 We shall now describe a test of the hypotheses (10.6.3) that does not require
 such grouping.
\end_layout

\begin_layout Standard
As before, we shall let 
\begin_inset Formula $F_{n}(x)$
\end_inset

 denote the sample c.d.f.
 Also, we shall now let 
\begin_inset Formula $D_{n}^{*}$
\end_inset

 denote the following statistic:
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
D_{n}^{*}=\sup_{-\infty<x<\infty}|F_{n}(x)-F^{*}(x)|\text{ (10.6.4)}
\]

\end_inset


\end_layout

\begin_layout Standard
In other words, 
\begin_inset Formula $D_{n}^{*}$
\end_inset

 is the maximum difference between the sample c.d.f.
 
\begin_inset Formula $F_{n}(x)$
\end_inset

 and the hypothesized c.d.f.
 
\begin_inset Formula $F^{*}(x)$
\end_inset

.
 When the null hypothesis 
\begin_inset Formula $H_{0}$
\end_inset

 in (10.6.3.) is true, the probability distribution of 
\begin_inset Formula $D_{n}^{*}$
\end_inset

 will be a certain distribution that is same for every possible contunuous
 c.d.f 
\begin_inset Formula $F^{*}(x)$
\end_inset

 and does not depend on the particular c.d.f.
 
\begin_inset Formula $F^{*}(x)$
\end_inset

 being studied in a specific problem.
 Tables of this distribution, for various values of sample size 
\begin_inset Formula $n$
\end_inset

, have been developed and are presented in many published collections of
 statistical tables.
 
\end_layout

\begin_layout Standard
It follows from the Glivenko-Catelli lemma that the value of 
\begin_inset Formula $D_{n}^{*}$
\end_inset

 will tend to be small if the null hypothesis 
\begin_inset Formula $H_{0}$
\end_inset

 is true, and 
\begin_inset Formula $D_{n}^{*}$
\end_inset

 will tend to be larger if the actual c.d.f.
 
\begin_inset Formula $F(x)$
\end_inset

 is different from 
\begin_inset Formula $F^{*}(x).$
\end_inset

 Therefore, a reasonable test procedure for the hypotheses (10.6.3) is to
 reject 
\begin_inset Formula $H_{0}$
\end_inset

 if 
\begin_inset Formula $n^{1/2}D_{n}^{*}>c$
\end_inset

, where 
\begin_inset Formula $c$
\end_inset

 is an appropriate constant.
\end_layout

\begin_layout Standard
It is convient to express the test procedure in terms of 
\begin_inset Formula $n^{1/2}D_{n}^{*}$
\end_inset

 rather than simple 
\begin_inset Formula $D_{n}^{*},$
\end_inset

 because of the following result
\end_layout

\begin_layout Theorem*
10.6.2.
 If the null hypothesis 
\begin_inset Formula $H_{0}$
\end_inset

 is true, then for each given value 
\begin_inset Formula $t>0$
\end_inset

,
\end_layout

\begin_layout Theorem*
\begin_inset Formula 
\[
\lim_{n\rightarrow\infty}Pr(n^{1/2}D_{n}^{*}\le t)=1-2\sum_{i=1}^{\infty}(-1)^{i-1}e^{-2i^{2}t^{2}}\text{ (10.6.5)}
\]

\end_inset


\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Standard
Thus, if the null hypothesis 
\begin_inset Formula $H_{0}$
\end_inset

 is true, then as 
\begin_inset Formula $n\rightarrow\infty$
\end_inset

, the c.d.f.
 of 
\begin_inset Formula $n^{1/2}D_{n}^{*}$
\end_inset

 will converge to the c.d.f.
 given by the infinite series on the right side on Eq.
 (10.6.5).
 For each value of 
\begin_inset Formula $t>0$
\end_inset

, we shall let 
\begin_inset Formula $H(t)$
\end_inset

 denote the value on the right side of Eq.
 (10.6.5).
 The values of 
\begin_inset Formula $H(t)$
\end_inset

 are given Table 10.32.
\end_layout

\begin_layout Standard
\begin_inset Graphics
	filename cdf of H.png

\end_inset


\end_layout

\begin_layout Definition*
10.6.2.
 Kolmogorov-Smirnov test.
 A test procedure that rejects 
\begin_inset Formula $H_{0}$
\end_inset

 when 
\begin_inset Formula $n^{1/2}D_{n}^{*}\ge c$
\end_inset

 is called a Kolmogorov-Smirnov test.
\end_layout

\begin_layout Standard
So in this situation, we focus on random variable 
\begin_inset Formula $n^{1/2}D_{n}^{*}$
\end_inset

, and table 10.32 show cdf of 
\begin_inset Formula $n^{1/2}D_{n}^{*}$
\end_inset

.
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Section*
The Kolmogorov-Smirnov Test for Two Samples
\end_layout

\begin_layout Standard
Consider a problem in which a random sample of 
\begin_inset Formula $m$
\end_inset

 observations 
\begin_inset Formula $X_{1},...,X_{m}$
\end_inset

 is taken from a distribution for which the c.d.f.
 
\begin_inset Formula $F(x)$
\end_inset

 is unknown, and an independent random sample of 
\begin_inset Formula $n$
\end_inset

 observations 
\begin_inset Formula $Y_{1},...,Y_{n}$
\end_inset

 is taken from another distribution for which the c.d.f.
 
\begin_inset Formula $G(x)$
\end_inset

 is also unknown.
 We shall assume that both 
\begin_inset Formula $F(x)$
\end_inset

 and 
\begin_inset Formula $G(x)$
\end_inset

 are continuous functions and that it is desired to test the hypothesis
 that these functions are identical, without specifying their common form.
 Thus, the following hypotheses are to be tested:
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{0}:F(x)=G(x)\text{ for }-\infty<x<\infty
\]

\end_inset


\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{1}:\text{ The hypothesis }H_{0}\text{\text{ is not true.}(10.6.6)}
\]

\end_inset


\end_layout

\begin_layout Standard
We shall let 
\begin_inset Formula $F_{m}(x)$
\end_inset

 denote the sample c.d.f calculated from the observed values of 
\begin_inset Formula $X_{1},...,X_{m}$
\end_inset

 and let 
\begin_inset Formula $G_{n}(x)$
\end_inset

 denote the sample c.d.f.
 calculated from the observed values of 
\begin_inset Formula $Y_{1},...,Y_{n}.$
\end_inset

 Furthermore, we shall consifer the statistic 
\begin_inset Formula $D_{mn},$
\end_inset

which is defined as follows:
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
D_{mn}=\sup_{-\infty<x<\infty}|F_{m}(x)-G_{n}(x)|\text{ (10.6.7)}
\]

\end_inset


\end_layout

\begin_layout Standard
The value of 
\begin_inset Formula $D_{mn}$
\end_inset

 is illustrated in Fig.
 10.5.
 for a typical example in which 
\begin_inset Formula $m=5$
\end_inset

 and 
\begin_inset Formula $n=3$
\end_inset

.
\end_layout

\begin_layout Standard
\begin_inset Graphics
	filename 10.6D_mn.png

\end_inset


\end_layout

\begin_layout Standard
When the null hypothesis 
\begin_inset Formula $H_{0}$
\end_inset

 is true and 
\begin_inset Formula $F(x)$
\end_inset

 and 
\begin_inset Formula $G(x)$
\end_inset

 are identical functions, the sample c.d.f.'s 
\begin_inset Formula $F_{m}(x)$
\end_inset

 and 
\begin_inset Formula $G_{n}(x)$
\end_inset

 will tend to be close to each other, In face, when 
\begin_inset Formula $H_{0}$
\end_inset

 is true, it follows from the Glivenko-Cantelli lemma that 
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
D_{mn}\rightarrow^{p}0\text{ as both }m\rightarrow\infty\text{ and }n\rightarrow\infty\text{ (10.6.8)}
\]

\end_inset


\end_layout

\begin_layout Standard
It seems reasonable, therefore, to use a test procedure that specified rejecting
 
\begin_inset Formula $H_{0}$
\end_inset

 when 
\begin_inset Formula $D_{mn}$
\end_inset

 is large.
 The following theorem, whose proof is beyond the scope of this text, gives
 us the asymptotic distribution of 
\begin_inset Formula $D_{mn}$
\end_inset

, which we can use to construct an approximate test.
\end_layout

\begin_layout Theorem*
10.6.3.
 Two-Sample Kolmogorov-Smirnov Statistic.
 For each value of 
\begin_inset Formula $t>0$
\end_inset

, let 
\begin_inset Formula $H(t)$
\end_inset

 denote the right side of Eq.
 (10.6.5).
 If the null hypothesis 
\begin_inset Formula $H_{0}$
\end_inset

 in (10.6.6) is true, then 
\end_layout

\begin_layout Theorem*
\begin_inset Formula 
\[
\lim_{m\rightarrow\infty,n\rightarrow\infty}Pr[(\frac{mn}{m+n})^{1/2}D_{mn}\le t]=H(t)\text{ (10.6.9)}
\]

\end_inset


\end_layout

\begin_layout Theorem*
Values of the function 
\begin_inset Formula $H(t)$
\end_inset

 are given in Table 10.32.
 The large-sample approcimate test of the hypotheses in (10.6.6) makes use
 of the statistic in (10.6.9).
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Definition*
10.6.3.
 Two-Sample Kolmogorov-Smirnov Test.
 A test procedure that rejects 
\begin_inset Formula $H_{0}$
\end_inset

 when
\end_layout

\begin_layout Definition*
\begin_inset Formula 
\[
(\frac{mn}{m+n})^{1/2}D_{mn}\ge c\text{ (10.6.10)}
\]

\end_inset


\end_layout

\begin_layout Definition*
where 
\begin_inset Formula $c$
\end_inset

 is an appropriate constant, is called a Kolmogorov-Smirnov two-sample test.
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Section*
Summary
\end_layout

\begin_layout Standard
We instroduced Kolmogorov-Smirnov tests for testing the null hypotheses
 that a random sample arose from particualr distribution and that two independen
t random samples arose from the same distribution.
 For the one-sample test, we compute 
\begin_inset Formula $D_{n},$
\end_inset

 the largest different between the sample c.d.f.
 and the null hypothesis c.d.f., and we reject the null hypothesis at level
 
\begin_inset Formula $\alpha_{0}$
\end_inset

 in 
\begin_inset Formula $n^{1/2}D_{n}^{*}\ge H^{-1}(1-\alpha_{0})$
\end_inset

, where 
\begin_inset Formula $H$
\end_inset

 is the c.d.f shown in Table 10.32.
 For the two-sample test, we compute 
\begin_inset Formula $D_{mn},$
\end_inset

the largest different between the two sample c.d.f.'s from the two different
 samples.
 We then reject the null hypothesis that the two samples arose from the
 same distribution at level 
\begin_inset Formula $\alpha_{0}$
\end_inset

 if 
\begin_inset Formula $(mn/(m+n))^{1/2}D_{mn}\ge H^{-1}(1-\alpha_{0})$
\end_inset

 
\end_layout

\begin_layout Section*
10.7.
 Robust Estimation
\end_layout

\begin_layout Standard
Exercise 9 prove that the M.L.E of 
\begin_inset Formula $\theta$
\end_inset

 is indeed the sample median when the sample comes from a Laplace distribution.
 
\end_layout

\begin_layout Standard
We also like the derivative to flatten out like the Laplace case for 
\begin_inset Formula $\theta$
\end_inset

 near the extremes so that the actual values of the extreme observations
 do not affect the estimate.
 A pdf with these properties is the following:
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
g_{k}(x|\theta,\sigma)=c_{k}e^{h_{k}([x-\theta]/\sigma)}\text{(10.7.6)}
\]

\end_inset


\end_layout

\begin_layout Section*
Comparison of the Estimators
\end_layout

\begin_layout Standard
When 
\begin_inset Formula $X_{1},...,X_{n}$
\end_inset

 form a random sample from the normal distribution with mean 
\begin_inset Formula $\theta$
\end_inset

 and variance 
\begin_inset Formula $\sigma^{2}$
\end_inset

, the probability of 
\begin_inset Formula $\bar{X}_{n}$
\end_inset

 and the probability distribution of each of the robust estimators described
 in this chapter will be symmetric with respect to the value 
\begin_inset Formula $\theta$
\end_inset

.
 Therefore, the mean of each these estimators will be 
\begin_inset Formula $\theta$
\end_inset

, the M.S.E.
 of each estimator will be equal its variance, and this M.S.E.
 will have a certain constant value for each estimator regardless of the
 true value of 
\begin_inset Formula $\theta$
\end_inset

.
\end_layout

\begin_layout Paragraph*
Summaration
\end_layout

\begin_layout Itemize
To say that the new estimators are more robust, we mean that they perform
 well compared to the old estimators, in terms of 
\series bold
M.S.E.
 
\series default
, regardless of which distribution gives rise to the data.
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Section*
10.8.
 Sign and Rank Tests
\end_layout

\begin_layout Standard
In this section, we describe some popular nonparametric tests for hypotheses
 about the median of a distribution or about the difference between two
 distributions.
\end_layout

\begin_layout Section*
One-Sample Procedures 
\end_layout

\begin_layout Standard
Are there methods that are appropriate when we are not willing to make assumptio
ns about the form of the distribution.
\end_layout

\begin_layout Standard
Suppose that 
\begin_inset Formula $X_{1},X_{2},...,X_{n}$
\end_inset

 form a random sample from an unknown distribution.
 In chapter 9, we considered the case in which the form of the inknown distribut
ion was known, but there were some specific parameters that were still unknown.
 For example, the distribution might be a normal distribution with unknown
 mean and/or variance.
 Now we shall assume onlythat the distribution is continuous.
 Since we shall not assume that the distribution of the data has a mean,
 then we cannot test hypotheses about the mean of the distribution.
 However, every continuous distribution has a median 
\begin_inset Formula $\mu$
\end_inset

 that satisfies 
\begin_inset Formula $Pr(X_{i}\le\mu)=0.5$
\end_inset

.
 The median is a popular measure of location for general distributions,
 and we shall now present a test procedure for testing hypothees of the
 form 
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{0}:\mu\le\mu_{0}
\]

\end_inset


\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{1}:\mu>\mu_{0}\text{ (10.8.1)}
\]

\end_inset


\end_layout

\begin_layout Standard
The test is based on the following simple fact: 
\begin_inset Formula $\mu\le\mu_{0}$
\end_inset

 if and only if 
\begin_inset Formula $Pr(X_{i}\le\mu_{0})\ge0.5$
\end_inset

.
 For 
\begin_inset Formula $i=1,...,n$
\end_inset

, let 
\begin_inset Formula $Y_{i}=1$
\end_inset

 if 
\begin_inset Formula $X_{i}\le\mu_{0}$
\end_inset

, and let 
\begin_inset Formula $Y_{i}=0$
\end_inset

 otherwise.
 Define 
\begin_inset Formula $p=Pr(Y_{i}=1)$
\end_inset

.
 Then testing wheter 
\begin_inset Formula $\mu<\mu_{0}$
\end_inset

 is equivalent to testing wheter 
\begin_inset Formula $p\ge0.5$
\end_inset

.
 Since 
\begin_inset Formula $X_{1},...,X_{n}$
\end_inset

 are independent, then so too are 
\begin_inset Formula $Y_{1},...,Y_{n}$
\end_inset

.
 This makes 
\begin_inset Formula $Y_{1},...,Y_{n}$
\end_inset

 a random sample from the Bernoulli distribution with parameter 
\begin_inset Formula $p$
\end_inset

.
 We already know how to test the null hypothesis that 
\begin_inset Formula $p\ge0.5$
\end_inset

 (See Example 9.19).
 We compute 
\begin_inset Formula $W=Y_{1}+....+Y_{n}$
\end_inset

 and reject the null hypothesis if 
\begin_inset Formula $W$
\end_inset

 is too small.
 To make the test have level of significance 
\begin_inset Formula $\alpha_{0}$
\end_inset

, choose 
\begin_inset Formula $c$
\end_inset

 so that 
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
\sum_{w=0}^{c}\binom{n}{w}\binom{1}{2}^{n}\le\alpha_{0}<\sum_{w=0}^{c+1}\binom{n}{w}\binom{1}{2}^{n}
\]

\end_inset


\end_layout

\begin_layout Standard

\series bold
Note: 
\series default
This result is presented from Example 9.1.9.
 We should notice that null hypostheses in this section (
\begin_inset Formula $p\ge0.5$
\end_inset

) is opposite to null hypotheses in Example 9.1.9 (
\begin_inset Formula $p\le p_{0})$
\end_inset

.
 
\series bold
So power function if 
\begin_inset Formula $P(W<c)$
\end_inset


\end_layout

\begin_layout Standard
Then the test would reject 
\begin_inset Formula $H_{0}$
\end_inset

 if 
\begin_inset Formula $W\le c$
\end_inset

.
\end_layout

\begin_layout Standard
The test that we have just described is called the 
\series bold
sign test 
\series default
because it is based on the number of observations for which 
\begin_inset Formula $X_{i}-\mu_{0}$
\end_inset

 is negative.
 A similar test can be constructed if we wish to test the hypotheses
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{0}:\mu=\mu_{0}
\]

\end_inset


\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{1}:\mu\ne\mu_{0}
\]

\end_inset


\end_layout

\begin_layout Standard
Once again, let 
\begin_inset Formula $p=Pr(X_{i}\le\mu_{0})$
\end_inset

.
 The null hypothesis 
\begin_inset Formula $H_{0}$
\end_inset

 is now equivalent to 
\begin_inset Formula $p=0.5$
\end_inset

.
 To perform the test at level of significance 
\begin_inset Formula $\alpha_{0}$
\end_inset

.
 We would choose a number 
\begin_inset Formula $c$
\end_inset

 such that 
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
\sum_{w=0}^{c}\binom{n}{w}\binom{1}{2}^{n}\le\frac{\alpha_{0}}{2}<\sum_{w=0}^{c+1}\binom{n}{w}\binom{1}{2}^{n}
\]

\end_inset


\end_layout

\begin_layout Standard
We would then reject 
\begin_inset Formula $H_{0}$
\end_inset

 if either 
\begin_inset Formula $W\le c$
\end_inset

 or 
\begin_inset Formula $W\ge n-c$
\end_inset

.
 We use the symmetric reject region because the binomial distribution with
 parameters 
\begin_inset Formula $n$
\end_inset

and 
\begin_inset Formula $1/2$
\end_inset

 is summetric about 
\begin_inset Formula $n/2$
\end_inset

.
\end_layout

\begin_layout Section*
Comparing Two Distributions
\end_layout

\begin_layout Standard
Comparing Copper Ores.
 Consider the comparision of copper ores in Example 9.6.5.
 This is Two-sample 
\begin_inset Formula $t$
\end_inset

 test, but in that example, two sample formed by normal distribution.
 But Can we still test hypotheses about wheter the distributions are the
 same of whether they have the same medians ?
\end_layout

\begin_layout Standard
Next, we shall consider a problem in which random sample of 
\begin_inset Formula $m$
\end_inset

 observations 
\begin_inset Formula $X_{1},...,X_{m}$
\end_inset

 is taken from a continuous distribution for which the c.d.f.
 
\begin_inset Formula $F(x)$
\end_inset

 is unknown, and an independent random sample of 
\begin_inset Formula $n$
\end_inset

 observations 
\begin_inset Formula $Y_{1},...,Y_{n}$
\end_inset

 is taken from another continous distribution for which the c.d.f.
 
\begin_inset Formula $G(x)$
\end_inset

 is also unknown.
 We desire to test the hypotheses 
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{0}:F=G
\]

\end_inset


\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
H_{1}:F\ne G\text{ (10.8.2)}
\]

\end_inset


\end_layout

\begin_layout Standard
One way to test the hypotheses (10.8.2) is to use the Kolmogorov-Smirnov test
 for two samples described in Sec.
 10.6.
 
\end_layout

\begin_layout Standard
In this section we shall present another procedure for testing the hypotheses
 (10.8.2).
 This procedure, which was introduced separately by F.
 Wilcoxon and by H.
 B.
 Mann and D.
 R.
 Whitney in the 1940s, is known as the Wilcoxon-Mann Whitney ranks test.
\end_layout

\begin_layout Subsection*
The Wilcoxon-Mann-Whitney Ranks Test.
 
\end_layout

\begin_layout Standard
In this procedure, we begin by arrangming the 
\begin_inset Formula $m+n$
\end_inset

 observations in the two samples in a single sequence from the smallest
 value that appears in the two smaples to the largest value that appears.
 Since all the obsevations come from continuous distribution, it may be
 assumed that no two of the 
\begin_inset Formula $m+n$
\end_inset

 observations have same values.
 Thus, a total ordering of these 
\begin_inset Formula $m+n$
\end_inset

 values can be obtained.
 Each observation in this total ordering is then assigned a rank from 
\begin_inset Formula $1$
\end_inset

 to 
\begin_inset Formula $m+n$
\end_inset

 corresponding to its position in the ordering.
\end_layout

\begin_layout Standard
The Wilconxon-Mann-Whitley ranks test is based on the property that if the
 null hypothsis 
\begin_inset Formula $H_{0}$
\end_inset

 is true and the two samples are actually drawn from the same distribution,
 then the observations 
\begin_inset Formula $X_{1},...,X_{m}$
\end_inset

 will tend to be dispersed throughout the ordering of all 
\begin_inset Formula $m+n$
\end_inset

 observations, rather than be concentrated among the smaller values or among
 the larger values.
 In fact, when 
\begin_inset Formula $H_{0}$
\end_inset

 is true, the ranks that are assigned to the 
\begin_inset Formula $m$
\end_inset

 observations 
\begin_inset Formula $X_{1},..,X_{m}$
\end_inset

 will be the same as if they were a random sample of 
\begin_inset Formula $m$
\end_inset

 ranks drawn at random without replacement from a box containing the 
\begin_inset Formula $m+n$
\end_inset

 ranks 
\begin_inset Formula $1,2,3,...,m+n$
\end_inset

.
\end_layout

\begin_layout Standard
Let 
\begin_inset Formula $S$
\end_inset

 denote the sum of the ranks that are assigned to the 
\begin_inset Formula $m$
\end_inset

 observations 
\begin_inset Formula $X_{1},...,X_{m}$
\end_inset

.
 Since the average of the ranks 
\begin_inset Formula $1,2,...,m+n$
\end_inset

 is 
\begin_inset Formula $(1/2)(m+n+1)$
\end_inset

, it follows from the discussion just given that when 
\begin_inset Formula $H_{0}$
\end_inset

 is true, 
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
E(S)=\frac{m(m+n+1)}{2}\text{ (10.8.3)}
\]

\end_inset


\end_layout

\begin_layout Standard
Also, it can be shown that when 
\begin_inset Formula $H_{0}$
\end_inset

 is true,
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
Var(S)=\frac{mn(m+n+1)}{12}\text{ (10.8.4)}
\]

\end_inset


\end_layout

\begin_layout Standard
Furthermore, when the sample sizes 
\begin_inset Formula $m$
\end_inset

 and 
\begin_inset Formula $n$
\end_inset

 are large and 
\begin_inset Formula $H_{0}$
\end_inset

 is true, the distribution of 
\begin_inset Formula $S$
\end_inset

 will be approximately the normal distribution for which the mean and the
 variance are given by Eqs.
 (10.8.3) and (10.8.4).
 The Wilcoxon-Mann-Whitney ranks test 
\series bold
rejects 
\begin_inset Formula $H_{0}$
\end_inset

 if the value of 
\begin_inset Formula $S$
\end_inset

 deviates very far from its mean value given by Eq.
 (10.8.3
\series default
).
 In other words, the test specifies rejecting 
\begin_inset Formula $H_{0}$
\end_inset

 if 
\begin_inset Formula $|S-(1/2)m(m+n+1)|\ge c$
\end_inset

, where the constant 
\begin_inset Formula $c$
\end_inset

 is chosen appropriately.
 In particular, when the approximate normal distribution of 
\begin_inset Formula $S$
\end_inset

 is used, the constant 
\begin_inset Formula $c=[Var(S)]^{1/2}\Phi^{-1}(1-\alpha_{0}/2)$
\end_inset

 makes the test have level of significance 
\begin_inset Formula $\alpha_{0}$
\end_inset

.
\end_layout

\begin_layout Standard
For small values of 
\begin_inset Formula $m$
\end_inset

 and m, the normal approximation to the distribution of 
\begin_inset Formula $S$
\end_inset

 will not be appropriate.
 Tables of the exact distribution of 
\begin_inset Formula $S$
\end_inset

 for small sample sizes are given in many published collections of statistical
 table.
 Many statistical software.
\end_layout

\begin_layout Subsection*
Note: Tests for paired Data.
 
\series medium
Versions of the sign test and ranks test for paired data are developed in
 Exercises 1 and 15.
\end_layout

\begin_layout Section*
Ties
\end_layout

\begin_layout Standard
The theory of Wilcoxon-Mann-Whitney signed ranks test is based on the assumption
 that all of the observed values of 
\begin_inset Formula $X_{i}$
\end_inset

 and 
\begin_inset Formula $Y_{j}$
\end_inset

 will be distinct.
 Since the measurements in an actual experiment may be made with only limited
 precision, however, there may actually be observed values that appear more
 than once 
\begin_inset Formula $X_{i}=Y_{j}$
\end_inset

.
 In this case the ranks test should be carried out twice.
 In the first test, assume that 
\begin_inset Formula $X_{i}>Y_{j}$
\end_inset

.
 In the second test, assume that 
\begin_inset Formula $X_{i}<Y_{j}$
\end_inset

.
 If the tail areas found from the two tests are roughly equal, then the
 ties are a relatively unimportant part of the data.
 If, on the other hand, the tail areas are quite different, then the ties
 can seriously affect the inference that are to be made.
 In this case that data may be inclusive.
\end_layout

\begin_layout Standard
Other reasonable methods for handling ties have been proposed.
 When two or more values are the same, one simple method is to consider
 the successive ranks that are to be assigned to these values and then assign
 the average of these ranks to each of the tied values.
 
\end_layout

\begin_layout Section*
Power of the Wilcoxon-Mann-Whitney Ranks Test
\end_layout

\begin_layout Standard
The Wilcoxon-Mann-Whitney ranks test rejects the null hypothesis that the
 two distributions are the same when the sum 
\begin_inset Formula $S$
\end_inset

 of the 
\begin_inset Formula $X$
\end_inset

 ranks is either too large or too small.
 This would be a sensible thing to do if one thought that the most important
 alternatives were those in which 
\begin_inset Formula $X_{i}$
\end_inset

 values tended to be larger than 
\begin_inset Formula $Y_{j}$
\end_inset

 values of those in which the 
\begin_inset Formula $X_{i}$
\end_inset

 values tended to be smaller than the 
\begin_inset Formula $Y_{j}$
\end_inset

 values.
 However, there are other situations in which 
\begin_inset Formula $F\ne G$
\end_inset

, but 
\begin_inset Formula $S$
\end_inset

 tends to be close to the mean in 
\begin_inset Formula $Eq.(10.8.3)$
\end_inset

.
\end_layout

\begin_layout Paragraph*
Note: 
\series medium
The idea in here is if distribution 
\begin_inset Formula $F$
\end_inset

 of 
\begin_inset Formula $X$
\end_inset

 nearly equals distribution 
\begin_inset Formula $G$
\end_inset

 of 
\begin_inset Formula $Y$
\end_inset

, then samples of 
\begin_inset Formula $X$
\end_inset

 and 
\begin_inset Formula $Y$
\end_inset

 will spread out in ordering combined sample 
\begin_inset Formula $m+n$
\end_inset

 instead of concentrated in one place.
 
\end_layout

\begin_layout Definition*
10.8.1.
 Stochastically Larger.
 Let 
\begin_inset Formula $X$
\end_inset

 be a random variable with c.d.f.
 
\begin_inset Formula $F$
\end_inset

, and let 
\begin_inset Formula $Y$
\end_inset

 be a random varible with c.d.f.
 
\begin_inset Formula $G$
\end_inset

.
 Let 
\begin_inset Formula $F^{-1}$
\end_inset

 and 
\begin_inset Formula $G^{-1}$
\end_inset

 denote the respective quantile functions.
 We say that 
\begin_inset Formula $F$
\end_inset

 is stochastically larger than 
\begin_inset Formula $G$
\end_inset

 or, equaivalently, that 
\begin_inset Formula $X$
\end_inset

 is stochastically larger than 
\begin_inset Formula $Y$
\end_inset

 if 
\begin_inset Formula $F^{-1}(p)\ge G^{-1}(p)$
\end_inset

 for all 
\begin_inset Formula $0<p<1$
\end_inset

 ; that is, every quantile of 
\begin_inset Formula $X$
\end_inset

 is at least as large as the corresponding quantile of 
\begin_inset Formula $Y$
\end_inset

.
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Standard
It is easy to see that if 
\begin_inset Formula $X_{i}$
\end_inset

 is stochastically larger than 
\begin_inset Formula $Y_{j}$
\end_inset

, then the ranks of the 
\begin_inset Formula $X_{i}'s$
\end_inset

 in the combined sample will tend to be at least as large as the ranks of
 the 
\begin_inset Formula $Y_{j}'s$
\end_inset

 .
 This will make large values of 
\begin_inset Formula $S$
\end_inset

 more likely than small values.
 Similarly, if 
\begin_inset Formula $Y_{j}$
\end_inset

 is stochastically larger than 
\begin_inset Formula $X_{i}$
\end_inset

, 
\begin_inset Formula $S$
\end_inset

 will tend to be small.
\end_layout

\begin_layout Section*
Summary 
\end_layout

\begin_layout Standard
The sign test was introduced as a nonparametric test for hypotheses about
 the median of an unknown distribution.
 The Wilcoxon-Mann-Whitley ranks test was developed as another nonparametric
 test for hypotheses about equality of two c.d.f's.
 The Wilcoxon-Mann-Whitley ranks test was designed to have large power function
 when one of two distribution is stochastically larger than the other.
\end_layout

\begin_layout Chapter*
10.7.
 Robust Estimation
\end_layout

\begin_layout Standard
In many statistical problems, we might not feel comfortable assuming that
 the distribution of our data 
\begin_inset Formula $X$
\end_inset

 is a member of a single parametric family.
 Suppose that we consider using an estimator 
\begin_inset Formula $T=r(X)$
\end_inset

 of some parameter 
\begin_inset Formula $\theta$
\end_inset

.
 It might be that 
\begin_inset Formula $T$
\end_inset

 has good properties if 
\begin_inset Formula $X$
\end_inset

 is a random sample from, say, a normal distribution.
 On the other hand, we might be concerned about how 
\begin_inset Formula $T$
\end_inset

 would behave if 
\begin_inset Formula $X$
\end_inset

 were actually a sample from a different distribution.
 In this section, we introduce a new class of distributions and several
 new statistics.
 We then compare the behaviors of these statistics (and some old ones) when
 the data arise from one of the new distributions (and from some old ones).
 An estimator is called robust if it performs well compared to other estimators,
 regardless of the distribution that gives ries to the data.
\end_layout

\begin_layout Standard
In this section, we shall define a class of distributions called contaminated
 normals that we shall use for assessing the performance of various estimators.
 We shall also introduce special types of robust estimators known as trimmed
 means and 
\begin_inset Formula $M-$
\end_inset

estimatos.
\end_layout

\begin_layout Section*
Contaminated Normal Distributions.
\end_layout

\begin_layout Standard
For example, suppose the the bulk of the data in which we are interested
 comprise a sample from the normal distribution with unknown mean 
\begin_inset Formula $\mu$
\end_inset

and variance 
\begin_inset Formula $\sigma^{2}$
\end_inset

.
 But suppose that, for each observation, there is a small probability 
\begin_inset Formula $\epsilon$
\end_inset

 that the observation actually comes from a different distribution with
 p.d.f.
 
\begin_inset Formula $g$
\end_inset

.
 That is, the pd.f.
 of our observable data is actually
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
f(x)=(1-\epsilon)(2\pi\sigma^{2})^{-1/2}exp(-\frac{1}{2\sigma^{2}}[x-\mu]^{2})+\epsilon g(x).\text{ (10.7.1)}
\]

\end_inset


\end_layout

\begin_layout Definition*
Contaminated Normal Distributions.
 A distribution whose p.d.f.
 has the form of Eq.
 (10.7.1) is called a constaminated normal, and the distribution with p.d.f.
 
\begin_inset Formula $g$
\end_inset

 is called contaminating distribution.
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Section*
Trimmed Means
\end_layout

\begin_layout Definition*
10.7.2 Trimmed Means.
 for each positive integer 
\begin_inset Formula $k$
\end_inset

 such that 
\begin_inset Formula $k<n/2$
\end_inset

, ignore the 
\begin_inset Formula $k$
\end_inset

 smallest observatons 
\begin_inset Formula $Y_{1},...,Y_{k}$
\end_inset

 and the 
\begin_inset Formula $k$
\end_inset

 largest observations 
\begin_inset Formula $Y_{n},Y_{n-1},...,Y_{n-k+1}$
\end_inset

 in the sample.
 The average of the remaining 
\begin_inset Formula $n-2k$
\end_inset

 intermediate observations is called the 
\begin_inset Formula $k$
\end_inset

th level trimmed mean.
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Standard
Clearly, the 
\begin_inset Formula $k$
\end_inset

th level trimmed mean can be represented as a weighted average of the order
 statistics having the form
\end_layout

\begin_layout Standard
\begin_inset Formula 
\[
\frac{1}{n-2k}\sum_{i=k+1}^{n-k}Y_{i}\text{ (10.7.4)}
\]

\end_inset


\end_layout

\begin_layout Standard
The sample median is an example of a trimmed mean.
 When 
\begin_inset Formula $n$
\end_inset

 is odd, the sample median is the 
\begin_inset Formula $[(n-1)/2]th$
\end_inset

 level trimmed mean.
 When 
\begin_inset Formula $n$
\end_inset

 is even, it is the 
\begin_inset Formula $[(n-2)/2]$
\end_inset

 th level trimmed mean.
 In either case, the sample median is the 
\begin_inset Formula $k$
\end_inset

th level trimmed mean, where 
\begin_inset Formula $k$
\end_inset

 is lower bound of 
\begin_inset Formula $(n-1)/2$
\end_inset

 is the largest integer less than or equal to 
\begin_inset Formula $(n-1)/2$
\end_inset

.
\end_layout

\begin_layout Section*
Robust Estimation of Scale
\end_layout

\begin_layout Standard
In addition to the median of a distribution, there are other parameters
 that might be worth estimating even when we are not willing to model our
 data as arising from a particular parametric family.
 For example, scale parameters might be valuable for giving an idea how
 spread out a distribution is.
 The standard deviation, if it exists, is one such measure.
 The general class of scale parameters is defined here.
\end_layout

\begin_layout Definition*
10.7.3.
 Scale Parameters.
 An arbitrary parameter 
\begin_inset Formula $\sigma$
\end_inset

 is a scale parameter for the distribution of 
\begin_inset Formula $X$
\end_inset

 if, for all 
\begin_inset Formula $a>0$
\end_inset

 and all real 
\begin_inset Formula $b$
\end_inset

, the corresponding parameter for the distribution of 
\begin_inset Formula $aX+b$
\end_inset

 is 
\begin_inset Formula $a\sigma$
\end_inset

.
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Standard
Although the standard deviation is a scale parameter, there are many distributio
ns (such as the Cauchy) for ehich the standard deviation does not exists.There
 are alternative measures of spread to the standard deviation that exist
 and are finite for all distributions.
\end_layout

\begin_layout Standard
One scale parameter that exists for every distribution is the interquartile
 range (IQR) as defined in Definition 4.3.2.
 An estimator of the IQA if the sample IQR, the difference between the 0.75
 and 0.25 sample quantiles.
 (Sample quantiles are just quantiles of the sample c.d.f.)
\end_layout

\begin_layout Standard
Another scale parameer that exists for every random variable 
\begin_inset Formula $X$
\end_inset

 is the median absolute deviation.
\end_layout

\begin_layout Definition*
10.7.4.
 Median Absolute Deviation.
 The median absolute deviation of a random variable 
\begin_inset Formula $X$
\end_inset

 is the median of the distribution of 
\begin_inset Formula $|X-m|$
\end_inset

, where 
\begin_inset Formula $m$
\end_inset

 is the median of 
\begin_inset Formula $X$
\end_inset

.
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Standard
If this distribution of 
\begin_inset Formula $X$
\end_inset

 is symmetric around its median, then the median absolute deivation is one-half
 of the IQR.
 For asymmetric distributions, the median absolute deviation is the half-length
 of the symmetric interval around the media that contain 50 percent of distribut
ion.
 The sample median absolute deviation is the sample median of the values
 
\begin_inset Formula $|X_{i}-M_{n}|$
\end_inset

, where 
\begin_inset Formula $M_{n}$
\end_inset

 is the sample media of 
\begin_inset Formula $X_{1},...,X_{n}$
\end_inset

.
\end_layout

\begin_layout Section*
M-Estimators of the Median
\end_layout

\begin_layout Standard
The sample mean is heavily influenced by one extreme observation.
 For example, if one observation 
\begin_inset Formula $x$
\end_inset

 in the sample of size 
\begin_inset Formula $n$
\end_inset

 in replaced by 
\begin_inset Formula $x+\Delta$
\end_inset

, the sample mean changes by 
\begin_inset Formula $\Delta/n$
\end_inset

.
 If 
\begin_inset Formula $\Delta$
\end_inset

 is large, this will be a big change.
 The sample median, on the other hand, is influenced very little, or not
 at all, by a change in one observation.
 However, the sample median is inefficient in that it makes use of very
 few of the observed values.
 Trimmed means are one attempt to compromise between the sample median and
 the sample mean by forming estimators estimators that make use of more
 than just the one or two observations in the middle of sample while maintaining
 insensitiviry to extrme observations.
 There are other estimators are M.L.E's of 
\begin_inset Formula $\theta$
\end_inset

 under different assumptions about the p.d.f.
 of the observations.
\end_layout

\begin_layout Standard
The sample mean is the M.L.E of 
\begin_inset Formula $\theta$
\end_inset

 if we assume that 
\begin_inset Formula $X_{1},...,X_{n}$
\end_inset

 form a random sample from a normal distribution with mean (and median)
 
\begin_inset Formula $\theta$
\end_inset

 and arbitrary variance.
 The sample median is also an M.L.E.
 it is the M.L.E of 
\begin_inset Formula $\theta$
\end_inset

.
 It is the M.L.E.
 of 
\begin_inset Formula $\theta$
\end_inset

 if we assume that 
\begin_inset Formula $X_{1},...,X_{n}$
\end_inset

 form a random sample from one of the following distributions.
\end_layout

\begin_layout Definition*
10.7.5 Laplace Distributions.
 Let 
\begin_inset Formula $\sigma>0$
\end_inset

 and 
\begin_inset Formula $\theta$
\end_inset

 be real numbers.
 Th distribution whose p.d.f.
 is 
\end_layout

\begin_layout Definition*
\begin_inset Formula 
\[
f(x|\theta,\sigma)=\frac{1}{2\sigma}e^{-|x-\theta|/\sigma}\text{ (10.7.5)}
\]

\end_inset


\end_layout

\begin_layout Definition*
is called the Laplace distribution with parameters 
\begin_inset Formula $\theta$
\end_inset

 and 
\begin_inset Formula $\sigma$
\end_inset

.
\end_layout

\begin_layout Definition*
It would be nice to have a compromise between these two types of behavior
 withour arbitrarily discarding a fixed amount of data.
 We would like the derivative of the logarithm of the likelihood to be approxima
tly proportional to 
\begin_inset Formula $\sum(x_{i}-\theta)$
\end_inset

 for 
\begin_inset Formula $\theta$
\end_inset

 near middle of the data, where the summation is only over the middle observatio
ns.
 This will allow the estimator to make use of more data than just the very
 middle observation.
 Also, we would like the derivative to flatten out like the laplace case
 for 
\begin_inset Formula $\theta$
\end_inset

 near the extremes so that the actual values of the extreme observations
 do not affect the estimate.
 A pdf with theses properties is the following:
\end_layout

\begin_layout Definition*
\begin_inset Formula 
\[
g_{k}(x|\theta,\sigma)=c_{k}e^{h_{k}([c-\theta]/\sigma)}\text{ (10.7.6)}
\]

\end_inset


\end_layout

\begin_layout Definition*
where 
\begin_inset Formula $\sigma$
\end_inset

 is a scale number,
\end_layout

\begin_layout Definition*
\begin_inset Formula 
\[
h_{k(y)}=\begin{cases}
-0.5y^{2} & \text{if}-k<y<k\\
0.5k^{2}-k|y| & \text{otherwise}
\end{cases}
\]

\end_inset


\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Standard
We see that 
\begin_inset Formula $k$
\end_inset

 can be chosen to reflect how many multiples of 
\begin_inset Formula $\sigma$
\end_inset

 a data value can be away from 
\begin_inset Formula $\theta$
\end_inset

 before we think that it starts to lose importance for estimating 
\begin_inset Formula $\theta$
\end_inset

.
 Typical choices are 
\begin_inset Formula $1\le k\le2.5$
\end_inset

.
 If we suppose that 
\begin_inset Formula $X_{1},...,X_{n}$
\end_inset

 form a random sample from a distribution with pdf 
\begin_inset Formula $g_{k}(x|\theta,\sigma)$
\end_inset

, the MLE of 
\begin_inset Formula $\theta$
\end_inset

 can be a compromise between the sample media and the sample mean.
\end_layout

\begin_layout Definition*
10.7.6.
 M-Estimators.
 The M.L.E of 
\begin_inset Formula $\theta$
\end_inset

 under the assumption that the data have p.d.f.
 
\begin_inset Formula $g_{k}$
\end_inset

 in Eq.
 (10.7.6) is called an M-estimators.
\end_layout

\begin_layout Standard
\begin_inset Separator plain
\end_inset


\end_layout

\begin_layout Paragraph*
Note: Simultaneous M-Esimators Exist for the Median and Scale parameters.
 
\series medium
It is possible to estimate the median and a scale parameter simultaneously
 using a method very similar to that described for M-estimators.
 That is, instead of just picking a value for 
\begin_inset Formula $\hat{\sigma}$
\end_inset

 in the M-esitmator algorith,, we can construct a more complicated algorithm
 that estimates both the median and a scale parameter.
 
\end_layout

\begin_layout Section*
Comparison of the Estimators
\end_layout

\begin_layout Standard
It is acknowledged that a robust estimator will perform better than 
\begin_inset Formula $\bar{X}_{n}$
\end_inset

in a situation of the type just described.
 However, if 
\begin_inset Formula $X_{1},...,X_{n}$
\end_inset

 actually do form a random sample from a normal distribution, then 
\begin_inset Formula $\bar{X}_{n}$
\end_inset

will perform better than a robust estimator.
 Since we are typically not certain which situation obtains in a particular
 problem, it is important to know how much larger the M.S.E.
 of a robust estimator will be than the M.S.E.
 of 
\begin_inset Formula $\bar{X}_{n}$
\end_inset

 when the actual distribution is normal.
 In other words, it is important to known how much is lost if we use robust
 estimator when the actual distributio is normal.
\end_layout

\begin_layout Standard
If there is reason to believe that p.d.f.
 
\begin_inset Formula $f(x)$
\end_inset

 is approximately by normal, then 
\begin_inset Formula $\theta$
\end_inset

 might be estimated by using s trimmed mean, which is obtained by omitting
 about 10 or 15 percent of the observed values at each end of the ordered
 sample.
 Alternatively, an M-estimator with 
\begin_inset Formula $k=2$
\end_inset

 or 
\begin_inset Formula $2.5$
\end_inset

 could be used.
 If the p.d.f.
 
\begin_inset Formula $f(x)$
\end_inset

 might be far from normal or if several of the observations might be outliers,
 then the sample median might be used to estimate 
\begin_inset Formula $\theta$
\end_inset

, or one could be used an M-estimator with 
\begin_inset Formula $k=1$
\end_inset

 or 
\begin_inset Formula $1.5$
\end_inset


\end_layout

\end_body
\end_document