©2001 CRC Press LLC
Library of Congress Cataloging-in-Publication Data Sprent, Peter. Applied nonparametric statistica...
360 downloads
3620 Views
5MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
©2001 CRC Press LLC
Library of Congress Cataloging-in-Publication Data Sprent, Peter. Applied nonparametric statistical methods.--3rd ed. I P Sprent, N.C. Smeeton. p. cm. -- (Cbapman & Hall/CRC texts in statistical science series) Includes bibliographical references and index. ISBN 1-58488-145-3 (alk. paper) 1. Nonparametric statistics. I. Smecton, N.C. II. Title. III. Texts in statistical science. QA278.8 S74 20W 519.5'4--dc2l
00-055485
This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. Apart from any fair dealing for the purpose of research or private study, or criticism or review, as permitted under the UK Copyright Designs and Patents Act, 1988, this publication my not be reproduced, stored or transmitted, in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage or retrieval system, without the prior permission in writing of the publishers, or in the case of reprographic, reproduction only in accordance with the terms of the licenses issued by the Copyright Licensing Agency in the UK, or in accordance with the terms of the license issued by the appropriate Reproduction Rights Organization outside the UK. All rights reserved. Authorization to photocopy items for internal or personal use, or the personal or internal use of specific clients, my be granted by CRC Press LLC, provided that $.50 per page photocopied is paid directly to Copyright Clearance Center, 222 Rosewisd Drive, Danvers, MA 01923 USA. The fee code for users of the Transactional Reporting Service is ISBN 1-58489-1453/0150.004.50. The fee is subject to change without notice. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. The consent of CRC Press LLC does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific permission must be obtained in writing from CRC Press LLC for such copying. Direct all inquiries to CRC Press LLC, 2000 N.W. Corporate Blvd., Boca Raton, Florida 33431. Trademark Notice: Product or corporate names my be trademarks or registered trademarks, and are used only for identification and explanation, without intent to infringe.
Visit the CRC Press Web site at www.crcpress.com © 2001 by Chapman & Hall/CRC No claim to original U.S. Government works International Standard Book Number 1-58488-145-3 Library of Congress C~ard Number 00-05.5485 Printed in the United States of America 3 4 5 6 7 8 9 0 Printed on acid-free paper
©2001 CRC Press LLC
Contents Preface 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7
Introducing nonparametric methods Basic statistics Samples and populations Hypothesis tests Estimation Ethical issues Computers and nonparametric methods Further reading Exercises
2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8
Centrality inference for single samples Using measurement data Inferences about medians based on ranks The sign test Transformation of ranks Asymptotic results Robustness Fields of application Summary Exercises
3.1 3.2 3.3 3.4 3.5 3.6 3.7
Other single-sample inference Inferences for dichotomous data Tests related to the sign test Matching samples to distributions Angular data A runs test for randomness Fields of application Summary Exercises
4.1 4.2 4.3 4.4 4.5
Methods for paired samples Comparisons in pairs A less obvious use of the sign test Power and sample size Fields of application Summary Exercises
2
3
4
©2001 CRC Press LLC
5
Methods for two independent samples 5.1 Centrality tests and estimates 5.2 Rank based tests 5.3 The median test 5.4 Normal scores 5.5 Tests for survival data 5.6 Asymptotic approximations 5.7 Power and sample size 5.8 Tests for equality of variance 5.9 Tests for a common distribution 5.10 Fields of application 5.11 Summary Exercises
6 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8
Three or more samples Compaarisons with parametric methods Centrality tests for independent samples Centrality tests for related samples More detailed treatment comparisons Tests for heterogeneity of variance Some miscellaneous considerations Fields of application Summary Exercises
7.1 7.2 7.3 7.4 7.5
Correlation and concordance Correlation and bivariate data Ranked data for several variables Agreement Fields of application Summary Exercises
8.1 8.2 8.3 8.4 8.5 8.6
Regression Bivariate linear regression Multiple regression Nonparametric regression models Other multivariate data problems Fields of application Summary Exercises
7
8
9 9.1 9.2
Categorical data Categories and counts Nominal attribute categories
©2001 CRC Press LLC
9.3 9.4 9.5 9,6 9.7
10
Ordered categorical data Goodness-of-fit tests for discrete data Extension of McNemar's test Fields of application Summary Exercises
10.1 10.2 10.3 10.4 10.5 10.6
Association in categorical data The analysis of association Some models for contingency tables Combining and partitioning of tables Power Fields of application Summary Exercises
11.1 11.2 11.3 11.4 11.5 11.6
Robust estimation When assumptions break down Outliers and influence The bootstrap M-estimators and other robust estimators Fields of application Summary Exercises
11
Appendix References Solutions to odd-numbered exercises
©2001 CRC Press LLC
Preface T h e s e c o n d e d i t i o n o f t h i s b o o k w a s w r i t t e n b y th e fi r s t - n a m e d a u t h o r t o pr o v i d e a th e n ( 1 9 9 3 ) up - t o - d a t e in t r o d u c t i o n to n o n p a r a m e t r i c a n d d i s t r i b u t i o n - f r e e m e t h o d s . It t o o k a mi d w a y c o u r s e b e t w e e n a b a r e d e s c r i p t i o n of te c h n i q u e s a n d a de t a i l e d e x p o s i t i o n of th e th e o r y . I n d i v i d u a l me t h o d s a n d li n k s be t w e e n th e m w e r e il l u s t r a t e d ma i n l y b y e x a m p l e s , M a t h e m a t i c s w a s k e p t t o t h e m i n i m u m n e e d e d fo r a c l e a r u n d e r s t a n d i n g o f s c o p e a n d l i m i t a t i o n s . Th e b o o k w a s d e s i g n e d t o me e t t h e n e e d s b o t h o f s t a t i s t i c s s t u d e n t s m a k i n g fi r s t c o n t a c t w i t h t h e s e m e t h o d s a n d o f re s e a r c h w o r k e r s , m a n a g e r s , re s e a r c h a n d d e v e l o p m e n t s t a f f , c o n s u l t a n t s a n d o t h e r s w o r k i n g in v a r i o u s fi e l d s w h o ha d a n un d e r s t a n d i n g of ba s i c s t a t i s t i c s a n d wh o , a l t h o u g h th e y h a d li t t l e p r e v i o u s k n o w l e d g e o f n o n p a r a m e t r i c m e t h o d s , n o w fo u n d o r th o u g h t t h e y mi g h t fi n d th e m u s e f u l in t h e i r w o r k . A p o s i t i v e re s p o n s e fr o m re a d e r s a n d re v i e w e r s h a s e n c o u r a g e d u s t o re t a i n t h e b a s i c fo r m a t w h i l e t a k i n g t h e o p p o r t u n i t y t o i n t r o d u c e n e w to p i c s a s w e l l a s c h a n g i n g th e e m p h a s i s to re f l e c t b o t h d e v e l o p m e n t s in c o m p u t i n g a n d n e w a t t i t u d e s to w a r d s da t a a n a l y s i s . N o n p a r a m e t r i c me t h o d s a r e ba s i c a l l y a n a l y t i c to o l s , bu t da t a c o l l e c t i o n , a n a l y s e s a n d t h e i r i n t e r p r e t a t i o n a r e i n t e r r e l a t e d . Th i s i s w h y we ha v e e x p a n d e d th e c o v e r a g e of to p i c s s u c h a s e t h i c a l c o n s i d e r a t i o n s an d ca l c u l a t i o n of po w e r an d of s a m p l e s i z e s ne e d e d t o a c h i e v e s t a t e d a i m s . Th e s e m a k e t h e i r m a i n i m p a c t a t t h e p l a n n i n g s t a g e , bu t a l s o in f l u e n c e th e a n a l y t i c a n d in f e r e n t i a l ph a s e s . T h e r e h a s b e e n w i d e s p r e a d c r i t i c i s m in re c e n t y e a r s b y ma n y s t a t i s t i c i a n s of in a p p r o p r i a t e a n d e v e n im p r o p e r us e of s i g n i f i c a n c e t e s t s a n d t h e re l a t e d c o n c e p t o f P- v a l u e s . H o w e v e r , t h e s e t o o l s h a v e a p o s i t i v e ro l e w h e n p r o p e r l y u s e d a n d u n d e r s t o o d . To e n c o u r a g e b e t t e r u s e th e s e c t i o n o n h y p o t h e s i s t e s t i n g i n C h a p t e r I h a s b e e n re w r i t t e n , a n d th r o u g h o u t th e bo o k th e r e i s mo r e e m p h a s i s on ho w t h e s e c o n c e p t s s h o u l d be us e d a n d wa r n i n g s a b o u t po t e n t i a l mi s u s e . T h e la y o u t o f C h a p t e r s I to 1 0 fo l l o w s t h e b r o a d p a t t e r n o f t h e c o r r e s p o n d i n g c h a p t e r s in th e s e c o n d e d i t i o n bu t th e r e a r e ma n y c h a n g e s in or d e r a n d ot h e r a s p e c t s of pr e s e n t a t i o n in c l u d i n g ne w a n d m o r e de t a i l e d e x a m p l e s . O n e or tw o to p i c s h a v e be e n dr o p p e d or a r e t r e a t e d in le s s de t a i l , a n d ne w ma t e r i a l ha s be e n in s e r t e d w h e r e a p p r o p r i a t e . As we l l as c o m m e n t s on e t h i c a l co n s i d e r a t i o n s a n d d i s c u s s i o n s on po w e r a n d s a m p l e s i z e , th e r e a r e ne w s e c t i o n s on th e
©2001 CRC Press LLC
a n a l y s i s of a n g u l a r da t a , th e u s e of c a p t u r e - r e c a p t u r e me t h o d s , th e m e a s u r e m e n t of a g r e e m e n t be t w e e n ob s e r v e r s a n d s e v e r a l le s s e r a d d i t i o n s . Ex a m p l e s h a v e b e e n c h o s e n fr o m a w i d e r r a n g e o f d i s c i p l i n e s . Fo r a fe w mo r e a d v a n c e d to p i c s s u c h a s re g r e s s i o n s m o o t h i n g te c h n i q u e s a n d M - e s t i m a t i o n we ha v e no t gi v e n de t a i l s of s p e c i f i c me t h o d s bu t on l y a br o a d ov e r v i e w of e a c h to p i c to e n a b l e r e a d e r s to j u d g e w h e t h e r i t ma y b e re l e v a n t t o th e i r p a r t i c u l a r n e e d s . I n s u c h c a s e s re f e r e n c e s a r e g i v e n to s o u r c e s th a t c o n t a i n th e d e t a i l n e e d e d fo r i m p l e m e n t a t i o n . C h a p t e r 1 1 h a s b e e n re w r i t t e n to g i v e a n e l e m e n t a r y in t r o d u c t i o n t o in f l u e n c e fu n c t i o n s , th e n o n p a r a m e t r i c b o o t s t r a p a n d ro b u s t e s t i m a t i o n g e n e r a l l y , a g a i n w i t h re f e r e n c e s to s o u r c e m a t e r i a l fo r t h o s e w h o w a n t to m a k e fu l l u s e o f t h e s e id e a s . M a t e r i a l th a t a p p e a r e d in C h a p t e r 12 of th e s e c o n d e d i t i o n ha s be e n u p d a t e d a n d i n c o r p o r a t e d a t re l e v a n t p o i n t s in th e te x t . W e h a v e n o t in c l u d e d ta b l e s fo r b a s i c n o n p a r a m e t r i c p r o c e d u r e s , m a i n l y be c a u s e mo r e s a t i s f a c t o r y in f o r m a t i o n is pr o v i d e d by mo d e m s t a t i s t i c a l s o f t w a r e , ma k i n g ma n y s t a n d a r d ta b l e s in s u f f i c i e n t or s u p e r f l u o u s fo r s e r i o u s u s e r s o f t h e m e t h o d s . Th o s e w h o n e e d s u c h t a b l e s be c a u s e th e y ha v e no a c c e s s to s p e c i a l i z e d s o f t w a r e a r e we l l c a t e r e d fo r b y s t a n d a r d c o l l e c t i o n s o f s t a t i s t i c a l ta b l e s . W e g i v e r e f e r e n c e s to th e s e th r o u g h o u t th e bo o k a n d a l s o wh e n r e l e v a n t to s o m e s p e c i a l i z e d t a b l e s . W e h a v e re t a i n e d t h e s e c t i o n o u t l i n i n g s o l u t i o n s to od d - n u m b e r e d e x e r c i s e s . W e a r e g r a t e f u l to m a n y re a d e r s o f th e e a r l i e r e d i t i o n s w h o ma d e c o n s t r u c t i v e c o m m e n t s a b o u t th e c o n t e n t a n d tr e a t m e n t , or s o m e t i m e s a b o u t t h e l a c k o f t r e a t m e n t , o f p a r t i c u l a r t o p i c s . Th i s i n p u t t r i g g e r e d m a n y of th e c h a n g e s ma d e in th i s e d i t i o n . O u r s p e c i a l t h a n k s go to J i m M c G a n r i c k fo r h e l p f u l d i s c u s s i o n s o n p h y s i o l o g i c a l m e a s u r e m e n t s a n d to Pr o f e s s o r R i c h a r d H u g h e s fo r a d v i c e o n th e G u i l l a i n - B a r r é s y n d r o m e . W e h a p p i l y re n e w t h e t h a n k s re c o r d e d i n t h e s e c o n d e d i t i o n t o Ti m o t h y P . D a v i s a n d C h r i s T h e o b a l d w h o s u p p l i e d u s w i t h d a t a s e t s u s e d in i t i a l l y i n th a t e d i t i o n fo r e x a m p l e s t h a t w e h a v e re t a i n e d . P. Sprent N. C. Smeeton July 2000
©2001 CRC Press LLC
1 Introducing nonparametric methods
1.1 BASIC STATISTICS One need know only a little statistics to make sensible use of simple nonparametric methods. While we assume that many readers will be familiar with the basic statistical notions met in introductory or service courses of some 20 hours instruction, nevertheless those without formal statistical training should be able to use this book in parallel with one of the many introductory general statistical texts available in libraries or from most academic bookshops, or one recommended by a statistician colleague or friend. The choice may depend upon how familiar one is with mathematical terms and notation. For example, Essential Statistics (Rees, 1995) adopts a straightforward approach that should suffice for most purposes, but some may prefer a more advanced treatment, or an introductory text that emphasizes applications in particular areas such as medicine, biology, agriculture, the social sciences and so on. Readers with considerable experience in general statistics but who are new to nonparametric methods will be familiar with some background material we give, but we urge them at least to skim through this to see whether we depart from conventional treatments. For example, our approach to testing and estimation in Sections 1.3 and 1.4 differs in certain aspects from that in some general statistics courses. In this chapter we survey some concepts relevant to nonparametric methods. It is the methods and not the data that are nonparametric. 1.1.1 Parametric and nonparametric methods Statistics students meet families of probability distributions early in their courses. One of the best known is the n ormal or Gaussian family, where individual members are specified by assigning constant values to two quantities called parameters. These are usually denoted by µ and σ2 and represent the mean and variance.
©2001 CRC Press LLC
The notation N(µ , σ 2) denotes a normal distribution with these parameters. Normal distributions are often associated with continuous data such as measurements. Given a set of independent observations (a concept explained more fully in Section 1.2) from a normal distribution, we often want to infer something about the unknown parameters. The sample mean provides a point (i.e. single value) estimate of the parameter µ (sometimes called, in statistical jargon, the population mean). Here the well-known t-test is used to measure the strength of the evidence provided by a sample to support an a priori hypothesized value µ0 for the population mean. More usefully, we may determine what is called a confidence interval for the ‘true’ population mean. This is an interval for which we have, in a sense we describe in Section 1.4.1, reasonable confidence that it contains the true but unknown mean µ. These are examples of parametric inferences. The normal distribution is strictly relevant only to some types of continuous scale data such as measurements, but it often works quite well if the measurement scale is coarse (e.g. for examination marks recorded to the nearest integer). More importantly, it is useful for approximations in a wide range of circumstances when applied to other types of data. The binomial distribution is relevant to some types of counts. The family also has two parameters n, p, where n is the total number of observations and p is the probability that a particular one from two possible events occurs at any observation. Subject to certain conditions, the number of occurrences, r, where 0 ≤ r ≤ n, of that event in the n observations has a binomial distribution, which we refer to as a B(n, p) distribution. With a binomial distribution if we observe r occurrences of an event in a set of n observations, then p^ = r/n is a point estimate of p, the probability of success at each independent observation. We may want to assess how strongly sample evidence supports an a priori hypothesized value p0, say, for p or obtain a confidence interval for the value of p for the population. The binomial distribution is often relevant to counts in dichotomous outcome situations. For example, the number of male children in a family of size n is often assumed to have a binomial distribution with p = 1/2, but we see in Section 1.1.3 that this is only approximate. The number of ‘sixes’ recorded in 10 casts of a fair die has a B(10, 1/6) distribution. The outcome of interest is sometimes called a ‘favourable’ event, but this is hardly appropriate if, for example, the event is a positive diagnosis of illness.
©2001 CRC Press LLC
Other well-known families include the uniform (or rectangular), multinomial, Poisson, exponential, double exponential, gamma, beta and Weibull distributions. This list is not exhaustive and you may not be, and need not be, familiar with all of them. It may be reasonable on theoretical grounds or on the basis of past experience to assume that observations come from a particular family of distributions. Also experience, backed by theory, suggests that for many measurements inferences based on the assumption that observations form a random sample from some normal distribution may not be misleading even if the normality assumption is incorrect. A theorem called the central limit theorem justifies these and other uses of the normal distribution, particularly in what are called asymptotic approximations. We often refer to these in this book. Parametric inference is sometimes inappropriate or even impossible. To assume that samples come from any specified family of distributions may be unreasonable. For example, we may not have examination marks on, say, a percentage scale for each candidate but know only the numbers of candidates in banded and ordered grades designated Grade A, Grade B, Grade C, etc. Given these numbers for two different schools, we may want to know if they indicate a difference in performance between schools that might be due to unequal standards of teaching or the ability of one school to attract more able pupils. The method of inference is then usually nonparametric. Even when we have precise measurements it may be irrational to assume a normal distribution because normality implies certain properties of symmetry and spread. We may be able to see that a sample is not from a normal distribution simply by looking at a few characteristics like the sample mean, median, standard deviation and range of values. For example, if all observations are either zero or positive and the standard deviation is appreciably greater than the mean then, unless the sample is very small, it is unreasonable to assume it comes from a normal distribution (see Exercise 1.9). There are well-known distributions that are flatter than the normal (e.g. the continuous uniform, or rectangular, distribution) or skew (e.g. the exponential and gamma distributions). In practice we are often able to say little more than that our sample appears to come from a distribution that is skew, or very peaked, or very flat, etc. Here nonparametric inference may again be appropriate. In this latter situation some writers prefer the term distribution-free to nonparametric. The terms ‘distribution-free’ and ‘nonparametric’ are sometimes regarded as synonymous. Indeed, in defining a distribution-free
©2001 CRC Press LLC
method Marriott (1990) states that ‘Distribution-free inference or distribution-free tests are sometimes known as nonparametric but this usage is confusing and should be avoided’. We believe that view ignores subtle distinctions, though these usually need not worry us unduly. Indeed, the nature of these distinctions depends to some extent on how one defines terms like parameter and distribution. Statisticians agree that the constants like µ , σ2 that define family members in the normal distribution are parameters; many also argue that the mean, say, of any distribution that has a mean (precisely to what family the distribution belongs need not be specified) is a parameter even if it does not appear as a constant in the distribution function. This concept accords with the broader mathematical definition of a parameter as an unknown quantity that may take one of a set of possible values. Those who take this view argue that if we make inferences about the mean of a distribution without making any assumption about its nature or its belonging to a particular family it is still an inference about a parameter and so the name nonparametric is unjustified. However, it would be appropriate in these circumstances to say the inference is distribution-free when no assumption is made about the nature of the distribution. In practice this is usually an over-simplification; in nearly all inferences we make some distributional assumptions. For example, we may assume a sample is from a symmetric distribution. This does not restrict us to a particular family such as the normal but it does mean we exclude all asymmetric distributions. That there is room for argument about precisely what we mean by the terms nonparametric or distribution-free is reinforced by the growth in compromise terms for methods that do not fall clearly into areas covered by these names. Two examples are the descriptors semiparametric and asymptotically distribution-free. Paradoxically, many tests that are generally regarded as nonparametric or distribution-free involve parameters and distributions (often the normal or binomial distributions). This is because the tags ‘nonparametric’ or ‘distribution-free’ relate not to the distribution of the test statistics, but to the fact that the methods can be applied to samples from populations having distributions which need only be specified in broad terms, e.g. as being continuous, symmetric, identical, differing only in median or mean, etc. They need not belong to a specified family. There is a grey area between what is clearly distribution-free and what is parametric inference. Some of the association tests described in Chapters 9 and 10 fall in this area.
©2001 CRC Press LLC
1.1.2 When we might use nonparametric methods Even if a parametric test does not depend critically on an assumption that samples come from a distribution in a particular family, when in doubt we may prefer a nonparametric test needing weaker assumptions. More importantly, nonparametric methods are often the only ones available for data that simply specify order (ranks) or counts of numbers of events or of individuals in various categories. Nonparametric methods are not assumption-free. In most statistical problems what we can deduce, by either parametric or nonparametric methods, depends upon what assumptions can validly be made. An example illustrates this. Example 1.1 A contract for making metal rods specifies, among other requirements, that not more than 2.5 per cent should have a cross-sectional diameter exceeding 30 mm. Some quality control methods that are used to see if this condition is met are based on counting the proportion of defectives (i.e. rods exceeding 30 mm diameter) in samples. Such tests can be extended to assess whether it is reasonable to assume that two machines each produce the same proportion of defectives. Even if we conclude that they do, it does not follow that the distribution of diameters is the same for each. For example, suppose that diameters of items produced by the first machine have a normal distribution with mean 27 mm and standard deviation 1.53 mm; normal distribution theory then tells us that 2.5 per cent of all items produced will have a diameter exceeding 30 mm. This is because, for any normal distribution, 2.5 per cent of all items have a diameter at least 1.96 standard deviations above the mean; in this case, 2.5 per cent exceed 27 + 1.96 × 1.53 ≈ 30. If the second machine produces items with diameters that are uniformly distributed between 20.25 and 30.25 mm (i.e. with mean diameter 25.25 mm) it is easy to see that once again 2.5 per cent would have diameters exceeding 30 mm (since any interval of 0.25 mm between 20.25 and 30.25 mm contains a proportion 1/40, i.e. 2.5 per cent, of all production). This uniform distribution situation is unlikely to be met in practice but this example shows that we may have the same proportion of defectives in two populations, yet each has a different mean and their distributions do not even belong to the same family. We consider a more realistic situation involving different means in Exercise 1.1. Clearly then the numbers of defectives alone cannot tell us whether the mean diameters of items produced by each machine are the same. However, if we also assume that the distributions of diameters for items from each machine differ, if at all, only in their means, then if we know the proportion of defectives in samples of, say, 200 from each machine, we could test whether the means can reasonably be supposed to be identical. The test would not be efficient. It would be better to measure the diameter of each item in smaller samples, and dependent upon the distributional assumptions one makes, use an appropriate parametric or nonparametric test. We say more about hypothesis tests in Section 1.3.
©2001 CRC Press LLC
Means and medians are widely used to indicate where distributions are centred. Both are often referred to broadly as ‘averages’. Because many people think of an average as the arithmetic mean we shall often be more formal and call them measures of centrality. Another name is measures of location but we use this only in a special context relating to what are called location and scale parameters. Tests and estimation procedures for measurement data are often about averages or centrality measures, e.g.: • Is it reasonable to suppose that a sample come from a population with a pre-specified mean or median? • Do two samples come from populations whose means differ by at least 10? • Given a sample, what is an appropriate estimate of the population mean or median? How good is that estimate? Increasingly, for example, in industrial quality control, interest is turning also to spread or dispersion, often measured by variance or standard deviation. Buyers of a new car or computer want not only a good average performance but also consistency. They expect their product to perform as well as those of other purchasers of that model. Success of a product often depends upon personal recommendations, so mixed endorsements – some glowing, others warning of niggling faults – are not good publicity. There are parametric and nonparametric methods for assessing spread or variability. Some nonparametric techniques require little information. We may test if it is reasonable to assume that weights of items have a prespecified median 2 mg, say, if all we know is how many items in a sample of n weigh more than 2 mg. If it were difficult, expensive, or impossible to get exact weights, but easy to determine numbers above (or below) 2 mg this nonparametric approach may be cost effective. Simple nonparametric methods are also useful if data are in some way incomplete, like those in Example 1.2. Example 1.2 In medical studies the progress of patients is often monitored for a limited time after treatment; often anything from a few months to 5 or 6 years. Dinse (1982) gives data for survival times in weeks for 10 patients with symptomatic lymphocytic non-Hodgkin’s lymphoma. The precise survival time is not known for one patient who was alive after 362 weeks. The observation for that patient is said to be censored. Survival times in weeks were 49, 58, 75, 110, 112, 132, 151, 276, 281, 362* The asterisk denotes a censored observation.
©2001 CRC Press LLC
Is it reasonable to suppose that these data are consistent with a median survival time of 200 weeks? Censored observations cause problems in many parametric tests but in Section 1.3.2 we use a simple nonparametric test to show there is no strong evidence against the hypothesis that the median is 200. For that interpretation to be meaningful and useful we have to assume the data are a random sample from some population of patients with the disease.
To confirm that the median might well be 200 is not in itself very helpful. It would be more useful if we could say that the data imply that it is reasonable to assert that the median survival time is between 80 and 275 weeks or something of that sort. This is what confidence intervals (Section 1.4.1) are about. Dinse was interested, among other things, in whether the median survival times differed between symptomatic and asymptomatic cases; he used this sample and another for 28 asymptomatic cases to compare the survival time distributions in more detail. In this other sample 12 of the 28 observations were censored at values of 300 or more. We show in Example 5.9 that, on the basis of his data, we would conclude that there is strong evidence that the medians for symptomatic and for asymptomatic cases are different. These data were also considered by Kimber (1990). 1.1.3 Historical note The first chapter of the Book of Daniel records that on the orders of Nebuchadnezzar certain favoured children of Israel were to be specially fed on the king’s meat and wine for 3 years. Reluctant to defile himself with such luxuries, Daniel pleaded that he and three of his brethren be fed instead on pulse for 10 days. After that time the four were declared ‘fairer and fatter in flesh than all of the children which did eat the portion of the king’s meat’. This evidence was taken on commonsense grounds to prove the superiority of a diet of pulse. In Example 1.4 and throughout the book we illustrate how we test evidence like this more formally to justify the commonsense conclusion arrived at in the Old Testament. Although the biblical analysis is informal it contains the germ of a nonparametric, as opposed to a parametric, test. John Arbuthnot (1710) observed that in each year from 1629 to 1710 the number of males christened in London exceeded the number of females. He regarded this as strong evidence against the probability of a male birth being 1⁄2 . The situation is somewhat akin to observing 82 heads in 82 consecutive tosses of a coin.
©2001 CRC Press LLC
Francis Galton (1892) developed a measure – which he termed a ‘centisimal scale’ – to assess agreement between patterns (categorical data) on corresponding fingertips on left and right hands. Karl Pearson (1900) proposed the well-known, and sometimes misused, chi-squared goodness-of-fit test applicable to any discrete distribution, and C. Spearman (1904) defined a rank correlation coefficient (see Section 7.1.3) that bears his name. Systematic study of nonparametric inference dates from the 1930s when attempts were made to show that even if an assumption of normality stretched credulity, then at least in some cases making it would not greatly alter conclusions. This stimulated work by R.A. Fisher, E.J.G. Pitman and B.L. Welch on randomization or permutation tests which were then too time consuming for general use, a problem now overcome with appropriate statistical software. About the same time it was realised that observations consisting simply of preferences or ranks could be used to make some inferences without too much computational effort. A few years later F. Wilcoxon and others showed that even if we have precise measurements, we sometimes lose little useful information by ranking them in increasing order of magnitude and basing analyses on these ranks. Indeed, when assumptions of normality are not justified, analyses based on ranks or on some transformation of them may be the most efficient available. Nonparametric methods then became practical tools either when data were by nature ordinal (ranks or preferences) or as reasonably efficient methods that reduced computation even when measurements were available providing those measurements could be replaced by ranks. While hypothesis testing was usually easy, unfortunately the more important interval estimation (Section 1.4) was not, a difficulty that has now been largely overcome In parallel with the above advances, techniques relevant to counts were developed. Counts often represent the numbers of items in categories which may be either ordered, e.g. examination grades, or nominal (i.e. unordered), e.g. in psychiatry characteristics like depression, anxiety, psychosis, etc. Many advanced and flexible nonparametric methods are tedious only because they require repeated performance of simple but monotonous calculations, something computers do well. The dramatic post-war development of feasible, but until recently often tedious to carry out, nonparametric procedures is described by Noether (1984). Kendall and Gibbons (1990) give more than 350 references for the one topic of rank correlation.
©2001 CRC Press LLC
Computers have revolutionized our approach to data analysis and statistical inference. Hope, often ill-founded, that data would fit a restricted mathematical model with few parameters and emphasis on simplifying concepts such as linearity has been replaced by the use of robust methods (Chapter 11) and by exploratory data analyses to investigate different potential models. These are areas where nonparametric methods sometimes have a central role. Generalized linear models described by McCullagh and Nelder (1989) at the theoretical level and by Dobson (1990) at the practical level often blend parametric and nonparametric approaches. Nonparametric methods are in no sense preferred methods of analysis for all situations. A strength is their applicability where there is insufficient theory or data to justify, or to test compatibility with, specific distributional models. At a more sophisticated level they are also useful, for example, in finding or estimating trends in large data sets that are otherwise difficult to detect due to the presence of disturbances usually referred to as ‘noise’. Recent important practical developments have been in computer software (Section 1.6) to carry out permutation and other nonparametric tests. Results for these may be compared with those given by asymptotic theory, which, in the past, was often used where its validity was dubious. Modern computing power has also led to increasing use of the bootstrap stemming from work by Efron and others since the early 1980s. Bootstrapping is a computer-intensive technique with both parametric and nonparametric versions. We give an introduction to this and related topics in Section 11.3. 1.2 SAMPLES AND POPULATIONS When making statistical inferences we often assume that observations are a random sample from some population. Specification of that population may or may not be precise. If we select 20 books at random (i.e. so that each book in the population has the same probability of inclusion in the sample) from 100 000 volumes in a library, and record the number of pages in each book in the sample, then inferences made from the sample about the mean or median number of pages per book apply strictly only to the population of books in that library. If the library covers a wide range of fiction, non-fiction and reference works it is reasonable to assume that any inferences apply, at least approximately, to a wider population of ©2001 CRC Press LLC
books, perhaps to all books published in the United Kingdom or the United States or wherever the library is situated. If the books in the library are all English language books, inferences may not apply to books in Chinese or in Russian. We seldom deal with strictly random samples from a so clearly specified population. More commonly, data form samples that may be expected to have the essential properties of a random sample from a vaguely-specified population. For example, if a new diet is tested on pigs and we measure weight gains for 20 pigs at one agricultural experimental station we might assume that these are something like a random sample from all pigs of that or similar breeds raised under such conditions. This qualification is important. Inferences might apply widely only if the experimental station adopted common farming practices and if responses were fairly uniform for many breeds of pig. This may not be so if the experimental station chose an unusual breed, adopted different husbandry practices from those used on most pig farms, or if the 20 pigs used in the experiment were treated more favourably than is usual in other respects, e.g. if they were kept in specially heated units during the experiment. The abstract notion of random sampling from an infinite population (implicit in most inference based upon normal distribution theory) often works well in practice, but is never completely true! At the other extreme there are situations where the sample is essentially the whole population. For example, at the early stages of testing a new drug for treating a rare disease there may be just, say, nine patients available for test and only four doses of the drug. One might choose at random the four patients from nine to receive the new drug, the remaining five are untreated or may be treated with a drug already in use. Because of the random selection, if the drug has no effect, or is no better than one currently in use, it is unlikely that a later examination would show that the four patients receiving the new drug had responded better than any of the others. This is possible, but it has a low probability, which we can calculate on the assumption that the new drug is ineffective or is no better than the old one. Clearly, if the drug is beneficial or better than that currently in use the probability of better responses among those treated with it is increased (see Example 1.4, Section 1.3.3). Because of the many ways data may be obtained we must consider carefully the validity and scope of inferences. For example, the ten patients for whom survival times were measured in Example 1.2 came from a study conducted by the Eastern Co-operative
©2001 CRC Press LLC
Oncology Group in the USA and represented all patients afflicted with symptomatic lymphocytic non-Hodgkin’s lymphoma available for that study. In making inferences about the median or other characteristics of the survival time distribution, it is reasonable to assume these inferences are valid for all patients receiving similar treatment and alike in other relevant characteristics, e.g. with a similar age distribution. The patients in this study were all male, so clearly it would be unwise to infer, without further evidence, that the survival times for females would have the same distribution. Fortunately the same nonparametric procedures are often valid whether samples are from an infinite population, a finite population or when the sample is the entire relevant population. What is different is how far the inferences can be generalized. The implications of generalizing inferences is described for several specific tests by Lehmann (1975, Chapters 1–4) and is discussed in more detail than in this book in Sprent (1998, Chapter 4). In Section 1.1.1 we referred to a set of n independent observations from a normal distribution. Independence implies that knowing one value tells us nothing about other values. For any distribution a sequence of n independent observations all having the same distribution is a random sample from that distribution. Independence is an important, and by no means trivial, requirement for the validity of many statistical inferences. 1.3 HYPOTHESIS TESTS 1.3.1 Basic concepts Estimation (Section 1.4) is often a key aim of statistical analysis. It is easy to explain in terms of testing a range of hypotheses, so we need to understand testing even though it is a technique that tends to be both overused and misused. We assume familiarity with simple parametric hypothesis tests such as the t-test and chi-squared test, but we review here some fundamentals and changes in interpretation made possible by modern computer software. Until such software became widely available hypothesis testing was often based on tables. The familiar t-test is often used to test hypotheses about the unknown mean µ of a normal distribution. We specify a null hypothesis, H0, that µ takes a specific value µ 0 and an alternative hypothesis, H1, is that it takes some other value. Formally this is stated as: ©2001 CRC Press LLC
Test H0: µ = µ0 against H1: µ ≠ µ0
(1.1)
The test is based on a statistic, t, that is a function of the sample values calculated by a formula given in most statistics textbooks. The classic procedure using tables was to compare the magnitude, without regard to sign, of the calculated t, often written |t|, with a value tα given in tables, the latter chosen so that when H0 was true Pr(|t| ≥ tα) = α.
(1.2)
In practice α nearly always took one of the values 0.05, 0.01 or 0.001. These probabilities are often expressed as equivalent percentages, i.e. 5, 1 or 0.1 and are widely known as significance levels. Use of these particular levels was dictated at least in part by available tables. In this traditional approach if one obtained a value of |t| ≥ tα the result was said to be significant at probability level α or at the corresponding 100α per cent level, and these levels were often referred to respectively as ‘significant’, ‘highly significant’ and ‘very highly significant’. If significance at a particular level was attained one spoke of rejecting the hypothesis H 0 at that level. If significance was not attained the result was described as n o t significant and H 0 was said to be accepted. This is unfortunate terminology giving – especially to nonstatisticians – the misleading impression that nonsignificance implies that H0 is true, while significance implies it is false. The rationale behind the test (1.1) is that if H0 is true then values of t near zero are more likely than large values of t, either positive or negative. Large values of |t| are more likely to occur under H1 than under H0. It follows from (1.2) that if we perform a large number of such tests on different independent random samples when H0 is true we shall in the long run incorrectly reject H 0 in a proportion α of these; e.g. if α = 0.05 we would reject the null hypothesis when it were true in the long run in 1 in 20 tests, i.e. in 5 per cent of all tests. The traditional approach is still common, especially in certain areas of law, medicine and commerce, or to conform with misguided policy requirements of some scientific and professional journals. Modern statistical software lets us do something more sensible though by itself still far from satisfactory. The output from any good modern software for a t-test relevant to (1.1) gives the exact probability of obtaining, when H0 is true, a value of |t| equal to or greater than that observed. In statistical jargon this probability is usually called a P-value and it is a measure of the strength of evidence against H 0 provided by the data – the smaller P is, the stronger is that evidence.
©2001 CRC Press LLC
Using such evidence one formal possibility when it is strong is to decide H 0 is implausible. When we decide what P-values are sufficiently small for H0 to be implausible we may speak formally of rejecting H 0 at the exact 100P per cent significance level. This avoids the difficulty that rigid application of the 5 per cent significance level leads to the unsatisfactory situation of H 0 being rejected for a P-value of 0.049 but accepted for the slightly larger P-value of 0.051. It does not, however, overcome a more serious objection to this approach, namely, that with small experiments one may never observe small P-values even when H0 is n o t true. Suppose a coin is tossed 5 times to test the hypothesis H0 : the coin is fair (equally likely to fall heads or tails) against H1: the coin is biased (either more likely to fall heads or more likely to fall tails). Clearly in 5 tosses the strongest evidence against H0 is associated with the outcomes 5 heads or 5 tails. Under H0 the probability (P) of getting one of these outcomes is given by the sum of the probabilities of r = 0 or r = 5 ‘heads’ for a binomial B(5, 1 /2) distribution, so P = 2 × (1/2)5 = 0.0625. This is the smallest attainable P-value when n = 5 and p = 1 /2 and so we never reject H0 at a conventional P = 0.05 level whatever the outcome of the 5 tosses – even if the coin is a double-header! That the experiment is too small is the only useful information given by the P-value in this example. The situation is very different if we increase the experiment to 20 tosses and get 20 heads or 20 tails. This weakness of hypothesis testing together with the perpetration of myths such as equating accepting a hypothesis to proof it is true has led to justified criticism of what is sometimes called the P-value culture. Krantz (1999) and Nelder (1999) both highlight dangers arising from inappropriate use of and misunderstandings about what a P-value implies. We draw attention also to an ethical danger near the end of Section 1.5. In the real world policy decisions are often based on the findings of statistical analyses; if the evidence against H 0 is strong a formal rejection of H 0 on the ground that it is implausible may be appropriate. To appreciate the implications of either a formal rejection of H0 or of a decision not to reject it at a given significance level we need some additional concepts. If we decide to reject H 0 whenever our P-value is less than some fixed value P0, say, then, if in all cases where we do so H0 is true, we would in the long run reject it in a proportion P0 of those cases. Rejection of H0 when it is true is an error of the first kind, or Type I error. What a P-value therefore tells us is the probability that we are making an error of the first kind by rejecting H0. In a t-test if we make a decision to reject ©2001 CRC Press LLC
H0 whenever we observe a P ≤ P0 we do so when |t| ≥ k where k is such that when H0 holds Pr(|t| ≥ k) = P0 and these values of t define a critical or rejection region of size P0. Using a critical region of size P0 implies we continue to regard H 0 as plausible if |t| < k. If we follow this rule we shall sometimes (or as we saw above for the 5 coin tosses, in extreme cases, always) continue to regard H0 as plausible when, in fact, H1 is true. Continuing to accept H0 when H1 is true is an error of the second kind, or Type II error. Let β denote the probability of a Type II error. The probability of a Type II error depends in part on the true value of µ if indeed it is not equal to µ0. Intuition correctly suggests that the more µ differs from µ0, the more likely we are to get large values of |t|, i.e. values in the critical region, so that β decreases as |µ – µ 0| increases. If we decrease P0 (say from 0.03 to 0.008) our critical region becomes smaller, so that for a given µ we increase β because the set of values of t for which we accept H0 is larger. The other factor affecting β is the sample size, n. If we increase n we decrease β for a given µ and P0. Thus β depends on the true value of µ (over which we have no control) and the value of n and of P0 determining the size of the critical region (for both of which we often have some choice). Despite obvious limitations P-values used properly and constructively have a basic role in statistical inference. In Section 1.4 we shall see that a null hypothesis that specifies one single value of a parameter is usually just one of many possible hypotheses that are not contradicted by the sample evidence. Donahue (1999) and Sackrowitz and Samuel-Cahn (1999) discuss various distributional properties of the P-value that relate indirectly to uses we discuss here and in Section 1.4. Fixing the probability of an error of the first kind, whether we denote it by the conventional symbol a or the alternative P0 does not determine β. We want β to be small because, in the t-test for example, we want the calculated t-value to be in the critical region when H0 is not true. Clearly 1 – β is the probability of getting a t-value in the critical region when H0 is not true. The quantity 1 – β is called the power of the test; we want this to be large. For samples from a normal distribution and all choices of n, P and for any µ, the t-test is more powerful than any other test of the form (1.1). The historical choice of significance levels 5, 1 and 0.1 per cent as the basis for tables was made on the pragmatic grounds that one does not want to make too many errors of the first kind. It clearly would be silly to choose a significance level of 50 per cent, for then we would be equally likely to accept or to reject H0 when it were
©2001 CRC Press LLC
true. Even with conventional significance levels or other small Pvalues we may often make errors of the second kind in a t-test (or any other test) if it has low power for one or more of these reasons: • • • •
µ is close to the value specified in H0, or the sample size is small, or we specify a very small P-value for significance, or assumptions required for the test to be valid are violated.
How we proceed when a test gives a nonsignificant result depends on our aims and objectives. In later chapters we consider for some tests the often non-trivial problem of determining how big an experiment is needed to ensure reasonable power to achieve given objectives. Using small P-values in place of traditional 5, 1 and 0.1 per cent significance levels gives more freedom in weighing evidence for or against a null hypothesis. Remembering that P = 0.05 corresponds to the traditional 5 per cent significance level long used as a reasonable watershed, one should not feel there is strong evidence against a null hypothesis if P is much greater than 0.05, but values of P not greatly exceeding 0.05 often point at least to a case for further studies – often to a need for larger experiments. In this book we shall usually discuss the evidence for or against hypotheses in terms of observed P-values but in some situations where it is appropriate to consider a hypothetical fixed P-value we use for this the notation α with an implication that we regard any P-value less than that α as sufficient evidence to prefer H 1 to H 0. This complies with certain longestablished conventions. The test in (1.1) is called a two-tail test because the critical region consists both of large positive and large negative values of the statistic t. More specifically, large positive values of t usually imply µ > µ0 and large negative values of t imply µ < µ0. Specification of H 0 and H1 is determined by the logic of a problem. Two other common choices are (i) (ii)
Test H0: µ = µ0 against H1: µ > µ0 Test H0: µ ≤ µ0 against H1: µ > µ0
(1.3) (1.4)
both leading to a one-tail (here right or upper-tail) test, since in each case when the t-test is relevant large positive values of t favour H1, whereas a small positive value or any negative value indicates that H0 is more likely to hold. The modifications to a one-tail test if the inequalities in (1.3) or (1.4) are reversed are obvious. The critical region then becomes the left, or lower, tail. ©2001 CRC Press LLC
For example, if the amount of a specified impurity in 1000g ingots of zinc produced by a standard process is normally distributed with a mean of 1.75g and it is hoped that a steam treatment will remove some of this impurity we might steam-treat a sample of 15 ingots and determine the amount of impurity left in each ingot. If the steam is free from the impurity the treatment cannot increase the level and either it is ineffective or it reduces the impurity. It is therefore appropriate to test H0: µ = 1.75 against H1: µ < 1.75. If ingots have an unknown mean impurity level, but a batch is acceptable only if µ ≤ 1.75, the appropriate test is H0: µ ≤ 1.75 against H1: µ > 1.75. For the t-test some computer packages give a P-value appropriate for a one-tail test, e.g. Pr(t ≥ tP) = P. Because the distribution of t is symmetric one doubles this probability to obtain P for a two-tail test. The doubling of one-tail probabilities to give the corresponding twotail test probability or significance level applies in other parametric tests such as the F-test for equality of variance based on samples from two normal populations, but in these cases the two subregions are not symmetric about the mean. However, in many applications where relevant statistics have a chi-squared or F-distribution a onetail (upper-tail) test is appropriate. A common misconception is that a low P-value indicates a departure from the null hypothesis that is of practical importance. We show why this is not necessarily true in Section 1.4.2. Knowing exact tail probabilities is, however, useful when comparing the performance of different tests. 1.3.2 Some nonparametric test statistics Some statistics used for nonparametric tests have, at least approximately, familiar continuous distributions such as the normal, t, F or chi-squared distribution. However, we shall use many statistics that have discontinuous distributions and this raises a further problem that has a practical impact mainly with small samples. Example 1.2 (continued) In this example we want to test the hypothesis that the median θ of survival times for the population is 200 against the alternative of some other value, i.e. to test H0: θ = 200 against H1: θ ≠ 200
©2001 CRC Press LLC
(1.5)
A simple test needs only a count of the number of sample values exceeding 200 (recording each as a ‘plus’). By the definition of a random sample and that of a population median, if we have a random sample from any distribution with median 200 each sample value is equally likely to be above or below 200. This means that under H 0 the number of plus signs has a binomial B(10, 1/2) distribution. The probability of observing r plus signs in 10 observations when p = 1/2 is given by the binomial formula pr = Pr ( X = r ) = (10r )( 1 2 )10 where the binomial coefficient (1r0) = 10!/[r!(10–r)!] and r! is the product of all integers between 1 and r (called factorial r) and 0! = 1. The values of these probabilities, pr, for each value of r between 0 and 10, correct to 4 decimal places, are R P
0
1
0.0010 0.0098
2 0.0439
3 0.1172
4 0.2051
5 0.2461
6 0.2051
7 0.1172
8 0.0439
9 0.0098
10 0.0010
In the data 3 observations exceed 200 so there are 3 plus signs and from the table above we see that the probability of 3 or less plus signs in a sample of 10 is 0.1172 + 0.0439 + 0.0098 + 0.0010 = 0.1719 when H0 is true. There is clearly no strong evidence against H0, since the probability of getting 3 or less plus signs or 7 or more plus signs (the two relevant ‘tail’ probabilities for our observed statistic, the number of plus signs) is 2 × 0.1719 = 0.3438, implying that departures from the expected number of plus signs, 5, as large or larger than that observed will occur in slightly more than one-third of all samples when H 0 is true. This simple test, called the sign test, is discussed more fully in Section 2.3. In t-tests all values of P between 0 and 1 are possible, but for the sign test only certain discrete P-values occur. In this example for a two-tail test the three smallest are P = 2 × (0.0010) = 0.0020 (rounded to 4 decimal places) corresponding to 0 or 10 plus; and P = 2 × (0.0010 + 0.0098) = 0.0216 corresponding to 1 or 9 plus; then P = 2 × (0.0010 + 0.0098 + 0.0439) = 0.1094 corresponding to 2 or 8 plus; next comes the observed P = 0.3438. For a one-tail test these P–values are all halved. Our statistic – the number of plus signs – has a discrete distribution. This means there is no direct way of obtaining a critical region of exact size 0.05 for a two-tail test; we must choose between regions of size 0.0216 or 0.1094.
A device called a randomization procedure has been proposed with the property that in the long run an error of the first kind occurs in repeated testing with a probability at some prechosen nominal level, e.g. at 5 per cent, rather than at an exact level in each case. In practice our prime interest is what happens in our one test, so it is better, when we know them, to use exact levels, rather than worry about nominal arbitrary levels. There is, however, when there are discontinuities, a case for forming a tail probability by allocating
©2001 CRC Press LLC
only one half of the probability that the statistic equals the observed value to the ‘tail’ when determining the size of the ‘critical’ region. This approach has many advocates. We do not use it in this book, but if it is used this should be done consistently. Most modern computer programs calculate exact P-values for many parametric and for some nonparametric tests, but pragmatism means we may deplore, but cannot completely ignore, the established concept of nominal levels like 5 or 1 per cent entirely, for if we have no program that gives an exact P-value we must use tables where information for many tests (e.g. the Wilcoxon–Mann– Whitney test described in Section 5.2) is sometimes only given for such nominal levels, thus forcing users to quote these nominal levels rather than the exact P-value. Care is needed in interpreting P-values especially in one-tail tests, because most computer programs for nonparametric tests quote the probability that a value greater than or equal to the test statistic will be attained if this probability is less than 1/2, otherwise they give the probability that a value less than or equal to the test statistic is obtained. This is the probability of errors of the first kind in a onetail test if we decide to reject at a significance level equal to that probability. In practice the evidence against H0 is only rated strong if this ‘tail’ probability is sufficiently small and is in the appropriate tail! In general, we recommend doubling a one-tail probability to obtain the actual significance level for a two-tail test, but see Example 1.3 and the remarks following it. If the test statistic has a symmetric distribution doubling is equivalent to considering equal deviations from the mean value of the statistic. If the statistic does not have a symmetric distribution, taking tails equidistant from the mean is not equivalent to doubling a one-tail probability. Example 1.3 shows another difficulty that sometimes arises due to discontinuities in P-values; namely, that if we only regard the evidence against H0 as strong enough to reject it if P ≤ 0.05 (or at any rate a value not very much greater than this), we may never get that evidence because no outcome provides it, a problem we have already alluded to with small experiments. Example 1.3 In a dental practice, experience has shown that the proportion of adult patients requiring treatment following a routine inspection is 3/4, so the number of individuals requiring treatment, S, in a sample of 10 independent patients has a binomial B(10, 3/4) distribution. Here the probabilities for the various values r of the statistic S, where r takes integral values between 0 and 10 are given by
©2001 CRC Press LLC
pr = Pr ( S = r ) = (10r )( 3 4 ) r ( 1 4) 10− r
The relevant probabilities are __________________________________________________________________ r 0 1 2 3 4 5 6 7 8 9 10 pr 0.0000 0.0000 0.0004 0.0031 0.0162 0.0584 0.1460 0.2503 0.2816 0.1877 0.0563
_______________________________________________________________ Thus, if we had data for another practice and wanted for that practice to test H0: p = 3/4 against H1: p > 3/4, the smallest P-value for testing is in the upper tail and is associated with r = 10. i.e. P = 0.0563, so if we only regard P ≤ 0.05 as sufficiently strong evidence to discredit H0 such values are never obtained. There is no problem here for a one-tail test of H0: p = 3/4 against H1: p < 3/4 since in the appropriate lower tail P = Pr(S ≤ 4) = 0.0162 + 0.0031 + 0.0004 = 0.0197. This example also shows a logical difficulty associated with the rule that the appropriate level for a two-tail test is twice that for a one-tail test, for if we get S = 4 the two-tail test level based on this rule is 2 × 0.0197 = 0.0394. This presents a dilemma, for there is no observable upper tail area corresponding to that in the lower tail. This means that if a two-tail test is appropriate we shall in fact only be likely to detect departures from the null hypothesis if they are in one direction; there may well be a departure in the other direction, but if so we are highly unlikely to detect it at the conventional level P ≤ 0.05, and even if we did it would be for the wrong reason (see Exercise 1.10). This is not surprising when, as shown above, the appropriate one-tail test must fail to detect it, for generally a one-tail test at a given significance level is more powerful for detecting departures in the appropriate direction than is a two-tail test at the same level. The implication is that we need a larger sample to detect departures of the form H1: p > 3/4 in this example. Again the fairly large P-value associated with the result most likely under this H1 only tells us our sample is too small. The stipulation that the patients be independent is important. If the sample included three members of the same family it is quite likely that if one of them were more (or less) likely to require treatment than the norm, this may also be the case for other members of that family.
Statisticians do not all agree that one should double a one-tail probability to get the appropriate two-tail significance level [see for example, Yates (1984) and the discussion thereon]. An alternative to doubling the one-tail probability is that once the exact size of a onetail region has been determined, we should, for a two-tail test, add the probabilities associated with an opposite tail situated equidistant from the mean value of the test statistic to that associated with our observed statistic value. In the symmetric case we have already pointed out that this is equivalent to doubling the probability, but it seems inappropriate with a non-symmetric distribution. In Example 1.3 the region r ≤ 4 is appropriate for a lower-tail test. The mean of the test statistic (the binomial mean np) is 7.5. Since 7.5 – 4 = 3.5, the corresponding deviation above the mean is 7.5 + 3.5 = 11. ©2001 CRC Press LLC
Because Pr(r ≥ 11) = 0, the two-tail test based on equidistance from the mean would have the same exact significance level as the onetail test. 1.3.3 Permutation tests Example 1.4 In Section 1.2 we envisaged a situation where four from nine patients are selected at random to receive a new drug. After three weeks all nine patients are examined by a skilled consultant who, on the basis of various tests and clinical observations ‘ranks’ the patients’ conditions in order from least severe (rank 1) to most severe (rank 9). If there is no beneficial effect of the new drug, what is the probability that the patients who received the new drug are ranked 1, 2, 3, 4? Selecting four patients ‘at random’ means that any four are equally likely to be given the new drug. If there really is no effect one would expect some of those chosen to end up with low ranks, some with moderate or high ranks, in the post-treatment assessment. From a group of nine patients there are 126 ways of selecting a set of four. This may be verified by using the well-known mathematical result that the number of ways of selecting r objects from n is n!/[r!(n – r)!]. We give all 126 selections in Table 1.1. Ignore for the moment the numbers in brackets after each selection. If the new drug were ineffective the set of ranks associated with the four patients receiving it are equally likely to be any of the 126 quadruplets listed in Table 1.1 Possible selections of four individuals from nine labelled 1 to 9 with the sum of the ranks in brackets. _______________________________________________________________ 1,2,3,4 (10) 1,2,3,5 (11) 1,2,3,6 (12) 1,2,3,7 (13) 1,2,3,8 (14) 1,2,3,9 (15) 1,2,4,5 (12) 1,2,4,6 (13) 1,2,4,7 (14) 1,2,4,8 (15) 1,2,4,9 (16) 1,2,5,6 (14) 1,2,5,7 (15) 1,2,5,8 (16) 1,2,5,9 (17) 1,2,6,7 (16) 1,2,6,8 (17) 1,2,6,9 (18) 1,2,7,8 (18) 1,2,7,9 (19) 1,2,8.9 (20) 1,3,4,5 (13) 1,3,4,6 (14) 1,3,4,7 (15) 1,3,4,8 (16) 1,3,4,9 (17) 1,3,5,6 (15) 1,3,5,7 (16) 1,3,5,8 (17) 1,3,5,9 (18) 1,3,6,7 (17) 1,3,6,8 (18) 1,3,6,9 (19) 1,3,7,8 (19) 1,3,7,9 (20) 1,3,8,9 (21) 1,4,5,6 (16) 1,4,5,7 (17) 1,4,5,8 (18) 1,4,5,9 (19) 1,4,6,7 (18) 1,4,6,8 (19) 1,4,6,9 (20) 1,4,7,8 (20) 1,4,7,9 (21) 1,4,8,9 (22) 1,5,6,7 (19) 1,5,6,8 (20) 1,5,6,9 (21) 1,5,7,8 (21) 1,5,7,9 (22) 1,5,8,9 (23) 1,6,7,8 (22) 1,6,7,9 (23) 1,6,8,9 (24) 1,7,8,9 (25) 2,3,4,5 (14) 2,3,4,6 (15) 2,3,4,7 (16) 2,3,4,8 (17) 2,3,4,9 (18) 2,3,5,6 (16) 2,3,5,7 (17) 2,3,5,8 (18) 2,3,5,9 (19) 2,3,6,7 (18) 2,3,6,8 (19) 2,3,6,9 (20) 2,3,7,8 (20) 2,3,7,9 (21) 2,3,8,9 (22) 2,4,5,6 (17) 2,4,5,7 (18) 2,4,5,8 (19) 2,4,5,9 (20) 2,4,6,7 (19) 2,4,6,8 (20) 2,4,6,9 (21) 2,4,7,8 (21) 2,4,7,9 (22) 2,4,8,9 (23) 2,5,6,7 (20) 2,5,6,8 (21) 2,5,6,9 (22) 2,5,7,8 (22) 2,5,7,9 (23) 2,5,8,9 (24) 2,6,7,8 (23) 2,6,7,9 (24) 2,6,8,9 (25) 2,7,8,9 (26) 3,4,5,6 (18) 3,4,5,7 (19) 3,4,5,8 (20) 3,4,5,9 (21) 3,4,6,7 (20) 3,4,6,8 (21) 3,4,6,9 (22) 3,4,7,8 (22) 3,4,7,9 (23) 3,4,8,9 (24) 3,5,6,7 (21) 3,5,6,8 (22) 3,5,6,9 (23) 3,5,7,8 (23) 3,5,7,9 (24) 3,5,8,9 (25) 3,6,7,8 (24) 3,6,7,9 (25) 3,6,8,9 (26) 3,7,8,9 (27) 4,5,6,7 (22) 4,5,6,8 (23) 4,5,6,9 (24) 4,5,7,8 (24) 4,5,7,9 (25) 4,5,8,9 (26) 4,6,7,8 (25) 4,6,7,9 (26) 4,6,8,9 (27) 4,7,8,9 (28) 5,6,7,8 (26) 5,6,7,9 (27) 5,6,8,9 (28) 5,7,8,9 (29) 6,7,8,9 (30) ___________________________________________________________________________
©2001 CRC Press LLC
Table 1.2 Sums of ranks of four items from nine. _______________________________________________________ Sum of ranks Number of occurrences
10 11 12 13 14 15 16 17 18 19 20 1 1 2 3 5 6 8 9 11 11 12
Sum of ranks 21 22 23 24 25 26 27 28 29 30 Number of occurrences 11 11 9 8 6 5 3 2 1 1 _______________________________________________________ Table 1.1. Thus, if there is no treatment effect there is only 1 chance in 126 that the four showing greatest improvement (ranked 1, 2, 3, 4 in order of condition after treatment) are the four patients allocated to the new drug. It is more plausible that such an outcome may reflect a beneficial effect of the drug. In a hypothesis testing framework we have a group of 4 treated with the new drug and a group of 5 (the remainder) given nothing or a standard treatment in what is called a two independent sample experiment. We discuss such experiments in detail in Chapter 5. The most favourable evidence for the new drug would be that those receiving it are ranked 1, 2, 3, 4; the least favourable that they are ranked 6, 7, 8, 9. Each of these extremes has a probability of 1/126 of occurring when there is no real effect. If we consider a test of H0: new drug has no effect against the two-sided alternative H1: new drug has an effect (beneficial or deleterious) the outcome 1, 2, 3, 4 and 6, 7, 8, 9 are extremes with a total associated probability P = 2/126 ≈ 0.0159 if H0 is true. In classic testing terms we might speak of rejecting H0 at an exact 1.59 per cent significance level if we observed either of these outcomes. This small P-value provides strong evidence that the new drug has an effect. What if the patients receiving the new drug were ranked 1, 2, 3, 5? Intuitively this evidence looks to favour the new drug, but how do we test this? We seek a statistic, i.e. some function of the four ranks, that has a low value if all ranks are low, a high value if all ranks are high and an intermediate value if there is a mix of ranks for those receiving the new drug. An intuitively reasonable choice is the sum of the four ranks. If we sum the ranks for every quadruplet in Table 1.1, and count how many times each sum occurs we easily work out the probability of getting any particular sum, and hence the distribution of our test statistic when H0 is true. In Table 1.1 the number in brackets after each quadruplet is the sum of the ranks for that quadruplet, e.g. for 1, 2, 7, 9 the sum is 1 + 2 + 7 + 9 = 19. The lowest sum is 10 for 1, 2, 3, 4 and the highest is 30 for 6, 7, 8, 9. Table 1.2 gives the numbers of quadruplets having each given sum. Because there are 126 different but equally likely sets of ranks the probability that the rank sum statistic, which we denote by S, takes a particular value is obtained by dividing the number of times that value occurs by 126. Thus, for ©2001 CRC Press LLC
example, Pr(S = 17) = 9/126 ≈ 0.0714. To find what outcomes are consistent with a P-value not exceeding 0.05 we select a region in each tail (since H1 implies a two-tail test) with a total associated probability not exceeding 0.025. Clearly, if we select in the lower tail S = 10 and 11, the associated probability is 2/126 and if we add S = 12 the associated total probability, i.e. Pr(S ≤ 12) = 4/126 ≈ 0.0317. This exceeds 0.025, so our lower-tail critical region should be S ≤ 11 with P = 2/126 ≈ 0.0159. By symmetry, the upper-tail region is S ≥ 29 also with P = 0.0159. Thus, for a two-tail test the largest symmetric critical region with P ≤ 0.05 is S = 10, 11, 29, 30 and the exact P = 4/126 ≈ 0.0317. Some statisticians suggest choosing a critical region with probability as close as possible to a target level such as P = 0.05 rather than the more conservative choice of one no larger. In this example adding S = 12 and the symmetric S = 28 to our critical region gives a two-tail P = 8/126 ≈ 0.0635. This is closer to 0.05 than the size (0.0317) of the region chosen above. We reaffirm that ideally it is best to quote the exact P-value obtained and point out again that the practical argument (though there are further theoretical ones) for quoting nominal sizes such as 0.05 is that many tables give only these, although a few, e.g. Hollander and Wolfe (1999, Table A6) give relevant exact P-values for many sample size c o m b i n a t i o n s an d d i f f e r e n t v a l u e s o f S . Co m p u t e r p r o g r a m s g i v i n g ex a c t P-values overcome any difficulty if the latter type of table is not readily available. Unless there were firm evidence before the experiment started that an effect, if any, of the new drug could only be beneficial, a two-tail test is appropriate. We consider a one-tail test scenario in Exercise 1.5. In Section 1.2 we suggested that in preliminary testing of drugs for treating a rare disease our population may be in a strict sense only the cases we have. However, if these patients are fairly typical of all who might have the disease, it is not unreasonable to assume that findings from our small experiment may hold for any patients with a similar condition providing other factors (nursing attention, supplementary treatments, consistency of diagnosis, etc.) are comparable. When our experiment involves what is effectively the whole population and the only data are ranks, a permutation test is the best test available. Random allocation of treatments is essential for the test to be valid; this may not always be possible in the light of some ethical considerations that we discuss in Section 1.5.
Tests based on permutation of ranks or on permutation of certain functions of ranks (including as a special case the original measurements) are central to nonparametric methods and are called permutation or randomization tests. Not only do many of these tests have a commonsense intuitive appeal but they comply with well established theoretical criteria for sound inference. This theoretical basis is discussed in the literature and is summarized by Hettmansperger and McKean (1998) for many different procedures. Small scale tests of a drug like that in Example 1.4 are often called pilot studies. Efficacy of a drug in wider use may depend on factors like severity of disease, treatment being administered sufficiently early, the age and sex of patients, etc., all or none of
©2001 CRC Press LLC
which may be reflected in a small group of ‘available patients’. An encouraging result with the small group may suggest further experiments are desirable; a not very small P-value associated with what looks to be an intuitively encouraging result may indicate that a larger experiment is needed to tell us anything useful. 1.3.4 Pitman efficiency We pointed out in Section 1.3.1 that the power of a test depends upon the sample size, n, the choice of the largest P-value to indicate significance in the classical sense (usually denoted in power studies by α), the magnitude of any departure from H 0 and whether assumptions needed for validity hold. Most intuitively reasonable tests have good power to detect a true alternative that is far removed from the null hypothesis providing the data set is large enough. We sometimes want tests to have as much power as possible for detecting alternatives close to H0 even when these are of no practical importance because such tests are usually also good at detecting larger departures, a desirable state of affairs. If α is the probability of a Type I error and β is the probability of a Type II error (the power is 1 – β), then the efficiency of a test T2 relative to a test T1 is the ratio n1/n2 of the sample sizes needed to obtain the same power for the two tests with these values of α, β. In practice, we usually fix α at some P-value appropriate to the problem at hand; then β depends on the particular alternative as well as the sample sizes. Fresh calculations of relative efficiency are required for each particular value of the parameter or parameters of interest in H1 and for each choice of α, β. Pitman (1948), in a series of unpublished lecture notes, introduced the concept of asymptotic relative efficiency for comparing two tests. He considered sequences of tests T1, T 2 in which we fix α but allow the alternative in H1 to vary in such a way that β remains constant as the sample size n1 increases. For each n1 we determine n2 such that T2 has the same β for the particular alternative considered. Increasing sample size usually increases the power for alternatives closer to H0, so that for large samples Pitman studied the behaviour of the efficiency, n 1/n2, for steadily improving tests for detecting small departures from H0. He showed under very general conditions that in these sequences of tests n1/n2 tended to a limit as n1 → ∞, and more importantly, that this limit which he called the asymptotic relative efficiency (ARE) was the same for all choices of α , β . Bahadur (1967) proposed an alternative definition that is less widely ©2001 CRC Press LLC
used, so for clarity and brevity we refer to Pitman’s concept simply as the Pitman efficiency. The concept is useful because when comparing two tests the small sample relative efficiency is often close to, or even higher, than the Pitman efficiency The Pitman efficiency of the sign test relative to the t-test when the latter is appropriate is 2/π ≈ 0.64. Lehmann (1975, p.173) shows that for samples of size 10 and a wide range of values of the median θ relative to the value θ 0 specified in H0 with α fixed, the relative efficiency exceeds 0.7, while for samples of 20 it is nearer to, but still above, 0.64. Here Pitman efficiency gives a pessimistic picture of the performance of the sign test at realistic sample sizes. We have already mentioned that when it is relevant and valid the t-test for a mean is the most powerful test for any mean specified in H0 against any alternative. When the t-test is not appropriate, other tests may have higher efficiency. Indeed, if our sample comes from the double exponential distribution, which has much longer tails than the normal, the Pitman efficiency of the sign test relative to the t-test is 2. That is, a sign test using a sample of n is (at least for large samples) as efficient as a t-test applied to a sample of size 2n. 1.4 ESTIMATION 1.4.1 Confidence intervals The sample mean is widely used as a point estimate of a population mean. The sample mean varies between samples so we need a measure of the precision of this estimate. A confidence interval is one such measure. One way to specify a 100(1 – α) per cent confidence interval for a parameter θ is to define it as the set of all values θ for which, if any such θ were specified in H0, for the given data the test would lead to a P > α. Thus, if a confidence interval includes the value of a parameter that is specified in H0 there is no strong evidence against H0, whereas a value specified in H0 that lies well outside that confidence interval indicates strong evidence against H0. If a P-value notation is preferred, setting P = P0 as the critical value for determining significance, one may use the notation P0 in place of α. Example 1.5 We saw in Section 1.3.2, Example 1.2 that for a sign test for a median using a sample of 10 and a two-tail test at the 2.16 per cent level we accept H0 if we get between 2 and 8 plus signs.
©2001 CRC Press LLC
Consider the data in Example 1.2, i.e. 49, 58, 75, 110, 112, 132, 151, 276, 281, 362* where the asterisk represents a censored observation. Clearly we have between 2 and 8 plus signs if the median specified in H0 has any value greater than 58 but less than 281, so the interval (58, 281) is a 100(1 – 0.0216) = 97.84 per cent confidence interval for θ, the population median survival time. Since we would retain an H0 that specified any value for the median greater than 58 but less than 281 there is considerable doubt about the population median value. It is almost an understatement to say our estimate lacks precision.
Another common interpretation of a 100(1 – α ) per cent confidence interval is in terms of the property that if we form such intervals for repeated samples, then in the long run 100(1 – α) per cent of these intervals would contain (or cover) the true but unknown θ. Confidence intervals are useful because: • They tell us something about the precision with which we estimate a parameter. • They help us decide (a) whether a significant result is likely to be of practical importance or (b) whether we need more data before we decide if it is. We elaborate on these points in Section 1.4.2. A useful way of looking at the relationship between hypothesis testing and estimation is to regard testing as answering the question: •
Given a hypothesis H0: θ = θ0 about, say, a parameter θ, what is the probability (P-value) of getting a sample as or less likely than that obtained if θ0 is indeed the true value of θ?
whereas estimation using a confidence interval answers the question: •
Given a sample, what values of θ are consistent with the sample data in the sense that they lie in the confidence interval?
1.4.2 Precision and significance and practical importance Example 1.6 Doctors treating hypertension are often interested in the decrease in systolic blood pressure after administering a drug. When testing an expensive new drug they might want to know whether it reduces systolic blood pressure by at least 20 mm Hg. Such a minimum difference could be of practical importance. Two clinical trials (I and II) were carried out to test the efficacy of a new drug (A) for reducing blood pressure. A third trial (III) was carried out with a second new drug (B). Trial I involved only a small number of patients, but trials II and III ©2001 CRC Press LLC
involved larger numbers. The 95 per cent confidence intervals for mean blood pressure reduction (mm Hg) after treatment at each trial were: Drug A Drug A Drug B
Trial I Trial II Trial III
(3, 35) (9, 12) (21, 25)
In each trial a hypothesis H0: drug does not reduce blood pressure would be rejected at a 5 per cent significance level (because the confidence intervals do not include zero), implying strong evidence against H0. Trial I is imprecise; we would accept in a significance test at the 5 per cent level any mean reduction between 3 and 35 units. The former is not of practical importance to a doctor; the latter is. This small trial only answers questions about the ‘significant’ mean reduction with low precision. The larger Trial II, using the same drug, indicates an average reduction between 9 and 12 units, a result of statistical significance but not of practical importance in this context. Compared to Trial I, it has high precision. Other relevant factors being unchanged, increasing the size of a trial increases the precision, this being reflected in shorter confidence intervals. Trial III using drug B also has high precision and tells us the mean reduction is likely to be between 21 and 25 units, a difference of practical importance. Drug B appears to be superior to Drug A.
For a given test, increasing sample size increases the probability that small departures from H0 may provide strong evidence against H0. The art of designing experiments is to take enough observations to ensure a good chance of detecting with reasonable precision departures from H 0 of practical importance, but to avoid wasting resources by taking so many observations that trivial departures from H0 provide strong evidence against it. An introduction to sample size calculation is given by Kraemer and Thiemann (1987) and it is also discussed with examples by Hollander and Wolfe (1999). Practical design of experiments is best done with guidance from a trained statistician although many statistical software packages include programs giving recommendations in specific circumstances. In later chapters we show for some tests how to find sample sizes needed to meet specified aims. Our discussion of hypothesis testing and estimation has used the frequentialist approach to inference. The Bayesian school adopt a different philosophy, introducing subjective probabilities to reflect prior beliefs about parameters. Some statisticians are firm adherents of one or other of these schools, but a widely accepted view is that each has strengths and weaknesses and that one or the other may be preferred in certain contexts. However, for the procedures we describe sensible use of either approach will usually lead to similar conclusions despite the different logical foundations, so for consistency we use the frequentialist approach throughout.
©2001 CRC Press LLC
1.5 ETHICAL ISSUES Ethical considerations are important both in general working practices (Gillon, 1986) and in the planning and conduct of investigations (Hutton, 1995). The main principles are respect for autonomy, nonmaleficence, beneficence and justice. Many research proposals need to be assessed by ethical committees before being approved. This applies particularly in the field of medicine but increasing attention is being given to ethical issues in environmentally sensitive fields like biotechnology and also in the social sciences where questions of legal rights or civil liberties may arise. The role of statistics and statisticians in what are known as research ethics committees has recently been discussed by Williamson et al. (2000). The related issue of the development of guidelines for the design, execution and reporting of clinical trials is described by Day and Talbot (2000). It would be unacceptable to study some issues by allocating individuals to one of the possible groups at random as we did in Example 1.4. For instance, in a study of the effects of smoking on health, one could not instruct individuals to smoke or to abstain from smoking as this disregards the autonomy of the study participants. In such a situation, the individual’s choice of whether to smoke cigarettes or not must be respected, and an alternative type of study which makes this possible must be chosen. Many ethical issues are less clear and in practice until recently some have often been ignored. At the early stages of development, a new treatment for an illness may be associated with harmful sideeffects and may clash with the principle of nonmaleficence. When the benefits of a new treatment are fairly clear, many doctors reason from the principle of beneficence that it is wrong to deny patients care by giving them the previously standard treatment. If early results from a trial indicate that one treatment is clearly superior, it is unethical to allow the trial to continue. All new studies should be based on information gleaned from a comprehensive search of findings from related work by using, for instance, MEDLINE, a continually updated source of information on articles from medical and biological journals. It is unethical to conduct research that ignores previous work that may be relevant because it is then likely that any or all of time, money and scarce resources will not be used to best effect. The results of literature searches nevertheless need to be interpreted with caution. Studies with interesting findings are more likely to appear in print, leading to publication bias (see Easterbrook et al., 1991). ©2001 CRC Press LLC
When there is little relevant prior knowledge it may be prudent to conduct an initial pilot study to highlight potential problems that might arise. Results from a pilot study can also be useful in choosing an appropriate number of participants for the main study. A sufficiently large number should be involved even at the pilot stage to have a reasonable chance of finding the expected difference between the two groups if it really exists. The intended method of statistical analysis also influences the sample size requirement. Small studies often fail to yield useful findings and are thus a poor use of resources. On the other hand, resources can be wasted by recruiting more participants than needed. In medical research, in either situation more patients than necessary are at risk of receiving an inferior treatment. Careful planning should consider the composition of the sample with respect to age, sex, ethnic group, etc. as this will enable problems under investigation to be answered more effectively. In medical investigations each potential participant should receive a written information sheet that outlines the main points about the study. Ideally, complete information about the possible efficacy and side-effects of the treatments involved in the study should be given to the patient. In practice, not all patients will understand or even wish to receive details beyond those given in the information sheet, particularly in a sophisticated trial. In this situation, the patient should be given a choice about what information is supplied. Once a trial has been completed, patients who feel that they have received an effective treatment for their health problem may wish to continue with it. Financial constraints and/or the concerns of the patient’s general practitioner may prevent long-term use of the treatment; this should be discussed in advance as part of the patient information. The autonomy of the patient should be respected and the patient should only make a decision on whether or not to enter the trial following careful consideration of the information provided. This is particularly important with tests for inherited diseases that only become evident in later life. A positive finding may distress the patient, have serious implications for any children and prejudice life assurance proposals. Patients should give informed written consent to the investigator(s) prior to being entered into a trial and they should be allowed to withdraw from the trial at any time. It has been suggested, for instance, that in a comparison of two treatments patients who do not wish to be entered into a randomized controlled trial should be allowed instead to choose which of the treatments they wish to receive. Doing this would produce three
©2001 CRC Press LLC
groups: the patients who request Treatment A, the patients who request Treatment B and those who are randomized. This must be taken into account in any statistical analysis. Data collected in studies should be kept confidential. In the United Kingdom, for example, computer records should adhere to the principles laid down in the Data Protection Act. Data used for statistical purposes should not contain patients’ names or addresses. Suppose that in the situation described in Example 1.4, there were high hopes that the new drug might greatly relieve suffering in severe cases but only enough doses were available to treat four patients. The principles of beneficence and justice may suggest that the four patients to receive the drug should be those with the most severe symptoms. In a situation like this, the drug may reduce suffering, but such patients may still, after treatment, be ranked 6, 7, 8, 9 because, although their condition may have improved, their symptoms may still be more severe than those of patients not given the drug. The consequent statistical difficulties might be overcome by basing ranks not on relative condition after treatment but on ‘degree of improvement’ shown by each patient. At the other extreme, an experimenter might allocate the new drug to the patients with the least severe symptoms. From a research point of view this is misleading as even if it were ineffective or no better than an existing treatment, these patients may still be ranked 1, 2, 3, 4 after treatment. However, if it is likely that only patients in the early stages of the disease will benefit it will be more appropriate from an ethical viewpoint to give the new drug to these patients. Even when patients are allocated to treatments at random and we find strong evidence to suggest we should abandon a hypothesis of ‘no treatment effect’, the statistically significant outcome may be of no practical importance, or there may be ethical reasons for ignoring it. A doctor would be unlikely to feel justified in prescribing the new treatment if it merely prolonged by three days the life expectation of terminally-ill patients suffering considerable distress, but may from the principle of beneficence feel bound to prescribe it if it substantially improved survival prospects and quality of life. An example of unethical statistical behaviour is that where a statistician performs a number of competing tests – parametric or nonparametric – each producing a different P-value, but only publishes the P-value that is most favourable to the conclusion he or she wants to establish, regardless of whether it is obtained by using an appropriate test. This is unethical suppression of evidence. Ethical considerations are relevant no matter whether parametric ©2001 CRC Press LLC
or nonparametric methods of data analysis are used and they may influence not only how an experiment is carried out (the experimental design) but also what inferences are possible and how these should be made. 1.6 COMPUTERS AND NONPARAMETRIC METHODS Software that rapidly computes exact P-values for permutation tests for small to medium-sized samples and that provides accurate estimates of tail probabilities by simulation for larger samples has revolutionized application of nonparametric methods. The software and hardware situation is changing rapidly. At the time of writing StatXact 4.0 distributed by Cytel Software Corporation, Cambridge, MA gives exact P-values or Monte Carlo estimates of these for a range of tests. Large sample or asymptotic results are also given and there are facilities for computing confidence intervals and also the power of some of the tests for assigned sample sizes and specified alternative hypotheses. Some of the exact tests in StatXact are now available as options in the general statistical packages SAS and SPSS. Testimate, distributed by IDV Daten-analyse und Versuchsplanung, Munich, Germany has considerable overlap with StatXact but some methods are included in one but not both these packages. There are specialized programs dealing with particular aspects of the broad fields of nonparametric and semiparametric inference. These include LogXact and EGRET which are especially relevant to logistic regression, a topic not covered in this book. Popular general statistical packages such as Stata, Minitab and the standard versions of SPSS, SAS, etc., include some nonparametric procedures. In a few of these exact tests are given but many rely heavily on asymptotic results sometimes with insufficient warning about when, particularly with small or unbalanced sample sizes, these may be misleading. S-PLUS is particularly useful for the bootstrap described in Chapter 11. Monte Carlo approximations to exact P-values or for bootstrap estimation can often be obtained from standard packages by creating macros that make use of inbuilt facilities for generating many random samples with or without replacement. Developments in statistical computer software are rapid and much of what we say about this may be out of date by the time you read it. Readers should check advertisements for statistical software in
©2001 CRC Press LLC
relevant journals and look for reviews of software in publications such as The American Statistician to trace new products. The efficiency of StatXact programs stems from the use of algorithms based on the work of Mehta and his co-authors in a series of papers including Mehta and Patel (1983; 1986), Mehta, Patel and Tsiatis (1984), Mehta, Patel and Gray (1985), Mehta, Patel and Senchaudhuri (1988, 1998). Similar and other efficient algorithms are used in Testimate, but understanding the algorithms is not needed to use these packages. Users should test all programs using examples from this book and other sources to ensure that they can interpret the output. In some cases the output will inevitably be different, being either more or less extensive than that given in the source of the examples. For instance, output may give nominal (usually 5 or 1 per cent) significance levels rather than exact P-values. Appropriate software is virtually essential for implementation of all but the simplest methods. This book is mainly about longestablished methods, but only modern computing facilities allow us to use them in the way we describe
1.7 FURTHER READING Hollander and Wolfe (1999), Conover (1999) and Gibbons and Chakraborti (1992) give more background for many of the procedures described here. Each book covers a slightly different range of topics, but all are recommended for those who want to make a more detailed study of nonparametrics. Daniel (1990) is a general book on applied nonparametric methods. A moderately advanced mathematical treatment of the theory behind basic nonparametric methods is given by Hettmansperger and McKean (1998). Randles and Wolfe (1979) and Maritz (1995) are other recommended books covering the theory at a more advanced mathematical level than that used here. Two classics are the pioneering Bradley (1968) and Lehmann (1975). The latter repays careful reading for those who want to pursue the logic of the subject in more depth without too much mathematical detail. Books on applications in the social sciences include Marascuilo and McSweeney (1977), Leach (1979) and more recently Siegel and Castellan (1988), an update of a book written by Siegel some 30 years earlier. Noether (1991) uses a nonparametric approach to introduce basic general statistical concepts. Although dealing ©2001 CRC Press LLC
basically with rank correlation methods, Kendall and Gibbons (1990) give an insight into the relationship between many nonparametric methods. Agresti (1984; 1990; 1996) and Everitt (1992) give detailed accounts of various models, parametric and nonparametric, used in categorical data analysis. A sophisticated treatment of randomization tests with emphasis on biological applications is given by Manly (1997). Good (1994) and Edgington (1995) cover randomization and permutation tests. Sprent (1998) extends the treatment of some of the topics in this book and shows how they interconnect with other aspects of statistical analysis. Books dealing with the bootstrap are Efron and Tibshirani (1993), Davison and Hinkley (1997) and Chernick (1999). Many nonparametric procedures use ranks, and the theory behind rank tests is given by Hájek, Sidak and Sen (1999). There are a number of advanced texts and reports of conference proceedings that are mainly for specialists. EXERCISES 1.1
1.2
1.3
As in Example 1.1, suppose that one machine produces rods with diameters normally distributed with mean 27 mm and standard deviation 1.53 mm, so that 2.5 per cent of the rods have diameter 30 mm or more. A second machine is known to produce rods with diameters normally distributed with mean 25 mm and 2.5 per cent of rods it produces have diameter 30 mm or more. What is the standard deviation of rods produced by the second machine? A library has on its shelves 114 books on statistics. I take a random sample of 12 and want to test the hypothesis that the median number of pages, θ, in all 114 books is 225. In the sample of 12, I note that 3 have less than 225 pages. Does this justify retention of the hypothesis that θ = 225? What should I take as the appropriate alternative hypothesis? What is the largest critical region for a test with P ≤ 0.05 and what is the corresponding exact P-level? The numbers of pages in the sample of 12 books in Exercise 1.2 were: 126 142 156 228 245 246 370 419 433 454 478 503
1.4 1.5
Find a confidence interval at a level not less than 95 per cent for the median θ. Use the sum of ranks given in brackets after each group in Table 1.1 to verify the correctness of the entries in Table 1.2. Suppose that the new drug under test in Example 1.4 has all the ingredients of a standard drug at present in use and an additional ingredient which has proved to be of use for a related disease, so that it is reasonable to assume that the new drug will do at least as well as the standard one, but may do better. Formulate the hypotheses leading to an appropriate one-tail test. If
©2001 CRC Press LLC
the post-treatment ranking of the patients receiving the new drug is 1, 2, 4, 6 assess the strength of the evidence against the relevant H0. 1.6 An archaeologist numbers some articles 1 to 11 in the order he discovers them. He selects at random a sample of 3 of them. What is the probability that the sum of the numbers on the items he selects is less than or equal to 8? (You do not need to list all combinations of 3 items from 11 to answer this question.) If the archaeologist believed that items belonging to the more recent of two civilizations were more likely to be found earlier in his dig and of his 11 items 3 are identified as belonging to that more recent civilization (but the remaining 8 come from an earlier civilization) does a rank sum of 8 for the 3 matching the more recent civilization provide reasonable support for his theory? 1.7 In Section 1.4.1 we associated a confidence interval with a two-tail test. As well as such two-sided confidence intervals, one may define a onesided confidence interval composed of all parameter values that would not be rejected in a one-tail test. Follow through such an argument to obtain a confidence interval at level not less than 95 per cent based on the sign test criteria for the 12 book sample values given in Exercise 1.3 relevant to a test of H0: θ = θ0 against a one-sided alternative H1: θ > θ0. 1.8 We wish to compare a new treatment with a standard treatment and only 6 patients are available. We allocate 3 to each treatment at random and after an appropriate interval rank the patients in order of their condition. What is the situation (i) for testing H0: treatments do not differ against H1: the new treatment is better and (ii) for testing the same H0 against H 1 : the two treatments differ in effect? 1.9 In Section 1.1.1 we stated that if all observations in a not-too-small sample are zero or positive and the standard deviation is appreciably greater than the mean, then the sample is almost certainly not one from a normal distribution. Explain why this is so. (Hint: consider the probability that a normally distributed variable takes a value that differs from the mean by more than one standard deviation.) 1.10 In Example 1.3 we remarked that a situation could arise where we might reject a null hypothesis for the wrong reason. Explain how this is possible in that example. 1.11 State the appropriate null hypothesis for the example from the book of Daniel about diet (p.7). How could you use ranks to calculate the probability that those receiving the diet of pulses were ranked 1, 2, 3, 4? Obtain this probability assuming that there were 20 young men involved altogether. 1.12 In Section 1.3.1 we pointed out that 5 tosses of a coin would never provide evidence against the hypothesis that a coin was fair (equally likely to fall heads or tails) at a conventional 5 per cent significance level. What is the least number of tosses needed to provide such evidence using a two-tail test, and what is the exact P-value? If in fact a coin that is tossed this least number of times is such that Pr(heads) = 2/3 what is the probability of an error of the second kind? What is the power of the test?
©2001 CRC Press LLC
2 Centrality inference for single samples
2.1 USING MEASUREMENT DATA In this book we illustrate the logic of many inference methods using examples where, with a few exceptions, we discuss various aspects under the headings: The problem Formulation and assumptions Procedure Conclusion Comments Computational aspects The summary and exercises at the ends of chapters are preceded by indicative but not exhaustive lists of fields of application. This chapter covers inferences about centrality measures for a single sample. These provide simple examples for both parametric and nonparametric methods, although in most real problems we compare two or more samples. The methods then used are often easily described as extensions of or developments from those for single samples. 2.1.1 Some general considerations Choosing the right inference procedure depends both on the type of data and the information we want. In this chapter the initial data are usually measurements but some of the methods are directly applicable to other kinds of data such as ranks, or various types of counts. In Example 1.2 we wanted to make inferences about average survival time, but there was the complication that the precise survival time was not known for one patient. The sign test let us use the partial information we had for that patient but the test used only a small part of the total information available; namely, whether or not each observation was above or below 200. We could have ©2001 CRC Press LLC
carried out the test if we had been given only this partial information. Even when data sets are measurements occasionally we do as well or better with nonparametric methods that do not use all the available information. We develop several nonparametric tests in this chapter and compare their performance using simple data sets. In Sections 2.1 — 2.4 we show how to compute and interpret some exact P-values and confidence intervals. In practice even the best currently available software gives exact P-values only for small to medium-sized samples. We present in Section 2.5 asymptotic results that often provide good approximations for larger samples. If programs for exact tests are not available a sensible compromise is to use Monte Carlo sampling to get good estimates of exact Pvalues. StatXact provides this facility, but if this package is not available one may still create the relevant samples for Monte Carlo approximation using any software that allows one to generate rapidly many random samples. 2.1.2
A raw-data randomization test for centrality
Pitman (1937a) developed ideas put forward by Fisher (1935) for a randomization test for hypotheses about the mean or median of a symmetric distribution when we have a random sample of observations from a population that has that distribution. For a symmetric distribution if the mean and median both exist they coincide at the point of symmetry. A few symmetric distributions have no mean, but this complication is seldom relevant in real data problems, so for the symmetric case we tend to use the terms population mean or median almost as though they are interchangeable. For asymmetric (skew) distributions the mean and median are no longer equal. Pitman s prime aim was to show that his test, which only assumed symmetry and did not require the distribution to belong to a specific family such as the normal family would, at least for samples that were not very small, lead to similar inferences to those based on a t-test which requires a normality assumption for strict validity. In effect he verified an earlier assertion by Fisher that using the t-test or other normal theory tests gave a good approximation to an exact randomization test even when the normality assumption was no longer valid. This justified continued use of the then easier-to-compute t-test based inference. Without modern computers the calculations for the Pitman test were prohibitive except for trivially small samples. The test is seldom used in practice (see Comment 5 on Example 2.3 for some reasons why) but we describe it in some detail partly because of its historical ©2001 CRC Press LLC
interest but also because it highlights several principles that are taken into account in more useful nonparametric methods. We call the test a Pitman test, but the names raw data test and Fisher–Pitman test are also used. Modern computer packages such as StatXact or Testimate make computation easy. However, illustrating the test with a simple numerical example helps us understand its main features and comparing it with other tests highlights some weaknesses. The basic idea behind the test is that if a random sample is from a symmetric distribution with unknown mean or median θ then symmetry implies it is equally likely that any sample value will differ from θ by some positive amount d, say, or by the same negative amount, –d, for all values of d. The test is valid even if the observations x1, x2, . . . , xn are not each from the same distribution providing they are independent of one another and all have symmetric distributions with the same value for θ. This situation might arise, for example, if n observatories each independently measure the diameter of the same star. Because each observatory uses different equipment, some of it more sophisticated than other equipment, it is likely that the precision with which observations are made will differ between observatories. If one assumes that these measurements are all ‘centred’ symmetrically about the true diameter, θ, but differ only in the (usually unknown) spread about θ and makes no further assumptions, the Pitman test is still valid. Suppose n observations x 1, x 2, . . . , x n satisfy the conditions above and we want to test the hypothesis that the median θ has a fixed value θ 0, i.e. H 0: θ = θ0 against the twosided alternative H1: θ ≠ θ0; the test proceeds on the basis that under H0 the sign of each of the n differences d i = x i – θ 0, i = 1, 2, . . . , n is equally likely to be positive or negative. Clearly, when H 0 does not hold there is more likely to be a preponderance either of positive or of negative signs associated with the d i. The situation has similarities to the sign test scenario in Example 1.2, but there we considered only the numbers of positive or negative signs and not the magnitudes of the associated d i and this required no assumption of symmetry for its validity. ©2001 CRC Press LLC
Example 2.1 The problem. The heart rate (beats per minute) when standing was recorded for seven members of a tutorial group. We assume that in the population of such students the distribution of heart rates when standing is symmetric. Given observations of 73, 82, 87, 68, 106, 60 and 97 test the hypothesis H0: θ = 70 against the alternative H1: θ > 70 using the Pitman test. Formulation and assumptions. Symmetry implies that under H0 sample values should have a near-symmetric scatter about θ = 70, so the magnitudes of the sums of positive and of negative deviations from 70 (the di) should not differ greatly. The one-sided alternative H1 implies that when it holds there are likely to be more positive deviations from 70 than there would be under H0 and that these positive deviations will tend to be larger than the negative deviations. Procedure. For the given data deviations from 70 are 73 – 70 = 3, 82 – 70 = 12 and similarly those remaining are 17, –2, 36, –10, 27. Since, under H0 the sum of the posit ive and negat ive devia tions shoul d be nearl y equal where as under H1 we expect the sum of the positive deviations to be appreciably larger than the sum of the negative deviations, it is clear that the sum of the positive deviations (i.e. differences from 70) is an intuitively reasonable statistic to help assess the strength of the evidence against H0 in support of the specified onesided alternative H1. If the alternative had been the two-sided H1: θ ≠ 70 small as well as large sums of positive deviations would also have supported H1. For a randomization test based on S+, the sum of the positive deviations, we form the appropriate randomization distribution in a manner similar to that in Example 1.4 for the statistic S used there. The key to obtaining the randomization distribution of S+ when H0 holds lies in the fact that under H0 the sign of each deviation di is equally likely, on account of the symmetry assumption, to be positive or negative. Thus any combination of plus and minus signs attached to a set of deviations having the observed magnitudes is equally likely. Clearly in the current example with 7 observations there are 27 = 128 possible allocations of sign (2 to each of the 7 differences) each allocation being equally likely under H0. Recording and ordering all of them without a computer is tedious. We need not do so if we only want to obtain the relevant P-value when that is small because we then only need to know how many of the equally likely allocations of signs give an S+ greater than or equal to that for our sample, i.e. S+ = 3 + 12 + 17 + 36 + 27 = 95. Obviously the greatest possible value of S+ occurs when all the deviations have positive signs and then S+ = 3 + 12 + 17 +2 + 36 + 10 +27 = 107. Clearly the next highest sum (105) is attained when only the smallest deviation, 2, has a negative sign. In order of decreasing magnitude it is easily verified that other sums greater than the observed S+ = 95 occur only when negative signs are attached to 3 only (sum 104), 2 and 3 (sum 102) and 10 only (sum 97). If a negative sign is attached to 12 only or to both 10 and 2 (as above) the sum is 95. Thus there are seven sums greater than or equal to the observed S+ = 95. Since all are equal ly likel y the relev ant P-valu e for asses sing the stren gth of evidence against H0 is P = 7/128 ≈ 0.055. Conclusion. The evidence against H0 when P = 0.055 is sufficiently strong to perhaps warrant a similar study with a larger group of students.
©2001 CRC Press LLC
Comments. 1. These data show that, despite our warning in Section 1.3.1 that small exper iments may never produ ce P-valu es indic ating stron g evide nce again st H0, an interesting effect may sometimes be suggested even with very small samples; this is encouraging given the current emphasis on small group work in teaching, where data concerning individuals in the group are often used for illustrative purposes. A teaching advantage of using such data from small groups is that in working through the arithmetic one is not bogged down as would be the case with larger data sets. However, this example draws attention to a major limitation of a conventional hypothesis test. While it tells us that there is some doubt about the wisdom of retaining H0 it tells us nothing about what other hypotheses might be plausible. We need a confidence interval to give this information. Unfortunately, while modern computer programs enable us to perform a Pitman hypothesis test with relative ease it is harder to obtain a confidence interval for the true θ, a matter discussed in more detail in Example 2.3. 2. Modifications for a two-tail test when that is appropriate are straightforward and were mentioned under Procedure. 3. We chose S+ as our test statistic but we could equally well have chosen S–, the sum of the negative deviations. This is because the sum of the magnitudes of the deviations remains constant for all possible allocation of signs (in this example always having the value 107). Thus the statistic has a symmetric distribution because for any of the 128 sign permutations we always have S+ + S– = 107 or S– = 107 – S+. Computer software sometimes calculates the smaller of S+, S– but each leads to the same relevant P-value. Another statistic that is often used is Sd = |S+ – S–|. For this example Sd = |2S+ – 107| so there is a one-to-one ordered relationship between Sd and S+. One may even use the one sample t-test statistic instead of S+ or S– because it is not difficult to show (Exercise 2.2) that again there is a one-to-one ordered relationship between the values of t and S+. Remember though that the distribution of this statistic t is not that given by the usual t-tables that assume normality, but is identical with that for S+. 4. The approach has a strong intellectual appeal because it uses all the information in the data but it lacks robustness against departures from symmetry. This is explained in Section 2.6. 5. Because the possible values of S+ depend upon values of di which in turn depend upon the data values it follows that the S+ corresponding to a given Pvalue varies between data sets even for samples of the same fixed size. The test is called a conditional test because it is conditional upon the data values. A practical implication of this is that the P-value associated with H0 must be computed afresh for each data set. 6. If we had followed the classic convention requiring P ≤ 0.05 before asserting significance we would not have rejected H0 in a formal significance test at the 5 per cent level. We see no merit in following such an arbitrary rule in this example, preferring to argue that the case for retaining H0 is not strongly supported even though 0.055 slightly exceeds 0.050. Remember also our comment in Section 1.3.1 that a not very small P-value may sometimes only indicate that our experiment is too small to yield valuable information. Computational aspects. 1. Using StatXact the P-value obtained above and the complete distribution of S+ can be calculated for this example in less than one ©2001 CRC Press LLC
second. That program also gives an asymptotic approximation that is strictly valid only for large samples. Such approximations are discussed in Section 2.5. For these data the asymptotic P-value for the one-tail test is P = 0.051, which, despite the small sample size, is close to the exact P-value, but such close agreement is not always obtained for small samples. In passing we note that the normal theory t-test for this example also gives P = 0.051. We comment further on the relation between the Pitman test and the t-test in Example 2.2. 2. To estimate P by Monte Carlo sampling one would randomly allocate, each with probability p = 0.5, a plus or a minus sign to every di for, say, 10 000 samples and record the number, r, of these samples giving values of S+ greater than or equal to 95. Then P* = r/10 000 is an estimate of the exact one-tail P.
Students of literary style often want to compare characteristics of the work of a particular author with those of other authors. The distribution of the number of words per sentence (sentence length) is one facet that is often studied. The mean or the median sentence length and the spread in sentence length, for example, varies between authors, especially in scientific or technical writing where average sentence length may even differ between several works by the same author, reflecting differing complexity of subject matter. When writing about some subjects there may even be varying patterns of sentence length in different positions in paragraphs or in different chapters. For example, the first sentence in any paragraph may tend to be longer than subsequent ones. This might happen in technical writing if each paragraph introduces a new basic idea and the first sentence may have to be long to expound that idea, while later sentences are subsidiary in nature to point out exceptions, limitations, fields of usage or other relevant matters. Both hypothesis testing and estimation play a part in studies of this kind. For example, suppose that an expert on writing skills asserts that for clarity in technical writing the average length of sentences should not exceed 30 words. A sample of sentences could be taken from the work of some author. Looking at the median sentence length, θ, the strength of evidence against the hypothesis H 0: θ ≤ 30 would be of interest; if the evidence is strong, the alternative H 1: θ > 30 seems more acceptable. If no prior assertion about ideal sentence length had been made one might be more interested in the median or mean sentence length of the work of a particular author and this leads naturally to an estimation type problem where finding a confidence interval is appropriate. Examples 2.2 and 2.3 bring out some difficulties that arise if the Pitman test is used when a symmetry assumption may not be reasonable. The consequences are discussed under the Comments headings.
©2001 CRC Press LLC
Example 2.2 The problem. The number of words in the first sentences in 12 randomly selected paragraphs from Fisher (1948) were: 12 18 24 26 37 40 42 47 49 49 78 108 These are arranged in ascending order but the analysis is not affected by this reordering of the original data. Test the hypothesis H0: θ ≤ 30 against H1: θ > 30 where θ is the population mean or median using the Pitman test. Formulation and assumptions. We assume that sentence lengths are symmetrically distributed (but see Comment 1 below). Procedure. We apply the Pitman randomization test to the differences formed by subtracting 30 from each datum, giving –18 –12
–6 –4 7 10 12 17 19 19 48 78
Clearly the sum of the negative differences is smaller than the sum of the positive differences so we may use as our test statistic S– = 18 + 12 +6 + 4 = 40. If θ = 30 each of the 12 differences is equally likely to be positive or negative and there are now 212 = 4096 possible allocations of signs to the differences and it is a formidable task to find how many of these lead to an S– ≤ 40. Software such as that in StatXact comes to our rescue and using that program for a one-tail test gives P = 0.0405. Conclusion. The P-value indicates fairly strong evidence that the median exceeds 30. Comments. 1. We remarked that Pitman (1937a) showed that results for his test are usually close to those given by a t-test. A one-tail t-test in this example gives an exact P = 0.0462, not markedly different from that given by the Pitman test. However it is doubtful whether the distribution of sentence length is symmetric, and certainly doubt about whether it is normally distributed. There are good grounds for these doubts. Firstly, for a symmetric distribution the means and medians coincide and thus in a sample one expects them to be close. For these data the median is 41 and the mean 44.17 so there is a recognizable difference between them. Inspecting the sample values shows that this difference is largely due to the influence on the mean of the value 108 which is 67 more than the median, whereas the minimum observed length is only 29 less than the median. Secondly, when one considers the nature of these data population skewness is not unreasonable. The shortest possible sentence must contain at least one word, but there is no fixed upper limit for the number of words in a sentence. In practice most writers sometimes use a sentence of 100 or more words even if their average sentence length is between 25 and 40, a common range for the median for many writers. A formal test for normality developed in Section 3.3.3 provides strong evidence against these data being a sample from a normal distribution. 2. A breakdown of the symmetry assumption affects the performance of both the Pitman test and the t-test in an undesirable way, and although we do not prove it, the general effect of asymmetry is to reduce the power of both tests. 3. Number of words is only a crude measure of sentence length, especially when comparing works by different authors because some habitually use longer ©2001 CRC Press LLC
words than others. Alternative measures are the total number of letters in a sentence or, if all sentences are set in exactly the same type-face, the length in centimetres between the start of the first word and the end of the last word. This last method has the disadvantage that for typographic reasons letters and words are usually not evenly spaced throughout a sentence. It is likely that any of the three methods for measuring sentence length mentioned here would lead to broadly similar inferences providing one could agree on the appropriate equivalent null hypothesis value of θ for each measure – that in itself might call for some sample-based experiments! For example, is a 30-word sentence broadly equivalent to a 180-letter sentence? Computational aspects. StatXact gives exact one- or two-tail permutation test P-values for small to moderate sample sizes and also an asymptotic result which is reliable only for larger samples. The program also provides a Monte Carlo approximation for P if required. A few general statistical programs also provide an asymptotic approximation to a Pitman test P-value.
Confidence intervals for θ based on the Pitman test are seldom used for reasons that will soon become apparent so we omit the theory for computing them. It is given by Sprent (1998, Section 5.2) and by Maritz (1995, Section 2.4). We know of no readily available software package that gives confidence intervals directly in this case, but a trial-and-error method based on a hypothesis testing program may be used. Example 2.3 The problem. Given the data in Example 2.2 determine a confidence interval with at least 95 per cent coverage for the mean or median θ based on the Pitman procedure. Remember that we are, with little justification, assuming symmetry. Formulation and assumptions. Assuming symmetry we may obtain the limits that defin e the inter val by deter mining value s θ1 and θ2 of θ where θ1 < θ2 such that we would retain the hypothesis that this were the true parameter value if θ = θ1 but not for any θ < θ1 or if θ = θ2 but not for any θ > θ2, in each case with a relevant one-tail probability as close as possible to, but not exceeding 0.025. The interval is then based on a two-tail test with P not exceeding 0.05. We consider the modification for a one-sided interval in Comment 2. Procedure. A trial and error method is used to establish the required limits starting with suitable initial values. Because the Pitman test usually gives results not very different from a t-test applied (however inappropriately) to the same data it is reasonable to start with 95 per cent confidence limits based on the t-test procedure. These are 27.2 and 61.1 and are obtainable from most statistical software packages. Our trial and error procedure begins by using a Pitman test program to obtain one-tail P-values for testing the hypotheses θ = 27.2 and θ = 61.1. The data editor in most relevant statistical programs will generate the relevant deviations directly by subtraction of the appropriate constants 27.2 and 61.1 from the data. The relevant P-values found are 0.016 in the lower tail and 0.026 in the upper tail. To adjust each of these probabilities to be as close as
©2001 CRC Press LLC
possible to 0.025 without exceeding that value clearly we should try a higher value for θ for the lower limit (to incre ase P) and also a highe r value for the upper limit (to decrease P). We first chose 28 and 61.5 giving respectively P = 0.021 and P = 0.025. Our next choice was 28.4 and 61.6 giving P = 0.024 and P = 0.024. We then took 28.5 as a revised lower value giving P = 0.025. Both lower and upper limits are close to the target P = 0.025 but for further refinement we took 28.6 giving P = 0.026. It is now clear that in two-tail tests at the 5 per cent level we would favour the hypotheses θ = 28.6 or perhaps θ = 61.5, but we would not retain the hypotheses if θ = 28.5 or θ = 61.6. As one further piece of fine tuning, really unnecessary in practice, we found that the P-values associated with θ = 28.59 and θ = 61.51 were respectively P = 0.025 and P = 0.024. Concl usion. A Pitma n test based inter val (28.6 , 61.5) has confi dence level [1 – (0.025 + 0.024)]100 = 95.1 per cent. This follows because if we take values of θ just outside these limits, i.e. 28.59 or 61.51 we would reject these hypothetical values while we would accept the values 28.6 and 61.5 in conventional tests if we used an exact 4.9 per cent significance level. Comments. 1. Although discontinuities in possible P-values stop us forming an exact 95 per cent interval our coverage of 95.1 per cent is close to 95 per cent and the interval (28.6, 61.5) is not very different from the t-test 95 per cent interval (27.2, 61.1). The Pitman interval is slightly shorter and shifted a little to the right, but these differences are of no practical importance. Another effect of discontinuities is that the exact P-values may not be equal in each tail. 2. Both the t-test and the Pitman test intervals indicate that with a two-tail test there is no strong evidence against the hypothesis θ ≤ 30. Indeed the rule for doubling the one-tail probability for getting an appropriate two-tail probability leads to the same concl usion if we had modif ied our H1 appro priately in Example 2.2. Confidence intervals are usually applied in a two-tail context but we can form intervals relevant to a one-tail test. Corresponding to the test in Example 2.2, for such a 95 per cent interval we find a lower cut-off point that gives a critical region of size P = 0.05 in the lower-tail. The upper limit becomes infinity and a trial and error process similar to that used for the two-tail situation gives a one tail P = 0.050 corresponding to θ = 30.74 and P = 0.051 corresponding to θ = 30.75. Thus a 95 per cent one-sided confidence interval is (30.75, ∞). In other words, hypothetical values of θ ≥ 30.75 appear more plausible than values of θ < 30.75, consistent with our finding in Example 2.2 that a one-tail test gave fairly strong evidence against H0: θ = 30. The upper limit ∞ is in practice somewhat meaningless, being a mathematically symbolic way of saying that the true median is more likely to be some (unspecified) value above 30.75 rather than one at or below that value. One only need glance at the sample values to see that the median is extremely unlikely to be 200! 3. In this example it is sensible and suffices to give limits only to one dec-imal place. Also, when quoting P-values, one or two significant figures suffice, although some programs may produce values like P = 0.025631 in some examples. It is sensible to report P-values below 0.001 simply as P < 0.001. 4. We show in Example 2.11 that using a method not assuming a symmetric distribution gives a shorter confidence interval for the median. Indeed, in that example we obtain a 96.2 per cent interval (24, 49) for these data. Although the lower limit is only slightly below that for the t-test or Pitman test intervals the ©2001 CRC Press LLC
upper limit is markedly lower. We discuss why this is so in Example 2.11 remarking here only that breakdown of the symmetry assumption may have a considerable effect on confidence intervals obtained when assuming symmetry. 5. Inferences based on the t-test and the Pitman test tend to be similar and both are unsatisfactory when there is asymmetry. The Pitman test procedure for computing confidence intervals is cumbersome.
2.2 INFERENCES ABOUT MEDIANS BASED ON RANKS An alternative to the Pitman test is to apply a similar test not to the signed deviations from the hypothesized median but to the signed ranks of the absolute deviations. This leads to the signed-rank test proposed by Wilcoxon (1945). The population distribution is again assumed symmetric and ideally, continuous. In theory this means there is zero probability of two sample values coinciding. This may not be the case in practice; equal or tied values may be present in a data set. A modified form of the test allows for ties. 2.2.1 The Wilcoxon signed-rank test Given a sample of n independent measurements we first determine, as we did in Section 2.1.1, the magnitude of departures from the hypothetical mean or median θ0 specified in H 0. We arrange these absolute deviations in order of magnitude and assign ranks to these in ascending order (1 for the smallest, n for the largest). We next attach a negative sign to each rank that corresponds to a negative deviation (i.e. to data values below θ 0). We expect a near equal scatter of positive and negative ranks if θ0 is the true mean or median, implying that the sum of all positive ranks and the sum of all negative ranks should not differ greatly. A high sum of the positive (negative) ranks relative to that of the negative (positive) ranks implies θ0 is unlikely to be the population mean. We now apply a test that is mechanically equivalent to the Pitman test where signed ranks of deviations replace the signed deviations themselves. Example 2.4 The problem. For a group of 12 female students, the changes in heart rate (beats per minute) when standing up from lying down are: –2 4 8 25 –5 16 3 1 12 17 20 9 If we assume population symmetry, we may use the Wilcoxon signed-rank test to test H0: θ = 15 against the alternative H1: θ ≠ 15.
©2001 CRC Press LLC
Formulation and assumptions. We arrange the deviations from 15 in ascending order of magnitude and rank these, associating with each rank the sign of the corresponding deviation. We calculate the lesser of the sum of positive and negative signed ranks and, if available, use appropriate computer software to determine P. Procedure. Subtracting 15 from each sample value gives the deviations –17, –11, –7 , 10, –20, 1, –12, –14, –3, 2, 5, –6. We rearrange these in increasing order of magnitude while retaining signs, i.e. 1, 2, –3, 5, –6, –7, 10, –11, –12, –14, –17, –20, whence the signed ranks are 1, 2, –3, 4, –5, –6, 7, –8, –9, –10, –11, –12. The sum of the positive ranks, S+ = 1 + 2 + 4 + 7 = 14, is less than the sum, S– of the negative ranks. StatXact, Testimate and some general statistical programs give exact P-values in programs designed specifically for the Wilcoxon test where data may be input either in its original form or as signed ranks and the program then effectively carries out a Pitman test using signed ranks in place of the exact signed deviations. For this example these programs indicate that the two-tail P-value corresponding to S+ = 14 is P = 0.052. Conclusion. The evidence against H0 is not very strong, but the P-value is sufficiently small for a further study involving a larger group to be considered. Comments. 1. Doctors often assume that heart rate measurements are symmetrically distributed. In this example, the median, 8.5, is close to the mean, 9.0, indicating that symmetry is possible. There are several tests for symmetry when we are given only the sample values. One proposed by Randles et al. (1980) is described by Hollander and Wolfe (1999, Section 3.9) and also by Siegel and Castellan (1988, Section 4.4) but caution is needed in assessing significance if n < 20 in cases where the asymptotic result is on the borderline of significance. However, for these data the test statistic has a value well below any that would suggest asymmetry. There is, however, some evidence for a departure from normality because there is no particular cluster of values around the mean of 9.0; instead several values are close to zero and there is another group of values around 17. Given the likely degree of symmetry in the distribution the Wilcoxon test should not be seriously misleading. 2. The two clumps of values could reflect two distinct groups among female students: those who are heavily involved in sports such as hockey or tennis and those who rarely participate. The females who are physically fit may not experience a large increase in heart rate on standing compared with those who follow a more sedentary life style. 3. One must be cautious about generalizing the above results to the community at large. For a variety of reasons female students are probably not typical of young adults in general. Computational aspects. 1. Not all statistical packages give an exact P-value for the Wilcoxon signed-rank test, some only having a program that calculates S+ or S– and then sometimes gives only a P-value based on asymptotic theory (see Section 2.5). If one has no program to give exact P-values and is not prepared to seek a Monte Carlo approximation for small to medium samples it may be better to refer to published tables even though many give only values required for significance at a ‘nominal’ 5 or 1 per cent level in one- or two-tail tests. Due to discontinuities in the distribution of P-values the exact levels are not precisely 5 ©2001 CRC Press LLC
or 1 per cent. For most published tables the exact values are the closest possible values that do not exceed the nominal significance levels although a few tables give a value as close as possible to the nominal level even if that closest value exceeds the nominal level. Most major published statistical tables [e.g. Neave (1981) or Lindley and Scott (1995)] and some books on nonparametric methods include tables for the Wilcoxon signed-rank test and usually describe which of the above options they use for defining the nominal level. A more satisfactory table in Hollander and Wolfe (1999, Table A4) gives most of the exact P-values likely to be of interest for sample sizes 3 ≤ n ≤ 60, but these tables are not needed if programs like that in StatXact or Testimate are available. 2. The asymptotic P-value given by StatXact for these data for a two-tail test is P = 0.050 but this does not include a continuity correction we describe in Section 2.5. That correction increases the value to P = 0.0545. For these data the two-tail t-test value is P = 0.046. These values do not differ greatly from the exact value but give a timely warning on the dangers of adhering to a rigid ‘significance’ level such as α = 0.05 as a decision-making basis for accepting or rejecting H0.
2.2.2 Theory of the Wilcoxon signed-rank test We pointed out in Section 2.2.1 that the Wilcoxon signed-rank test is a special case of the Pitman test with signed ranks of the differences replacing the signed differences themselves. Using ranks gives theoretical simplifications and also has implications for what happens if an assumption of symmetry is not justified. Availability of computer software to give exact P-values means that mastery of the theory and its implications is not so important as it once was, but a broad understanding helps to remove misconceptions and is needed to show how to get confidence intervals. We describe the exact permutation test for a sample size n = 7, assuming no deviations are tied in magnitude. As already indicated, when H0 is true the symmetry assumption implies the sums of ranks for positive and negative ranked deviations should be nearly equal, whereas one sum should be much larger than the other if H 0 is not true. In the extreme case when all sample values are above the mean or median hypothesized in H0 all ranks will be positive so S– = 0 and S+ = 1 + 2 + 3 + 4 + 5 + 6 + 7 = 28. Clearly, for any allocation of signs to ranks S+ = 28 – S – when n = 7. More generally, for n observations S+ = 1⁄2n(n + 1) – S– since the sum of the ranks 1, 2, 3, . . . , n is 1⁄2n(n + 1). This implies a symmetry between S+ and S – in the sense that Pr(S– = s) =Pr[S+ = 1/2n(n + 1) – s].
©2001 CRC Press LLC
As in the Pitman test we may use any of the statistics S+, S– or SW = |S+ – S –|, since there is an ordered linear relationship between them. When n = 7, signs may be allocated randomly to ranks in 27 = 128 different and equally likely ways. The distribution of the test statistic S+ now depends only on the sample size, 7, since all data sets then give the rank magnitudes 1, 2, 3, 4, 5, 6, 7. To get the distribution of S+ we write down the S+ for all possible sign allocations and it is left as an exercise for the reader to establish that the relevant sums and associated probabilities are those in Table 2.1. Exact P-values for one- or two-tail tests may be obtained using that table and are applicable to a test for any data without tied values when n = 7. For example, in a one-tail test where a small lower-tail value of S+ is relevant the exact P-value corresponding to S+ = 3 is P = Pr(S+ ≤ 3) = (1 + 1 +1 + 2)/128 = 5/128 ≈ 0.039. This probability is doubled for a two-tail test. Manual calculation of probabilities associated with all values of S+ or S – for larger n is time consuming and error prone. StatXact obtains these values rapidly for sample sizes of practical importance. The number of different values of the rank sums increases as the sample size, n, increases and the differences between the discrete probabilities associated with each become smaller, especially in the tails. Also, the distributions of S+, S– approach that of a normal distribution as n increases. When n = 12 there are 2 12 = 4096 possible associations of signs with ranks and the possible signedrank sums range from 0 to 78. Symmetry means we need only know the probabilities for each sum between 0 and 39. Table 2.2 is adapted from StatXact output and gives sufficient information to calculate the complete permutation test distribution of S+ or S –. From Table 2.2 we easily find that Pr(S– ≤ 13) = 87/4096 ≈ 0.021 while Pr(S– ≤ 14) = 107/4096 ≈ 0.026. For two-tail tests we double these probabilities giving 0.042 and 0.052. Thus values of the smaller signed-rank sum not exceeding 13 indicate significance at level 4.2 per cent or less, i.e. below a conventional 5 per cent level. Again from Table 2.2 we see that Pr(S– ≤ 9) = 33/4096 ≈ 0.008, while Pr(S– ≤ 10) = 43/4096 ≈ 0.0105; it follows that for a one-tail test at a level not exceeding 1 per cent that S – ≤ 9 forms an appropriate critical region with associated P = 0.008. Figure 2.1, based on Table 2.2, shows the probability mass function (also known as the frequency function) of S+ or S– for n = 12 in the form of a bar chart. The shape resembles that of the familiar probability density function for the normal distribution. In Exercise ©2001 CRC Press LLC
2.4 we ask the reader to draw and comment upon a bar chart based on Table 2.1 for the case n = 7. Table 2.1 Exact probabilities that S+, S– equal a given value k in a Wilcoxon signed-rank test for a sample of size n = 7 when H0 is true. ________________________________________________________ k
Probability
k
Probability
0 1/128 15 8/128 1 1/128 16 8/128 2 1/128 17 7/128 3 2/128 18 7/128 4 2/128 19 6/128 5 3/128 20 5/128 6 4/128 21 5/128 7 5/128 22 4/128 8 5/128 23 3/128 9 6/128 24 2/128 10 7/128 25 2/128 11 7/128 26 1/128 12 8/128 27 1/128 13 8/128 28 1/128 14 8/128 ________________________________________________________ Table 2.2 Exact probabilities that S+, S– equal a given value k in a Wilcoxon signed-rank test for a sample of size n = 12 when H0 is true. Probabilities for k > 39 follow from the symmetry property Pr(S– = k) = Pr(S– =78 – k). ____________________________________________________________ k Probability k Probability k Probability 0 1/4096 13 17/4096 26 78/4096 1 1/4096 14 20/4096 27 84/4096 2 1/4096 15 24/4096 28 89/4096 3 2/4096 16 27/4096 29 94/4096 4 2/4096 17 31/4096 30 100/4096 5 3/4096 18 36/4096 31 104/4096 6 4/4096 19 40/4096 32 108/4096 7 5/4096 20 45/4096 33 113/4096 8 6/4096 21 51/4096 34 115/4096 9 8/4096 22 56/4096 35 118/4096 10 10/4096 23 61/4096 36 121/4096 11 12/4096 24 67/4096 37 122/4096 12 15/4096 25 72/4096 38 123/4096 39 124/4096
©2001 CRC Press LLC
Figure 2.1 A bar chart illustrating the distribution of the Wilcoxon signed-rank statistic when n = 12.
2.2.3 The Wilcoxon test with ties In theory, when we sample from a continuous distribution both the probabilities of tied observations or of a sample value exactly equalling the population mean or median are zero. Real observations never have a distribution that is strictly continuous in the mathematical sense either because of their nature or as a result of rounding or limited measuring precision. We measure lengths to the nearest centimetre or millimetre; weights to the nearest kilogram, gram or milligram; the number of pages in a book in complete pages, although chapter layout often results in part-pages of text; numbers of words in a sentence must be integers. These realities may produce data with rank ties or zero departures from a hypothesized mean or median. If they do, the exact distribution of S, the lower of the positive or negative rank sums, requires fresh computation for different numbers of ties and ties in different positions in the rank order. Before the advent of software to compute exact P-values for permutation distributions with ties, a common
©2001 CRC Press LLC
procedure was, after adjusting the scoring method based on ranks to allow for ties, to use the asymptotic normal approximation we give in Section 2.5. This usually works well for a few ties if n is reasonably large, but may have bizarre consequences for small n. Another approach for smaller samples was to use tables for the notie case assuming that the relevant value of S+ would be little affected by ties. This need not be so. When two or more ranks are equal in magnitude, but not necessarily of the same sign, we replace them by their mid-rank, an idea easily explained by an example. If 7 observations are 1, 1, 5, 5, 8, 8, 8 and we want to use a Wilcoxon signed-rank test with H0 : θ = 3 the relevant signed differences from 3 are –2, –2, 2, 2, 5, 5, 5. The magnitude of the smallest difference is 2 and four differences have that magnitude. The mid-rank rule assigns to these ties the mean of the four smallest ranks 1, 2, 3, 4; i.e. we allocate to these observations the signed ranks –2.5, –2.5, 2.5, 2.5. The remaining three differences are all 5 and we give each the mean of the remaining ranks 5, 6 and 7; i.e. each is ranked 6. The exact permutation test is based on the signed mid-ranks –2.5, –2.5, 2.5, 2.5, 6, 6, 6. An appropriate test statistic is S– = 5, the sum of the negative mid-ranks. If we test using the permutation distribution for the ‘no-tie’ case given in Table 2.1 we find Pr(S– ≤ 5) = 10/128 = 0.078. However, with ties, the distribution of S– under H0 is not that in Table 2.1. The exact distribution with ties depends both on how many ties there are and where they lie in the rank sequence. We can work out exact Table 2.3 Exact probabilities that S+, S– equal a given value k in a Wilcoxon signed-rank test for a sample of size n = 7 when H0 is true and mid-ranks are 2.5 (four times) and 6 (three times). _______________________________________________________ k
Probability
k
Probability
0 1/128 14.5 12/128 2.5 4/128 16 3/128 5 6/128 17 18/128 6 3/128 18 1/128 7.5 4/128 19.5 12/128 8.5 12/128 20.5 4/128 10 1/128 22 3/128 11 18/128 23 6/128 12 3/128 25.5 4/128 13.5 12/128 28 1/128 _____________________________________________________
©2001 CRC Press LLC
significance levels with ties for small to medium sized samples. For any tie pattern StatXact or Testimate give exact P-values for oneand two-tail tests and the former will also compute the complete distribution of S+ or S– for samples that are not too large. For the tie pattern above for a sample of 7, StatXact gives the exact distribution in Table 2.3. Comparing Tables 2.1 and 2.3 we see that in each case the distribution of the test statistic is symmetric (that is why we may base our test on either S+ or S –). However, the distribution with noties is unimodal (i.e. it has one maximum) with values of the statistic confined to integral values increasing in unit steps, but that in Table 2.3 is heavily multimodal with S taking unevenly spaced and not necessarily integer values. Discontinuities are also more marked. From Table 2.3 we easily find that Pr(S– ≤ 5) = 11/128, compared to the no-tie probability 10/128 found from Table 2.1. Despite many ties in this example and the differences in the distribution of the test statistic in the two cases, it may appear that ties have little effect on our conclusion. However, this is not always the case. Example 2.5 Ties often result from rounding. The data 1, 1, 5, 5, 8, 8, 8 considered above could arise from rounding to the nearest integer in either of the sets: 0.9, 1.1, 5.2, 5.3, 7.9, 8.0, 8.1 0.9, 1.0, 4.8, 4.9, 7.9, 8.0, 8.1 For a Wilcoxon test with H0: _ = 3 the signed differences from 3 are: –2.1, –2.1,
–1.9, –2.0,
2.2, 1.8,
2.3, 1.9,
4.9, 4.9,
5.0, 5.0,
5.1 5.1
with signed ranks: –2, –4,
–1, –3,
3, 1,
4, 2,
5, 5,
6, 6,
7 7
There are now no tied ranks in either set and for the first set S– = 3, while for the second set S– = 7. Table 2.1 gives Pr(S– ≤ 3) = 5/128 ≈ 0.039 and Pr(S– ≤ 7) = 19/128 ≈ 0.148. Thus a one-tail test at a level not exceeding 5 per cent would indicate an implausible hypothesis in the first case but not in the second. Rounding to the nearest integer thus greatly affects the apparent strength of evidence against the null hypothesis. This example is a simple illustration of difficulties of interpretation with ties due to rounding, a process that makes small perturbations or changes to most observations. M any statistical analyses are sensitive to such changes. The impact of ties due to rounding in this example is more extreme than one is likely to meet in practice, resulting in only two different magnitudes for ranks. For larger samples and not too many ties, the effect on P-values is less marked. ©2001 CRC Press LLC
Example 2.6 The problem. Consider the data in Example 2.2 for the numbers of words in sentences in Fisher (1948), i.e. 12 18 24 26 37 40 42 47 49 49 78 108 Use the Wilcoxon signed-rank test to test for the population median θ the hypothesis H0: θ ≤ 30 against the alternative H1: θ > 30. Formulation and assumptions. To justify the Wilcoxon test, we assume the population is symmetric. The deviations of observations from 30 are ranked, with due regard to sign using mid-ranks for ties. The lesser of S+ and S– is then found. Procedure. The signed deviations from 30 are –18, –12, –6, –4, 7, 10, 12, 17, 19, 19, 48, 78 and the corresponding signed ranks using mid-ranks for the tied magnitudes at 12 and 19 are –8 –5.5 –2 –1 3 4 5.5 7 9.5 9.5 11 12 giving S– = 16.5 (or S+ = 61.5). A one-tail test is appropriate. StatXact gives the exact probability of observing S– ≤ 16.5 with this pattern of tied deviations when n = 12 as P = 0.0405. Conclusion. The evidence implies H0 is implausible at a 4.05 per cent significance level, so there is fairly strong evidence against H0. Comments. 1. Although the P-value here is identical to that for the Pitman test in Example 2.2, in general this is not the case. Large differences between results for the two tests, although possible, are unusual (see Example 2.13). 2. Before we had software that gave the exact distribution of the test statistic when there are ties, the advice to resort to asymptotic (large sample) results of a type we describe in Section 2.5 had little justification for small samples. 3. The data values 78, 108 throw doubt on the validity of the symmetry assumption. Had these observations been replaced by, say, 51, 53 our Wilcoxon rank-sum test would give the same result but there then would have then been no intuitive reason to doubt the symmetry assumption. However the Pitman test would be affected by such a change and in Exercise 2.5 we ask you to confirm that with those alterations for that test P = 0.0505. Computational aspects. With packages like StatXact it is a useful exercise to generate exact distributions for a number of tied situations for small sample sizes to develop a feeling for the effect of ties on standard results. An extreme case arises with the observations 4, 4, 8, 8, 8, 8, 8 for the Wilcoxon test when we set H0: θ = 6. Then all ranks of deviations are tied apart from signs and it is easy to see (Exercise 2.6) that the Wilcoxon test statistic distribution is now equivalent to that for the sign test. In this sense a sign test is a special case of the Wilcoxon test where all deviation magnitudes are tied at the same value.
If one or more sample values equal the mean hypothesized under H 0, the associated ‘deviations’ are zero. Statisticians are divided on how best to proceed in this case. Some argue that we should omit such observations from our computations because they are
©2001 CRC Press LLC
uninformative about any alternative hypothesis. The test is then applied to a reduced sample. An alternative we recommend is to rank all deviations temporarily giving any zero the rank 1 because it is the smallest difference (or the appropriate tied rank if there is more than one zero). After all signed ranks are allocated we then change the rank(s) associated with zero difference to 0, leaving the other ranks as allocated. To obtain exact tests in the latter situation a suitable computer program is needed for the exact distribution of the test statistic, else we may resort to asymptotic results in a way we indicate in Section 2.5. Deleting zero differences is the simplest procedure if no suitable program for exact probabilities of S for the modified rank statistic is available, but the second procedure has the theoretical advantage of providing a more powerful test. 2.2.4 Confidence intervals based on signed ranks For many nonparametric methods the more important aim of calculating confidence intervals is not so simple as hypothesis testing, but the procedures, once mastered, are often straightforward and not difficult to program by writing a macro if no package program is available. Although at the time of writing only a few packages deal in a completely satisfactory way with confidence intervals based on Wilcoxon signed-rank test theory, the position is improving steadily. The trial-and-error approach suggested in Section 2.1.2 for the Pitman test is avoided by using an alternative way of calculating the statistic S – (or S+). We outline the theory, but a reader prepared to take this on trust may move directly to Example 2.7 where this approach is applied to both hypothesis testing and determining confidence intervals. For convenience we assume a sample of n observations with no ties has been arranged in ascending order x 1, x 2, . . . , x n. The null hypothesis for the mean or median is specified as H 0: θ = θ 0. In practice θ0 will usually lie between x1 and x n, but this is not a theoretical requirement. A deviation xi – θ0 will have a negative signed rank for any xi less than θ 0. Since the xi are in ascending order, among all negative ranks, if there are any, that of greatest magnitude will be associated with x1, that of next greatest magnitude with x 2, and so on. Similarly, all xj greater than θ0 have positive signed ranks and of these xn will have the highest rank, x n–1 the next highest and so on. ©2001 CRC Press LLC
Consider now the paired averages of x1 with each of x1 , x2 , . . . , xn. Suppose x1 has associated signed rank –p. A moment’s reflection shows that each of the averages 1/2(x1 + xq) will be less than θ0 providing the deviation associated with xq has either a negative rank or a positive rank less than p. Since we are assuming there are no rank ties no observation has positive rank p (since the rank p is already associated with x 1). If xq is the smallest observation associated with a positive rank greater than p, this implies |xq – θ0| > |x1 – θ0 | and it follows that the average of x1 and xq is greater than θ 0. Then, because the data are in ascending order, so is the average of x1 with any xr > x q. Thus the number of averages involving x1 that are less than θ0 is equal to the negative rank associated with x1. Similarly, if we form all the paired averages of any xi less than θ0 with each of xi , xi+1 , . . . , xn the number of these averages less than θ0 will equal the (negative) rank associated with xi . Clearly, when xi has a positive signed rank none of the averages with itself or greater sample values will be less than θ0 . The complete set of averages 1 /2(xi + xj) for i = 1, 2, . . . , n and all j ≥ i are often called Walsh averages, having been proposed by Walsh (1949a, b). Clearly the number of Walsh averages less than θ0 equals S – and the number greater than θ0 equals S+. We may use Walsh averages to calculate the test statistic S for a given H0 and to obtain confidence limits for a population mean or median. Example 2.7 The problem. Given the changes in heart rate for the female students in Example 2.4, viz. –5 –2 1 3 4 8 9 12 16 17 20 25 obtain an estimate of the population mean with confidence intervals at levels at least 95 and 99 per cent, assuming a symmetric distribution. Formulation and assumptions. We use Walsh averages and the principle enunciated in Section 1.4.1 that the 100(1 – α) per cent confidence interval contains those values of the parameter that would be accepted in a significance test at the 100α per cent significance level. Procedure. Table 2.4 shows a convenient way to set out Walsh averages. Both the top row and the first column (in italics) of that table give the sample values in ascending order. The triangular matrix forming the body of the table gives the Walsh averages. Comment 2 below gives tips for calculating these if a program to do so is not available. An intuitively reasonable point estimator of the population median is the median of the Walsh averages since it follows from the arguments given before this example that for this estimate the sums of both the negative and the positive rank deviations are equal, both being equal to the number of Walsh averages ©2001 CRC Press LLC
Table 2.4 Walsh averages for changes in heart rate from lying to standing.
–5 –5 –2 1 3 4 8 9 12 16
–2
3
4
8
–5.0 –3.5 –2.0 –1.0 –0.5 1.5 –2.0 –0.5 0.5 1.0 3.0 1.0 2.0 2.5 4.5 3.0 3.5 5.5 4.0 6.0 8.0
17 20 25
1
9
12
16
17
20
25
2.0 3.5 5.0 6.0 6.5 8.5 9.0
3.5 5.0 6.5 7.5 8.0 10.0 10.5 12.0
5.5 7.0 8.5 9.5 10.0 12.0 12.5 14.0 16.0
6.0 7.5 9.0 10.0 10.5 12.5 13.0 14.5 16.5 17.0
7.5 9.0 10.5 11.5 12.0 14.0 14.5 16.0 18.0 18.5 20.0
10.0 11.5 13.0 14.0 14.5 16.5 17.0 18.5 20.5 21.0 22.5
25.0
above (or below) this median. The median of the Walsh averages may be determined by inspecting Table 2.4. We see that entries in any row increase from left to right and those in any column increase as we move down, so that the smallest entries are at the top left of the table and the largest are at the bottom right. For a sample size n there are 21 n(n + 1) Walsh averages, i.e. 78 in this example. Thus the median of these lies between the 39th and 40th ordered Walsh average and inspection of the table establishes that both equal 9.0; so this is the appropriate estimate of the population median (or mean). To establish a 95 per cent confidence interval we select as end points values of θ that will just be acceptable if P = 0.05. From Example 2.4, P is only slightly more than 0.05 if the sum of the lesser of positive or negative ranks is 14. Thus, if we choose a value less than the 14th smallest Walsh average we would reject H0. Similarly if we choose a value greater than the 14th largest Walsh average we also reject H0 since in either case our test statistic S would be such that S ≤ 13. Thus for n = 12 a confidence interval at not less than a 95 per cent confidence level is the interval (14th smallest Walsh average, 14th largest Walsh average). From Table 2.4, by counting from the most extreme value we see that this is the interval (2.5, 16.0). We noted in discussing Table 2.2 that using S ≤ 13 as a criterion for significance corresponded to an exact significance level of 4.25 per cent (two tails). It is intuitively reasonable therefore to regard the interval (2.5, 16.0) as an actual 95.75 per cent confidence interval. Owing to discontinuities in the distribution of the test statistic and the fact that the end points of the confidence interval often coincide with observed values we must regard values outside this interval as ones indicating rejection at a 4.25 per cent significance level. We must also, if our exact level is critical, test the end values to see if they would be accepted or rejected at this significance level. Exact tests using StatXact show that we would find evidence against the hypothesized means of 2.5 and 16.0. Thus in this case all points in the open interval 2.5 < θ < 16.0 would be acceptable, but the closed interval (2.5, 16.0) is a conservative 95 per cent interval, having a confidence level exceeding 95 per cent. In practice it is usual to cite a confidence interval ©2001 CRC Press LLC
simply as the interval (2.5, 16.0) without being specific as to whether it is open or closed, bearing in mind that discontinuities inherent in the Wilcoxon test statistic distribution may imply (and it will vary from case to case) that the end points may or may not be included as acceptable values at a nominal significance level such as 5 per cent. One should not get excited about subtleties due to discontinuity at the end point of intervals. If relevant programs are available it may be worthwhile checking the actual confidence level associated with any quoted interval. It is left as an exercise to establish that a nominal 99 per cent confidence interval (i.e. with at least 99 per cent coverage) is the closed interval (0.5, 18). Conclusion. An appropriate point estimator of the population median is 9.0 and actual 95.75 and 99.08 per cent confidence intervals are respectively (2.5, 16.0) and (0.5, 18), where it is understood that if values of θ outside these intervals are specified in H0 there will be evidence against these at levels implied by the relevant P-values. Comments. 1. A reasonable alternative point estimator to the median of the Walsh averages is the mean of all sample values. It is unbiased in the sense that if we sampled repeatedly and took the mean of the sample values as our estimate in all cases, the mean of these estimates would converge to the population mean. However, the median of the Walsh averages is in a certain technical sense, which we do not discuss here, generally closer to the population median; this estimator is called the Hodges–Lehmann estimator, having been proposed by Hodges and Lehmann (1963). A more formal justification for its use when basing the interval on signed ranks is given by Hettmansperger and McKean (1998, Section 1.3). 2. If confidence limits only are required we need write down only a few Walsh averages in the top left and bottom right of the triangular matrix in Table 2.4. If a computer program is available for forming this matrix there is no problem in computing all the averages. If the matrix has to be formed manually once we have the Walsh averages in the first row, the required averages in the second row may be obtained by adding the same constant, namely half the difference x2 – x1, to the average immediately above it. To verify this denote by xij the average of xi and xj. This is the entry in row i and column j of the matrix of Walsh averages. Now for any k > 2 x2k =
1 1 1 (x2 + xk) = (x2 – x1 + x1 + xk) = (x2 – x1) + x1k. 2 2 2
The idea generalizes for all subsequent rows, each entry in row r + 1, say, being obtained by adding the same constant 1 (xr+1 – xr) to the entry immediately 2 above it. 3. In Example 2.4 we found when testing H0: θ = 15 that S+ = 14. In Table 2.4 there are 14 Walsh averages exceeding 15.0. Computational aspects. 1. Minitab includes a program for computing Walsh averages. It gives asymptotic confidence intervals (see Section 2.5). These may not be satisfactory for small n, especially when there are many ties. If one has the matrix of Walsh averages generated by some available program one may easily establish nominal 95 or 99 per cent confidence limits using tables of critical values of the statistic S.
©2001 CRC Press LLC
2. StatXact includes a program giving the Hodges-Lehmann estimator and confidence intervals at any pre-assigned nominal level and confirms the values obtained above.
With ties in signed ranks the method for confidence intervals in this section needs modification. If computer software for confidence intervals in these circumstances is not available one might use the Walsh average method described above, then use an available computer program to study appropriate end point adjustments to those for the untied case along the lines indicated in Example 2.3 for the Pitman interval. Such adjustments are usually small unless there is heavy tying. Example 2.8 The problem. Consider again the numbers of words in sentences used by R. A. Fisher with the sample given in Example 2.6, i.e. 12 18 24 26 37 40 42 47 49 49 78 108 Obtain a confidence interval at a level not less than 95 per cent for the median sentence length based on the Wilcoxon signed-rank test. Formulation and assumptions. We form the Walsh averages. We already know from Example 2.7 that for samples of 12 in the no-tie case an interval with nominal 95 per cent coverage is that between the 14th smallest and 14th largest Walsh average. With a small number of ties one hopes these will be good approximations and this may be tested and adjustments made as indicated under Procedure if an exact hypothesis testing program is available. Procedure. Walsh averages may be calculated using either some appropriate computer program or manually in the way used to obtain Table 2.4. This is left as an exercise (Exercise 2.8). The 14th smallest average turns out to be 27.5 and the 14th largest to be 63 suggesting an interval (27.5, 63) slightly longer than that obtained in Example 2.3 based on the Pitman test. Fine tuning of end points using a trial-and-error method with StatXact leads to an interval of (27.5, 62.5) as an exact 95.3 per cent interval (lower-tail P = 0.022, upper-tail P = 0.025). Conclusion. The 95.3 per cent confidence interval is (27.5, 62.5). Comments. This interval differs little from those associated with the t-test or the Pitman test approach and is again strongly influenced by the outlying values of 78 and more especially 108. This may seem surprising after we intimated in the comments on Example 2.6 that the hypothesis test result would not have been altered if these two observations were replaced by values appreciably more in line with the rest of the data. The reason these values influence the upper confidence limit is evident from a full table of Walsh averages (Exercise 2.8), for it will be apparent from this that the 14 highest averages all include at least one of the values 78, 108 Computational aspects. Obtaining Walsh averages is reasonably straightforward even when no program is available. Ties create a situation where fine ©2001 CRC Press LLC
tuning of the limits is best achieved by a trial-and-error procedure using an exact test program such as those in StatXact or Testimate, commencing with limits that ignore tying, i.e. that are appropriate to a no-tie case. However, if an exact program for a confidence interval such as that in StatXact is available fine tuning is not needed.
2.3 THE SIGN TEST 2.3.1 The sign test and the effect of sample size We look more closely at assumptions and possible complications and extend concepts developed in Example 1.2. For the sign test – like most tests – the larger the sample the higher the power and the shorter the confidence interval for a given confidence level. A symmetry assumption is not needed and providing all sample xi values are independent they need not all have the same distributions providing that they are from distributions with identical medians. Remember that for skew distributions the median does not equal the mean. Example 2.9 The problem. The percentage water content of agricultural land often varies only slightly over an area such as a large field apart from a few local patches of exceptionally high or low percentages associated with especially poor or good drainage. The potential range of percentage water content is exemplified in a set of readings for nearly 400 soil quadrats on 22 June 1992 given by Gumpertz, Graham and Ristiano (1997). We use two small subsets (with values slightly rounded) from this large data set to illustrate some points about single sample nonparametric analysis. The use we make of these data has no direct relevance to the paper from which our samples were taken. The percentage water contents for our sampled quadrats, arranged in ascending order of magnitude, are: Sample I
5.5
6.0
6.5
7.6
7.6
7.7
8.0
8.2
9.1
15.1
Sample II
5.6 8.0
6.1 8.1
6.3 8.1
6.3 8.2
6.5 8.4
6.6 8.5
7.0 8.7
7.5 9.4
7.9 14.3
8.0 26.0
We use these data to test in each case H0: θ = 9 against the alternative H1: θ ≠ 9 for the population median θ using the sign test. Formulation and assumptions. If the population median is 9, then any observation in a random sample is equally likely to be either above or below 9. Proceeding as in Example 1.2 we associate a plus sign with values greater than 9, and if the hypothesis H0: θ = 9 holds, the number of plus signs has a binomial distribution with p = 21 , n = 10, i.e. a B(10, 21 ) distribution for the first sample 1 and a B(20, 2 ) distribution for the second sample. For the alternative H1: θ ≠ 9, a two-tail test is appropriate.
©2001 CRC Press LLC
Procedure. In the first sample there are 2 values, 9.1 and 15.1, greater than 9, implying 2 plus signs. We gave a table of B(10, 12 ) probabilities in Example 1.2 (p. 17) from which we easily calculate the probability of 2 or less or 8 or more plus signs is P = 0.109, a value confirmed by any computer program for the sign test. In the second sample there are 3 values, 9.4, 14.3 and 26.0 greater than 9. 1 The probability of 3 or less plus signs for a B(20, 2 ) distribution is easily calculated although there is seldom any need to do this with the ready availability of computer software or tables for a sign test, and any relevant program should give P = 0.003 for the relevant two-tail test. Conclusion. With the sample of 10, since P = 0.109 there is no convincing evidence against H0 since this clearly exceeds the conventional 5 per cent level. With a sample of 20 the evidence against H0 becomes very strong since P = 0.003 corresponding to a 0.3 per cent significance level. Comments. 1. We used a two-tail test since there was no a priori reason to confine alternatives to only greater or only less than the value specified in H0. 2. The results are consistent with our remark in Section 1.3 that increasing sample size generally increases the power of a test. Computational aspects. With adequate tables of binomial probabilities when p = 12 for various n, computer programs might almost be regarded as optional extras for the sign test but StatXact and Testimate and nearly all major general statistics packages include programs for this or equivalent procedures.
If one or more sample values coincide with the value of θ specified in H 0 we cannot logically assign either a plus or minus to these observations. They may be ignored and the sample size reduced by 1 for each such value, e.g. if in the sample of 10 in Example 2.9 we specify H0 : θ = 8.0 we treat our problem as one with 9 observations giving 3 plus signs. Doing this rejects evidence which strongly supports the null hypothesis, but which is uninformative about the direction of possible alternatives. Another proposed approach is to toss a coin and allocate a plus to a value equal to θ if the coin falls heads and a minus if it falls tails: yet another is to assign a plus or minus to such a value in a way that makes rejection of H 0 less likely. The coin-toss approach usually makes only a small difference and has little to commend it since it only introduces additional randomness, being in a sense more ‘error’; the last approach is ultraconservative. Lehmann (1975, p. 144) discusses pros and cons of these choices in more detail. 2.3.2 Confidence intervals Example 2.10 The problem. Obtain approximate 95 per cent confidence intervals for the median water content using the sign test with the samples in Example 2.9.
©2001 CRC Press LLC
Determine the exact confidence level in each case. Formulation and assumptions. We seek all values for the population median _ that would not be rejected in a two-tail test at a 5 per cent significance level. Procedure. We use the argument developed in Section 1.4.1. Consider first the sample of 10. From the table given in Example 1.2 (or any published tables or using a computer program that calculates probabilities for any given n, p) we easily find that a critical region consisting of 0, 1, 9 or 10 plus signs has size 2 × (0.0010 + 0.0098) = 0.0216 and that this is the largest exact size less than 0.05. Thus we reject H0 if it leads to 1 or fewer or 9 or more plus signs. We retain H0 if we get between 2 and 8 plus signs. It is clear from the sample values 5.5 6.0 6.5 7.6 7.6 7.7 8.0 8.2 9.1 15.1 in Example 2.9 that we have between 2 and 8 plus signs if θ lies in the open interval from 6 to 9.1, i.e. θ > 6 and θ < 9.1. This implies an exact confidence level of 100(1 – 0.0216) = 97.84 per cent for the interval (6, 9.1). For any θ not in this interval we would reject the hypothesis that it is the population median at the 100 – 97.84 = 2.16 per cent significance level. We leave it as an exercise for the reader, using appropriate tables or a relevant computer program, to show that for the sample of 20 in Example 2.9 the interval (6.6, 8.4) is an actual 95.86 per cent confidence interval. Conclusion. For the sample of 10 an exact 97.84 per cent confidence interval for the median is (6, 9.1) and for the sample of 20 an exact 95.86 per cent interval is (6.6, 8.4). Comments. 1. The sample median is an appropriate point estimator of the population median when we make no assumption of symmetry. For our samples these are respectively 7.65 and 8.0. The rationale for this choice is that it gives equal numbers of plus and minus signs, the strongest supporting evidence for H0. Hettsmansperger and McKean (1998, Section 1.3) justify this choice on even stronger theoretical grounds. 2. Increasing the sample size shortens the confidence interval. 3. The samples contain values 14.3, 15.1 and 26.0 that all appreciably exceed the median, suggesting a skew distribution of water content with a long upper tail. This is in line with the common experience that after rain, for example, one often finds a few small areas in a field excessively wet due to poor local drainage. 4. Skewness makes normal theory (t-distribution) tests or confidence intervals inappropriate. Remember that the mean and median do not coincide for a skew distribution. In passing we note that normal theory t-test based estimation for the sample of 10 in this example gives a mean of 8.13 with a 95 per cent confidence interval (6.2, 10.0) and for the sample of 20 a mean of 8.77 and a 95 per cent confidence interval (6.7, 10.9). Both intervals are slightly longer than those obtained using the sign test and lead to the somewhat bizarre result that the confidence interval for the larger sample is longer than that for the smaller one. Further they apply to the means rather than the medians and thus are shifted to the right, being symmetric about the sample mean; a moment’s reflection will show this shift is a consequence of the presence of the high right-tail values 14.3, 15.1 and 26.0, i.e. those that suggest skewness.
©2001 CRC Press LLC
Computational aspects. Given suitable tables there is little need for a computer program to calculate sign-test confidence intervals. Many general statistical packages will generate the relevant binomial probabilities if these are not available in tables. In Comment 4 above we considered t-distribution based 95 per cent confidence intervals. For direct comparison with the nonparametric intervals, many statistical packages provide parametric (normal theory, tdistribution) confidence intervals at any specified level such as 95.86 or 97.84 per cent if one wants to make closer comparisons at exact levels.
In Comment 4 above we remarked that t-distribution confidence intervals are not appropriate because the data are skew. We suggested when discussing in Examples 2.3 and 2.8 the Fisher sentence length data that there is doubt there about the t-test based intervals. However, we found in those examples that the Pitman, ttest and Wilcoxon test based intervals were not very different. It is interesting to consider the sign-test based interval for those data. Example 2.11 Following arguments similar to those in Example 2.10 and applying them to the sentence length data set given in Example 2.2, i.e. 12 18 24 26 37 40 42 47 49 49 78 108 1
it is easily verified (using appropriate binomial tables when n = 12, p = 2 or a suitable computer program) that the interval (24, 49) is an exact 96.14 per cent confidence interval and this is appreciably shorter than the interval (27.5, 62.5) obtained in Example 2.8. The sign test does not assume a symmetric distribution and the consequence when that assumption is violated is sometimes (but not always) that it has higher Pitman efficiency resulting in greater power in the hypothesis testing context which often translates to shorter confidence intervals. However, where a symmetry assumption is valid the sign-test approach usually leads to wider confidence intervals than either the Pitman or Wilcoxon test methods. An exception is certain symmetric distributions with very long tails, e.g. the double exponential, where the sign test has higher Pitman efficiency than the signed-rank test and may give a shorter confidence interval.
2.4 TRANSFORMATION OF RANKS
2.4.1 Normal scores In essence transformation of a sample of n continuous ordered data to ranks replaces the sample values by something like values we would expect when sampling from a uniform or rectangular distribution over (0, n) – more precisely the rank r corresponding to the r/(n + 1)th quantile of that distribution. Might further ©2001 CRC Press LLC
transformation of these ranks increase the Pitman efficiency or have other advantages? In many parametric analyses a normality assumption is a key feature and when it can validly be made it often leads to mathematical and computational simplicity. In Section 2.1 we indicated that exact permutation tests were only practical for small to medium sample sizes even with the best software. We show in Section 2.5 how we sometimes face this difficulty using asymptotic results which themselves carry an assumption of normality. It is not unreasonable to hope that if we start with data that resemble a sample from a normal distribution that such asymptotic or ‘large sample’ results will approach exact results more rapidly in the sense that they may be reasonable for not very large samples. Such hopes inspired a study of transformations to give data more like those from a normal distribution as a basis for exact tests while retaining some of the desirable properties such as the test being unconditional on the actual data as was the case for the Wilcoxon signed-rank test (but not for the Pitman test). A possible transformation is suggested by our remark that ranks correspond to quantiles of a uniform distribution. Why not transform ranks to values (often called scores) corresponding to quantile values for a normal distribution? More specifically, why not choose the standard normal distribution, for then it is easy to make the transformation using tables of the standard normal cumulative distribution function (which we now abbreviate to cdf). For example, if we have the very nonnormal sample 2, 3, 7, 21, 132 we first replace these values by the ranks 1, 2, 3, 4, 5 and transform these to normal scores which are the 1 /6 th, 2 /6 th, 3/6 th, 4 /6 th and 5 /6th quantiles of the standard normal distribution. These may be read from standard normal cumulative distribution function (cdf) tables, but many software programs compute these as one of a set of possible data transformations. The normal score corresponding to the 1 /6 = 0.1667 quantile is the x value such that the standard normal cdf, commonly denoted by Φ (x), is Φ (x) = 0.1667. From tables (e.g. Neave, 1981, p.18) or from software we find x = –0.97. For the 2 /6 = 0.3333 quantile we find x = –0.43 and for the 3/6th (mean or median) x = 0. By symmetry the remaining quantiles are x = 0.43 and x = 0.97. These quantile scores are often called van der Waerden scores having been proposed by van der Waerden (1952; 1953). Alternatives to the above scores are discussed by Conover (1999, Section 5.10) and others. These include expected normal scores where the ith ordered sample value in ascending order, sometimes
©2001 CRC Press LLC
called the ith order statistic, is replaced by the expectation of the ith order statistic for the standard normal distribution. The ith order statistic is often written x(i). Fisher and Yates (1957, Table XX) give expected normal scores corresponding to ranks for n ≤ 50. In practice using van der Waerden or expected normal scores usually lead to similar conclusions, but both need adapting for use in the one sample situation. We consider here only van der Waerden scores. Direct application of van der Waerden scores in an analogue to the Wilcoxon signed-rank test is not possible because the rationale of the test demands that we allocate signs to magnitudes of ranks but van der Waerden scores are of equal magnitudes but opposite signs. One practical way around this difficulty is first to add a constant k to all van der Waerden scores so that they are nonnegative. The choice k = 3 achieves this for sample sizes less than 700. Adding 3 in effect gives normal scores for a distribution of mean 3 and standard deviation 1. The next step is to attach a negative sign to the score corresponding to a negative deviation. Programs for raw data permutation tests analogous to the Pitman test for the original data may be used with these scores now treated as the raw data. Example 2.12 The problem. Use the normal scores test procedure outlined above to test the median hypothesis H0 : θ = 30 against H1: θ > 30 for the sentence length data given in Example 2.2, viz: 12 18 24 26 37 40 42 47 49 49 78 108 Formulation and assumptions. We replace the signed ranks in Example 2.6 by van der Waerden scores, then add three to each and perform a randomization test on these regarding them as raw data. Procedure. Statistical software may be used to form the scores and add the constant k = 3. In StatXact (or most software programs) this is a one-step process but for completeness we give the relevant scores here. The unmodified van der Waerden scores in order of increasing absolute ranks of differences used in the Wilcoxon test in Example 2.6 are: –1.43 –1.02 –0.74 –0.50 –0.19 –0.19 0.10 0.29 0.62 0.62 1.02 1.43 Adding 3 to each of the above scores gives a set of all positive scores and we then assign negative signs to those that correspond to negative deviations, giving the modified scores –1.57 –1.98 2.26 2.50 –2.81 2.81 3.10 –3.29 3.62 3.62 4.02 4.43 The raw data permutation test (effectively a Pitman test) applied to these scores as data gives a one-tail P = 0.062, not in close agreement with the values in Examples 2.2 and 2.6. ©2001 CRC Press LLC
Conclusion. There is only fairly weak evidence against H0 and in favour of H1. Comments. 1. The result is somewhat out of line with common experience that results using these scores usually differ little from those using the Wilcoxon test. In Section 2.5 we consider the situation regarding asymptotic tests for each. 2. Discontinuities in possible P-values tend generally to be smaller for this test than for the Wilcoxon test because the test statistic used here takes more possible values. 3. The distribution is symmetric in the sense that we may use either the sum of the positive scores or the sum of the negative scores as our test statistic. Ties have been dealt with in this example by assigning quantiles corresponding to the mid-ranks. An alternative sometimes recommended is to assign instead the mean of the quantiles corresponding to the ranks that are tied, e.g. if there is a tie for the rth and the (r + 1)th ordered data values we take the mean of the quantiles corresponding to each. Generally the difference between these approaches is small. 4. So far as we know there is no completely satisfactory way of getting a confidence interval based on these scores. Not only do the same difficulties arise as in the Pitman test situation for determining end points but the interval pertains to the transformed differences and there appears to us no satisfactory way of back-transforming to the original data. Problems of this sort arise in many data transformation situations both in parametric and nonparametric inference. 5. Normal score tests in this and other situations met in later chapters have Pitman efficiency 1 when the normal theory t-test is appropriate and in any other case the Pitman efficiency is at least 1. Any euphoria generated by this property must be tempered by a realization that there are other tests with Pitman efficiency greater than 1 relative to normal theory tests when certain other appropriate distributional assumptions are made and also that an asymptotic result does not guarantee equivalent small sample efficiency, although Pitman efficiency is usually a good guide. We remarked in Comment 3 on Example 2.6 that these data suggest skewness.
So far as we are aware the procedure in Example 2.12 was first formally described in the second edition of this book but it is so straightforward we would be surprised if it or something similar has not been used earlier. An alternative method for obtaining positive scores to which appropriate signs may be allocated is to use only some of the positive quantiles (i.e. those lying above the median) of the standard normal distribution as scores. Appropriate signs are then attached to these. Recommended quantiles corresponding to the ranks 1, 2, 3, . . . , n are the (n + 2)/(2n + 2)th, (n + 3)/(2n + 2)th, (n + 4)/(2n + 2)th, . . . , (2n + 1)/(2n + 2)th quantiles. The attached signs are those for Wilcoxon signed ranks. One then proceeds using these scores as though they were the original data in a Pitman-type test. The fact that these quantiles are not symmetrically distributed about their median detracts from the intuitive appeal of the
©2001 CRC Press LLC
transformation, but again our experience with a few examples is that it generally gives results similar to a Wilcoxon signed-rank test. Maritz (1995, Example 2.15) gives an example using these scores or closely related scores based on expected values and proceeds to discuss hypothesis testing and point estimation using these and even more general scores. 2.4.2 Other scores Scores other than normal scores are sometimes appropriate. We use some of these in more sophisticated problems. They are needed when data that are essentially nonnormal occur or when there are complications due to censoring of data. An example is given in Section 5.5.1. In analyzing survival data what are called Savage or exponential scores are often appropriate. We introduce these in Section 5.5.2 and they are also discussed by Lehmann (1975, p. 103) and by Sprent (1998, Section 4.9). 2.5 ASYMPTOTIC RESULTS Widely available software for exact permutation tests for small to medium-sized samples removes the need to use asymptotic results for such samples where one knows these may not be reliable unless samples are moderately large (how large depending partly upon the procedure being used). All centrality test and estimation procedures given so far in this chapter – Pitman, Wilcoxon, sign test, rank-transformations – have been based on statistics that are the sum of certain scores, e.g. the sum of the positive deviations from a hypothesized mean or median θ0 in the Pitman test, the sum of the positive signed ranks in the Wilcoxon test, the number of positive signs in the sign test (i.e. the sum of positive scores if each deviation is scored as +1 or –1). Here the deviations, the signed ranks, the ±1s and likewise the signed modified van der Waerden scores are each relevant scores for a particular procedure. In general for a sample of n observations our procedures have been based on the sum, S, of positive (or negative) scores where the score allocated to observation i may in general be denoted by si , i = 1, 2, . . ., n. Under the null hypothesis a key feature is that each score si is equally likely to have a positive or negative sign. It follows (Exercise 2.16) that any such statistic S has mean value 1/2Σ|si| and ©2001 CRC Press LLC
variance 1/4Σsi2 where the summation is over all n scores. From the central limit theorem it follows that for large n the distribution of Z=
∑| s | ∑s
S − 12 1
2
i
(2.1)
2 i
is approximately standard normal. Formula (2.1) simplifies for a number of special cases (Exercise 2.16). For example, with no rank ties it is easy to verify for the Wilcoxon signed-rank test E(S+) or E(S –) reduces to 14 n(n + 1) and the var(S+) or var(S–) to n(n + 1)(2n + 1)/24. The formula in this case can be improved by a continuity correction that allows for the fact that we are approximating to a continuous distribution with a discrete distribution where S can take only integer values. If S is the smaller of the sums of the positive or negative ranks the appropriate correction is to replace S by S + 12 , while if S is the larger sum it is replaced by S – 12 . When there are tied ranks it is easiest to revert to the general formula (2.1) using the tied ranks as scores. If the statistic SW = |S+ – S –| is used it is easily verified that now (2.1) reduces to Z = SW/√[Σ(si2)] and if there are no ties the denominator reduces to √[n(n+1)(2n+1)/6]. For the Wilcoxon test the asymptotic approximation is usually satisfactory for n ≥ 20, and sometimes for smaller values of n also. For the sign test where S is the number of observations greater than (or less than) the hypothesized median it is easily verified that E(S) = 21 n and var(S) = 41 n whence (2.1) reduces to Z=
X − 12 n . 1 2 n
(2.2)
Again the approximation is improved by a continuity correction replacing S by S + 21 if S is less than its mean and by S – 21 otherwise. Programs giving asymptotic results for many standard nonparametric tests are included in most major statistical software packages and as options in specialist programs like StatXact and Testimate. They work well for large samples but the approximations may or may not be good for medium or small samples. In Examples 2.13 and 2.14 we consider some uses of asymptotic results in testing and estimation, comparing them where possible with exact results. Manual computation is possible but tedious if software is not available to carry out asymptotic tests automatically.
©2001 CRC Press LLC
Example 2.13 The problem. For the data sets on water content of soil used in Example 2.9, viz. 5.5
6.0
6.5
7.6
7.6
7.7
8.0
6.3
6.3
6.5 6.6 7.0 7.5 7.9 8.4 8.5 8.7 9.4 14.3
8.2
9.1
15.1
8.0 8.0 26.0
8.1
and 5.6
6.1
8.1
8.2
explore the use of asymptotic results for the Pitman, Wilcoxon and sign tests for testing the hypothesis that the median content is θ = 9 against a two-tail alternative. Compare results with those for exact tests where possible. Formulation and assumptions. Any assumptions of symmetry implicit for relevant exact tests also apply to asymptotic tests. Generally increasing sample size will bring the asymptotic result closer to the exact result. Procedure. Although manual computation of Z is feasible using (2.1) or (2.2) as appropriate, we assume relevant statistical software is available. For testing H0: θ = 9 against H1: θ ≠ 9 StatXact gives the following exact and asymptotic P-values: Test
Sample of 10 Exact Asymptotic
Sample of 20 Exact Asymptotic
Pitman Wilcoxon Sign
0.352 0.102 0.109
0.916 0.015 0.0026
0.305 0.092 0.057
0.817 0.017 0.0017
StatXact does not use the continuity correction for the sign test. That has quite a large effect in these examples. For instance for the sample of 20 its use increases the two-tail asymptotic P to 0.0037, but this is not closer to the exact value, whereas surprisingly for the smaller sample it does better, increasing P to 0.114 which is closer to the exact value. This indicates the need for caution about using asymptotic values even for moderate n. It is particularly in the tails (i.e. for low values of P) that asymptotic values sometimes show the greatest relative departure from exact probabilities. When there are ties a continuity correction is not usually used for the Wilcoxon test as it has little theoretical justification, although in practice it sometimes tends to bring results closer to those for the exact permutation test. The t-test P = 0.33 and 0.82 are close to the values for the Pitman test. Conclusion. The data in both samples are, as already pointed out, slightly skewed to the right, especially in the sample of 20. Both the Pitman and Wilcoxon tests indicate lower efficiency relative to the sign test in the larger sample, but the effect of skewness is less for the Wilcoxon test than it is for the Pitman test because the transformation to ranks does not give the more extreme values of 14.3, 15.1 and 26.0 the influence they have in the Pitman test. Indeed they would have had the same rankings had they had the values 12.6, 12.7 and 12.8.
©2001 CRC Press LLC
Comment. In view of the conclusion that our rankings would not have been altered if the three largest observations had been replaced by 12.6, 12.7 and 12.8 when there would be only a small indication of skewness, it is interesting to see how the Pitman test would perform if these values are used in it. Intuition suggests that the P values should then be nearer those given by the Wilcoxon test (which, of course, are unaltered by the change). Indeed with this change the exact P-value for the Pitman test reduces to P = 0.1230 for the smaller sample and to P = 0.0355 for the larger, both closer to the results for the Wilcoxon test. This should cause no surprise because there is now little indication of skewness and so a key validity requirement for both tests is nearer to being satisfied. Computational aspects. The wide availability of software to carry out asymptotic tests makes them appealing. Three notes of caution are: (i) Large sample sizes are needed to ensure consistently good approximations, especially in the tails, so beware of asymptotic tests when n < 20. (ii) A breakdown of an assumption such as a requirement for symmetry may make asymptotic results of little value (as is also true for exact tests). (iii) Practice varies regarding use of continuity corrections; it is usually easy to check whether or not these are used either from documentation included with software or from the given output which often includes the test statistic value together with its mean and standard deviation.
Asymptotic confidence intervals are obtainable using the form of (2.1) relevant to the Wilcoxon test without ties or for the sign test using (2.2). For the Wilcoxon test the asymptotic result enables us to obtain an approximate value of S for significance at a suitable level and Walsh differences may then be used to get the approximate confidence interval. This works quite well if there are no ties and it gives a starting point for trial-and-error refinement when there are many ties. For the sign test we can estimate the number of positive or negative signs that determine the end points of our interval. An example shows how this works. Example 2.14 The problem. For the larger data set in Example 2.13 use the asymptotic formulae for the Wilcoxon test and that for the sign test to establish approximate 95 per cent confidence intervals for the population median. Formulation and assumptions. The key to establishing end points of 95 per cent confidence intervals is that if the statistic used for computing P-values for some hypothetical value of the median θ is such that |Z| = 1.96 then that θ is the end point of the approximate confidence interval. Procedure. Consider first the Wilcoxon test. For the sample of 20 observations in Example 2.13 either direct computation or use of statistical software for the asymptotic test establishes that the statistic S has mean 105 and standard deviation 26.78. Thus, if for some hypothesized mean a calculated S is such that (S – 105)/26.78 = ± 1.96 ©2001 CRC Press LLC
it follows that such S are appropriate values for calculating the end points of the confidence interval using Walsh averages. Solutions are S = 52.82 and 157.18. For practical purposes we round these to 53 and 157. In fact we only need calculate one of these in view of the symmetry in counts of positive and negative ranks; the smaller value 53 indicates that we should reject the 53 largest and 53 smallest Walsh averages. If no program is available to compute these averages it is only a matter of tedious arithmetic computing the more extreme averages to establish that the approximate interval is (7.15, 8.45). For the sign test it follows immediately from (2.2) that when n = 20 the mean value of the test statistic (the number of positive signs or of negative signs) is 10 and the standard deviation is 12 √20 = 2.236 whence the lower limit is the value S such that (S – 10)/2.236 = –1.96, i.e. S = 5.62. Rounding to the nearest integer suggests that we should reject H0 if there are 6 or fewer positive or negative signs leading for these data to the approximate interval (7, 8.2). Conclusion. The asymptotic results give approximate 95 per cent confidence intervals based on the Wilcoxon statistic as (7.15, 8.45) and based on the sign test statistic as (7, 8.2). Comments. 1. The 95 per cent confidence interval based on the t-test statistic is (6.7, 10.9); this is considerably longer than those obtained in this example and reflects the inappropriateness of the t-test procedure when data are clearly skew. Despite the fact that we found in Examples 2.3 and 2.8 for a different data set that the Wilcoxon based intervals may be similar to the t-test based intervals in the presence of skewness, its performance here is better. In practice how well the Wilcoxon method performs in estimation problems depends upon the number of outlying observations in the sample as these influence only the more extreme Walsh averages. Only a few outliers when n is large are less influential than the same number of outliers for smaller n. 2. Given programs for exact tests such as that in StatXact it is easy to determine the exact coverage of the intervals and also to adjust the limits by a trial and error method to give better coverage. In this example the Wilcoxon interval (7.15, 8.45) is associated with a lower-tail P = 0.0235 and an upper-tail P = 0.030. Although one is below and the other above 0.025 the interval is reasonable for most practical purposes. For the sign-test interval with these end points some observations coincide with the limits so it is useful to perform an exact test for P-values taking values just above and below these end points. Using StatXact we find the interval (6.95, 8.25) has only an 88.46 per cent coverage but the interval (7.05, 8.15) has an even lower 73.7 per cent coverage. Again there is an indication here that one needs to be cautious about asymptotic results. Use of a continuity correction in determining the asymptotic limits may help. In Example 2.10 we established the 95 per cent interval based on the sign test was (6.6, 8.4). Computational aspects. Although programs giving exact P-values or good Monte Carlo approximations thereto are needed to refine asymptotic limits, limits based on asymptotic theory are usually reasonable for moderately large samples and can be computed using any program that deals adequately with the relevant asymptotic nonparametric tests. Some programs (e.g. Minitab) quickly ©2001 CRC Press LLC
generate tables of Walsh averages, so if the Wilcoxon intervals are required once S corresponding to Z = ± 1.96 has been computed it is easy to get the Walsh averages corresponding to end points, although Minitab will itself give asymptotic limits for the Wilcoxon test and exact limits for the sign test.
2.6 ROBUSTNESS In Example 2.14 we found an approximate 95 per cent confidence interval for the population mean using the Wilcoxon signed-rank procedure was (7.15, 8.45) while that based on the sign test was displaced slightly to the left and was slightly shorter being (7, 8.2). However, the confidence levels were not quite identical. The interval (6.7, 10.9) based on the normal theory t-test is appreciably longer, reflecting the inappropriateness of the t-test approach when applied to obviously skew data. On the other hand one commonly meets situations where the sign-test interval is longer than that given by the Wilcoxon test which in turn is longer than that based on the ttest. In such examples although there might be some evidence of departure from normality, if the departure is not severe and there is not much evidence of skewness these results are in general agreement with the finding based on Pitman efficiency that when a sample comes from a normal distribution the efficiency of the t-test is greater than that of any other test. The Pitman efficiency of the Wilcoxon test can be shown to be 3/π (approximately 95.5 per cent) while that of the sign test is 2/π (approximately 63.7 per cent) when a sample is from a normal distribution. Two kinds of departure from normality that are common are one where the sample may come from some distribution such as the exponential or gamma distribution which are known to be asymmetric, or one where the sample comes from a mixture of distributions. Mixtures may arise in several ways. For example our data might be results of chemical analyses carried out in two different laboratories, one of these giving less precise results than the other and perhaps also incorporating a consistent error (called bias) due to an equipment fault. Mixtures may also occur in accurate data. We explained why this was so for the data in Example 2.10 (Comment 3). Another way skew or longtail distributions may occur is by the intrusion of rogue observations often called outliers. If we know why they are rogues there may be a good case for rejection, but often we do not know if suspect observations are really rogues (e.g. incorrectly recorded readings, observations on units that are not members of the population of interest, etc.) or if we are sampling from a population where a small
©2001 CRC Press LLC
proportion of outlying values is the norm. While many parametric tests like the t-test are sensitive to certain departures from assumptions, the Wilcoxon test may be less so because it makes fewer assumptions, and the sign-test even less sensitive because it in turn makes even fewer assumptions. These latter tests exhibit a property called robustness which we discuss more fully in Chapter 11. Test and estimation procedures are robust if they are little influenced by fairly blatant departures from assumptions. In Example 2.15 we indicate that the Wilcoxon test may be more robust than the t-test if there is a marked departure from symmetry. We also see that in such circumstances the sign test, not unexpectedly, may do even better since it makes no symmetry assumption.
Example 2.15 Suppose that the data in Example 2.4, i.e. –5 –2 1 3 4 8 9 12 16 17 20 25 are amended by omitting the observation 25 and replacing it (i) by 35 and (ii) by 65. These changes induce obvious upper-tail skewness. Table 2.5 gives the nominal 95 per cent and 99 per cent confidence intervals based on the normal theory t, the Wilcoxon signed-rank statistic and the sign test in each case. For the moderately skewed set (i) we have a mixed picture, the t-test does best at the 95 per cent level and the Wilcoxon test at the 99 per cent level but the other methods do nearly as well in each case. For the more markedly skewed set (ii), the t limits are highly unsatisfactory, and those based on Wilcoxon are moderately satisfactory at the 95 per cent level but not satisfactory at the 99 per cent level. However, the binomial-based sign-test limits are the same as those for set (i) and are not influenced by the extreme observation at 65. For set (ii) they are the same width as the Wilcoxon limits at the 95 per cent level although the exact coverage (96.14 per cent) is slightly greater than the exact Wilcoxon coverage (95.5 per cent) but this is more a reflection of the discontinuities in possible available levels than anything else. At the 99 per cent level the sign test limits are the clear winner. This is not surprising since no symmetry assumption is needed. Are such extreme values as that in data sets (i) and (ii) realistic from a medical point of view? A change in heart rate of 35 beats per minute might just be possible but the value of 65 in set (ii) is a data error. It is good statistical practice to check data carefully to eliminate errors, but one often has to analyze data where there is no indication that an extreme value (often referred to as an outlier) is not a correct observation. This is when one may do better with the simple sign test than with a test requiring more stringent assumptions. On the other hand it is generally inappropriate to use the sign test when there is little or no evidence of skewness. ©2001 CRC Press LLC
Table 2.5 95 and 99 per cent confidence intervals for data sets (i) and (ii) in Example 2.15 based on normal theory (t), Wilcoxon signed ranks (W) and the binomial sign test (B). For each set the shortest interval at each confidence level is indicated by an asterisk. _____________________________________________________ 95 per cent 99 per cent t (2.79, 16.88) * (–0.12, 19.79) Data set (i) W (2.5, 17.0) (0.5, 20.0)* B (1.0, 17.0) (–2.0, 20.0)
Data set (ii)
t W B
(0.70, 23.97) (2.5, 18.5)* (1.0, 17.0)*
(–4.10, 28.77) (0.5, 34.5) (–2.0, 20.0)*
2.7 FIELDS OF APPLICATION Insurance The median of all motor policy claims paid by a company in 1999 is £870. Early in 2000 the management thinks claims are higher. To test this, and to estimate the likely rise in mean or median, a random sample of 25 claims is taken. The distribution of claims will almost certainly be skew, so a sign test would be appropriate. Medicine The median systolic blood pressure of a group of boys prior to physical training is known. If the blood pressure is taken for a sample after exercise the sign test could be used to test for a shift in median. Would you consider a one- or a two-tail test appropriate? If it appears reasonable to assume a symmetric distribution a Wilcoxon test or even a normal theory t-test is more appropriate. Even if an assumption of symmetry in systolic blood pressures before exercise is reasonable, this may not be so after exercise. For instance, the increase after exercise might be relatively higher for those above the median blood pressure at rest and this could give rise to skewness. A physiologist asking questions about changes should know if this is likely. Engineering Median noise level under the flight path to an airport might be known for aircraft with a certain engine type (the actual level will
©2001 CRC Press LLC
vary from plane to plane and from day to day depending on weather factors, the precise height each plane flies over the measuring point, etc.). If the engine design is modified a sample of measurements under similar conditions to that for the old engine may indicate a noise reduction. A one-tail test would be appropriate if it were clear the modification could not increase noise. We are unlikely to be able to use a true random sample here, but if taken over a wide range of weather conditions, the first, say, 40 approaches using the new engine may broadly reflect characteristics of a random sample. Biology Heartbeat rates for female monkeys of one species in locality A may have a symmetric distribution with known mean. Given heartbeat rates for a sample of similar females from locality B, the Wilcoxon test could be used to detect a shift or to obtain confidence limits for the true mean. Education A widely used test of numerical skills for 12-year-old boys gives a median mark of 83. A new method of teaching such skills is used for a class of 27 such pupils. The asymptotic approximation to the Wilcoxon test could be used to test for a shift in median if symmetry could be assumed. If not, a sign test would be preferred. Management Records give the mean and median number of days absent from work for all employees in a large factory for 1998. The number of days absent for a random sample of 20 is noted in 1999. Do you think such data are likely to be symmetric? In the light of the answer to this question one may select the appropriate test for indications of changes in the absentee pattern. Geography and environment Estimates of the amount of cloud cover at the site of a proposed airport are taken at a fixed time each day over a period. The site might be rated unsuitable if the median cover were too high. A confidence interval for the true median would be useful. Local differences A psychologist is told that the ‘national average IQ of drug abusers aged between 16 and 18 is 103’. He assesses the IQ of a sample of ©2001 CRC Press LLC
abusers in that age group from an area where drug abuse is rife. He might use a Wilcoxon test to assess whether it is reasonable to assume the mean is 103 for that area. It is not unusual in a number of contexts to find that national averages applicable to a large area do not apply locally. In the UK, for instance, the average price per litre of petrol in London is very different from that in the Scottish Highlands and both (especially the latter) differ from the nationwide average. The average price in euros of a litre of milk in Italy, France and Germany may each differ from the average for the whole European community. Industry The median time people stay in jobs in a large motor assembly plant in Germany is known to be 5.2 years. Employment times for all 55 employees who have left a UK plant in the last 3 months are available. Although this may not be a random sample it may be reasonable to use the data to test whether the median time for UK workers is also 5.2 years. If the distribution appears skew a sign test would be appropriate. Combining information from different sources Tests are often carried out in more than one laboratory to determine, for example, the mean or median level of some impurity in a product. If it is reasonable to consider that the population means are equal for all the material sent for analysis and the laboratory means are centred about this value even if the distributions of the test results differ in other respects an appropriate nonparametric test of the combined data may be used depending upon whether assumptions of symmetry are or are not appropriate. Astronomy Astronomers frequently estimate quantities such as the mass of a particular star or the diameter of a planet in several observatories each using non-identical equipment. The resulting measurements are often far from normally distributed, forming either a longer tailed or skewed distribution of readings. Providing reasonable assumptions can be made about the nature of measurement errors nonparametric methods of analysis may often be appropriate. Some comments on this situation were made in Section 2.1.2.
©2001 CRC Press LLC
2.8 SUMMARY Pitman test. A suitable test statistic (Section 2.1.2) is the sum S+ of the positive data deviations from a hypothesized mean or median θ assuming the sample is from a symmetric distribution. Appropriate computer software is required for significance tests and even with such software confidence limits can only be obtained by a cumbersome trial-and-error approach. The procedure lacks robustness and often gives similar results to a t-test even when the latter is not appropriate. It is conditional upon each particular data set and is not widely used in practice. Wilcoxon signed-rank test. The test statistic S+, the sum of the positive signed-rank differences from the mean or median specified in H0 (Section 2.2.1), is widely used. Tables, or more satisfactorily, appropriate computer software may be used for significance tests. Ties require special treatment (Section 2.2.3) and suitable computer software is then needed to determine exact permutation distributions. Walsh averages (Section 2.2.4) provide an alternative hypothesis testing procedure and they are particularly useful for estimation and calculating confidence intervals. An assumption of population symmetry is needed for validity of test and estimation procedures. Asymptotic normal approximations (Section 2.5) work well for sample sizes n ≥ 20 if there are no (or very few) ties; modification of the denominator in the test statistic is required for numerous ties even for relatively large n. Sign test. The test statistic is the number of observations above or below a median specified in H0 (see e.g. Example 2.9). Significance is determined using binomial probability tables for p = 12 and various n or else from a suitable computer program. The method is easily used to obtain confidence intervals (Section 2.3.2). Normal theory approximations (Section 2.5) may be used for sample sizes greater than 20. Modifications are required to deal with values equal to the median specified in H 0 (Section 2.3.1). No assumption of symmetry is required. Normal scores. These scores (Section 2.4) aim to make data more like samples from a normal distribution. Test results are usually not very different from those given by the Wilcoxon test. There appear to be no simple satisfactory methods for obtaining confidence intervals using these transformations. ©2001 CRC Press LLC
EXERCISES 2.1 2.2
2.3
2.4
2.5 2.6
2.7
For the data in Example 2.1 carry out a test of H0: θ = 80 against H1: θ ≠ 80 using the Pitman test. In Comment 3 on Example 2.1 we asserted that the usual t-statistic could be used in place of S+ as the test statistic for the Pitman test because there was a one-to-one correspondence between the ordering of the two statistics. Establish that this is so. [Hint: show that the denominator of the t-statistic is invariant under all permutations of the signs of the di.] In Comment 3 on Example 2.4 we suggested that for a variety of reasons one should be cautious about extending inferences about heartbeat rates for female students to the population at large. What might some of these reasons be? Using the data in Table 2.1 for the distribution of the Wilcoxon S when n = 7, construct a bar chart like that in Figure 2.1 showing the probability function for S. Discuss the similarity, or lack of similarity, to a normal distribution probability density function. Verify the P-value quoted in Comment 3 in Example 2.6 for the data modified in the way suggested in that comment. Establish that the permutation distribution of the Wilcoxon signed-rank statistic for testing the hypothesis H0: θ = 6, given the observations 4, 4, 8, 8, 8, 8, 8 has a distribution equivalent to that for the sign test of the same hypothesis. Would this equivalence hold if the null hypothesis was changed to H0: θ = 7? Establish nominal 95 per cent confidence intervals for the median based on the Wilcoxon signed-rank test for the following data sets. If an appropriate computer program is available use it to comment on the discontinuities at the end points of your estimated intervals based on Walsh averages. Set I Set II
2.8 2.9
1, 1, 1, 1, 1, 3, 3, 5, 5, 7, 7 1, 2, 2, 4, 4, 4, 4, 5, 5, 5, 7
Form a table of Walsh averages for the Fisher sentence length data given in Example 2.8. and use it to confirm the confidence interval obtained in that example and also to obtain an approximate 99 per cent confidence interval. The numbers of pages in the sample of 12 books referred to in Exercise 1.2 were 126 142 156 228 245 246 370 419 433 454 478 503
Use the Wilcoxon signed-rank test to test the hypothesis that the mean number of pages in the statistics books in the library from which the sample was taken is 400. Obtain a 95 per cent confidence interval for the mean number of pages based on the Wilcoxon test and compare it with the interval obtained on an assumption of normality. 2.10 Apply the sign test to the data in Example 2.4 for the hypotheses considered there. 2.11 For the sample of 20 in Example 2.9 if θ is the population median test the hypothesis H0: θ = 9 against the alternative H1: θ ≠ 9 using the sign test by
©2001 CRC Press LLC
computing any relevant binomial probabilities directly from the binomial probability formula. Also perform the test for H0: θ = 7.5 against the alternative H1: θ > 7.5. 2.12 Before treatment with a new drug 11 people with sleep problems have a median sleeping time of 2 hours per night. A drug is administered and it is known that if it has an effect it will increase sleeping time, but some doctors doubt if it will have any effect. Are their doubts justified if the hours per night slept by these individuals after taking the drug are: 3.1 1.8 2.7 2.4 2.9 0.2 3.7 5.1 8.3 2.1 2.4 2.13
Assuming symmetry, carry out a test relevant to the situation and data in Exercise 2.12 using a normal scores procedure. 2.14 Kimura and Chikuni (1987) give data for lengths of Greenland turbot of various ages sampled from commercial catches in the Bering Sea as aged and measured by the Northwest and Alaska Fisheries Center. For 12-year-old turbot the numbers of each length were: Length (cm) 64 65 66 67 68 69 70 71 72 73 75 77 78 83 No. of fish 1 2 1 1 4 3 4 5 3 3 1 6 1 1
2.15 2.16
2.17 2.18
Would you agree with someone who asserted that, on this evidence, the median length of 12-year-old Greenland turbot was almost certainly between 69 and 72 cm? Use (i) the Wilcoxon signed-rank test and (ii) modified van der Waerden scores to test the hypothesis that the median length of 12-year-old turbots is 73.5 using the data in Exercise 2.14. Establish the values given in Section 2.5 for the mean and variance of the statistic S relevant to certain one-sample location tests, considering in particular the simplifications for the no-tie Wilcoxon test and for the sign test. Determine a P = 0.01 two-tail test critical region for the Wilcoxon statistic S+ when n = 12 based on the asymptotic result using equation (2.1). The first application listed in Section 2.7 involved insurance claims. The 1999 median was £870. A random sample of 14 claims from a large batch received in the first quarter of 2000 were for the following amounts (in £): 475 483 627 881 892 924 1077 1224 1783 1942 2013 2719 4650 6915
What test do you consider appropriate for a shift in median relative to the 1999 median? Would a one-tail test be appropriate? Obtain a 95 per cent confidence interval for the median based upon these data. 2.19 The weight losses in kilograms for 16 overweight women who have been on a diet for 2 months are as follows: 4 6 3 1 2 5 4 0 3 6 3 1 7 2 5 6 The firm sponsoring the diet advertises ‘Lose 5 kg in 2 months’. In a consumer affairs radio programme they claim this is an ‘average’ weight loss. You may be unclear about what is meant by ‘average’, but assuming ©2001 CRC Press LLC
the sample is effectively random do the data support a median weight loss of 5 kg in the population of dieters? Test this without an assumption of symmetry. What would be a more appropriate test with an assumption of symmetry? Carry out this latter test. 2.20 A pathologist counts the numbers of diseased plants in randomly selected areas each 1 metre square on a large field. For 40 such areas the numbers of diseased plants are: ______________________________________________ 21 18 42 29 81 12 94 117 88 210 44 39 11 83 42 94 2 11 33 91 141 48 12 50 61 35 111 73 5 44 6 11 35 91 147 83 91 48 22 17 ______________________________________________ Use an appropriate nonparametric test to find whether it is reasonable to assume the median number of diseased plants per square metre might be 50 (i) without assuming population symmetry, (ii) assuming population symmetry. For these data do you consider the latter assumption reasonable? 2.21 A traffic warden notes the time cars have been illegally parked after their metered time has expired. For 16 offending cars he records the time in minutes as: 10 42 29 11 63 145 11 8 23 17 5 20 15 36 32 15 Obtain an appropriate 95 per cent confidence interval for the median overstay time of offenders prior to detection. What assumptions were you making to justify using the method you did? To what population do you think the confidence interval you obtained might apply? 2.22 Knapp (1982) gives the percentage of births on each day of the year averaged over 28 years for Monroe County, New York State. Ignoring leap years (which make little difference), the median percentage of births per day is 0.2746. Not surprisingly, this is close to the expected percentage on the assumption that births are equally likely to be on any day, that is, 100/365 = 0.274. We give below the average percentage for each day in the month of September. If births are equally likely to be on any day of the year this should resemble a random sample from a population with median 0.2746. Do these data confirm this? 0.277 0.277 0.295 0.286 0.271 0.265 0.274 0.274 0.278 0.290 0.295 0.276 0.273 0.289 0.308 0.301 0.302 0.293 0.296 0.288 0.305 0.283 0.309 0.299 0.287 0.309 0.294 0.288 0.298 0.289
©2001 CRC Press LLC
3 Other single-sample inferences
3.1 INFERENCES FOR DICHOTOMOUS DATA 3.1.1 Binomially distributed data The binomial distribution is relevant to certain counts associated with only two possible outcomes —often referred to as dichotomous data. These counts may be the raw or primary data as in Example 1.3, where the count was the number of patients in a sample of 10 requiring treatment. In that example we used binomial distribution properties to assess the strength of evidence against a hypothesis that the sample came from a B(10, 3/4) distribution, i.e. to test the hypothesis H0: p = 3/4 against H1: p > 3/4. In Example 1.2 we applied the sign test to hypotheses about the median of measured survival times (the primary data), basing the test not on these primary data but on the derived or secondary data of numbers of observations above or below a hypothesized median. For the sign test the parameter p = 1/2 is relevant to H 0, and although the test was based on that parameter, we were interested not in the parameter itself but in the median of an underlying continuous distribution of survival times. In this section we give other examples of inferences about a binomial parameter p. It is convenient to refer to each of the n observations as the outcome of a trial. The key assumptions needed for a count to have a binomial distribution are : • the n trials are mutually independent; • only one of two possible outcomes A, B is observed at each trial; • at each trial there is a fixed probability p associated with the outcome A and a probability q = 1 — p associated with the outcome B. The outcomes, or events A, B are often labelled success and failure, but for reasons explained in Section 1.1.1 this may be inappropriate, so we designate the outcomes as event A and event B. ©2001 CRC Press LLC
The requirement that p be constant is often violated in a strict sense. For example, when UK children are skin-pricked for sensitization to grass, dust mites and cats around 7 per cent show a skin reaction to all three allergens. This does not imply that there is a probability p = 0.07 that a child given the skin tests will react to all three tests positively. For instance, for children with asthma the probability of three positive reactions is around 0.14, whereas for children without asthma it is about 0.05. However, if a random sample is taken from a large group of UK children which is representative of the population in terms of asthma, it is reasonable to expect approximately 7 per cent of those in the random sample to have reactions to all three allergens. For a similar sample of children from a country with a desert climate coupled with few domestic cats there is medical evidence to suggest that the proportion of children reacting to the grass and cat allergens may be lower. An appropriate statistical test could be based on the binomial distribution corresponding to a sample of n with the null hypothesis H0: p = 0.07 against the alternative H 1: p < 0.07. However, in a comment on Example 3.5 we point out another complication that may invalidate the test. Tests based on the binomial distribution also arise in quality control problems where we are interested in the proportion of items in large batches having a specific attribute — often a defect. A widely used procedure is to take a random sample of n items from a batch of N items and for a buyer to accept the whole batch if the number of defective items, r, in that sample does not exceed some small fixed number s. Otherwise the batch is rejected. The consequences of such a procedure can be studied using binomial distribution theory. The results are only approximate because the usual sampling method of selecting items without replacing each before we select later items (called sampling without replacement) does not lead to a binomial distribution for r. However, if N is very much larger than n, as it often is in practice, the binomial approximation is satisfactory. The scheme we outline below is usually embellished in practice, but we do not consider here the many possible elaborations. Example 3.1 The problem. A contract for supplying rechargeable batteries specifies that not more than 10 per cent should need recharging before 100 hours of use. To test compliance with this requirement a potential purchaser takes a random sample of 20 from a large consignment and finds that 3 need recharging after ©2001 CRC Press LLC
being used for less than 100 hours. Assess the strength of evidence that the batch is below specification. Formulation and assumptions. The test is distribution-free in the sense that we make no assumption about the distribution of times before a recharge is needed. However, we assume that there is a fixed probability p, say, that any one battery will need recharging in less than 100 hours and that the failure time for each battery is independent of that of any other battery. The null hypothesis is H0: p ≤ 0.10 and the alternative is H1: p > 0.10. Procedure. Tables for the binomial distribution or standard statistical software indicate that when p = 0.10 the probability that a sample of 20 contains 3 or more batteries that fail to meet specifications is P = 0.323. Conclusion. Since the probability that 3 or more batteries will need recharging in less than 100 hours when p = 0.10 is nearly one-third there is little evidence for preferring the alternative that the failure rate in the large consignment exceeds 10 per cent. Comments. 1. This example does little more than highlight serious limitations of hypothesis testing. Three failures in 20 items represents a 15 per cent failure rate in the sample, so the data would clearly support a null hypothesis that specified p = 0.15 and also many higher values. Confidence intervals for p will be informative. We show in Example 3.2 that a two-sided confidence interval with at least 95 per cent coverage for the true p is the interval (0.032, 0.379). This implies that we would only reject a hypothesized value of p falling outside that interval when we use the conventional P ≤ 0.05 as sufficient evidence against that hypothesis, i.e. the sample evidence here suggests only that the batch percentage failing to live up to specification is likely to be between about 3.2 and 37.9 per cent. 2. Having obtained a confidence interval, two further questions of interest are (i) what is the power of a test for H0:p = 0.1 against various alternatives and (ii) how may we use larger samples to make a test more precise (in the sense of shortening the confidence interval or increasing the power for specific alternatives)? We address these questions in Examples 3.3 and 3.4. Computational aspects. In Examples 3.1–3.4 we discuss computational aspects under Procedure as they form an integral part of that.
3.1.2 Confidence intervals for binomial proportions For a binomial distribution, if an event A occurs in r out of n trials the sample estimate of the binomial probability, p, is p^ = r/n. We use the notation X = r where r = 0, 1, 2, . . . , n to indicate that the event A occurs in r among n trials. To find a two-sided confidence interval for p with at least 95 per cent coverage we seek for its upper limit a value pu such that if the binomial parameters are n, p u, then the value of Pr(X ≤ r) is as close as possible to, but does not exceed, 0.025. Well-known binomial distribution theory indicates that pu must satisfy the equation ©2001 CRC Press LLC
Σ i (ni)(pu)i (1 – pu )n – i = 0.025
(3.1)
where the summation is over i from 0 to r. A similar approach provides an equation for a lower limit, p l. These equations may be modified to give any required level by replacing 0.025 by another appropriate value, e.g. 0.005 for a 99 per cent interval. The equation (3.1) generally has no simple analytic solution for pU. The classic way of overcoming this difficulty was recourse to tables or charts obtained by numerical methods where these limits were presented for several confidence levels (usually 90, 95 and 99 per cent) for a range of values of n and r. Table A4 in Conover (1999) is one such table. Many computer packages including Minitab and StatXact compute exact confidence limits at any level chosen by the user. For reasonably large n if neither exact tables nor relevant computer programs are available an asymptotic approximation may be used. If we observe X = r occurrences of event A in n trials for any p such that 0 < p < 1, then Z=
p − pˆ pˆ (1 − pˆ ) / n
has for large n approximately a standard normal distribution. The approximation is good for n > 20 if p is close to 0.50, but appreciably larger values of n are needed if p is close to 0 or 1. To obtain an approximate 95 per cent interval for p we follow the usual normal distribution confidence interval procedure and find the upper and lower limits as the value of p that satisfy the equations Z=
p − pˆ = ±1.96 pˆ (1 − pˆ ) / n
(3.2)
To obtain limits for other confidence levels 1.96 is replaced by the appropriate value, e.g. 1.64 if a 90 per cent interval is required. If the event does not occur at all (r = 0) then the estimate of the binomial probability is zero. An upper limit for p can be readily obtained (using a tail probability of 0.05 as the lower tail is not meaningful) and equation (3.1) reduces to (1 – pu )n = 0.05 Correspondingly, if r = n a lower limit can be obtained by solving (pl )n = 0.05.
©2001 CRC Press LLC
Sackett et al. (1991) tabulate p l and p u when r = 0 and r = n for a selection of sample sizes between 1 and 300. Example 3.2 The problem. For the data in Example 3.1 confirm that an exact 95 per cent confidence interval for p is (0.032, 0.379). Determine also 90 and 99 per cent intervals. Obtain an asymptotic approximation to the 95 per cent interval. Formulation and assumptions. Assuming that the number of batteries in a sample of 20 that fail to last 100 hours before needing a recharge has a binomial distribution, the exact 95 per cent confidence limits are given by equation (3.1) and the corresponding equation for the lower limit. For the 90 per cent and 99 per cent limits the 0.025 on the right-hand side is replaced by 0.05 and 0.005 respectively. Because the equations have no analytical solution the required limits are obtained in practice from tables or by using appropriate software. The relevant asymptotic limits are given by (3.2). Procedure. From Conover (1999) Table A4 one finds the following exact intervals for this problem where n = 20 and r = 3. 90 per cent (0.042, 0.344) 95 per cent (0.032, 0.379) 99 per cent (0.018, 0.450) Both Minitab and StatXact give the same values and these should also be obtained from any other software correctly claiming to give exact intervals. For the asymptotic limits we use (3.2) with n = 20 and ^p = 3/20 = 0.15 and the solutions give the 95 per cent interval (–0.0065, 0.306). Most computer software will give these limits as an alternative to exact limits, but not all include the timely warning in Minitab that these may be unreliable for small samples. Conclusion. Evidence from this sample of 20 indicates only that it seems likely that the overall proportion in a large batch lies somewhere between about 3 – 4 per cent and about 35 – 40 per cent. Comments. 1. We have already indicated that the asymptotic approximation requires n to be considerably greater than 20 for small p. The asymptotic 95 per cent lower limit, –0.0065 is a statistical nonsense, being outside the permissible range 0 ≤ p ≤ 1. Tables of exact values in Conover cover values of n ≤ 30 but with small values of p the asymptotic result may still be unreliable even when n > 30, meaning that unless one has access to a table for larger values of n one needs statistical software for an exact interval. For the case n = 30 and r = 2 an exact 95 per cent interval for p is (0.008, 0.221) while the normal approximation gives the nonsense interval (–0.023, 0.156). However, if n = 30 and r = 13 the exact 95 per cent interval is (0.255, 0.626) and the asymptotic approximation is a reasonable (0.256, 0.611). 2. In this example we considered a two-sided confidence interval whereas in Example 3.1 we were interested in a one-tail hypothesis test. The lower limit of the one-sided 95 per cent confidence interval for the test considered there corresponds to the lower limit for the two-sided 90 per cent interval, i.e. the 95 per cent one-sided interval is in this example (0.042, 1). ©2001 CRC Press LLC
3.1.3 Power and sample size In Examples 3.1 and 3.2 there is considerable doubt about the population value of p. In practical quality control problems a common situation is that where a purchaser would ideally like all batches of items such as the rechargeable batteries that are accepted to contain not more than 10 per cent that require recharging before 100 hours of use. However, taking marketing realities such as production cost into account, the purchaser may be prepared to accept a few batches where as many as, say, 20 per cent fail to live up to this requirement, but be reluctant to accept batches where more than 20 per cent fail. By the same token the supplier does not want the purchaser to reject batches where less than 10 per cent will fail. If we test H 0: p ≤ 0.10 against the alternative H 1: p > 0.10 and agree the evidence is strong enough to reject H0 if the associated P-value is P ≤ 0.05 this means there is a probability not greater than 0.05 that the purchaser will reject a batch that would be acceptable (i.e. that in reality contains 10 per cent or less that are defective). Be careful in your reading to distinguish between the binomial probability (lowercase p) and the test P-value (capital P). In acceptance sampling terms P = 0.05 is called the producer’s risk because in agreeing to accept these terms it is the maximum long run probability that an acceptable batch will be rejected. In conventional hypothesis testing terms producer’s risk is the probability of an error of the first kind. If the consumer is prepared to accept a few batches with more than 10 but less than 20 per cent defective the consumer and producer might agree that they should devise a test procedure that ensures a certain preassigned high probability, say P* = 0.90 that one does not accept a batch with 20 per cent or more defective. This means there is only a probability 1 – P* = 0.10 that a batch with 20 per cent or more defective will slip through. This last probability is called the consumer’s risk because in agreeing to accept these terms it is the maximum probability that an unacceptable batch will be purchased. It is the probability of an error of the second kind, and P* is the power. A practical problem is to choose the sample size n such that there is both a preassigned small probability P that batches for which p ≤ 0.10 will be rejected and also a preassigned large probability P* that batches for which p ≥ 0.20 will also be rejected. Here the concept of power (Section 1.3) is useful. Power depends on the value of P chosen to indicate significance, i.e. as sufficient evidence to make H0 unacceptable, and also upon the sample size n
©2001 CRC Press LLC
and upon the value of the binomial p specified for an alternative hypothesis. Two relevant questions are: • Given a sample of size n what is the power of a test of H0 : p = 0.1 against H 1: p = 0.2 if we agree to reject H 0 for any P less than some fixed preassigned value in a one-tail test? • What sample size n will ensure that a test of H0: p = 0.1 against H1: p = 0.2 where we reject H 0 for any P less than some fixed value (e.g. P = 0.05) has a given power P* for that alternative? The rationale for these choices of H 0, H 1 is that if the true p is less than 0.1 we are less likely to reject a good batch than would be the case if p = 0.1 and that if p > 0.2 the power of our test will exceed that when p = 0.2. A moment’s reflection shows these are desirable characteristics for both producer and consumer. If we observe r occurrences of event A in a sample of size n and want to test H 0: p = p 0 against H 1: p > p 0 and agree that there is sufficient evidence to prefer H1 for all P-values less than or equal to some fixed value α, then the critical region of size α consists of all values of r ≥ r0 where r0 is the smallest value of r such that the binomial distribution B(n, p0) probabilities for all r ≥ r0 have a sum not exceeding α. In practice we either read r0 from tables or now more commonly obtain it using computer software. Having determined r0 it is easy to find the power of this test against a single-valued alternative H 1: p = p1 because this is simply the Pr(r ≥ r0) when H 1 is true. This probability may be read from tables or obtained using appropriate computer software. For large n asymptotic approximations may be used. These are based on the fact that under H0 the distribution of pˆ − p0 (3.3) Z= p0 (1 − p0 ) / n
is approximately standard normal. If, for example, α = 0.05 for an upper-tail test we set Z = 1.64 in (3.3) and solve for p^ given n and p0. To get an asymptotic expression for the power when H 1: p = p1 is true we replace p0 by p1 in (3.3) and insert the solution p^ that we have just obtained in this amended expression for Z and calculate the resulting Z, say Z = z1. The asymptotic approximation to the power is Pr(Z ≥ z1). Example 3.3 The problem. For a sample of 20 batteries from a large batch suppose we wish to test H0: p = 0.1 against H1: p = 0.2 where p is the population proportion
of batteries requiring recharging in less than 100 hours and decide we will prefer H1 for any P-value P ≤ 0.05. What is the minimum number of faulty batteries in the sample for which we prefer H1? Obtain both exact and asymptotic expressions for the power of this test. Formulation and assumptions. Let X be the number of occurrences, r, of the event A (battery needs recharging in less than 100 hours), then under H0, X has a binomial (20, 0.1) distribution. We prefer H1 if our sample gives r0 or more occurrences of A where r0 is the least r such that Pr (X ≥ r0) ≤ 0.05 when H0 is true. Once we know r0 the power is obtained by computing for that r0 the corresponding probability under H1. Asymptotic results for large n use the normal approximations given above for these probabilities. Procedure. If suitable tables are available these may be used although it is often easier to use software. We outline the procedure for both methods. When n = 20, p = 0.10, Table A3 in Conover (1999) indicates that Pr(X ≤ 4) = 0.9568 whence Pr(X ≥ 5) = 1 – 0.9568 = 0. 0432. Thus we prefer H1 if and only if r ≥ 5. To determine the power we require Pr(X ≥ 5) for a B(20, 0.2) distribution. The same tables indicate that now Pr(X ≥ 5) = 1 – Pr(X ≤ 4) = 1 – 0.6296 = 0.3704. In the asymptotic approach under H0 we have, using (3.3) Z = (p^ – 0.1) /√(0.1 × 0.9/20) = 1.64 whence p^ = 0.2100. Thus the relevant power is given by Pr[Z ≥ (0.2100 – 0.2)/√(0.2 × 0.8/20)] = Pr(Z ≥ 0.1118). From standard normal distribution tables this probability is 0.4555. Most computer software easily generates P-values corresponding to any number of successes for one- or two-tail tests. For example, the 1-proportion program in Minitab confirms that Pr(X ≥ 5) = 0.043 and the same program using p = 0.2 indicates Pr(X ≥ 5) = 0.370 confirming our result for the power. Minitab, version 12, also gives a separate program relating power to sample size but uses an asymptotic approximation. This gives an approximate power 0.432 which differs somewhat from our 0.4555. The discrepancy is explained by the estimate being sensitive to round off. If we replace 1.64 by the more precise 1.645 the estimated power is 0.434, close to the Minitab estimate. The discrepancy between the true power and the asymptotic estimate reflects the unsatisfactory nature of the asymptotic result for even moderate n associated with small p. StatXact, version 4 gives the exact power directly for any sample size for any pair of values of p specified in H0 and H1. Conclusion. The power of our test against an alternative p = 0.20 is only 0 . 37 0 if w e ch o os e P ≤ 0. 05 ( o r mo re p re ci se l y be ca us e of d i sc on ti nu it ie s P ≤ 0.043) as our criterion for not accepting p = 0.10. Comments. 1. In the discussion preceding this example we suggested a practical aim might be to reject a batch with probability (power) 0.90 if p ≥ 0.20. The power 0.370 is far short of this requirement. 2. The unsatisfactory nature of asymptotic power calculations with the n and p used here is less acute with larger samples. 3. In Section 1.3 we indicated that reducing the P-value chosen as acceptable evidence against H0 generally results in reduced power against a particular
©2001 CRC Press LLC
alternative. In this example tables or appropriate software indicates that if we set the requirement that P must not exceed 0.02 the power of the one-sided test against the alternative p = 0.2 is reduced from 0.370 to 0.196. However, due to discontinuities in possible P-values the exact P is not 0.02 but is 0.011.
Modern software yields exact P-values for one- or two-tail tests for any n, p and r which effectively means we can obtain the values of Pr(X ≥ r), 0 ≤ r ≤ n for any B(n, p) distribution. This is useful for developing tests such as those for attribute sampling considered in this section that have preset producer’s risk (i.e. P-values for rejection of H 0) and consumer’s risk (i.e. power against a specified H1 that gives the maximum p the consumer considers tolerable). Example 3.4 The problem. Determine the power of the test H0: p = 0.10 against H1: p = 0.20 when P ≤ 0.05 when n = 50, 100, 150, 200, 300. Compare exact values with the asymptotic approximation. Also determine the minimum n to ensure a power of at least 0.9 for this test. Formulation and assumptions. For illustrative purposes we use programs in Minitab, version 12 and in StatXact, version 4, but modifications for other packages with binomial distribution programs are generally straightforward. Procedure. We describe the procedure fully for the case n = 100 and only quote results for other n. We urge readers to confirm the results using available software. When n = 100 and p = 0.10 it is intuitively reasonable to expect a P-value not exceeding 0.05 if r, the number of batteries needing recharging before 100 hours, is close to 15. Minitab and StatXact both give the following values r P
15 0.073
16 0.040
17 0.021
establishing r = 16 as the critical value. To obtain the power we need to know Pr(X ≥ 16) when p = 0.20. StatXact gives this probability, the exact power, to be 0.871 directly without the need to first ascertain, as we just did, that r = 16 is the critical cut-off value. The sample size and power program for proportions in Minitab gives an asymptotic approximation which in this case indicates a power of 0.897 for a nominal P = 0.05. For our exact P-value 0.040 the asymptotic power approximation is 0.882. The asymptotic approximation is reasonable for this sample size. For n = 100 and the remaining sample sizes of interest with the largest exact P-values meeting the requirement P ≤ 0.05, the exact power and the asymptotic estimate of power based on the corresponding exact P are: n P
50 0.025
100 0.040
150 0.044
200 0.043
300 0.038
Exact power 0.556 Asymptotic power 0.617
0.872 0.882
0.963 0.962
0.989 0.988
0.999 0.999
How best to estimate the sample size n needed to ensure an exact power 0.9 may depend on available software. Clearly the size lies between 100 and 150. Since ©2001 CRC Press LLC
asymptotic power calculations seem reasonable for samples exceeding 100 one might use an asymptotically based facility in Minitab that gives an estimate of sample size having any required power for the relevant critical P-value (here P = 0.05). One may fine tune this using a program for exact binomial probabilities in a way we now describe. For our test the power-sample size program in Minitab indicates the required n = 102. We used this as a starting estimate for the exact result. When n = 100 we found the critical region was defined by r ≥ 16. When n = 102 the Minitab binomial program relevant to our one-tail test shows that when r = 16 the corresponding P-value is P = 0.047, while r = 15 gives P = 0.083. The exact power for the alternative p = 0.20 corresponding to P = 0.047 is given by Minitab as 0.890. In practice this may be accepted as a suitable approximation or one might seek some insurance by choosing a slightly greater sample size such as n = 105. With that latter choice Minitab gives the exact P = 0.032 with power 0.865! The apparent anomaly of reduced power with increased sample size is simply a reflection of the impact of discontinuities in Pvalues and possible values of the power (which remember is also a probability with associated discontinuities). Our nearest P-value above P = 0.032 would have been P = 0.058 and the associated power then becomes 0.914. StatXact also gives exact power for the above or other sample sizes directly. These indicate that a sample size of 109 has power 0.901 in the problem considered here. Conclusions. In general for a specified single-value alternative hypothesis power increases with sample size for a fixed P-value but there are some irregularities due to discontinuities in possible exact P-values. These irregularities are generally of little practical importance. Comment. Although one should be aware of the effects of discontinuities in P-values in assessing evidence for abandoning a hypothesis and in calculating power and sample sizes to achieve certain aims the importance of such effects should not be overemphasized. While it is clearly unwise, for example, to use asymptotic results for small samples, the discrepancies due to discontinuities in possible P-values are usually small in larger samples and in this context it is also worth remembering that nearly all mathematical models of real world situations are only approximations. Hopefully these will reflect key features of reality, but there will always be some discrepancies. For instance, we have already pointed out that when we take a sample of n from a large batch of N items by sampling without replacement the binomial distribution is only an approximation, albeit quite a good one.
3.1.4 A caution In Section 3.1.1 we considered a situation where, in the population at large, 7 per cent of all children skin-pricked for sensitization to grass, dust mites and cats had a positive skin reaction to all three allergens. However, the presence or absence of asthma meant that each child did not have the same probability of showing three positive skin tests. We indicated there that if a random sample were taken from that larger population it is not unreasonable to expect this to reflect the population proportion of 7 per cent showing three
©2001 CRC Press LLC
positive tests, because statistical theory backs an intuitive hunch that a random sample should reflect fairly closely characteristics of a population. On these grounds we indicated that it may be reasonable to take a random sample of children from a country with a desert climate and few domestic cats to see whether there is a lower proportion of positive skin tests in that population. We formalize a simple test based on the binomial assumption in Example 3.5 but warn in the Comment that the analysis may still not be valid. Example 3.5 The problem. It is known that 7 per cent of UK children give positive skin tests to grass, dust mites and cats. It is thought that a desert climate and few domestic cats may reduce the proportion of positive tests. A random sample of 54 children from such a country were given the allergen tests and only one of these had positive reactions to all three allergens. Is there sufficient evidence of a lower incidence compared with the UK? Formulation and assumptions. We justify a one-tail test of H0: p = 0.07 against H1: p < 0.07 because there are good medical reasons for assuming that the proportion should be lower under such conditions. If the direction of any effect on sensitization were unknown then a two-tail test of H0: p = 0.07 against H1: p ≠ 0.07 would be appropriate. Procedure. As exact tables for n = 54 are not widely available and a small p makes asymptotic results unreliable one should resort to appropriate software. For the one-tail test StatXact or Minitab gives an exact P = 0.101. The exact two-sided nominal 95 per cent confidence interval for p is (0.0005, 0.0989). Conclusion. The evidence of a reduction is not strong. If it were important to detect even a small reduction to, say, p = 0.05 a larger experiment would be needed to give reasonable power. Comment. In Section 3.1.1 we pointed out possible difficulties if particular groups of patients have markedly different probabilities of showing a reaction. We stressed the need for a random, or effectively random sample, to overcome this problem. Another type of problem may arise in, for instance, the testing of a new drug. For the new drug there may be a quite different set of factors influencing the likelihood of an individual showing side-effects compared with the original drug. For instance, with one antibiotic for the treatment of pneumonia people with asthma may be more likely to exhibit side-effects, whereas for a new antibiotic the reverse may be the case. In these circumstances what could be an overall beneficial effect can sometimes be masked or distorted in our sample. The solution lies in either further experimentation or at least in using more detailed observations and tests. For example, one may require an analysis that takes into account other factors that might influence outcomes for individuals, e.g. factors such as weight, age, sex of patients. The result is a more complex analytic problem and indeed most real life statistical problems are more complicated than the simple ones discussed in this section! ©2001 CRC Press LLC
3.2 TESTS RELATED TO THE SIGN TEST 3.2.1 An alternative approach to the sign-test We described the sign test in detail in Section 2.3. An alternative but equivalent approach more in the spirit of the work in Section 3.1 is sometimes used and we illustrate this by an example. Example 3.6 The problem. For the second data set in Example 2.9, i.e. 5.6 6.1 6.3 6.3 6.5 6.6 7.0 7.5 7.9 8.0 8.0 8.1 8.1 8.2 8.4 8.5 8.7 9.4 14.3 26.0
if θ is the population median test the hypothesis H0: θ = 9 against the alternative H1: θ ≠ 9 using the sign test. Obtain a 95 per cent confidence interval for θ using the methods developed in Section 3.1. Formulation and assumptions. Wide availability of programs giving exact Pvalues provides a useful facility for testing and estimation in a sign-test situation. Procedure. For illustrative purposes we use the basic statistics 1-proportion option in Minitab. Here n = 20 and the number of observations below the hypothesized median is r = 17. The sign test calls for evaluation of P for a twotail test with p = 0.5 and the Minitab program gives at the same time a confidence interval for the true p at a nominal level which we here set at 95 per cent. The program gives P = 0.003 and a nominal 95 per cent confidence interval for p of (0.621, 0.967). Allowing for rounding our P-value agrees with that obtained in Example 2.9 and indeed here we did nothing essentially different from what was done there. To obtain a confidence interval for θ we now decrease r by steps of 1 until we get a 95 per cent confidence interval for p that just includes the value p = 0.5. This straightforward stepwise process indicates that when r = 15 the confidence interval is (0.508, 0.913) while for r = 14 it is (0.457, 0.881). Thus when r = 14 we would regard p = 0.5 as acceptable with P > 0.05 but not when r = 15. The symmetry of the binomial distribution when p = 0.5 implies that we would accept when r = 6 but not when r = 5. This implies that an appropriate 95 per cent confidence interval for the median is from the sixth to the fifteenth largest observations, i.e. the open interval (6.6, 8.4), agreeing with the result in Example 2.10. Computational aspects. The procedures in Examples 2.9 and 2.10 and in this example are basically equivalent. Which is preferred is largely determined by the availability of appropriate tables or computational software.
3.2.2 Quantile tests The sign test is relevant to population medians. We are often interested in other quantiles of a population. For example a quantile such that three-quarters of the population values lie below it and one quarter above is known as the upper (or third) quartile while a
©2001 CRC Press LLC
quantile such that r tenths of the observations lie below it and (10–r) tenths of the observations lie above it where r = 1, 2, . . . , 9 is called the rth decile. Many tests and estimation procedures involving quantiles proceed along similar lines to those about medians by simply replacing p = 0.5 by a value appropriate to the relevant quantile. For continuous distributions quantiles are unique, but for discrete distributions conventions given in most standard textbooks are needed to give unique quantiles. We have already met the usual convention for the median of a set consisting of an even number, say, 2m, observations. We arrange them in ascending order and denoting the rth ordered observation by x(r) we define the median as 1 /2(x(m) + x(m + 1)). Note that the fifth decile is the same as the median. Example 3.7 The problem. A central examining body publishes the information that ‘threequarters of the candidates taking a mathematics paper achieved a mark of 40 or more’ (i.e. the first population quartile is 40). One school entered 32 candidates for this paper of whom 13 scored less than 40. The president of the Parents’ Association argues that the school’s performance is below national standards. The headmaster counters by claiming that in a random sample of 32 candidates it is quite likely that 13 would score less than the lower quartile mark even though 8 out of 32 is the expected proportion. Is his assertion justified? Formulation and assumptions. Denoting the first quartile by Q1 a test for the headmaster’s assertion is a test for H0: Q1 = 40 against H1: Q1 ≠ 40. Procedure. We associate a minus with a mark below 40; thus for our ‘sample’ of 32 we have 13 minuses (and 19 pluses). If the first quartile is 40 then the probability is p = 0.25 that each candidate in a random sample has a mark below 40 (and thus is scored as a minus). The distribution of minuses is therefore binomial B(32, 1/4). Using any program that gives a 95 per cent confidence interval for p given r = 13 minuses (event A) when n = 32 we find the relevant confidence interval is (0.24, 0.59). This includes p = 0.25. The associated P-value is P = 0.0755. Conclusion. If we observe 13 minus signs we would not reject the hypothesis H0: first quartile is 40 at a conventional 5 per cent significance level. Nevertheless there is some evidence against this hypothesis – enough to worry many parents and some may be reluctant to give the headmaster’s claim the benefit of the doubt until further evidence were available. Comments. 1. Remember that even if we use formal significance levels, nonrejection of a hypothesis does not prove it true. It is only a statement that the evidence to date is not sufficient to reject it. This may simply be because our sample is too small. We may make an error of the second kind by non-rejection. Indeed, since the above confidence interval includes p = 0.5 we would not reject the hypothesis that the median is 40. 2. We used a two-tail test. A one-tail test would not be justified unless we had information indicating the school’s performance could not be better than the ©2001 CRC Press LLC
national norm. For example, if most schools devoted three periods per week to the subject but the school in question only devoted two, we might argue that lack of tuition could only depress performance. Indeed, a one-tail test at the conventional 5 per cent level would reject H0 if there are more than 12 minuses. If we observe 13 minuses and feel a one-tail test with P = 0.05 as a significance indicator is appropriate and want to use the method described in this example we might determine a 90 per cent confidence interval for p and reject H0 if p = 0.25 is below the lower limit for this interval. Think carefully about this to be sure you see why. Have you any reservations about the accuracy of this approach? 3. The headmaster’s claim said ‘if one took a random sample’. Pupils from a single school are in no sense a random sample from all examination candidates. Our test only establishes that results for this particular school are not too strongly out of line with national results in the sense that they just might arise if one took a random sample of 32 candidates from all entrants. 4. Tests for a third quartile are symmetric with those for a first quartile if we interchange plus and minus signs.
3.2.3 The Cox–Stuart sign test for trend Cox and Stuart (1955) proposed a simple test for a monotonic trend, i.e. an increasing or decreasing trend. A monotonic trend need not be linear; it need only express a tendency for observations to increase or decrease subject to some local or random irregularities. Consider a set of independent observations x 1, x 2, . . . , xn ordered in time. If we have an even number of observations, n = 2m, say, we take the differences xm+1 – x 1, xm+2 – x 2, . . . , x2m – xm. For an odd number of observations, 2m+1, we may proceed as above omitting the middle value xm+1 and calculating xm+2 – x1, etc. If there is an increasing trend we expect most of these differences to be positive, whereas if there were no trend and observations differed only by random fluctuations about some median these differences (in view of the independence assumption) are equally likely to be positive or negative. A preponderance of negative differences suggests a decreasing trend. This implies that under the null hypothesis of no trend, the plus (or minus) signs have a B(m, 1/2) distribution and we are in a sign test situation. Example 3.8 The problem. The US Department of Commerce publishes estimates obtained from independent samples each year of the mean annual mileages covered by various classes of vehicles in the United States. The figures for cars and trucks (in thousands of miles) are given below for each of the years 1970–83. Is there evidence of a monotonic trend in either case?
©2001 CRC Press LLC
Cars 9.8 9.9 Trucks 11.5 11.5
10.0 9.8 9.2 9.4 9.5 9.6 9.8 12.2 11.5 10.9 10.6 11.1 11.1 11.0
9.3 8.9 8.7 9.2 9.3 10.8 11.4 12.3 11.2 11.2
Formulation and assumptions. The figures for each year are based on independent samples so we may use the Cox–Stuart test for trend. Without further information a two-tail test is appropriate as a trend may be increasing or decreasing. Procedure. For cars relevant differences are 9.6 – 9.8, 9.8 – 9.9, 9.3 –10.0, 8.9 – 9.8, 8.7 – 9.2, 9.2 –9.4, 9.3 – 9.5 and all are negative. Appropriate tables or computer software immediately establishes that seven negative signs or seven positive signs when p = 0.5 has an associated P = 0.016. Conclusion. There is strong evidence of a downward monotonic trend. Comments. 1. For trucks the corresponding differences have the signs –, –, –, –, +, +, +. Clearly when we only have seven data values 3 plus and 4 minus (or 3 minus and 4 plus) signs provides the strongest possible evidence to support a hypothesis of no monotonic trend. The fact that the first four differences are all negative and the last three all positive suggests the possibility of a decreasing trend followed by an increasing trend (i.e. a non-monotonic trend) rather than random fluctuations. The sample is too small to establish this, but in Section 3.5 we describe a ‘runs test’ appropriate when we have larger samples for testing whether in circumstances like these fluctuations are random. 2. Periodic trends are common. For example, at many places in the northern hemisphere mean weekly temperature tends to increase from February to July and decrease from August to January. A Cox–Stuart test applied to data of this type might either miss such a trend (because it is not monotonic) or indicate a monotonic trend for records over a limited period (e.g. from February to June). Conover (1999, Examples 4 and 5, pp. 172–175) shows how in certain circumstances the Cox–Stuart test may be adapted to detect periodic trends by re-ordering the data. 3. If the same samples of cars and trucks had been used each year the independence assumption would not hold; inference would then only be valid for vehicles in that sample, for anything atypical about the sample would influence observations in all years. With independent samples for each year, anything atypical about the sample in any one year will be incorporated in the random deviation from trend in that year.
3.3 MATCHING SAMPLES TO DISTRIBUTIONS Test and estimation procedures for centrality measures were covered in Chapter 2 but other population characteristics such as spread or skewness may be of interest. In particular, we often want to know whether observations are consistent with their being a sample from some specified continuous distribution. Kolmogorov (1933; 1941) devised a test for this purpose. ©2001 CRC Press LLC
Given observations x1, x 2, . . . , x n, we ask if these are consistent with our sample being from some completely specified distribution. This might be a uniform distribution over (0, 1) or over (20, 30), or a normal distribution with mean 20 and standard deviation 2.7. Kolmogorov’s test is distribution-free because the procedure does not depend upon which distribution is completely specified under H 0. In Section 3.3.3 we look at modifications to answer questions like ‘Can we suppose these data are from a normal distribution with unspecified mean and variance?’ 3.3.1 Kolmogorov’s test The continuous uniform distribution (sometimes called the rectangular distribution) is a simple continuous distribution that arises when a random variable has the same probability of taking a value in any small interval of length δx within a fixed and specified finite interval (a, b). For example, suppose pieces of thread each 6 cm in length are clamped at each end and a force is applied until they break; if each thread breaks at the weakest point and this is equally likely to be anywhere along its length, then the breaking points will be uniformly distributed over the interval (0, 6), where the distance to the break is measured in centimetres from the lefthand clamp. The probability density, or frequency, function has the form f(x) = 0, x ≤ 0; f(x) = 1/6, 0 < x ≤ 6; f(x) = 0, x > 6. The rectangular form of f(x) over (0, 6) explains the name rectangular distribution. If the thread always breaks, the probability is 1 that it breaks in the interval (0, 6), so the total area under the density curve must be 1; clearly only the density function above satisfies both this condition and that of equal probability of a break in any small segment of length δ x no matter where that segment lies within the interval. The function is graphed in Figure 3.1 and is essentially a line running from 0 to 6 at height 1 /6 above the x-axis. The lightly shaded area represents the total probability of 1 associated with the complete distribution. The heavily shaded rectangle PQRS between x = 3.1 and x = 3.8 represents the probability that the random variable X, the distance to the break, takes a value between 3.1 and 3.8. Since PQ = 3.8 – 3.1 = 0.7 and PS = 1/6 , clearly this area is 0.7/6 ≈ 0.1167. The probability of a break occurring in any segment of length 0.7 lying entirely in the interval (0, 6) is also 0.1167.
©2001 CRC Press LLC
Following convention we use X, Y . . . as names for random variables, and corresponding lowercase letters x, y . . . (with or without subscripts) for specific values of these variables. The Kolmogorov test uses not the probability density function, but the cumulative distribution function (cdf), a function that gives Pr(X ≤ x) for all x in (0, 6). In our example the probability the break occurs in the first two centimetres is Pr(X ≤ 2). Clearly this is 1/3. The cdf is written F(x) and for any x between 0 and 6, F(x) = x/6. It has the value 0 at x = 0 and 1 at x = 6. It is graphed in Figure 3.2. These notions generalize to a uniform distribution over any interval (a, b) and the cdf is F(x) = (x – a)/(b – a), a < x ≤ b, specifying a straight line rising from zero at x = a to 1 at x = b. Clearly F(a) = 0 and F(b) = 1. This is illustrated in Figure 3.3. Y
0.6
0.4
0.2
0
2
1
S
R
P
Q
5
X
6
Figure 3.1 Probability density function for a continuous uniform distribution over (0, 6).
1.0
Y
1 unit
0
X 1
2
3
4
5
6
Figure 3.2 Cumulative distribution function (cdf) for a uniform distribution over (0, 6). ©2001 CRC Press LLC
Y 1.0
1 unit
a
0
X
b
Figure 3.3 Cumulative distribution function (cdf) for a uniform distribution over (a, b).
Y 1.0
0.5
-3
-2
-1
0
1
2
3
X
Figure 3.4 Cumulative distribution function (cdf) for the standard normal distribution.
For any continuous distribution the cdf is a curve starting at zero for some x and increasing as we move from left to right until it attains the value 1. It never decreases as x increases and is said to be monotonic non-decreasing (or monotonic increasing). Figure 3.4 shows the cdf for the standard normal distribution. The Kolmogorov test compares a population cdf with a related curve S(x) based on sample values and called the sample (or empirical) cumulative distribution function. For a sample of n observations S( x ) =
©2001 CRC Press LLC
number of sample values ≤ x n
(3.4)
Example 3.9 The problem. The distances from one end at which each of 20 threads 6 cm long break when subjected to strain are given below. Evaluate and graph S(x). For convenience the distances are given in ascending order. 0.6 2.5
0.8 2.9
1.1 3.1
1.2 3.4
1.4 3.4
1.7 3.9
1.8 4.4
1.9 4.9
2.2 5.2
2.4 5.9
Formulation and assumptions. From (3.4) it is clear that S(x) increases by 1/20 at each unique x value corresponding to a break, or by r/20 if r break distances coincide. Procedure. When x = 0, S(x) = 0; it keeps this value until x = 0.6, the first break. Then S(0.6) = 1/20, and S(x) maintains this value until x = 0.8 when it jumps to 2/20, a value retained until x = 1.1. It jumps in steps of 1/20 at each break value until x = 3.4, where it increases by 2/20 since two breaks occur at x = 3.4. S(x) is referred to as a step function for obvious reasons. Its value at each step is given in Table 3.1. Conclusion. Figure 3.5 shows the form of S(x) for these data. Comment. S(x) is a sample estimator of the population cdf F(x). If a sample comes from a specified distribution, the step function S(x) should not depart markedly from the population cdf F(x) unless the sample is very small, say, n = 10 or less. In this problem if breaks are uniformly distributed over (0, 6) one expects, for example, about half of these to be in the interval (0, 3), so that S(3) should not have a value very different from 0.5 and so on. In Figure 3.5 we show also the cdf for a uniform distribution over (0, 6) reproduced from Figure 3.2. Intuitively one may feel that this is not a good fit.
Recognizing that S(x) should not depart violently from F(x) for a sample from a distribution with that specific cdf is basic to Kolmogorov’s test. The test statistic is the maximum difference in magnitude between F(x) and S(x). Example 3.10 The problem. Given the 20 breaking points and the corresponding S(x) values in Table 3.1, is it reasonable to suppose breaking points are uniformly distributed over (0, 6)? Table 3.1 Values of S(x) at step points. ___________________________________________________________ x 0.6 0.8 1.1 1.2 1.4 1.7 1.8 1.9 2.2 2.4 2.5 S(x) 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 x 2.9 3.1 3.4* 3.9 4.4 4.9 5.2 5.9 S(x) 0.60 0.65 0.75 0.80 0.85 0.90 0.95 1.00 ___________________________________________________________ * Repeated sample value ©2001 CRC Press LLC
Y
1.0 0.8 0.6 0.4 0.2
0
1
2
3
4
5
6
X
Figure 3.5 Sample cumulative distribution function (cdf) for thread breaks (step function) together with the cdf for a uniform distribution over (0,6) (straight line). Table 3.2 Comparison of F(x) and S(x) for thread breaks. xi
F(xi)
S(xi)
F(xi)–S(xi)
F(xi)–S(xi–1)
0.6 0.8 1.1 1.2 1.4 1.7 1.8 1.9 2.2 2.4 2.5 2.9 3.1 3.4 3.9 4.4 4.9 5.2 5.9
0.10 0.13 0.18 0.20 0.23 0.28 0.30 0.32 0.37 0.40 0.42 0.48 0.52 0.57 0.65 0.73 0.82 0.87 0.98
0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.75 0.80 0.85 0.90 0.95 1.00
0.05 0.03 0.03 0.00 –0.02 –0.02 –0.05 –0.08 –0.08 –0.10 –0.13 –0.12 –0.13 –0.18 –0.15 –0.12 –0.08 –0.08 –0.02
0.10 0.08 0.08 0.05 0.03 0.03 0.00 –0.03 –0.03 –0.05 –0.08 –0.07 –0.08 –0.08 –0.10 –0.07 –0.03 –0.03 0.03
Formulation and assumptions. The maximum difference in magnitude between F(x) and S(x) is compared with a tabulated value to assess the goodness of fit. Or better, software is used to determine the exact P-value.
©2001 CRC Press LLC
Procedure. Table 3.2 shows values of F(x) and S(x) at each break point given in Table 3.1. It also gives at each of these points the difference F(xi) – S(xi). In view of the stepwise form of S(x), the maximum difference may not be in this set. Inspecting Figure 3.5 it is clear that a greater difference may occur immediately before such a step. There S(x) has the value attained at the previous step, so a maximum may occur among the F(xi) – S(xi – 1). These differences are also recorded in Table 3.2. Figure 3.5 shows that for much of the interval (0, 6) S(x) lies above F(x). The entry of greatest magnitude in the last two columns of Table 3.2 is –0.18 when xi = 3.4. Published tables give values of the minimum difference required for significance at nominal 5 and 1 per cent levels for various n, and in some cases also for other quantiles but modern software makes these somewhat redundant. If required, tables may be found in Gibbons and Chakriborti (1992), in Conover (1999) or in many general statistical tables. The theory for computing exact P-values is more complicated than that for situations we have met so far and Gibbons and Chakriborti (1992, Section 4.3) give a detailed account. Tables indicate that the statistic must have a value of at least 0.294 for significance at the 5 per cent level in a two-tail test. The exact test gives P = 0.458. Conclusion. There is no evidence against H0. Comments. 1. A two-tail test is appropriate if the alternative hypothesis is that the observations may come from any other unspecified distribution. Situations may arise where the only alternative is that the cdf must lie either wholly at or above or wholly at or below that specified in the null hypothesis. For example, in the thread problem we may know that the test equipment places strains on the thread that, if not uniformly distributed, will be greater at the left and decrease as we move to the right. This might increase the tendency for breaks to occur close to the left end of the string. Then the cdf would always lie above the straight line representing F(x) for the uniform distribution. In that case a maximum difference with S(x) greater than F(x) for the uniform distribution would, if significant, favour the acceptable alternative, making a one-tail test appropriate. 2. In commenting on Figure 3.5 we noted that the step function did not appear to match the population cdf very well. The Kolmogorov test does not indicate a meaningful departure, a point we take up again in Section 3.3.2. Computational aspects. Some general statistical packages include programs for the Kolmogorov test but may use only asymptotic results applied at nominal significance levels and these are not satisfactory even in that context unless sample sizes are at least 20, or sometimes more. The exact test program in StatXact avoids these difficulties and allows a comparison with an asymptotic approximation to the P-value, which often shows a surprising difference.
The Kolmogorov test appears to waste information by using only the difference of greatest magnitude. Tests that allow for all differences exist but tend to have few, if any, advantages. This is not as contrary as intuition may suggest, for the value of S(x) at any stage depends on how many observations are less than the current x and therefore at each stage we make the comparison on the basis of accumulated evidence. ©2001 CRC Press LLC
For a discrete distribution the Kolmogorov test tends to give too few significant results and in this sense is conservative. We consider preferred tests for discrete distributions in Section 9.4. 3.3.2 Comparison of distributions: confidence regions If, in Figure 3.5 we had drawn step functions everywhere at distances 0.294 units above or below that representing S(x), with adjustments at each end to stop them falling below zero or above one, we would, since 0.294 is the critical difference for significance at the 5 per cent level given by tables, have a 95 per cent confidence region for F(x) in the sense that any F(x) lying entirely between these two new step functions would be an acceptable F(x) when testing at a 5 per cent significance level in a two-tail test. This interval is not very useful in practice. A more common practical situation is one where it is reasonable to assume that our data may come from one of a (usually small) number of completely specified distributions. We might then use a Kolmogorov test for each. In each test the relevant S(xi) will be the same for a given data set. As a result of these tests we may reject some of the hypotheses but still find more than one acceptable. It is then useful to get at least an overall eye comparison of how well the match of S(x) is to each acceptable F(x). Ross (1990, p. 85) gives a useful procedure for such eye comparisons. Basically it consists of plots of the maximum of the two deviations given for the last two columns of each row in a table like Table 3.2 against the S(xi) for that row. Plots may easily be done for several different hypothesized population distributions on the same graph. Example 3.11 The problem. Compare the fit of the thread break data in Example 3.9 to (i) a uniform distribution over (0, 6) and (ii) the distribution with cumulative distribution function over (0, 6) given by F(x) = x/5, 0 < x ≤ 3; F(x) = 0.2 + 4x/30, 3 < x ≤ 6.
(3.5)
Formulation and assumptions. All information needed for the Kolmogorov test procedure for the distribution (i) was obtained in Example 3.10. We require similar information for the alternative distribution in (ii). We make a graphical comparison using the format suggested by Ross. Procedure. For (i) the values of S(x) of interest are those in column 3 of Table 3.2. In each case the corresponding deviation is the value of greatest magnitude in columns 4 and 5. For (ii) the values of S(x) are again those in Table 3.2. However, fresh values of F(x) for each xi must be calculated using (3.5).
©2001 CRC Press LLC
0.3 case (1)
0.2
case(ii) 0.1 S(x)
0
1
–0.1 –0.2 –0.3
Figure 3.6 Plot of deviations F(x) – S(x) against S(x) for the two F(x) functions specified in Example 3.5. Table 3.3 Values of xi, S(xi), di = max[F(xi) – S(xi), F(xi) – S(xi–1)] for population distributions specified in (i) and (ii) of Example 3.11.
xi
S(xi)
di(i)
di(ii)
0.6 0.8 1.1 1.2 1.4 1.7 1.8 1.9 2.2 2.4 2.5 2.9 3.1 3.4 3.9 4.4 4.9 5.2 5.9
0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.75 0.80 0.85 0.90 0.95 1.00
0.10 0.08 0.08 0.05 0.03 0.03 –0.05 –0.08 –0.08 –0.10 –0.13 –0.12 –0.13 –0.18 –0.15 –0.12 –0.08 –0.08 0.03
0.12 0.11 0.12 0.09 0.08 0.09 0.06 0.03 0.04 0.03 –0.05 0.03 –0.04 –0.10 –0.08 –0.06 –0.05 –0.06 0.04
©2001 CRC Press LLC
For example, when xi = 1.2, (3.5) gives F(1.2) = 1.2/5 = 0.24. Using this and the values of S(1.1) and S(1.2) in Table 3.2 we find that F(1.2) –S(1.2) = 0.24 – 0.20 = 0.04 and F(1.2) – S(1.1) = 0.24 – 0.15 = 0.09. Thus, corresponding to x = 1.2, the maximum deviation is 0.09 just before the step in S(x) at x = 1.2. In Table 3.3 we set out for each xi the corresponding S(xi) and maximum deviations for the population distributions specified in (i) and (ii). These deviations are plotted against S(x) in Figure 3.6. Although it is not essential we have joined consecutive points in each case by straight line segments. The parallel broken lines at y = ± 0.294 represent the critical value for significance at the 5 per cent level. We reject H0 at a formal 5 per cent significance level only if we observe a deviation outside these lines. Conclusion. There is no strong evidence against either of the population hypotheses specified in (i) and (ii). Comment. The deviations, except for the early observations, are rather smaller for (ii), suggesting (but only marginally) that this may be a slightly better fit. The hypothesis (ii) implies a uniform but higher probability of breakages between 0 and 3 cm relative to a uniform distribution, but a lower probability between 3 and 6 cm. If other hypotheses are of interest relevant deviations for these may be entered in Figure 3.6.
3.3.3 A test for normality The Kolmogorov test is valid for any completely specified continuous distribution. If we specify in H0 only that the sample has a normal distribution and estimate the parameters µ , σ2 from the data we may still use the Kolmogorov statistic but the associated P-values differ from those for the fully specified model. For this important practical situation a test proposed by Lilliefors (1967) is useful. We assume observations are a random sample from some unspecified continuous distribution, and test whether it is reasonable to suppose this is a member of the normal family. To this extent our test is not distribution-free, but it is nonparametric in the sense that we do not specify parameter values. The basic idea can be extended to test compatibility with other families of distributions such as a gamma distribution with unspecified parameters, but separate tables of critical values or appropriate software to estimate P-values are needed for each family of distributions. Conover (1999, Section 6.2) describes some difficulties in obtaining critical values and indeed one may only get approximate values of the statistic corresponding to precise P-values such as P = 0.05 or 0.01, or alternatively Monte Carlo approximations for the exact P-value corresponding to an observed value of the statistic. StatXact provides a program for the latter. Conover
©2001 CRC Press LLC
(1999, Table A14) gives approximate critical values for sample sizes up to 30 and asymptotic approximations for larger samples. Another test for normality is the Shapiro–Wilk (1965) test which again requires special tables or appropriate software for implementation. An account of how the test works, without detail of the theory, is given by Conover (1999, Section 6.2) who also provides extensive tables that are required if relevant computer software is not available. This test has good power against a range of alternatives but the rationale is less easy to describe by intuitive arguments. StatXact includes a program for asymptotic estimates of P-values. Except for very small samples, where exact results are available, these usually prove adequate. Tests for normality may be important in deciding whether to apply nonparametric methods in a particular problem. We describe Lilliefors’ test and also quote results for analysis of the same data using the Shapiro–Wilk test. Example 3.12 The problem. In the Badenscallie burial ground in Wester Ross, Scotland, Sprent recorded the ages at death in years of males on all tombstones for four clans in the district and reported these in the first edition of this book. The complete list is given in the appendix. From all 117 ages recorded a random sample of 30 was taken and the ages for that sample were, in ascending order: 11 13 14 22 29 30 41 41 52 55 56 59 65 65 66 74 74 75 77 81 82 82 82 82 83 85 85 87 87 88 Is it reasonable to suppose the death ages are normally distributed? Formulation and assumptions. In Lilliefors’ test the Kolmogorov statistic is used to compare the standard normal cdf Φ (z) with the standardized sample cdf S(z) based on the transformation x −m zi = i s where m is the sample mean and s the usual estimate of population standard deviation, i.e. s = 2
∑x i
2 i
−(
∑x ) i
2
/n
i
n −1
Procedure. We find m = 61.43, s = 25.04. Successive zi are calculated: e.g. for x1 = 11, we find z1 = (11 – 61.43)/25.04 = –2.014. We treat this as a sample value from a standard normal distribution and tables (e.g. Neave, 1981, pp. 18–19) show Φ (–2.014) = 0.022. Table 3.4 is set up in analogous manner to Table 3.2. Here Φ (z), S(z) represent the standard normal and sample cdfs respectively. These computations are done automatically in StatXact. It is not essential, but ©2001 CRC Press LLC
we have included the x values, an asterisk against a value implying it occurs more than once. The largest difference is 0.192, occurring in the final column when x = 74. Tables such as Table A14 in Conover (1999, p.548) give the 1 per cent critical value when n = 30 for a one-tail test to be 0.183. StatXact computes a Monte Carlo approximation to the exact P-value for this particular data set as a worked example in the manuals for StatXact versions 3 and 4. These approximate P-values differ slightly between simulations. That in the version 3 manual is P = 0.0057 while that in the version 4 manual is P = 0.0055. Conclusion. The Monte Carlo approximations to exact P-values provide strong evidence that the sample is not from a normal distribution. Comments. 1. The StatXact manuals for versions 3 and 4 also use these data to illustrate the Shapiro–Wilk test and obtain an asymptotic estimate P = 0.0007. Shapiro, Wilk and Chen (1968) carried out studies that show this test has good power against a wide range of departures from normality. 2. A glance at the sample data suggests that a few males died young and that the distribution is skew with a large number of deaths occurring after age 80. Figure 3.7 shows these data on a histogram with class interval of width 10. Readers familiar with histograms for samples from a normal distribution will not Table 3.4 The Lilliefors normality test: Badenscallie ages at death. x
z
Φ(z)
S(z)
Φ(zi)–S(zi)
11 13 14 22 29 30 41* 52 55 56 59 65* 66 74* 75 77 81 82* 83 85* 87* 88
–2.014 –1.934 –1.894 –1.575 –1.295 –1.255 –0.816 –0.377 –0.257 –0.217 –0.097 0.142 0.183 0.502 0.542 0.622 0.781 0.821 0.861 0.942 1.021 1.061
0.022 0.026 0.029 0.058 0.098 0.105 0.207 0.353 0.399 0.414 0.461 0.556 0.572 0.692 0.706 0.733 0.782 0.794 0.805 0.827 0.846 0.856
0.033 0.067 0.100 0.133 0.167 0.200 0.267 0.300 0.333 0.367 0.400 0.467 0.500 0.567 0.600 0.633 0.667 0.800 0.833 0.900 0.967 1.000
–0.011 –0.044 –0.071 –0.075 –0.069 –0.095 –0.060 0.053 0.066 0.047 0.061 0.089 0.072 0.125 0.106 0.100 0.115 –0.006 –0.028 –0.073 –0.121 –0.144
*repeated value
©2001 CRC Press LLC
Φ(zi)–S(zi–1) 0.022 –0.007 –0.038 –0.042 –0.035 –0.062 –0.007 0.086 0.099 0.081 0.094 0.156 0.105 0.192 0.139 0.133 0.149 0.127 –0.005 –0.006 –0.054 –0.111
Y 12 10 8 6 4 2 0 0–9 10–19 20–29 30–39 40–49 50–59 60–69 70–79 80–89 Age at death
X
Figure 3.7 Histogram of death ages at Badenscallie. Y 15
10
5
0–9 10–19 20–29 30–39 40–49 50–59 60–69 70–79 80–89 90–99 X Age at death
Figure 3.8 Histogram of death ages for clan McAlpha. be surprised that we reject the hypothesis of normality. Indeed ‘life-span’ data, whether for man, animal or the time a machine functions without a breakdown, will often characteristically have a non-normal distribution. Computational aspects. We indicated above that these data are used to illustrate Lilliefors’ test in StatXact. Because there is no analytic formula for exact P-values only a Monte Carlo approximation is available for Lilliefors’ test. These are given together with a confidence interval for that estimate and this will usually be quite short if the estimate is made on a large number of Monte Carlo ©2001 CRC Press LLC
samples. In practice it is customary to use anything between 1000 (minimum) and perhaps 1 000 000 samples if high precision is required. For the Shapiro–Wilk test the distribution of the test statistic appears to be known only for sample sizes of six or less and is very nonnormally distributed for larger sample sizes. However it may be transformed to approximate normality and the transformation provides a basis for determining the asymptotic P-value given in StatXact.
3.3.4 Relevant inference Situations arise where inferences about means or medians are by their nature somewhat irrelevant to sensible deductions and here tests such as Kolmogorov’s or Lilliefors’ test may be more useful. We comment upon one such case. Example 3.13 For one particular clan – it would be invidious to give it the real name, so we shall call it the McAlpha clan (but the data are real) – ages at death were recorded for all males of that clan buried in the Badenscallie burial ground. For all 59 burials the ages (arranged for convenience in ascending order) were: 0
0
1
2
3
9 14 22 23 29 33 41 41 42 44 52 56 57 58 58
60 62 63 64 65 69 72 72 73 74 74 75 75 75 77 77 78 78 79 79 80 81 81 81 81 82 82 83 84 84 85 86 87 87 88 90 92 93 95 A histogram of these data with class interval 10 is given in Figure 3.8. The mean and median are respectively 61.8 and 74 while the age range with most deaths is 80 – 89. Striking features in Figure 3.8 are the skewness of the data to the left and the bimodal nature with a small peak (secondary mode) of infant deaths in the range 0–9 followed by lower death rates in 10-year intervals until age 60. The death rate then rises sharply to the primary mode at the 80–89 range. One might go through formal motions of attaching confidence intervals to the mean or median using an approach based on the t-test (obviously irrelevant because of nonnormality – Lilliefors’ test indicates P < 0.001), the Wilcoxon test (not valid because of asymmetry) or the sign test (valid but not very interesting in the light of the bimodality). Such approaches are at worst wrong and at best pointless. In this example the histogram is probably the most useful aid in interpreting the data. What is the relevance of any formal inference procedure in a case like this? From what population is this a random sample? It is clearly not a random sample of all males buried at Badenscallie, for clans and families have differing lifespan characteristics. Perhaps the pattern of death ages is a reasonable approximation to that for the McAlpha clan in Wester Ross, Sutherland and the Western Isles (where the clan is well established). The main use we can envisage for these data would be for comparisons with patterns for other clans. Difficulties like these may arise with data collection generally and are not confined to nonparametric methods. Analyses making various assumptions often draw attention to interpretation problems.
©2001 CRC Press LLC
Using modern software it is easy to apply several tests to the same data. Indeed, for centrality tests StatXact allows one to use Wilcoxon and normal scores test and also to base tests on any scores one chooses. Do not abuse this freedom; a scoring system should be chosen on logical grounds. Remember our ethical warning in the penultimate paragraph of Section 1.5! If different choices are logical under differing assumptions and these lead to inconsistent results (as in this example for means or medians) further analysis (e.g. applying Lilliefors’ test) or more simply drawing a histogram as in Figure 3.8, or sometimes even just looking at the data critically, may suggest the most appropriate choice. In this example clearly a sign-test-type confidence interval for the median is less inappropriate than many others but this begs the question as to whether the median is a useful summary statistic for bimodal data.
3.4 ANGULAR DATA In some investigations measurements are made on directions, e.g. wind directions in a certain town at noon on successive days, the bearings at which released pigeons disappear over the horizon and the successive stopping positions of a roulette wheel. In other studies the data, although not measured directly as angles or directions,