Developments in Dual System Estimation of Population Size and Growth
This page intentionally left blank
Developments in Dual System Estimation of Population Size and Growth Edited by KarolJ.Krotki
The University of Alberta Press 1978
First published by The University of Alberta Press Edmonton, Alberta, Canada 1978 This book has been published with the help of a grant from the Humanities Research Council of Canada, using funds provided by the Canada Council. Copyright © 1978 The University of Alberta Press Canadian shared cataloguing in Publication Data Main entry under title Developments in dual system estimation of population size and growth Includes bibliographies. ISBN 0-88864-017-X 1. Demography - Methodology - Addresses, essays, lectures. 2. Population forecastingAdresses, essays, lectures. I. Krotki, Karol J. HB881.D49 301.32 C77-002073-9
All rights reserved No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form, or by any means, electronic mechanical, photocopying, recording or otherwise without the prior permission of the copyright owner. Design: P. Bartl, Dept. of Art and Design Cover design: P. Bartl, Chi Lee Printed in Canada by Printing Services of The University of Alberta iv
Contents
Acknowledgments xiii Preface xiv by W. Parker Mouldin, Acting President, the Population Council, New York Introduction xvi Chapter 1 The Role of PGE/ERAD/ECP Surveys Among the Endeavours to Secure More and Improved Demographic Data 1 by Karol J. Krotki 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8
The need for improved demographic data 1 An overview of the available techniques 2 The theory behind the PGE/ ERAD/ ECP technique 8 The PGE handbook and some essential PGE/ERAD/ECP features 10 An ideal PGE/ ERAD/ ECP exercise 12 An overview of the world uses of the PGE/ERAD/ECP technique 13 Modes of securing the advantages of the technique 25 Inadvertent modes of destroying the advantages of the technique 27 1.8.a Matters affecting independence 27 l.S.b Matters affecting matching procedure 27 l.S.c Matters affecting samples 28 l.8.d Matters affecting organization 28 1.9 Quasi PGE/ ERAD/ ECP estimates 29 1.10 The paucity and importance of costing data 29 1.11 The past history and the probable future of the technique 48 Discussion by Lee L. Bean 49 Chapter 2 The State of the Art in Dual Systems for Measuring Population Change 53 by H. Bradley Wells and Daniel G. Horvitz 2.1 2.2
Introduction 53 Sample design 54 2.2.a Some general sampling considerations 54 2.2.b The need for realistic parameters 55 2.2.c Rotation designs 56 V
2.2.d Some aids to improved estimation 56 2.3 The data collection elements of the dual collection system 57 2.3.a The essential elements of a dual collection system 57 2.3.b The continuous recording procedure 58 2.3.c The household survey procedure 61 2.3.d Matching procedures 63 2.3.e Field identification procedures and other factors common to both collection procedures 64 2.4 Conclusion 69 Discussion by Ivan P. Fellegi 69 Chapter 3 Dual Estimation in Demography Employing Time Series and Cross Section Data 74 by P. Krishnan 3.1 3.2 3.3 3.4
Introduction 74 Pooling cross section and time series data 74 Best estimator 75 Suggested estimation strategies 75 Estimator 1 75 Estimator 2 76 Estimator 3 77 An appendix to chapter 3: Correlation pattern in Canadian data on births and deaths 79 Discussion by Eli S. Marks 79
Chapter 4 Dual System Estimators Based on Multiplicity Surveys 81 by Monroe G. Sirken 4.1 4.2 4.3 4.4 4.5 4.6 4.7
Introduction 81 Counting rules 81 Multiplicity estimators 82 Dual system multiplicity estimators 83 Variance of sample estimators 84 Relative precision of different rules 84 Conclusion 86 Appendix A to chapter 4: Variance of DSM sample estimators 87 Appendix B to chapter 4: Simplification of Hr 87 Discussion by Ivan P. Fellegi 89
Chapter 5 The Collection of Demographic Data in Francophone Africa and Liberia Using the PGE/ERAD/ECP System 92 by Francois Pradel de Lamaze 5.1 5.2 5.3 5.4 5.5 5.6 5.7 vi
Dual collection 92 Some observations 92 Some undertakings in Madagascar 94 The Tunisian experiment 95 The case of Senegal and Cameroun 96 An endeavour in Algeria 96 Morocco: an extended experiment 97
5.8 Dual collection in Liberia 99 Discussion by William Seltzer 100 Chapter 6 The Egyptian Study to Measure Vital Rates: Some Estimates by Dual Collection 104 by K. E. Vaidyanathan 6.1 6.2 6.3 6.4 6.5
Introduction 104 Study design 104 Findings of the study 106 An evaluation of the estimates 108 Conclusion 109 Discussion by William Seltzer 110
Chapter 7 Some Practical Problems Suggested by the Application of the PGE/ERAD/ECP System in Morocco 113 by Mohamed Rachidi 7.1 7.2 7.3 7.4 7.5 7.6
Moroccan demography 113 The periodic household survey and its rotating objective 114 Cluster size and some consequences 114 The efficiency of personnel: resident and outside 115 The numbering of structures 116 Organization in the field 117 7.6.a Interval between two household surveys 117 7.6.b Relation between rounds in periodic surveys 117 7.6.c Events in medical institutions 117 7.6.d The problem of supervision in the field 118 Discussion by Charles Nobbe 119
Chapter 8 PGE/ERAD/ECP Matching Experiences in Morocco 122 by El Arbi Housni, Frances Notzon, and Marie-Daniele Picket 8.1 8.2 8.3 8.4 8.5
Features of field work most relevant to matching 122 The purpose of matching 123 Experimental matching: procedure used 124 Experimental matching: problems encountered 126 Production matching 128 8.5.a Vital rates 129 8.5.b Field verification 132 8.6 Conclusion 133 8.7 An appendix to chapter 8: The use of and experimental study for reaching decisions on matching rules 135 by Gad Nathan 8.A.I General 135 8. A.2 Data available from an experimental matching study 135 8. A.3 The variances of the estimator and its estimation 137 8. A.4 Comparison of matching rules 139 8. A.5 A proposal for a decision process 141 vii
Chapter 9 The PGE/ERAD/ECP System of Data Collection in Africa and a Comparison of its Results With Those of Analytic Techniques 145 by Roderic P. Beaujot 9.1 Introduction 145 9.2 Some African experiences with the PGE/ERAD/ECP 145 9.3 Analytic techniques 148 9.4 Some African experiences with analytic techniques 150 Discussion by Ansley J. Coale 153 Chapter 10 The Role of Dual System Estimation in Census Evaluation 156 by Eli S. Marks 10.1 The importance of publishing a census evaluation 156 10.2 Methods of census evaluation and their biases and variances 157 10.3 Internal and external consistency analysis 158 10.4 Dual system estimation 159 10.5 The two types of PES (post-enumeration survey) 161 10.6 A PES with a PGE/ ERAD/ ECP approach 162 10.7 Out-of-scope error in a PES 164 10.8 Widening the available options for census evaluation 165 Discussion by William Seltzer 166 PGE/ERAD/ECP issues 166 Census evaluation issues 167 General survey research issues 167 An appendix to chapter 10: PGE/ERAD/ECP evaluation of the Korean and Paraguayan results 168 10.A.I Introduction to the Korean and Paraguayan results 168 10.A.2 Overall results: Korea 170 10.A.3 Overall results: Paraguay 171 10. A.4 Differences in completeness between migrants and non-migrants 173 10.A.5 Differences in completeness between types of area 173 10.A.6 Completeness differences correlated with sex and age 174 10.A.7 Differences in completeness related to household composition and migration status 181 10.A.8 Post-stratification 185 10. A.9 Handling of migrants with "insufficient information for matching" 186 Chapter 11 The 1974 Post-Enumeration Survey of Liberia—A New Approach 189 by Eli S. Marks and John C. Rumford 11.1 11.2 11.3 11.4 11.5 11.6 11.7 viii
The traditional PES approach 189 Dual system estimation 191 Independence 193 Other biases 196 Handling of migrants 197 Use of one-way matching 198 Elimination of field verification 199
11.8 PES results 199 11.9 Some defects of the Liberian PES 200 11.10 Conclusion 202 Chapter 12 The Problem of Independence and Other Issues 205 12.1 12.2
C. Scott on Sources of error in the dual system approach 205 V.H. Muhsam on The bias of the PGE/ERAD/ECP estimates due to overenumeration 208 Discussion by Eli S. Marks 210
Appendix International Association of Survey Statistians Meeting, Vienna, 1973: Organizer's Report 215 Glossary 218 About the Authors 229 References 235 Index 248
ix
List of Tables
1.1 1.2 1.3 1.4 1.5 2.1 4.1 4.2 5.1 5.2 5.3 5.4 5.5 6.1 6.2 6.3 7.1 8.1 8.2 8.3 8.4 8.5 X
An overview of the known world involvement of PGE/ERAD/ECP techniques 14 Summary of world involvement in PGE/ERAD/ECP 25 Survey costs divided by office/field and by primary/secondary units 30 Total and proportionate survey costs 37 Costs with varying types of surveys 42 Estimated completeness rates and crude birth rates by area, periods under different recording operations, switch-back trial; Colombia, 1971-73 62 Estimates of 6,0,0,and fl based on alternative counting rules for enumerating deaths 85 Proportion of deaths that were missed by type of housing unit in the survey experiment, Los Angeles, July-October 1969 86 Estimated completeness rates in two Madagascar dual collections, 1967-70 94 Estimated completeness rates in two dual collection Sheikhdoms of Tunisia, 1968-69 95 Estimated completeness rates in dual collection allowing for delayed registration, two Sheikhdoms of Tunisia, 1968-69 96 Vital rates in Liberia, 1969 100 Migrants found by the procedures of a PGE/ ERAD/ ECP enquiry. Liberia: District of Voinjama, 1969 100 Dual collection estimation of vital events in the Lower Egypt Survey, 196566 106 Estimated vital rates from the two dual system procedures and the percentage completeness, the Lower Egypt Survey, 1965-66 107 A comparison of the dual collection vital rates with those of civil registration 109 Results of matching 1972 births and deaths in Morocco 119 Experimental matching of birth documents by characteristics and tolerance limits in the CeRED region of Morocco, 1972-73 127 Experimental matching of death documents by characteristics and tolerance limits in the CeRED region of Morocco, 1972-73 128 Matching outcomes of combinations of characteristics for birth documents in the CeRED region of Morocco, 1972-73 129 Matching outcomes of combinations of characteristics for death documents in the CeRED region of Morocco, 1972-73 130 Production matching of vital events in the CeRED region of Morocco, 197273 131
8.6 Crude vital rates per 1,000 population in the CeRED region of Morocco, 197273 131 8.7 PGE/ ER AD/ ECP estimates of vital events in the CeRED region of Morocco, 1972-73 according to various homogeneous groupings 132 8.8 The outcome of field verification of vital events reported in the urban and rural parts of the CeRED region of Morocco, 1972-73 133 9.1 Background information of PGE/ERAD/ECP studies conducted in seven African countries 146 9.2 Estimated completeness of reported births and deaths for each collection procedure in six African studies 147 9.3 Percentage distribution of estimated total births, by PGE/ ERAD/ ECP category for six African studies 148 9.4 Percentage distribution of estimated total deaths, by PGE/ ERAD/ ECP category, for six African studies 149 9.5 Stable population and PGE/ERAD/ECP estimates for Egypt 150 9.6 Stable population estimates for Morocco, 1960, 1965, 1971 151 9.7 Analytic and PGE/ ERAD/ECP estimates for Liberia 152 10.1 Estimates of completeness in the 1972 census of Paraguay 169 10.2 Estimates of completeness in the 1970 census of Korea 170 10.3 Completeness of census enumeration by age, sex, and area for Paraguay 1972 PES 175 10.4 Estimated census completeness by relationship to household head for Paraguay 1972 PES 182 10.5 Estimated census completeness by migration status for Paraguay 1972 PES 184 11.1 Persons recorded in both the census and the post-enumeration survey by age and sex: Liberia, 1974 190 11.2 Persons recorded in the post-enumeration survey by age and sex: Liberia, 1974 190 11.3 Estimated completeness of the census by age and sex: Liberia, 1974 191
xi
List of Figures
1.1 1.2 2.1 2.2 2.3 2.4 2.5 4.1 7.1 8.1 8.2 8.3 8.4 8.5 8.6 11.1
xii
Schematic presentation of the categories of events obtained by a dual collection system 6 Hypothetical examples of the outcome of three sets of matching rules: average, strict, relaxed 7 The four types of registration procedure 59 Minimum questionnaire content for record keeping and matching operations 64 Relative differences in average number of children ever born and proportion surviving for self and proxy reporters, ever married women, urban and rural areas, Mindanao Center for Population Studies, July 1972 66 Estimated completeness rates by reported month of birth, recording, and survey procedures, by area, Dual Record Study, Mindanao Center for Population Studies, January-June 1972 68 Interrelations between error and sample size for different types of survey 70 Counting rules tested in a multiplicity survey experiment for enumerating deaths, Los Angeles, July-October 1969 85 Summary results of most useful personnel by area and type of event, Morocco, July 1971-December 1972 116 Characteristics and tolerance limits used in experimental matching of birth records 125 Characteristics and tolerance limits used in experimental matching of death records 126 Events with reports in both sources by matching status in each source 137 Breakdown of reports by matching status 138 Hypothetical example: data on matching rules 143 Hypothetical example: relationship between rule A(l) and A(2) 144 The questionnaire used in the Liberian post-enumeration survey, Form PES2 194
Acknowledgements
The first thanks are due to Dr. Ivan P. Fellegi, Assistant Chief Statistician of Canada who, in his capacity as organizer of the first meeting of the International Association of Survey Statisticians in Vienna, August 1973, invited the editor of this book to organize a session on dual system estimation in demography. The original four contributors of invited papers, the two authors of contributed papers, and the four discussants, were happy to be joined later by eight other contributors and four discussants. We were all united in our interest in producing high quality data and obtaining the greatest possible analytic use from them. We may have disagreed in the pursuit of this objective, but we can only hope that in the process we helped to clarify some of the issues. Mr. Benson Morah, then a graduate student at the University of Alberta, acted as assistant editor for all the parts of the book. Miss A. Candace Fedoruk, then a graduate student at the University of Alberta and currently a Civil Servant in Ottawa, translated chapters 5 and 7 from their original French. Mr. Khalid Siddiqui, a graduate student at the Population Studies Center of the University of Michigan, worked out the tables in chapter 1. Figures 2.3, 2.4 and 2.5 were produced in the Population Research Laboratory at the University of Alberta by Mrs. Ilze Hobin. William Seltzer edited chapter 7. Finally, the appreciation of the whole international community of statisticians and demographers is expressed to the countless field workers in all five continents who laboured, often under trying conditions, to obtain more and better demographic data and in this manner increased our understanding of and knowledge about humanity, its present, and future.
xin
Preface W. Parker Mauldin
Many countries of the world do not have statistical systems that generate reasonably accurate data on population size and rate of population growth. Censuses are often one-time affairs with no continuing professional growth for planning and supervising data collection and subsequent analysis. Even so, data on population size are generally thought to be moderately good, in spite of the fact that we do not know the population of Nigeria within a figure of 10 million persons, nor that of China within 50, or possibly 100, million. But there has been an appreciable improvement in census data during the past 25 years and there is enough interest to ensure that further improvements are likely. But much time is yet to pass before adequate attention is given to systematic and scientific assessment of the accuracy of census data. Vital statistics systems are less complete and far less well maintained than are censuses. Most current estimates of the numbers and rates of births and deaths and of natural increase in less developed countries are based on sample surveys or are derived from censuses; they are not based on actual counts of births and deaths. The prospects for upgrading vital statistics systems to a moderately high level within a decade or so are not promising and, therefore, we shall continue to be dependent on census data and surveys. During the past decade or two there have been a number of promising new theories and techniques backed with some empirical data for the collection of vital statistics by one-time sample surveys. Indeed, the largest effort in man's history to collect current information on levels of and changes in fertility, the World Fertility Survey, is based on a single survey in each of about 40 less developed countries with about 19 of the more developed countries also participating in this important undertaking. Such surveys have been the norm for almost 40 years, and with recent improvements and with the care that is going into this massive effort, undoubtedly a great deal of useful information will be generated. This book explores a different philosophy, a different approach: that of collecting vital statistics and estimating population size by two independent systems, and comparing the results on a name-by-name basis. Such systems are more complicated than single systems, and typically are more costly. They are subject to a number of problems. But they have one striking advantage for the statistician who seeks certainty of knowledge as to the accuracy of results. If the two systems give somewhat different results, the statistician knows that there are deficiencies in at least one of the systems, and a field check will disclose whether the deficiencies are confined to one system or if they exist in both systems. Such a result is not comforting but it reveals xiv
the need for further improving the systems. Sometimes technicians are acutely aware that there are problems with a single system, but less rigorous technicians often do not see or understand the existence and nature of such defects. This book discusses a number of theoretical issues related to dual systems of data collection, practical problems that arise in carrying out such systems, reports in detail on selected surveys (particularly in Africa where vital statistics systems are notably weak), and summarizes actual surveys as well as the state of the art. It is an important and timely book because as the world seeks a new economic order it must have basic information about population as well as about the economy in order to measure changes, and to judge the process of progress. New York, N.Y. January 1976
W. Parker Mauldin Acting President The Population Council
XV
Introduction
The ink was not yet dry on the final draft of the PGE handbook (Population Growth Estimates: A Manual of Vital Statistics Measurement, Marks et al., 1974) when the organizer of the first conference of the International Association of Survey Statisticians, Vienna, August 1973, set up a session on dual system estimation. Five of the 12 chapters in this book were presented at the conference (Wells and Horvitz, Sirken, Pradel, Vaidyanathan, and Marks); two were presented orally in general outline (Krotki and Scott-Muhsam-Marks); and five were invited after the conference (Krishnan, Rachidi, Housni et al., Beaujot, and Marks-Rumford) to complete the subject matter. An important appendix has also been added (Nathan). The purpose of this collection of 12 chapters is to present the next stage in the development of the vastly expanding field of dual system estimation. The previous stage was wound up with the publication of the PGE handbook. At various points, but particularly at the end of chapter 1 and throughout chapters 3 (Krishnan), 4 (Sirken), and 10 (Marks), we attempt to glean an indication of future developments that are likely to take place in this field. A further purpose is to continue the clarification required to preserve the advantages of the method. The broth of the PGE method [as it is known among English-speaking professionals for Population Growth Estimation, ERAD among francophones (Estimation du Rhythme d'Accroissement Demographique), ECP among Spanish speakers (Estimation del Crecimiento de la Poblacion)] runs the risk of being spoiled by too many cooks itching to interfere with the ideal recipe and unable to buckle down to producing a superior but standard product. For example, several well documented errors have been committed in the practice of social surveying when lists of households (of increasing staleness) were used as the sampling frame instead of area samples. The PGE/ERAD/ECP technique counsels the numbering of structures as an extension and strengthening of the detailed mapping required by area sampling. Yet the critical distinction between structures and households is blurred, and alleged "refinement" is proposed in the form of listing of families and individuals (Scott, 1973b: 412). The 12 chapters fall conveniently into three groups. After the background information and articulation of some of the critical issues posed before collectors of demographic data in chapter 1, the next three chapters picture the present and some possible developments in the future state of the art in the field. The middle four chapters (five through eight) report on recent empirical evidence and actual experiences. The four final chapters deal with particularly topical and troublesome issues, on the solution of which much of the future development of the method will xvi
Introduction depend. They also report some practical application of the innovative techniques suggested theoretically in other parts of the book. Not all the topics covered in this book are of equal importance, nor have all the contributors been brainwashed by the PGE/ERAD/ECP gospel to the same extent. Consequently not all the chapters have the same status and not all of them have been included with the same degree of editorial imprimatur. However, we thought that there would be a distinct advantage in assembling declared partisans of the technique, sceptical spectators, partisans of other techniques who are downright hostile to dual estimation, and also those who tend to walk off at a tangent while professing sympathy; the latter kind is likely in the process to destroy the essence of the technique, but may — who knows — stumble into further fruitful developments. The content of the 12 chapters can be conveniently summarized. First the background story is given and the key issues are highlighted in chapter 1 — in a somewhat partisan fashion for the sake of brevity of presentation. In chapter 2, Wells and Horvitz state carefully some of the more recent results achieved and indicate some of the work being done in the area that will, it is hoped, show some promise. In chapter 3 (Krishnan) the classical textbook approach is taken of aiming at the minimum variance and/or maximum likelihood. The focus on variance is at variance with a principle PGE handbook concern to worry at all times about biases as well. The mathematics will be of no immediate utility to demographers and statisticians but they suggest some interesting possibilities in the future, and the emphasis on regression techniques in combining past and present data is innovative. Chapter 4 (Sirken) continues the well-known work of the author in multiplicity estimators and takes up their role in dual systems. The four chapters on the general review of francophone Africa and Liberia (5: Pradel), on Egypt (6: Vaidyanathan), the detailed report on Moroccan fieldwork (7: Rachidi), and the presentation of the matching procedures worked out for Morocco (8: Housni et al.) give some data that were never published in accessible sources, some data that were never published in anglophone media, and much experience and many conclusions based on actual endeavours in the field. The empirical derivation of matching rules of the Moroccan practitioners in the field reported in chapter 8 is sharply contrasted in the appendix (Nathan) with the complex and demanding principles of such a derivation based on theoretical requirements and experimental consideration. Chapter 9 (Beaujot) continues these contributions with rare data and rare experiences but, more importantly, poses a problem quite critical for the future of the PGE/ERAD/ECP: how far can we hope to go with analytic techniques while using grossly imperfect data in the estimation of vital rates. In chapter 10 Eli Marks makes a contribution in the all important field of census evaluation. As stated in section 10.4 "the theory and practice of dual system estimation for census evaluation has exhibited very little development during the past 10 to 15 years" even though increasingly competent exercises were carried out, particularly in Canada and the United States. For purposes of census evaluation all single methods of evaluation have weaknesses and a combination of dual system evaluation with demographic techniques is proposed. In an appendix to chapter 10 and in chapter 11 (Marks and Rumford) practical applications of some of the new advances carried out in Korea, Paraguay, and Liberia are reported upon and supported with original data not otherwise generally available, and not otherwise conveniently accessible. Chapter 12 concludes, somewhat unsatisfactorily, with some fireworks that are, we hope more illuminating than heat generating. Some of the issues are left hanging in the air but they will not go away. They will stay with us for a long time, until they are solved. This unsatisfactory state of the technique, and for that matter of much of the field of data collection, or rather the incomplete state of the art, has consequences. xvii
Introduction Many of our pages read like papers at an unfinished series of seminars at a university, but they cannot be finished seminar-like through further theoretical clarification. They can only be clarified through further field work on the basis of the issues defined in this volume and in the previous writings. What the editor did attempt was to clarify the avowed differences of opinions and differences of experiences among the contributors either in the form of additional endnotes in chapters or through direct textural intervention (sanctioned in each case by the contributor concerned). One non-negotiable textual intervention concerned terminology. This editor is not an academic prima donna and tries to follow previous examples whenever possible, but the situation is difficult. The POPLAB dictionary (Chanlett, 1974) does not follow the terminology of the PGE handbook. Not all POPLAB writers follow the POPLAB dictionary. French language literature tempts the reader with expressive words like "exploitation" for analysis and "confrontation" for matching, but one resists these temptations for the sake of uniformity. Our choices are given in the glossary. They fall halfway between the PGE handbook and the POPLAB dictionary. It might be useful at the outset to admit the necessity for certain peculiarities essential to the PGE/ERAD/ECP success. One of them, unreasonable to an outsider on commonsense grounds, is the insistence on not correcting data, especially not in the field. Hence the insistence on separating supervision from evaluation, on allowing no reconciliation, etc. In section 10.7 we suggest that even in census evaluation "Reconciliation is not necessary with the newer PES techniques" (post-enumeration survey). We, therefore, note with some concern that Parker Mauldin in the preface hopes that field checks "will disclose whether deficiencies are confined to one system or if they exist in both systems." He knows much about PGE/ ERAD/ ECP — that is why he was invited to write the preface — and he must remember that this is a sure way to invite collaboration between the two systems. Some of the elaborations proposed by previous writers in this field have since been found to be either empirically or, on further reflection, not very fruitful pursuits in the practical world. While in theory (Chakraborthy, 1963; Das Gupta, 1964) there is no difference between matching documents from two sources or from three and more sources, there seems to be no actual advantage in using more than two sources (Marks et al., 1974: 401-402). Probably the largest PGE/ERAD/ECP enquiry ever undertaken is that of the Indian Sample Registration Scheme. Falling under the incomplete type of a dual system enquiry discussed in section 1.8 it raises the fundamental question of how to evaluate such incomplete PGE/ ERAD/ ECP enquiries. The objection to the complete exercise apparently, is that not all biases can be removed (there is always some lingering suspicion that some biases remain). The reasoning then continues that it is better in such circumstances not to remove any. Neither the PGE handbook nor the present volume pretend to be definitive works. "Such a work could only be attempted after many of the design alternatives discussed had been systematically tested" (Marks et al., 1974: 5). When clean and definite progress has been made in almost every direction in the field of population studies and in an understanding of their problems, one segment refuses stubbornly to yield: that of vital statistics registration. It is different from the others inasmuch as for results it requires a sustained effort. PGE/ERAD/ECP also requires sustained attention and long nursing, but being more selective, and operating on a more reduced scale, it provides a viable alternative. Ann Arbour, Michigan, January 1975
xvin
KJK
Chapter 1 The Role of PGE/ ERAD/ ECP Surveys Among Endeavours to Secure Improved Demographic Data Karol J. Krotki
1.1 The need for improved demographic data The inadequacy of demographic data for the majority of humanity owing to the paucity and recency of population censuses, and even more to the non-existence or incompleteness of the registration of vital events, has caused much ingenious searching for substitute data. Another stimulus to such a search is given by the need for data required by formulators of population policies, and by designers and evaluators of family planning programmes. Both groups have been brought into existence because of the emergence of the so-called "population problem" in precisely those countries with inadequate demographic data. Demographers, particularly those working under the auspices of the United Nations, have made important contributions to the development of analytic techniques (e.g. United Nations Population Studies, 39 and 42) for the assessment and correction of faulty and unreliable data. Such assessment has often resulted in an estimation of demographic parameters, possibly and probably closer to reality than the reported data. While we are very much interested in and in sympathy with these endeavours, we show in chapter 9 the strong subjective element that enters into the estimation process and which throws into question these analytic techniques when in untried hands. There is no substitute for real data. Hence the ingenuity in the development of new analytic techniques aimed at increasing the usefulness of such unreliable data as are made available traditionally, was paralleled by ingenuity in the collection of new types of data and new methods of collecting data. The sources of demographic data for purposes of population studies can be grouped in four categories: (i) the population census, often combined with a census of housing, sometimes agriculture, occasionally fertility or some other specific, population-related topics; (ii) civil registration of vital events, such as births, deaths, stillbirths, adoptions, marriages, divorces, separations; (iii) various kinds of demographic and socio-economic surveys, either sample censuses, if this contradiction in terms can be permitted, or intensive enquiries to fill in gaps left by insufficient content of census questionnaires; (iv) recent and often unorthodox attempts to obtain demographic data in the absence of traditional data or to meet new needs usually involving special surveys, particular record keeping, and new observational techniques. This chapter is concerned with the last category, but it does not deal with such sources as electoral lists, poll-tax lists, administrative records with a demographic content, or health records.1 The chapter deals with surveys or other related arrangements specifi1
/./
Karol J. Krotki
cally made to secure more and improved demographic data. It ignores situations where demographic data become available as a by-product of other activities. Among the many new survey and non-survey sources of data, several distinct groups have become important in recent years: the service statistics from family planning programmes, KAP surveys (knowledge, attitude, practices in family planning), a group of intensive and expensive population observations, usually out of reach of underdeveloped countries (such as the national and local fertility surveys in the United States, Canada, Great Britain, and other countries, often involving panels of women pursued for years in the quest for data); highly specialized surveys such as those based on photogrammetry, called by some writers sociogrammetry; multiround surveys employing retrospective questions; multiround surveys employing the household change technique; and finally PGE/ ERAD/ ECP surveys which are the subject of this book. 1.2 An overview of the available techniques At the conclusion of the previous section in the listing of types of surveys we did not mention the single round survey. Although the most popular and most frequently used kind of survey, it has become by now somewhat discredited and it is common ground among many professionals working in the field that it should be used only in exceptional circumstances. As a rule the results are of such quality that it is probably better not to have them. One is less misled in their absence. The main reason why single round surveys give unsatisfactory results when evaluated is the unreliability of human recollection. Events of interest to the survey are forgotten or the most relevant respondents are no longer there or the events are placed in the wrong time scale. These disadvantages are compounded by the ease with which such a survey can be launched. Even when there is the weakest statistical organization, the smallest sum of money, when the purpose is not clearly understood, some kind of survey can be put together, some kind of group of interviewers can be trained and let loose onto the population. It is true that the particularly dismal results obtained from single round surveys, which will be quoted later in this section, are made to look particularly bad because of this compounding aspect. Let it, therefore, be repeated that they suffer from an inherent disadvantage, irrespective of the organization; namely the frailty of the human memory, unchecked and unsupported in conditions of a single round survey. The single round survey is also called ad hoc with the slightly pejorative implication of the phrase. "It is generally accepted that this cannot give reliable information on births and deaths" (Scott and Coker, 1971: 253). "Single retrospective surveys cannot be depended upon to provide valid or reliable estimates of births and deaths" (Mauldin, 1966: 652). "On the basis of evidence a number of those with wide experience in the developing world have concluded that single ... [round] household surveys using simple retrospective questions on births are particularly vulnerable to error" (Seltzer, 1973: 23). Similar experiences have been reported from francophone Africa. "L'humilite est certes necessaire car les difficultes sont considerables et inedites; il semble en particulier que les resultats obtenus se pretent difficilement a un traitement d'ensemble; Fimportance et la diversite des erreurs d'observation qui paraissent les effecter rendent d'autre part fort illusoire toute tentative d'adjustement general" (Blanc, 1964: 85). The first Population Growth Estimation seminar held at the Population Council offices in New York on 23 January, 1969 gathered a group of international experts and practitioners together to consider the matter. Its conclusion was that "a single system is unsatisfactory in the present state of the art in underdeveloped countries; a balanced 2
Role of PGEI ERA D/ ECP surveys
1.2
dual system is essential" (Mauldin and Bean, 1969). The language of numerous reports emanating from or through the United Nations is often diplomatic, but unequivocal is the ultimate meaning that single round surveys with retrospective questions are no good (e.g. United Nations, 1971: 157). A carefully sifted collection of evidence for the purposes of the PGE handbook (Marks et al., 1974: 54, table 2.11) has shown that the completeness of reporting deaths in single rounds in a large number of Asian surveys is hardly higher on the average (51 percent) than that of the official civil registration (49.5). It is precisely because of the inadequacy of the civil registration system that the need for alternative sources of data arises. What is the use of the alternatives if they cannot do better? For births the completeness seems to be better (67 percent) than in registration (56 percent). The range of reported completeness in the instances for which data are available is markedly wider for single round surveys than in civil registration. The reported range was 68 and 67 percentage points for the completeness of births and deaths respectively in single round surveys as against 43 and 49 percentage points in civil registration. In other words, not only was the completeness virtually the same as in the unsatisfactory civil registration, but the "product" was of a more uneven standard. "An attempt was made during the 1960 census of Morocco to obtain information on births and deaths in the last twelve months. An examination of a number of census returns suggested to census officials that these data were very incomplete" (Sabagh and Scott, 1967: 760, note 2).2 In endnote 8 to our chapter 9, the survey in North Carolina is recalled where among 3,000 households selected from birth and death registrations, only 92 percent reported to the survey the previously registered births and only 83 percent the previously registered deaths (Horvitz, 1966).3 Still, our evidence suggests that some single round surveys do less poorly than others. In other words, by trying hard some better results can be obtained in spite of the disadvantages inherent in the single rounds. At the end of section 10.5 we discuss the meaning, effectiveness, and operationality of doing a "better" job, selecting "best" enumerators, giving them "intensive training," and "reconciliation" of confusing answers and the like. This belief in the ultimate improvement of human endeavour is, of course, endearing, but less realistic than the approach of using all sources and a readiness to live in error, but measure it. To go back again to the end of the previous section, the choice between methods of collecting more and improved data for demographic purposes is, thus, limited to multiround surveys employing retrospective questions, multiround surveys employing household change technique, and the PGE/ ERAD/ ECP surveys. Multiround surveys with retrospective questions collect data more than once from the same and/ or from rotating households; their main advantage stems from the long term commitment of the organizers and the consequent cumulation of experiences and presumably gradual improvement in data. Multiround surveys recording household changes collect data at least twice from the same households; their main advantage lies in the record of household structures from the earlier round being available to the interviewers during the more recent round. The PGE/ ERAD/ ECP surveys consist of two independent means of collecting data, hence "dual collection system" and a case-by-case comparison of the reports collected; these surveys are unique in having a built-in means of evaluating the completeness rates of each of the two procedures in the dual system and, consequently, the joint omissions in both procedures. The corollary of the advantage of multiround surveys (accumulated experience and increased quality) is the risk of losing with time the financial and administrative support necessary for continuing existence. This disadvantage does not affect the consideration because, if the commitment is really that light, then the results are probably not worth having. The advantage of experience on the other hand is considered essential in countries where the evaluation of quality of data is taken seriously. 3
1.2
KarolJ. Krotki
Statistics Canada was told in the early 1970s to augment in size and content its monthly survey at the cost of — it is said — $8 million (Canadian). It was thought necessary to take many monthly rounds before the survey would be ready and public data released. In the context of our interest, such surveys have also the advantage of overlapping recall periods when taken sufficiently frequently. For example, during the original Pakistan PGE exercise, the periodic survey was taken quarterly with a recall period of 12 months. Thus each vital event had the theoretical chance of being reported four times. The "extra" information was useful in getting a solid grip on the household and helping in the matching operations (about which more later). Some of this additional value might have been due to the frequency of the survey rounds as opposed to the number of times the same event was reported. The Pakistan exercise had the benefit of two tricks of trade, small but important, as will be argued later: for three of each four visits the interviewer had the benefit of the household composition as reported at previous interviews, but not the records of the reported vital events; every fourth interview the chain was broken and a virgin listing of households and their composition had to be started. The National Sample Survey (NSS) of India tried for a long time to obtain retrospectively data on births and deaths. When each survey was taken as a "stand alone" survey, then their results were not different from the discouraging results elsewhere in the world with single round surveys. As elsewhere, the reported rates were so low that arbitrary adjustments had to be made so that published data would not offend too much against commonsense.4 In the 7th, 14th, and 15th rounds of the NSS interesting innovations were attempted (Som, 1973). The interviews were held monthly with a reference period of one year: each vital event thus had a chance of being reported 12 times, and 12 monthly consecutive interviews produced 144 observations. Each month of the year was represented 12 times. The averaging proceeded by the 12 months for which the recall period was one month, for which the recall period was two months, for which the recall period was three months and so on. An estimate of memory decay was in this way obtained, the celebrated-in-literature "Som curve" was created, and the events with "zero" recall period were estimated. The experiment was extended into a two-year recall period so that data became available for "last year" and "the year before last". If one believes that some respondents tend to advance the dates of vital events, that is, bring events from the 13th and 14th month into the 12th month then the Som curve is underestimating memory decay; and vice versa if respondents stretch out the recall period and put events further back. The two examples of Pakistan and India highlight two uses to which multiround surveys can be put usefully: strengthening the grip of the survey organization on the population, and multiplying the amount of data available for averaging purposes. A third legitimate use, matching events reported during different rounds for the purposes of the PGE/ ERAD/ ECP technique, will be discussed later on in this section. The attempt to collect data through the household change technique in multiround surveys raises other questions. The faith in this method rests on the list of members of households collected during the first round and the changes recorded in this list during subsequent rounds. It is clear that the method reduces some omissions. Persons who have died since the previous round, even the smallest babies, should be reported as having died, without fail, provided they were entered onto the list of members of the household during the previous round. The method also reduces some of the problems of dating. For some of the events that have taken place between visits the error of dating can be at most equal to the length of the interval between rounds.5 It is clear that a method "in which the enumerator merely has to check a list of names provided, obviously offers a strong temptation to the enumerator who is less than fully conscientious" (Scott, 1973: 7). Actually the problems with the household change 4
Role of PGEj ERA D/ ECP surveys
1.2
technique are more fundamental than merely tempting lazy enumerators. The first complaint is that it ties the survey to the listed households and their composition as reported at the time of the first round.6 Partly for this reason, and partly because of the interviewer's tendency to take the list of members of the household seriously, the survey works with a "panel". Such a panel starts as a not very representative sample (most samples suffer some selective omissions) and gradually becomes less and less representative of the current, live population. The problem of the de facto and the de jure definitions and their application to mobile members of the society are severe whatever the procedure adopted, but the household change technique is probably more clumsy in dealing with them than others. Similarly the reason for rotating a sample and avoiding staleness arises in all methods, but the household change technique has the additional disadvantage of "tying" the interviewer to whatever happens to have been listed in the first round. The problem is not the development of "significant respondent resistance" (Scott, 1973:7) with repeated calls on the same household. The technique tends to bias the sample, in place of throwing freely and probabilistically the enumeration net onto the population and its vital events in each round afresh. The long history of unsatisfactory results delivered by the household change technique (India, 1958-59; Cambodia, 1958-59; Brazil, 1961; Morocco, 1961-62; Indonesia, 1961-62; Nigeria, 1965-66; Tunisia, 1968-69) was not sufficiently discouraging and it was further applied in Algeria, 1970; Senegal, 1970-71; Lesotho, 1971-72; Haiti, 1972; Honduras, 1972; and Saudi Arabia, 1972. For countries where reports are available the experience was no different.7 For a probably atypical experience of overenumeration, more significant than underenumeration in births and deaths reporting, see the Moroccan multi-purpose survey of 1961-62 (Sabagh and Scott, 1967: 768). The experience is most sobering in that it dissects the complexities and unreliability of retrospective reporting, whatever the method.8 The high priest of the household change technique tried at the last moment to rescue something of it by suggesting that the changes between rounds could be done actually "in the blind" (Scott, 1973: 7).9 Having then obtained two or more listings of household members these are matched on a case-by-case basis, that is individual by individual, not just totals. However, the suggestion that the PGE/ ERAD/ ECP technique has "something in common with multiround surveys" is dismissed after a page of thinking aloud as "generally... impracticable" (ibid). The technique, should there ever be a field worker and researcher unrealistic enough to try it, involves "reconciliation" in the field. Obviously the differences between the two rounds: the original and the "blind" could not be accepted at the face value. They could merely be differences of recording and not real differences. As will be shown in sections 1.7 and 1.8 of this chapter reconciliation is a dirty word in PGE/ ERAD/ ECP parlance. For this and for other reasons the change of household technique can have nothing in common with a true PGE/ ERAD/ ECP survey. The fact that some PGE/ ERAD/ ECP surveys used the three months round is "analagous to the multiround surveys" (Scott, 1973:6) using the household change approach formally but without much expectation of meaning. We come now to the PGE/ ERAD/ ECP technique proper. Its arithmetical justification is given in the next section of this chapter. Some of its underlying principles are explained in other sections, particularly section 1.4 that summarizes the PGE handbook (Marks et al., 1974). Its history is sketched in section 1.11. Here we limit ourselves to an introduction comparable to the introductions of the other methods. Briefly, the technique consists of two procedures of collecting data and a case-bycase comparison of individual reports. From this comparison or matching of records, three kinds of records result. There are records in respect of events reported by both procedures (two records for each event, one from each procedure); records for events reported by one procedure only (one record for each event, all from one procedure);
5
Karol J. Krotki
1.2
and records for events reported by the other procedure only (one record for each event, all from the other procedure). Provided that the two procedures and sets of events were independent of each other we are then justified, in accordance with section 1.4, in making an estimate of events with regard to which neither procedure produced a record. This picture is summarized in figure 1.1. Figure 1.1 Schematic presentation of the categories of events obtained by a dual collection system Procedure R (- continuous recording) Events observed
Events not observed
All events
N(RS) = M
U(S)
N(S)
Events not observed
U(R)
U(R)U(S)/M = Z
V(S)
All events
N(R)
V(R)
N
Events observed
Independence between the lines and independence between the columns implies the following relationships: For procedure R Pr(R) = N(R)/N = M/N(S) = U(R)/V(S) For procedure S Pr(S) = N(S)/N = M/N(R) = U(S)/V(R) In short, the probability of inclusion is the same whether one talks about all events, or events caught by the other procedure, or events omitted. The problem of independence is sounded repeatedly throughout this chapter and in fact throughout all the other chapters. Particularly in sections 1.6 and 1.7 we show how simple devices can increase independence, and in section 1.8 how easily thoughtlessness can destroy it. Pradel in chapter 5 seems to be less convinced of the possibilities of ensuring independence and right to the very end of chapter 12 we are concerned with it. The matching problem is simpler and is negotiable. It is obvious that if we work with very strict matching rules we will end up with few cases in the first category, many in the other two categories, and consequently with a large estimate of the fourth category. On the other hand, if we set up lax matching rules, we will finish with a large first category, two small lots in the second and third category, and consequently a small estimate for the fourth category. The outcome of a hypothetical case with three sets of matching rules has been summarized in figure 1.2. Many features of carrying out a survey are applicable to PGE/ ERAD/ ECP surveys in the realm of sampling theory, sampling practice, questionnaire design, field work, office procedure and the like. The two needs of high independence and matching rules with a zero net error are peculiar to this technique. For that price, we obtain a method that is selfchecking. None of the others are. While further features of the PGE/ ERAD/ ECP technique are elaborated in other sections of this chapter and, indeed, in other chapters, it will be useful at this stage to return to the question of matching between rounds. The PGE handbook presents in chapter 4, and in some detail, three alternative systems. One of these is a comparison between two field procedures very similar to each other, namely single round surveys. 6
1.2
Role of PGEI ERA D/ ECP surveys
Figure 1.2 Hypothetical examples of the outcome of three sets of matching rules: average, strict, relaxed. (One procedure observed 840 events, the other 850) Average Recording
Observed Not observed
Observed
Not observed
790
60
50
4
estimate of N = 904 (± 4 at 95%) Strict Recording
Observed Not observed
Observed
Not observed
750
100
90
12
estimate of N = 952 (± 8 at 95%) Relaxed Recording
Observed Not observed
Observed
Not observed
820
30
20
1
estimate of N = 871 (± 2 at 95%) Single round surveys follow each other at, say, every six months with a recall period of, say, twelve months. In this manner, each vital event should be reported twice, once in each of two neighbouring surveys. Other combinations of frequencies and recall periods are possible as long as sufficient overlaps occur. The question then arises, why are single round surveys acceptable in this case? Why is not the sauce of the goose of PGE/ ER AD/ ECP good enough for the gander of the "stand alone" single round survey or at least for the household change technique? Primarily, the third example of the PGE handbook leans over backwards in stressing the need for independence and suggesting innumerable tricks of the trade to achieve it (Marks et a/., 1974, e.g. 250253,425-428). Even so, between-round comparisons in the Pakistan PGE would not have been worthwhile because of the lack of independence between the rounds of the survey.10 In fact, we do not have enough experience in the world to talk with confidence about such an approach (survey with survey). The frequent and popular examples of 7
1.2
Karol J. Krotki
PGE/ ERAD/ ECP systems combine typically survey with civil registration (first example in the PGE handbook: 227-228) or survey with special registration (second example in the PGE handbook: 238-250). The Director-General of the next Central Statistical Office that will undertake a PGE/ ERAD/ ECP survey, can perform a real service to the international fraternity of statisticians, and probably to himself and his country, by trying a really well-founded third example suggested in the PGE handbook. We are aware of many other possible approaches in the field of measuring vital events. Those particularly hurtful are taken up in section 1.8. There is merit in others, but for the given outlay per hour of time and per unit of money they repay less than a correct PGE/ ERAD/ ECP survey. Among other approaches handicapped from the start we would list the following: pregnancy histories, expensively elaborate in their interview dynamics and resulting in longitudinal data unwieldy for traditional methods of analysis which are mostly cross sectional (Bogue and Bogue, 1967; Bogueand Bogue, 1970); maternity histories, less troublesome and somewhat more realistic than pregnancy histories, but also resulting in data surplus to available means of analysis; sample registrations following civil registration principles (Hauser, 1954; Cavanaugh, 1963) because of the basic incompatibility between the type of personnel, attitudes, finance, and a host of other considerations relevant to the establishment of a legal system and a statistical system (Linder, 1971); and panels of respondents (Vaidyanathan, 1973) because of the staleness of the sample increasing with time. 1.3 The theory behind the PGE I ERAD I ECP technique The comparative novelty of the PGE/ ERAD/ ECP technique does not lie in a new principle. In fact, the technique is based on a proposition in probability that has been with us for a long time. If two events take place independently of each other then the probability that they will take place simultaneously is equal to the probability of one of the events happening multiplied by the probability of the other event happening. In terms of the notation used in figure 1.1, if R takes place independently of S then the probability of R and S both taking place simultaneously is equal to R times S. The algebraic formulations are well known (e.g. Krotki, 1969; Krotki, 1971): the probability of (R and S) = probability of R times probability of S (1.1) where probability of R = R/(R + non R) (1.2) probability of S = S/(S + non S) (1.3) Let it be recalled once more that equation (1.1) is true on the condition that the occurrence and frequency of R has no effect on the occurrence and frequency of S and vice versa. Equation (1.1) can then be rewritten as follows: Pr(R) = [Pr(R and S)]/[Pr(S)] (1.4) The three terms of equation (1.4) can be redefined as follows: Pr
Role of PGE/ ERA D/ ECP surveys
1.3
M = number of events observed by both procedures. The notation from equation (1.4) can now be replaced by the new notation from equations (1.6) and (1.7) and we obtain: Pr(R) = ( M / N ) / (Ns/N) = M/N S (1.8) Supposing we could obtain through field work the exact value of M and N (or Ns), a subject with which most of this book is concerned, equation (1.8) would give us the completeness rate of procedure R. Combining algebraically equations (1.5) and (1.8) we obtain:
NR/N = M/N S
Carrying out the same exercise for the other procedure we obtain:
N S /N = M/NR
(1.9) (i.io)
The left hand sides of equations (1.9) and (1.10) give the completeness rate of procedures R and S respectively (indicated in the numerator). The completeness rate (see: Glossary) of a procedure is the proportion of all events (births or deaths) that were reported by the given procedure. The right hand side of the two equations gives the match rate of procedures S and R respectively (indicated in the denominator). The match rate (see: Glossary) is the proportion of events (births or deaths) reported by one procedure that were also reported by the other procedure. Equations (1.9) and (1.10) show, then, that the match rate of one procedure in the PGE"/ ERAD/ ECP system is equal to the completeness rate of the other procedure. More generally, in a dual collection system, the match rate of either procedure is equal to the completeness rate of the other procedure.11 Clearly a loss of independence will increase the match rate and consequently increase and overestimate the completeness rate. With full loss of independence when, for example, the records of one procedure are copied for the purposes of the other procedure, all records will be matched and the estimated completeness will be 100 percent. From the two equations (1.9) and (1.10), that give the completeness rate for either procedure, we can obtain, through a minor transformation, an estimate of the total number of all events: N' = (N R N S )/M (1.11) where NR, Ns, and M have the same values as defined for the purposes of equations (1.5), (1.6), and (1.7) and where N' = estimate of all events (or N) Equation (1.11) is an old friend and has been with us not only for the last two pages of discussion but more generally for centuries in the field of probability. It is popular and has been used for many different purposes. In order to respond to the special needs of PGE/ERAD/ECP estimation we have introduced into figure 1.1 some new notation: N R = M + UR
(1.12)
Ns = M + Us (1.13) where NR, Ns, and M continue to carry their earlier meanings, while UR = events observed by procedure R only; Us = events observed by procedure S only. One can now rewrite equation (1.11) in terms directly suitable for the PGE/ ERAD/ ECP technique: N'= [(M + UR) (M + Us)] / M (1.14) By simple transformation we obtain: N' = (M 2 + U R M + U S M + U R U S ) / M (1.15) N ' = M + U R + Us + (U R U s )/M (1.16) Equation (1.16) carries a clear message. It is couched in terms of the celebrated four 9
1.3
KarolJ. Krotki
categories of the PGE/ ERAD/ ECP estimation process. The four categories are: (i) events caught by both procedures, (ii) events caught by one procedure but not by the other, (iii) events caught by the other procedure but not by the first one, and finally (iv) events missed by both procedures. The last category is the proverbial "fourth category" of the PGE/ ERAD/ ECP literature. The four categories have been shown schematically in figure 1.1. The estimate of N, which is N' as defined in equation (1.14), experiences sampling variations. The variance of the estimate, as a first approximation, can be estimated as follows: aN2 = N{[l -Pr(R)][l -Pr(S)]}/[Pr(R)Pr(S)] = (NZ)/M (1.17) where Pr(R), Pr(S), N, M have been defined earlier and where Z is the "fourth category" defined in equation (1.16). It is clear that the variance will be smaller and the estimate of N will be more precise the nearer to unity are the values of Pr(R) and Pr(S). The smaller the fourth category the smaller the variance, until in the limiting case when the fourth category dwindles to zero the variance disappears altogether. For this to happen, the second and third category will also tend to disappear. In other words, the reliability of the sample is inversely proportionate to the completeness rate and the match rate of both procedures. 1.4 The PGE handbook and some essential PGE I ERAD I ECP features The PGE handbook has been written to provide a solution to the inadequacy of registration data and retrospective surveys data. While the PGE/ ERAD/ ECP technique itself might use retrospective survey data, it does not rely on them exclusively. It uses them merely as one component in a system of other procedures that crosscheck and evaluate each other. It is recognized that other techniques and approaches are available. Some are analytic and require no collection of data(e.g. United Nations Population Studies 42), but they cannot function when data are subject to more than maximally acceptable margins of error. Some are directed towards the correction of errors on the basis of a taxonomy of errors arising traditionally (e.g. ibid; or Som, 1973), but they are subject to the uncertainty as to which feature is correcting which and the possibility that in some societies the anthropological reasons causing a typical set of error patterns work the other way round. Some rely on intensive concentration of effort either at the interview (e.g. Bogue and Bogue, 1967) or in the setting up of the field work (e.g. Arretx and Somoza, 1965). Those with disappointing experiences in field work might find this faith in the efficacy of human effort quite touching. Some of the suggestions rest more on assertion than actual experience. Until tried under difficult conditions they are no more than postulates. In any case, every one of these approaches is, in comparison with the PGE/ ERAD/ ECP technique, short of the built-in measure of reliability that works automatically when set up properly. It is because of this unique feature that the 500-page handbook has been written. However, this unique feature is not a robust panacea. It can be fully useful only if put into wider context with other important and underlying principles. These were seven in number (Marks et al., 1974:7): i. No one design is optimal, either theoretically or practically, for all situations; ii. Certain underlying principles of good design exist and can be specified; iii. A theoretically sound design is worthless if it is either impractical to implement or likely to be implemented improperly; iv. No matter how carefully a study is designed, its execution will be flawed—in other words, no set of data will be error free; 10
Role of PGEj ERA D/ ECP surveys
1.4
v. Producers of statistical data have an obligation to investigate and report on the amount of error that may be present in the data they collect; vi. Any given design, good or bad, can be improved on the basis of administrative experience and the statistical evaluation of results; vii. Any given study is a system of interrelated parts, and an improvement of one feature of the design without considering its effect on the balance of the design is wasteful and undesirable. The last principle, in particular, led to another feature in the handbook: the careful balancing of errors rather than their elimination. To realize that one lives in an error-full world is not a characteristic requirement of the PGE/ ER AD/ ECP approach alone, but with PGE we have an increased chance and actual means of doing something about it. With regard to sample design, the continuing concern to balance errors leads to the realization that ceterisparibus, increasing the number of primary sampling units (psu), will have two counterbalancing effects: it will decrease the sampling errors of the estimates; it will also, ordinarily, increase the nonsampling errors of these estimates. It is the manager's responsibility to decide whether at the end he will be better off or worse with a larger number of psu's. In other words, a sample designer thinking in terms of the sampling errors will tend to have many, small clusters. A sample designer impressed with nonsampling errors will tend to have fewer and larger clusters. We give a relevant example including some cost considerations in section 1.10. An empirical investigation of the impact of cluster size in Liberia showed the interrelatedness of the statistical considerations with the question of independence so critical to the PGE/ ERAD/ ECP approach. The completeness rate of reported births, deaths, and infant deaths decreased with cluster size in the sample (Rumford, 1971) but the estimated values increased with cluster size. There is no reason to think that the smaller areas had smaller fertility, mortality, and infant mortality. Either matching was more difficult in larger areas and the same matching rules produced an overestimate of the events or there were losses of independence in small areas that resulted in an underestimate of events. A comparison with analytic results suggests the latter. Pradel in chapter 5 shows that the correlation leading to loss of independence is of two kinds: correlation due to the nature of the population and correlation due to communication between the two procedures. An example of the first one would be field workers of two religious groups tending to work in the households of their coreligionists. An example of the second would be field workers of the two procedures comparing notes in the evening or using the same faulty map. The religious separation would result in an overestimate of events, the communication between the field workers in an underestimate of events (though the faulty map could work either way, depending on whether the estimating personnel at headquarters worked under the same misapprehension). One somewhat unusual feature of the PGE/ ERAD/ ECP type of matching is that it need not get each match "right" as is necessary, for example, in the case of matching purposes of record linkage in administrative tasks. There is no harm if document AI is matched with document 82 and document A2 is matched with Bi, where letters of the alphabet refer to events and subscripts to PGE/ ERAD/ ECP procedures, but one would not like to have too many such gross errors even if resulting in zero net error. High gross errors, even if balancing, probably increase variances and give an uncomfortable feeling about the reliability of the whole operation. It could be that the discussant's unhappiness with this approach to matching in chapter 2 is traceable to the responsibilities he carries in his professional life, which are different from those of PGE/ ERAD/ ECP workers in less developed countries whose task is to fill in some of the demographic lacunae.12 11
1.4
KarolJ. Krotki
Much of this section may have appeared to the uninitiated in PGE/ ERAD/ ECP mysteries, as confidential exchanges between members of a close fraternity. For this reason, in the next section, a brief outline of an "ideal" PGE/ ERAD/ ECP exercise is provided. It is a highly abridged and systematized picture. It is not intended for reading by experts familiar with the issues.
1.5 An "ideal" PGE I ERAD I ECP exercise It will be based on a national sample. The dearth of data, and even more crucial, the shortage of personnel capable of carrying out reliable exercises, is such, that it is wasteful to engineer experimental investigations and local enquiries. The additional cost of carrying out a nationally representative sample is usually marginal. Often, the most expensive parts of the population need not be included in the sampling frame, because their exclusion is unlikely to affect the national estimate. For this reason, the Chittagong Hills in East Pakistan(now Bangladesh) and Baluchistan in West Pakistan were excluded from the Pakistan PGE (Pakistan, 1968). Similarly, the Saharan fringe in Morocco could be excluded (Fellegi, 1973) without loss to the estimation procedure and with considerable savings in terms of personnel and money. A national sample will secure continuing support from such clients of national data as the State Planning Commission, the Central Statistical Office, the Ministry of Finance, and the Department of Health and Welfare. If there is genuine desire for experimentation with new approaches, new techniques, or new questions, these can usually be drafted onto the main exercise in the form of rotating questions and the like. The sample will be drawn by a specialist in the field (e.g. Fellegi, 1974). Meticulous attention will be paid to the mapping, with various scales required for unambiguous field operations (e.g. Cooke, 1970). All structures in the sample areas, whether inhabited or not, will be numbered through the PGE headquarters regardless of the fact that in some parts some local "addresses" might exist (Markset al., 1974:173-177). No money or effort will be saved on these three counts. In all other respects, and with regard to lesser considerations, the PGE handbook will be followed closely unless there are clear indications to the contrary. No departures from its recommendations and experiences will be taken in ignorance. Typically the two procedures might involve a specially-set-up registration by residents or visiting recorders who continuously (once a month or every fortnight) enquire about vital events. Their rounds are highly structured so that the frequency of their visits does not depend on their zeal. The other pf ocedure might be a household survey every six months with an overlapping recall period of 12 months. The two procedures are organizationally independent of each other, except through the Director-General at the headquarters. Their documents are different, the emphasis of their enquiries is different (vital events in the case of recording, and household structure in the case of the survey), even the order of the questions on similar topics is different. The recorder's documents are withdrawn just before the survey visit, so that even if collusion was intended it is not possible. However, the main point of the organization should be that collusion is not worth anybody's while. Not only are the two procedures almost entirely different, but the results from one are not used to correct the others, even less used to "improve" the steps of the other procedure. There is as elaborate a system of supervision within each procedure as sound principles of management might require. There is complete separation between supervision under the procedure director (correcting the mistakes found and redoing the whole job if sequential quality control so demands) and evaluation 12
Role of PGEI ERA D/ ECP surveys
1.5
(when errors are measured, but not corrected), the latter exclusively under the impartial Director-General. The case-by-case comparison is done through objective matching rules and, incidentally, always by hand (the heaps of documents for matching are never high enough to justify mechanical matching). A complete PGE/ ERAD/ ECP estimate is arrived at and published with all the attendant data swiftly. The managers will think in terms of total cost and total errors. They will seldom agree to a sample very different from 100,000 persons; a smaller sample would not give enough vital events and a larger one is unwieldy administratively and is unnecessary from the point of view of estimating annual vital rates of any typical population. They will seldom have clusters smaller than 1,000 persons, however much the highly skilled and highly paid statistical expert talks about intraclass correlations, because consideration of Mean Square Error would warn them against it. They would seldom, if ever, have sample clusters greater than 5,000, however attractive that might be administratively, because with larger clusters the intraclass correlation might truly become important. They would also be anti-intellectual in the sense that they would frown on overelaborate suggestions and requests for experimenting, when perfectly commonsensical and well-tried approaches are available. They would probably come out against subsampling of one procedure by the other.13 They would be impatient with elaborate matching procedures suggested by egg-headed back room boys and they could well afford much impatience if all the structures in their clusters were meticulously numbered.
1.6 An overview of the world uses of the PGE I ERAD I ECP technique In chapter 2 of the PGE handbook a summary of the uses of the PGE type of estimation has been provided up to and including the time when it was necessary to lock up the draft. Since then, further information has become available from the POPLAB survey in Morocco reported in our chapters 7 and 8; new data from the sister surveys in Colombia and the Philippines are presented in chapter 2; an African overview with some new African data and references is given in chapter 5; an Egyptian exercise is reported in chapter 6; some Malawi and Madagascar data are given in chapter 9. In table 1.1 we present a world summary of such involvement in the PGE/ ERAD/ ECP approach, country by country, as it was possible to identify on the basis of data available. Needless to say, the data are not strictly comparable as the large number of footnotes to the table testifies. Any PGE/ ERAD/ ECP involvement, however marginal, secured a country a place in table 1.1. When countries are excluded that never had a national PGE/ ERAD/ ECP exercise, then the proportion of regional populations given in table 1.1 dwindle in the manner summarized in table 1.2. Another reason for exclusion from the imposing proportions in table 1.1 would be national endeavours undertaken in the PGE/ ERAD/ ECP spirit, but then not followed through for one reason or another. For example, in the 1969-70 Algerian national sample for the purposes of a PGE/ ERAD/ ECP exercise 27 clusters were selected, but only in nine was the field work necessary for ERAD estimation started (Algeria, 1971: 624-27, 76-89). No report on the exercise is available. On the other hand, and to use the same country as an example, matching was carried out between the three rounds of the national enumeration through a re-copying procedure (ibid., 91) so that administrative control could increase the grip of the survey organization on its population, and possibly to establish longitudinal household data without, however, following the ERAD requirements. 13
Table 1.1 An overview of the known world involvement in PGE/ERAD/ECP techniques Study name and area within country (3)
Main purpose of the study (4)
Description of first procedure
Description of second procedure
Approx. population covered
References to reports published
(5)
(6)
(7)
(8)
Continent and country
Period covered
(1)
(2)
AFRICA Algeria
1969-70
Etude Statistique Nationale de la Population
a
Egypt
1965-66
Study undertaken by Cairo Demographic Centre and Egyptian Central Agency for Public Mobilization and Statistics-rural lower Egypt
b
Three semi-annual surveys
Continuous recording
100,000
Kenya
1973-76 (?)
Study by Demographic Studies Unit — seven contiguous districts
c
Semi-annual and annual surveys
Continuous recording
93,000
Bahri, 1969
Chapter 6 of this book
Estimated 1972 population in thousands (9) 364,000 15,270
34,839
Myers and Lingner 12,067 1973
Liberian Fertility Survey, round one — * whole country
a
Semi-annual survey Monthly survey by by staff supervisor local part-time recorders
Madagascar 1967-78 1969-70
Study by Service Statistique de Madagascar (INSRE)— Ambinanitelo (commune), Ankazoabo (prefecture)
b
Multiround survey
Malawi
1971-72
Malawi Population Change Survey1*
a
Morocco
1972-73
Study by Centre de Recherches et d' Etudes Demographiques — Northern Morocco coast to desert
a
Rural
c
Liberia
Tanzania
1969-70
Urban 20,000 Rural 50,000
Rumford, 1970 and 1972; Liberia, 1971a, 1971b, and 1971c
1,571
38,000
Chapter 5 of this book
6,750
Semi-annual survey Recording
30,000
Blacker, 1971 Kazeze, 1974
4,666
Serni-annual survey
84,000
Myers and Lingner, 1973; Chapter 7 of this book
15,286
Marks et al., 1974: 48
13,996
Civil registration
Continuous recording
X
Table J.I Continent and country
Period covered
(1)
(2)
Tunisia
1968-69
AMERICA, NORTH Barbados 1960
Study name and area within country (3)
Main purpose of the study (4)
Description of first procedure
Description of second procedure
Approx. population covered
References to reports published
(5)
(6)
(7)
(8)
Civil registration
5,000
Tunisia, 1970; Vallin, 1971
Estimated 1972 population in thousands (9) 5,377
Enquete Nationale Demographique — Oued el Khatef and Goraa (Cheikhats)2
a
Household survey of three rounds
Study in association with 1960 Eastern Caribbean population census — *
b
1960 census special Civil registration schedules for infants under 3 months of age
Byrne, 1966
233,000 240
21,848
Canada
1931
Canadian birth registration study3. All provinces excluding two territories
b
Census enumeration Civil registration enumeration of 1931 of infants under 1 year of age
Tracey, 1941
Dominica
1960
Study in assoc. with 1960 Eastern
b
1960 census special Civil registration schedules for infants
Byrne, 1966
73
Caribbean population census—*
under 3 months of age
Study in assoc. with 1960 Eastern Caribbean population census — *
b
1960 census special Civil registration schedules for infants under 3 years of age
Byrne, 1966
115
Trinidad 1960 and Tobago
Study in assoc. with 1960 Eastern Caribbean population census — Tobago
b
1960 census special Civil registration schedules for infants under 3 years of age
Byrne, 1966
1,043
United States
1936
Study conducted by Bureau of the Census —26 counties of Georgia4
b
One time retrospec- Civil registration tive household survey
Hendrich et al, 1939
208,841
1936
Study conducted by Bureau of the Census —26 counties of Georgia
b
Mail-out /mail-back postal survey
Civil registration
Hendrich et al, 1939
1937
Study conducted by Bureau of the Census- Maryland
b
One time retrospec- Civil registration tive household survey
Hendrich et a/., 1939
St. Lucia
1960
Table LI Continent and country
Period covered
(1)
(2)
Study name and area within country (3)
December 1939March 1940
United States birth registration Study — * 48 states and District of Columbia
Jan.March 1950
Description of first procedure
Description of second procedure
Approx. population covered
References to reports published
(5)
(6)
(7)
(8)
b
Census reports of infants less than 4 months old
Civil registration
Grove 1943
Infant Enumeration Study—* 48 states and District of Columbia
c
Census reports of infants less than 3 months old
Civil registration
U.S. Bureau of the Census, 1953
Jan.March 1950
United States birth registration test study* —48 states and District of Columbia
b
Census reports of infants less than 3 months old
Civil registration
Shapiro, 1954
1965
Study by the Research Triangle Institute—seven counties of North Carolina
c
Household survey with two recall periods5
Civil registration
Horvitz, 1966
Main purpose of the study (4)
Estimated 1972 population in thousands (9)
1965
AMERICA, SOUTH 1967Chile 1968 Colombia
1971
ASIA Bangladesh9 Jan. 1962 Dec. 1965
Study conducted in areas with poor registration in 1950-* sample universe consisted of 50 states and District of Columbia
b
Household surveys (CPS and HIS)6
Civil registration
Koons, 1971
Valdivia pilot study Valdivia commune7
b
Multiround household survey
Civil registration
Tacla, 1969
Pilot study for the Colombian Pop lab — one of Colombia's 16 departments
e
Retrospec-survey
Monthly household 60,000* canvass in some areas and continuous recording in other areas
Colombia, '71; Chapter 2 of this book, CIMED, 1973
Population Growth Estimation (PGE)-*
a
Continuous Household survey every three months recording with one year overlapping recall periods
42,000
Pakistan, 1968 Farooqui and Farooq, 1971
300,000 10,045
22,491
2,154,000 60,675
Table 1.1 Description of first procedure
Description of second procedure
Approx. population covered
References to reports published
(5)
(6)
(7)
(8)
Continent and country
Period covered
(1)
(2)
India
1945
Singuar Health Centre, Bengal
e
One time household Civil registration survey with approximately 2 year recall
64,000
Chandrasekaran and Deming, 1949
1946
Singuar Health Centre, Bengal
e
One time household Civil registration survey with approximately 1 year recall
64,000
Chandrasekaran and Deming, 1949
1950-
Mysore Population study, three rural zones
d
One time household Civil registration survey with recall period varying from about 1 to 2 years
23,300
United Nations, 1961
1950-
Mysore Population study, Bangalore city
d
One time household One time fertility survey with recall attitude survey with period varying from female interviewer about 1 to 2 years
1,011
United Nations, 1961
1964
Gujarat Sample
e
Household survey
36,000
Merita and
Study name and area within country (3)
Main purpose of the study (4)
Special registration
Estimated 1972 population in thousands (9) 563,490
registration scheme (pilot) — rural
conducted by supervisor every 6 months
by local resident part-time registrar
Shah, 1966 and 1968
19641966
Vaso (pilot) — urban Gujarat
e
One time household Civil registration survey with recall period of up to 28 months
19631966
Mehrauli pilot study. Five villages in Mehrauli block, South Delhi
e
Household survey conducted every 6 months by female interviewer
Special registration 5,582 by part-time non-resident registrar
Murty and Jain, 1967
1965
Kerala sample registration scheme
e
Household survey every 6 months by supervisor
Special registration by local resident as part-time registrar
260,000
Kerala, 1967 and 1968
Korea
19631965
Taeduck Gun matching project
b
One time household Civil registration survey with recall period of up to 2 years
100,000
Choe, 1967 and 1968
32,527
Pakistan10
19621965
Population Growth Estimation (PGE)*
a
Continuous recording 42,000 Household survey conducted every 3 by full-time recorder months with 1 year living in the area overlapping recall periods
Pakistan, 1968, Farooqui and Farooq, 1971
56,065
9,480
Mehta, 1967; Mehta and Shah 1968
Table 1.1 Continent and country
Period covered
(1)
(2)
Philippines
1972
Mindanao Center for Population Studies — two research areas
Thailand
19641965
Turkey
19651966
Description of first procedure
Description of second procedure
Approx. population covered
References to reports published
(5)
(6)
(7)
(8)
e
Periodic household survey
Continuous recording
Survey of Population Change (SPC)*
a
Household survey conducted every 3 months
Civil registration
Turkish Demographic Survey (TDS)-*
a
Household survey conducted every 6 months by staff supervisor
Continuous recording by part-time local recorder
Study name and area within country (3)
Region 1, urban Region 1, rural Region 2, urban Region 2, rural Ankara City Istanbul City Izmir City
Main purpose of the study (4)
Estimated 1972 population in thousands (9)
46,000
Madigan, 1973
39,040
172,000
Thailand, 1967
36,286
Rumford et al., 1968 Turkish Demographic Survey, 1967 and 1970
37,010
32,500 38,100 29,700 37,600 24,900 15,600 13,400
U.S.S.R. U.S.S.R.
1939
Vostrikova 1963
1959
Study by Central Statistical Board
b
Ten percent Civil registration systematic sample of village Soviets
1970"
ditto
b
ditto
ditto
Dmitrieva 1969
Column (3) *. Sample intended to be nationally representative. Codes used in column (4) a. To estimate basic demographic parameters. b. To assess completeness of civil registration. c. To assess completeness of reporting of enumeration procedure. d. To investigate interrelationship between population and economic and social change. e. Demonstration or pilot project to establish system for estimation of missing demographic data. Footnotes 1. 2. 3.
for various columns Sample design used was inappropriate to local conditions (Blacker, 1971). Non-randomly selected Cheikhats. Somewhat different sampling and matching procedures were used in various provinces, and precise details of certain aspects of the study are not available. 4. Sample under the household survey was not probability sample due to budgetary constraints. 5. This study was designed to test accuracy of retrospective questions on births and deaths in survey and also to compare two recall periods. 6. CPS = Current Population Survey; HIS = Health Information Survey.
247,459 247,459
7. 8. 9. 10. 11.
Civil registration records were available only for three-quarters of 191 sampled villages, so that estimates were biased. A guestimate by the author. No figure given in CIMED, 1973. Formerly West Pakistan. Formerly East Pakistan. It was announced that procedure of 1959 study would also be employed in connection with 1970 census. (Dmitrieva, 1969.)
1.6
Role of PGEI ERA D/ ECP surveys Table 1.2 Summary of world involvement in PGE/ERAD/ECP Population of countries who ever had a PGE study on national basis
%
(4)
(5)
(6)
1,447,115
38
405,542
11
109,822 232,160 32,536 825,097
30 100 11 38 0 0 100
6,237 209,269
2 90 0 9 0 0 0
Region
Population of region
Population of % countries who ever had a PGE study
(1)
(2)
(3)
World
3,787,700
Africa Northern America Latin America Asia Europe Oceania U.S.S.R.
364,000 233,000 300,000 2,154,000 469,000 20,200 247,500
247,500
190,036
N.B. Population totals are in thousands and refer to 1972. Source: 1. Col. (2): United Nations, 1973. 7972 Demographic Yearbook, New York, United Nations 2. Cols (3) and (5) are summaries from table 1.1 Finally, to increase the meaning of table 1.1 one would have to provide similar tabulations for single round surveys (14), multiround surveys with retrospective questions (1), multiround questions with the household change technique (3), and possibly other surveys, such as panels (2). This would be a formidable task and would result in a bulky product. However, a rough indication may be provided by the summary of literature. In a random search, during the time necessary to find at least one reference for each type of survey mentioned above (in parentheses are given the numbers actually found), no less than 14 references were found for the single round survey. Presumably all types of surveys are beaten for the first place and by many lengths by the least reliable, least elegant, often useless type of survey, namely the single round survey. 1.7 Modes of securing the advantages of the technique In this section are listed such tricks of the trade as are particularly applicable to the PGE/ ERAD/ ECP technique. Some are merely elaborations of the "ideal" presented in section 1.5. They centre mainly on the two outstanding features of the technique: the need for independence and the requirements of matching. However, several other pointers relevant not exclusively to the PGE/ ERAD/ ECP technique are also added. Some of these points, and their counterpoints in the next section would necessarily find their way to a minimum qualifying list for a PGE/ ERAD/ ECP franchise.14 1.7.a With regard to the need for independence the following comments can be offered: i. The first and fundamental requirement is that the two procedures in the dual system estimation be as independent of each other as possible; the principle has already been indicated in sections 1.3 and 1.5 and a long list of suggestions, practicable and in 25
1.7.a
KarolJ. Krotki
most cases well-tried, are given in the PGE handbook; the whole book is shot through with these concerns so no particular page reference can be given, but one section is particularly devoted to this topic (7.F; pp. 425-428). ii. In the next section we give examples of how independence can be destroyed, some hypothetical, most from actual practice. There is one positive suggestion that can be made: the two procedures in any one PGE/ ERAD/ ECP system should have no common organization, no common supervisors (except the Director-General at the top), no mutual checking arrangements or reward systems. 1.7.b A separate manual could be written for matching, but some principles deserve to be pointed out even in a brief chapter. i. There should be articulate matching rules. Experience has shown that commonsense checking ("Surely this is the same birth/ death"), leads to errors that are systematic, non-random, and biasing the results in an unknown direction. ii. Experience has shown repeatedly that dwelling numbers (better still structure numbers irrespective of whether inhabited or uninhabited, whether inhabitable or uninhabitable) are the matching item with the strongest discriminating power. As reported in section 7.5 it is poor economy to try to use existing administrative numbers in such parts of the sample as might already have numbers. iii. The emphasis on matching and the amount of resources devoted to matching should be related to the degree of the completeness of the reporting of vital events. With full completeness no matching would be necessary: all events would fall into the first category and none into the second and the third categories; consequently, the estimate of the fourth category would be zero. The importance of the net matching error increases with the degree of omissions (Fellegi, 1974: 21). iv. Field follow-up of matching decisions or matching doubts are necessary, not so much to avoid or cut down gross errors, but to check empirically that matching rules result in zero or near-zero net error and that there has been no change since the rules were established. 1.7.c Then there are arrangements that differentiate good field work and superior office organization from poor field work and sloppy office organization, not necessarily peculiar to the PGE/ ERAD/ ECP approach, but important probably in most types of surveys. They may read simplistically, but experience has proven their critical importance. i. There should be a structured routine round for field workers, so that they know at any given time in which part of their area they should be working; their supervisors would then know where to find them. ii. A very considerable amount of managerial time and of the monetary budget will be spent on mapping. The senior levels of maps at least, will be based on topographical maps or aerial photographs, though junior maps may be based on the senior maps produced for the purposes of PGE/ ERAD/ ECP. iii. There will be a routine for the orderly recording of changes on maps and other documents in the organization that have been produced in quantity. iv. One of the procedures will usually require frequent calls on the respondent. Within the taxonomy suggested in figure 2.1 this will be the "active" seeking of events. To avoid respondent fatigue and in circumstances where social ties are strong and social cohesion considerable (read: "predilection to gossip mongering high") every second or third household only need be called upon at every round, but which household during which round acted as the informant for its small neighbourhood should be explicit. It is likely that in most urban areas calls on all households will be desirable.
26
Role of PGE I ERA D/ ECP surveys
/. 8
1.8 Inadvertant modes of destroying the advantages of the technique This section is devoted to the listing of arrangements that with the best of intentions are bound to destroy the fundamental or ancillary advantages of a PGE/ ERAD/ ECP system. Some are based on illogicality or ignorance. Some of them, for ease of identification, have been given geographical labels. Often this is to the great credit of the respective organizers, because they reported their findings so that others could avoid the same pitfalls. 1.8.a Matters affecting independence i. Field workers of one of the procedures being charged with supervisory responsibilities in the other procedure (the early Indian procedure). ii. Verification of documents in the field to make certain that they do or that they do not cover the same event. This is permissible, provided it is done by a third party to the two main procedures. It is not permissible to reconcile details of characteristics for differences on the two types of documents. The distinction may be talmudic, but it makes a world of difference to the preservation of independence between the two procedures. The suggestion for the improvement of one method through the other made by the discussant of chapter 2 is, therefore, no less than sabotage of the pure PGE/ ERAD/ ECP technique. iii. Subsampling.'5 This could become the thin edge of the wedge of destruction of the main PGE/ ERAD/ ECP concept. Having obtained the completeness rate through the subsampled procedure, the "natural" corollary—it might occur to the managers — would be to apply the result as means of calibrating and "improving" the other procedure. As we repeatedly argue in this chapter and elsewhere, "improving" a PGE/ ERAD/ ECP procedure through the other one is the kiss of death for the whole system. iv. The preaching of hard work, careful work, high completeness ("100 percent completeness" — indeed!) and the like. Cases were known in the history of survey taking when crude birth rates of 100 per thousand were obtained through preaching and propaganda (Sudan, 1958). The vital events recorded then cease being a reflection of the demographic processes of a given society and become a result of the persuasive process. The field procedures should be neutral in their effect and such that the reporting of vital events does not depend on the temporary zeal of any field worker. v. Implicit faith in any one part of the work being greater than in another. It is unjustified and dangerous to see in the so-called base line survey or base line study anything more than the first of many endeavours. It is not particularly better than others, and is probably poorer than subsequent ones. Like "reconciliation", the faith in "base line" discloses the tendency to improve matters. Such a tendency is destructive in itself for PGE/ ERAD/ ECP. We preach instead the more realistic tendency to live in sin and error, but to evaluate both and arrive at improved estimates. Within this spirit the basic register described in section 5.8 is not in the spirit of a true PGE/ ERAD/ ECP pursuit. 1.8.b Matters affecting the matching procedure i. Cutting the size of clusters more and more in the hope that coupled with frequent rounds the burden of matching will disappear (and as a by-product the sampling error of cluster sampling will dwindle to the level of simple random sampling). By zeroing in on one or two vital events in this manner we abstract from all those in between that escape the net of any one procedure either through their own peculiarities or the disadvantages of the procedure. 27
1.8 b
Karol J. Krotki
ii. Rushing into using the civil registration system as one of the procedures for a PGE/ ERAD/ ECP system, without ascertaining first that the civil registration bookkeeping justifies and makes possible comparisons with the size of clusters one usually draws for sampling purposes. Typically the books of civil registration are kept for areas much larger than those required for PGE/ ERAD/ ECP. iii. Making the recall period 15 months long with half yearly rounds should the next study be so adventurous as to try two periodic household surveys as the two procedures with an overlapping recall period.16 The field procedure would confuse the respondent, exasperate the field worker, and confound the analyst. iv. Offering prizes to field workers for discovering vital events not discovered by the other field worker (the Liberian case). They will surely find a way to share the bounty equitably in line with the allegedly Chinese proverb: "Two officials think up a law; two million members of the public think of ways of getting round it; who will get the upper hand?" In Liberia the field workers must have taken to dividing fifty-fifty all their vital events and estimating rates must have rocketed sky high. (We are thus in disagreement with the suggestion made in section 7.6.d.) The organizational effort required by prizes and bonuses is better spent on structuring supervision. 1.8.c Matters affecting samples i. Having a bizzare sampling design such as covering a country in five annual lots (with none of the fifths being selected probabilistically to boot) and attempting to design suitable de facto and dejure definitions (Upper Volta). Or covering one-tenth in the first year, two-tenths in the second year, three-tenths in the third year and so on, until the whole territory is covered in the tenth year (the five countries of UDEAC). While it must be hoped that no analyst worth his salt would just lump all these questionnaires together without regard to their temporal origin, any successful office treatment would require arrangements of considerable complexity. ii. Having enormous clusters such as the 20,000 they had in Cameroun. Any minimal homogeneity must result in intraclass correlations so large that the few clusters that can be enumerated do not justify the expenditure. Furthermore, it is likely that even from the standpoint of nonsampling errors, 20,000 is well beyond an optimum. iii. Using a list of households as a sampling frame, from an old census to boot (Tunisia, 1970). Such a list ties the sample to a stale population and institutionalizes old errors. In particular it excludes newly created households. l.S.d Matters affecting organization i. Hitching the PGE/ERAD/ECP survey onto a multipurpose survey (Maroc, 1967). For a discussion of the disadvantages of multipurpose surveys see Mauldin, 1966: 652. ii. Wasting time on such frills as asking females "Are you pregnant now?" (Scott, 1973: 6), in the hope that the disappearance of pregnancy at the next round will lead to the sure reporting of a birth. This Liberian initiative is like catching fish with a net whose holes are larger than the fish. iii. Having calls other than six monthly in the periodic survey procedure. Few organizations will have the stamina to carry out the quarterly matching (Pakistan, 1961-65) and yearly intervals result in too much memory loss and too great a burden on the second and third PGE/ ERAD/ ECP category (see figure 1.1). iv. Throwing out all of the data because part of them are not available or are faulty. This may arise when seemingly important parts of the sample drop out because of insufficient response or for other reasons. This may be the case when the number of areas affected is large with matching between rounds. In such a case, interround 28
Role of PGEIERAD/ECP surveys
L8.d
tabulations for the clusters for which two correct data sources are available should be printed separately from the tabulations for all the clusters. In this manner a bridge might be created between the two kinds of tabulations. v. Taking the trouble and expense of mounting an operation without seriously considering the possibility of making it nationally representative. This error can arise either through quota sampling or through concentrating the whole field work in one area for "experimental" purposes (the Philippines, 1971-75). 1.9 Quasi PG EI ERA DI ECP estimates There are instances where the PGE/ ERAD/ ECP approach is taken in all essentials except that the fourth category is not estimated (for the example of India see Ramabhadran, 1971). This is understandable when the quality and outcome of the matching rules is so uncertain that a large net error in either direction is expected. But then we must have no faith in the second or third category either (see figure 1.1) and the survey must be restructured. The sampling error is neutral either way and provides no reason for ignoring the fourth category. The possibly inadequate independence between the two procedures remains. All known examples of correlation are positive and one would be hard put to think of examples of negative correlation. One would be the bizzare case suggested in section 1.8.b iv, the case when the two field workers carved up the vital events in the area between them in order to share fairly and to the maximum the prize offered. In all more realistic cases the likelihood of the three categories being underestimated is markedly greater than overestimated. It makes no sense, as in the Indian case, to undergo the expense of matching and then only partly use the results. It would be valid to say that the expense of matching is not worthwhile to undertake because of insufficient independence, particularly should there exist a better set of data with which to match. This would be consistent with the attitude taken in this chapter and in the PGE handbook that matching between rounds of the same multiround survey is seldom worthwhile because of their dependence on the same type of procedure. It is only when extreme steps have been taken to ensure independence in conformity with the PGE handbook (Marks et a/., 1974:250-253,425428) that matching is justifiable. Once justifiable, the fourth category is valid. 1.10 The paucity and importance of costing data Table 1.3 assembles such data on expenditure on social surveys as became easily available. One of the reasons why little data on financing social surveys are published is that most social surveys are financed not only by their own budget but also by the provision of services and assistance from sources not easily identifiable for accounting purposes. There is a need to direct some effort to identifying these sources in particular cases and suggesting imputed figures in order to arrive at real, economic costs, not only the financial costs of a social survey. This is important for two reasons: to assess the total costs of an endeavour in order to compare it with the benefits of the endeavour or with the costs of an alternative way of reaching the same objective; the second is the impact that cost items have on the survey design. With regard to the first reason, international comparisons would be particularly interesting. The strong feeling of the present writer is that relative to the average national income, social surveys in underdeveloped countries are more expensive than in developed countries by a factor of three or four. In Canada the annual salary of a 29
Table 1.3 Survey costs divided by office/field and by primary/secondary units
Survey
Currency
Total
Office
Field
(1)
(2)
(3)
(4)
(5)
Algeria 1966 Algeria 1969 Anon 1967(?) Burundi 1970/71 Cameroon 1971/82 Cameroon (?) Canada-GAFS 1973 Congo 1971/82 Dahomey 1961 Gabon 1960/61 Gabon 1969 Gabon 1971/82 Guyana 1974 Ivory Coast 1970 Madagscar (?) Malawi 1970/71 Mauritania 1974 Morocco 1961/62 Morocco 1972/73 (macro) ' ' Morocco 1972/73 (micro)12 Morocco 1972 (recording)
D.A.1 D.A.1
20,493,000 6,350,000 432,000 345,600 86,400 2,822,000 357,100,000 31,500 39,879 14,281 117,089,000 18,000,000 33,000,000 20,950,000 86,959,000 268,630 416,000,000 46,000 71,000 178,950,000 46,840 3,350,000 1,600,000'°1, 750,000 516,000
u.s.s
FiBu3 F CFA5 pao Can4 FCFA 5 FCFA 6 F CFA6 FCFA5 F CFA5 G7 FCFA5 F8
u.s.s
F CFA5 D.H.9 D.H.9 D.H.9 D.H.9
303,000 285,330
Field costs: (5) as % for for of (3) PSUs SSUs (6) (7) (8)
20
43,200
43,200
average cost per one PSU SSU Cost ratio 9/10 (9) (10) (11)
0.32?
36
0.026
561
3.213
175
Morocco 1972 (household) Morocco — project Pakistan 1962/65 R.C.A. 1971/82 Senegal: Sine-Saloum 1963/65 Senegal: Khombol Thienaba Senegal: Pikine 1967/69 Senegal 1970/71 Sudan 1953 Sudan 1955/56 Tchad 1971/82 Tunisia (?) USA-GAF 1955 USA-GAF 1960
D.H.9
350,637
D.H.9
u.s.s
130,000 122,880,000
F CFA5
F19
160,000
p20
46,000
F21
16,000 75,000,000 14,880 469,939 350,832,000 1,500,000
F CFA5 L.E.22 L.S.22
FCFA 5 F23
Algeria 1966
Orstom, 1973, p. 370 Orstom, 1973, p. 370 Seltzer, 1971, p. 101 Orstom et al., 1971, p. 277 Orstom, 1973, p. 374
Burundi 1970/71 Cameroon 1971/82
12,005 375,519
2,875 94,420
2317
19 20
9,000
1,063 69,220
1,288 20,700
50018
912 26,200
17.8 6.7
3.214 40215 0.21 '6 2,380
0.015 1,187 0.014 479
40,990 58,823
(3)
Anon 1967(?)
215,000 29,700
U.S.$ U.S.S
Sources
Algeria 1969
100,300
(4)
(5)
(7)
(8)
(9)
arbitrary arbitrary arbitrary arbitrary
(10)
24
(4)
Sources
(3)
Cameroon (?)
Orstom, 1973, p. 373
Canada-GAFS 1973
25
Congo 1971/82
Orstom, p. 374 Orstom, p. 369 Orstom, p. 370 Orstom, p. 370 Orstom, p. 385 Orstom, p. 374 Guyana,, p. 21 Orstom, p. 369 Orstom, p. 373 Blacker,
Dahomey 1961 Dahomey 1966 Gabon 1960/61 Gabon 1969 Gabon 1971/82 Guyana 1974 Ivory Coast 1970 Madagascar (?) Malawi 1970/71 Mauritania 1974 Morocco 1961/62 Morocco 1972/73 (macro)
25
1973, 1973, 1973, 1973, 1973, 1973, 1974, 1973, 1973,
1970
Orstom, 1973, p. 368 Orstom et al., 1970, p. 262 10 Fellegi, 1971a, letter of 6 April
(5)
(7)
(8)
(9)
(10)
Morocco 1972/73 (micro)
Fellegi, 1973, p.ll
Morocco 1972 (recording) Morocco 1972 (household) Morocco-project (?)
Orstom, 1973, p. 385 Orstom, 1973, p. 385
Pakistan 1962/65
Seltzer, 1971 p. 101 Orstom, 1973, p. 374
R.C.A. 1971/82 Senegal: Sine-Saloum 1963-65 Senegal: Khombol Thienaba
(?)
Senegal: Pikine 1967/69 Senegal 1970/71 Sudan 1953 Sudan 1955/56 Tchad 1971/82 Tunisia (?)
ibid., p.ll ibid., p.ll
Fellegi, 1973
ibid., p. 4 ibid., p. 4
26
27
ibid., p. 78 ibid., p. Ill
ibid., p. 124 ibid., p. 1ll derived
Orstom, 1973, p. 373 Orstom, 1973 p. 373 Orstom, 1973 p. 373 Orstom, 1973, p. 369 Krotki, 1955, p. 78 Sudan, 1958, p. 87: 444,544, p. Ill: 25,395 Orstom, 1973, p. 374 Orstom, 1973, p. 372
Sources USA-GAF 1955 USA-GAF 1960
1
(3)
(4)
(5)
(7)
(8)
(9)
(10)
SRC, ISR Ann Arbor verbal SRC, ISR Ann Arbor verbal
Multiply all values by 0.2026 to bring them to U.S.: dollars It is believed that there was more than one single visit in the survey. Following footnote 18, a division by 4 is suggested ($0.08). 3 Multiply all values by 0.0114 to bring them to U.S. dollars 4 Multiply all values by 1.0016 to bring them to U.S. dollars. 5 Multiply all values by 0.0036 to bring them to U.S. dollars. 6 Multiply all values by 0.0041 to bring them to U.S. dollars. 7 Multiply all values by 0.50 to bring them to U.S. dollars. 8 Multiply all values by 0.1813 to bring them to U.S. dollars. 9 Divide all values by 4.6 to bring them to U.S. dollars. 10 The office expenditure in column (4) is a minimal estimate relating to office treatment of questionnaires, i.e. it excluded all overheads, etc. 11 The "macro" estimate for Morocco of 1972/73 was derived for budgetary purposes prior to the exercise in global terms. 12 The "micro" estimate for Morocco of 1972/73 was derived for analytic purposes after the exercise on the basis of assumed "per household" performance. 13 On the assumption of 5 persons per each of the 17,000 households in the Moroccan PGE/ERAD/ECP study. 14 On the assumption of 5 persons per each of the 9,600 households in the proposed national PGE/ERAD/ECP survey in Morocco. 15 The Moroccan-project cost ratio in column (11) rises to 805 when only one survey is considered. 16 Average annual figures are quoted for the 4-year Pakistan study. 17 High ratios due to some overheads spread over 4 years (Seltzer, 1971:101). 18 In any one year there were 3 multi-visits and equivalents of 6 longitudinal registrations. To compare with single visits, divide by 9 ($56 and $0.023). 2
19
Multiply all values by 0.2041 to bring them to U.S. dollars. Multiply all values by 0.1813 to bring them to U.S. dollars. Multiply all values by 0.2021 to bring them to U.S. dollars. 22 Multiply all values by 2.8 to bring them to U.S. dollars. 23 Multiply all values by 0.20 to bring them to U.S. dollars. 24 On the assumption of 5 persons per each of the 27,000 households (in the "anon" country). 25 The cost estimates were based on the revised 1973 budget. 26 On the assumption of 18 recorders, 18 interviewers, 9 supervisors, at U.S.S790 annually. 27 4/5 of the time of the 18 recorders and 18 interviewers. 20 21
7.70
Karol J. Krotki
field worker will be twice the national income per capita, but in a typical underdeveloped country it will be six or eight times higher, though rather extreme variations in similar and neighbouring countries will occur. With regard to the second reason, it is probably even more difficult to obtain details of the expenditure unless very good care is taken by the accountant to post each item of expenditure to the heading most relevant analytically, which is not necessarily the same as for accounting purposes. For example, travel and subsistence while away from one's place of residence might be one accounting heading, but the expenses of the interviewers will affect the size of clusters and other design aspects differently from the expenses of their supervisors. Fortunately we need not depend on actual monetary values. One can use substitutes in the form of time possibly multiplied by the fraction by which the salaries of supervisors and their other emoluments exceed those of the interviewers. There were not enough data to pursue this alternative but an attempt has been made in table 1.4 to ensure some international comparability by bringing all reported costs to the 1975 price level and expressing them in the most convenient international currency, namely the U.S. dollar. The table has the same blanks as table 1.3, but we consider the effort worthwhile if it will encourage a wider reporting of cost items. The available information has been pushed to further limits in table 1.5 where the cost per unit of contact was calculated in order to show the variability between countries and between types of survey. The data are still so limited that we will leave any verbalization that the table might deserve to the reader. In a discussion of PGE/ ERAD/ ECP costs and those of a multiround survey it has been shown that with three rounds in a multiround survey annually, the PGE/ ERAD/ ECP system begins to be less expensive and wins on the grounds of costs alone, not to mention the built-in feature of self evaluation (Seltzer, 1971). The two procedures of a PGE/ ERAD/ ECP system and the costs of matching are under some circumstances less than the cost of three rounds of a multiround survey. In the course of the work on tables 1.3,1.4, and 1.5 certain simple principles of accounting for analytic purposes occurred and it might be useful for future reference to spell them out: i. Have own costs posted into at least three ledger headings: overheads, costs tied to primary sampling units, and costs tied to secondary sampling units; ii. Impute costs covered from external budgets, and put these under the same three ledger headings; iii. Unusual costs (international or subsidized) should be identified and some indication given of what the normal local cost would be; iv. Indicate changes in the local price index when the survey spreads over a sufficiently long period of time; v. Indicate the rate of exchange with one or two major international currencies, particularly when changes in exchange rates are taking place in the course of the survey (consider an average between official and black-market rates if relevant); vi. State the starting or average salary of a field worker and those of a supervisor; state their employment in terms of person-years or person-months; vii. Give sources of the data meticulously, if necessary in tabular form, separately for each column, not unlike those in tables 1.3,1.4 and 1.5; viii. Publish the cost data. The person-years or person-months substitutes are not always a complete answer to the problem. Only rarely are the totals of the "time on the job" a direct reflection of all the costs. In organizational circumstances when field workers spend only a small proportion of their time on the job and the rest "just hanging around" the time-andmotion measured cost may actually be misleading and only a very small proportion of 36
Table 1.4 Total and proportionate survey costs Survey
Currency reported
Total costs
(1)
(2)
(3)
Algeria 1966 Algeria 1969 Anon 1967 (?) Burundi 1970/71 Cameroon 1971/82 Cameroon (?) Canada-GAFS 1973 Congo 1971/82 Dahomey 1961 Dahomey 1966 Gabon 1960/61 Gabon 1969 Gabon 1971/82 Guyana 1974 Ivory Coast 1970 Madagascar (?) Malawi 1970/71 Mauritania 1974 Morocco 1961/62 Morocco 1972/73 (macro)8 Morocco 1972/73 (micro)
D.A. D.A.
20,493,000 6,350,000 432,000 86,400 2,822,000 357,100,000 31,500 14,281 39,879 117,089,000 18,000,000 60,000,000 33,000,000 20,950,000 86,959,000 268,630 416,000,000 46,000 71,000 46,840 178,950,000 3,350,0007 1,750,000 516,000
0.2026 0.2026 1.0000 0.0114 0.0036 0.1813 1.0016 0.0036 0.0041 0.0036 0.0041 0.0036 0.0036 0.5000 0.0036 0.1813 1.0000 0.0036 0.2400 0.2200
1.623 1.435 1.578 1.328 1.300 1.300 1.624 1.300 1.760 1.623 1.778 1.806 1.300 1.300 1.356 1.300 1.328 1.300 1.751 1.300
303,000
0.2200
1.300
u.s.s
FiBu FCFA F Can FCFA FCFA FCFA FCFA FCFA FCFA G FCFA F U.S.S FCFA D.H. D.H. D.H.
Field Multipliers used to Total cost costs bring costs (3) & (4) in US $ of 1975 to US$ (3)X(5) to 1975 X(6) (4) (5) (6) =(7) 6,736,912 1,845,705 681,696 359,036 1,671,635 7,424 46,425 548,110 128,337 350,653 237,691 98,070 406,968 17,461 2,031,241 10,842 94,288 873,486 1,407,804
Field cost in US $ of 1975 (4)X(5) X(6) =(8)
No. of households
(9)
Avg. cost per household in US $ of 1975 Total Field costs costs (7):(9) (8):(9) -(10) -(11)
136,339
70,000' 27,000
26.37 25.25
5.05
16,613
3,0002 1,045
2.47 44.42
15.90
14,0003
9.17
104,0004
2.29
5,000 19,0005 5,0006 10,000
34.92 106.91 2.17 9.43
735,420 147,576
66,000 16,000
21.33
86,658
17,000
219 11.14 9.22
5.10
Table 1.4 Survey
Currency reported
Total costs
0)
(2)
(3)
D.H. Morocco 1972 (recording) Morocco 1972 D.H. (household) D.H. Morocco-project (?) Pakistan 1962/65 U. S.S F CFA R.C.A. 1971/82 Senegal: Sine-Saloum F 1963/65 F Senegal: Khombol Thienaba (?) Senegal: Pikine 1967/69 F F CFA Senegal 1970/71 L. E. Sudan 1953 Sudan 1955/56 L. S. Tchad 1971/82 F CFA F Tunisia (?) USA-GAF 1955 U. S.S USA-GAF 1960 U. S.S
Field Multipliers used to Total cost costs bring costs (3) & (4) in US $ to US$ of 1975 (3)X(5) to 1975 X(6) (6) =(7) (4) (5)
Field cost in US $ of 1975 (4)X(5) X(6) =(8)
No. of households
(9)
Avg. cost per household in US I f of 1975 Total Field costs costs (7):(9) (8):(9) =(10) -(11)
285,330
0.2200
1.300
81,604
16,000'°
5.10
350,637
0.2200
1.300
100,282
1 6,00010
6.27
0.2200 1.0000 0.0036 0.2041
1.300 1.707 1.300 1.697
221,910 575,219 55,412
46,000
0.1813
1.356
16,000 75,000,000 14,800 469,939 350,832,000 1,500,000
0.2021 0.0036 2.8000 2.8000 0.0036 0.2000 1.0000 1.0000
130,000 122,880,000 160,000
215,000 29,700
2,875 94,420 40,990 58,823
61,490 50,698
9,600 20,000
11.10
10,600"
5.23
11,309
2,500'2
4.52
1.514 4,896 1.356 366,209 1.968 81,995 1.953 2,569,814 1.300 1,642,294 1.514 454,200 1.967 1.778
15,842 516,326
80013 100,00014 10,000 376,600
6.12 3.66 8.20 6.82
28,000 15 2,913 3,322
16.22
80,627 104,587
6.41 2.53
1.58 1.37 27.68 31.48
Sources (1)
(3)
Algeria 1966
Orstom, 1973, p.370
Algeria 1969
Orstom, 1973, p.370
Anon 1967(?)
Seltzer, 1971, p. 101
Burundi 1970/71 Cameroon 1971/82
Orstom et al, 1971,p.277 Orstom, 1973, p.374
Cameroon (?)
Orstom, 1973, p.373
Canada-GAFS 1973
16
Congo 1971/82
Orstom, 1973, p.374
Dahomey 1961
Orstom, 1973 p.369
Dahomey 1966
Orstom, 1973 p.370
Gabon 1960/61
Orstom, 1973, p.370
Gabon 1969
Orstom, 1973, p.385
Gabon 1971/82
Orstom, 1973, p.374
Guyana 1974
Guyana, 1974, p.21
Ivory Coast 1970
Orstom, 1973, p.369
(4)
16
(5)
(6)
Derived from exchange rate reported in IMF, 1971, p.43 Derived from exchange rate reported in IMF, 1971, p.43
U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973
Derived from exchange rate reported in IMF, 1972, p.73 Derived from exchange rate reported in IMF, 1972, p.373 Derived from exchange rate reported in IMF, 1972, p. 143 Derived from exchange rate reported in IMF, 1974, p.77 Derived from exchange rate reported in IMF, 1972, p.373 Derived from exchange rate reported in IMF, 1964, p.241 Derived from exchange rate reported in IMF, 1972, p.373 Derived from exchange rate reported in IMF, 1964, p.241 Derived from exchange rate reported in IMF, 1972, p.373 Derived from exchange rate reported in IMF, 1972, p.373 Derived from exchange rate reported in IMF, 1972, p. 165 Derived from exchange rate reported in IMF, 1972, p.373
(9)
Orstom, 1973, p.370 Seltzer, 1971, p.101
Orstom, 1973, p.373
Orstom, 1973, p.385
Guyana, 1974, p. 6 Orstom, 1973, p.369
Sources (1)
(3)
Madagascar (?)
Orstom, 1973, p.373
Malawi 1970/71
Blacker, 1970
Mauritania 1974
Orstom, 1973, p.368 Orstom et a/., 1971, p.262
Morocco 1961/62 Morocco 1972/73 (macro) Morocco 1972/73 (micro) Morocco 1972 (recording) Morocco 1972 (household) Morocco-project (?) Pakistan 1962/65 R.C.A. 1971/82 Senegal: Sine-Saloum 1963/65 Senegal: Khombol Thienaba (?) Senegal: Pikine 1967/69
(4)
(5) Derived from exchange rate reported in IMF, 1972, p. 143
ibid., p.368
Derived from exchange rate reported in IMF, 1972, p.373
Fellegi, 197 la letter of 6 April, 1971 Fellegi, 1973, p.ll Orstom, 1973 p.385 Orstom, 1973, p.385 Fellegi, 1973, p.4 Seltzer, 1971, p.101 Orstom, 1973, p. 374 Orstom, 1973, p.373 Orstom, 1973, p.373 Orstom, 1973, p.373
Derived from exchange rate reported in IMF, 1972, p.373 Derived from exchange rate reported in IMF, 1971, p. 140 Derived from exchange rate reported in IMF, 1971, p. 143 Derived from exchange rate reported in IMF, 1971, p. 146
(6) U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973
(9) Orstom , 1973, p.373
Fellegi, 1973, p.39 Fellegi, 1973, p.ll Orstom, 1973, p.385 Orstom, 1973, p.385 Fellegi, 1973, p.4
Orstom, 1973, p.373 Orstom, 1973, p.373 Orstom, 1973, p.373
Senegal 1970/71
Orstom, 1973, p.369
Sudan 1953
Krotki, 1955, p.78
Sudan 1955/56 Tchad 1971/82
Sudan, 1958 p.87:444,544 p.l 11:25,395 Orstom, 1973, p.374
Tunisia (?)
Orstom, 1973, p.372
USA-GAF 1955 USA-GAF 1960 1
SRC, ISR Ann Arbor verbal SRC, ISR Ann Arbor verbal
Derived from exchange rate reported in IMF, 1972, p.373
U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973
Derived from exchange rate reported in IMF, 1972, p.373 Derived from exchange rate reported in IMF, 1972, p. 143
U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973 U.S. Dept. of Commerce, 1973
Orstom, 1973, p .369
Orstom, 1973, p. 372
SRC, ISR Ann Arbor verbal SRC, ISR Ann Arbor verbal
On the assumption of 5 persons per household, the number of households was derived from dividing the reported 350,000 individuals by 5. Following footnote 1, the reported number of 15,000 individuals was divided by 5. 3 Following footnote 1, the reported number of 70,000 individuals was divided by 5. 4 Following footnote 1, the reported number of 520,000 individuals was divided by 5. 5 Following footnote 1, the reported number of 95,000 individuals was divided by 5. 6 Following footnote 1, the reported number of 25,000 individuals was divided by 5. 7 The total cost in Morocco of 1961/62 included only minimal office expenditure relating to the office treatment of questionnaires, i.e., it excluded all overhead. 8 The "macro" estimate for Morocco of 1972/73 was derived for budgetary purposes prior to the exercise in global terms. 9 The "micro" estimate for Morocco of 1972/73 was derived for analytic purposes after the exercise on the basis of assumed "per household" performance. 10 Following footnote , the reported number of 80,000 individuals was divided by 5. 11 Following footnote , the reported number of 53,000 individuals was divided by 5. 12 Following footnote , the reported number of 12,500 individuals was divided by 5. 13 Following footnote , the reported number of 40,000 individuals was divided by 5. 14 Following footnote , the reported number of 500,000 individuals was divided by 5. 15 Following footnote , the reported number of 140,000 individuals was divided by 5. 16 The cost estimates were based on the revised 1973 budget. 2
Table 1.5 Costs varying with types of surveys
Survey
Number of households
(1)
(2)
Avg. annual cost per household in US$ of 1975 Total cost Field cost (3) (4)
70,000' 27,000
26.37 25.25
Algeria 1966 Algeria 1969 Anon 1967 (?) Burundi 1970/71 Cameroon 1971/82 Cameroon (?) Canada-GAFS 1973 Congo 1971/82 Dahomey 1961 Dahomey 1966 Gabon 1960/61 Gabon 1969 Gabon 1971/82 Guyana 1974 Ivory Coast 1970 Madagascar (?) Malawi 1970/71 Mauritania 1974 Morocco 1961/62 Morocco 1972/73 (macro)7
3,0002 1,045
44.42
14,0003
9.17
104,0004
2.09
5,000 19,0005 5,0006 10,000 66,000 16,000
Number of contacts
(5)
(6)
Avg. field cost per contact (4): (6) -(7)
National survey National statistical study on population 4
5.05
15.90
106.91 9.43 21.33
Type of survey
11.14 9.22
Regional project on population Pilot for fertility study KAP and fertility Regional project on population National survey Regional survey National survey National census Regional project on population Fertility survey GFS/WFS National survey Pilot for fertility survey PGE/ERAD/ECP National census Multi-purpose PGE/ERAD/ECP
1.26
5 1
15.90
1 4 3 6
3.71 1.54
Morocco 1972/73 (micro)8 Morocco 1972 (recording) Morocco 1972 Morocco-project (?) Pakistan 1962/65 R.C.A. 1971/82 Senegal: Sine-Saloum 1963/65 Senegal: Khombol Thienaba (?) Senegal: Pikine Senegal 1970/71 Sudan 1953 Sudan 1955/56 Tchad 1971/82 Tunisia (?) USA-GAF 1955 USA-GAF 1960
17,000 16,0009
5.10
16,0009 9,600 20,000
6.27 11.10
10,60010
5.23
5.10
PGE/ERAD/ECP
6
0.85
6.41 2.53
PGE/ERAD/ECP PGE/ERAD/ECP Regional project on population Pilot for a fertility survey
6 8
1.07 0.32
Pilot for a fertility survey
6
Pilot for a fertility survey National survey Pilot for a national census National sample population census Regional project on population National survey KAP and fertility KAP and fertility
3
2,500 11 80012 100,00013 10,000 376,600 28, 00014 2,913 3,322
6.12 3.66 8.20 6.82
1.58 1.37 27.68 31.48
4
1 1
1.58 1.37
3 1 1
27.68 31.48
Sources N.B. Cols (2), (3), (4) are copied from cols (9), (10), (11) of table 1.4.
(1) Algeria 1966 Algeria 1969 Anon 1967 (?) Burundi 1970/71 Cameroon 1971/82 Cameroon (?) Canada-GAFS 1973 Congo 1971/82 Dahomey 1961 Dahomey 1966 Gabon 1960/61 Gabon 1969 Gabon 1971/82 Guyana 1974 Ivory Coast 1970 Madagascar (?) Malawi 1970/71 Mauritania 1974 Morocco 1961/62 Morocco 1972/73 (macro) Morocco 1972/73 (micro)
(2)
Orstom, 1973, P- 370 Seltzer, 1971, P- 101 Orstom, 1973, P- 373 Orstom, 1973, P- 369 Orstom, 1973, P- 385 Guyana,, 1974.. P.21 Orstom, 1973, P- 369 Orstom, 1973, P- 373 Maroc, 1967, P- 111 Fellegi, 1973 Fellegi, 1973, P- 11
(6)
(5)
Orstom, 1973, p. 370 ibid., p. 370 Orstom, 1973, p. 374 ibid., p. 373 Orstom, 1973, ibid., p. 369 Orstom, 1973, Orstom, 1973, ibid., p. 385 Orstom, 1973, ibid., p. 1 ibid., p. 369 ibid., p. 373
ibid., P .370 arbitrary ibid., P- 373
p. 374 p. 370 p. 370 p. 374
Orstom, 1973, p. 368 ibid.
ibid., P . 2 1 ibid., P .373 ibid., P . Ill 2 visits annually by recorders are 2 visits annually by recorders are
by interviewers and assumed that the 12 visits equivalent of 4 visits by interviewers and assumed that the 12 visits equivalent of 4 visits
Morocco 1972 (recording) Morocco 1972 (household) Morocco-project (?)
Orstom, 1973, p. 385
Pakistan 1962/65
Seltzer, 1971, p. 101
Orstom, 1973, p. 385 Fellegi, 1973, p. 4
R.C.A. 1971/82 Senegal: Sine-Saloum Orstom, 1973, p. 373 1963/65 Senegal: Khombol Orstom, 1973, p. 373 Thienaba (?) Senegal: Pikine 1967/69 Orstom, 1973, p. 373 Orstom, 1973, p. 369 Senegal 1970/71 Krotki, 1955, p. Ill Sudan 1953 Sudan, 1958, p. Ill Sudan 1955/56 Tchad 1971/82 Orstom, 1973, p. 372 Tunisia (?) SRC, ISR Ann Arbor USA-GAF 1955 verbal USA-GAF 1960 SRC, ISR Ann Arbor verbal
2 visits annually by interviewers and assumed that the 12 visits by recorders are equivalent of 4 visits 3 visits annually by interviewers and assumed that the 12 visits by recorders are equivalent of 5 visits Orstom, 1973, p. 374 ibid., p. 373
ibid., p. 373
ibid., p. 373
ibid., p. 373
ibid., p. 373 ibid., p. 369 ibid. ibid. Orstom, 1973, p. 374 ibid., p. 372
ibid., p. 373 ibid., p. 369
ibid., p. 372
1
2 3 4 5 6 7 8
9 10 11 12 13 14
On the assumption of 5 persons per household, the number of households was derived from dividing the reported 350,000 individuals by 5. Following footnote 1, the reported number of 15,000 individuals was divided by 5. Following footnote 1, the reported number of 70,000 individuals was divided by 5. Following footnote 1, the reported number of 520,000 individuals was divided by 5. Following footnote 1, the reported number of 95,000 individuals was divided by 5. Following footnote 1, the reported number of 25,000 individuals was divided by 5. The "macro" estimate for Morocco of 1972/73 was derived for budgetary purposes prior to the exercise in global terms. The "micro" estimate for Morocco of 1972/73 was derived for analytic purposes after the exercise on the basis of assumed "per household" performance. Following footnote , the reported number of 80,000 individuals was divided by 5. Following footnote , the reported number of 53,000 individuals was divided by 5. Following footnote , the reported number of 12,500 individuals was divided by 5. Following footnote , the reported number of 40,000 individuals was divided by 5. Following footnote , the reported number of 500,000 individuals was divided by 5. Following footnote 1, the reported number of 140,000 individuals was divided by 5.
Role of PGEI ERA D/ ECP surveys
1.10
the total cost. The field workers may work very hard when on the job (reports of one hundred household interviews with pretty hefty interview schedules are not uncommon) precisely because they know that they can loaf afterwards.17 In such cases the neat arithmetical exercise that allots time proportionately to the variance in stratified sampling or to intraclass correlation in cluster sampling becomes rather pointless, or rather baseless. The need for accurate cost coefficients can be illustrated by the choice of cluster size for estimating birth rates with the proposed Moroccan national dual collection sample (Fellegi, 1973). It was assumed (a) that fulltime resident recorders would be used for the continuous recording procedure, (b) that all the clusters selected for continuous recording would also be subjected to the periodic household survey and that there would be no subsampling of one procedure by the other procedure; in other words, that in terms of the notation used in 2.2.b, ki = k2 and h = H, (c) that the periodic surveys will be conducted by survey teams travelling from cluster to cluster and (d) that the continuous recording cost component CR = O (see 2.2.b), since the data collection budget was not to be charged to the resident recorder costs. The last assumption reduced the problem to one of determining the appropriate cluster size for the periodic survey procedure. The cost for the two surveys to be conducted each year was estimated to be (in terms of person-days): C= 1288 k2+ 1.6 k2 H, (1.18) where 1288 represents the cost per cluster for initial delineation and travel costs18 between two clusters and 1.6 is the per person enumeration cost with two surveys per annum. From formula (1.18) we express k2 k2 = C/(1288 + 1.6H) (1.19) The classical formula for the variance of a cluster sample is Variance of an estimated total = N2 (a 2 /k 2 H) (1 + Hd) (1.20) where a2 = pq p = birth rate q= 1 -p N = total population size k2 = the number of selected clusters H = the number of persons per cluster minus 1 (Kish, 1966: 216) d = intraclass correlation. The empirical intraclass correlation for births was reported to be 0.0008. Substituting the values from (1.19) into (1.20) we obtain: Var (B) = (a 2 /k 2 H)(l+0.0008 H) (1.21) = (a 2 /C) [(1288 + 1.6H)/H](1 + 0.0008H) (1.22) = (a21C) [(1288/H) + 0.00128 H + 2.6304] (1.23) where B = total births. Taking the derivative of var (B) with respect to H and setting it equal to O, we obtain (1288/H 2 ) = 0.00128 (1.24) Therefore, we have the following optimal cluster size for the estimation of births Hop, = 1000 (1.25) Thus, for a simple random sample of clusters and intracluster correlation of .0008 for births, the optimal cluster size H is 1,000 persons. This result is interesting from two standpoints. First, it is based on "no subsampling" assumptions, although the actual cost ratio C R /C S = 1.420/4.322 for a national sample of 50,000 or 100,000 persons suggests subsampling of one of the procedures may, in fact, be worthwhile.19 Second, H = 1000 is on the lower side of the limits suggested in section 1.5 but is 47
1.10
Karol J. Krotki
still larger than those found in other African countries. Scott and Coker (1971) report household survey cost coefficient ratios of 80 to 300 for team enumeration under African conditions as compared to the Moroccan ratio in equation (1.18) above of 1288/1.6 = 805. This suggests possible overestimation of the cluster cost coefficient (1288) or underestimation of the within cluster cost coefficient (1.6). It points up the need for accurate determination of variable cost components for specification of optimum or nearly optimum dual collection designs.20 Other considerations enter into the choice of cluster size than merely that value of H which, together with appropriate choices of ki, k2 and h, minimizes the relvariance of the estimated number of births (or the estimated birth rate) for a given survey budget, but the example is illuminating.
1.11 The past history and the probable future of the technique It is satisfying to a Canadian writer to report that an early example of case-by-case matching occurred in Canada. The occasion was the 1931 population census and the PGE technique was used as means of evaluating the quality of the civil registration. The impressive series of monographs resulting from the 1931 census must have been facilitated by the ease with which the then Dominion Bureau of Statistics could recruit at that time of high unemployment university graduates at cut rates. The test was a oneway match from the census data to the civil registration. No attempt was made to test the completeness of census enumeration of young children by searching among census records for children registered as born in the years immediately preceding the census. This was to come years later when, particularly after the censuses of 1961,1966, and 1971, the so-called reverse record check (RRC) was used in order to assess the quality of census enumeration. For RRC sources other than birth registrations were also used to build up the control list. As we indicated in section 1.2 there are many uses of dual collection and dual comparison, but for our purposes at least two other characteristics are required: a caseby-case comparison (not a comparison to totals) and an estimate of omission by one or either procedure. Table 1.1 gives an overview of the world involvement in PGE/ ER AD/ ECP surveys. Some background history is presented in chapter 2 of the PGE handbook, brought up to date in some chapters of this book. In this connection the relation of the PGE/ ERAD/ ECP approach with the "capture/ tag/ recapture" (C-TR) techniques may be noted. What of the future? In the main, progress can be expected because of increasing disillusionment with unsatisfactory types of surveys. Presumably more and more countries will use the one method that, when properly conducted, gives a built-in estimate of omissions. Another possible development is in the direction of estimating the denominator of vital rates through dual collection of data on all persons at risk that should be entering the denominator. A suggestion to that effect has been made by Deming and Keyfitz, 1965. Their suggestion can be adapted to an enumeration of nomads. Nomads are usually enumerated through administrative efforts of "getting at them" (e.g. Brenez, 1971). A cross of the C-T-R technique and what was to become the Deming-Keyfitz suggestion has been used to enumerate the nomads of Sudan (Krotki, 1955). Our own chapter 10 elaborates the possibility of making the traditional PES (post-enumeration survey) less dependent on the original survey through adopting PGE/ ERAD/ ECP features. The field is wide and the possibilities are many, but the greatest and most immediate benefit is likely to accrue through the pursuits of orthodox — dare one say so of 48
Role of PGEI ERA D/ ECP surveys
1.11
this still new to some — technique, following the principles explained in the PGE handbook and brought up to date in the present volume.
Discussion by Lee L. Bean
Faced with the problem of generating or evaluating valid and reliable demographic information, administrators and researchers are faced with the problem of selecting from among various methodologies. The distinction between generating and evaluating data is an important one because various discrete methods of research, often in combination with others, are used for the same process. Whether particularly methodological elements in various combinations, however, produce equally acceptable results for generation and evaluation is an important issue. PGE/ ERAD/ ECP utilizes, in part, techniques which have been standard for years in the evaluation of demographic data. The concept of matching records from two sources—both independent and correlated (such as cohorts in successive censuses adjusted for mortality and migration) is central to much demographic analysis as A.J. Jaffe(1951: 85) has argued on the basis of the "principle of comparison". PGE/ERAD/ ECP, thus, builds upon well-established demographic methods which are widely accepted. Why then must one begin a volume of this type with a defence of the "role of PGE/ ERAD/ ECP surveys relative to other endeavours to secure more and improved demographic data",—the title of Krotki's chapter? Obviously because the PGE/ ERAD/ ECP method has yet to be accepted as the appropriate alternative to other methods. Krotki deals at some length with various alternative methods, particularly single round and multiround surveys, but the chapter provides much more than simply an evaluation of PGE/ ERAD/ ECP relative to other "endeavours to secure more and improved demographic data". There are discussions of ideal models of PGE systems, what one can do to maximize the "pay-off, and how one might destroy this fragile creature. The reader is provided with a concise, coherent treatment of PGE/ ERAD/ ECP, but not so systematically that one is able to weigh the evidence related to all issues that an administrator or a demographer may wish to consider in selecting PGE or some other method. Indeed, by focusing on the "closest competitors" to PGE/ ER AD/ ECP—single and multiround surveys—some important questions are ignored. The chapter is seemingly based upon an important assumption. That is, PGE/ ERAD/ ECP (and other survey methods) are an alternative to a fullblown, accurate vital statistics system.21 Should it be? Particularly if one is concerned with the development of demographic data in the Third World, that assumption is worth some reflection. At the time the PGE program in Pakistan was being discussed at the beginning of the 1960s, many demographers, including several associated with the Pakistan PGE project, argued that PGE was the right method at the right time because of the immediate need for demographic data to stimulate population policy and to provide a system by which family planning programs could be evaluated as they would begin to force down birth rates. The rationale for the adoption of PGE reflected an element of expediency: the data were needed then and in the short run; developing a vital statistics system which would generate accurate statistics would take at least 20 years. In spite of the fact that Pakistan operated a most efficient PGE system, Pakistan lacks a vital statistics system, current demographic data, and a system to generate data. One wonders, 15 years later, whether one should not take the long view. If one had said invest all available resources in developing an efficient vital statistics system, would we now be five years away from having such a system in Pakistan instead of being at least, still, 20 years away from such a state?22
49
Discussion
Lee L. Bean
Perhaps there were not enough resources available in Pakistan in the 1960s to begin a comprehensive vital registration system, but what resources are required for such a program relative to PGE/ ERAD/ ECP? Indeed another important element not covered in Krotki's discussion is the question of resource utilization. I mean not solely costs (which Krotki treats, noting we need more information) but also human resources. Given that the question of accuracy of the data may clearly override some cost and manpower considerations, particularly evident in the comparison of single round and PGE/ ERAD/ ECP systems, for long-term planning purposes one needs to know what types of skills and how many individuals will be required for one program versus another. Ignoring that particular question, I submit, is largely responsible for the breakdown of the elegantly designed Algerian ERAD experiment. In most developing countries the lack of trained manpower is critical, and how does PGE/ ERAD/ ECP stack up relative to developing a vital registration system, using single round or multiround surveys? We don't know. Demographic data which refer to one point in time are almost completely useless. The strength and importance of a full-blown vital statistics system (or the continuous registration system of selected European countries) is its continuity. Certainly that continuity can be achieved by other systems. A PGE/ ERAD/ ECP system which remains in operation, using exactly the same techniques, over a series of years would constitute an important and reasonable alternative to a vital registration system. In evaluating PGE/ ERAD/ ECP relative to other systems, what is its record of continuity? That issue is not treated systematically by Krotki, but there is some evidence. Using the more generous definition of what has constituted a PGE system (see table 1.1) 26 PGE studies covered one year,23 seven covered two years, three ran for three years and only one was operated for as long as five years. We do not have comparative data for competitive systems, but the evidence suggests lack of continuous support for PGE/ ERAD/ ECP and unless one can generate such support, is it worth the manpower, money, and effort to mount a PGE/ ERAD/ ECP exercise? There are many factors which one must consider in determining the selection of PGE/ ERAD/ ECP relative to other methods if one is interested in securing more and improved demographic methods, and Krotki does deal efficiently and directly with many of these issues: the benefits in terms of accuracy and efficiency through use of independent systems and matching problems of determining cluster size and so on. Often, however, these arguments are presented in such a fashion as to reflect the continuation of an intellectual debate between Kr6tki and certain other statisticians and demographers.24 And it appears that such arguments may ignore more important issues and may actually result not in progress but rather in retrogression in the generation of demographic data in the developing countries which continue to face the problem of poor quality demographic data. The important questions are not size of cluster or two versus three rounds per year, but can PGE or any other survey method receive such a degree of commitment from a government that it will support—providing the resources in terms of manpower first, and then money—such a system for not one or two years, but rather five, 10 or 15 years and do so without changing elements of the methods so often that observed changes cannot be attributed to nonmethodological factors? Is PGE/ ERAD/ ECP worth doing for five, 10 or 15 years, or should the limited manpower and financial resources be devoted to developing more classical systems? Those are some of the questions which I believe remain to be answered as one attempts to fully assess the "role of PGE/ ERAD/ ECP surveys among the endeavours to secure more and improved demographic data".
50
Role of PG El ERA Dj ECP surveys
Endnotes
Endnotes to Chapter 1 1. Among health records, data on epidemics are worth mentioning because of their historical importance, at least for the time being. When used for demographic purposes they seldom give good results, because of uncertain completeness, sometimes even uncertain coverage. 2. Under exceptional circumstances, here and there, even single round surveys will report what appear to be reliable, high quality data on small areas consistent with commonsense (Goldberg and Adlakha, 1968); death reports in the example quoted. 3. With this evidence accumulated over the years, the United Nations, the International Union for the Scientific Study of Population, and the International Statistical Institute decided to base the World Fertility Survey (WFS) on single round surveys. There are, of course, also other purposes in front of the WFS than measuring birth rates and the high mobilization of talent in the WFS headquarters will presumably justify this flying in the face of world opinion and evidence. If not, then the world might obtain intricate answers to sophisticated questions on fertility and reproductive practices and attitudes, but still not know how much of its population is being born and how much is dying. 4. Arbitrary adjustments have now been with us for a long time and are likely to remain if the stream of data flowing from the World Fertility Survey conforms to the experience with single round surveys in the collection of vital rates. (That is, without prejudice to the other more sophisticated ratio measures at which the WFS is aiming.) 5. This error statement abstracts from other sources of error such as interviewers' mistakes in writing down dates, especially soon after the beginning of a new calendar year, errors in transcribing dates between different types of calendar, punching errors, coding errors, and a host of others. For the sake of brevity, the need to refer repeatedly to these ceteris paribus conditions is largely ignored in this chapter. 6. A "reform" has been proposed for the structure numbering required as addresses for PGE/ ERAD/ ECP surveys. These numbers are, it must be repeated, addresses or structures, not codes for housholds. The reform suggests that not only structures should be listed, but also households and individual persons (Scott, 1973: 412). In this manner the original sin of the household change technique would become an acquired sin of the PGE/ ERAD/ ECP. The "reform" would also assist in institutionalizing the Tunisian error described in l.S.c iii. 7. For an example of an arbitrary correction in Cambodia on the basis of what the figures "should" be, see Kannisto, 1973: 397. Even after the correction, four out of 14 provinces were found to show a fertility "too low". They were further upped. For Indonesia the crude death rates had to be similarly and arbitrarily, though "cautiously" doctored. The birth rates were beyond repair and had to be estimated from age distributions (ibid., 400). It is curious to note that both in Cambodia and Indonesia the deaths caused less trouble, that is less under-reporting, than births. It could be that the anthropology of these two countries is different from the rest of the world; or that there were some peculiar features in the two surveys. 8. The allegedly over-reported birth rates are below long term estimates derived from a painstaking analysis of three consecutive censuses (Krotki and Beaujot, 1975). 9. In Pakistan a match has been attempted between the households as reported during the last or fourth round of a series of survey rounds and the first of the next series of four rounds. With the resources available, and they were considerable, this proved an impossible task. The matching task was just too great. 10. Matching in the Pakistan PGE has been carried out between the documents collected by the continuous recorders and the documents of one of the four surveys. 11. Algebraically, equations (1.9) and (1.10) look like simple inversions and as such 51
Endnotes
Karol J. Krdtki
hardly deserve being written out fully. Also, in PGE/ ERAD/ ECP surveys with two way matching (see: Glossary) either formula will do. However, it is worthwhile to write out both of them because in one way matching(see: Glossary) only one of the formulae is operational. 12. It is less certain what is the correct PGE/ ERAD/ ECP attitude in this regard when it comes to using PGE/ ERAD/ ECP techniques to evaluate national censuses in the manner proposed in our chapter 10. 13. We offer a discussion of subsampling in section 2.2.a. For additional reviews of the subject see Fellegi, 1974: 15,23,24 and Markset a/., 1974: 388-400. 14. At a Paris seminar, when all sorts of outlandish suggestions had been made which disregarded the need to preserve the essential features of a PGE exercise, the possibility was advanced that three prestigious government agencies should appoint themselves a court that would grant the ERAD franchise only to proposals that met minimum ERAD requirements (Krotki, 1972). 15. Subsampling is dealt with again in section 1.10 in connection with costing. Briefly, instead of including the whole of each sample area in both procedures, only a proportion (say, a third) are included in one of the procedures, but all the area in the other procedure. This will pay if the cost of carrying out the work under one procedure is markedly higher than under the other procedure and provided that savings are possible in the former procedure, though this will seldom occur to the full extent of the cut in proportion. 16. The possibility of using two household surveys as the two procedures for a PGE/ ERAD/ ECP system is indicated towards the end of section 1.4 and in subsection l.Sb.iii. 17. The two rows for Morocco 1972-73 (macro and micro) in table 1.5 seem to provide some evidence consistent with this observation. When watched individually, the field workers give field costs about similar to those reported from Pakistan. However, when the actual and total expenditure is considered then they almost double for the same unit of output. 18. Presumably, all these initial overheads were charged to one year. For a different allocation of the initial costs see section 7.4. 19. It should be noted that the PGE handbook came out rather unfavourably disposed towards the idea of subsampling (Marks et a/., 1974: 338-399). Nevertheless, the matter is not yet closed. 20. The many blanks and the wide variations in the cost ratio shown in column (11) of table 1.3 reinforce the need for more data, if practising statisticians are to suggest to demographers realistically optimum designs. Furthermore, it will be noted that the empirical intraclass correlation available to us was lower than the lowest suggested schematically in an overview of African samples (Scott, 1968: table 2) and the empirical cost ratio available to us was higher than the highest in the same overview. Both considerations move the optimum cluster size away from the alleged 300 in Africa towards our 1,000 and above it. And, the discussion has not yet taken into account the nonsampling components of the total error as defined by the mean square error. 21. Here I am assuming that only the matching of two independent operational systems — census and vital statistics, for example — and the use of estimation techniques as outlined by Krotki do not constitute a PGE/ ERAD/ ECP system. 22. The discussant made a notable contribution to numerous attempts to evaluate the quality of Pakistan demographic data (e.g. Bean, 1974). [Editor's note.] 23.1 am assuming dates such as 1966-67 refer to fiscal years and have therefore counted these as one year. 24.1 personally find that the personalized references, such as referring to Christopher Scott as the "high priest of the household change technique" detract from the argument. 52
Chapter 2 The State of the Art in Dual Systems for Measuring Population Change H. Bradley Wells and Daniel G. Horvitz 2.1 Introduction During the 1960s, major national trials of different dual collection designs have been carried out in Pakistan, India, Thailand, Turkey, Liberia, (Wells, 1971; Marks et al., 1974) and the Philippines (Philippines, 1973). The largest of these, the Indian Sample Registration System, was started with pilot studies in 1963 and is now firmly established as the major source of current vital statistics data for the country (India, 1972). The original Pakistani, Thai, and Turkish systems have been deactivated but plans are being made to establish new systems in both Pakistan and Turkey. The Liberian system has been in operation since 1969. The Philippines system (not to be confused with the Mindanao Center for Population Studies) is the newest and is in the third year of operation. Morocco is planning to expand its current program to a national system. These national systems were developed in an effort to meet demands of social and economic planners for basic population data, especially birth, death, and growth rates. Some have succeeded to a large extent in meeting those needs — some have failed, for various reasons. All have encountered difficulties both administrative and technical. Many of the difficulties were of the type that occur in the implementation of any new statistical program. Other problems were due to the need for co-ordinating the two procedures so that the collection of records, processing, and matching could be done with acceptable levels of accuracy. Considerable effort in these ongoing programs is devoted to methods research in dual collection systems. Many other studies utilizing the dual collection system approach have been carried out for areas smaller than a country. These include the experimental field trials conducted in Colombia, Kenya, Morocco, and Northern Mindanao by collaborating institutions in the International Program of Laboratories for Population Statistics (POPLAB) under the direction of Dr. Forrest E. Linder at the University of North Carolina. Much more research is still being done in single than in dual system methodology, but a great deal of that is also pertinent to dual systems. Detailed reviews of many of the single system studies, as well as all of the country systems and early historical developments of dual collection systems, may be found elsewhere(Mauldin, 1966; Wells, 1971; Marks etal., 1974). Our aim in this chapter is to review systematically some of the major methodological issues in the design and management of dual collection systems and attempt to describe procedures for making rule of thumb decisions. Because of space limitations the amount of detail will necessarily be restricted. We make no claim that the ideas and 53
2. /
H. Bradley Wells and Daniel G. Horvitz
recommendations are new, indeed most of them have been described and advocated by others. In particular, for a much more detailed and comprehensive discussion of the problems and alternatives in designing dual systems the Population Council Handbook for Population Growth Estimation Studies (PGE handbook) by Eli Marks, William Seltzer, and Karol Krotki (1974) is highly recommended. Dual collection systems are without question more complex to design and administer than two single systems but their accuracy in terms of mean square error may more than offset the added cost in well executed dual systems. Unlike a single system, a dual system includes nonsampling error detection and correction procedures as an integral component and this is the unique feature that distinguishes the dual system and makes it potentially more cost effective than most single systems. Thus the dual system is a logical extension of the usual sample survey model in that it includes measurement of an adjustment for nonsampling as well as sampling error components. In spite of the general conclusion that dual systems are more efficient, in the statistical sense, there is a great need for detailed data about design parameters and cultural variations in order to develop optimal or nearly optimal design in given situations. We will largely limit our considerations to the three essential components of the dual collection system: (i) a continuous recording procedure; (ii) a periodic survey procedure; and (iii) procedures for matching of records from the two sources. Some sample design questions will be discussed first. We assume that the sampling frame will consist of clusters of households located in geographic areas with identifiable boundaries.
2.2 Sample design 2.2.a Some general sampling considerations A major advantage of probability samples is that the data collected can be used to assess the magnitude of the sampling error in estimates of population means and totals derived from the same data. The usual formulae for computing the error variance for complex sample designs, when measurement errors and other nonsampling errors are also present in the data, will include their contribution to the total error of estimates, provided these measurement errors are uncorrelated and have expectation zero. In this situation no special modification to the sample design is necessary. Steps should be taken, of course, to reduce the added component of variance due to the nonsampling errors if that component seriously affects the total error. This may be accomplished by improving the measurement process with tighter statisticial control, or by merely increasing the sample size. This not too unhappy circumstance does not obtain when the errors of measurement are correlated, in which case the usual variance formulae are biased and increasing the sample size will not significantly reduce the contribution of this component to the total error variance. Finally, if the measurements on all observation units in the sample are subject to an unknown systematic error component or bias, then estimates computed from the sample will be subject to this same bias. There is little doubt that sample household surveys designed to measure vital rates suffer from measurement errors which are correlated and which contain systematic components. Since these errors and biases, in fact, seriously affect the accuracy of vital rates estimates, it is essential that surveys concerned with demographic measurement be so designed that a proper balance is achieved between the magnitude of the sampling and nonsampling error components of the total error of estimate (measured by the mean square error). Dual data collection systems for measuring population change offer not only an excellent opportunity to accomplish a significant increase in the 54
State of the art
2.2. a
accuracy of estimates of vital events, but also to achieve this more efficiently through appropriate allocation of resources (Marks, 1971). 2.2. b The need for realistic parameters When dual collection systems for collecting data on vital events were first introduced, there was but limited knowledge concerning the most appropriate choices to make among the large number of design parameters under the control of the investigator. Information of specific value in judging or assessing the relative efficiency of alternative dual system data collection procedures was just not available. For example, completeness rates for active registration with local recorders or for quarterly household surveys versus annual surveys were unknown. And there was little if any accumulation of data on variances, covariances, intracluster correlations, biases (e.g. due to reporting of out-of-scope events, temporal or geographic), costs and other indices essential to a cost effective evaluation of alternative dual collection systems. Although still far from ideal, the situation is much improved today to the point that serious consideration can be given to choosing parameters which will tend to optimize a dual collection system, in the sense of minimizing the total error of estimated vital events or vital rates for a given data collection budget. It should be recognized, of course, that the optimal design for estimating total births will not necessarily be optimal for estimating the birth rate or any other measures such as total deaths or death rates. Some guidance in the choice of more efficient designs has been provided by Marks (1971) and Marks et al. (1974), with initial emphasis on the number of sample clusters or subclusters to select for each procedure which will minimize the relvariance of the dual system estimate of the adjusted number of births (or deaths) for a given cost with a simple random sample of clusters. The design parameters to be specified are: ki = the number of sample clusters to select for continuous recording; k2 = the number of sample clusters to select for periodic surveys; the k2 clusters are either identical to or a subsample of the ki clusters, i.e., fc^k,;1
H = the size of the sample clusters in terms of the number of subclusters of households for continuous recording; h = the number of sample subclusters to select at random in each cluster for the periodic surveys, i.e., h ^ H. The usual dual collection system for estimating the number of births (or number of deaths) will have k2 = ki and h = H = 1. These are not necessarily the optimum design choices. Under the assumptions (i) that the intraclass correlation within clusters is the same for Ni, N2 and M (as defined in Marks et al. Chapter 2), (ii) that the match rate for each procedure is independent of the number of events reported by that procedure, (iii) that the number of vital events occurring in a cluster per unit of time is a Poisson variate, and (iv) that the two procedures are in fact independent, Marks (1971) shows (for h = H) that the optimum subsampling rate for k2 is given by: where P1 and P2 are the probabilities that the vital events of interest will be reported by the respective procedures, CR and Cs are the respective variable costs per sample subcluster and the total variable cost is C = CakiH + Csk2h. If Pi = P2 = -7, selecting a subsample of the ki sample clusters for coverage by the periodic survey would only be more desirable than surveying all ki if Cs were somewhat greater than CR. For P\ > P2 > .8, it still might be advisable to select a subsample of the ki clusters for the periodic survey even if CR = Cs. 55
2.2.b
H. Bradley Wells and Daniel G. Horvitz
The optimum subsampling rate f = h/ H (within clusters) for the periodic survey is also given by the right side of equation (2.1) above, under the additional assumptions that k2 = ki, that full-time resident recorders must be used with the continuous recording procedure in order to reduce costs and to maintain better statistical control, and that the intracluster correlation is zero. The effect of a positive intracluster correlation is to reduce the optimum within cluster subsampling rate for the periodic surveys. For PI = P2 = .7, equation (2.1) suggests that within cluster subsampling would be desirable if Cs is somewhat larger than CR (when k2 = ki). The gain in precision brought about by subsampling within clusters and using the savings to increase the number of clusters should be weighed against the added design complexity. The above results, while providing some guidance, are still restricted to particular parameter combinations (i.e., h = H in the first instance and k2 = ki in the second) and therefore yield only locally optimum designs. For countries or cultures for which estimates of the variances and covariances for Ni, N2 and M for one or more cluster sizes are available, and for which an appropriate cost function and reasonably accurate variable cost coefficients are known, unconditional solutions for the optimum combination of the design parameters ki, k2, h and H should be computed. Other considerations enter into the choice of cluster size than merely that value of H which, together with appropriate choices of ki, k2 and h minimizes the relvariance of the estimated number of births (or the estimated birth rate) for a given survey budget. The cluster size must be such that boundaries for the area containing the cluster can be specified and easily identified in the field. As H decreases, the boundary identification problem increases. This can lead to the inclusion of events which do not belong to sample clusters and to the exclusion of events which do belong. Also migration or local moves can become a significant factor for small clusters. Further, the variation in the adjusted number of events n may be somewhat greater for small clusters since the number matched M, might then be too small for stable estimates of N resulting in a larger ratio bias. Thus, dual collection systems for estimating vital events and vital rates require somewhat larger cluster sizes than would ordinarily be appropriate in household surveys. 2.2.c Rotation designs An important objective in most national vital event dual collection systems is to be able to detect changes in fertility and mortality, as well as to estimate their magnitude accurately. If an estimate of change from the preceding year were the only objective, then the same sample of clusters should be used for both years. If an estimate of average level for each of the two years were the only objective, then a fresh sample of clusters should be selected for the second year. Since there is interest in both change and level, a rotation sample in which some clusters are retained and other clusters are dropped each year would be most appropriate. The argument for some rotation of sample clusters is strengthened further by the need to minimize respondent conditioning effects and respondent fatigue, both of which can lead to unknown biases of response (Marks et al., 1974,396 and 397). 2.2.d Some aids to improved estimation Despite a sound sample design and appropriate field procedures with adequate controls for quality, the dual system estimate of the number of vital events may still be subject to unknown biases. Correlation bias occurs when the two procedures are not operating independently. Completeness bias occurs when vital events are included or 56
State of the art
2.2.d
excluded erroneously in either or both procedures due to errors in reporting the time of the event or the geographic location of the event. When completeness of reporting is decidedly different for subgroups of the population for both procedures, correlation bias will result. This can be reduced by dividing the sample into appropriate groups (post-stratification), such as by age of mother or type of event (i.e. infant death and non-infant death) and estimating the number of events separately for each group. This procedure will reduce the correlation bias provided that each group is homogeneous with respect to completeness of reporting. Care should be taken that the groups are not so small as to increase the ratio bias substantially (Chandrasekaran and Deming, 1949). Completeness bias can be reduced in a similar fashion, that is by grouping the population or the events into homogeneous strata with respect to the probability of out-of-scope errors, preparing separate estimates for each group, and then combining the group estimates. For example, to adjust for boundary errors, events reported for households in areas on or near the boundaries of the cluster, or events reported in the first and last months of the reference period can be classified into a single stratum for events with a "high probability of out-of-scope error" and the remaining events into a second stratum. The usual matching and adjusted estimate can be computed for the latter stratum. The estimate for the first stratum would not be based on matching, but rather on an average of the events observed by the two procedures. Rules for classifying out-of-scope events are necessary, such that the number included in the first stratum balances (approximately) the estimated number of in-scope omissions (Marks et a/., 1974). Completeness bias may be introduced by migration. The continuous recording procedure may miss births which occurred to in-migrants prior to moving to the sample cluster, but during the survey period. The periodic survey may pick up such births, but miss births to out-migrants that the continuous recording picked up. In order to obtain an improved estimate, the sample population is divided into two groups. Those households which resided in a sample cluster for the entire period are placed in the first group. All migrant households are placed in the second stratum. Again, the usual dual system estimate n is computed for the first stratum. Births (Nn) for the households that moved out of the cluster can be estimated by multiplying the number reported by the reciprocal of the completeness rate for the periodic survey for the non-migrant households (i.e. N2/ M). Births (N22) for the households that moved into the cluster can be estimated by multiplying the number reported by the reciprocal of the completeness rate for the continuous recording procedure for the non-migrant households (i.e. Ni/M). The average of these latter two estimates for the migrant households is used for the second stratum estimate.
2.3 The data collection elements of a dual collection system The first three essential data collection elements of a dual collection system are: 2,3.a The essential elements of a dual collection system i. A (continuous) recording procedure. This is the first source for recording births and deaths and it may be an existing record source such as civil registration. More commonly it is a special procedure somewhat different in nature from either the civil registration system or the usual sample survey. ii. A (periodic) survey procedure. This almost always is designed specifically as the second source of data to be matched with the recordingprocedure. This may be a single
57
2.3.a
H. Bradley Wells and Daniel G. Horvitz
visit survey but more often it will be a longitudinal periodic (multiple visit) household survey. iii. Procedures for the matching of recording and survey events. These will include designs for record flow, data processing, and matching of records from both procedures as well as procedures for field recheck or verification of unmatched and doubtfully matched events. The dual system of data collection for measuring births and deaths is much more complex than the simple sum of the complexities due to two single procedures producing the same reports. It requires co-ordinating the design, definitions, training of field and office staff, scheduling of field work, and at the same time maintaining as much independence as is practicable between the two procedures. In designing data collection procedures for each of the procedures, a number of basic questions must be answered bearing in mind the objectives and the cost factors in relation to the precision required. These include questions concerning the content of event records and interview schedules; the wording of items; the field procedures, including frequency of visits to informants or households; the use of full-time versus part-time staff (part-time continuous or full-time intermittent employment); staff qualifications in terms of education and experience; residency of the field staff (local or non-local residents); training and supervisory procedures; and the flow of records and reports. Clearly the answers to these questions are interrelated. They are also related to sample design factors including cluster size, subsampling, and whether the sample is fixed or rotating, and to the levels of literacy of the survey population, and to the availability of qualified persons to staff the system at all levels. Whether or not subsampling is used, a relatively fool-proof field identification system is required as a fourth essential element in a dual collection system. iv. Field identification procedures. These are necessary in order that the sample units can be consistently and unambiguously located by field workers of either procedure. In the event that only a single survey is to be conducted for identifying events to be matched with a recording procedure that already exists (for example, measuring the completeness of civil registration), the field identification procedure must be capable of correctly assigning records from either procedure to the same (small) geographic areas in the matching process. Field identification procedures are crucial in minimizing the biases that can result if area boundaries of the two procedures do not correspond. Even when a good field identification system has been developed there will be errors in using it. See Cooke (1971) for a general description and Madigan et al. (1974) for a specific illustration of approaches to the field identification system. 2.3.b The continuous recording procedure The major objective of a recording procedure is to obtain as complete a list as possible of births and deaths in the sample population. Recording procedures can encompass a wide range but may generally be classified into four groups based upon the combination of legal status and effort required of field staff as depicted in figure 2.1. Most civil registration systems fall into group i. Examples of dual collection efforts related to this are the U.S. birth registration tests of 1940,1950, and 1968; Thailand, 1964-66; Singur 1946-47; and Vaso Town 1966-68 (Marks et al., 1974). We are not aware of any recording procedures in group ii nor of purely group iii procedures. Dual systems in which active efforts by field staff to supplement the usual passive civil registration are relatively few but are illustrated by the Peruvian effort (Cavanaugh, 1963) and a large national system now operated in the Philippines by the Bureau of Census and Statistics.2 Such efforts are a combination of i and iii. 58
State of the art
2.3.b
Figure 2.1 The four types of registration procedure Effort of Field staff Passive Active
Legal status Civil registration Special registration i iii
ii iv
Most recording procedures, including the current Indian system and the early Pakistan and Turkish efforts, fall into group iv. The dual collection systems being tested by POPLAB use group iv recording procedures. There are two important variations in the activities followed in recording procedures of group iv. These two recording variations can also be used together, resulting in three types ranked in terms of the intensity of recording contact with households: i. Informant or "routine round contact" procedure (contact household only if vital event is reported). Pakistan and early Indian experience were based upon this method. ii. Periodic household visits (whether or not vital event is reported). The early Turkish and the Liberian efforts were of this type. iii. Combination of i and ii. The POPLABs at Xavier University in Mindanao and Kenya are experimenting with this approach and India is now using it in some states. The recording procedure based on routine round (RR) contact must at a minimum include the following four activities by the recorder designed specifically to minimize household visits: i. Setting up a group of local persons (routine round contacts) who are expected by virtue of their professional or social status to be aware of vital events occurring in the sample area and who are willing to report them to the recorder. ii. Visiting the RR contacts on a regular basis (weekly or bi-weekly) and asking about families which experienced vital events. iii. Visiting those families for which events are reported and completing a detailed vital event report for each event. iv. Transmitting individual vital event reports to headquarters. In addition the recorder might optionally be required to maintain a special register of vital events, update a household listing,3 and up-date maps periodically. The recording procedure based upon repeated household visits is, in effect, a periodic survey since every household is supposed to be contacted at intervals, whether or not they have had a vital event. The household visit approach requires that the recorder, at a minimum: (i) visit each household at specified intervals and inquire directly about whether any vital events have occurred; (ii) complete a vital event report for each reported event; (iii) transmit the reports to headquarters. In addition the recorder may, as in the informant system, be required to maintain a register of vital events, and up-date household listings, maps, dwelling lists, and the numbering system. The recorder may also be required to report migrations, for example, as in Turkey and Liberia. In India, after the same sample units had been covered for a number of years, estimated completeness rates for the informant-based recording procedure began to decline. Recorders are now required to conduct quarterly household visits in addition to their usual visits to informants (India, 1972).
59
2.3.b
H. Bradley Wells and Daniel G. Horvitz
Preliminary results from part of a rather complicated experiment done in the Colombian POPLAB (table 2.1) show higher recording completeness rates in both rural and urban samples for household visits than for the informant system. Overall birth rates are also higher where the recording procedure used household visits. This suggests a lack of independence between the recording and survey procedures. The results are difficult to interpret since survey completeness was better than recording completeness when household visits were used in the first period, but poorer than recording completeness when household visits were used in the second period. There was an overall decline in survey completeness in the second period which was expected, since two three-month surveys were used in period I and one six-month survey in period II. On the other hand overall recording completeness increased in the second period. Taken at face value these preliminary Colombian results are disturbing because the estimated birth rates vary so greatly with the different factors and combination of factors. It is possible that unknown correlations may be responsible for the observed differences. If this is borne out in the more detailed analyses now underway it would suggest that more intensive efforts are required to develop data collection and estimation techniques appropriate for quasi-independent procedures rather than continuing to use the independence assumption.
Table 2.1 Estimated completeness rates and crude birth rates by area, periods under different recorder operations, switch-back trial*, Colombia, 1971-72.
Area and procedure
Sequence of recorder activities and period Household visits Period I, Informants Period I, informants Period II household visits Period II Period Period I II Change I II Change Completeness rates (percent)
Rural area Recorder Survey Urban area Recorder Survey
81 86
88 68
+7 -18
68 73
90 65
+22 -8
76 82
77 60
+1
-22
62 70
87 70
+25 0
41 29
58 33
+ 17 +4
Crude birth rates per 1.000 Rural area Urban area
53 28
42 25
-11 -3
* At the end of Period I recorders in sample areas where monthly household visits had been used were trained to use the informant system in Period II and vice versa for the other half of the sample areas. Note: Period I is 1 October, 1971 - 31 March, 1972; Period II is 1 April, 1972 - 30 September, 1972. Source: Adapted from data in table 4 shown by permission of Departmento Administrative National de Estadistica in Myers and Lingner (1973). 60
State of the art
2.3.c
2.3.c The household survey procedure Experience with survey techniques and procedures for measuring population phenomena is much more widespread and well known than experience with the recording procedure. There are, however, a much wider variety of survey options which can be used as a component in a dual collection system. The first objective of the survey procedure is to obtain reports of a high proportion of the vital events which occurred in the sample during a time period which overlaps completely with that of the recording procedure. Except in instances where the population base for vital rates is obtained from an outside source, such as a census, the second objective of the survey is to provide the population base. When the household survey is repeated periodically, a third objective can be to collect substantive data of special interest and this content may be varied from time to time.4 A major feature of any survey design is the frequency of interview. This can be crucial in a periodic multiround survey. On the one hand frequent interviewing may cause respondent conditioning and/ or fatigue. On the other hand, interviews spaced too far apart may result in many survey households being lost because of migration and in under-reporting due to recall lapse about events of interest. A dual collection system requires a minimum of one survey to collect vital event reports for the same period prior to interview for which events were covered by the recording procedure. Theoretically there is no limit to the maximum frequency of survey interview, but the practical constraints of costs and respondent fatigue force limits to be set. The optimum survey frequency is not known, either for dual collection systems or for single surveys. Furthermore, what is optimal for one country may be less than optimal in another. The most popular survey frequency in continuing dual collection studies has been six months, although Pakistan and Thailand used three-month intervals. Experimental testing of three, six and 12 month survey frequencies with nonoverlapping recall periods has been carried out in the Colombia POPLAB. Preliminary results are shown in table 2.2. Three-month frequencies were investigated only in Santander. With the exception of period II survey results for births in Santander (lines d and e) the results support the conclusion that survey coverage is more complete for more frequent visits; two three-month surveys are more complete than one six-month survey, and similarly two six-month surveys are more complete than one twelve-month survey. Differences are more marked for deaths than for births. Also the estimated completeness rates in Santander increased rather markedly from period I to period II, perhaps due to better training and practice or to increasing dependence between procedures or to a combination of these. It is also possible that the frequency of survey results have been confounded with other factors in the design (see table 2.1 and the discussion in section 2.3.b). The effect of using different frequencies with overlapping recall periods is being tested in India and in several other POPLABs. A closely related question is the length of the recall period for which respondents are asked to report vital events. A recall period which is longer than the interval between repeated interviews is called an overlapping period. In Pakistan interviews were conducted every three months but respondents at each interview were asked to report all events for the previous 12 months. Thus each event had four chances to be reported in the survey. In Turkey and Liberia interviews were done at six-month intervals, in January and July. The July interview used a non-overlapping recall period of only six months while the January interview used a recall period of 12 months, a sixmonth overlap.5 A major virtue of overlapping reference periods in a single system periodic multiround survey is that it minimizes the effects of time telescoping of events, provided 61
H. Bradley Wells and Daniel G. Horvitz
2.3.c
Table 2.2 Estimated completeness rates for births and deaths by recording and survey procedures by period and frequency of survey, Santander and Bolivar, Colombia, 1971-73. Study period and dates
Number and frequency ofsurvey(s)
Births Recorder Survey
Births Recorder Survey
Santander Period I Oct. 1971 to March 1972 Period II April 1972 to March 1973
a) Two 3-month b) One 6-month
71 77
85 73
68 58
65 41
c) One 6-month, Oct. 1972 to March 1973 d) Two 6-month e) One 12-month
83
82
83
53
84 85
75 76
81 85
58 54
81 80
77 65
83 75
58 46
Bolivar Period II April 1972 to March 1973
f) Two 6-month g) One 12-month
Source: Data adapted from Colombia Departmento Administrativo National De Estadistica, 1973, "Primeros Resultados del Estudio ERED". Centro de Investigaciones en Metodos Estadisticos Para Demographia, Bogota, Junio 1973.
records from different rounds are matched and duplicates are eliminated. It is not clear that this virtue of a single system multiround survey will be desirable in a dual collection system. If it is used, the record keeping and matching operation within the survey itself increases in volume and quite likely in complexity. A compromise approach uses the overlapping recall period for the interview and then processes only those event records which are reported in the most recent nonoverlapping reference period of interest. This approach may improve time reporting and will minimize the number of reports for processing. Further research on the value of overlapping periods in a dual collection system is needed. Measurement bias is a major consideration in determining optimal survey frequency and length of recall periods. The manner in which recall lapse, telescoping errors, respondent fatigue, respondent conditioning, and the changing composition of the sample over time influence measurement errors, has not been systematically studied in dual collection systems, nor indeed in single systems, for measuring population change. As indicated by Marks el al. (1974) the range of choices for survey procedure designs depends so directly upon the recording procedure that it is unwise to make recommendations without considering a fairly wide range of feasible alternatives. If the objectives of the survey are to provide both birth and death events for matching and population data for the denominator, we would be inclined to recommend six-month survey intervals, provided the budget can support them.
62
State of the art
2.3.d
2.3.d Matching procedures The objective of the matching operation in a dual collection system is to yield minimum matching bias and matching variance as defined by Seltzer and Adlakha (1974), and in Marks et al. (1974). On the basis of experience to date Marks et al. (1974) recommend, and we agree, that matching be done by hand rather than machine or computer even in a large scale, country-wide dual collection system, especially in the developmental stages. It is also considered essential that matching procedures be developed specifically for the two field procedures as they actually operate in a given setting. A set of vital reports whose "true" match status is known is required to assess the value of using different characteristics in alternative matching rules. This is essentially a "boot-strap" operation as recommended by Marks et al. (1974) and summarized as follows by Madigan and Wells (1973). "Developing matching procedures for a new dual collection system involves a number of steps: i. Select or create a sample of 300-600 reports of births or deaths from each of the two procedures; these could come from a pretest or from the first stages of an actual collection program. The completed reports would of course have to contain the characteristics which are to be considered in developing the matching rules. Different characteristics will usually be required for deaths than for births. ii. Match the events from the two procedures as accurately and carefully as possible. At this stage all information on the reports, and knowledge of cultural characteristics, and interviewing procedures should all be utilized implicitly in arriving at decisions as to what constitutes a match. It is recommended that this be done independently by three junior or senior professionals. After the three have independently agreed on a match or a non-match these are considered as 'true or correct'. If only two of the three agree on a report then all are asked to reconsider. After reconsideration the remaining reports on which the three disagree as well as the non-matches are sent back to the field for verification or additional information.6 After the field check some outof-scope events may be eliminated and final decisions are made regarding the 'true' match status of the remaining reports thus creating the standard against which alternative matching rules can be judged. iii. Erroneous matches and non-matches for individual matching characteristics are then compared over the feasible range of tolerance limits by matching only on single characteristics. The sum of erroneous matches and erroneous non-matches is defined as the gross matching error and their difference is defined as the net matching error for a characteristic. iv. The data on gross and net matching error as an aid in choosing the characteristic set or subsets and the corresponding weight for each characteristic which will become the basis for the matching rules either explicit or implicit to be applied in the subsequent matching operation. There are usually an extremely large number of possible characteristic sets but a reasonably good explicit matching procedure can usually be developed by examination of the gross errors of individual characteristics. For an individual characteristic gross error shows the relative ability of a specific cutting point or tolerance limit to place an event in the correct class and tolerance limits should be chosen to minimize gross matching error for the characteristic. Net error, taking account of sign, indicates the direction of bias for the individual characteristic and therefore helps determine the relative weight which should be given to a match or nonmatch on that characteristic and vice versa for a variable which yields a high erroneous match rate". One approach to choosing a set of characteristics with minimum net matching error is to rank the individual characteristics (at chosen tolerance limits) from low to 63
2.3.d
H. Bradley Wells and Daniel G. Horvitz
high in terms of erroneous matching error (reverse of discriminating power) and consider the first, the first two, and first three, etc. in combination and choose the set which minimizes net matching error for low gross matching error. Another approach is to assign different weights to the different characteristics so as to achieve the same goal. Weights may be based upon judgement and empirical evidence, i.e. the gross and net errors, and can be derived approximately without considering all possible characteristic sets. Alternatively, weights may be derived through statistical methods such as discriminant or cluster analysis, ideally by considering all combinations of characteristics and cutting points through computer analysis. Although the above procedures have been proposed, no application has yet been reported in which these procedures have been compared within the same study against alternative rules. Madigan and his co-workers are conducting such studies (Madigan and Wells, 1973). Myers and Lingner (1973) described the characteristics and tolerance limits being used for matching in the various POPLABs and there were few similarities other than the use of names of decedents and newborns and their parents. There is great need for studies in which results of such "approximate" rules are compared with "optimum" rules based upon consideration of all combinations of characteristics. 2.3.e Field identification procedures and other factors common to both collection procedures Several other factors which influence the design of interview, editing, and supervisory procedures are relevant. These all relate more or less directly to the content of one or both collection procedures. For convenience we include them in this section although most are discussed only briefly. Individual vital event reports in each procedure must, as a minimum, include sufficient detail to determine whether the event matches or corresponds to an event for the same time period in the other procedure. If the survey is repeated periodically the same basic detail would be required in each round to match within the survey and eliminate duplicate survey event reports. Elimination of duplicate survey events is especially critical when there are overlapping reference periods for the survey (Sabagh and Scott, 1967). Minimum essential items for event reports in any survey which is part of a dual Figure 2.2 Minimum questionnaire content for record keeping and matching operations Content
Survey only Household schedule
Survey and recorder Birth Death report report
Geographic identification Details on household members Births in report period Deaths in report period Place event occurred Particulars: infant /decedent Mother /father Fat her /wife
X X X X
X
64
X X
X
X X X X
X X X X
State of the art
2.3.e
collection system to provide reports of births and deaths and denominator data for estimation of vital rates are shown in figure 2.2. Abernathy and Lunde (1972) described the record content in dual collection systems in India, Pakistan, Turkey, and Liberia. Marks et al. (1974) include a detailed checklist of suggested items for inclusion and an indication of how useful each item is for the different purposes of matching, demographic analyses, and survey control. Decisions must be made at the outset regarding counting rules for the numerator events, birth or death, and for the denominator, total population or women exposed to risk. Two common ways of calculating vital rates are: (i) on the basis of residence (de jure} i.e. events to residents during the year divided by the number of persons residing in the area at midyear, and (ii) on the basis of occurrence (de facto) i.e. events which occur in the area during the year divided by the number of persons occupy ing (living in) the area at midyear. Usually the denominators for crude rates, the total populations at midyear are not very different for the two rates; however, for age specific rates or sex-age specific rates "residents" and "occupants" might differ more in some groups than others, depending upon migration rates and the definitions of migrants. Further, when vital events and population data are from civil registration and census (or projections of the census) respectively, the denominators often are identical in either case. Dual collection systems usually collect data for both numerator and denominator because they attempt to follow residents/ occupants in sample areas over time. Migration, changes in household composition, and travel outside the sample area for delivery of an infant or for hospitalization before death create special problems. Decisions must also be made on handling vital events which occur to inmates of institutions such as prisons, hotels, and long-term hospitals. The continuous recording procedure, especially if it relies upon RR contacts, is more likely than the periodic survey procedure to find and record events occurring to non-residents or visitors in the sample area. The periodic survey on the other hand is perhaps more likely to find events which occurred elsewhere to newcomers or to residents temporarily absent; it also is more likely to miss early infant deaths, or deaths which caused the breakup and migration of a family which has been resident in the area. The de facto approach was used in the Pakistan country system (Marks et al., 1974), Philippines dual collection system, and in the Moroccan POPLAB. India and the MCPS collect both de facto and dejure vital events but after matching and field verification,7 analysis is based upon dejure events. The Turkish Demographic Survey, the Liberian, the Colombian, Kenyan, and the proposed new Turkish systems use the dejure approach to counting vital events. Determination of "residence" classification for the population requires at a minimum that a question on "usual" residence be asked for each individual at the time of survey. It is possible to ask a whole series of additional questions to establish residence more accurately, such as split places of abode, student status, intent to reside here, and duration of stay here. It is recommended however that the simplest counting rules consistent with the usual statistical definitions of the country be used by both procedures. Both defacto or dejure approaches should yield the same result for a representative country-wide sample. In either case it is usually best to use other means to measure vital events occurring to the institutional population. In the pre-test phases of developing a system it would be wise to test procedures for collecting both de facto and dejure events and to develop and test matching and field reverification procedures which would assist in arriving at "the most probable" 65
2.3.e
H. Bradley Wells and Daniel G. Horvitz
classification status of each questionable event. This would provide a basis for deciding whether analysis should be based upon a dejure or a de facto approach. Whatever approach is used the procedures and subject matter content must be designed to include questions in sufficient detail to permit residence classification of an event or a person. It should also minimize possible multiple counting of events for those who move within the sample area. Some POPLABs are collecting data on children ever born and children surviving, in addition to children born last year, so that estimates of fertility obtained by stable population or Brass estimation procedures (United Nations Population Studies 42) can be compared with dual collection estimates. Data on children ever born and children surviving can be obtained in a number of ways. As a minimum, for each woman (or ever married woman) age 15 years to some age beyond childbearing, two questions must be asked: (i) How many live births have you (she) had, and (ii) how many are still alive? At the other extreme, the data can be obtained by reconstructing a complete pregnancy history for each woman in the childbearing ages. The timing of each live birth from puberty onward should be asked for and the subsequent survival status of each live birth. As well, there should be an accounting for any long interlive birth intervals by probing for the use of contraceptives or foetal death. Because there may be differential levels of completeness in reporting births by sex, survival status, and whether the child is living in the home, it is sometimes recommended that three bits of data be collected separately by sex of offspring for each woman: children born alive and (i) living here; (ii) living elsewhere; (iii) now dead. Because of concern with reporting errors Madigan and Herrin(1973) also recorded whether or not the data on children ever born and children surviving were self or proxy reported. Some results are shown in figure 2.3.
Figure 2.3 Relative differences in average number of children ever born and proportion surviving for self and proxy reporters, ever married women, urban and rural areas, Mindanao Center for Population Studies, July 1972 66
State of the art
2.3.e
Selfreporting ever married women in the urban area reported higher average numbers of children ever born at every age than proxy reported women. The differences between self and proxy reporters were greatest between 20-29 years of age, somewhat lower under age 20, and decreased to roughly less than 10 percent at ages 40 and above. For rural women a similar age trend in differences in the average of children ever born was observed, but the relative differences between self and proxy reporters were quite a bit lower than the urban ratios at every age above 20 years. For rural women above age 40 there were no clear cut differences between self and proxy reporters. Ratios of the proportion of children surviving show much smaller differences between self and proxy reporters. Considered as a whole these ratios decline with age for both urban and rural women. At almost every age the ratio of self to proxy reports of the proportion surviving is lower for urban than rural women. Taken at face value it would appear that something more than simple proxy under-reporting of children born and children surviving may be responsible for such results. One can speculate that: i. Under-reporting is related to familial relationship and the degree of association of proxy reporters and the women for whom they report. These would be closer in the rural than urban areas. ii. Cultural factors, such as reluctance to report deaths, might also be related to the age of the respondent. Proxy respondents would tend to be older for young mothers and younger for old mothers; for example daughters might report more often for women over age 40. This might be a possible explanation of the decline with age in the reported survival rates. Further research into the possible effects of proxy versus selfreporting should be conducted because this obviously has potential for introducing bias into any survey regardless of the method of analysis used. Additional interesting preliminary results provided by Madiganef a/., (1973) from the Philippine POPLAB experience shown in figure 2.4 illustrate negative correlation. In almost every month, both for rural and urban areas, the completeness rates for one procedure increases while the rates for the other decreases. Baseline household listings were done in the summer of 1971 and the continuous recording procedure was started immediately thereafter. Retrospective surveys were conducted in January/ February 1972 with a recall period from time of interview to Christmas 1970, and again in July/August 1972 using essentially a non-overlapping recall period from date of interview to Christmas 1972. Hence events reported in the January/ February survey had a second chance of being reported again in the July/ August survey. In addition, the recorder or interviewer who first reported an event was paid a bonus. Recorders were instructed not to make calls in their area during the period when survey interviewers were in the field. Analysis and matching, however, were based upon the period 1 January - 30 June, 1972. Events reported as occurring outside of this period were excluded. The fall and rise in survey completeness and the rise and fall of recording completeness over the reference period suggests that the observed changes occur because of what happens near the end points of the reference period. Taking the estimated completeness rates at face value the survey pattern might well be due to a combination of double coverage in January and the recall lapse phenomenon. The low completeness levels for the recorder at the start and end of the period may be due to a combination of the incentive effect and time delays in reporting. Similar analyses in studies without incentives, for different interview frequencies, overlapping recall periods and for informant based recording procedures will help to throw light upon the influence of some of these factors. 67
2.3.e
H. Bradley Wells and Daniel G. Horvitz
Figure 2.4 Estimated completeness rates by reported month of birth, recording and survey procedures, by area, Dual Record Study, Mindanao Center for Population Studies, JanuaryJune 1972
Innovative advances in single system survey methods for measuring population change have also been made in recent years. Procedures developed are useful also in dual collection systems for control of field work and processing records from the two procedures. Sabagh and Scott (1967) describe the results and limitations of the multiple round 1961-63 Moroccan survey. The matching and reconciliation procedures followed in that survey are recommended reading for anyone faced with the task of designing field and office procedures for matching and maintaining household and individual records for multiple-round periodic surveys, whether or not they are part of a dual collection system. Similar procedures are described for the Population Growth Survey in Liberia (Liberia, 1969). The longitudinal periodic survey and person-years of exposure accounting procedures for denominators developed by CELADE (Arretx and Somoza, 1965), and applied on a national scale in Honduras (1972), bear further investigation as an alternative or supplementary approach to the usual periodic survey component in dual collection systems. Further study, through analytical and simulation models, is needed to explore the magnitude of possible biases in various estimation procedures such as Brass, and other stable population procedures, and Chandrasekaran-Deming in the presence of different types of nonsampling errors such as age-misreporting, telescoping of vital events in time and space (location), memory lapse or biases in reporting of children born alive but now dead, and differential reporting by sex or other characteristics. 68
State of the art
2.4
2.4 Conclusion We have attempted to examine some of the inherent limitations as well as problems yet to be solved in the design of dual collection systems. It certainly was not a profound discussion. In all fairness we probably have devoted more effort to the strong or potentially strong features of dual collection system than we have to enumerating the difficulties. On balance we do feel that some form of the dual collection approach is essential to cope adequately with measurement errors in population surveys. Although concrete results are indeed sparse, the upsurge in the use of the method to obtain substantive demographic data, coupled with the increased interest in experimental testing of the method, would seem to indicate that the dual collection era is either here or is just around the corner. The dual collection system movement certainly appears to be growing in popularity very rapidly. Nevertheless, the state of the art probably should still be characterized as developing. In spite of our optimism about its potentials, we are very much aware of the practical difficulties in achieving the degree of technical and administrative control that is absolutely essential in operating a dual collection system. Other proponents of the dual collection are also well aware of these difficulties (Krotki, 1972, Marks et al., 1974). The basic question, for which there probably is a long list of possible answers, is: How good do the data have to be? It seems clear that dual collection systems are making significant contributions in helping to determine how good the data are. Discussion by Ivan P. Fellegi
The authors have to be congratulated for their bravery in agreeing to undertake a brief review of the "state of the art" relating to a topic as broad as dual collection systems. It is just recently that Marks, Seltzer, and Krotki have written a book of several hundred pages covering the same topic; and it is only recently that I, in commenting on that book, found myself writing well over 100 pages in discussion. The present discussion will be much shorter. I will start with some general comments on dual collection systems, prompted by the present paper but not exclusively related to it. At several places in the paper sweeping statements are made about the superiority of estimates derived from dual collection systems over those obtained from single source surveys. For example, the statement has been made, without qualification, that "while the variances may be greater for the PGE estimates the mean square error will be less". It is a fact [shown in Marks (1971) and Fellegi (1974)] that the variance of PGE estimates will be greater than the corresponding estimate derived from the survey alone. However, this result is derived without taking into account the fact that if the survey was conceived of as a "stand-alone" operation, rather than one part of a dual collection system, it would have available to it a significantly larger budget, it would be free of some significant design constraints (e.g. it could typically utilize considerably smaller clusters, leading to increased sampling efficiency), and thus a stand-alone survey at the same budget (rather than same sample size) as that of a corresponding PGE survey would probably have its sampling variance reduced by a factor of up to two or three. Of course, it would be subject to the well known biases of under-reporting. This discussion highlights some important points: i. One is not justified in assuming that under all circumstances the best strategy is to trade off the variance in favour of reducing the bias. This should be the result of a deliberate examination of the likely parameters of any given situation. If the budget is 69
Discussion
Ivan P. Fellegi
small, resulting in very small sample sizes, the improvement of the sampling variance may outweigh the potential reduction in bias. ii. Even where the overall strategy calls for the PGE approach, the estimates derived from the survey alone have a smaller sampling error. At the national level (or other large aggregation) the PGE estimate, because of its smaller bias, presumably has a smaller mean squared error (otherwise we should not have designed a dual collection system), so the PGE estimate would be preferred. As we pass to more detailed disaggregations, the variance will inevitably increase while the order of magnitude of the bias will remain more or less the same. There must be a cross-over point beyond which the variance begins to dominate the bias and therefore the survey estimate alone, or better still the average of survey and recorder estimates, is preferable to the PGE estimate. The argument is illustrated in figure 2.5 (assuming that the PGE estimate is unbiased): clearly for higher aggregates (to the right of the "cross-over point") the PGE estimate is to be preferred; for finer disaggregations the single source or average estimates are preferred. iii. The "best of both worlds" might prevail if we could use the single source estimate but reduce its bias through means other than the PGE estimate. It is not inconceivable that using a dual collection system, but using it differently from the usual PGE estimation, such an "ideal" could be approximated. What I have in mind is the use of the recording procedure as a direct quality control activity with feedback to the survey, rather than an independent operation. This would involve a careful analysis of the types of vital events uncovered by continuous recorders but missed by the survey interviewers, tightening the periodic survey operations accordingly, pointing out to survey interviewers the specific vital events they missed and analysing the reasons, etc. Clearly, using the continuous recording procedure in this way would introduce a positive correlation between continuous recording and the periodic survey data but probably (although generally not known) positive correlations exist anyway (except in
Figure 2.5 Interrelations between error and sample size for different types of survey 70
State of the art
Discussion
situations involving some specific incentive system, such as that reported in the discussion centering on figure 2.4 regarding the experience in the Philippines). In connection with point (iii) above, the material in table 2.1 is very instructive. Let us denote by Pr the overall completeness rate of continuous recording, by Pr|s the completeness rate of continuous recording of those vital events which are also covered (i.e., not missed) by the household survey. Then the expected value of the PGE estimate is (approximately) equal to (Pr/Pr|s)X
(2.2)
where x is the number of vital events. If Pr = Pr|s then registration and survey are independent. More often (and at least in the case of the Colombian experience reported in table 2.1) some positive correlation exists, in which case Pr
(2.3)
so that (Pr/P,|.)
Discussion
Ivan P. Fellegi
The problem of reporting vital events for migrants at the end of section 2.2.c is treated in something of a cookbook fashion. The proposed solution is a plausible one, but it clearly is not unbiased and its use should not be prescribed without at least some cautionary notes. In the discussion centering on table 2.2, the use of overlapping reference periods is questioned in the context of a dual collection system operation — presumably on the basis of the unstated premise that to the extent the survey completeness might be reduced without the use of overlapping reference periods, the PGE/ ERAD/ ECP estimate will correct for it anyway. This would be a dangerous assumption. For one thing, it is known that the variance of the estimate will be increased if the completeness of either procedure is reduced — even apart from biases. However, as the earlier discussion indicates, the bias may also well be increased if the completeness of either procedure is poor — unless one is willing to assume away the correlation between the two data collection procedures. The higher the completeness of the survey, the less dependent one is on the PGE/ ERAD/ ECP correction and all of its built-in assumptions and problems. The question raised by the authors would be more valid if they put it in the context of cost-benefit considerations; if overlapping reference periods are the source of significant costs, can those costs be more usefully expended in other directions? The discussion on matching follows Marks et al., (1974). I disagree with the proposed procedure. If the choice of matching characteristics is carried out by considering the gross error of individual characteristics, one will be unlikely to arrive at an optimal matching strategy: for example, one would never use "sex" as a matching variable because it has an obviously high gross matching error — yet everybody intuitively knows that sex is a very useful matching variable. One has to consider variables from two points of view: their contribution to confirming as "matched" those records which refer to identical events and their (negative) contribution to disproving the match status of those vital event reports which refer to different events. It can be shown that variables with a high discrimination (i.e. which create fine code classes) are ideal from the first point of view; variables with a low frequency of reporting errors (e.g. sex) are ideal from the second point of view. A set of matching variables can then be combined into a matching rule through the use of weights: agreement on a characteristic contributes a positive weight to the overall matching weight (test statistic), the size of the positive weight being dependent on the discrimination of the given variable (agreement on a rare characteristic obviously carries a higher positive weight); disagreement on a characteristic contributes a negative weight whose absolute value is larger if the characteristic has a low error reporting rate (disagreement on a very reliably reported characteristic has a high value in disproving the match status of records). Finally, alternative sets of matching characteristics are compared to one another as follows: one sets the decision point(i.e. the value above which the combined matching weight would indicate "match", below which it would indicate "no match") for each set of matching characteristics at a level for which the net matching error is zero (this can always be done!); one then compares the gross matching error for the alternative set of matching characteristics and chooses the one for which it is the smallest. For more detail the audience is referred to the paper by Fellegi and Sunter (1969). In conclusion, let me congratulate the authors once again for a thoughtprovoking paper. My specific criticisms are all attributable to a tendency of the paper to deal with complex issues in a somewhat simplified fashion. For this reason my review is, almost by definition, unfair: it can only deal with the paper as it is and not as it would be had the authors had less severe constraints of time and space. 72
State of the art
Endnotes
Endnotes to Chapter 2 1. For clarity of exposition the case k2 < ki is stipulated. Obviously, and depending on the cost functions of any actual empirical situation, sub-sampling might become worthwhile in the case where k2 > ki. [Editor's note.] 2. The National Philippine sample consists of 600,000 persons in 470 census enumeration districts for which detailed maps are prepared, special incentives are paid to the local civil registrar and to local officials to augment the report to the legal system, and periodic annual surveys are conducted by regional personnel of the Bureau of Census and Statistics. (Philippines, 1973.) 3. It is clear that extending the duty of the continuous recorder to obtaining the household composition is making it alike to the duties of the periodic survey interviewer. The consequent contribution to the loss of independence must be weighed against the alleged advantage of structuring the duty of the continuous recorder in this manner. [Editor's note.] 4. However, for a discussion of the disadvantages of multipurpose surveys, see Mauldin, 1966:652. [Editor's note.] 5. A clear distinction must be made between data collection by survey interviewers inquiring about events during a recall period without reference to a list of residents at some previous time, and data collection by follow-up surveys in which a list of persons present at some previous time is checked. See Scott's discussion of this in chapter 11. 6. There are no PGE/ ERAD/ ECP objections to field verification during the experimental matching for purposes of establishing matching rules. [Editor's note.] 7. There are strong PGE/ ERAD/ ECP objections to field verification during production matching. As argued repeatedly in chapter 1 and elsewhere, such verification advances the possibility of competitiveness between procedures. [Editor's note.]
73
Chapter 3 Dual Estimation in Demography Employing Time Series and Cross Section Data P. Krishnan 3.1 Introduction One of the primary aims of statistical estimation theory is to provide the best estimator (to be defined in some sense) of a population parameter. Toward this end, estimators of the same parameter by different procedures, or from different sources, are prepared and combined to obtain a more (or most) efficient one. In demographic research this strategy has found an application in what is called the PGE/ ERAD/ ECP technique, employed in some countries to estimate fertility, mortality, and natural increase in the absence of reliable data, and in some others to estimate the completeness of the civil registration system. Examples were cited in chapters 1 and 2. The different techniques suggested to date with regard to dual estimation in demography do not consider demographic theories, or make use of other available prior information for improving the estimators. In this chapter an attempt is made to suggest some strategies in that direction by integrating demographic theories and statistical theory of estimation with a view to deriving best estimators in the minimum variance sense. 3.2 Pooling cross section and time series data In countries that have a good civil registration system, the researcher can lay hands on a time series of the data of interest. The sample survey for the particular year under consideration provides a cross section. In the regular PGE/ ERAD/ ECP technique one does not make use of the past data (prior information). Matching is done and the sources are combined to secure the PGE/ ERAD/ ECP estimator in accordance with chapter 1. Suppose one wishes to make the best use of the prior information supplied by the time series. However poor the civil registration system, it has some information to offer and that has to be exploited for improving the estimator. Econometricians have developed several procedures for pooling cross section and time series data. There is voluminous literature in this area. Some of the recent developments can be found in Maddala (1971) and Swamy (1971). Most of the work done by econometricians is not of immediate relevance to the problem at hand despite its similarity to the pooling issue in econometrics. The attempt here is to integrate 74
Time series and cross section data
3.2
demographic and statistical estimation theories for the development of improved estimation procedures. The different strategies are discussed in section 3.4. 3.3 Best estimator If the best estimator of a parameter is defined in the minimum variance sense, and if TI and Ta are two independent unbiased estimators of the parameter, then Let the variances be denoted by One can secure a more efficient estimator by combining TI and T2. If a linear combination is considered, it is shown (see Rao, 1970: 137) that is more efficient than TI and Ta. It is also clear that This logic can be exploited to some extent with regard to the problem at hand. 3.4 Suggested estimation strategies Estimator 1: Combining a time series regression estimator with the PGE estimator. For the year under consideration, one can find the PGE estimator, say Ni, for the events under consideration. The variance of the PGE estimator is given by > where p = the probability of detecting the ith source (i = 1, 2), NI = the actual number of events in the universe. If ft is the rate of incidence of the events, then where P is the base (risk) population of choice. P may be estimated or known from a census or other sources. We shall assume that P is not estimated. An estimate of /3, say /3, is
When we regress the number (or incidence) of the events at time t on the number (or incidence) at time t-1, t-2, t~3, . . . linearly, we obtain an autoregressive predictor. Working with the Canadian data on births and crude birth rates, it was seen that a second order auto-regression equation is a good predictor (see appendix). Furthermore, the Canadian data revealed that the correlation pattern existing between the different auxiliary variables is approximated by the geometric pattern ptt+ k = pk (k = 1, 2, . . . ). We hope that this pattern would hold for other countries as well. Under this pattern, let Na be the predicted number of events for time t (the year under consideration). Then
3.4
P- Krishnan
where a2 is the variance of the error term, C the row vector of the coefficients of the predictor, ft the column vector of regression coefficients, and X the matrix of observations (Johnston, 1972: 126, 153). If the geometric pattern does not hold true, where R2 is the square of the multiple correlation coefficient of the dependent variable on the independent variables. The advantage one can reap with the geometric pattern is that only one independent variable js needed for prediction. Equation (3.11), in conjunction with the PGE estimate NI yields the best estimator N of the number of events under consideration employing time series and cross section, viz V(N) is given by equation (3.4). Since ai 2 , a-i are unknown, the sample estimates can be substituted in (3.14). Estimator 2: Combining time series and cross section under rigorous assumptions with regard to error term. Estimator 1 is rather simplistic with regard to the composition of the error term. The "matching" part need not be of concern to us at this stage. Depending on whether an estimate of the extent of under- (over) registration is required or not, matching need or need not be done. We shall consider only the time series and the cross section data. Let the time series and cross section models be
where Y t ( 1 ) = the number of events at time t in the macro unit; Pt = population at time t in the macro unit; Yi (2) = the number of events in the ith micro unit; Pi = population of the ith micro unit; and /u t , Vj = error (disturbance) terms. ft is assumed to remain the same throughout this period. Let us further assume that the disturbance terms have zero mean and finite variance, and are serially uncorrelated.1 Then we have: estimate of ft from cross section data only
estimate of ft from time series only Case I: Let V(M.) = a,2 = V(vi) = a22 = a2 (say) Under this condition, we treat cross section and time series as one set yielding the pooled estimator
Now let us compare the variances ot'/J, ft, ft.
76
Time series and cross section data
3.4
It is clear from (3.20), (3.21), and (3.22) that (3)—the pooled estimator—is the most efficient of the three. Case II: o-i = k 2 ai 2 (k known) The pooled estimate (3) in this case is:
For k = 1, (3.23) reduced to (3.22). Under large sample assumptions, the variance of (3.23) follows from Cramer's result on the asymptotic normality of maximum likelihood estimators (MLE), (see Rao, 1970: 158). Case III: <5,Va 2 2 The likelihood function of the parameters is
The MLEs of o2, oi2 and ft are obtainable on solving equations (3.25), (3.26), and (3.27) simultaneously.
An iterative procedure yields the MLEs. Usually the cross section sample is large and assuming that the time series also is large, the large sample estimates of oi2 and a22 can be employed in (3.27) to arrive at a large sample estimate of ft. Cramer's theorem yields the variance of the MLE of ft. Estimator 3: An integrated estimator from demographic and statistical theories. Demographers have tried to develop a theory of fertility based on the influences of social, economic, psychological, and biomedical factors. There is no need to enumerate here the various contributions to the sociology of fertility. At family (individual) level, duration of marriage, knowledge and effective use of contraception, infant mortality, desired family size, present parity, family income, schooling, etc. are all known to be predictors of fertility. At societal level, it has been noted that literacy, age at marriage, infant mortality rate, degree of urbanization, per cent population in non-agricultural activities, extent of female labour force participation, income per capita etc. are all associated with fertility. A similar set of socio-economic correlates are available for mortality prediction also. Since the demographic journals are replete with empirical regression applications, there is no need to refer to specific studies here. The PGE/ ERAD/ ECP technique has not exploited these auxiliary variables for improving the estimates of fertility and/ or mortality. At micro (family/ individual) level, data on the relevant auxiliary variables can be collected at the time of the survey without adding much to the cost. 77
P. Krishnan At macro (societal) level, the data are available from censuses, or from other reliable official sources. Hence, the methodology discussed in the following paragraph does not present any difficulty as such in the generalization of the PGE/ ERAD/ ECP technique. Let be the time series model, where Z t (1) = rate of events under consideration Xit = ith auxiliary variable (i = 1, 2, . . . k) 7, = ith regression coefficient (i = 1, 2, . . . k) Ut = disturbance term with E(u,) = 0, E(u.u.+s) The auxiliary variables may be lagged over time to suit the demographic theories. The least squares predictor of Z t (1) given X;t (i = 1, 2, . . . k) is Variance of Z t (1) is given by the standard formula available in Johnston (1972: 153). Suitable modifications are necessary if the problems of multicollinearity, heteroscedasticity etc. are present in the regression set up. Let the cross section model be where
The cross section predictor is with V(Z(2)) = o22 (say). (3.29) and (3.31) can be combined now to arrive at the best estimator of the rate(s) under consideration. Large sample estimates of ai2 and oi2 may be applied in the estimation strategy. No illustrations are presented here to show how this pooling technique works. The main aim of this chapter has been to indicate that, if suitable time series and cross section data are available, the pooling technique could be employed to improve the PGE estimator.
78
Time series and cross section data
Appendix
An appendix to Chapter 3
Correlation pattern in Canadian data on births and CBRs Canadian data on crude birth rates (CBR) and total number of births from 1948 through 1970 yielded the following results: CBR Matrix of Correlation Coefficients time t t-1 t-2 t 1 .985 .953 t-1 1.000 .977 t-2 1.000
Mean Maximum Minimum Standard deviation
Total No. of Births Matrix of Correlation Coefficients t t-1 t-2 1 .948 .827 1.000 .942 1.000
Crude Birth Rates t 24.8 28.5 17.4 4.02
t-1 25.3 28.9 17.6 3.77
t-2 25.7 28.9 17.6 3.39
Statistical testing reveals that the geometric pattern holds here. Discussion by Eli S. Marks
Section 3.1 of this chapter notes that: "The different techniques suggested to date with regard to dual estimation in demography do not consider demographic theories, or make use of other available prior information for improving the estimators." This statement is somewhat misleading since the role of PGE/ ERAD/ ECP estimation is to try to reduce the biases in the data, and a number of the references in the PGE/ ERAD/ ECP field have emphasized the desirability of combining demographic analysis with dual system estimates. However, the point that one needs to use other available information to improve dual system estimates is very important and cannot be overstressed. We badly need some concrete examples of how PGE/ ERAD/ ECP can be used to improve the results of some of the recognized techniques of demographic analysis, and we also need specific suggestions of new ways of using other available data to improve PGE/ERAD/ECP results.2 Unfortunately the chapter misses the major problem. That is, in "integrating demographic theories and statistical theory of estimation" the goal is to minimize the mean square error (rather than the variance) of our estimates. Thus Krishnan presents quite adequately the standard statistical theory for combining two independent unbiased estimators to get a third estimator having a lower variance than either of the originals; but the real problem is to combine two methods which produce (different) biased estimators to get results having lower mean square errors than the results of other methods used alone. It is true, however, that, because PGE/ ERAD/ ECP methods usually involve the use of small samples, some of the dual system estimates will have high variances and one must look for means to reduce those variances. But we must guard against reducing the variance by increasing the bias unduly. For example, the use of time series or of relationships derived from previously observed data can be very useful if (i) 79
Discussion
Eli S. Marks
PGE/ ERAD/ ECP techniques or some other methods have been used to reduce the bias in both the current and past data to manageable levels and (ii) the relationships used in the time series model are stable over time. Thus, for example, one can use estimates of fertility and mortality rates by age from previous years to reduce the variance of our estimates of current mortality and fertility rates and of the growth rate and the current age-sex structure. But we will have to correct past mortality and fertility rates for under-reporting biases (by using dual system estimates or some other means) just as we need to correct the current mortality and fertility rates for these biases. Furthermore, we cannot use a model which assumes stable age-specific rates unless there is reasonable evidence that the rates have been stable over time. And, even with such evidence, it would be rather foolhardy to use fertility rates which project past trends, if the objective of our estimates is to measure the effectiveness for reducing fertility of a national family planning program. In general, the use of past relationships will be best accomplished by using a specific model rather than the generalized linear regressions presented by Krishnan. For example, due to high variance, the PGE/ ERAD/ ECP estimate of the death rate in Thailand in 1964-67 was higher for females 1-4 than for males-a most unlikely situation. The civil registration for this period shows a masculinity ratio more in line with demographic expectations. A model which assumes that the ratio of deaths of males 1-4 to females 1-4 is that given by the civil registration will probably give lower mean square errors than one based on the regression of the observed death rates on age and sex. However, we will still need to use PGE to correct the general level of fertility. One minor point is the assumption that "the variance of the PGE/ ERAD/ ECP estimator" is given by Krishnan's equation (3.5): V(N,) = ( N , q i q 2 ) / ( p i p 2 ) This equation only applies if (i) one has reports for the whole population in both sources (or uses only simple random samples or some other type of fixed size sample with zero intraclass correlations); and (ii) reporting errors are truly independent in both systems. In a well designed and well executed PGE/ERAD/ECP system, the second condition will obtain approximately but the first condition (having data for the whole population in both sources or using only simple random samples) is rather rare.3 While the limitations on equation (3.5) are not a major point, it is mentioned because the misunderstanding is rather common and people assume that equation (3.5) is "the" variance of a dual system estimate. More general variance formulas are given in chapter 7 of Marks, Seltzer and Krotki (1974). Endnotes to Chapter 3 1. For a detailed description of serial or auto-correlated disturbance terms, see Johnson, 1970:243-66. 2. We are suggesting one such approach in our chapter 9. [Editor's note.] 3. In the 1940 and 1950 U.S. birth registration tests, both sources (civil registration and census) did try to obtain reports for all members of the population.
80
Chapter 4 Dual System Estimators Based on Multiplicity Surveys Monroe G. Sirken
4.1 Introduction We are concerned with the problem of improving the reliability of dual system estimators of vital statistics derived from single retrospective sample surveys. Our objective is to investigate the effect of alternative counting rules in single retrospective surveys on the sampling errors of dual system estimators of vital statistics, especially mortality statistics. In addition to the sample design itself, there are several other factors that affect the accuracy of the survey statistics over which the statistician exercises control in designing single retrospective surveys. These design factors include, for example, the length of the reference period and the kind of respondent rule adopted in the survey. The former defines the calendar period for which vital events are reported retrospectively. The latter designates who may serve as eligible respondents in the survey. Another design factor which had received no attention until recently (Sirken, 1970, 1972a, 1972b) is the counting rule. In single retrospective surveys, this design factor designates the housing units where the vital events that occurred during the reference period are eligible to be enumerated. Clearly, the distribution of the vital events among the housing units is a function of the counting rule. By changing the counting rule, the statistician modifies this distribution and thereby alters also the sampling distribution of the survey statistics. The counting rule strategy is to select the rule that minimizes the variance of the sampling distribution. 4.2 Counting rules We differentiate between conventional and multiplicity rules in single retrospective surveys. The conventional counting rule distributes the vital events that occurred during the reference period among the housing units such that every event is uniquely linked to and hence eligible to be enumerated at only one housing unit. In household surveys, conventional counting rules are often referred to as residence rules. For instance, the de jure residence rule links vital events that occurred during the reference period to the usual places of residence of the individuals who experienced the event. For most vital events the usual place of residence would be the housing unit where the person who experienced the event was residing when the 81
4.2
Monroe G. Sirken
survey was conducted. In the event of death, however, the usual place of residence would be the housing unit where the person resided at death. A multiplicity rule distributes vital events among the housing units such that every event is linked to one or more housing units where it is eligible to be enumerated. Multiplicity rules have been proposed that would link persons who experienced vital events to the residences of their relatives. For instance, one of these rules links adults who died during the reference period to the residences of their surviving spouse, siblings, and children. Another rule links children who died during the reference period to the residences of their mother, grandparents, aunts, and uncles. 4.3 Multiplicity estimators Several unbiased estimators have been proposed (Birnbaum and Sirken, 1965) for estimates based on surveys using multiplicity rules. We will use one of these, the socalled multiplicity estimator, to estimate N, the number of persons that died during the reference period specified by the survey. Denote the L housing units in the population by Hi, . . . , Hi, . . . , HL, and the N deaths that occurred to persons who resided in the L housing units during the reference period by Ii, . . . , Ia, • • • , IN. Let the indicator variable, r 6 a ,i, specify the links between the L housing units and the N deaths based on counting rule r. Thus,
A multiplicity estimator of N is
where = the multiplicity of I<*(a = 1, . . . , N), that is, the number of housing units linked to Ia by the rule r. This estimator assumes that all deaths are reported by the housing units which are eligible to report them, and that deaths are not reported by housing units ineligible to report them. It is an unbiased estimator if the counting rule links every death to at least one housing unit, that is, if sa> 1 (a = 1, . . • , N). The unbiased sample estimator of Nr, is
where ii, . . . , ij, . . . , i^ represent the indices of the Chousing units selected in the sample. It will be noted that the estimator Nr requires the multiplicity of every death enumerated in the sample housing units. We are proposing (Sirken, 1970) to obtain the multiplicities from the housing units where deaths are enumerated in the survey. Suppose the counting rule stated that decedents are linked to the housing units of their surviving spouse, children, and siblings. A death would be enumerated at a sample housing unit whenever a surviving spouse, child, or sibling of the decedent lived in the unit. The multiplicity of an enumerated death would be ascertained by asking the housing units which reported a death also to report the number of other housing units which contained either a surviving spouse, child, or 82
Multiplicity surveys
4.3 A
sibling of the decedent. If Nr were based on a conventional rule, however, it would be unnecessary for the survey to obtain the multiplicities of the enumerated deaths since by definition sa = 1 (a = 1, . . . , N). Thus, the estimator based on a multiplicity rule requires the collection of supplementary information which is not needed by a conventional rule. 4.4 Dual system multiplicity estimators Typically, deaths are grossly underenumerated in single retrospective surveys (Seltzer, 1969). This appears to occur because deaths are not enumerated at the housing units to which they are linked by the counting rule (Sirken and Royston, 1970). Therefore, the estimator Nr understates N. Hence we consider an alternative estimator, rj,, which we refer to as a dual collection estimator of N based on rule r. The estimator rjr involves matching the deaths enumerated in the survey with the file of deaths recorded by the registration system. We will assume that deaths are incompletely registered as well as incompletely enumerated. Otherwise, there would be no need to conduct the household survey. Before defining rjr we introduce some notations. Let the indicator variable r/u 0,1 identify the deaths that are enumerated in a survey based on rule r, where 1 i f I a ( a + 1 , . . . , N ) is linked to as well as enumerated at rua,i
Hi(i + 1, . . . , L) 0 otherwise
The biased estimator of N based on the deaths actually enumerated in the survey using counting rule r, is
where
denotes the weighted number of deaths enumerated at
Hi (i = 1, . . . , L). We assume that the multiplicities are not subject to measurement errors so that A one way match of deaths enumerated in the survey is made against the complete file of registered deaths. Let the indicator variable ,i>a,i specify the enumerated deaths that are matched with registered deaths, where 1 if Ia(a +1, . . . , N) is linked to as well as enumerated at Hi(i + 1, . . . , L) and matched with a registered death rua,i 0 otherwise The number of registered deaths enumerated in the survey based on counting rule r, is
where
denotes the weighted number of deaths enumerated
at Hi (i = 1, . . . , L) that were matched with registered deaths. The dual system multiplicity estimator of N is where pr = N r / X r and X is the number of registered deaths. The sample estimate of rjr is 83
4.4
Monroe G. Sirken
The sample estimators of Xr and Nr based on a simple random sample of £ housing units are
where i i , . . . , i j , . . ., i& represent the indices of the sample housing units. Despite the fact that the survey underenumerates deaths and the registration system fails to record some deaths, r)r is an unbiased estimate of N if Xr/ X = Nr/ N. 4.5 Variance of sample estimators Assuming a simple random sample of £ housing units selected without replacement, the sampling variance of ij, is The derivation of V(»7r), presented in appendix A to this chapter, assumes no more than one death is linked to a housing unit. The parameter, T =X/ L is the proportion of housing units that was a de jure residence of a registered death. Two parameters depend on rule r.
, = Xr/ X is the proportion of registered deaths that are enumerated in the survey, and 0r = (1 - l/p r ) r Ex + (l/p r ) r EN-X where and
respectively are based on the multiplicities of registered and unregistered deaths enumerated in the survey. Since 0 < r ^ 1 and 0 < 6 ^ 1, it follows that Hr = 6,1$, > 0 and thus Hr serves well as a measure of the sampling efficiency of counting rule r. The efficiency of counting rule r is inversely proportional to flr. If the survey is based on a conventional counting rule, flc = 1/<£ C since cEx = C£N-X = 1, and it follows that the sampling variance of i If the survey is based on a multiplicity rule, the estimator is and from (4.7) and (4.8) we have Assuming pc = pm, it follows that If 0C = 0m, formula (4.10) implies that 4.6 Relative precision of different rules The effect of counting rule r on the variance of the dual system sample estimator of N, is shown by formula (4.7) to depend on the parameter Hr = 0r/ r. Survey experiments are needed to estimate 0r and t for different counting rules. In this section, we describe the design of such a survey experiment and present findings that compare the sampling precision of selected counting rules. The findings are considered provisional and they are presented for illustrative purposes only. A small experiment (Sirken and Royston, 1970) investigated the counting 84
Multiplicity surveys
4.6
Figure 4.1 Counting rules tested in a survey experiment for enumerating deaths, Los Angeles, July - October, 1969 Rule
Statement of rule
1 2
Decedents are linked to their former housing Decedents are linked to their former housing surviving children Decedents are linked to their former housing surviving siblings Decedents are linked to their former housing surviving children and siblings
3 4
units units and to units of their units and to units of their units and to units of their
rules listed in figure 4.1. Rule 1 is a conventional rule and rules 2, 3, and 4 are multiplicity rules. Each of the multiplicity rules links the decedents to their former housing units and thereby guarantees that every death is linked to at least one housing unit. Should it be established empirically that virtually all decedents have at least one surviving close relative, it would be sufficient to link deaths to housing units of these relatives. The experiment was based on a small sample involving 142 noninstitutionalized white adults who died in Los Angeles during the four month period, July/October 1969. The sample deaths were selected from the files of the Los Angeles County Department of Health. The survey was conducted during the three month period January/March 1970. First, interviews were conducted at the sample decedent's former housing units as identified from the address listed as the usual place of residence on the death record. Subsequently, interviews were conducted at housing units of the surviving relatives of the sample deaths, who were living in Los Angeles County. These relatives were identified in interviews completed at the decedents' former housing units. Household respondents were asked the same question in the first and second stage interviews. Screening questions identified the deaths that occurred during the preceding 12 month period to persons who either formerly lived in the housing unit or to persons who had a surviving child or sibling living in the housing unit. Whenever a death was screened, its multiplicity was ascertained by asking a series of questions about the location of the decedent's former residence and the locations of the residences of his surviving children and siblings. The estimates Hr = 0 r /0 r derived from the survey experiment are shown in table 4.1. The estimates of Hr range from about .54 for rule 4 to about 1.6 Table 4.1 Estimates of 6,
1 2 3 4
1.00 .61 .59 .38
.63 .70 .63 .70
1.59 .87 .94 .54 85
Monroe G. Sirken
4.6
for conventional rule. In other words, for fixed sample size the variance of the estimated number of deaths based on either rule 2 or 3 would be about three-fifths as large as the variance based on the conventional rule. Limitations of the survey experimental design forced us to make a number of assumptions in order to estimate r and 6,. On the basis of these assumptions, which are discussed in Appendix B to this chapter, the estimate of 6, is where x is the number of registered deaths in the experiment that were enumerated at the decedents' former residence and rsa is the multiplicity of LXa = 1, . . . , x) based on rule r as reported by the household residing at L's former residence. The estimate of r is where qc = proportion of sample deaths that were not reported at their former residences = proportion of sample deaths that were not reported by surviving relatives. The estimates of qc and rqm based on the survey experiment are presented in table 4.2. Thus, qc = .37 and rqm is equal to .38, .20, and .26 respectively for housing units of surviving siblings, surviving children, and surviving children or siblings. It is apparent from table 4.2, however, that qc and rqm would have been smaller had more resources been available to reduce the noninterview rate. Had we estimated qc and rqm solely on the basis of the missed deaths in completed interviews, we would have qc = .23 and rqm = .29, .08, and .15 respectively for housing units of surviving siblings, surviving children, and surviving children or siblings. Table 4.2 Proportion of deaths that were missed by type of housing unit in the survey experiment, Los Angeles, July - October 1969 Units formerly occupied by key deaths Sample size Deaths enumerated Deaths missed Interview completed Interview not completed
142
Other units occupied by relatives Total Siblings Children
46 .63 .37 .19 .18
16 .74 .26 .13 .13
30 .62 .38 .25 .13
.80 .20 .07 .13
4.7 Conclusion We have investigated the effect of different counting rules in single retrospective surveys on the sampling errors of dual system estimators of vital statistics. First we derived an expression for the variance of the survey estimates which contains a parameter that serves as an index of the sampling efficiency of the counting rule. Then, we presented estimates of this index for several different counting rules for linking persons who died during the reference period to housing units where they are
86
Multiplicity surveys
4.7
eligible to be enumerated in the survey. These estimates, though provisional, being based on the findings of a small survey experiment, indicate that the counting rule would have a decided effect on the sampling errors of dual system estimators of the number of deaths in the United States. Further experiments are needed to substantiate these findings. Since counting rules are culture bound, survey experiments to test alternative counting rule strategies in single retrospective surveys are needed particularly in countries lacking viable registration systems. In limiting the scope of this report to the effect of counting rules on the sampling errors of dual system estimators, we did not consider the effect of counting rules on the bias of these estimators. It has been shown (Sirken, 1973b), that the counting rule has a substantial effect on the number of deaths that are missed in the survey, and there can be little doubt that the bias of the dual system estimators would vary considerably depending on the counting rule adopted in the survey, although this remains to be confirmed experimentally.
Appendix A to chapter 4 Variance of DSM sample estimators In a prior paper (Sirken, 1973b), we derived the sampling variance of Pr = X r /N r , which is the dual system multiplicity estimator of vital registration completeness. The derivation assumed that counting rule r links no more than one vital event to a housing unit. We apply that prior result here to derive the sampling variance of rj r , the dual system multiplicity estimator of the number of vital events. Since it follows that Assuming a simple random sample of £ housing units is selected with replacement, we showed (Sirken, 1973b) that where 0r = (1 - P r ) r E x + (P r )rE N =x. Substituting (4.14) into (4.13), noting that Pr = l/p r , we have where Appendix B to chapter 4 Simplification of Hr The measure of samoline efficiency of rule r is where
and We assume that rEx = r E N -x so that (4.18) reduces to (4.19) Additional assumptions were made in order to facilitate estimating 6T and r on the 87
Appendix B
Monroe G. Sirken
basis of the restricted information collected in the survey experiment. These assumptions are described below. We assume that counting rule r links every decedent to his former de jure residence, and to the residences of specified surviving relatives. Let 1 if a registered death is enumerated at its de jure residence, reai 0 otherwise and 1 if a registered death is enumerated at a linked housing unit other than its de jure residence, r f ai 0 otherwise. We assume also that r c a ,i and r fo,i are binomial random variables with expected values: E(rca,i) = (1 - qc) = probability the registered death is enumerated at its de jure residence and E(rf a ,i) = (1 ~ r q m ) = probability the registered death is enumerated at a relative's residence, other than the decendent's de jure residence. With these assumptions, 6, = 8', and r = 'r become random variables. The expected value of 6', is
{ {
where and
Since every Ia(a = 1, . . . , N) is linked to its de jure residence by rule r, each Itt is also linked to exactly ( r s a - 1) other residences of relatives. Thus, we have
and
Substituting (4.21) and (4.22) in (4.20) we have V
Y
since R is generally a negligible quantity. If sa = s, R = 0. From (4.21) and (4.23), it follows that the expected value of 0'r is
Now if we assume that a random sample of x registered deaths were enumerated in the experiment, we have 88
Multiplicity surveys
Discussion
Discussion by Ivan P. Fellegi The development of multiplicity counting rules for rare events can justifiably be considered as one of the few genuinely original developments in household survey methodology during the last 10 years. I am very glad indeed that Sirken has attempted to take the step of marrying the multiplicity counting rule technique with dual system estimation. The first technique is primarily a variance reduction device (most valuable for rare events which, therefore, are subject to high variances), the second is designed to reduce biases due to non-reporting. If the combination of techniques can successfully be made to work, a major step forward in measuring population change may well have been accomplished. Like all ingenious ideas, the concept of multiplicity counting rules is very simple. It can be illustrated in the context of an oversimplified example as follows. Suppose the proportion of deaths in a population is to be estimated from a simple random sample of n persons (living or dead). The usual estimate, p, is unbiased and is the sample mean of the counting process whereby we associate a 1 with every person who died in the reference period and a 0 with all other persons. The variance of p is approximately given by where S2 = p(l-p) If, now, with every person who died in the reference period we can associate another (k - 1) persons uniquely (e.g. their surviving siblings or children), then we can change the counting rule to associate 1 / k with every person who died in the reference period or who is associated with one who died. We can expect in the sample nkp observations of 1/k and n(l - kp) observation of O's (instead of np observations of 1 and the remainder O's). The number k is the "multiplicity". The estimate will clearly be unbiased still, but the dispersion of the observations is significantly reduced resulting in a reduced variance: where Sk2 = p ( l / k - p ) so long as kp ^ 1 If p is small, the variance will be reduced (in comparison with the traditional counting rule) by a factor of k. This oversimplified presentation of the principle underlying the concept of multiplicity counting might be useful because it illustrates not only how the rule reduces the variance of an estimate but also because it indicates another phenomenon at work: from a pure variance point of view it would clearly be advantageous to increase k without limit. A problem arises, however, because a high multiplicity can only be achieved by defining the rule of association in terms of increasingly tenuous relationships (surviving children, surviving children or siblings, surviving first cousins, second cousins, etc.). Since the counting rule must be one that can be understood by respondents, it must be defined in terms of relationships meaningful to the respondent. Thus the multiplicity (as in the examples above) is not a predetermined constant but varies with the particular death involved, and thus it must be reported in the survey by the respondents. Clearly, the more tenuous the rule of association, the more likely it is that respondents will under-report the number of persons they are defined by the counting rule to be in association with. Since this number enters the denominator of the estimate Nr (based on a single survey), an upward bias may 89
Discussion
Ivan P. Fellegi
well result. The bias is likely to increase as the multiplicity increases. So the variance reduction may well be accompanied by an increase in the bias of the estimates. At some (a priori unknown) level of multiplicity an optimal balance may be reached. The matter is further complicated by the fact that the likely upward bias due to multiplicity counting rules may at least partially be offset by the traditional underreporting of rare events. Still further complication is introduced by the fact that the estimated multiplicity is subject not only to bias, but also to response variance. The variance of the resulting multiplicity estimates is likely to be higher, therefore, than one would guess on the basis of the chapter's assumed error-free reporting of the estimated multiplicity. Worse yet, if the reported multiplicity is subject to response variance, given the generally small magnitude of the estimated multiplicities, the resulting multiplicity estimates may be subject to a not insignificant ratio estimate bias. Moving to dual system multiplicity estimation, the author considers a case where the first collection procedure is a census (100 percent registration) and the second a survey. In order to assess the proposed estimator, a variety of other biases have to be considered. Since the probable upward bias of the single source multiplicity estimation will, in the dual system estimator, appear both in the numerator and in the denominator, this particular source of bias may partially cancel out. There may well be additional biases, however. In particular, an essential aspect of all dual system estimators is related to matching between two data sources. Clearly, the quality of matching is fundamentally dependent on the identification of the events which are to be matched (in the case of deaths, information such as name, last address, age, etc. of the deceased). The more tenuous the "association" between the person reporting the death and the deceased, the more error-prone the matching information will be. This will probably result, other things being equal, in an understatement of the number of matched cases and hence an overstatement of the dual system multiplicity estimator. At any rate, as the author himself indicates, the proportion of missed events may well be affected by the counting rule (it may be reduced to the point where dual system estimators become unnecessary); also the correlation between the two sources may be affected in an unpredictable fashion. Ascending even further in this presentation of escalating difficulties, the author does not treat the case where both registration and survey are based on samples—by far the most frequent situation in developing countries. No doubt, the consideration of the different trade-offs between biases and variances would be even more complex in this case. In particular, if the continuous recording is also based on a sample, it has to collect information using the same counting rule as that of the survey (otherwise matching would not be possible—the proposed post facto reconstruction of the multiplicity estimator based on the events matched between the survey using a multiplicity rule and the registration using traditional counting is predicated on the fact that the register is a complete census, rather than the result of a sample survey). The value of data collection for the registration would, therefore, be substantially altered since a more than usual amount of information would have to be collected. This, in turn, would quite possibly affect the correlation between registration and survey. Of course, there would be other possibilities also worth exploring: such as collecting the registration information using the traditional counting rule, collecting the survey information in such a way as to facilitate the compilation of a survey estimate using both the traditional and the multiplicity counting rule, and compiling a "hybrid" estimate in which Nr is the multiplicity estimate from the survey, X is the "traditional" registration estimate, M is the
90
Multiplicity surveys
Discussion
estimate based on the matched events (using only those deaths in the survey which correspond to the "traditional" counting rule): As can be seen, the chapter presents a new technique with exciting possibilities. The author himself is well aware of the fact that much further exploration of the behaviour of the mean square error of the proposed estimates is required before the technique can be said to be ready for large scale applications. The present discussion attempts to focus future experimentation on some specific sources of bias. The intention is not to indicate a long list of problems with a view to discouraging potential users of the technique, but rather to encourage the profession to accumulate the necessary empirical evidence. The technique is of major potential pay-off: by significantly reducing the sampling variance of estimates of births and deaths, it may permit a major reduction of sample sizes and hence costs. However, a significant initial investment in the form of experimentation is required before we can expect to begin to reap the dividends.
91
Chapter 5 The Collection of Demographic Data in Francophone Africa and Liberia Using the PGE/ ERAD/ ECP System Francois Pradel de Lamaze 5.1 Dual collection The insufficient findings in the various demographic surveys, as well as underenumeration—obvious but difficult to quantify—in the registration of vital statistics of third world countries, led in Africa (among other attempts) to the reciprocal verification between two a priori independent operations (the PGE/ ERAD/ ECP technique). In the cases known to us, the use of this method has a pedestrian goal: the precise determination of the number of births and deaths. These in turn permit by reference to the population, obtained at the same time or otherwise, the calculation of the rate of natural increase. To the best of our knowledge the technique has not been used in Africa for other reasons: rectification of the age pyramid, measurement of the completeness of a census, etc.' No doubt the difficulties encountered even when a relatively simple objective is chosen would increase considerably with more difficult objectives. Most experiments with dual collection have been done on samples, and the method has sometimes been condemned because of sampling problems. The method may be put to work on a total area, even a very large one. Nevertheless, because the cost is considerable, it is expedient — at least in African circumstances — to work with samples. The general problems presented by a sample design are the same as for an ordinary survey, but with the added difficulty that the two samples should coincide exactly. The "slips" that are almost always produced in a survey (poor boundaries, non-responses, absences, etc.) then have an even greater significance. Finally, note that "dual collection" as usually presented in Africa is a matching of two types of operations: (i) a continuous recording of vital events analogous in principle to the civil registration, but distinct from the corresponding administrative operation; (ii) a survey of vital events in households, in retrospect, taken with the same recall as that of the continuous recording. But this schema is not immutable and diverse variations can be used: comparison of the civil registration and a survey, comparison of two retrospective surveys, comparison of one retrospective survey and a multiround survey, etc. 5.2 Some observations The application of the PGE/ ERAD/ ECP technique presupposes the total independence of the two operations. Two aspects of this independence should be considered. 92
Francophone Africa and Liberia
5.2
The first is operational independence: it is essential that the field workers participating in the household survey completely ignore those engaged in continuous recording. Good organization can resolve this problem. The second is causal independence: this problem appears to be more difficult to resolve. The PGE/ ERAD/ ECP formula used supposes, in effect, that within each procedure of observation, each event has an equal probability of being reported, and that this probability is not affected by the existence of the other procedure. In practice, there will not be an equal probability of collection for all events within a procedure. Thus, for example, births in young families in which the heads of the households are educated have presumably a greater chance of being correctly reported than those happening in older families in which the heads are illiterate. Similarly, deaths of isolated individuals are more easily omitted than those occurring in normal families. The problem can be resolved by stratification, whether a priori — stating the principle causes of omissions elsewhere—or a posteriori—stating omissions as a function of the characteristics of the events. The final estimation is the sum of the strata estimates. Note that, in so far as it is otherwise possible to estimate adequately the corresponding populations for each strata, this eventally permits the calculation of differential rates, though with variable precision. Even more serious is the fact that a connection will be established, in a quasiautomatic way, between the two procedures during the surveys. The probability of a household already surveyed by one of the procedures declaring an event in the other procedure, depends strongly on its previous response. Because of the fact that an interviewer has already arrived, asked judicious questions, and has been able to revive the memories of the household, the recorder following him some days later will certainly obtain a similar response. If, as is often the case, one of the procedures carries out frequent visits to the households, at a certain time the two procedures will give nearly identical results, but be hampered by the same systematic errors and omissions. These problems for which we see no suitable solution, handicap all operations of too long a duration, and leave suspect operations which are too short. Then there are the problems of matching. When an event is reported in both procedures, the difficulty arises of ascertaining, with sufficient certainty, whether the two documents refer to the same single event. In African reality, and this problem is even more difficult when the two sources are really independent, the characteristics given are rarely coherent. The given and family names are often unstable, the ages or birth dates are poorly determined and a fortiori even more so, more remote characteristics such as the age of the parents or the occupation. In chapter 1 it has been shown that very strict matching rules tend to underestimate the quantity M in figure 1.1 (events observed in both procedures) and consequently, to overestimate the quantities Ui (events observed only by the first procedure), U2 (events observed only by the second procedure), and Z (events not observed in either procedure). If, on the other hand, the matching rules are too lax, M will be overestimated and the other quantities will be underestimated. In conditions of African uncertainty and changeability the striking of matching rules close to a zero net matching error might be particularly difficult and frustrating. The PGE/ ERAD/ ECP technique is of Anglo-Saxon origin and has been applied so far mainly to the territories formerly part of the British Empire. Abundant literature, most often in English, describes the more important operations. Francophone countries have usually displayed a certain reluctance to take the full plunge. This reluctance might explain some of the mediocrity of the results. The only experience of a respectable magnitude is the field exercise started by the Moroccan CeRED, which we have described to the extent required by this chapter. The fuller findings of the Moroccan PGE/ ERAD/ ECP enquiry are presented in chapters 7 and 8. 93
5.3
F. Pradel de Lamaze
5.3 Two undertakings in Madagascar Having decided to undertake a large national demographic survey, the statistical service of Madagascar (INSRE) launched two successive pilot studies, one in 1967-68 on the Ambinanitelo Commune2 (approximately 14,000 inhabitants), and the other in 1969-70 on the subprefecture of Ankazoabo (population 24,000) (ORSTOM et al, 1971: 154-158). In the two cases the dual collection system was applied by comparing the documents arising out of the survey (repeated enumerations) with those of the civil registration. The concordance between the administrative boundaries permitted the use, without difficulty, of the existing registration data. The two procedures were independent of each other. There were, however, some difficulties in boundary interpretations between the two procedures. There were difficulties over time. The fact that many events were not registered until after a significant delay required the use of a longer period of registration, rectified according to the declared dates of the events, so as to ensure coverage of duration equal to that of the survey. The registration being permanent, this procedure did not present difficulties in collection. There were difficulties over space. The problem of de facto and dejure populations and events has arisen. Because the survey observed dejure events it was necessary to "retrieve" the events that occurred in neighbouring communes, but which concerned families residing in the survey area. This necessitated, theoretically, the analysis of the entire civil registration because at least some of the vital events were registered de facto in the "foreign" communes. In practice only the registration for communes neighbouring to the survey area was analyzed. The matching of events was done according to diverse matching rules as much for theoretical reasons (that is, stratification) as to permit analyses of: births and deaths by village births and deaths by month births by sex births by age of mother deaths by sex and age. On the whole, the declarations of sex and place of residence were consistent between the two sets of documents. This was not so for ages and dates of birth. As a result, the stratification that could have been the most useful (age) was the most difficult in terms of matching. The gain in precision given by a good stratification was thus counterbalanced by the mediocrity of the matching. In addition, numerous uncertainties about family and given names made the manual work difficult. The results are shown in table 5.1. It seems, then, that the quality of the household survey was appreciably the same as that of the civil registration. It was possible to Table 5.1 Estimated completeness rates in two Madagascar dual collections, 1967-70 Procedure
Births
Deaths
Household survey Civil registration Estimated completeness in the absence of survey Estimated crude rates per 1,000 pop.
81% 77% 73% 55
72% 74% 66% 18 '
Source: Orstom et al., 1971: 157 94
Francophone Africa and Liberia
5.3
estimate than an improvement in civil registration was produced by the fact of the household survey: under "normal operation" the completeness of the civil registration would only have been 73 percent for births and 66 percent for deaths. The measurement of this amelioration was obtained from a comparison of the results month by month, the completeness increasing rapidly during the period of the household survey. It is evident that this loss of independence calls into question the final result (raw total births and deaths), but it does not weaken a certain number of observations that could be made: lower declaration of female births and deaths lower declaration of births to young women (or overestimation of age) poor appreciation of dates, ages and so on. The Madagascar experience confirms, then, the importance of the risks incurred by a mechanical application of the PGE/ ERAD/ ECP technique: the probability of noting an event varies quite strongly over time (a problem that can be resolved by a "stratification"), but this variation results from the interaction of the two procedures. The prolonged application of the PGE/ ERAD/ ECP technique risks leads to a convergence of results, and nothing indicates that this convergence would be produced without systematic omission of the same events by both procedures. 5.4 The Tunisian experiment The national demographic survey conducted in Tunisia in 1968-69 put priority on the measurement of the rate of natural increase. After having adopted the principle of a multiround survey, the problem of eventually measuring omissions was raised. The survey plan chosen (surveying a sample of households in very small clusters) did not lend itself to the application of a dual collection system. After some fruitless attempts to double survey households it was decided to modify the survey plan, in two reduced zones, in such a way that comparison with the civil registration would be possible.3 Two sheikhdoms were chosen, Goraa and Oued el Kharef, in which the survey was extended to the total population coterminous with the population of the civil registration offices, and then the events reported in the household survey were compared to those recorded at the civil registration. The results are shown in table 5.2. It can be seen that, contrary to the case in Madagascar, the quality of the household survey was largely superior to that of the civil registration/ A motivational survey done in "non-registering" households established that in most cases the non-registrants ignored the civil registration or did not see it as useful. The improvement of the civil registration which has taken place as a result of delayed registration has been estimated. The results after the survey are shown in Table 5.2 Estimated completeness rates in two dual collection Sheikhdoms of Tunisia, 1968-69
Household survey Civil registration
Births
Deaths
%
%
97 79
88 62
Source: Tunisia, 1973: 77
95
5.4
F. Pradel de Lamaze
table 5.3. The estimated vital rates of table 5.3 are lower than the implied estimated vital rates of table 5.2 by the extent of late registration. Table 5.3 Estimated completeness rates in dual collection, allowing for delayed registration, two Sheikhdoms of Tunisia 1968-69
Survey Registration Estimated crude rates Goraa Oued el Khatef
Births %
Deaths %
97 86.4
88 64.5
40.5 47.0
20.5 15.6
Source: Tunisia, 1973: 39 and 78.
5.5 The case of Senegal and Corner oun During the years 1965 to 1970 various pilot studies were done in Senegal (Cantrelle, 1969) and Cameroun(Podlewski, 1970). The common idea in all these experiments was to establish permanent observation of vital events. To start with, a basic register of individuals was created, and then kept up to date by repeated surveys. In order to check this updating, another source, usually a village notable, was asked to keep a book in which the principle events were noted. It is not really possible, then, to speak of a dual collection operation but rather simply of a checking mechanism. To our knowledge, the use of the PGE/ ERAD/ ECP formula for estimation of omissions, though possible, has not yet been attempted in these two countries. 5.6 An endeavour in Algeria In 1969-70, Algeria undertook a large multiround household survey very close in its conception to the Tunisian study. The sampling design, by clusters of approximately 500 persons, lent itself very well to the setting up of the dual collection system. The size of the sample was too large (about 700 clusters) for a dual collection system. A subsample was chosen of about 80 clusters (distributed within 20 communes, that is four clusters per commune) for which continuous recording was set up, independent of the household survey itself; The procedure of matching the vital event reports from the continuous recording and the periodic household survey was thus possible for a sample of nearly 40,000 people. Essentially the goal of the sub-sample was to measure the eventual under-reporting during the national survey intended to cover about one-third of a million persons (Pradel, 1970). For several reasons this operation was not continued: i. Difficulty in recruiting field personnel for the continuous recording was experienced. Though it was possible, at the start, to find suitable candidates in all 20 communities, most were not able or did not wish to continue the work for the 18 months during which the dual collection was to take place. Many reasons were given as 96
Francophone Africa and Liberia
5.6
the causes for their abandoning the work: an apparent lack of interest in a job which in practice duplicated the civil registration, limited proposed remuneration, travel difficulties (small motorcycles had been considered but proved to be unsuitable), etc. ii. Insufficient supervision: those directing the implementation, having sufficient problems with the main survey, were apt to neglect the continuous recording, which was considered secondary. iii. Difficulties in the exploitation of results: in spite of the preceding difficulties, the continuous recording functioned satisfactorily in five or six communes where the personality of the continuous recorder was fairly strong.5 The exploitation of the results of the main survey had been difficult enough because of changes among planners, and this led to the operation of matching events being considered secondary even in the 20 or 24 clusters where it was possible. An important lesson may be gained from the Algerian project. The decision to start a dual collection system must not be taken if it is not certain to be carried to term. It must not be an auxiliary operation lest it is neglected and unusable in the end. 5.7 Morocco: an extended experiment The CeRED (Centre de Recherches et d'Etudes Demographiques) is an organization which, located in Rabat, Morocco, has as one of its main purposes the implementation of a dual collection system. In so far as the PGE/ ERAD/ ECP technique relates to CeRED's essential mission, it can be expected that, contrary to the Algerian experience just cited, this organization will be concerned with the resolution of problems that may be posed by a dual collection system. The results which it will obtain will furnish valuable lessons. Morocco has at its disposal some experience in the collection of a demographic data, having brought to successful conclusion two censuses and a multiple-purpose survey, and having planned a civil registration which was obligatory, though far from universal in coverage and less in completeness. In this context, the role of the CeRED was facilitated by a number of elements: the presence of competent personnel at all levels, an administration already favourable to statistical operations, and a receptive population. On the other hand, the clearly experimental character of the operations undertaken in the framework of the dual collection system freed the officials from the worry and restraints of immediate and productive "pay offs". A sample of experimental areas was chosen, the initial object not being national estimation. The CeRED region, equal to about a quarter of the country, constituted a field of experimentation with differing problems. Out of this region a sample of some 84,000 people was drawn, 48,000 in rural areas and 36,000 in towns. Meanwhile, certain adjustments were made in such a manner that the results could eventually be extrapolated from the CeRED region to the level of national estimates. The definition of areas of observation took into account several criteria: i. The necessity of constituting clusters of a size such that the effect of the cluster (intracluster correlation) would not be too awkward. In the particular case, the choice of large clusters would not have posed estimation problems since it was not representativeness which was being sought. However, in order to maintain the value of the exercise as a test, it was necessary to take into account the needs of the eventual national system. The clusters finally adopted generally had a population in the order of 300 households. ii. The necessity of easily definable boundaries. In order to take two observations
97
5.7
F. Pradel de Lamaze
of the same population it was necessary that the territorial limits be sufficiently clear that neither field worker might over-run them. iii. The possibility of a subdivision into "daily rations". In order to be able to justify permanent work, and eventually to permit fairly strict controls, it was necessary that the continuous recorder determine each day, his intended work area. Given the topographical conditions, the number of visits possible in a day (to search for new events only) was estimated to be between 50 and 80 households. A cluster thus could be covered by a continuous recorder in four or five days. Assuming the desirability of touring households monthly, either one continuous recorder could be devoted fulltime to the observation of four clusters, or the employment of part-time recorders had to be considered, each to be employed for a week during each month. After having chosen, as a function of these diverse criteria, areas of observation (often corresponding to the census districts), the creation of a detailed map permitted clear identification of structures and, through them, households. Large scale plans were made and each structure was numbered on the sketch and also in the field, by means of a number painted on the door. Several particular cases should be mentioned: the presence of moveable dwellings (tents) in certain districts, a complex habitat of several households in the same building, and the appearance of new buildings in the course of the study. These do not appear to have presented major problems, the "CeRED number" having been rapidly adopted by the non-mobile population in the study zone. The de facto convention was adopted in the periodic survey and in continuous recording. An exception was made for births taking place in specialized institutions (hospitals, maternity clinics) which were counted at the place of residence of the mother. This exception was justified by the importance of this phenomenon, particularly in the city of Rabat. The two procedures, continuous recording and periodic survey, were operated by distinct and separate teams, in principle without any contact. The continuous recording requires frequent information from each household. Each event occurring between two interviews is noted with its characteristics. For each event a separate questionnaire is made out in triplicate, one copy of which is left in the home. In urban areas all households are visited monthly in a systematic way, whereas, in rural milieux the continuous recorder has chosen a number of "routine round contacts" (see Glossary) or information points and only visits those households for which he has obtained "first intelligence" (see Glossary) that an event has taken place. The periodic survey is of the multi-round interview type, normally with six-month intervals between rounds. In practice it does not seem that the exact constancy of the interval can be respected. At each round the survey interviewers fill out a complete household questionnaire and also a separate document for each vital event. The periodic household survey and the continuous recording are strictly controlled, as much to ensure their own quality as to avoid contacts between the field workers which might be prejudicial to the independence of the two procedures.6 All the questionnaires corresponding to the vital events are then entrusted to a third team, working in the headquarters office, who are charged with the case-by-case matching. Returns to the field to verify cases in doubt are foreseen, but it is not yet specified whether these will be done by the third group. The dual collection system having been established late in 1971 and gone on for only a few years, few data are as yet available. They are presented in chapter 7. However, already several general remarks may be made: i. Resolutely organized and using the necessary means, this operation appears to have a more serious chance than others to lead to important results. ii. The existence of statistical and census operations as well as of civil registration 98
Francophone Africa and Liberia
5.7
in the same areas risk prejudicing the dual collection system: a certain confusion must develop in the population between all these operations, and the tendency to furnish every field worker with the same responses becomes more convenient, more especially since appropriate reference documents have been created and distributed to respondents. Thus, even though the independence of the field workers appears effectively assured, the independence of their observations seems to be contestable.7 iii. Such matching experiences as are available suggest and will suggest subtle problems, the reported characteristics of events varying enormously: spelling of family (or Kounia) and given names, age of the deceased, ages of parents, etc.8 It seems that, in the final analysis, the only element in the two questionnaires which can almost certainly be matched is the date of the event. But this element is one of the most susceptible to the risk of non-independence. A fuller report on the matching experience is given in chapter 8. 5.8 Dual collection in Liberia Started in 1969, the Liberian Population Growth Survey was based entirely on the principles of a dual collection system. It had as its objectives the provision not only of estimates of births and deaths, but migration as well. The operation, carried out by means of a survey using a sample of about 70,000 people (one-twentieth of the total population of the country) is conducted in a manner similar to that of Morocco's: the monthly recording of events by a continuous recorder, who is resident in the PGE area; and an independent, periodic household survey, twice yearly, by outside interviewers. At the start, an overview of the population is conducted by field workers of both procedures together, who have also jointly prepared the necessary geographic maps. During this overview every household is visited and a household list is drawn up without enquiries about vital events. It is only after this baseline survey that the two teams become independent.9 The continuous recorders then visit each household on a monthly basis, recording events and keeping the household's list up to date. The survey interviewers, in groups of three, visit the households after six months, prepare a new list, and ask about events that have taken place in the meantime. Note that the continuous recorders are also equipped with a fertility questionnaire, allowing them to note pregnant women. The originality of the Liberian system lies in its utilization of various methods of motivation, for the field workers of both procedures designed to enhance their behaviour as well as to insure that they remain independent. To this end "encouragement bonuses" are given to each continuous recorder who records a subsequently verified event, and to the survey interviewer for each event un