From Goals to Data and Back Again: Adding Backbone to Developmental Intervention for Children With Autism

From Goals to Data and Back Again

of related interest Giggle Time – Establishing the Social Connection A Program to Develop the Communication Skills of Children with Autism Susan Aud Sonders ISBN 1 84310 716 3

Playing, Laughing and Learning with Children on the Autism Spectrum A Practical Resource of Play Ideas for Parents and Carers Julia Moor ISBN 1 84310 060 6

Demystifying the Autistic Experience A Humanistic Introduction for Parents, Caregivers and Educators William Stillman ISBN 1 84310 726 0

Addressing the Challenging Behavior of Children with High-Functioning Autism/ Asperger Syndrome in the Classroom A Guide for Teachers and Parents Rebecca A. Moyes ISBN 1 84310 719 8

Freaks, Geeks and Asperger Syndrome A User Guide to Adolescence Luke Jackson ISBN 1 84310 098 3

From Goals to Data and Back Again Adding Backbone to Developmental Intervention for Children with Autism Jill Fain Lehman and Rebecca Klaw

Jessica Kingsley Publishers London and New York

All rights reserved. Except for the quotation of short passages for the purposes of criticism and review, no part of this book or the accompanying CD-ROM except those portions appearing in the appendices may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written permission of the publisher. Material that appears in the appendices of this book may be reproduced without modification by educational institutions. Warning: The doing of an unauthorised act in relation to a copyright work may result in both a civil claim for damages and criminal prosecution. The right of Jill Fain Lehman and Rebecca Klaw to be identified as authors of this work has been asserted by them in accordance with the Copyright, Designs and Patents Act 1988. First published in the United Kingdom in 2003 by Jessica Kingsley Publishers Ltd 116 Pentonville Road London N1 9JB, England and th th 29 West 35 Street, 10 fl. New York, NY 10001-2299, USA www.jkp.com Copyright © 2003 Jill Fain Lehman and Rebecca Klaw Library of Congress Cataloging in Publication Data Lehman, Jill Fain. From goals to data and back again : adding backbone to developmental intervention for children with autism/Jill Fain Lehman, Rebecca Klaw. p.cm. Includes bibliographical references and index. ISBN 1-84310-753-8 (alk.paper) 1. Autism in children. 2. Autism in children--Treatment. 3. Children with disabilities--Rehabilitation. I. Klaw, Rebecca, 1952- II. Title. RJ506.A9L446 2003 618.92’898206--dc21 2003041614

British Library Cataloguing in Publication Data A CIP catalogue record for this book is available from the British Library ISBN 1 84310 753 8 Printed and Bound in Great Britain by Athenaeum Press, Gateshead, Tyne and Wear

For Philip, Charles, and Sarah Kaye

— JFL

For Art, Ben, Jeremy, and Bram

— RK

Contents

1.

2.

List of Exercises

11

List of Figures and Tables

11

Acknowledgments

14

Why We Wrote This Book

15

The Big Picture

18

How to use this book

19

Identifying Goals

21

Attention and basic social relatedness

21

Imitation

23

Affect

23

Self-regulation

25

Play

26

Social play

26

Pretend play

27

Drawing

27

Communication

28

Eye gaze

29

Receptive communication (understanding language)

30

Expressive communication: body language and affect

30

Expressive communication: the use of symbols

31

Conversationsl skills/pragmatics

32

Sensory issues

33

Restricted interests and perseverative behaviors

34

Concept development

34

Increasing awareness of others

35

Social skills with peers

36

Respecting social norms

37

School and camp skills

38

Leisure

38

Things to remember

39

3.

Writing Measurable Goals Change in the child and change in the goals

4.

5.

6.

41 42

Phase 1: Emergence

43

Phase 2: Consistency

43

Phase 3: Extension

43

Writing goals that can be measured

44

Ways of recording increasing or decreasing frequency

47

Using quantitative scales

47

Using a qualitative scale

50

Ways of recording increasing or decreasing duration

52

Ways of recording an increasing range of behavior

53

Ways of recording a decreasing prompt level

55

Considering goal-independent factors

57

Things to remember

58

Collecting the Data

61

Data questions

61

Formatting the data sheet

63

Frequency of data collection

65

Special suggestions for teachers

67

Monitoring goal-independent factors

68

Things to remember

71

Putting It All Together — Joey, Tyler, and Mai Lin

73

Joey

73

Tyler

77

Mai Lin

81

Things to remember

84

From Data Collection to Data Analysis

85

Types of data

85

Translating words to numbers to create raw data

89

Transferring the raw data to spreadsheets

92

Keeping in mind what the numbers really mean

95

First impressions: numbers versus pictures

96

Creating simple line graphs

98

7.

Real data is not pretty

101

Things to remember

102

Detecting Change — The Mean Patterns of change

105

Computing the mean

111

Evaluating change using mean values

111

Step 1: Dividing the data for the goal

112

Step 2: Computing the means and charting bar graphs

115

Step 3: Comparing the means

116

Things to remember

8.

9.

105

Measuring Reliability — The Standard Deviation

118

121

Distance as a measure of representation

121

Computing and graphing the standard deviation

124

Evaluating change using mean and standard deviation

125

Things to remember

132

Exceptions That Prove the Rule — Factor Data

133

Identifying an outlier in the data set

133

Predicting sub-patterns of behavior from factors

137

Correlation is not cause

138

Translating factor data to numbers and computing correlation

139

Negative correlation

145

You get what you ask for

146

Data analysis, revisited

151

Things to remember

151

10. Coming Full Circle — Joey, Tyler, and Mai Lin

153

Joey

153

Tyler

164

Mai Lin

171

Things to remember

179

11. Conclusions

181

Appendix A Basic Intervention Goals for Children with Autism Attention and basic social relatedness Imitation Affect Self-regulation Play Increasing the play repetoire Pretend play Drawing Communication Receptive communication (understanding language) Eye gaze Expressive communication (body language and affect) Expressive communication (the use of symbols for communication) Conversational skills/Pragmatics Sensory issues Restricted interests and perseverative behaviors Concept development Increasing social awareness Social skills with peers Social norms School and camp skills Leisure

Appendix B Charts and Handouts The Three Phases of Intervention How Do We Know When Goals Are Complete? Writing Measurable Goals Detecting Change Using The Comparison of Means Sample Data Sheet — 2-year-old, more compromised Sample Data Sheet — 4-year-old, more compromised Sample Data Sheet — 6-year-old, moderately compromised Sample Data Sheet — school-aged child, more able Glossary Directions for Using Excel A very quick reference guide Selecting Deleting Inserting Changing or modifying Copying and Pasting Miscellaneous

Appendix C Exercises

183 183 184 184 185 185 185 185 186 186 186 187 188 188 190 191 191 192 193 193 194 195 195

197 198 199 200 201 202 203 204 205 206 211 211 211 211 211 212 212 213

215

Exercises 3.1

What’s wrong?

216

4.1

Creating data sheets using tables

219

5.1

Writing goals for Joey

221

5.2

Making Joey’s goals measurable

222

5.3

Writing data questions for Joey’s goals

223

5.4

Writing goals for Tyler

224

5.5

Making Tyler’s goals measurable

225

5.6

Writing data questions for Tyler’s goals

226

5.7

Writing measurable goals for Mai Lin

227

5.8

Writing data questions for Mai Lin’s goals

228

6.1

How to produce a line graph

229

6.2

Graphing shortcuts

233

7.1

Computing the mean

237

7.2

Computing multiple means on a single sheet

239

7.3

Computing “before” and “after” means

240

7.4

Producing a bar graph from data for a single goal

241

7.5

Producing a bar graph with multiple goals for one child

242

8.1

Computing the standard deviation

243

8.2

Adding standard deviation information to a bar graph

244

8.3

Experimenting with raw data, mean, and standard deviation in the Statistics Lab

245

9.1

Experimenting with outliers in the Statistics Lab

247

9.2

Computing correlations with the Pearson r

248

9.3

Understanding negative correlation

250

Figures and Tables Figures 4.1

A sample data collection sheet in an easy-to-use format

64

4.2

A sample data collection sheet for multiple children working at the same time on different goals

67

4.3

A sample data collection sheet for multiple children working at the same time on the same goal

67

4.4

A sample data collection sheet showing goal-independent factors marked by an asterisk

69

4.5

The data collection sheet from Figure 4.2 with additional goal-independent factors

70

6.1

Every item on a data collection sheet has two independent properties. We note the first property—whether it’s a goal data or factor data—at the left. The second property—whether the item results in a categorical, ordinal, or interval data set—is labeled on the right. Both properties are used to determine the appropriate statistical tests to apply to the data.

87

6.2

Joey’s data sheet after translation

91

6.3

A portion of Joey’s raw data after it has been entered on a spreadsheet

93

6.4

A portion of Anton’s data from the ABC Workbook

96

6.5

The line graph for Anton’s raw data shows a “rising stairs” pattern

97

6.6

The legend box is necessary when you combine data for two children receiving group intervention on the same dates

99

6.7

The line graph for Anton’s raw data for goal Y

99

6.8

The line graph for Becca’s raw data. Has she made progress on goal X?

100

6.9

The line graph for Celeste’s raw data. Has she made progress?

101

7.1

A single regular pattern of behavior versus two different regular patterns of behavior

106

7.2

One pattern of behavior (pre-transformation) followed by a period of transformation followed by a new pattern of behavior (post-transformation)

107

7.3

The line graph for Anton’s data for goal X

107

7.4

Is this a case of change or not?

109

7.5

With more data before and after, the pattern of change emerges

109

7.6

Our model of cognition assumes that if change occurred within the intervention period then behavior went through three phases: pre-transformation (“BEFORE”), when old patterns dominate; transformation, when learning occurs; and post-transformation (“AFTER”), when behavior is best characterized by the new patterns learned during intervention

114

7.7

Anton’s performance on goal X “before” and “after” intervention

115

7.8

Celeste’s performance on goal X “before” and “after” intervention

117

7.9

Becca’s performance on goal X “before” and “after” intervention

118

8.1

Good representation displayed as a tight clustering of data points around the mean line

122

8.2

The more data points spread out around the mean line the poorer the mean is as a representative value

123

8.3

Standard deviations displayed as error bars around the means. The bar extends both above and below the mean by the amount of the standard deviation. The oval indicates the area of overlap between the bars 124

8.4

The more overlap in the error bars, the less reliable the change indicated by the difference in means

128

8.5

A clear example of change

129

8.6

The large standard deviations argue against the evidence of change provided by the difference in the means

130

8.7

Comparison of means for Celeste

131

9.1

Overlap in the error bars suggests we look further

134

9.2

Post-transformation behavior looks regular except for a single unexpected low value in Session 27

134

9.3

The expected pattern becomes clear after removing the outlier

135

9.4

Translation of the categorical factor specifying location into numbers

140

9.5

Is Becca still learning or is there something interfering systematically with her performance?

141

9.6

Celeste seems to show a systematic regression (followed by recoupment) after every 5 sessions

144

9.7

There is no way to ask the general question, “Does performance vary with location?” Instead, the Pearson r forces us to test a set of more specific questions, each reflecting a different assignment of locations to numbers

149

10.1

Joey’s data collection sheet, as created in Chapter 5

154

10.2

Change in Joey’s ability to show preferences

158

10.3

Change in Joey’s ability to match a photo to an item

159

10.4

Apparent change in Joey’s ability to approximate the word “go”

161

10.5

Lack of change in Joey’s ability to pull an adult to a desired object

162

10.6

Lack of change in Joey’s ability to imitate mouth movements

163

10.7

Tyler’s data collection sheet, as created in Chapter 5

165

10.8

A case of poorer performance after intervention?

167

10.9

Possible change in Tyler’s conversational ability

169

10.10 Mai Lin’s data collection sheet, as created in Chapter 5

172

10.11 Possible change in Mai Lin’s ability to control her voice

174

10.12 Lack of change in appropriate facial orientation

176

10.13 Change in Mai Lin’s ability to form requests as questions

178

C.1

The icon to start the Chart Wizard in Excel 97 is circled in black

229

C.2

The options to select the Chart Type dialog box. Also note the tabs circled at the top of the dialoge box. Each tab gives you access to a different view of some or all of the information to be specified at the step. 230

C.3

When you select the whole chart the data is highlighted on the spreadsheet and a pull-down menu for the chart appears.

C.4

The locations of the function symbol, the rounding up symbol, and the formula bar. 237

235

Tables 10.1

The statistical information for Joey’s goals over a three-month period

156

10.2

The statistical information for Tyler’s goals

166

10.3

The statistical information for Mai Lin’s goals

173

Acknowledgments Writing a book that is intended for both parents and professionals presents unique challenges. Meeting those challenges would not have been possible without the help of Ivie Torres, our parent reader, who reviewed the manuscript with intelligence, honesty, and dedication. Her words of praise, suggestions for improvement, and sense of humor made this project considerably more enjoyable. Our professional reviewers, Stanley Greenspan, Lee Marcus, Diane Twachtman-Cullen, Susan Izeman, and Cory Shulman, were no less helpful. We are particularly grateful to them for providing us with comments in the all too brief time we gave them. Thanks must also go to Mary Hart and Renee Porterfield for lending their expertise in step-by-step reviews of the spreadsheet exercises. Content is one thing, clarity and style another. Philip Lehman was painstaking in his role as editor and heroic in his role as document formatter. Rosann Lehman also read the entire manuscript, searching for inconsistencies, incoherence, and typographical errors. We are grateful for every minute of their time, effort, and patience. Finally, we would like to acknowledge the special roles played by Karen Berkman, Raj Reddy, and Richard Wyckoff in the writing of this book. Each was responsible in his or her own way for professional support or guidance at a critical juncture on the path to its publication.

Chapter 1

Why We Wrote This Book

Intervention for children with autism has come a long way in the last half century. In the early years after Kanner recognized the disorder, the diagnosis went hand in hand with terms like “unreachable” and “uneducable” and with recommendations like “institutionalize.” With the work of Lovaas and others in the 1970s that view began to change. It was finally demonstrated that children with Autistic Spectrum Disorders (referred to as “ASD” throughout this book) could learn, could respond, could even engage effectively (and “affectively”) in the neurotypical world. Since the 1970s many other types of intervention have been used with children with ASD, each based on a particular medical, nutritional, or psychological theory, each with supporters and detractors. Despite the differences among interventionists, the importance of intervention itself is no longer doubted. Parents and professionals know that providing intervention is critical in helping the child with autism cope with the demands of daily living, access education, and achieve long-term independence. If we knew what every child needed, if in fact every child needed exactly the same thing, intervention would be a relatively simple matter. But the last half-century has also shown us that the skills and deficits that can accompany autism are complex. What benefits one child may have little or no impact on the symptoms of another. As a result, at least for now, there is no single intervention or set of interventions that works for every child. All of which leaves us with a fundamental question: how do you choose among interventions for a child? How do you choose among interventions for your child? We believe that whether you are a professional service provider or a family member/service consumer, you answer that question in two steps. The first step is taken when you choose a theoretical approach to intervention. Theory provides the framework or general point of view for structuring the overall program of intervention. As a professional your choice of approach might be based on prior training, an assessment of your own strengths and weaknesses, passionate personal belief, or combinations of these. As a family member your choice might be based on what fits into your family culture, what matches your 15

FROM GOALS TO DATA AND BACK AGAIN

own observations about your child’s learning style, the recommendations of professionals and other parents, and, frankly, how you feel about the people who represent the service options in your community. However you make the decision, the choice of theoretical approach is an important one, but it is not the final one. Once the first step is taken, hundreds upon hundreds of “second steps” must follow. Every day, every therapeutic session, every class period, choices must be made about what to teach and how to teach it now. And here is where the problem occurs, because although theory is a wonderful “top down” organizer, it still leaves a lot to the imagination. Theory lays out a set of possible interventions bounded by principles and guidelines. It may even dictate some specific methods and curricula. But a child—your child—may need help learning almost anything. Theory cannot cover all the possibilities. There is no curriculum for life. If we can’t take the everyday steps of intervention based on theory alone, what other source of information is there to help us? Within the framework of any particular approach, how do you choose which goals to work on and which methods to use during this moment of intervention? We believe the answer to that question is only partly determined by theory. The other crucial factor in making those intervention choices is data. If theory is top down, data is bottom up. Theory says “Choose your goals from column A and your methods from column B,” and “Don’t attempt the third one on the left until you cover the two before it.” Data adds a different voice, saying, “It’s time to move on, that concept is mastered,” and “This child responds best to the second method in the third row.” The two types of information—theoretical and empirical (data-driven)—work together. Time is too short to wander through the vast set of all the things you could possibly do until your data tells you what works. That’s why you need theory. But theory doesn’t cut down the possibilities enough to tell you the best choice for each individual child. And that is why you need data. There is, of course, a third type of information that is critical to making intervention choices: the intangible sources, intuition and feeling. We don’t mean to discount these sources when we focus on the theoretical and empirical. After all, how many of us have believed in a child’s potential despite every objective measure to the contrary? How many of us have watched children do what the theory or data said they couldn’t? While there is no question that intuition and feeling count, sometimes what we feel is uncertainty. For every person who has trusted his or her instinct there is another person who has wondered, “Is this really working?” Subjective information—intuition and feeling—works best when it is tempered by objectivity, and vice versa. We must listen to what theory and data have to tell us but, at the same time, understand the limitations of what they can say.

16

WHY WE WROTE THIS BOOK

Everybody we know who works with children with ASD has a theoretical position. Some use TEACCH, some use a Floor-time model, some use ABA or a discrete trial approach, some use special education curricula, and some mix and match. This book does not assume that you are using any particular model. It does assume, however, that your approach is consistent with a developmental perspective. In other words, we (the authors) have theoretical positions, too. One of us (Rebecca) was trained as a developmentalist. The other (Jill) was trained as a cognitive scientist. Our theoretical biases were evident long before we started to work with children with autism. When these children captured our hearts and minds we saw a chance to apply what we knew in ways that made sense to us, based on our theoretical “upbringings.” In this respect our experiences are not so different from those of our colleagues. What we have found to be different, however, is our attitude toward data. We often come across the belief that developmentally based intervention is incompatible with an empirical mindset. Interestingly, we encounter this belief about as often from those who are empiricists as from those who are developmentalists. But as far as we can see, there is nothing incompatible between the decision to use any particular theory and the decision to use data. When we first came across the notion that there was, we were surprised. When we came across it again and again we were dismayed. That is why we wrote this book. In some sense having a cognitive or developmental predisposition is superfluous to what follows because we can state the ideas behind this book without any mention of cognitive or developmental principles: · Theory alone is not enough. · Data alone is not enough. · If you are a service provider, you need both sources of information to

make good decisions.

· If you are a service consumer, you should expect that both sources will

be used by those who offer intervention for your child.

On the other hand, just saying that data is good doesn’t tell you how to collect good data. And saying that data should inform intervention isn’t the same as telling you how to extract something informative from a pile of numbers. The specifics—the how tos—require an empirical method and the nature of that method is profoundly affected by your approach to intervention. What kind of data you can collect, what statistics you can use, what measurements of progress apply—all of these things depend on the assumptions and expectations that are part of your theory. We are committed to a form of intervention that is based on the use of both theory and data because we know from experience that decisions based on their combination are more powerful than decisions based on either alone. Moreover, 17

FROM GOALS TO DATA AND BACK AGAIN

we want to show you that, contrary to popular belief, such an approach is possible, even straightforward, when the framework for intervention is developmental theory. For the past six years it has worked for us and for our staff. It can work for you. In this book we want to show you the power of empirical developmental intervention.

The Big Picture What we describe in this book is the empirical piece of an empirical developmental approach to intervention. This book is not about child development, or child development in autism, or how to do developmental intervention per se. This book is about collecting and analyzing valid and reliable data in a way that makes sense if you are a developmental interventionist. To that end, the book is organized to follow the flow of intervention from goals to data and back to goals. In Chapter 2 we start by reviewing the sorts of goals that are commonly found in a developmental approach. These are the basis for our examples in all the other chapters. Chapters 3 and 4 focus on how to take a goal and turn it into a statement that describes activity or behavior that is both observable and measurable. The key feature of this process is that the goal always comes first. We don’t ask what we can measure and then choose our goals; we choose our goals and then transform them into data questions in a way that preserves their connection to the underlying theory. By the end of Chapter 4 you will know how to write goals addressing typical developmental issues for children with autism. You will also know how to create an easy-to-use data collection sheet that helps measure a child’s progress on these goals. In Chapter 5 you have the opportunity to practice those skills in the context of case histories for three children: Joey, Tyler, and Mai Lin. Of course, data collection is only half the story. Data analysis is the other half. Chapter 6 begins the process of taking the observations collected during intervention and producing an objective measure of the child’s progress. In this day and age there is no reason why data analysis needs to be a burdensome or mathematically complex chore. Computer spreadsheet programs can be used to organize data efficiently and perform the statistical computations easily. We know the idea of “going on-line” may seem intimidating to some, so Chapter 6 takes you step-by-step through the process of setting your data up on the computer and creating an initial representation of progress in the form of a line graph. Chapters 7, 8, and 9 form the core of the section on data analysis. Each chapter introduces a different statistical tool to the analytical toolbox. Since we have developed a method that relies on spreadsheet functions to do the computational work, however, our focus in these chapters is not on developing the 18

WHY WE WROTE THIS BOOK

mathematical ideas behind the statistics. Instead, the major portion of each chapter discusses what role its statistic plays in interpreting data within a developmental framework. By the end of Chapter 9 you will understand a great deal about how both cognitive and developmental theories characterize a child’s progress toward a goal and about how that characterization can be detected in your data. Chapter 10 takes the analytical methods we’ve developed and applies them to the case histories of Joey, Tyler, and Mai Lin. By revisiting their stories we have the opportunity to close the loop started in Chapter 2, demonstrating not just how analysis is done but also how it feeds back into the next round of intervention decisions and goals. After this exercise, Chapter 11 steps back to review briefly where we are and how we got there. A final point before you get started: the methods we describe are not the kind used in academic research. They are not the stuff of higher mathematics or the ones used by statisticians. The methods are practical and designed for use in the real world with real kids. They are designed to be used by educators, clinicians, and parents even when tired, rushed, and overworked. We believe that using them should be part of your everyday routine. They cannot replace theory and they cannot replace intuition, but they can help to make sense of both.

How to use this book People read books like this one for a lot of different reasons. Although no book can be all things to all readers, we would like to make it as easy as possible for you to use the book in whatever way it serves you best. If you want to learn to use data as part of a developmental approach to intervention, we suggest you read the chapters in order. In addition, we urge you to do each exercise when it is mentioned in the text. The exercises themselves have been separated from the main text as Appendix C so that the casual reader is not distracted. Nevertheless, they offer an opportunity to practice the methods we discuss and are integral to using this book as a self-teaching guide. To use the exercises and other interactive materials on the CD-ROM effectively you should read Chapters 5 through 10 with a computer that has Microsoft® Excel (97 or higher) and Word (97 or higher) programs available. Some of you may be interested in only the data collection method we describe. For you, Chapters 3, 4, and 5 are the most relevant. Those who are interested in only our approach to data analysis will find the relevant material in Chapters 6 through 9. Individuals who do not need to learn to apply the methods in a practical sense should feel free to skip over the exercises and ignore suggestions in the text to open files on the CD-ROM. Teachers or administrators who want to use this book with their students or staff may find the charts and handouts in the three appendices particularly useful, as they summarize various aspects of our approach in a convenient form. 19

FROM GOALS TO DATA AND BACK AGAIN

Although the text of this book is copyrighted, permission is granted for the copying and distribution of material in the appendices for educational purposes. To make distribution easier, the appendices are also available as Microsoft® Word files on the CD-ROM. Finally, all readers should note the existence of a Glossary in Appendix B. In the main body of the book, a new term or idea is identified by boldface italics the first time it is discussed. The Glossary reorganizes all such terms alphabetically so that you can review a definition as needed without having to find the initial occurrence in the text.

20

Chapter 2

Identifying Goals

A great deal has been written both about typical development and about how it is compromised by the symptoms of ASD. We assume that readers of this book are already knowledgeable about these topics (or will learn about them from another source). Our purpose is to teach you how to translate basic developmental growth, as it occurs in children with autism, into measurable goals. Before we can begin to explore the measurement and analysis of progress over time, however, we have to establish a common vocabulary. This chapter reviews areas of developmental focus that are essential to remediation in order to introduce the basic set of topics from which we will create individually designed goals in the examples that follow. Is this basic set of topics or goals adequate to pinpoint areas of need in every child with ASD? Of course not. Each child presents a unique profile of strengths and areas of compromise. Each child requires a team of parents and/or professionals to work and rework the targeted areas of development until the goals match the child’s needs. To be clear, then, the following set of topics is included in this guide in order to ground our discussion in concrete and realistic examples. It is not intended to be an exhaustive list, curriculum, or intervention plan. Not every goal on this list is appropriate for every child and many goals appropriate for your child may not be included. For those who wish to use this framework as a starting point in their discussions with the rest of the team responsible for designing intervention for their children, a listing of basic developmental goals is available as Appendix A.

Attention and basic social relatedness Attending and relating to others is a skill that serves as the precursor to nearly all forms of early learning. This basic skill is often compromised in children with autism. If you don’t attend to the world of people that surround you, you are quite limited in your opportunities for learning. In designing goals that target this area, you need to look for markers of emerging social skills and figure out what typically developing children do to 21

FROM GOALS TO DATA AND BACK AGAIN

indicate that they are willingly attending to social opportunities. These markers, then, become the outcomes that measure progress in growing attention and sociability. For example, a developmentally early goal in this area might be the following: Child will respond to the overtures of familiar/preferred adults with smile, frown, reach, vocalization, or other intentional behavior.

Here you have decided that this child needs to learn to intentionally acknowledge the overtures of familiar people. The child just needs to react when approached by familiar and loving adults. This predictable social acknowledgment can be painfully absent in children with ASD. Another goal at this stage in the development of social awareness might be: Child will stay engaged with familiar adults for increasing lengths of time before attempting to leave or becoming disengaged.

After the child is beginning to socialize with key adults, you look for social interaction to get easier and easier for the child over time. How do you know it is getting easier? By tracking the amount of time that the child can tolerate social give-and-take before attempting to retreat to more solitary activities. Another goal might target more sophisticated acknowledgment of the key players in the child’s life: Child will call family members/teacher and aides by name.

We know from research that individuals with autism have a poorly developed system for recognizing faces. Researchers have suggested that typically developing people have a special center in the brain that distinguishes one face from another in order to help us define, label, and remember others in our world. We are keenly aware of tiny details when it comes to the people in our communities. Individuals with ASD, on the other hand, seem to lack this refined ability to recognize faces. Therefore, learning the names of specific individuals, which is cued by facial recognition, is often very difficult for children with ASD. To address this area of compromise, we write a goal that measures an outcome behavior (calling family members or teachers by name). This is our developmental marker that indicates growing social awareness. Although our goals measure a certain behavior, it is always important to remember what that behavior really means in terms of the child’s social, emotional, and cognitive development. The goals used as examples in this section are merely suggestions. They are intended to give the reader an idea about how to address early skills that fall under the heading of social attention (more advanced skills that mark social awareness are addressed under the heading “Increasing awareness of others,” 22

IDENTIFYING GOALS

below). Remember that even the full list, “Basic Intervention Goals for Children with Autism,” in Appendix A is not intended to be all encompassing. Each child you meet will have his or her own set of skills and deficits in the area of basic social attention.

Imitation Imitating what someone else is doing is the most basic way a child picks up skills. Long before children can manipulate symbols, long before they comprehend a complex communication system, children imitate the actions of others. In children who develop typically (and seemingly effortlessly), imitation is usually motivated by a desire to do just what their siblings, their mothers, or their fathers are doing. It is actually “social imitation” and it leads swiftly to the use of facial expression, gesture, and finally symbols as a means of communication. In many children with ASD, however, imitation is not predominantly social. It is often more mechanical (learning to open a door, fix a toy, or unwrap a candy bar) and thus does not aid in the development of communication and socialization. It is important, therefore, to help children with ASD practice and expand upon this primary technique for acquiring skills. In order to do this, we break down the imitation process into small steps and teach each step, if need be, one at a time: ° Child will imitate with object after demonstration of use of object. ° Child will simultaneously imitate with objects. ° Child will imitate hand movements. ° Child will imitate body movements. ° Child will imitate mouth movements. ° Child will imitate sounds. ° Child will imitate words.

As the child progresses through these steps he or she gains important methods for learning about both the physical and social mechanics of our complex worlds.

Affect What is “affect?” It is the expression of emotion. It is behavior that involves all or some of the following: facial expression; body language; gesture; tone, pace, and volume of communication; and content of communication. Affect has the capacity to influence, impress, touch, and move others. It has social consequence.

23

FROM GOALS TO DATA AND BACK AGAIN

Typically developing children reliably express emotion and appear to understand emotional expression in others. Children with ASD, however, often have great difficulty with both the expression of their own emotions and the understanding of emotional expression in others. It is important, therefore, in the course of designing intervention for children with ASD to find ways to teach the complex components of emotional expression. You might begin with the following goal: Child will look up to caregiver/teacher using a smile as a way of securing ongoing adult attention.

This action, meaning, “I like this…please continue,” is an early social skill upon which survival depends. It is the way that human infants and children engage others in interaction that comforts, excites, interests, and teaches. It is essential for ongoing development. Children with ASD often have difficulty using affect (in this case, a smile) to continue a pleasurable or interesting social experience. They are often ineffectual in their expression of pleasure. Therefore, it may be important to target this skill at the beginning of your intervention. For a child who is developmentally more advanced, you might write the following goal: Child will respond appropriately to common expressions of emotion in family members/familiar individuals in the classroom.

Many children with ASD have difficulty knowing how to respond when someone in their life is angry or sad or excited or scared. They might become distraught when someone is happy or become silly when someone is sad. They might become withdrawn at the expression of any strong emotion. Learning to react appropriately and with some social grace is an important skill not only within one’s family but also at school and in the community. As a final example, consider this goal as representing advanced work in developing appropriate affect: To achieve precision and subtlety in the expression of emotion, Child will use qualifiers to describe gradation of emotional experience (e.g., really disappointed versus a little disappointed).

In learning to understand affect in self and others, it is important not only to learn the basics but also the gradation of each emotion. Emotions occur on a scale for most people. Sometimes they are a little happy, sometimes very happy, and sometimes ecstatically happy. Typically developing children figure out common emotional scales without difficulty. Children with ASD don’t. They tend to see things as black or white, good or bad, always or never. It is important to help children with ASD develop the skills for understanding and regulating

24

IDENTIFYING GOALS

emotional response according to a scale that is similar to the scales of their families and school friends.

Self-regulation Having difficulty remaining alert, focused, and available for learning is a well-studied component of ASD. The ability to control the flood of emotional response when angry or sad or frustrated is also compromised in many of our children. There are entire curricula that help teach children to regulate their state of arousal. Your child may be getting extensive help with this area of difficulty from another professional. However, you might want to include goals from this area in your work with the child as well because difficulty with self-regulation affects your child in the home or in the classroom. Prior to writing this kind of goal, imagine that your team commented on how long it takes the child in question to recover after becoming distressed. The behaviors that express distress in this child linger long after the distressing event is over, interfering with the learning process. So your team decides to target this problem. The team may have collectively figured out four different strategies for calming. They will, perhaps, rock the child in a rocker. They will also repeat calming phrases, such as “It’s okay,” many times. They will give the child a stress ball to squeeze and will help the child focus on a favorite picture. After introducing these strategies to the child, the team can then measure the success of this process by looking for a decrease in the time it takes the child to recover and regroup after an incident that caused distress. Thus, you might write a goal such as: Child will recover from distress within 5 minutes with help from a familiar adult.

Over time, however, you might want to see this child become more independent in his or her self-calming. So a later goal might be: Independently using any or all of four different self-calming strategies, Child will recover from distress within 5 minutes.

Self-regulation is a problem in children with autism. Each child you meet may have a different set of challenges in this area. In Appendix A there is a brief list of suggestions pulled from intervention plans that the authors reviewed. These goals are basic enough to be useful in a number of cases, particularly since they are written in general terms. Of course, you must figure out what each child’s exact areas of difficulty are and create specific goals that reflect the needs of that child.

25

FROM GOALS TO DATA AND BACK AGAIN

Play Children with ASD often have significant difficulty with all or most aspects of play. Play is a pivotal skill in young children. It provides the content of learning and the method by which children practice and solidify information. As a designer of intervention, you need to approach this topic wholeheartedly. In general, you might consider the following areas as targets for intervention: · Learning to enjoy social play. · Developing a repertoire of playful activities. · Beginning to pretend. · Moving from concrete to abstract thinking. · Using drawing as a vehicle for creativity.

A list of common goals that correspond to these general topics is collected in Appendix A. For the purposes of discussion, however, we’ll just look at how to write goals that target social play, pretend play, and drawing.

Social play

Let’s consider a developmentally early goal: Child will joyfully participate in sensory-motor play with a familiar adult.

Sensory-motor play is the earliest form of social play in early childhood. It is not dependent on language or on rules. It rarely depends on sophisticated routine. It is just joyous, sensory-based interaction. Every child needs to begin here—even the child who is good at puzzles or computer games. A more advanced goal might focus on the need to expand the child’s repertoire of playful activities. Children with ASD can be very narrow in their interests. To explore the world in all its complexity, a child needs to get involved with a wide range of playful activities. A goal that reflects this might be: Child will expand his/her play repertoire to include manipulation, sensory-motor activities, art, music, building/construction activities, and early cognitive activities (matching, sorting, and puzzles).

This goal is designed to measure a steadily expanding “vocabulary” of behavior. Progress is indicated by an increasing range of skills. This method of measuring progress is discussed in Chapter 3. The purpose of the goal, however, should be clear. It is important to expand your child’s horizons, giving him or her a myriad of opportunities that promote learning.

26

IDENTIFYING GOALS

Pretend play

Pretend play is a complex skill that is noticeably compromised in most children with ASD. Lack of pretend play is, in fact, a diagnostic marker of autism. In order to teach a child to pretend you need to be aware of the typical sequence with which this skill emerges in young children. Starting at the beginning, your first goal might be something like this: Child will develop interest in the content of pretend play as opposed to the simple mechanics (e.g., interest will move from how the bottle fits into baby’s mouth to helping hungry baby).

Many children with ASD are fascinated by the way things work. They examine toys with the eye of an engineer and the skill of a mechanic. It is a difficult step to move from interest in the movements of a little train’s wheels, for instance, to moving the train around the room making train noises and imagining a real train experience. Children with ASD need help with this transition. When they have achieved this goal with your persistent modeling and encouragement, a new world becomes available to them. Later in the process of promoting pretend play skills with a child, you might write the following: Child will participate, with adults, in increasingly elaborate make-believe, moving from concrete themes (involving everyday, common experience) to abstract themes (involving material never experienced).

This represents a shift into fantasy—the ability to create a mental image of something new and different. This skill is needed for all later learning when the school-age child is presented with information foreign to his or her experiences. In order to learn about pioneer life, for example, a child needs to be able to conjure up a life that has little relationship to the child’s own experience. School-age children learning about pioneers must be able to imagine a world without electricity or cars. Typically developing children can do this, some better than others, because of the imagination that has been developing from the time they were 18 months old. Many children with ASD, however, need to be taught how to pretend. It doesn’t just happen.

Drawing

A final area that might be targeted in your attempt to promote play skills in children with ASD is drawing. Drawing is important because it is another vehicle for creativity and abstract thinking. It is an activity that children often do together and it is a requirement for most children who enter kindergarten. For a variety of reasons, children with ASD are often stunted in their ability to draw and create representations (some children with ASD, however, are very gifted in this area). If your team decides that this skill needs to be strengthened in a partic27

FROM GOALS TO DATA AND BACK AGAIN

ular child, they should be aware of how drawing progresses in the typically developing child. For example, your goals could progress in the following sequence: ° Child will scribble with crayon. ° Child will imitate drawing of vertical line. ° Child will imitate drawing of circle. ° Child will add three parts to incomplete human drawing. ° Child will copy drawing of square. ° Child will draw unmistakable human with body, arms, legs, feet, nose,

eyes, and mouth.

Communication For many children with autistic spectrum disorders, learning to communicate meaningfully and effectively opens the door to the social world. But learning to communicate is often a huge challenge. As educators, parents, and therapists, we are aware of the complexity of teaching someone to communicate. We are also aware that children with ASD don’t always respond to the cues that allow a typically developing child to acquire communication so easily. We can’t just provide an environment rich in language. We can’t simply count on incidental learning of language. There are too many barriers for children with autism. What we can do to teach children with ASD to communicate is break down the process of becoming an effective communicator into small steps and explicitly teach the steps one at a time. This process is not, however, just about the production of language. It is about social connection, all the way from a meaningful look to a whole sentence. In teaching children to communicate, it is helpful to divide the communication goals into the following categories that address the particular problems associated with the diagnosis of ASD: · eye gaze · receptive communication (understanding language) · expressive communication: body language and affect · expressive communication: using symbols · conversational skills (pragmatics).

A lengthy list of common goals that correspond to these sections is provided in Appendix A. For the purpose of discussion, however, let’s look at the categories listed above in greater detail.

28

IDENTIFYING GOALS

Eye gaze

Eye gaze is an early and effective means of communicating. It is the vehicle for two processes known to be problematic in children with autism—joint attention (“Hey, look at that…Did you see that?”) and social referencing (“Is this stranger at the door safe? Is it okay to climb on this shelf ?”). These two processes are thought by some to be the key areas of compromise in children with ASD because it is through joint attention and social referencing that we learn to establish meaning for events. Meaning is constructed, researchers tell us, from three sources of information: 1.

what we see, hear, and feel

2.

what we remember and

3.

what we learn from the feedback our parents, teachers, and therapists give us about a situation.

Consider this: you are a very young child. A stranger comes to your home. Your senses tell you there is someone new in the room. Your memory tells you that this person is unfamiliar, which creates feelings of unease. You then seek more information by looking at Grandma. She is smiling. She looks at you and the stranger and continues to smile. Her feedback interprets the event for you as safe. You adjust your initial reaction and welcome the new person into your environment. How did you construct meaning for this event? You used your senses, your memory, and social referencing. Looking at other people to see if they are experiencing what you are experiencing is a crucial mechanism for attaching common meaning to your world. Attuning your reactions to the reactions of others is a key process in the development of social and emotional skills. Both of the above require consistent eye gaze, which is often difficult for children with ASD. To begin to address this complex area of deficit, you might consider writing the following goal: Child will look back and forth to make sure adult sees what child sees in order to share enjoyment.

or: Child will look towards adult to make sense of an ambiguous situation.

These goals can be shaped into measurable objectives, as discussed in the next chapter. Don’t ignore them because they seem hard to measure. They are, perhaps, the crux of intervention. The lack of joint attention and social referencing may be the point of divergence between typical and atypical development in the young child with autism.

29

FROM GOALS TO DATA AND BACK AGAIN

Receptive communication (understanding language)

It is quite easy to write goals that track language understanding in highly practiced tasks or highly simplified environments. What is less straightforward, however, is writing goals that look for the emergence of language understanding in context. Despite the difficulty of measuring progress in this way, we must keep in mind that the underlying cognitive change we want is not well characterized by behavior in which a child points to a picture in response to a familiar question. The goal of language intervention is increased understanding as part of the child’s natural world. That is where behavior that indicates underlying change will emerge. As an example: Child will look for family members when asked, “Where is Mommy?” or “Where is Daddy?”

This goal not only highlights an early step in the development of receptive language, it also puts that step in the context of the child’s social world. The typically developing child is driven to keep track of important people. This is part of normal social and emotional development. Teaching a child with ASD to understand “Where is Mommy?” or “Where is Daddy?” not only supports the development of communication but heightens the child’s awareness of his/her parents. To further illustrate this point, here are more goals that measure progress in receptive skills as they occur in a situation that is meaningful to both the child and the caregivers. ° Child will respond to his/her name. ° Child will stop action in response to “No!” ° Child will appropriately respond to the command “Stop!” ° Child will move in response to a one-step direction. ° Child will take object or food to someone when requested. ° Child will indicate approval when asked a “Do you want?” question. ° Child will appropriately respond to simple and familiar “Where?” ques-

tions with searching movements.

Expressive communication: body language and affect

It is important to target expressive communication as it occurs before symbolic language develops. Babies, for instance, are great communicators. They use body language and facial expression to communicate complex ideas. There is much to learn about communication before we use words. Consider this goal, for example:

30

IDENTIFYING GOALS

Child will respond to gestures with intentional gestures of his/her own (e.g., reaches out in response to outstretched arms).

or: Child will indicate disapproval using gestures and body language.

Both of these highlight the need to communicate effectively long before language is expected. It is important to work on improving communication while respecting the developmental process. If you emphasize the use of words, for instance, when there is little to no non-verbal expression, you may create communicators that are rote and ineffective in the real world.

Expressive communication: the use of symbols

Typically developing children begin to communicate using symbols with apparent ease. We label things for children and praise them for their fine efforts in repeating those labels, but beyond that, we are merely amused observers of the process. Typically developing children simply begin to talk one day and keep on practicing until they have perfected the process. Children with autism, however, often have great difficulty learning to use symbols for communication. Whether you are using signs, pictures, or words, you are essentially teaching that these almost random items represent real objects or actions in the world. The word “chair” represents all those things you sit upon. The word “swing” represents that wonderful activity you do with Dad. Does it make sense? It doesn’t, really. It is simply a system of symbolic representation upon which society depends. So how do we begin to teach symbolic communication? We teach symbolic communication in the context of the child’s daily life, duplicating, as best we can, the typical developmental process. We teach in small steps, with meaningful strategies. We help the child with autism understand the fundamental concept of symbolic reference through actions that are meaningful to that particular child. The following list represents a small sample of developmentally early goals that target the process of learning to communicate with symbols. Additional goals are included in Appendix A. ° Child will learn fill-in-the-blanks of familiar songs, rhymes, and/or

familiar verbal routines (e.g., “Ready, set, go”).

° Child will use word/sign/picture for “more.” ° Child will indicate that he/she is done with an activity by saying or

signing, “All done.”

° Child will develop consistent vocabulary of symbols used in the absence

of concrete gestures (e.g., child will come into the dining room and say “juice” to mother to request juice without needing to take mother to refrigerator and touch the juice bottle). 31

FROM GOALS TO DATA AND BACK AGAIN

° Child will respond to question, “What’s this?” ° Child will ask question, “What’s this?” ° Child will spontaneously add single words to play, beginning to narrate

play actions.

° Child will ask for help.

Conversational skills/pragmatics

Just because a child with ASD can use many words does not necessarily mean that he or she can use these words meaningfully in conversation. The art of reciprocity is a separate and vital part of mastering language. Choosing the right words and saying them the right way so that you and your conversational partner remain “in sync” is something most of us do relatively easily. Individuals with autism rarely, if ever, find this process effortless. We teach social reciprocity the same way we teach everything else, by breaking it down into manageable pieces of learning, and writing goals that represent small, developmentally sequenced steps. For example, we might begin with something as simple as: Child will use attention-getting words such as “Hey!” when beginning a conversation with a peer in school.

or: Child will use appropriate distance between self and conversational partner.

As the child is becoming a more proficient communicator, we might add the following: Child will use eye contact to signal that he/she is finished talking and that it is now the partner’s turn to talk.

Finally, as the child is becoming quite skilled, you may still have to refine his or her skills with goals such as: Child will initiate conversation that is of interest to social partner.

or: Child will ask questions that are related to the topic, even when the topic is not a preferred subject, to maintain the conversational flow.

32

IDENTIFYING GOALS

Sensory issues In children with autism, there are many differences in the functioning of the central nervous system. For the purpose of this section, the relevant differences relate to how a child reacts to and processes sensory experience as well as how a child plans and organizes responses. Children with ASD typically have difficulty with the following: · Sensory modulation: they might be over- or under-reactive to touch,

sight, or sound.

· Sensory processing: they might have difficulty understanding language

or interpreting visual-spatial, tactile, vestibular, or proprioceptive information.

· Motor planning: they might have difficulty carrying out movement

sequences.

As developmental interventionists, we need to write goals for children who are experiencing difficulties with sensory issues in their home, in their school, and in the community. As we target the impact of sensory issues in the child’s environment, other professionals might be building skills in a more isolated context such as a therapy room. With intervention concerning sensory issues, and perhaps in all types of intervention, a team approach is essential because there are often two avenues for teaching skills. On the one hand, the child needs to learn and practice, one by one, the specific sub-skills that are embedded in each complex task. On the other hand, the child needs to learn when and how to use those sub-skills together and in the actual contexts in which they are required. It’s like teaching someone to play basketball. First you teach the person to dribble, to shoot foul shots, and to do lay-ups. You might need to teach him or her how to defend the basket and how to score. But that is only part of what the person needs to learn. The other piece he or she needs to become proficient in is the experience of putting the bits of learning into action in the appropriate situations. The person needs to practice playing the game. As a teacher, parent, or therapist, you are the one that helps the child “play the game” with sensory issues. You need to identify the areas of compromise and while your team members work to teach isolated skills in their therapy rooms with special equipment or exercises, you help the child put those skills to use together. As a result, your goals might look like the following examples: ° Child will eat a greater variety of foods. ° Child will gain comfort with activities in which his/her feet are off the

ground.

° Child will tolerate proximity of other children. ° Child will become sensitized to, and appropriately label, hot/cold/pain. 33

FROM GOALS TO DATA AND BACK AGAIN

° Child will walk around toys, pets, and people on floor. ° Child will successfully avoid bumping into people. ° Child will develop compensatory strategies for feeling comfort while in

large, open spaces.

° Child will become more comfortable with activities that involve hands

and face.

° Child will become more comfortable with multiple voices singing.

Restricted interests and perseverative behaviors Having restricted interests and/or perseverative behaviors is one of the hallmarks of ASD. Children at both ends of the spectrum can exhibit a need for sameness and a need for ritual, repetition, and control. Many researchers have documented that the incidence of idiosyncratic, solitary behavior decreases as the child becomes more invested in, and comfortable with, the social world. The more the child is engaged, the less the child perseverates on solitary activities. What is our role, then, as interventionists, in this particular phenomenon of autism? We must work to make the child’s world more comfortable. Sometimes we need to alter the environment so that the barriers to overcoming sensory differences are lowered. Sometimes we need to give the child other actions that comfort, yet are interactive. But we always keep the child engaged. Two examples of goals addressing perseverative or idiosyncratic behavior are: Reciting passages from books, videos, TV, and/or radio will decrease. Child will stay focused on shared conversation with caregivers instead of lapsing into private reference.

We write goals like these because we want the idiosyncratic behaviors to decrease. That is the change we want to observe over time. What that change should indicate, however, is not that we have finally eradicated a strange behavior, but that we have increased the child’s comfort and willingness to remain connected to his or her social world.

Concept development There are many concepts that we assume will come into focus if we simply give a child enough opportunities to experience information. But what educators, parents, and therapists have noted is that providing a concept-rich environment isn’t enough for some children with autism. Concept learning typically happens as a by-product of social interaction, something that is difficult for children with ASD. Therefore it takes structured, creative, and strategic teaching to help a child with autism learn common concepts such as time, quantity, or emotions. 34

IDENTIFYING GOALS

As interventionists, we target these apparent gaps in the cognitive profiles of our children. We note that although they can, perhaps, read and do math, they don’t categorize information in common ways, they don’t understand the passage of time, and they remain very puzzled by “Why?” questions. We therefore write goals that highlight the gaps in understanding that we observe in each child. We might write the following: Child will demonstrate understanding of function of familiar objects by selecting the correct item or insisting on the correct item when “mistakenly” given the wrong item.

This goal seeks to teach the function of familiar objects as these functions arise over the course of the child’s day. Perhaps the parent begins to brush the child’s hair with a shoe. What does the child do? Perhaps the therapist puts on slippers when getting ready to go outside, or attempts to open the door with a spoon. What does the child do? According to numerous theorists, the child will more easily retrieve this information in the future when both active problem solving and humor have been involved in the initial learning phase. Consider this goal: To demonstrate an understanding of locative state and prepositions, Child will be able to answer “Where?” questions.

This is a sizable task. It is not the beginning step in teaching the child about where things exist in his or her world; it is a final goal that targets the consistent and meaningful use of these skills. When you ask Johnny where his Woody doll is because together you need him for the house you just built out of Lincoln Logs, his answer will alert you to how well he understands the concept of locative state. That is what you are measuring—language that indicates a comprehension of certain concepts.

Increasing awareness of others This section is a follow-up to the first area for intervention, “Attention and basic social relatedness.” The goals discussed here target more developmentally advanced skills along this dimension. So, while before we were concerned with the child in the small, sheltered world of his or her home or classroom, now our expectations concern a larger world—one that includes various adults, neighbors, and peers. As interventionists, we need to alert the children with whom we work to the various facets of social reciprocity. In order to be truly connected to another human, one needs to be aware of what that other person is doing, is feeling, is seeing, and is hearing. One needs to have clues into what that other person might be thinking and what he or she probably knows. These are essential skills for the child within the family, school, and neighborhood. 35

FROM GOALS TO DATA AND BACK AGAIN

Goals for this topic might include the following: ° Child will note what others are doing and shape his/her behaviors

accordingly.

° Child will predict what others might see or hear in a given situation. ° Child will demonstrate an awareness of the needs of others by spontane-

ously offering help.

° Child will receive a daily compliment for being considerate. ° Child will predict what others might think or feel in a given situation.

Social skills with peers What a formidable task it is to figure out how to break down the process of making and keeping friends into little goals! Making friends is a complex human phenomenon with a rich array of skills required at every step. Some of the children with whom you work may need extensive help in developing social skills with peers and some may need just a few refinements to modesty or bossiness or empathy. You may, for example, need to target the following: Child will successfully initiate conversation/play with peer.

Such a goal is appropriate for children who don’t know how to begin. Even though they may be skilled at asking their parents or siblings to play or talk, these children become uneasy when confronted with kids on a playground or in a classroom. They need, with your help, to learn to invite another child into interaction. They need to make an overture to another that is effective and successful. As you help them learn to do this, you can measure their progress. Later, you may designate the following as the next step: Child will be able to join others already engaged in a play activity or in a conversation (as opposed to having a peer join him or her).

This seems to be a much harder skill for children with ASD. Joining something that is already happening takes a certain melding of behavior. You need to slide in, making few, if any ripples. You need to do what the others are doing so that you are included in their activity. How very difficult for children who feel more comfortable when they have some degree of control. Another aspect of social skills with peers is learning behavior that is conventional in a social setting. You might need, for instance, to teach the child the following: Child will apologize when he/she bumps into someone.

or: 36

IDENTIFYING GOALS

Child will develop tactful responses to describe dislikes and disagreements as they arise during interaction with peers.

This entire topic, “social skills with peers,” is vast and complex. Keep in mind that many of the other sections target related skills; look in the areas of play, expressive communication, conversational skills, respecting social norms, and school and camp skills. Finally (and always), remember that the team that surrounds the child will be the best designer of intervention. Goals may well emerge from the team that are better suited for a particular child than the goals highlighted above or those found in Appendix A.

Respecting social norms Hand in hand with increasing social awareness and developing social skills, most children with autistic spectrum disorders need to be taught standard social expectations. It is easy to assume that the children with whom we work will figure out what they should do and what they shouldn’t. After all, they are surrounded in school by children who know these basic rules, so won’t they learn them, too? The answer is often “no.” Learning to conform to the unwritten rules of social behavior is difficult. It is difficult because children with autistic spectrum disorders are not always aware of common expectations, and if they are aware of these social conventions they may not be motivated to restrain an impulse that runs counter to convention just because their friends might not approve. Think about nose picking, for example. In our experience, a certain child may be aware that a friend frowned upon his or her nose picking last time. But if his or her nose is uncomfortable, that feeling seems to take precedence over any social obligation to follow the rule “Don’t pick your nose in public.” For this rule and others, parents, teachers, and therapists end up teaching the rules of social conformity as the need arises. A formal goal for this particular issue might be: Child will wipe nose on tissue and throw tissue away.

You might find yourself needing to teach lessons in ownership and/or privacy, as demonstrated in the following goals: Child will demonstrate an understanding of ownership by refraining from taking someone else’s food or belongings.

or: Child will demonstrate an understanding of modesty and/or privacy by being fully clothed when leaving the bathroom in public places.

More goals that address common problems in learning about social norms can be found in Appendix A. 37

FROM GOALS TO DATA AND BACK AGAIN

School and camp skills As designers of intervention, we are often in the position of helping children learn a set of skills for successful functioning in a variety of environments. One of the greatest developmental challenges for all children is learning to move from the security and predictability of one’s home and family to the new demands created by multiple environments with multiple caregivers. The skills that are required in school, at camp, or in the community are often unlike skills needed at home or in a quiet, familiar setting. New environments mean new demands. What parents, teachers, and therapists need to do, once again, is figure out together which new demands are posing particular strain on the child in question and design intervention accordingly. You might need to target basic group skills such as: Child will raise hand when wishing to speak and will wait until he/she is called on before speaking out loud.

You might need to target skills specific to the difficulties commonly experienced by children with autistic spectrum disorders, such as: Child will tolerate changes in the schedule when prepared for these changes.

Finally, you might need to design goals that are specific to one child only because no one child with autism is ever identical to another child with autism. Your goal could look like this: Child will tolerate being at both the middle and the end of the line.

This child obviously had difficulty with line position. Is this true for many children? No. Is it true for many children with autism? Probably not. Is it true for at least one child with autism? Yes.

Leisure Leisure is defined by the American Heritage Dictionary as a noun meaning “rest” or “freedom from time-consuming duties, responsibilities, or activities.” If our American concept of leisure were just that, children with autistic spectrum disorders would be skillful masters of leisure time. In America, however, our concept of leisure time for a child is partly defined by doing things with family. Usually, children with autism are not skillful masters of the family trip. The family outing lacks predictability and the presence of novelty can be alarming. Simply leaving the house on the weekend can be irritating for some children with ASD. It is usually the parents of the children with whom we work that define leisure goals. They are the ones that know what needs to be learned so that the 38

IDENTIFYING GOALS

child can experience life as the rest of the family experiences it. They are the ones that point out the skills, big or little, that seem to be missing. We can help families improve leisure skills in their children. It takes the same strategy that everything takes—breaking down expectations into little steps and working on them one by one. Leisure time skills, like everything else, can be written into goals that can then be measured to mark progress over time. Here are a few examples: ° Child will increase repertoire of tolerable family outings. ° Child will keep caregivers informed of where he/she is going during

outings.

° Child will join routine family activities, from start to finish, without

needing to be continually prompted to stay involved.

° Child will join non-routine family activities, from start to finish, without

needing to be coerced.

° Child will be able to tolerate winning and losing.

Things to remember This was a long chapter with a lot of information to absorb. Still, the key ideas are straightforward: T Goals must be written in a way that reflects basic development growth. T Even though we are measuring small behavioral changes, these

behavioral changes represent shifts in the child’s social, emotional, and cognitive development.

T The goals suggested here and in Appendix A are only a beginning in the

process of designing intervention. The actual goals for an individual child are always a team effort that pinpoints specific and often unique areas of need.

39

Chapter 3

Writing Measurable Goals

Recognizing and writing developmentally appropriate goals, like those in the previous chapter, isn’t the same thing as recognizing and writing measurable goals. In our experience even the most well-meaning intervention teams often write goals or objectives that don’t mean what the team intends the goals to mean, and/or can’t be measured in a meaningful way. For example, we recently came across a goal that was written for a young boy who was receiving home-based services. The goal read: George will not be aggressive 80% of the time.

The flip side of this statement is that George can be aggressive 20% of the time and still meet the goal! Twenty percent of a 24-hour day is 4 hours and 48 minutes—meaning 4 hours and 48 minutes of aggression each day was OK. This child, we knew, slept 12 hours a night. He obviously wasn’t aggressive while sleeping. He also went to school for 7 out of the remaining 12 hours of the day and we knew that he wasn’t having any difficulties with aggression in school. So out of the 5 hours of time he was awake and not in school, George would have reached his in-home therapy goal if he remained non-aggressive for a mere 12 minutes. Is this what the team intended? We don’t think so. They may have meant: George will refrain from becoming aggressive with the therapist (kicking, biting, or hitting) four out of every five times he has an emotional outburst in the home.

or they may have meant: During therapy in the home, the frequency of physical aggression towards the therapist will diminish to only one aggressive act (hitting, biting, or kicking) every five sessions.

Or they may have meant something else altogether. If George’s team had written the goal in a clear, specific way, there would have been no confusion concerning 41

FROM GOALS TO DATA AND BACK AGAIN

either the meaning of the goal or how to measure it. That’s what this chapter is about: giving you the skills you need to write goals that reflect the actual behavior you want to measure in a meaningful way.

Change in the child and change in the goals What are we measuring when we write developmental goals for children with autism? Only one thing: change over time. When we work with a child what we want to see is a change in targeted skills and behaviors that may take days, weeks, or months to produce. We are not comparing groups of children to each other or comparing a child to some standard. We are concerned only with a single child’s progress over time. Much of the difficulty in understanding how to approach data collection for developmentally based intervention comes from two different ideas of “progress over time.” On the one hand we need to write goals that reflect changes in small, concrete pieces of behavior. We expect that these kinds of changes can be accomplished in a modest amount of time; this is progress on a small time scale. On the other hand, the knowledge and skills that underlie the behaviors—what we’re most interested in—may take a longer time to change in a general and lasting way. This is progress on a larger time scale. We unify the two time scales by realizing that every large-scale skill or behavioral change must be led through three distinct phases. In the first phase, the child demonstrates certain small-scale behaviors in a single, familiar environment with one teacher or caregiver. Then the child must progress to the second phase in which he or she demonstrates consistent use of the behavior whenever it’s appropriate. Finally, the child must progress again to a phase in which he or she is consistently using the skill in multiple environments with multiple caregivers and natural distractions. This means that each small, concrete goal we write is actually related to other goals in the child’s past and future as part of a deeper, more lasting kind of change. By viewing progress against this backdrop we begin to see how to take large developmental changes and make them measurable. What does the process of balancing small-scale and large-scale progress look like? You and your team begin by figuring out what the child needs to learn next. To help with this decision you might use a curriculum or a set of developmental guidelines. Suppose, for example, that Johnny’s team decides to target the social practice of waving goodbye. Within this large-scale goal there are at least three small-scale goals that follow the child’s progression through the phases of learning. Let’s look at that progression for Johnny, and for Johnny’s goals.

42

WRITING MEASURABLE GOALS

Phase 1: Emergence

In Phase 1 the goal statement describes the emergence of a skill given multiple and supported opportunities in a sheltered, consistent, therapeutic environment. So the first concrete instance of a small-scale goal contributing towards Johnny’s large-scale goal might be written: With multiple prompts and frequent modeling, Johnny will begin to wave goodbye to family members and caregivers leaving his home.

Because learning is most challenging in this phase, you begin at home or in a small classroom with a consistent caregiver. To the extent possible the interactions are based around the child’s interests and natural pursuits. You begin to work with the child in a manner that is tailored to his/her particular learning needs. You offer rigorous support and constant rewards in an effort to help the child move into new skill areas. In short, during the emergence phase, the targeted skills are taught and re-taught, supported and maintained. Only when a child shows progress in this setting under these circumstances does the purpose of intervention refocus.

Phase 2: Consistency

In Phase 2 the goal statements focus on consistency: the frequent and repeated practice of the behavior given specific surroundings. The next concrete instance of a small-scale goal contributing toward Johnny’s large-scale goal might be written: With a decreasing need for prompts, Johnny will wave goodbye to familiar adults and his siblings.

Phase 2 statements reflect your aim to have the child be able to retrieve the skill predictably and reliably. At first you might prompt for the new behavior at every possible opportunity, but once the child easily retrieves that piece of learning when it is needed, you pull back on your prompting so that the child becomes more independent. You may use drills for teaching, you may set up situations for learning within the child’s natural environment, or you may rely on incidental “teachable moments.” However you do it, this is the stage for practice. When a child shows consistency and independence in this setting and under these circumstances, the purpose of intervention is refocused again.

Phase 3: Extension

In the final phase of large-scale progress on this goal, Johnny’s team might write:

43

FROM GOALS TO DATA AND BACK AGAIN

With decreasing need for all prompts, Johnny will wave goodbye in home and school environments when someone who is leaving waves to him or says “goodbye” first.

Followed sometime later by: In all environments, Johnny will consistently wave goodbye when someone who is leaving waves to him or says “goodbye.”

In other words, in Phase 3 the small-scale goals target the extension of the skill to multiple environments and the degree of independence in those environments. You need to make sure the child can do whatever you expect him to do in the grocery store, at the playground, and in school. You also make sure the child can do whatever you expect him to do with other adults, siblings, cousins, neighbors, and peers. Increased support may be required temporarily in each new environment just as familiarity of environment may be required temporarily with each new person. But the temporary reintroduction of support should not be considered a step “backward.” It is part of the natural progression of large-scale change. Overall, the final phase of intervention helps the child take an independent and consistent skill out into the world. Only then is the large-scale goal complete. In designing developmentally based intervention, there is clearly a flow to the learning process that must be reflected in the goals you write. Charts 3.1 and 3.2 in Appendix B summarize the process visually. While understanding the flow is critical, it is not enough. To write small-scale measurable goals (like the examples in this section) we need to know more than just the phase of the large-scale goal they represent. We need to know both if we are looking for emergence, consistency, or extension and under what conditions we should see the emergence, consistency, or extension. We need to know the Who, What, When, and Where of each small-scale step. To get measurable we need to get specific.

Writing goals that can be measured Chapter 2 gave you some ideas about how to write goals that reflect both typical development and the areas of compromise in children with autistic spectrum disorders. The first section of this chapter demonstrated how to set the small concrete goals we work with day-to-day into a larger perspective. But once the team has created a list of objectives that represent the current needs of a certain child—each objective clearly identified as being in a particular phase—how do you shape each goal so that it can be measured reliably? You shape a goal first by qualifying each behavioral outcome and then by quantifying it. To qualify a goal you must be specific about the variables that surround the desired outcome. If you are unclear about the specific conditions under which the

44

WRITING MEASURABLE GOALS

behavior is to occur, the meaning of your data will be unclear as well. In particular, to qualify a goal you must add language that designates: · Where the behavior is to occur. Do you expect the child to exhibit

this behavior in a sheltered and familiar environment? If so, which environment? Do you mean familiar environments that are free of unexpected events? Or do you expect the child to show this behavior in community settings where things can be unpredictable? Qualifying where the behavior should occur means including phrases like “at home,” “in the regular classroom,” or “in multiple community settings.”

· When the behavior is to occur. It might be difficult to measure a new

behavior over a long period of time (like an entire school day) so you should limit the time that you measure a particular goal. Sometimes you might need to measure a targeted behavior during the same time each day to control for changing variables. Other times you might need to measure something in intervals throughout the day to look at the effect of changing variables. In either case, you need to be specific about your time segment. This type of qualification is made by phrases like “during ‘free play’,” “during transitions between classes,” or “in the 2-hour period before bed when Johnny is playing with his siblings.”

· Who observes the desired behavior. Do you expect it to emerge only

with the most trusted adults (“with Mom or Dad” or “with the instructional aide”)? Or do you think the child is ready to show that behavior to many people in his or her community (“with any child on the playground”)? You need to be specific.

· The level of support the child needs to demonstrate the behavior.

Demonstrating a behavior with multiple prompts is a very different skill from demonstrating that same behavior spontaneously. Although the degree of prompting is intimately tied to where the small-scale goal falls in the larger learning progression, specificity is still important. Do you expect the child to need multiple prompts because it is a new and difficult skill (“with multiple prompts”)? Do you expect that the child will need only a single prompt? Or are you intending that the child demonstrate the desired behavior independently, using only natural cues (for example, “when someone waves first”)?

· The type of measurement to be collected. Are you asking the person

collecting the data to estimate? Do you want to know what the best outcome was for the day’s work or do you want to know what the average outcome was? The more specific your goal statement is, the more certain the meaning of the measurements.

45

FROM GOALS TO DATA AND BACK AGAIN

After you’ve qualified it, you need to quantify the goal by choosing a system for measuring progress. Having reviewed many treatment plans for children with ASD, we have identified the following systems of measurement as both practical and meaningful for developmental outcomes. For each goal, choose one of the following types of quantification: · Increasing or decreasing frequency of behavior. You might want to

see a child wave goodbye more and more often when Mom leaves for work. You might want to see fewer and fewer incidents of self-talk in the middle of social play at school. This is the most common system for measurement. It is not, however, the only option you have.

· Increasing or decreasing duration of behavior. You might want to

see a child sustain attention to playful encounters for longer and longer periods of time. You might want to see a gradual reduction in the amount of time it takes a child to respond to his or her name.

· Increasing a range of behavior. This type of measurement is used

when you want to see the gradual acquisition of a set of skills over time. You use this system when you are looking for a child to expand his or her repertoire of play, or when you are measuring, over time, a child’s ability to respond favorably to a variety of people. However, a growing vocabulary—linguistic or behavioral—can’t be measured on a single day. This is a system that measures an accumulation of skills over time.

· Decreasing need for assistance/prompts. This is often the system of

measurement that is used when it is time for the large-scale goal to move from emergence to improvement and finally to consistency. If a child previously needed multiple prompts in order to answer a “Why” question, for example, and now only needs an expectant pause, that is progress! If a child used to need hand-over-hand assistance to wash his or her hands and can now do it with only a set of visual reminders, that is progress! A bonus to using this method of marking progress is that it makes the team aware of how they prompt. Becoming complacent with your level of support is never good for the child.

The process of qualifying and quantifying your goal statement makes it clear to everyone on the team both what you are measuring and how you are measuring it. To complete the process, however, you must choose a specific scale, unit, or quantity that will be used to record the child’s behavior. Each method of quantification allows for different types of observations, as we detail in the following sections.

46

WRITING MEASURABLE GOALS

Ways of recording increasing or decreasing frequency Frequency can be measured in one of three ways: by tally of occurrence (a quantitative scale), by percentage (another quantitative scale), or by a qualitative scale. Each method has appropriate circumstances for its use, as explained below.

Using quantitative scales

A quantitative scale measures progress by recording increasing or decreasing quantity. One type of quantitative scale, the tally of occurrence, is used for measuring progress in the development of a skill when opportunity is constant. Let us show you an example of such a goal. Goal: With increasing frequency, Peter will show pleasure by smiling during structured play with his aide and a small group of peers. Data question: How many times in a ½ hour of structured play did Peter smile to indicate pleasure? 0 1 2 3 4 5 >5

Each example in this section has one part labeled “goal” and one part labeled “data question.” The data question lays out the scale or set of alternatives for the way of measuring progress specified in the goal. We will explain data questions more fully in Chapter 4; for now we are focusing on the bigger picture. In Peter’s case, the team wants to see the emergence of a new behavior. Since a new behavior is most easily learned in a familiar environment the qualifying phrases “with his aide and a small group of peers” and “during structured play” are added to detail the specifics. The general quantifying phrase in the goal statement (“with increasing frequency”) is made specific in the data question (“times in a ½ hour of structured play”). Since there are unlimited opportunities for Peter to smile within the specified environment, a tally of occurrence is an appropriate system of measurement for this goal at this stage. Now let’s think about how this qualified and quantified goal relates to intervention with Peter…maybe Peter only rarely showed pleasure when this goal was written. During the first week of intervention, perhaps Peter smiles only once. The next week, there are three times when he smiles. The following week, Peter not only smiles at his aide, but begins to smile at the other children as well. Do you see what is happening? If you were keeping tallies of occurrence, your data would have registered change over time. Let’s think about another example. Shawnna is a child who has a very difficult time pretending to be anyone other than Shawnna. Her sisters want to play dress-up and Shawnna never joins them. Shawnna is awkward with little figures and doesn’t join her sisters when they play with dolls. A goal is written for Shawnna that is paired appropriately with a quantitative scale:

47

FROM GOALS TO DATA AND BACK AGAIN

Goal: With regular modeling from siblings and adults during therapy sessions in the home, Shawnna will take on the role of another person in play with increasing frequency. Data question: How many times did Shawnna pretend to be someone else while engaged in pretend play with siblings and therapist? 0 1 2 3 4 5 >5

Here again the focus is on the emergence of a new behavior. The goal statement is qualified with phrases indicating where and when (“during therapy sessions in the home”), who (“siblings and adults”), and level of support (“with regular modeling”). Within those constraints, Shawnna has a constant opportunity to initiate the new behavior, so the team chooses a tally of occurrence. The quantifier in the goal statement (“with increasing frequency”) is made specific in the data question. How does this goal relate to intervention and change in Shawnna’s behavior? The team wants to see if they can expand Shawnna’s play skills with modeling so that she can play more easily with her sisters. They set out to work on this every session, modeling and encouraging. The individual collecting data looks for incidents in which Shawnna participates in or initiates some form of role-playing and tallies the occurrences. As Shawnna’s comfort with role-playing increases so will her tally. In contrast to the tally of occurrences, the second type of quantitative scale, percentage, is used for any desired behavioral change in which frequency of opportunity is also measurable. Percentage is calculated by dividing the number of times a behavior does occur by the number of opportunities in which it might have occurred. For example: Goal: With increasing frequency, Peter will respond by saying “Hi” when familiar people say “Hi” to him. Data question: What percentage of the time did Peter respond by saying “Hi” to familiar people who came to his home during his morning therapy session? How many people came to his home? ______ How many times did Peter say “Hi”? ______ Calculate # of greetings ÷ # of opportunities x 100 = ______%

or: Goal: Needing only a single verbal prompt, Shawnna will wait appropriately for her turn on the playground equipment. Data question: What percentage of the time was Shawnna able to wait for her turn at the playground, needing only a single verbal prompt?

48

WRITING MEASURABLE GOALS

# of opportunities: ______ # of times Shawnna waited: ______ Calculate # of waits ÷ # of opportunities x 100 = ______%

Be aware that using percentages as your method of measurement in marking progress can be problematic in many instances. We have observed too often that teams rely on taking a percentage measurement even when it doesn’t make sense. We want you to be familiar with the four serious problems that can be associated with collecting data in this way: 1.

With goals that focus on frequently occurring behaviors, it is unlikely that a therapist or teacher has the luxury of counting all of the opportunities for the behavior in order to calculate the percentage correctly. Each time the observer fails to note that the child missed an opportunity the percentage becomes a little less accurate. Under these circumstances the ratio of successes to opportunities becomes inflated; the child appears to be far more masterful than he or she truly is. Estimating percentages is a common practice that is almost guaranteed to produce inaccurate data. If your team needs to estimate a percentage because counting the child’s successes and dividing them by the number of opportunities for success is too cumbersome, then you should use another method of measuring progress.

2.

Percentages can also be problematic for infrequently occurring behaviors. If there are only one or two opportunities for the target behavior each time you work with the child, the child’s behavior will be easy to record but the percentage data will be too crude to be informative. Why? Consider this: if there is only one opportunity, the child either behaves as desired (receives 100%) or does not behave as desired (receives 0%). If there are two opportunities, the child can score 0%, 50%, or 100%. These figures simply do not give enough gradation to accurately represent the child’s behavior in a meaningful way because the large jump between the possible values does not correspond to large differences in the child’s mastery. As the number of opportunities grows, the use of percentages does become more appropriate…but only until the opportunities become too frequent to count, as explained in 1.

3.

If the number of opportunities fluctuates due to the nature of the developmental skill you are trying to teach, percentages may be misleading. Perhaps Shawnna got a 50% one day and a 40% the next 49

FROM GOALS TO DATA AND BACK AGAIN

day. On paper, this looks like she is regressing slightly. But perhaps she showed a certain new behavior once on the first day (out of two opportunities) and four times on the second day (out of ten opportunities). Even though the percentage dropped, we might want to consider this progress simply because Shawnna managed to show the behavior four times. That information was lost when you calculated the percentages. 4.

It is possible to write goals that try to compensate for problems 1–3 by targeting behaviors that can be measured frequently but not too frequently, and by making sure each session with the child has about the same number of opportunities. Such an approach means letting your data analysis method determine your intervention style. The premise of this book is that the decision of intervention style must come first and is independent of the decision to collect data. Once the commitment to a particular style of intervention has been made then data collection and analysis methods appropriate to that intervention can be learned and practiced.

Using a qualitative scale

A qualitative scale is used for rating behavior that exists on a continuum. Of all the methods we discuss in this chapter, this technique for collecting data on developmental progress can result in the most specific and useful data. Yet we have found that teachers, parents, and therapists rarely use it. Why is this method under-utilized? Perhaps teachers and therapists worry that only a scale that has been established independently can be valid. This is not correct. You know the particular child in question and you understand that development is a process. So put those two areas of knowledge together to devise a series of steps that goes from not being able to do something all the way to being able to do something. By creating a scale of steps that is individualized to a specific child, the data you collect will be more valuable than data collected on a general scale. The four examples that follow are intended to give a feel for how to use qualitative scales to measure progress: Example 1: Goal: With increasing frequency, Peter will match his behavior to the behavior of his peers. Data question: How often did Peter watch what his friends were doing and do it too?

50

WRITING MEASURABLE GOALS

Rarely, even when prompted 1 About half the time, prompted Most of the time, prompted A few times, independently Many times, independently

Example 2: Goal: Shawnna will become more comfortable with messy art activities, that involve using her hands, when they are presented to her in the classroom. Data question: Please rate Shawnna’s participation level with activities involving tactile experiences. Very resistant Somewhat resistant A little resistant Guarded participation Full participation

Example 3: Goal: Isaac will begin to engage in parallel play during “free play” at preschool. Data question: Did Isaac engage in parallel play? Isolative throughout “free play” During gross motor activities only During gross motor and sensory activities During all activities, including playing with trains

Example 4: Goal: With increasing frequency, Beth will learn to use words to describe her distress while completing homework and chores in the evening. Data question: For the majority of the time, how did Beth communicate distress? TANTRUM 1

CRYING

WHINING

VERBALLY

We hear you asking yourself, “Why is it all right to estimate a percentage (‘about half the time’) here when you just finished telling me it was bad practice?” Excellent question! The estimate asked for on this qualitative scale is a choice among a set of broad categories that can be judged accurately. Most adults can accurately distinguish between events that have occurred rarely, about half the time, and most of the time. This is a very different sort of judgment than distinguishing accurately if an event occurred 25% of the time or 30% of the time or 60% of the time. (We’ll explain the difference between qualitative scales and percentages in more technical terms in Chapter 6.)

51

FROM GOALS TO DATA AND BACK AGAIN

As the examples show, the more specific you are about what you expect in the way of progress, the more informative your data results will be. Most systems of measurement look at whether the child did or did not do something with little regard for the process of going from A to Z. If you take the time to figure out the steps that the child will take to accomplish a new skill, you educate everyone who uses the scale. Further, when you look back on the child’s progress and your team notes that the goal has not yet been achieved, a qualitative scale gives you considerable feedback about where the child has stopped progressing. A good team takes this information seriously and plans new strategies accordingly.

Ways of recording increasing or decreasing duration Frequency is not the only aspect of learning that is important. Even a child who shows progress in knowing how and when to use a skill may not participate in the activity or behavior for a long enough period to fully benefit from its rewards. For this child, the team may decide that it is time to focus on duration. When we want to see behavior demonstrated for either longer or shorter periods of time we need quantitative scales that measure duration. Such scales need to be paired with qualifying phrases that clearly ask for average, longest, or shortest instances of the target behavior. The following examples are presented to give you a feel for measuring progress in this way: Example 1: Goal: In the home environment, Peter will stay engaged with familiar adults for increasing lengths of time. Data question: On the average, how long was Peter able to stay engaged with you today? 2–4 min.

5–7 min.

8–10 min.

>10 min.

Example 2: Goal: As Shawnna becomes more social, periods of time in which she chooses to remain self-absorbed and self-contained will diminish. Data question: When given a choice to be alone, what was the longest time before Shawnna sought the company of her mother or therapist? >15 min.

~15 min.2

~10 min.

The symbol “~” means “about” or “approximately.”

2

52

~5 min.

<5min.

WRITING MEASURABLE GOALS

Example 3: Goal: Miguel will learn to tolerate the presence of a dog from a distance without growing anxious or needing to leave the area. Data question: What is the longest period of time that Miguel tolerated the presence of a dog from a distance? 1–5 min.

5–10 min.

10–15 min.

15–20 min.

<20 min.

When considering using increasing or decreasing duration as a system for measurement, you will need to consider increments of time for your scale. Are you looking for an increase or decrease in 2-minute increments? In 5-minute increments? In half-hour increments? This depends completely on the child and what you are measuring. In the case of Peter, 2-minute intervals were picked for his scale because interacting with others is hard for him and it is realistic to expect this to improve in small increments of time. On the other hand, Shawnna is already becoming more social, already demonstrating an interest in reconnecting with familiar people after isolating herself briefly. We expect her interest in isolative play to diminish a little quicker than Peter’s ability to stay engaged increases. Why? Because they are different children in different phases of treatment. Shawnna’s skills are already improving. Peter’s are just emerging. The time intervals in each case reflect the difference in our expectations for change for each child. Let’s think about this with some other examples. If you are working with a child who occasionally presses his or her eye when bored and you want this behavior to diminish as you teach new, more appropriate behaviors to manage boredom, you might measure increasing intervals of time between eye presses. For this child, the appropriate time intervals might be 10 minutes. You know perhaps that he easily goes 10–15 minutes without pressing his eye and as a measure of progress, you want to see the time between eye presses expand to 20 minutes, 30 minutes, and 40 minutes. For another child, however, who is very reactive to noise, you might want the child to tolerate being in the school cafeteria for a minute, then 2, then 3, with a goal of only 5 minutes. For that child, a 1-minute interval might be most appropriate. Just remember that the time intervals you pick will depend on the skill you are targeting and on the child’s individual profile.

Ways of recording an increasing range of behavior A child who participates in an activity for an age-appropriate period of time may, nevertheless, be restricting his or her range of behavior. The child who always wants to play the same game with trains, the child who will play in parallel with peers only at the art table, and the child who describes every event as having occurred yesterday (even when it happened months ago) have at least one thing 53

FROM GOALS TO DATA AND BACK AGAIN

in common. They are all ready to work on large-scale goals to increase the range of their behaviors. Measuring an increasing range of behavior is different from the other systems of measurement we’ve seen. In this type of system, we are interested in steadily increasing a child’s “vocabulary,” as shown in the two examples that follow. Example 1: Goal: During “free play” at school, Peter will expand his repertoire of play to include gross motor, construction/building, puzzles, art, music, and early pretend play. Data question: Please circle the types of activities Peter participated in today during free play: gross motor art

construction/building music

puzzles early pretend play

Peter needs to expand his repertoire of play. For example, he may ask for the computer every day and the teaching staff want to see him, over time, try all different kinds of activities. They don’t expect him to do all six activities every day but their goal is that he eventually participates in all six. So an appropriate way to measure this goal is with an accumulating list. If they noted only how many things Peter did on a given day, they would never be able to figure out if he was doing the same things or different things. By working to complete a list over time, Peter’s team can ensure that he is expanding his interests. In another example, we meet Shawnna again. Shawnna is beginning to understand time. Her vocabulary for time-related words, which indicates her growing knowledge, is expanding. By the end of the school year, Shawnna’s team wants to know that she can use a list of 12 time-related words and phrases. They don’t expect Shawnna to use all 12 words or phrases in a single conversation, but over time they expect her to have used each one of the targeted words or phrases in a way that reflects appropriate understanding of its meaning. They write this goal: Example 2: Goal: To demonstrate a growing understanding of time and sequence, Shawnna will spontaneously use at least 12 time markers in conversation with her speech and language therapist. Data question: Please circle the time markers Shawnna used appropriately and meaningfully today:

54

WRITING MEASURABLE GOALS

now morning today

later afternoon tomorrow

before after night yesterday a long time ago

soon

One of the important things to remember here is that the “vocabulary” to be mastered must be part of a developmentally sound sequence. If you want a child to acquire a set of skills, a set of concepts, a set of words, or a set of activities, choose the set carefully. You don’t want to throw together behaviors that are unrelated to the skill level of the child. Nor does it make sense to expect a child to acquire a range of behaviors that includes skills normally acquired at very different ages.

Ways of recording a decreasing prompt level Monitoring the level of support, as opposed to frequency or duration, tends to be an intermediate step in the process of goal acquisition. It is a useful way to measure progress after a skill has emerged in the child’s repertoire of behaviors but before this skill is independent or reliable. This measurement method is especially important in this population because children with autism who are taught intensively to produce certain behaviors can become dependent on artificial prompts, something often referred to as “prompt dependence.” Therapists, teachers, and parents can also become complacent in their use of prompts, forgetting to push or even nudge the child towards greater independence. It is important to remember, however, that if you use a decreasing need for prompts to measure progress, you must individualize the qualitative scale of prompts according to the needs of the child, the learning task, and the situation. Example 1: Goal: Miguel will use a visual schedule to complete a five-step task (setting the table). Data question: What level of prompt was needed to help Miguel use a visual schedule as a guide in setting the table? needed frequent verbal and gestural prompts needed a few verbal and gestural prompts needed a single verbal prompt needed simple gestural prompt independent

55

FROM GOALS TO DATA AND BACK AGAIN

Example 2: Goal: With a decreasing need for prompts, Peter will wave goodbye to familiar adults in multiple environments. Data question: What level of prompt was needed to help Peter wave goodbye to familiar adults? verbal and physical prompt (hand-over-hand) verbal with modeling verbal only no prompt needed

Example 3: Goal: Isaac will sit quietly “like a pretzel” at circle time with a decreasing need for reminders. Data question: How frequently did Isaac need to be reminded to sit quietly “like a pretzel” at circle time today? aide needed to sit behind, frequent prompting aide sat nearby, few prompts given aide faded, teacher prompted a few times teacher reminded whole group to sit still sat appropriately without reminders

Let’s think about these children for a minute and why decreasing the level of prompting was chosen as a method for measuring progress. Each child—Miguel, Peter, and Isaac—has learned a certain targeted skill in Phase 1 of his treatment for that goal. Miguel had never set the table before but with full support from his mother or therapist he began to complete this task. Now the team wants him to become more independent with this chore (they are moving the goal into Phase 2), so they reduce the level of human-driven prompts by helping him to use a visual schedule. Similarly, Peter has learned to wave goodbye with a lot of support (Phase 1). Eventually, we want him to be able to take the natural cue from a departing family member or friend without our additional support. We want him to respond consistently when someone says, “Bye bye, Peter” and waves. This is a Phase 2 goal. We are reducing the artificial prompts in order for Peter to use an environmental cue to wave goodbye. Finally, in Isaac’s case, we want him to sit correctly at circle-time with only the natural reminder, if needed, from his teacher. We want to decrease the support needed from the aide and increase first his reliance on the teacher for guidance (reliance shared by his classmates) and finally on himself to sit back down independently should he get out of the “pretzel” position. We are moving him from a stage of using the targeted skill with persistent external support such 56

WRITING MEASURABLE GOALS

as the aide (Phase 1) to a stage of natural support and independence (Phase 2). In helping children move from Phase 1 to Phase 2 of goal acquisition in order to master large-scale change, it is very helpful to use decreasing need for prompts as the method for measuring progress. Chart 3.3 in Appendix B presents the material we have covered to this point in the chapter in a concise format. You may find the chart useful as a reminder of the steps in writing measurable goals when designing intervention or explaining the process to others.

Considering goal-independent factors When we measure progress we expect changes in a child’s behavior to be related to the effectiveness of our intervention. But children with autism are often greatly affected by variables that compromise performance even though these variables have nothing to do with the particular skill being taught. A child who is not feeling well, for instance, may communicate less and become harder to engage. Parents have told us repeatedly that when their child is getting ill, he or she can be thrown off course for days before and for days following the illness. We know that changes in routine, changes in environment, changes in teachers, even minor changes in the appearance of teachers can alter a child’s responsiveness as well. The time of the day and the time of the year can be influential. We’ve known many children who have difficulty adjusting to the colder weather, indoor heat, and decreasing light of autumn and winter, or to the early warmth of spring. Understanding and monitoring these goal-independent factors can make an enormous difference in knowing how to interpret the data you collect. Some of the variables that affect performance may be unique to a specific child. You will have to learn these from parents and other team members and through your own history with the child. However, there are common factors to consider monitoring with any child. These include: · overall assessment: availability to engage/learn · who is doing intervention: familiar/unfamiliar · where intervention is done: familiar environment/new environment · day of the week (or day of the “cycle” in some schools) · time of day: morning versus afternoon versus evening · changes in routine · medication: type and time of last dose.

Tracking these kinds of typical goal-independent variables allows you to note, for instance, that the days that a desired behavior is suddenly absent are Tuesday and Thursday—days when the child has a different aide in school. They allow 57

FROM GOALS TO DATA AND BACK AGAIN

you to see, perhaps, that the child is unsuccessful every afternoon when medication is wearing off. Or that the desired behavior is lacking on Mondays when the child returns from a three-day weekend. This information is valuable to the team, allowing you to understand the flow of progress with a keener eye or compensate for differences in your data due to factors like fatigue, illness, and novelty. Just as important, these variables can give you information about what does negatively affect the child and interfere with learning. The team can use that information to devise new treatment goals and/or make adaptations to the interfering variables in order to produce more effective intervention. Exercise 3.1 provides you with an opportunity to practice the skills presented in this chapter. It can be found in Appendix C and in the file “Goal Workbook.doc” on the CD-ROM that accompanies this book.

Things to remember T Developmentally driven objectives can be written into goals that are

measurable.

T Intervention for children with ASD measures only one thing: change over

time.

T There are regular phases to the treatment process. Children begin to

learn a new skill in a single, sheltered setting but before they become fully accomplished they need to demonstrate the skill consistently in multiple environments. The goals we write reflect this process, targeting the emergence, consistency, and extension of the skill over time.

T Every goal needs to be qualified. To qualify the goal you must be

specific about the variables that surround the desired outcome, indicating where, when, and with whom the desired behavior is to be demonstrated as well as the level of support needed for success.

T Every goal needs to be quantified by picking a system of measurement

that fits the particular child and situation. In our experience, one of four systems is adequate to the task: increasing or decreasing frequency of behavior, increasing or decreasing duration of behavior, increasing the range of the behavior, or decreasing the need for assistance or prompts.

T There are three methods for measuring increasing/decreasing frequency:

58

1.

Tally of occurrence (a quantitative scale of measurement) should be used when the behavior could occur any or all of the time.

2.

Percentages (a quantitative scale) should be used only in situations where the number of opportunities can be easily detected and does not fluctuate greatly in different sessions.

WRITING MEASURABLE GOALS

3.

Qualitative scales are used to measure progress that moves through a series of carefully defined steps. A valid qualitative scale can be based on the steps we expect the individual child to pass through on the way from not being able to do something to being able to do it.

T When you want a certain behavior to occur for either longer or shorter

periods of time, measure duration using a quantitative time scale.

T When you want to see a steadily increasing repertoire of skills you need

to measure the child’s range of behavior. The data accumulates new items as they emerge over time.

T Measuring progress by the decreasing need for assistance uses a

qualitative scale of prompts that reflects the needs of the child, the learning task, and the situation. This technique is particularly useful in the stage between the emergence of a desired behavior and the consistent display of the desired behavior.

T In addition to the goal data you collect, it is important to monitor

variables such as time of day, medication, and changes in routine. Such goal-independent factors can help explain uneven and unexpected patterns in a child’s behavior.

59

Chapter 4

Collecting the Data

In the previous chapter, you learned to take a simple statement defining developmental progress and shape it into something measurable by qualifying it and quantifying it. You are now familiar with various methods for quantifying goals and you know the importance of noting goal-independent variables that might affect the data. You also know that there is a flow and pattern to the process of designing intervention with three phases of treatment. In this chapter we discuss something that we have only touched upon in the examples. We are going to show you how to formulate data questions and collect the data. This is not difficult, really, but it does require asking yourself or your team questions about the child’s target behaviors that easily elicit valid responses.

Data questions You have already seen a number of examples of data questions in the sections on quantifying goals. Let’s look at some of these again: Example 1: Goal: Isaac will sit quietly “like a pretzel” at circle time with a decreasing need for reminders. Data question: How frequently did Isaac need to be reminded to sit quietly “like a pretzel” at circle time today? aide needed to sit behind, frequent prompting aide sat nearby, few prompts given aide faded, teacher prompted a few times teacher reminded whole group to sit still sat appropriately without reminders

61

FROM GOALS TO DATA AND BACK AGAIN

Example 2: Goal: As Shawnna becomes more social, periods of time in which she chooses to remain self-absorbed and self-contained will diminish. Data question: When given a choice to be alone, what was the longest amount of time before Shawnna sought the company of her mother or therapist? 15 min.

~15 min.

~10 min.

~5 min.

<5min.

Example 3: Goal: With increasing frequency, Peter will respond by saying “Hi” when familiar people say “Hi” to him. Data question: What percentage of the time did Peter respond by saying “Hi” to familiar people who came to his home during his morning therapy session? How many people came to his home? ______ How many times did Peter say “Hi”? ______ Calculate # of greetings ÷ # of opportunities x 100 = ______%

Example 4: Goal: With increasing frequency, Peter will show pleasure by smiling when with his aide and a small group of peers during structured play. Data question: How many times in a ½ hour of structured play did Peter smile to indicate pleasure? 0 1 2 3 4 5 >5

Example 5: Goal: With a decreasing need for prompts, Peter will wave goodbye to familiar adults in multiple environments. Data question: What level of prompt was needed to help Peter wave goodbye to familiar adults? verbal and physical prompt (hand-over-hand) verbal with modeling verbal only no prompt needed

62

COLLECTING THE DATA

Example 6: Goal: During “free play” at school, Peter will expand his repertoire of play to include gross motor, construction/building, puzzles, art, music, and early pretend play. Data question: Please check the types of activities Peter participated in today during free play: gross motor art

construction/building music

puzzles early pretend play

These are all examples of goals that have been turned into data questions. Each data question reflects both the content defined in the goal and the measurement system selected as most appropriate or useful. Each of these data questions, when answered, gives the team information about progress on that particular goal.

Formatting the data sheet It is important to put data questions into a format that is both easy to read and easy to use. In order to do so we specify the range of answers to the data question, except when requesting a tally or percentage. In this way, the person collecting the data can simply check the applicable answer or fill in an appropriate number. Figure 4.1 shows an example of what a data collection sheet would look like for three of Peter’s goals using our format. Remember that the skills of the individual child and the complexity of a targeted behavior for that child will determine the range of answers. Whether it is intervals of time, levels of prompting, or steps that go from not being able to do something to being able to do that thing, the range of answers that defines a scale are designed for the individual child. For the person checking off answers, the scales act as reminders about what steps this child will take on the road to change. Therefore, the data sheet not only serves the purpose of providing the data collector with an easy system of data collection, it also serves as a reminder about where this child is going and how he or she will most likely get there. Although writing out the goals with the range of possible answers can be time consuming for the team initially, this method serves an important function. It establishes clear definitions of what goals are being measured and how the goals are being measured. In some cases it also indicates the process the child might follow on the way to goal completion. Having this information on hand is critical to consistent data collection either by a single individual across sessions, or by more than one observer working on the same goals with the same child but at different times. Sometimes the person doing the intervention has no opportunity to collect information on a particular targeted skill. If you suspect that there might be 63

FROM GOALS TO DATA AND BACK AGAIN

Child’s name: Peter Smith

Observer’s name:____________

Date:______

Time:______

Please check the types of activities Peter participated in today during free play:

o o o o o o

What percentage of the time did Peter respond by saying “Hi” to familiar people who came to his classroom during his morning preschool session?

How many people came to his classroom?______

gross motor construction/building puzzles art music early pretend play

How many times did Peter say “Hi”?______ Calculate: ______% (# of greetings ÷ # of opportunities x 100)

What level of prompt was needed to help Peter say goodbye to familiar adults?

o Verbal & physical prompt (hand-over-hand) o Verbal with modeling o Verbal only o No prompt needed Figure 4.1 A sample data collection sheet in an easy-to-use format

times when you or a member of your team cannot measure a particular goal, it is important to put that option on your data collection form. You might add a box labeled “Not applicable” or “No opportunity for observation.” This makes it possible to distinguish between times when there is no opportunity and times when the child is unable to perform. One of Shawnna’s goals is an appropriate example: When given a choice to be alone, what was o >15 min. the longest amount of time before Shawnna o ~15 min. sought the company of her mother or o ~10 min. therapist? o ~5 min. o <5 min. o No opportunity for observation

The treatment team wants to measure Shawnna’s growing sociability by monitoring how long she remains self-involved when no one is pushing her to engage. Yet the therapist’s job is to work intensely with Shawnna and she doesn’t often allow Shawnna to disengage for long periods of time. On a day 64

COLLECTING THE DATA

when intervention lasts right until the therapist needs to leave, there would be no opportunity to measure the length of time Shawnna stays disengaged by choice. By adding the box labeled “No opportunity for observation” the data explicitly reflects this decision by the therapist. The importance of this point is discussed further in Chapter 6. In summary, the format for data questions must be chosen to promote and facilitate accurate observation. It is important that your data collection sheets be both easy to read and easy to use. Unless you are taking a tally or collecting numbers to calculate a percentage, your job is to create a set of data questions that can be answered by checking a box or circling the appropriate answer. This requires careful forethought about exactly what should be observed so that the data you collect is an informative measure of the child’s progress to the goal. Exercise 4.1 reviews the features of Microsoft® Word that are needed to create data sheets like the ones seen throughout this book. For additional examples of data sheets designed for individual children in different settings, see Appendix B.

Frequency of data collection Here is a general rule for deciding how often to collect data: more is better. With all children in general, and children with ASD in particular, it is essential that you do not base a conclusion about developmental progress on sporadic data. We all know that almost anything can throw off a child’s performance. Therefore, it is best to track progress steadily and consistently. Some teams with whom we’ve worked see a child daily but collect data only once a week. This system can lead to conclusions about progress that may not be valid. If, for instance, you work with a child daily but only measure progress every Friday, you might be collecting numbers that reflect accumulated fatigue or boredom for the task at hand. If you measure the progress on Wednesday when the normal routine for the child has been interrupted (perhaps Mom goes to work and a babysitter gets the child up and ready for school on those days), your numbers may be lower than usual and not accurately reflect the child’s skill level. The only way to avoid these problems is to take daily data and monitor for goal-independent variables so that you have a way of understanding the ups and downs in the child’s performance. Some teams with whom we’ve worked see a child daily in school but collect data only on Mondays, Wednesdays, and Fridays. Although this system seems to work fairly well (certainly better than a once-a-week system), it leaves the team little knowledge about Tuesdays and Thursdays. They could be missing some valuable piece of information that is only available on one of these other days, as shown in the following example:

65

FROM GOALS TO DATA AND BACK AGAIN

Years ago, we worked with a little boy named Jack whose school-based team tracked his progress on a daily basis by rating him on a scale of 1–5 (poor to excellent) for responsiveness to every subject of the day. What we noted within a few months of school was that his numbers looked significantly lower on Mondays and significantly higher on Thursdays than on the other days of the week. Why? Examining the data, the team discovered two things. On Mondays, following the weekend, Jack was very tired. It turned out that his parents regularly allowed Jack to stay up late on Friday and Saturday nights, sleeping late the following mornings. But on Sunday night, Jack always had difficulty falling asleep at the usual time for a school night. He would repeatedly get out of bed and often didn’t fall asleep until midnight. It was a struggle for his mother to get him up on Monday mornings and he was, the team decided, noticeably tired in school. As soon as the parents realized that they needed to keep Jack on a consistent schedule regardless of whether it was the weekend or not, his performance at school on Mondays improved. On Thursdays, Jack had music therapy as the first school activity in the morning. This session always appeared to calm and focus Jack. The team realized, through their data system, that every Thursday, all day long, Jack’s response to the learning opportunities at school was significantly better than on other days. The team speculated that music therapy exercises readied Jack for the school day and improved overall performance. In response, they were able to create a music period for Jack each morning. The period was designed by the music therapist but carried out by the regular staff. The team’s theory that music experiences readied Jack for school was proven correct, and Jack’s daily overall performance improved with his new schedule.

If Jack’s school team had collected data only on Mondays, Wednesdays and Fridays, they might have discovered and remedied the fatigue issue but they would have missed the relationship between Jack’s performance and his participation in music therapy. Every time you work with a child you have the opportunity to collect data. DO IT! If you have created data collection sheets that are easy to use the task will not be difficult or overwhelming. And if you have created data collection sheets that measure progress accurately the potential rewards for you and the child are enormous.

66

COLLECTING THE DATA

Special suggestions for teachers It is particularly difficult, teachers tell us, to collect data systematically and frequently for multiple children in a classroom. Reorganizing your data collection to be activity-oriented rather than child-oriented can help. For example, if you know that a certain activity is a good setting for data collection, aides or teachers can observe and take data on a variety of goals for a variety of children during that one activity or class period. To do this effectively, you need to have the different children’s goals pulled together on a single sheet,

Opening exercises Date:______

Observer:____________

Student

Data questions

Data 0 1 2 3 4 5 >5

Tanika

How many times did Tanika need to be reminded to remain in her seat?

Joey

Using fidget toys and adaptive o Needed frequent verbal prompts seating, was Joey able to keep o Needed minimal verbal prompts his hands quiet? o Needed visual prompt only o Hands were quiet

Juan

When responding to a question, what percentage of the time did Juan raise his hand instead of calling out the answer?

# of responses: _______________ # of times Juan raised hand: _______________________ Calculate: ______% (# hands up ÷ # responses x 100)

Figure 4.2 A sample data collection sheet for multiple children working at the same time on different goals

Date:______

Teacher:____________

Seatwork goal: Using a visual schedule, the following children will complete the targeted number of tasks independently.

Student

Target # of tasks

Harry

3

Cameron

6

Aishia

4

Tasks complete

Figure 4.3 A sample data collection sheet for multiple children working at the same time on the same goal

67

FROM GOALS TO DATA AND BACK AGAIN

as in Figure 4.2. The observations can then be transferred to each child’s Excel worksheet at a later time. Alternatively, you might design an activity that specifically targets the same goal in a number of children. Suppose, for example, that you want to teach all or most of your students how to complete tasks independently using a visual schedule. It then makes sense to set up time each day for independent work during which children complete tasks using visual support and you collect data using a sheet like the one in Figure 4.3. Once again, after the data has been taken you will enter the observations into each child’s separate Excel worksheet. Creating a sheet to take data for multiple children at one time simplifies the process of collecting the data regularly and requires a minimum of paper shuffling. It requires that you think about and take advantage of certain times of the day when the goals are actively being addressed and/or when the child has multiple opportunities to perform the targeted skills.

Monitoring goal-independent factors Each child has a different set of variables that might affect the steady display of progress. Some children might be thrown off by changes in routine, some by different times of the day, some by illness, and some even by weather or sunlight. When your team identifies a set of circumstances that might affect the child, it is important to begin to collect information about these factors along with the data 1 you collect concerning progress on the goals. How do you do this? You simply add a set of information-gathering questions at the beginning of your data collection sheet. For example, if Peter was a child who had significant ups and downs in his availability for social experience, you might want a global assessment of his availability each day you worked with him. Was it a day in which he was easily engaged? Was it a day in which he avoided interaction from the moment you came until the moment you left? Or was it a more typical day for Peter, in which he was a mixture of avoidant and sociable? Once you are tracking this variable you can consider its role in Peter’s progress. He may have achieved the goal in question but cannot retrieve the targeted skill when he is socially distancing himself. When you notice these circumstances your job changes to address the difficult days, rather than the average days. Figure 4.4 shows a data collection sheet for Peter, with the goal-independent factors marked by an asterisk (*).

We discuss some ways to uncover such factors in Chapter 9. Here we are interested in how you monitor variables that you have already uncovered or that you know to be likely candidates based on parent report, theory, or experience with other children.

1

68

COLLECTING THE DATA

Child’s name: Peter Smith

*Observer’s name:____________

*Date:______

*Time:______ 1 2 3 4 5

*On a scale of 1 to 5 (avoidant to sociable), please rate Peter’s overall mood for the day. Please check the types of activities Peter participated in today during free play.

o o o o o o

What percentage of the time did Peter respond by saying “Hi” to familiar people who came to his classroom during his morning preschool session?

How many people came to his classroom?______

What level of prompt was needed to help Peter say goodbye to familiar adults?

o o o o

gross motor construction/building puzzles art music early pretend play

How many times did Peter say “Hi”?_____ Calculate: ______% (# of greetings ÷ # of opportunities x 100) verbal & physical prompt (hand-over-hand) verbal with modeling verbal only no prompt needed

Figure 4.4 A sample data collection sheet showing goal-independent factors marked by an asterisk (*)

In Figure 4.4 we record an overall assessment of the day (defined here in terms of availability for social experience), who is doing the intervention and collecting the data, the day of the week, and the time of the day. In addition, we might be interested in where the intervention is taking place, whether or not there were any changes in routine that day, and whether the child was on medication. The sheet would then look like Figure 4.5.

69

FROM GOALS TO DATA AND BACK AGAIN

Child’s name: Peter Smith

*Observer’s name:____________

*Date:______

*Time:______ 1 2 3 4 5

* On a scale of 1 to 5 (avoidant to sociable), please rate Peter’s overall mood for the day. *Where did intervention take place?

o home o school o other:

* Were there any changes in routine today?

o yes o no If yes, please explain:______

* Did Peter take Ritalin today?

o yes o no What time did he take it?______

Please check the types of activities Peter participated in today during free play.

o o o o o o

What percentage of the time did Peter respond by saying “Hi” to familiar people who came to his classroom during his morning preschool session?

How many people came to his classroom?______

What level of prompt was needed to help Peter say good-bye to familiar adults?

o o o o

gross motor construction/building puzzles art music early pretend play

How many times did Peter say “Hi”? ____ Calculate______% (# of greetings ÷ # of opportunities x 100) verbal & physical (hand-over-hand) verbal with modeling verbal only no prompt needed

Figure 4.5 The data collection sheet from Figure 4.2 with additional goal-independent factors (*)

70

COLLECTING THE DATA

Figure 4.5 is an example of a data collection sheet that has seven common variables accounted for: · overall assessment of the day (defined in this case in terms of availability

for social experience)

· who is doing the intervention and collecting the data (here it is the same

person but not necessarily)

· day of the week (or day of the “cycle” in some schools) · time of day · location of intervention · possible changes in routine · medication.

Be aware that these are only common variables. Every child is unique, and because of this you might need to keep track of any number of individually designed variables that could be affecting this child’s performance. Goal-independent factors may be very useful or only occasionally useful (we discuss this further in Chapter 9). If you don’t track the variables for each session, however, you will find that it is extremely difficult to recreate them accurately in retrospect. It is far better to have the information available if and when you need it.

Things to remember In order to collect data, you must turn your carefully qualified and quantified goals into data questions. These data questions must then be posed to the parent, therapist, teacher, or other professional on a regular basis. T Data are the answers to the data questions. T Data questions reflect the carefully written goal. T The more specific you are in what you are measuring, the more

informative and useful the data.

T The more frequently you collect data, the more accurate your picture of

progress over time. More is better.

T The team is more likely to collect meaningful data if the data collection

sheet is easy to read and easy to use. Even though the data sheet is time-consuming to design, this “up-front” effort will be rewarded in the long run.

T In addition to monitoring progress on each goal, it is important to

monitor goal-independent variables that may affect treatment outcomes. 71

Chapter 5

Putting It All Together — Joey, Tyler, and Mai Lin

It’s time to put together what we’ve learned so far. Let’s look at three different children, whom we will call Joey, Tyler, and Mai Lin. For each child we’ll tell you a bit about where he or she is developmentally, then work with you to figure out a set of goals. We’ll shape these goals into something meaningful and measurable, design the needed data questions, and finally create a data collection sheet. By the time you’ve worked through these three cases, you’ll be ready to move on to data analysis. Although this chapter gives you a chance to practice goal writing and creating data questions from those goals to measure change over time, it does not discuss the intervention that actually creates the change. That discussion is for a different book! In this book, we want to teach you how to measure and document the results of your intervention. Don’t lose sight, however, of the fact that you can only measure progress if you have progress to measure. Your intervention is the key to the process. Before you meet Joey, Tyler, and Mai Lin, we suggest that you make a copy of all the worksheets for the exercises in this chapter. These can be photocopied from Appendix C (Exercises 5.1 through 5.8) or printed from the file “Goal Workbook.doc” on the CD-ROM that accompanies this book. If you have those sheets readily available, participation in this chapter will be easy.

Joey Joey’s parents describe their 3½-year-old son as a child who seems to have little need to communicate with either of them. He is happy just “doing his own thing.” Joey eats what they give him, he goes to bed without a problem, he tolerates all the daily routines, and when he is left alone, he appears content to spin the wheels on his large collection of cars and trains, or spin his collection of shiny items on the hallway floor. The only thing Joey requests regularly is a drink. He goes into the kitchen and gets 73

FROM GOALS TO DATA AND BACK AGAIN

his cup, which his parents usually leave in a spot that he can reach. If he can’t reach it for some reason, he whines and whimpers. The parents said that Joey would not come and find them for help. Instead, when they hear him whine or whimper, they come in to give him a drink. They wish that Joey would ask them for help more often. “I used to be proud of how independent my second child was,” said Joey’s mother, “but now I see that it isn’t so good.” Joey’s parents wonder if Joey will ever actually speak because he doesn’t even babble the way their older daughter did as a toddler. All he does, according to his mother, is make vowel sounds. His father pointed out that he rarely closes his mouth or lips except when he is eating and even then, it doesn’t look quite right to him. However, recently they have noticed that Joey is beginning to watch their mouths when they are making funny sounds and is trying to purse his lips to blow bubbles. They want to encourage this because they know he needs to use the muscles of his mouth and lips if he is ever to use speech. Joey likes to look at books. Although he mostly flips rapidly through the pages, he does pause and examine any pictures of trains or cars, and recently seems interested in pictures of food. His parents wonder if a picture exchange system would help him communicate while he is learning to talk. Joey loves physical activities, his parents report, and will stay engaged with them, even giving them sustained eye contact, when they push him in the swing. He seems happy to swing for hours, they say. He also loves it when they sit down with him with his collection of shiny objects. He sits and watches for relatively long periods of time if they just keep spinning the objects on the floor. His parents wonder how they could use these preferred activities to encourage communication. Joey’s parents say that more than anything they want to help Joey find a reason to communicate with them and a method that is within his current abilities. They feel strongly that they want to help Joey “make his mark upon the world.” They want to teach him that it is good to express himself and that they will listen.

Given this brief case history, what do you think might be a set of five goals for Joey that reflects what his parents have highlighted as most relevant to them and to him? To narrow the scope of this task, let’s focus all five goals on communication. Take a few minutes and jot down some ideas using the workbook page Exercise 5.1 (from Appendix C). Return here after completing Exercise 5.1. There are many different possibilities for goals for Joey. One group of professionals who looked at this scenario came up with the following list: 74

PUTTING IT ALL TOGETHER – JOEY, TYLER, AND MAI LIN

· To make his mark upon the world and to feel successful in

communicating, Joey will indicate preference when offered the choice of two options.

· To begin to use pictures to make requests, Joey will associate photographs

with preferred items.

· To encourage meaningful vocalization, Joey will attempt to say “go!” after

being cued with “Ready, set…”

· To develop more effective communication through contact gesturing,

Joey will begin to pull his parents to his object of desire.

· To encourage oral imitation as a precursor to oral language, Joey will

imitate mouth movements with increasing frequency.

As a set of goals, these are a good start. They do, however, need to be qualified and quantified further. Using the words in italic as the actual goals, try to shape the goals so that they become measurable (Exercise 5.2). Return here after completing Exercise 5.2. Here is one way to do it: 1. During the same 2-hour period each day, Joey will indicate preference 75% of the time when offered the choice of food or an object to hold or play with. 2. With increasing frequency, Joey will match photographs to preferred items when given the opportunity. 3. After hearing the phrase modeled twice, Joey will attempt to say “go!” after hearing “Ready, set…” during a familiar and preferred activity. 4. With a decreasing need for assistance, Joey will begin to pull his parents and/or familiar adults to his object of desire. 5. Joey will imitate the mouth movements of familiar adults with increasing frequency.

Now we have goals that clearly lend themselves to being measured over time with accuracy and precision. From these goals we compose our data questions. How would you do this? Try Exercise 5.3. Return here after completing Exercise 5.3. 75

FROM GOALS TO DATA AND BACK AGAIN

The team of professionals like you who originally identified Joey’s goals produced the data sheet below. Name: Joey Smith Name of therapist:____________ Date:______ Time:______ Location of intervention: “ home “ center On a scale of 1 to 5 (avoidant to sociable), please rate Joey’s overall mood for the day. 1 2 3 4 5

76

1. When offered a choice of food, or object with which to play or hold, was Joey able to indicate preference? (Use same 2-hour period each day when choices are routinely offered)

# of opportunities

# of successes

2. When given an opportunity, was Joey able to match a photograph to a preferred item?

o o o o

3. After hearing the language modeled two times, was Joey able to fill in the word approximation “go” after “Ready, set…” to start a preferred activity?

# of opportunities

4. What level of prompt was needed to get Joey to pull an adult to an object of desire?

o multiple physical and verbal prompts o a verbal prompt (“Show me!”) with outstretched hand o independently (a few times) o independently (consistently)

5. Did Joey attempt to imitate your mouth movements?

o o o o

Calculate:______ % (# of successes ÷ # of opportunities x 100) not today, even with modeling matched after modeling independently matched 1 or 2 times independently matched multiple times # of successes

Calculate:______% (# of successes ÷ # of opportunities x 100)

no attempts noted growing interest in mouth movements a few attempts multiple attempts

PUTTING IT ALL TOGETHER – JOEY, TYLER, AND MAI LIN

What do you think? How do these data questions differ from the ones you produced? Remember that the ones we’ve shown are not the only possible questions. There are many ways to ask valid questions. What’s important is that the questions measure a skill or behavior that you believe to be a good indicator of progress on the underlying goal. Joey is an example of a young child who has very little ability to communicate. He is clearly still in Phase 1 of treatment where his needs are intense and the scope of treatment must be narrowly focused on Joey, his family, and his home. Let’s look at Tyler next. He is a little older and has more communication skills.

Tyler Tyler is a 5-year-old boy who lives in a family where he is nestled between a 6-year-old sister and 4-year-old twin brothers. Tyler’s health is excellent except for allergies that are easily controlled with medication. The family came to us for advice on how to help Tyler, diagnosed recently with Pervasive Developmental Disorder (PDD), communicate more effectively. His parents describe him as being an integral part of their busy family life. “There is never a moment for him to be alone,” his mother said, “and someone is always talking with him.” Yet even within an environment rich with language and intense in social demands, Tyler’s social language is not developing normally or rapidly. One of the things that bothers Tyler’s parents the most is that he continually seems to talk to himself. Tyler recites parts of videos he loves and has memorized, he repeats conversations he has heard days before, and he likes to spell long words over and over again. Right in the middle of a conversation, Tyler will begin to say something totally out of context that has little or no meaning to his conversational partner. His parents think that this tendency towards private reference is a problem because communication should be a dialogue, they say, not a monologue. Another problem Tyler’s parents highlight is that when Tyler does talk to his family members, it isn’t very effective. He often fails to get his siblings’ attention before he speaks to them so his sister and brothers appear to ignore him much of the time. When Tyler comes to his parents to request something, he almost never looks at them, making it hard to know what he is requesting or to whom he is speaking. If they question him to find out the meaning of his communication or to require more information from him, Tyler appears to have difficulty responding. “It’s almost as if he shuts down whenever we start questioning him,” his mother admitted. His father notes that sometimes when he is busy fixing something and not paying attention to whether or not Tyler is talking, that’s when his son seems most willing to engage in verbal exchange. Finally, Tyler seems either overly passive (so different from their other children!) or easily upset. They feel that he needs to learn to effectively say “no” and “I don’t want that” and “not now” just like their other children. 77

FROM GOALS TO DATA AND BACK AGAIN

They feel that this would help Tyler control what happens to him and thus increase his level of participation.

Begin by writing down a few of your ideas for goals concerning Tyler’s communication needs. Return here after completing Exercise 5.4. The goals that were identified by Tyler’s parents with our help were the following: · To help Tyler sustain connection to his social world, incidents of private

reference occurring in the midst of social exchange will decrease in frequency.

· To help him communicate effectively with his siblings, Tyler will learn to

secure his family’s attention before talking.

· To help him effectively make requests, Tyler will look at his communicative

partner as he is talking, or right before he begins to talk, with increasing frequency.

· To help him sustain conversation, Tyler will be engaged in a daily verbal

exchange of increasing length in which the adult partner avoids questioning and quizzing, and relies instead on commenting, narrating, rephrasing, and expressing opinions.

· To help him maintain a more even temperament, Tyler will learn to

effectively and productively indicate disapproval or protest.

Now, just as you did for Joey, try to make these goals more amenable to meaningful data collection. Using the words in italic as the actual goals, shape the goals further so that they become measurable. In this case, write goals assuming that the family will be doing all of the intervention. Return here after completing Exercise 5.5. The original goals just needed a little tweaking. See if your tweaking looks something like ours. It’s okay if it doesn’t—there are a number of ways to qualify and quantify these goals so that progress can be measured.

78

PUTTING IT ALL TOGETHER – JOEY, TYLER, AND MAI LIN

1. During the 1-hour period of time when the family plays together each evening, incidents of private reference occurring in the midst of social exchange will decrease in frequency. 2. During the 1-hour period of time when the family plays together each evening, Tyler will learn to secure his family’s attention before talking at least half of the time. 3. Without needing to be reminded, Tyler will consistently look at his communicative partner as he is requesting food or drink. 4. With an increasing number of verbal exchanges, Tyler will have one conversation a day in which the adults use a facilitative approach (avoiding questioning and quizzing, and relying on commenting, narrating, rephrasing, and expressing opinions). 5. Without being coached by family members, Tyler will learn to effectively and productively indicate disapproval or protest.

What do the corresponding data questions look like? Create your version by doing Exercise 5.6. Return here after completing Exercise 5.6. Here’s our version. We designed this data sheet in conjunction with Tyler’s parents to mark his progress in communication skills within the family. At this time in Tyler’s life, he is both learning new skills (goals 4 and 5), extending skills to new environments (goals 1 and 2) and developing independence (goal 3). His goals span all three phases of treatment. This is not an uncommon picture for children who are progressing in their skill development.

79

FROM GOALS TO DATA AND BACK AGAIN

Name: Tyler Jones

Family member taking data:

Date:______

____________

Time:______ Did Tyler have his allergy medication today?

Yes

No

1 2 3 4 5

On a scale of 1 to 5 (self-absorbed to socially connected), please rate Tyler’s overall mood for the day. 1. Over a 1-hour period before bedtime when the family plays together, how many times did Tyler lapse into private reference?

o o o o

2. Over a 1-hour period before bedtime when the family plays together, how often did Tyler secure his listener’s attention before talking to him or her?

o rarely (needed prompts to do so) o only a few times o about half the time o more than half the time

frequently a few times once or twice not at all

3. On the average, when asking for food or drink, o needed multiple prompts o needed a single prompt did Tyler spontaneously look at you when making a request? o a few times without prompts o consistently without prompts 4. During a single conversation in which a facilitative approach rather than a directive approach was used, how many verbal exchanges were achieved? (tally) 5. Was Tyler able to effectively indicate disapproval/protest?

80

o remained passive or got upset without explanation o indicated disapproval with coaching from family o beginning to protest or say “no!” independently o consistently indicated disapproval when appropriate

PUTTING IT ALL TOGETHER – JOEY, TYLER, AND MAI LIN

Now let’s move on to our final example, Mai Lin.

Mai Lin Mai Lin is an 8-year-old with Asperger Syndrome. She is a gifted musician and an excellent math student. Mai Lin is enrolled in a typical 2nd grade class where she is doing very well. The teachers feel, however, that Mai Lin needs to refine her communication skills so that she can talk more effectively to the people around her. One thing that concerns the teaching staff is that with unfamiliar people, Mai Lin seems to forget the body language and voice tone that normally accompany conversation. She turns away from the listener while talking, talks in an inappropriately quiet or low tone, and her inflection practically disappears. This makes it difficult for Mai Lin to interact with other children at lunch and recess. Mai Lin also has difficulty making herself heard if her initial attempt isn’t successful. If a peer says, “What?” Mai Lin will repeat her words identically without making herself louder or clearer. If the friend says, “What?” again, Mai Lin will continue to repeat her statement or request like “a broken record.” The teaching staff feels that Mai Lin’s “poor mechanics” are impacting her ability to maintain social contact with her friends. In the middle of conversations, Mai Lin tends to suddenly switch topics to music, math, or her current favorite subject. The teachers think she may do this because she is more skilled talking about preferred topics. They would like Mai Lin to become more comfortable sustaining conversation on a range of common topics. Finally, Mai Lin needs to be prompted to make a request in the form of a question. She tends to simply make statements, such as “I can have a drink, please.” Her teachers want Mai Lin to use a question format independently for her requests.

When you write goals this time (Exercise 5.7) try to qualify and quantify them from the outset. For the sake of this exercise, let’s assume that all of Mai Lin’s intervention will occur in the school setting and will focus on communication. Return here after completing Exercise 5.7. Your goals may be different from the ones below but we do need to have a common set of goals for the rest of the exercise. So let’s use the following:

81

FROM GOALS TO DATA AND BACK AGAIN

1. With a decreasing need for reminders, Mai Lin will appropriately face unfamiliar people when she addresses them. 2. Needing no more than a single prompt, Mai Lin will increasingly use appropriate volume and inflection when speaking to unfamiliar people. 3. With consistency, Mai Lin will effectively repair her communicative attempts after a peer says “What?” or indicates in another way that Mai Lin’s words were not understood. 4. Once a day, Mai Lin will sustain general conversation for at least 5 minutes without attempting to suddenly introduce a preferred topic. 5. Needing only an expectant pause as a reminder, Mai Lin will make requests using a question format rather than a statement at least 75% of the time.

Next, design the data questions, making decisions about how to measure Mai Lin’s progress. Return here after completing Exercise 5.8. Once again, your data collection sheet may be different from the one we are going to suggest. Compare and see what you think. Did you remember to add goal-independent factors to your data sheet? Mai Lin is an accomplished young girl. We are mostly “fine-tuning” her skills so that her communication becomes more socially cognizant of and responsive to her partner. Mai Lin is now primarily in Phase 3, learning to extend her skills to multiple people in multiple environments. Note that as she is learning to generalize her skills to the school environment and to multiple peers, Mai Lin might need assistance again (something associated with earlier phases of treatment) until she has learned to generalize her social communication.

82

PUTTING IT ALL TOGETHER – JOEY, TYLER, AND MAI LIN

Name: Mai Lin Wu

Teacher/Aide’s name:

Date:______ Time: ______

____________

Were there any unexpected changes in the school routine today?

Yes

No

If yes, please explain: 1. How much prompting was needed to help Mai Lin appropriately face new or unfamiliar people when speaking to them?

o unable to face unfamiliar person, even with prompt

o o o o

needed multiple prompts needed a single verbal prompt needed a single physical prompt no prompting needed

o no opportunity for observation 2. After a single prompt, if necessary, did Mai Lin use appropriate volume and inflection when speaking to unfamiliar people?

o o o o o

continued in quiet monotone even after prompt appropriate volume/inflection, 1 prompt, 1 or 2 x appropriate volume/inflection, 1 prompt, consistent appropriate volume/inflection, no prompts, 1 or 2 x appropriate volume/inflection, no prompts, consistent

o no opportunity for observation 3. Did Mai Lin effectively repair her communicative attempts after a peer said, “What?” or indicated in another way that Mai Lin was not understood?

o o o o o

skill emerging with assistance from adults skill emerging with multiple cues from peer skill emerging with 1 additional cue from peer repairs attempts herself about half the time consistently repairs communicative attempts

o no opportunity for observation 4. During “talk time” in which Mai Lin has to talk to a teacher for 5 minutes on a variety of common subjects, how many times did Mai Lin attempt to switch the conversation to a preferred topic? 5. What percentage of the time did Mai Lin make requests in the form of a question—as opposed to a statement—needing only a facial cue (expectant pause) as a reminder?

tally of occurrence: # of times: ______

o no opportunity for observation # of requests made:______ # made in correct form or repaired:______ Calculate: ______%

(# of successes ÷ # of opportunities x 100)

o no opportunity for observation

83

FROM GOALS TO DATA AND BACK AGAIN

Things to remember T The process of figuring out appropriate goals, shaping the goals into

measurable objectives, and designing data collection systems gets easier and easier the more you do it.

T We deliberately made these examples complex in order to give you a

chance to see a range of problems and a range of solutions. Many of the goals you target for your own clients or students will be less complex and the process of collecting data will be correspondingly simpler.

T The more information you provide to the person responsible for

collecting the data about what he or she is measuring and how to do it, the better the data will be.

T Even though your data sheets may seem lengthy or wordy, detail and

specificity is what narrows the focus of your intervention and makes it measurable.

84

Chapter 6

From Data Collection to Data Analysis

We collect data to see if there have been measurable and reliable changes in a child’s behavior. We assume that when such changes exist they are a result of intervention and indicate some deeper changes in the child’s knowledge or emotional repertoire. How do we know when the data indicates that there has been change? And how do we know whether that change is reliable? Answering these questions is the topic of this and the following four chapters. It is what data analysis is all about. Before we jump into the particulars, however, we want to remind you of an important difference between this book and other publications in which data analysis is discussed. Some of you may be familiar with terms like t-test, F-test, ANOVA, and p-value from reading journal articles, taking courses in statistics, or being part of an academic research study. You will not find these concepts explained or used in this book because most of the statistical measures you may be familiar with from academic research simply do not apply under the conditions created by real intervention. We don’t want you to change how you do intervention in order to make it appropriate to use these tests. We do want you to use the tests that are appropriate for the kind of intervention you do, even if those tests are less mathematically rigorous. That said, let’s begin.

Types of data Chapters 3 through 5 took you through the steps involved in data collection, the process that turns a desire for behavioral change stated in general terms into a set of questions that can be answered by observing a child’s actions. Data analysis is the process that turns those observations back into meaningful statements about changes in the child’s behavior over a particular period of intervention. An unfilled data collection sheet represents the set of measurements we might make. Each time we work with a child, we choose from among these possibilities, observing the child’s actual performance in a particular situation and recording what we see and hear. So, when analysis begins, the data we have 85

FROM GOALS TO DATA AND BACK AGAIN

collected is spread across many sheets, reflecting behavior across many sessions. There might be many instances of the same sheet or there might be many instances of different sheets, as when different teachers are responsible for tracking different subsets of the child’s goals. Regardless, the same process must be applied to each sheet. We use the sample sheet in Figure 6.1 as the basis for our discussion in this section. If you look at Figure 6.1 you will see the record of Joey’s performance for five goals on January 3, 2001. Each thing that has been circled, checked, crossed out, or written in is called an observation, value, score, or data point (we use these terms interchangeably in what follows to help you get used to hearing all four). Groups of values are called a data set. Taken together, the observations in Figure 6.1 constitute a natural data set—in particular, the set of scores for Joey on January 3, 2001. But there is a second kind of data set that is more central to analysis—the data set that combines the observations for an individual item over time. Each item on a data collection sheet produces such a data set and each set has two properties. These properties determine the kind of analyses we can perform on the data set for that item. You already know the first property —whether the values represent goal data or goal-independent factor data. We’ve identified this property for each item on Joey’s data sheet for you. As the labels at the left of the figure show, Joey’s sheet contains five items that produce goal data (goals 1–5) and five items that produce factor data (the therapist’s name, the date, time, and location of intervention, and a number representing Joey’s overall mood that session). The second property of each item’s data set is its data type. At the beginning of analysis it is fairly easy to determine the type of data produced by each item because we can see all the information we need to decide. On Joey’s sheet we can see, for example, that some of the items produce data points that are words or phrases, some produce numbers from a scale, and some produce percentages. These simple characteristics are important clues to the data’s type. As analysis continues, however, we must translate all the data into numbers. Once we’ve finished the translation process, all the data will look the same and all the telltale clues will have disappeared. So, it’s important to take the time to identify the data’s type now, before we obscure the information by translation. There are three types of data on our collection sheets. Let’s examine them in order, from simplest to most complex. Look at the therapist’s name and the location of intervention on Joey’s sheet. These items produce the simplest sort of data, the kind that permits only the simplest mathematical functions. What sorts of things can we do with the data sets produced from these items? We can count how many observations there are in the data set and, if there are different possible values, how many observations we collected for each value. For example, when we look through the sheets for a particular time period we might find that Joey had eight sessions in 86

FROM DATA COLLECTION TO DATA ANALYSIS

Name: Joey Smith Factor data

Categorical (nominal) data

Name of therapist: Sasha Date: 1/3/01 Time: 3:00-5:00 Location of intervention:

 home

o center

On a scale of 1 to 5 (avoidant to sociable), please rate Joey’s overall mood for the day.

Interval data

1 2 3 4 5 Goal data

1. When offered a choice of food, or object with which to play or hold, was Joey able to indicate preference?

# of opportunities

2. When given an opportunity, was Joey able to match a photograph to a preferred item?

o o o o

3. After hearing the language modeled two times, was Joey able to fill in the word approximation “go” after “Ready, set…” to start a preferred activity?

# of opportunities

|||||||||

# of successes |||

Calculate: 33% (# of successes ÷ # of opportunities x 100) not today, even with modeling matched after modeling independently matched 1 or 2 times independently matched multiple times

||||||

# of successes |||

Calculate: 50% (# of successes ÷ # of opportunities x 100)

o multiple physical and verbal prompts 4. What level of prompt was needed to get Joey to pull an adult to an object of o a verbal prompt (“Show me!”) with outstretched hand desire? ý independently (a few times) o independently (consistently) 5. Did Joey attempt to imitate your mouth movements?

o ý o o

Ordinal data

no attempts noted growing interest in mouth movements a few attempts multiple attempts

Figure 6.1 Every item on a data collection sheet has two independent properties. We note the first property—whether it’s goal data or factor data—at the left. The second property—whether the item results in a categorical, ordinal, or interval data set—is labeled on the right. Both properties are used to determine the appropriate statistical tests to apply to the data

87

FROM GOALS TO DATA AND BACK AGAIN

total with six home visits (6/8 or 75%) and two center visits (2/8 or 25%). As you can see in Figure 6.1, data that captures only the categories or names associated with an observation is called categorical or nominal data. Although not usually very interesting by itself, categorical data may, nevertheless, contain important information when correlated with other data (a topic we discuss in Chapter 9). The next type of data can be seen in goals 2, 4, and 5. This type is a little more sophisticated because the data set generated by each item is made up of values from a 4-point scale with a specific and intentional order. We can still compute simple, categorical information about these items—perhaps Joey had 4/8 observations for goal 2 recorded as “Not today, even with modeling” and 4/8 recorded as “matched after modeling.” But we can also compute a new kind of information because, for this type of data, order counts. It meant nothing that “center” was listed after “home” in the location item and switching the order wouldn’t have change the meaning. In contrast, it means something definite that “matched after modeling” comes after “not today, even with modeling” in the list for goal 2. Specifically, it means that during intervention we expect the later choice on the list to be developmentally or cognitively more difficult to achieve than the earlier choice. Similarly, during analysis, we consider scores from later on the list to indicate more growth than scores from earlier on the list. Because the order of the list is meaningful, this type of data is called ordinal. All of the items on your collection sheet that have 4- or 5-point qualitative scales produce ordinal data. Each point on the scale is considered a different category or name, just as each choice of location was a different category or name. Formally, then, ordinal data is a more sophisticated type of measurement than categorical data because the name or category of a data point also indicates how the observation compares to the other possibilities in the ordering. To take another example: in goal 4 it is not just of interest whether Joey needed a verbal prompt, it is of interest particularly because this was a better outcome than if he needed a physical prompt and a less desirable outcome than if he had acted independently. In general, ordinal data can be subjected to all the same sorts of analyses as categorical data, but can, in addition, be used to make relational statements such as “this month’s performance is better than last month’s.” The third type of data that might appear on your collection sheets is also the most sophisticated type of data. Examples of this type for Joey include the date, time, and overall mood factors, and the percentages collected for goals 1 and 3. Each of these items generates a value from a quantitative scale: the date is a point taken from the scale of days in a year; the time is a point (or interval) taken from the scale of hours in a day; overall mood is taken from a numeric 5-point scale; and the percentages are points on a 100-point scale. These scales differ from the ordinal scales in goals 2, 4, and 5 in an important way. Ordinal scales can be used to express relative goodness, but they don’t tell how much of a difference exists 88

FROM DATA COLLECTION TO DATA ANALYSIS

between successive categories. We don’t know in goal 4, for example, that the amount of improvement Joey demonstrates by moving from multiple prompts to a single prompt is equal to the amount of improvement he demonstrates by moving from a single prompt to independence. But we do know, without question, that the amount of time between 3:00 and 5:00 is exactly the same as the amount of time between 4:00 and 6:00. We also know that the amount of increase in success rate between 30% and 40% is exactly the same as the amount of increase in success rate between 40% and 50%. Because the size of the interval between data points is equal (in the arithmetic sense) we call the data sets generated by these items interval data. As with categorical and ordinal data, each point on an interval scale defines a name or category, and as with ordinal data, the categories can be ranked or ordered. What makes interval data different, then, is just that the intervals between the categories are equal and obey the laws of arithmetic. But because of this simple difference interval data allows us to say not only that a child is doing better, but also how much better the child is doing. For example, suppose that during the last period we measured, Joey completed the prompted expression in goal 3 (“Ready, set…”) 40% of the time, while during the current period he completed the expression 80% of the time. We might characterize his performance by saying, “He is now doing twice as well as before,” or “He is now twice as likely to answer appropriately.” However, it is important to understand that such statements are supported by the data because they quantify behavior, not the underlying change. We cannot claim that Joey learned twice as much new information during the current interval, nor can we say that he is “half as autistic” as he was when his performance was only 40% correct.

Translating words to numbers to create raw data When you chose a scale for each goal on your data collection sheet you worded it carefully to reflect both the stage of the goal (for example, “emerging”) and the degrees of progress or change that you wanted to observe during the period of intervention. Using scales with meaningful category names (for example, “not today, even with modeling”) makes the job of the data collector easier and less susceptible to error or individual variation. Unfortunately, meaningful category names have the opposite effects on computers. The tools that are available to make data analysis easier require that we turn the words into numbers. Translating the data on your collection sheets into numbers will always be the first step in your data analysis. For the purposes of translation there are five different cases to consider. The examples provided for each case refer to the translated values that are circled in Figure 6.2.

89

FROM GOALS TO DATA AND BACK AGAIN

1.

Interval values require no translation. The calculated frequency in goals 1 and 3 and the number circled for the overall mood are the numeric values for those observations. The date and time can also be left as they are at this point in the analysis. (We’ll look at different ways to treat date and time in Chapter 9).

2.

Items in an accumulating list also require no translation.1 For this type of goal the data sheet specifies the actual items you want the child to add to his or her repertoire of words, skills, or emotions. In each session one or more of the items will be circled or checked. Every checked item is part of the observation to be recorded in the spreadsheet. Eventually, you want to see each of the targeted items listed at least once in the spreadsheet. Once each item has been listed, the child has achieved his or her goal of increasing a range of behaviors. This method of measurement does not depend on further analysis to verify change over time—change is evident from the data itself. Since no additional analysis is required, we do not discuss this type of goal again in the remaining chapters.

3.

Factors that have categorical values (the name of the therapist and the location of the intervention) need to be translated, but we delay the translation to numbers until we are ready to use the factor in computing a correlation statistic (see Chapter 9).

4.

Observations based on a qualitative scale (goals 4 and 5) must be changed to numeric form. Remember that each qualitative scale was created after identifying step-by-step changes in behavior. The team considered each step to be categorically different, a separate marker of progress. The translation process assigns sequential numbers to each step: 1 to the first value, 2 to the second value and so on. In this way the numbers reflect growth in functioning with 1 representing the lowest level expected of the child and 4 or 5 (depending on how many steps you identified) representing what we expect to be the best outcome for this period of intervention. Notice that the process of translating the scale to numbers creates an important characteristic in

Although there is no example of this kind on Joey’s sheet, you can see one in Chapter 5 (where the goal was to extend Peter’s play repertoire to include gross motor play, construction/building activities, puzzles, art, music, and early pretend play).

1

90

FROM DATA COLLECTION TO DATA ANALYSIS

Name: Joey Smith Name of therapist: Sasha Date: 1/3/01 Time: 3:00–5:00 Location of intervention:

 home

“ center

On a scale of 1 to 5 (avoidant to sociable), please rate Joey’s overall mood for the day. 1 2 3 4 5 1. When offered a choice of food, or object with which to play or hold, was Joey able to indicate preference?

# of opportunities

2. When given an opportunity, was Joey able to match a photograph to a preferred item?

o o o o

3. After hearing the language modeled two times, was Joey able to fill in the word approximation “go” after “Ready, set…” to start a preferred activity?

# of opportunities

4. What level of prompt was needed to get Joey to pull an adult to an object of desire?

o multiple physical and verbal prompts o a verbal prompt (“Show me!”) with outstretched hand ý independently (a few times) 3 o independently (consistently)

5. Did Joey attempt to imitate your mouth movements?

o ý o o

|||||||||

# of successes |||

Calculate: 33% (# of successes ÷ # of opportunities x 100) not today, even with modeling N/A matched after modeling independently matched 1 or 2 times independently matched multiple times

||||||

# of successes |||

Calculate: 50% (# of successes ÷ # of opportunities x 100)

no attempts noted growing interest in mouth movements a few attempts 2 multiple attempts

Figure 6.2 Joey’s data sheet after translation

91

FROM GOALS TO DATA AND BACK AGAIN

your data: numbers whose difference is 1 now represent different categories of behavior. 5.

Goals without associated observations (goal 2) must be explicitly recorded as “N/A,” “no observation,” or a similar phrase.

When you’ve assigned the appropriate type of value to each factor and observation, you’ve translated the child’s behavior into the raw data we work with in a spreadsheet. You may find that circling or highlighting the value for each observation on the data sheet will make it easier to enter the data accurately on the computer. As you scan down the circled values, notice that all the values for goals are numbers and that you can no longer tell the ordinal data from the interval data. Don’t let appearances fool you: we have changed the way the data looks, but not what the data means. The interpretation of the data must always be based on its underlying meaning.

Transferring the raw data to spreadsheets The second step in beginning a data analysis is to transfer the data on your collection sheets to a spreadsheet. Using a spreadsheet lets us further organize the data, apply statistical formulas to it, and automatically generate graphs that help make sense of it. If you haven’t used a spreadsheet program before, this process may seem difficult and time-consuming at first, but it will become easier and faster with practice. More importantly, taking the time to learn how to use a spreadsheet will make your analysis more reliable and more informative. There are many ways we might format each child’s data in the individual sheets, rows, and columns of an Excel workbook. The format we’ve chosen to use is shown in the portion of Joey’s spreadsheet in Figure 6.3. Whether you use this format or not, it is important that you use your chosen format consistently. Let’s look at the spreadsheet in Figure 6.3 more closely. You may find it helpful at this point to open the file “Joey Jan-Mar2001.xls” in Excel on your computer. There are 7 features of our organization that we want you to notice:

92

1.

The name of the file includes both the child’s name and the time period covered by the data. This makes it easy to find this particular period of Joey’s intervention without opening and closing a large number of files. It also helps you avoid overwriting this file by mistake with a new file for Joey on different dates (or with a new file for a different child on the same dates).

2.

The name of the sheet (circled at the bottom) tells us it contains the raw data for this period of intervention. Everything you do in data analysis begins with the raw data. For this reason, raw data should be placed on its own sheet in the workbook and all subsequent analysis performed on

FROM DATA COLLECTION TO DATA ANALYSIS

a copy. (The handout in Appendix B titled “Directions for Using Excel” covers the mechanics of copying data and other basic operations in Excel.) By observing this precaution you will not have to start over should you mistakenly corrupt the data by transforming it in an inappropriate way. By working with a copy of the data, you always have an on-line version to go back to if necessary. 3.

The top row of the sheet is used for labels that explain the information in each column. The remaining rows record the data for each session, one session per row, arranged in chronological order. The cells in the first column of each row label the session dates explicitly. This may not seem important now, but it will prove invaluable if you have to return to the collection sheets to check for data entry errors or comments that might have been written by the observer. In addition, your data may have different uses over time. For example, at some point in the future you may find yourself wondering how a child behaved during the same week or month in a previous year. As another example, someone may want to combine your data for a child with other professionals’ data sets to examine behavior across therapies on a day by day basis.

file name

worksheet name

Figure 6.3 A portion of Joey’s raw data after it has been entered on a spreadsheet

93

FROM GOALS TO DATA AND BACK AGAIN

4.

While the left-most column is used for the date, the remaining columns contain the data in the order dictated by the labels in the first row. Choose a meaningful label for each factor and goal on the collection sheet; do not use the number of the goal to label the column. Again, this may seem like unnecessary work at the moment, but over time you are likely to forget which column represents which goal, making it easy to go to the wrong data in response to a question. Moreover, if you are not careful when you perform tests using the data in a column, you may pick up the goal number as if it were a piece of data, changing the outcome of the test. Finally, consider that the same goal may occur in a different place on subsequent data sheets for this child or on the various data sheets used by different professionals working with this child. If you use a brief phrase (like “Match photo”) you will be able to track progress within a child’s development more easily. In short, the small effort taken to choose a meaningful label now will minimize confusion and make for more accurate statistics in the future.

5.

Along with meaningful labels for the goals, we include a reminder of the scale that was used. In the case of percentages we include the approximate number of opportunities per session that the child had for that goal during this period of intervention. We’ll need this information to interpret the results of the analysis. By including it now we save ourselves the trouble of having to go back to the data collection sheets to find it.

6.

Notice the blank entry (cell E4) under the “Match photo” goal for January 4. On that day Joey did not have the opportunity to match pictures to preferred items. Having no opportunity is different from having an opportunity but not being able to match (the lowest value on the qualitative scale for this goal).2 The observer who took the data left all the items unchecked to indicate that Joey had no opportunity to work on this goal. We translated the absence of a check mark into the raw data value “N/A” then, in turn, into a blank cell in the spreadsheet. For the statistical routines to work correctly it is critical that you leave blank any cell for which you did not collect data. Do not enter 0, “N/A,” “No opportunity,” or any other value in these cells.

In this way “Match photo” is different from “Imitate mouth” where, as long as the therapist talks and plays face-to-face with Joey, we can say that Joey had the opportunity to imitate.

2

94

FROM DATA COLLECTION TO DATA ANALYSIS

7.

Finally, note that the labels for the factor data are italicized while the labels for the goal data are in boldface. Use a visual convention to separate your factor data from your goal data. In the following chapters we will apply certain statistics to each column of goal data and other statistics to factor-goal pairs. Sometimes factor data will be represented by words on the raw data sheet (e.g., location in column B) but sometimes it will be represented as numbers (e.g., overall mood in column C). Having a simple visual cue that distinguishes between the two kinds of numbers (rather than relying solely on memory) will help to make sure the correct statistics are applied to the correct columns.

The accuracy of your raw data is critical to the accuracy of your statistics and resulting interpretation. Faulty data usually leads to faulty conclusions. Adopting a consistent approach to data organization across children is one way to improve accuracy. There are two additional good habits that are worth cultivating. First, you’ll make fewer transcription mistakes due to fatigue (and boredom) if you enter your data as it is collected rather than in one long session just before you’re ready to do an analysis. Second, don’t assume you did it right; always check the values in your spreadsheet against the values on the collection sheet after you’ve finished entering them. The best way to catch any transcription mistakes you’ve made (and we all make them) is to have a colleague or friend read the values from the collection sheet while you check the numbers on the spreadsheet. If this isn’t practical, waiting an hour or more before you check the collection sheets against the spreadsheets can help keep you from repeating an error.

Keeping in mind what the numbers really mean There is one final feature to notice about the data as it now appears on your spreadsheet: most of the context that defines the meaning of the numbers has been removed from view. As we focus increasingly on what we can do with the data when it is represented as numbers, don’t lose sight of the fact that the meaning of those numbers depends on the way the goals and scales have been worded. A value of 4 in one column does not necessarily mean the same thing as a value of 4 in another column. Even more importantly, keep in mind that the meaning of the numbers depends upon the accuracy and consistency of the observer and, most importantly, upon how well the behaviors being observed and the scales we have specified truly reflect the underlying cognitive and emotional changes we desire. This reminder is critical at this point in the book because we are about to switch our thinking from particular children with histories and meaningful goals to data sets with no real information behind them. In developing the tools 95

FROM GOALS TO DATA AND BACK AGAIN

for data analysis our examples will be based primarily on the information in the ABC Workbook on the CD-ROM that accompanies this book. The ABC Workbook contains data sets for three fictitious children: Anton, Becca, and Celeste. We’ve given names to the data sets because it makes the discussion easier to read, but we’ve chosen the names to remind you of the variables A, B, and C because these are our “variable” children. Their data has no real goals behind it and may, in fact, be used to represent different goals at different times for narrative purposes. To help you keep this in mind we have also labeled the goals in these data sets with variables (X, Y, and Z) rather than the meaningful abbreviations you would use for real data. If this sounds confusing, don’t worry, it will begin to make sense shortly. And rest assured that once we’ve developed them, we will bring our analytical tools back to the cases of Joey and the other children (see Chapter 10). In the meantime, we suggest you keep the ABC Workbook available on-line as you read the remainder of this and the next three chapters. The exercises designed to teach you how to use the analytical tools in Excel require the ABC Workbook as well.

First impressions: numbers versus pictures If you open the ABC Workbook to the sheet labeled “Anton Goal X” you will find 30 observations for the period between January 3rd and March 10th for the fictitious child named Anton (see Figure 6.4 if you are uncertain which sheet we mean).

96

FROM DATA COLLECTION TO DATA ANALYSIS

As you glance down the column for goal X you may notice that the first group of 10 values contains only 1s and 2s, the second group of ten values contains 2s and 3s, and the final group of ten values is made up of 3s and 4s. So, if we assume that lower numbers correspond to weaker performance and higher numbers correspond to stronger (more desired) performance, it seems clear that Anton’s behavior has become increasingly appropriate over this time period. We made the trend in this example fairly obvious and specifically pointed out the meaningful pattern in order to help you see it. In general, however, the ability to look at even a small column of numbers and judge whether or not change has taken place is a skill that takes a lot of practice. To develop this skill, and to work with larger or more complex data sets, you need a few simple but powerful tools. The first one we’ll add to your analytical toolbox is graphing. Graphing your data is often the second step of data analysis. The power of graphing can be seen by comparing Figure 6.5 to column B of Anton’s spreadsheet. For most people the visual representation of the data shown in Figure 6.5 is easier to understand and characterize than the column of numbers on the raw data sheet. You may also find that even when there is a clear trend in the data, as in Anton’s case, a picture is more compelling than a list of numbers when showing your data to others. Indeed, one of the reasons for entering your data in a spreadsheet is that the spreadsheet program has built-in graphing functions to help everyone on the team see the trends. This graph, for example, looks like a set of stairs rising, a good metaphor for (and fairly reliable predictor of ) positive change.

4-point scale (1,2,3,4)

Anton, Goal X

00 6/ 3/

0 28

/0

0 2/

21

/0

0 2/

/0 14

2/

0

00 7/ 2/

31

/0

0 1/

24

/0

0 1/

17

/0

0 1/

/0 10

1/

1/

3/

00

5 4 3 2 1 0

Figure 6.5 The line graph for Anton’s raw data shows a “rising stairs” pattern

97

FROM GOALS TO DATA AND BACK AGAIN

Creating simple line graphs Just as there are different types of data, there are different types of graphs that can be used to present each data type. Figure 6.5 is an example of the conven3 tional choice for showing a trend over time: the line graph. Exercise 6.1 (in Appendix C) takes you step-by-step through the creation of Anton’s line graph for goal X using Excel. Exercise 6.2 takes the basic method taught in Exercise 6.1 and introduces a number of helpful shortcuts and variations. There is a bit of an art to producing a good graphical representation of your data. On the one hand you must make sure that enough information is included in the various titles, labels, and legends to clearly explain what the picture represents. On the other hand, too much information in too small a space leads to visual clutter that may obscure the point you want to make. How you balance these two requirements—visual clarity and information accuracy—is partly a question of personal style. You don’t need to become an expert on graphical representation to create graphs that are clear, useable, and readable by others. There are, however, a few features you should be aware of: · Who and What: There should never be any question about which

child’s data it is or which goal is being graphed. We tend to put this information in the title of the graph because that is generally the most prominent text. In addition, when the child’s name and goal are in the title it isn’t necessary to include a legend, which helps cut down the visual clutter. The exception is for a graph containing data for more than one goal. In these cases the legend is necessary and the title may be used for other purposes, as shown in Figure 6.6.

· When: Because we are always interested in measuring change within

some time interval, the time interval must be clearly specified as well. In Figure 6.6 the dates of intervention have been associated with the increments shown by the “tick marks” (little lines) on the horizontal axis. Since the dates are explicit and recognizable, we don’t need an additional label for the axis. Again, our choice preserves information with an eye toward reducing visual clutter. Sometimes the tick marks aren’t informative enough, however, and additional information should be provided. As an example, see Figure 6.7 where we use the axis label to indicate that the tick marks represent a sequence of sessions.

· How: How was behavior measured for this goal? The vertical axis

reflects the values of the data points. Graphing programs usually

Technically, a line graph is not an appropriate way to represent ordinal data because the continuous line implies that the values between the points are well-defined. People are rarely misled by this aspect of the representation, however, so we will sacrifice a bit of technical correctness for the sake of convention.

3

98

FROM DATA COLLECTION TO DATA ANALYSIS

Comparison of Progress on Goal X

Anton

00

0

6/

3/

/0

0 2/

28

/0

0 2/

21

/0

00

14

2/

7/

0

2/

/0

0 1/

31

/0

0

24

/0

1/

17

1/

10

/0

0

Celeste

1/

1/

3/

00

4-point scale (1,2,3,4)

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0

Figure 6.6 The legend (box at right) is necessary when you combine data for two children receiving group intervention on the same dates

5 4 3 2 1

28

25

22

19

16

13

10

7

4

0

1

4-point scale (1,2,3,4)

Anton, Goal Y

Sessions

Figure 6.7 The line graph for Anton’s raw data for goal Y

determine the increments on the vertical axis automatically, based on the values of the data set being graphed. Division of the vertical axis is one of the few features of a graph that can be complicated to change. Unfortunately, the values chosen by Excel are often inappropriate for the kind of data we use. Someone looking at just the vertical axis in Figure 6.6, for example, might believe that the child’s behavior was rated on a continuous scale of values from 0 to 4.5. In fact, the only values for goal X that the observer could have chosen were the numbers 1, 2, 3, or 4. So we use the title of the axis to remind the reader of the actual scale and explicitly counter the misinformation in the graph. 99

FROM GOALS TO DATA AND BACK AGAIN

· How many: Figure 6.6 (and Exercise 6.2) demonstrates that more than

one data set can be placed on a single graph. Should you put all the data for a child in one graph? How many goals can you combine? The decision depends on both clarity and the purpose of the graph. Combining data sets in a single graph will usually make each trend a little bit harder to spot—the more you add the harder it gets. Sometimes the difficulty in reading the graph must be tolerated because you want a direct comparison among data sets. Keep in mind, however, that to combine data sets on a single graph the horizontal and vertical axes must mean the same thing. You cannot meaningfully combine goals with different size scales or goals in which the data was collected on different dates.

There is one additional aspect of graphing you need to understand: the visual effect of length versus height in the axes. It is possible to mislead someone about the meaning of your data by changing the ratio of these dimensions. For any single graph you don’t need to worry about this problem because Excel usually makes reasonable choices. If you are going to compare graphs side by side, however, it is important that they have the same relative dimensions and increment markings on their axes. Within the guidelines we’ve given you there is still room for variation (and you’ll see some of that variability in the graphs throughout this book). Developing your own style is fine, just remember that the general aims when producing graphs are similar to the general aims for your data collection sheets—be clear and require as little effort as possible on the part of the observer for a correct interpretation. Adopting consistent conventions helps both these aims. Becca, Goal X

4-point scale (1,2,3,4)

5 4 3 2 1

00 6/ 3/

0 28

/0

0 2/

21

/0

0 2/

/0 14

2/

0

00 7/ 2/

31

/0

0 1/

24

/0

0 1/

17

/0

0 1/

/0 10

1/

1/

3/

00

0

Figure 6.8 The line graph for Becca’s raw data. Has she made progress on goal X?

100

FROM DATA COLLECTION TO DATA ANALYSIS

Real data is not pretty Most of the time, the information in a data set is not as clear cut as in Anton’s case. Compare, for example, Anton’s data (Figure 6.5) with Becca’s data for the same goal (Figure 6.8). What should we conclude? Has Becca’s behavior changed? Can we be certain that she has learned the underlying information or skill we wanted her to learn? Perhaps her poor performance in the session on January 31st can be safely ignored; but maybe the additional low score toward the end of February should make us cautious when judging her progress. The case for Celeste (Figure 6.9) is even more confusing. She seems to have been making steady progress then appears to have regressed (around February 7th) then looks as if she may have gotten back on track. Should we conclude that she has reached her goal? Should we conclude that she has not? Celeste, Goal X

4-point scale (1,2,3,4)

5 4 3 2 1

00 6/ 3/

0 28

/0

0 2/

21

/0

0 2/

/0

2/

14

00 7/ 2/

0 31

/0

0 1/

24

/0

0 1/

17

/0

0 1/

/0 10

1/

1/

3/

00

0

Figure 6.9 The line graph for Celeste’s raw data. Has she made progress?

If your impulse is to conclude that Celeste has not reached her goal, then suppose for a moment that the scale being used to measure Celeste’s progress was translated as: 5 or more prompts →1 3–4 prompts →2 1–2 prompts →3 0 prompts →4 Looking at the scale, do we really want to conclude that Celeste’s data does not represent significant progress? Perhaps it is unreasonable, given her stage of development, to expect Celeste to do better than the pattern shown (one occurrence of unprompted behavior in the first ten sessions (10%), but three occurrences of unprompted behavior in the last ten sessions (30%)). Perhaps we wouldn’t expect better than 30% for any child of this age or level of develop101

FROM GOALS TO DATA AND BACK AGAIN

ment. Remember: the numbers themselves are meaningful only with respect to the ideas behind them. What constitutes progress depends critically on what is reasonable to expect for this particular child within the regularities of typical development. Becca and Celeste’s cases more accurately reflect the kinds of graphs you are likely to see when working with real data. Cognitive change is a gradual thing and rarely characterized by constant forward progress. Both adults and children grow tentatively, requiring small steps that make sense within the larger context of our lives to convince us of the usefulness of enduring the discomfort of change. As a result, real data, like real behavior, is often too “messy” to understand using only a tool as simple as the graphing function discussed in this chapter. A commitment to an empirical approach to intervention requires that we outfit ourselves with stronger tools: the mean and standard deviation. We turn to these in the next chapter.

Things to remember How our data is organized: T Spreadsheet file: the file name includes both the child’s name and the

time interval during which the data was collected. The file contains all the data collected for that child during that time interval.

T Sheet name within the file: indicates what kind of data (e.g., raw data). T Raw data sheet: the first row labels factors and goals, the remaining

rows contain the session date in the first column followed by columns containing the value assigned to each factor and goal on that date.

Important principles for labeling and entering your data: T Reduce data entry mistakes by typing in your data each time it is

collected (rather than in a marathon session) and by having your entries verified by a colleague or by yourself at a later time.

T Once your raw data set has been entered and checked, don’t touch it.

Always work with a copy.

T Label the session dates clearly and explicitly in the first column. T Choose a meaningful label for each factor and goal. T Leave empty any cell for which data was not collected during a session.

Empty cells correspond to the N/A or “No opportunity for observation” values on your data collection sheet.

T Use a visual convention (e.g., italics) to separate your factor data from

your goal data.

102

FROM DATA COLLECTION TO DATA ANALYSIS

T Consistency, consistency, consistency! Whether it is in organizing

your data, labeling your spreadsheets, or producing your graphs, the more consistent you are, the fewer mistakes you will make and the clearer your understanding and presentation of ideas will become over time.

T The meaning of the numbers depends on more than just the performance

of the child. Data must always be understood in terms of the way the goals have been phrased and the scales worded, the accuracy of the observer, and how well the behaviors being observed actually reflect the underlying changes we desire.

T Cognitive change is gradual and rarely characterized by constant

forward progress. For this reason, although line graphs can be helpful in understanding your data, they usually do not provide enough information to draw conclusions.

103

Chapter 7

Detecting Change — The Mean

In Chapter 6 we saw that data visualization tools such as graphing have strengths and weaknesses. When a trend in the data is clear-cut, a picture of that trend can be a compelling description. On the other hand, when there is no trend, or when the trend is unclear, a picture may require a thousand words of explanation and still produce no firm conclusion. In this chapter we introduce the use of simple statistical methods by examining the idea of change as it applies to developmental intervention. Our discussion of change leads us to a number of other naturally related concepts: regularity, expectation, predictability, and noise. Some of these concepts may be unfamiliar to you at first while others may seem obvious. Since these concepts are deeply intertwined, however, whichever concepts you’re most comfortable with will eventually lead you to the others.

Patterns of change When we work with a child to change a behavior, skill, or emotional response, we are taking part in what is, ideally, a three-step process. In the first step, the team identifies in the child a consistent way of acting that is ineffective, inappropriate, or maladaptive. The team believes that this way of acting represents the child’s responses to some stimuli or situation at this moment in time. In the second step, the team chooses intervention methods that modify the underlying beliefs, knowledge, or emotions that cause the child’s responses, and these methods are used with the child over a period of time. In the third step—and here’s the ideal part—the team identifies in the child a consistent way of acting that is more effective, appropriate, or adaptive than before. Is there a simpler way to say this? Sure. For the purposes of intervention we define change as the transformation of one regular pattern of behavior into another regular pattern of behavior. Consider the left-hand graph in Figure 7.1. Common intuition tells us that no change has occurred in this child’s behavior. Indeed, looking at a graph like this the team might conclude, “If something doesn’t change soon we’ll have to 105

FROM GOALS TO DATA AND BACK AGAIN

try a new approach.” Referring to our definition of change, we would say, more specifically, that no change has occurred because there is only one regular pattern of behavior, not two. Example of Abrupt Change

4 3 2 1 0 1

2

3

4

5

6

Sessions

7

8

9

10

4-point scale (1,2,3,4)

4-point scale (1,2,3,4)

Example of No Change 4 3 2 1 0 1

2

3

4

5

6

7

8

9

10

Sessions

Figure 7.1 A single regular pattern of behavior versus two different regular patterns of behavior

Now consider the right-hand graph in Figure 7.1. The child whose behavior is reflected by this data does show change, both intuitively and by our definition. First we see one regular pattern (the values of 2 in the first five sessions) then we see a second, different regular pattern (the values of 4 in the last 5 sessions). When we observe a regular pattern we form expectations about future behavior. To put it another way: regularity has predictive value. At the end of Session 5 in the right-hand graph, for example, we might predict that “if nothing changes” this child will continue to receive a score of 2 for this goal. When our expectations are violated and our prediction proven wrong (in Session 6), we assume it is because something has changed (indeed, unless we’ve identified other obvious factors, we assume that the cause of that change is related to the intervention). As time goes on and the new regularity is detected, we establish a new set of expectations and make new predictions. At the end of Session 10, for example, we might say, “I think he’s got it!” The kind of abrupt transformation of behavior shown in the right-hand graph of Figure 7.1—a large and consistent step forward occurring virtually overnight—is rare, but not impossible. Sometimes it does feel as if a child suddenly “gets it.” When we are trying to modify deep and largely unconscious behaviors, however, more gradual change is typical. Figure 7.2 shows an example of a more gradual transformation phase, similar to the “rising stairs” pattern of change we saw for Anton in Chapter 6. Our definition of change alone isn’t quite enough to help us interpret this graph. Are we looking at one occurrence of change or two? The definition is inadequate to settle this question because it is expressed in terms of behavior (as it should be!), not in terms of points on a graph. The definition doesn’t tell us, for example, how many data points are required to establish a regular pattern, or how many data points there are in the period of transformation. 106

DETECTING CHANGE – THE MEAN

4-point scale (1,2,3,4)

Example of Gradual Change

5 4 3

PRE POST

2 1 0 1

2

3

4

5

6

7

8

9

10

11

12

13

Sessions Figure 7.2 One pattern of behavior (pre-transformation) followed by a period of transformation followed by a new pattern of behavior (post-transformation)

A person who doesn’t know the meaning behind the data might see three regular patterns and two distinct episodes of change. With the developmentalist’s eye, however, we are likely to see only one episode of change; we explain the regularities in this way because they meet our expectation that cognitive change typically requires a gradual transformation of behavior. The important point in this example is that we can use our knowledge of development to make a prediction about the kind of pattern of change we are likely to see, then match the actual data to that prediction. In general: our interpretation of the data depends on our expectations and our expectations depend on our knowledge. When should a pattern be considered regular? This is not a trivial question. Figure 7.3 repeats Anton’s data from Chapter 6 to help clarify.

4-point scale (1,2,3,4)

Anton, Goal X

00 6/ 3/

0 28

/0

0 2/

21

/0

0 2/

/0 14

00

2/

7/ 2/

0 31

/0

0 1/

24

/0

0 1/

17

/0

0 1/

/0 10

1/

1/

3/

00

5 4 3 2 1 0

Figure 7.3 The line graph for Anton’s data for goal X

107

FROM GOALS TO DATA AND BACK AGAIN

When we said that the pattern of change in Figure 7.2 was similar to Anton’s we were making a judgment based on our visual perception of the overall shapes of the two graphs (“a set of rising stairs”). Visual perception is a powerful tool for judging regularity and that is one of the reasons we graph data. In Anton’s case, however, we need more than just visual perception to judge whether the graph meets our definition of change. Generalizing from the pattern in Figure 7.2 we might be comfortable picking out the middle portion of the data (the nine data points between 1/24 and 2/14) as the transformation stage. But how should we interpret Anton’s pre-transformation and post-transformation periods (the sessions prior to 1/24 and after 2/14, respectively)? Neither the pre-transformation behavior nor the post-transformation behavior is as obviously regular as it was in Figure 7.2. As in the previous example, it is knowledge that is not directly apparent from the data that allows us to say that there are two different regular patterns of behavior in Anton’s graph. We know, for example, that no child is entirely consistent in his or her behavior. This knowledge, alone, might be enough to make us confident in our interpretation. If we look back at the meaning of the scale used during observation, however, we might also find that the wavering in both the pre- and post-transformation scores is easy to dismiss as an artifact of that scale. For example, suppose the translation of the scale for this goal was: 3–5 prompts →1 1–2 prompts →2 no prompts (at least once) →3 no prompts (consistently) →4 Then if Anton’s pre-intervention pattern of behavior regularly required either two or three prompts, his scores would alternate between 1 and 2. In other words, the regularity in behavior would cross our category boundaries and create the sort of pattern shown in the pre-transformation portion of his graph. Similarly, if at the end of the observation period Anton still needed an occasional prompt, his scores would alternate between 3 and 4 and we would expect the post-transformation pattern in Figure 7.3, given our scale. Knowledge—of children in general and Anton’s scale in particular—makes the difference in our interpretation. Now consider Daniel’s graph in Figure 7.4. How many regular patterns do you see? Two people looking at this graph might conclude different things about whether and when change occurred, even if they had the same knowledge about development, rate of cognitive change, the particular child, and the particular child’s goals. Adding more knowledge—even knowledge about Daniel—won’t necessarily alter their interpretations. This kind of data is typical of what you see when measuring real behavior in real situations. In the previous chapter we called it “messy.” The technical term is “noisy.” Noise is the set of values that creates 108

DETECTING CHANGE – THE MEAN

an apparent lack of regularity when such regularity actually exists. You can think of noise as regularity’s “evil twin.” Noise interferes with our ability to perceive regularities. It undermines the certainty we feel about our interpretation. At its worst, a noisy graph of successful learning can be visually indistinguishable from a graph that shows true lack of change.

4-point scale (1,2,3,4)

Daniel, Noisy Change

5 4 3 2 1 0 1

2

3

4

5

6

7

8

9

10

11

12

13 14

15

Sessions Figure 7.4 Is this a case of change or not?

We used knowledge about how we measured Anton’s goals to compensate for the noise in Figure 7.2. Another way to reduce noise can be to extend the period over which we make observations. This may help because our sense of regularity depends on detecting a consistent pattern over time. The graph in Figure 7.5 shows the 15 sessions in Figure 7.4 as part of a longer stretch of time. If we had been able to measure Daniel’s behavior for this goal before Session 1 we would

5 4 3 2 1

22

19

16

13

10

7

4

0

1

4-point scale (1,2,3,4)

Daniel, View Extended Over Time

Sessions Figure 7.5 With more data before and after, the pattern of change emerges

109

FROM GOALS TO DATA AND BACK AGAIN

have seen the first regular pattern of behavior clearly. And if we could continue to measure Daniel’s behavior for this goal after Session 15 we would see a second regular pattern emerge. Two regular patterns separated by a period of growth—now that looks familiar! What just happened? The data that looked too noisy to interpret with any certainty in Figure 7.4 didn’t change, yet most of the noise seems to have disappeared. Why? Remember that our definition of behavioral change involves three parts: two different patterns of regularity separated by a period of transformation. When we tried to divide Daniel’s first graph into those three parts, it was difficult to see where one part ended and the next began. Every possible division left all three parts looking noisy. When we broadened our view in time the noise appeared as one segment. Moreover, it shifted into the segment in which it is most easily explained—the segment that reflects the period of learning transformation. Our knowledge, in combination with enough data, allowed us to explain 1 the noise. Unfortunately, for most kinds of meaningful change we can’t know how much data is going to be enough to picture the whole process for a particular child. We simply don’t know how long it will take a given child to learn a new skill, behavior, or emotional response in an adaptive and lasting way. In addition, we don’t always know exactly where the child is in the learning process when we begin to collect data. We often target a behavior when we think the time is ripe, which is just another way of saying that the data we collect at the beginning of intervention may already reflect some of the noise typical of the transformation stage. It seems as if everywhere we turn there is potential noise threatening the accurate interpretation of our data. How do we decide when a portion of our graph is just a little noisy and when that portion is really showing that lasting change requires additional intervention? The unfortunate truth is that we can’t eliminate noise completely, nor can we always explain it away. We’ve already discussed some ways we can minimize it: design goals to be meaningful reflections of the behavior we’re trying to teach, design scales to look for the degree of change that could occur within the time interval being measured, practice good data collection habits, and use our knowledge judiciously in interpreting the graphs. But, in the end, we have to accept that real behaviors, like real children, are inherently noisy beasts. If we can accept that our data has noise then we can acknowledge that not every data point carries useful information. But how do we decide which of the data points reflect true behavior and trends, and which ones are noisy? If we’re going to ignore some of the information in our data we need to be certain that If you think we’re going to point out that this example shows one more reason why more data is better, you’re right.

1

110

DETECTING CHANGE – THE MEAN

we do it fairly, not just in whatever way makes us (and the child) look good. In addition, if we’re going to use our interpretation to convince others, we need to be certain that they would make the same choices. In short, our method of dealing with noise must be consistent across children and repeatable by others.

Computing the mean The mean is a statistic that helps eliminate some of the effects of noise in a way that is both fair and uniform (it won’t do it by itself, so we’ll have to introduce its partner, the standard deviation, in the next chapter). It does its job by giving us a single representative value for a set of data. In essence, the mean clears away much of the visual clutter, thereby removing a good deal of the subjective component that comes with using line graphs for evaluation. Technically, the mean of a set of numbers is the set’s arithmetic average (the sum of the items in the set, divided by the number of items in the set). In Excel, the mean is computed using the AVERAGE function. Exercises 7.1 and 7.2 in Appendix C provide practice using this function. When you’ve completed Exercise 7.2 you’ll see that the mean is the same for each of Anton’s, Becca’s, and Celeste’s data sets. Recall from the previous chapter, however, that the line graphs for these data sets led us to qualitatively different evaluations of each child’s progress on goal X. Of course, when we looked at the line graphs we were looking for evidence of change. What part does the mean play in our search?

Evaluating change using mean values In well-designed psychology experiments, the typical method for detecting change requires three steps: first, the abilities of a group of participants are measured; second, the participants perform whatever activity the researcher has hypothesized will promote learning; and third, the participants’ abilities are measured again. The researcher then compares the participants’ test scores before the activity to their scores after the activity. A significant difference in those scores is taken as evidence that the experimental activity had an effect. What counts as a significant difference depends on the size of the group and the statistic the researcher chooses for the analysis. The choice of statistic depends, in turn, on the experimental design, access to a control group that did not undergo the experimental activity, any underlying assumptions about the participants being tested, and so on. Of course, this method is nearly useless as soon as you leave the psychology lab. Most of the statistics appropriate for studying large groups do not apply to studying a single child in intervention. Moreover, there is little or no “before” intervention measurement in our work. We can’t gather extensive baseline data for each goal before we begin to try to alter the child’s behavior, but instead must 111

FROM GOALS TO DATA AND BACK AGAIN

move almost immediately from identifying a goal to both collecting data on performance and providing behavior-changing experiences. Nor is there a clear period “after” intervention. If we have chosen natural activities for the context of our intervention, learning opportunities may occur when we are not observing and over which we have no control. Having reoriented the child so that the natural social world continues to intervene and reinforce a new behavior, the “experiment” goes on long after the goal has been marked “achieved” in our reports. So, we must adapt standard experimental design to our circumstances. Like the researcher we are concerned with demonstrating a difference between “before” and “after” behaviors. But we must replace formal group statistics with a test for change that makes sense for a single child in the real world. Not surprisingly, the test we use is based on our definition of change as a three-part process and our expectations about how that process unfolds. At the heart of the test is a comparison of means. There are three steps to performing such a comparison for each goal: 1.

Divide the data for that goal as closely as possible into thirds. We consider the first third of the data to correspond to “before” intervention and the last third of the data to correspond to “after” intervention. Copy the “before” and “after” data to a new spreadsheet. Ignore the middle third of the data.

2.

Compute the mean of the “before” data and the mean of the “after” data separately.

3.

Compare the two means in terms of the scale for the goal. If the two means reliably reflect different categories of behavior then conclude that change has occurred.

In the remaining sections of this chapter we look at each of these steps more closely.

Step 1: Dividing the data for the goal

Open the sheet titled “Anton Goal X Before-After” in the ABC Workbook to see a division of data as specified in Step 1. Because Anton’s data contains 29 observations, perfect thirds are not possible. To ignore the least amount of data, we copied the first ten and last ten observations from Anton’s raw data and left the nine middle cells behind. We also put blank rows between the sets to make sure it was easy to tell where the “before” data stopped and the “after” data started. The sheets titled “Becca Goal X Before-After” and “Celeste Goal X Before-After” were created in the same way (the first few steps of Exercise 7.3 in Appendix C take you through the mechanics).

112

DETECTING CHANGE – THE MEAN

At first glance dividing the data into thirds may seem like an arbitrary way to proceed, but some well-defined division is necessary if our method is going to be consistent and repeatable. The critical aspect of this step is not the fraction we’ve chosen (more about that below) but that the division is made independent of the characteristics of any particular data set. What we’re not allowed to do is look at the data and say, “Hmmm, we’ll just call this section over here ‘pre-transformation’ and that section over there ‘post-transformation’ and see what happens.” The rules of the game say that if we have successfully produced change through intervention then we must be able to fit the data to the hypothesis, not the other way around. In this case, our hypothesis is that during this time interval there has been a change in the child’s underlying knowledge, beliefs, or emotions such that he or she is able to behave in a way that is consistently 2 different from his or her behavior prior to intervention. But why thirds? The choice comes from the theory of how cognitive change is reflected in behavior. If change is occurring within the time frame of our analysis then each of change’s three phases—pre-transformation regularity, transformation, and post-transformation regularity—should be uniquely reflected in its own subset of the data. “Before” corresponds to that period of time during which the child’s behavior is best explained by his or her old ways of thinking and feeling. We expect pre-intervention patterns of behavior to dominate the first subset of the data even as we begin working on a goal because the kinds of lasting change we are interested in tend to occur slowly. Assuming our intervention is working, the further along we go in time, the less dominant the pre-intervention patterns should be. So, we expect the middle subset of our data to correspond to the transformation stage of change. The period of transformation is regular only in its irregularity. During this time we cannot predict the child’s actions from session to session. He or she will do better on some days than on others, try new responses in some situations but revert to old behaviors when too challenged or uncertain. It is the knowledge that this part of our data should be noisy that allows us to ignore it. Finally, our definition of change requires the evidence of a new pattern of regular behavior in the child. So we expect the remaining subset of our data to correspond to “after”—that period of time during which the child’s behavior is best explained by the new ways of thinking and feeling that have resulted from intervention. Figure 7.6 overlays these ideas on the line graph for Anton’s data.

2

In standard psychology experiments statistics are used to reject the null hypothesis. What this means is that you hypothesize that there has been no change as a result of the experiment. Then you hope that the statistic shows that the probability of the before and after scores being from the same set of participants is so low that you can safely reject the hypothesis of no experimental effect. Yes, our way of phrasing it is far less confusing. As long you remember to fit the data to the hypothesis and not the other way around, you’ll do fine with the simpler characterization.

113

FROM GOALS TO DATA AND BACK AGAIN

Does each phase of transformation really correspond to exactly one third of the data all the time? No, of course not. But thirds are a good approximate solution to the practical constraints. As a general rule, we want at least ten data points in each of the “before” and “after” data sets to compute reasonable statistics (don’t even bother to analyze data with fewer than five values in each set). If we have one observation every day for each goal this method requires a month of behavior (30 days) as the minimum time span for analysis. This works out nicely in cognitive terms—another general principal is that lasting changes in many kinds of behavior take at least a month of practice. Of course, if we collect data less frequently than once a day we have to adjust the time span for the analysis accordingly. For example, if you work with a child five days a week you need at least six weeks to collect ten “before” and ten “after” data points. If you work with a child only three times a week, you need nearly three months of data for an adequate pool of observations. Since quarterly analysis is required by many educational systems and service providers, dividing the data into thirds makes it possible to meet their constraints on a three-times-a-week-or-more 3 schedule and still have at least ten observations for each mean.

5 4 3 2 1 0

“BEFORE”

LEARNING

3/6/00

2/28/00

2/21/00

2/14/00

2/7/00

1/31/00

1/24/00

1/17/00

1/10/00

“AFTER”

1/3/00

4-point scale (1,2,3,4)

Anton, Goal X

Figure 7.6 Our model of cognition assumes that if change occurred within the intervention period then behavior went through three phases: pre-transformation (“BEFORE”), when old patterns dominate; transformation, when learning occurs; and post-transformation (“AFTER”), when behavior is best characterized by the new patterns learned during intervention

If you work with a child only once a week you need about seven months of data to perform a comparison of means as we describe it. This is impractical for professionals who must report on a quarterly basis. You can increase the amount of data you collect without increasing the number of intervention sessions by combining your data with data taken by another observer (a parent, for example) on days you do not work with the child. If you increase your data set in this way you must also check the correlation between observer and performance, as discussed in Chapter 9. You still may not manage to collect ten observations from each of the first and last third of the time interval; the fewer observations you have, the less certain you should be of your conclusions.

3

114

DETECTING CHANGE – THE MEAN

Theoretical and practical justifications aside, it is still the case that sometimes there will be change and that change will be obscured by the decision to divide the data into thirds. That is the price of consistency and repeatability. As we work through the examples in Chapters 8, 9, and 10 we will point out patterns that may arise in the data due to this decision. For now, let’s continue with our explanation of the comparison of means.

Step 2: Computing the means and charting bar graphs

Having separated out “before” and “after” from the raw data, the next step is to compute the mean for each subset. When we perform this step for Anton’s data, we find that the “before” mean equals 1.4 and the “after” mean equals 3.6. Exercise 7.3 (Appendix C) gives you a chance to practice this step, building on the skills learned in Exercises 7.1 and 7.2. As we noted in Chapter 6, a picture may give a clearer and more compelling explanation of the pattern in the data than the numbers alone. To create a graph that contrasts the “before” and “after” means for a goal we use the column bar graph, as shown in Figure 7.7. Exercise 7.4 (Appendix C) takes you step-by-step through the process of creating bar graphs like this one.

4-point scale (1,2,3,4)

Comparison of Means, Anton, Goal X 4.0 3.0 1/3-1/26

2.0

2/18-3/10

1.0 0.0 Before (1.4) versus After (3.6) Figure 7.7 Anton’s performance on goal X “before” and “after” intervention

Sometimes we want to start with a broad view of the child’s growth over the analysis period before focusing in goal by goal. Most graphing programs make it easy to show all of a child’s goals on a single graph (see Exercise 7.5, Appendix C). Unfortunately, these programs even allow us to combine goals that shouldn’t be combined. It is inappropriate to combine goals that have different size scales because the resulting graph can create perceptual biases. In other words, you can combine goals with 4-point scales in one graph but you must produce a separate graph for goals with 5-point scales and another for the goals with frequency values (100-point scales). 115

FROM GOALS TO DATA AND BACK AGAIN

Step 3: Comparing the means

All right, we’ve divided the data, computed the means and produced column bar graphs. It’s time to figure out what all this prep work actually tells us. Progress: yes or no? Since the differences in Anton, Becca, and Celeste’s line graphs for goal X were what originally sent us down this path, let’s see if using the mean has helped clarify the situation. We start with the easiest case: Anton. It seemed clear from his line graph that he had made real progress on this goal after about two and a half months of intervention. At the very least we expect that a comparison of means for Anton’s data will show the same result. In Step 2 we computed the “before” and “after” means for Anton’s data on goal X and found that they were 1.4 and 3.6, respectively. Figure 7.7 showed the same information in graphic form. Step 3 says that the means must represent different categories of behavior if we’re going to conclude that there has been change. How do we know whether the means reflect different categories of behavior? The label on the vertical axis of the graph and the label at the top of Anton’s spreadsheet remind us that this data was collected using a qualitative 4-point scale. Recall that for this type of data, each step on the scale reflects a different category of behavior. In other words, we chose each step of the scale to reflect what we felt would be a qualitatively different level of performance by Anton. When we translated the steps on the scale into numbers, each successive level of performance was assigned a number that was one higher than the level before. A score of 1 in Anton’s data means that Anton’s behavior that day fell in the lowest category on the scale. A score of 2 means Anton’s behavior was described by the next higher category, a 3 means the next higher, and a score of 4 means that Anton’s behavior fell in the highest category on the scale. Because of this correspondence, we define a qualitative scale’s category width to be 1.00 (Chapter 9 contains an example of how to calculate category width for percentage data and Chapter 10 has an example of category width for tallies). Since the difference between Anton’s means is about 2, Anton’s performance progressed across two category boundaries. Impressive progress in such a short time. Because Anton is one of our “variable” children, we don’t have access to the information behind the data to interpret the results of the analysis further. To understand the nature of this degree of change in a real child we would now go back to the wording of the scale that underlies this graph. What exactly was goal X and what are the skills associated with achieving it? What does it mean to have started at an average score of 1.4 in January and what does it mean to have ended up at an average score of 3.6 in March? Is this a shift in an emerging goal or does this show solidification of a skill across multiple environments? Without going back we cannot go forward—we can’t use our interpretation to think about what to do next. Although we have gone as far as we can in Anton’s case,

116

DETECTING CHANGE – THE MEAN

remember that what we do with statistical information is part of the larger on-going process of intervention. We will have a chance to explore the whole process in Chapter 10 when we apply the analytical tools we are developing here to the cases of Joey, Tyler, and Mai Lin. Let’s skip Becca for a moment, and jump ahead now to Celeste’s data. Recall that Celeste’s performance produced the most difficult line graph to interpret in Chapter 6. The comparison of means in Figure 7.8 makes the situation clear: the change we expected has not occurred. We know this because we must apply the same rules to Celeste’s data that we applied to Anton’s data. Given a qualitative scale with behaviorally distinct steps, different categories of behavior are defined by a numerical difference of at least 1.00. The difference in Celeste’s means is in a promising direction—perhaps Celeste is just entering the transformation stage—but the size of the difference (0.60) falls short of our boundary condition (a difference of at least one category width). Because it is much less than a category width (1.00), the difference is too small to justify the idea that there has been a “reliable” category change.

4-point scale (1,2,3,4)

Comparison of Means, Celeste, Goal X 4.0 3.0 1/3-1/26

2.0

2/18-3/10

1.0 0.0 Before (2.4) versus After (3.0)

Figure 7.8 Celeste’s performance on goal X “before” and “after” intervention

The question of just what we mean by “reliable” is at the heart of the analysis of Becca’s performance on goal X. The graph comparing Becca’s “before” and “after” means (2.1 and 3.1) is shown in Figure 7.9. Without the word “reliable” in Step 3 of our method, we could conclude that Becca’s data matches the pattern of change we hypothesized and move confidently on to interpreting the change in terms of the original scale. But our confidence has to be tempered by the actual value of the difference of the means (1.00), which is just enough to meet our boundary condition. Remember that the mean is a representative value indicating, on the average, what the child’s behavior was over a predicted stage of the intervention. Just as in the case of dividing the data into thirds, we need to 117

FROM GOALS TO DATA AND BACK AGAIN

have a consistent, well-defined boundary condition for category change; but when our interpretation relies completely on that arbitrary boundary, we need more evidence.

4-point scale (1,2,3,4)

Comparison of Means, Becca, Goal X 4.0 3.0 1/3-1/26

2.0

2/18-3/10

1.0 0.0 Before (2.1) versus After (3.1)

Figure 7.9 Becca’s performance on goal X “before” and “after” intervention

Without question there are other kinds of evidence we should always consider: parent report (“She’s really much more social than before the holidays. She joins the rest of the family after dinner almost every night now.”), teacher report (“Yes, I’ve started to see her interacting with the other kids at lunchtime, too.”), and so on. These typical sources are informative and important, but subjective. Since one of the reasons we use data is specifically to provide a different (objective) source of information, it would be a shame to give up on this new source after so much work. Luckily, our statistical toolbox has just the right-sized hammer for the problem. If the mean establishes a representative value for performance, its partner, the standard deviation, describes just how representative that value is. Standard deviation is the tool we use to measure the reliability of change. It is the topic of the next chapter.

Things to remember T Change is the transformation of one regular pattern of behavior into a

different regular pattern of behavior.

T Regularity has predictive value. This means that when we notice a

regular pattern we expect that pattern to continue and rely on that pattern to predict future behavior.

T Data analysis depends on our expectations; our expectations depend on

our knowledge.

118

DETECTING CHANGE – THE MEAN

T Noise is the phenomenon that hides regularity. Real behaviors, like real

children, are inherently noisy.

T To compute change for a goal:

1.

Divide the data for that goal into thirds. The first third of the data corresponds to “before” intervention; the last third of the data corresponds to “after” intervention. Ignore the middle third of the data, as it corresponds to the period of transition.

2.

Compute the mean of the “before” data and the mean of the “after” data independently.

3.

Compare the two means in terms of the scale for the goal. If the two means reliably reflect different categories then change has occurred.

T Whether the data matches our expected pattern of change is something

we can determine from a comparison of means (and standard deviations). But to understand the pattern of change in behavioral terms we must go back to the underlying meaning of the data that is found in the wording of the scale and the original discussion that led to the goal being measured.

119

Chapter 8

Measuring Reliability — The Standard Deviation

There are two reasons we chose a comparison of means as the statistical measure for detecting change. First, we recognize that the circumstances under which parents, teachers, and therapists collect data don’t satisfy the conditions for using more powerful statistical tests. Second, because the mean acts as a representative value for a set of observations, using it seems to be at least a partial solution to the problem of dealing with noise in a uniform and repeatable way. Our choice introduces a new problem, however. While it’s true that every set of numbers has a mean, there is nothing about the way we compute the mean that guarantees that it’s a good representative value, nothing that guarantees that it will allow us to predict behavior accurately in the future. How certain can we be that the information given by a comparison of means reflects a reliable change in behavior? How well each mean represents its values determines how certain we can be of the story our data tells. The standard deviation measures how representative the mean is, and therefore measures the certainty of our story.

Distance as a measure of representation When we vote for political candidates, we expect that, if elected, they will represent us by making the same sort of decisions we would make in their place. Realistically, we don’t expect every decision to be made exactly as we would have made it. Instead, our sense of how good or bad our representatives are depends upon how often they make decisions the way we would or how different their decisions are from what ours would have been. Typically, the language we choose to reflect our judgment of how well a candidate represents us is based on a distance metaphor. So, for example, we might talk about “how far apart” our views are or “how close we are” to the candidate on a particular issue.

121

FROM GOALS TO DATA AND BACK AGAIN

When judging whether the mean is a good representative for a set of observations, the intuition is similar. Of course in intervention, our candidate (the mean) has to be faithful to its constituency (the data) on only one issue: representing the child’s behavior for a particular goal. And although the concept of distance is used metaphorically in the political arena, in statistics distance is a real and measurable thing. To further our intuition about the relationship between distance and representation, let’s look at two different data sets. The first, Figure 8.1, shows Anton’s scores for goal X during the post-transformation phase of intervention and a line at the mean value (3.6) for this data set. Observations versus Mean

4-point scale (1,2,3,4)

Anton, Goal X

5 4 3 2 1 0 1

2

3

4

5

6

7

8

9

10

Session in "After" phase

Figure 8.1 Good representation displayed as a tight clustering of data points around the mean line

There are two things we want you to notice about this figure. First, each observation is close to the mean. In fact, the largest difference between the mean and any observation is 0.6, which is less than the size of one category width on the rating scale. The second thing to notice is that the mean line is actually closer to the values of 4 than to the values of 3. To push our political metaphor, we might say that when called upon to serve, the mean would act a little bit more like a 4 than like a 3. Since there are more 4s than 3s in the data set this makes sense: a value that is closer to 4 is more representative of the set than one that is closer to 3 or exactly half way between 3 and 4. The small distances between the data points and the mean make the mean value for Anton’s data a good representative of Anton’s performance in the past. Because of this, we feel confident using it to predict Anton’s performance in the future. By “electing” the value of 3.6 to represent his data we are saying that 122

MEASURING RELIABILITY – THE STANDARD DEVIATION

from this point forward we expect to see “votes” of 3 and 4, reflecting continuing regularity, in Anton’s behavior. Now let’s look at Figure 8.2, which contrasts Celeste’s post-transformation scores for goal X with the mean (3.0) for those values. Although the mean for Celeste’s data was computed with exactly the same formula as the mean for Anton’s data, the observations don’t seem to cluster around the line the way they did in Figure 8.1. Although a few values lie on the mean line, the smallest distance between any of the remaining observations and the mean is 1.00, which is both a full category width and almost twice as big as the largest distance in Anton’s data. Suppose you were the value of 1 observed in the fourth “after” session. Would you have elected 3 as your representative?

Observations versus Mean

4-point scale (1,2,3,4)

Celeste, Goal X

5 4 3 2 1 0 1

2

3

4

5

6

7

8

9

10

Session in "After" phase

Figure 8.2 The more the data points spread out around the mean line the poorer the mean is as a representative value

The large and varied distances between Celeste’s data points and her mean indicate that the mean is a poor representative for its “constituency.” Its value is a compromise among data points that have little in common. Because of this, we cannot be confident that the mean will be a good representative of Celeste’s behavior in the future, either. We should not use it to predict values of 3, or any other regularity, in her behavior going forward.

123

FROM GOALS TO DATA AND BACK AGAIN

Computing and graphing the standard deviation The standard deviation is a number that reflects our notion of representation as a function of distance from, or clustering around, the mean. The formula for this statistic is rather ugly and cumbersome if you are not a computer. Fortunately, we can let Excel handle the actual computation by applying the STDEV function in our spreadsheets. STDEV is taught in Exercise 8.1 (Appendix C). Since our method for judging change involves two means, we also need to compute two standard deviations, one for the “before” data and one for the “after” data set. Over time you will come to recognize the presence or absence of change directly from these four numbers. As you build that skill, however, you may find it helpful to look at a visual representation of the critical information; and, of course, it may be important to have a visual for explaining your analysis to others on the team. In Chapter 7 you learned how to produce bar graphs in Excel in order to contrast the “before” and “after” means. To show the standard deviation around each mean we augment the bar graph with error bars. An example is given in Figure 8.3. Exercise 8.2 takes you step-by-step through the process. Notice that the error bar for each mean extends both above and below that mean by the amount of its standard deviation. Also note the area of overlap between the bars indicated by the oval. This overlap will help us determine the reliability of change.

Comparison of Means mean + standard deviation

4-point scale (1,2,3,4)

6.0

mean – standard deviation

mean

4.0 2.0 0.0 Before

versus

After

mean = 2.6

mean = 3.9

sd = .85

sd = .92

Figure 8.3 Standard deviations displayed as error bars around the means. The bar extends both above and below the mean by the amount of the standard deviation. The oval indicates the area of overlap between the bars 124

MEASURING RELIABILITY – THE STANDARD DEVIATION

Evaluating change using mean and standard deviation The method of data analysis we introduced in Chapter 7 was problematic because it did not give a concrete definition of reliability. Now that you know how to compute the standard deviation for a set of observations, we can reformulate the method to remove that problem.

Computing the comparison of means statistic

To compare the “before” and “after” means for each goal: 1.

Divide the data for the goal into thirds. The first third of the data corresponds to “before” intervention and the last third of the data corresponds to “after” intervention. Copy the “before” and “after” data to a new spreadsheet. Ignore the middle third of the data.

2.

Compute the mean and standard deviation of the “before” data and the mean and standard deviation of the “after” data separately.

3.

Compare the two means in terms of the scale for the goal. If the difference between the means is not a full category width apart, assume that there has been no reliable change.

4.

If the two means are at least a full category width apart, graph the means and standard deviations and compare the reliability of the change by looking at the amount of overlap in the error bars.1 (a)

If there is no overlap conclude that the difference in means signals reliable change and a new regular pattern of behavior. If the “after” standard deviation is more than about half a category width consider continuing intervention to achieve consistency in the new skill or behavior. Otherwise, the goal is achieved in its current form.

(b)

If there is overlap, it is crucial to consider outliers and correlations (Chapter 9), as well as all the other sources of information available before coming to a conclusion. If no additional information is available, however, use the following guideline: • If the overlap is less than 1/3 the difference in the means, assume that the overlap reflects normal variability in performance and conclude that reliable change has occurred.

1

For those who prefer numbers to pictures, use the following formula: Overlap = Meanbefore + Standard Deviationbefore – (Meanafter – Standard Deviationafter).

125

FROM GOALS TO DATA AND BACK AGAIN

As in (a), if the “after” standard deviation is large, consider continuing intervention to achieve consistency.

• If the overlap is greater than or equal to 1/3 the difference between the means, assume that the change in behavior is not yet reliable. This procedure is also available as Chart 8.1 in Appendix B. Step 4b of this procedure is complex because the issues surrounding reliability can be complex and a judgment about the child’s performance requires both thought and discussion. Your decision—whether to ignore the overlap, to continue with the goal in modified form, or continue with the goal as is—must be sensitive to both the degree of overlap and why that overlap arises. In general, the more overlap there is, the less likely it is that reliable change has occurred. But in a specific instance there may be reasons why the overlap can exist even when reliable change has occurred. The two most common reasons (outliers and factors that systematically interfere with performance) are discussed in detail in the next chapter. However, even if those reasons do not explain the overlap, the team may identify other reasons for the child’s variability in performance through discussion and such discussion should be considered seriously. Remember: data analysis provides only one source of information. When all else fails, the team can choose to rely on the boundary value of 1/3 for their decision making. There is nothing magical about using this fraction—we need to choose some number to keep the analysis consistent and repeatable. Some teams may feel that this fraction suggests reliable change too easily, other teams may feel that it is too rigid a criterion. What is important is that you choose a value and use it consistently. What is more important still is that you understand what the boundary condition means. We add error bars to the graph to indicate how good a representative value the mean is and how much regularity there was in the data set it represents. If the means are at least a category width apart and they are good representatives of the data, then we can believe in the behavioral change the data signals. But, if the means are not good representatives of the data, we need to know just how poor they are. The poorer the representation, the larger the standard deviation will be. As the standard deviation around each mean grows, the amount of overlap between the error bars increases. If the standard deviations/error bars around each mean are large enough relative to the distance separating the means then we can no longer rely on the change in mean to indicate a change in behavior. Let’s see if this really makes sense. Figure 8.4 contrasts three graphs. In each graph the “before” mean and the “after” mean are the same, but as we proceed top to bottom through the figure the size of the standard deviations around the means grows. In the top graph, the top of the “before” mean error bar rises just a little bit above the bottom of the “after” mean error bar. In the middle graph, the overlap is larger—about a third of the total difference between the means. In the 126

MEASURING RELIABILITY – THE STANDARD DEVIATION

bottom graph the overlap is almost as big as the entire difference between the means: the top of the “before” error bar comes up almost as far as the “after” mean and the bottom of the “after” error bar comes down almost to the “before” mean. Glance back at Figures 8.1 and 8.2 to remind yourself visually that a bigger standard deviation means less clustering of observations around the mean. Now think about the fact that the observations represented by Figure 8.4 cannot be smaller than 1 or larger than 5 (given the 5-point scale). So, as the standard deviation grows around a “before” mean that is close to the bottom of the scale, the values that are very different from the mean are probably large values. Similarly, as the standard deviation grows around an “after” mean that is close to the top of our scale, the values that are very different from that mean are probably small values. To put it another way, as the standard deviations get bigger, the data in the pre-transformation phase and the data in the 2 post-transformation phase have more values in common. The overlap in area of the graph corresponds (very roughly) to an overlap in the data sets. The more the data sets share values, the less it seems that there are the two different regular patterns of behavior required by our definition of change. The graphs in Figure 8.4 were chosen to make a point; they are not the only patterns that you might see in your data. In the remainder of this chapter and again in Chapters 9 and 10 we examine and discuss a number of additional scenarios. But we don’t have the room (and you probably don’t have the patience) to examine all the possible cases. So instead of presenting hundreds of examples, we’ve set up a small laboratory in Exercise 8.3 to help you develop a feel for the relationships among raw data, mean, and standard deviation. We encourage you to spend some time experimenting in the lab before you continue. OK, we’re finally ready to tackle the problems that pushed us to examine the standard deviation in the first place: Anton, Becca, and Celeste’s data sets. As usual, Anton is our poster boy for change. His “before” and “after” means are two full categories apart and, when we add the error bars around the means, there is no overlap (Figure 8.5). His “after” standard deviation is much less than a category width. All in all, Anton’s data gives every indication that his behavior has changed reliably and settled in to a new, more desirable pattern. If, for example, goal X is measuring emergence of a skill with support, it looks like it’s time to move this goal into Phase 2, reducing support and solidifying the skill 2

If the two means are not close to the top and bottom of the scale then they are less likely to have at least a category’s width between them. If they are close to the middle of the scale and still a category apart, the argument is more complicated but the conclusion is the same: the more overlap, the less you should trust the reliability of the change. For those of you who are worried about the effect of outliers on the comparison of means, we discuss outliers in Chapter 9.

127

FROM GOALS TO DATA AND BACK AGAIN

A small overlap in the standard deviations 5 4 3 2 1 0

The same means, but some overlap 5 4 3 2 1 0

The same means with a large overlap 5 4 3 2 1 0

Figure 8.4 The more overlap in the error bars, the less reliable the change indicated by the difference in means

128

MEASURING RELIABILITY – THE STANDARD DEVIATION

with more natural rewards. If the goal is already in Phase 2 and the scale measures progress in a familiar surrounding, it looks like it’s time to consider Phase 3 for this goal, generalizing it to new environments. However we decide to build on Anton’s progress, we can at least feel comfortable predicting that “if nothing changes” his behavior will continue (under the current goal qualifiers) in the new, more adaptive way. Becca’s data was a problem in Chapter 6 and remained a problem in Chapter 7. Recall that the means for her “before” and “after” data sets were exactly one category width apart. Since that’s a boundary value in our statistical method, an interpretation of progress could have gone either way depending largely upon the evidence contributed by other sources. Adding standard deviation information to her graph, however, argues against an interpretation of reliable change (Figure 8.6), as the overlap in the error bars covers nearly all of the difference between the means. Comparison of Means, Anton, Goal X

4-point scale (1,2,3,4)

5 4 3 2 1 0 Before (mean=1.4, sd=.52

After mean=3.6, sd=.52)

Figure 8.5 A clear example of change

If we look at Becca’s data (“Becca Goal X Before–After” in the ABC Workbook), it’s clear that the overlap in the standard deviations around Becca’s “before” and “after” means comes from two sources. First, there are a few values of 3 in the pre-transformation phase that would look more at home in the “after” data set. Second, there is a value of 1 and a value of 2 in the post-transformation phase that are more indicative of pre-transformation behavior.

129

FROM GOALS TO DATA AND BACK AGAIN

4-point scale (1,2,3,4)

Comparison of Means, Becca, Goal X

6.0 4.0 2.0 0.0 Before

After

(mean=2.1, sd=.74 mean=3.1, sd=.99)

Figure 8.6 The large standard deviations argue against the evidence of change provided by the difference in the means

On the surface these results look like Becca might be on the way to a new regular pattern of behavior but needs more practice. Another way of saying this is that Becca’s post-transformation phase may have been longer than we expected. If we keep providing intervention for this goal we will have the opportunity to extend our view over time, gathering more data as we did for Danny in Chapter 7. And as we saw in that example, extending our view can push a portion of the data more solidly into the transformation stage where its lack of regularity fits our expectations. So, if Becca is on the way to lasting change and we continue intervention, then we would expect to see a smaller standard deviation around the “after” mean of the extended data set as her new behavior becomes reliably consistent. If you find yourself questioning the story we just told about Becca’s graph, arguing that there are other explanations for these results…good for you! No interpretation can be considered complete until we’ve gone back and looked at the original ideas and arguments that caused us to choose the goal and scale that underlie the data. Of course, since Anton, Becca, and Celeste are cases we made up for illustrative purposes we don’t have that information and can’t perform that critical step of the analysis. Don’t worry, though; we’ll have the chance to complete three interpretations in Chapter 10 when we analyze Joey, Tyler, and Mai Lin’s data. In the meantime, let’s turn to Celeste. Celeste’s data failed the test for change at step 3 of the comparison of means, so adding error bars to the graph we created in Chapter 7 isn’t strictly necessary. Still, it is useful to note that the tale told by the standard deviations is consistent with our previous interpretation. Figure 8.7 shows that what difference there is in the means is overwhelmed by variability in behavior both “before” and “after” intervention. We design intervention based on the expecta130

MEASURING RELIABILITY – THE STANDARD DEVIATION

tion that the three phases of change can occur in terms of a given measurement scale and within a given interval of time. In Celeste’s case this expectation has not been met.

4-point scale (1,2,3,4)

Comparison of Means, Celeste, Goal X

6.0 4.0 2.0 0.0 Before

After

(mean=2.4, sd=.84 mean=3.0, sd=1.05)

Figure 8.7 Comparison of means for Celeste

Perhaps Celeste is not ready for goal X as it is currently qualified. She may not be ready to act adaptively without a great deal of support. She may not be ready to use the new skill outside of familiar settings or with unfamiliar people. On the other hand, perhaps the type of quantification chosen for goal X is the problem. If the scale contains categories that are too far beyond Celeste’s reach at this point in time, she may be responding to intervention but over too small a portion of the scale to be noticeable. The point we want to make here is that although analyzing the data does not give us the information we expected, it gives us useful information nonetheless. In Celeste’s case it signals to the team that it is time to re-evaluate the assumptions that underlie the choice of this goal. So, we’re done, right? With graphs in hand we’re ready to add our data analysis to the other sources of information presented at the next team meeting. Actually, we’re close, but no, we’re not done quite yet. While we have wrung pretty much all the information we can out of the goal data alone, there’s still the factor data to consider. What it has to add to the story is the topic of Chapter 9.

131

FROM GOALS TO DATA AND BACK AGAIN

Things to remember T The standard deviation is a statistic that measures representation as a

function of distance from, or clustering around, the mean. The smaller the standard deviation, the better the mean is as a representative of prior behavior and the better it predicts future behavior.

T If the standard deviations around the “before” and “after” means are large

enough relative to the difference separating the means then we can no longer rely on the difference in mean to indicate a reliable change in behavior.

T Not even the most compelling graph of the “before” and “after” means

and standard deviations completes an analysis. Every interpretation requires that we go back and look at the original ideas and arguments that caused us to choose the goal and scale that underlie the data before making the next set of intervention decisions.

132

Chapter 9

Exceptions That Prove the Rule — Factor Data

Like all statistical methods, the comparison of means is based on the idea of finding similarities and differences between the data we expected and the data we collected. When we analyze data we do it against the backdrop of those expectations. So far our method allows us to recognize the pattern of change that is typical of developmental learning when it appears in the data over a given time period. But what happens when our expectations aren’t met? The tools in our computational toolbox—graphing, mean, and standard deviation—are good ones to have when things go right, but they aren’t particularly useful when things go wrong. When we don’t see the behavioral growth we’re expecting, we’d like to be able to tell a better story than just “too much noise” or “no change.” We’d like to know if there are reasons behind the apparent lack of progress. We’d like to know if there are regularities hidden within the noise. Having had the forethought to collect factor data, now is the time to look to that source of information to help clarify the picture.

Identifying an outlier in the data set Figure 9.1 shows the means for Anton’s data for goal Y. The standard deviation of the “after” data is large and overlaps a bit of the space marked off by the error bar of the “before” data. Applying the comparison of means method, we know that we must consider other relevant sources of information before attributing developmental change to these results. It’s possible that the large standard deviation in the “after” data does reflect ongoing difficulty in the consistent use of the new skill. When we look at Figure 9.2, however, we see that another possibility exists. The values for goal Y in most of Anton’s last ten sessions are exactly what we would expect if Anton’s progress was solid. The exception, a value of 1, appears in Session 27. This unusually low value has two effects on the comparison of means in the previous figure: it pulls 133

FROM GOALS TO DATA AND BACK AGAIN

the “after” mean down and it contributes most of the standard deviation around that mean. A single value that stands out against an otherwise regular pattern of behavior is called an outlier.

Comparison of Means, Anton, Goal Y

4-point scale (1,2,3,4)

5.0 4.0 3.0 2.0 1.0 0.0 Before

After

(mean=1.9, sd=.57 mean=3.4,sd=.97)

Figure 9.1 Overlap in the error bars suggests we look further

4-point scale (1,2,3,4)

Anton, Goal Y

6 4 2 0 1

2

3

4

5

6

7

8

9 10

21 22 23 24 25 26 27 28 29 30 Sessions

Figure 9.2 Post-transformation behavior (sessions 21 through 30) looks regular except for a single unexpected low value in Session 27

If we recompute the means without the observation in Session 27 (see Figure 9.3), Anton’s data set for goal Y looks every bit as clear and exciting as his data for goal X did in the previous chapter. As tempting as it is to simply ignore the outlier and move onto Anton’s next goal, however, good practice demands that we look for some explanation for his performance in Session 27 before dismissing it as uncharacteristic. Broadly speaking, there are three possible explanations for Anton’s difficulties: 134

EXCEPTIONS THAT PROVE THE RULE – FACTOR DATA

4-point scale (1,2,3,4)

Comparison of Means, Anton, Goal Y, Revised (session 27 omitted)

5.0 4.0 3.0 2.0 1.0 0.0 Before

After

(mean=1.9, sd=.57 mean=3.7, sd=.50)

Figure 9.3 The expected pattern becomes clear after removing the outlier

1.

His score is related to a goal-independent factor we are already tracking.

2.

His score is related to a goal-independent factor we are not tracking.

or 3.

His score is related to a goal-dependent factor of which we are unaware.

Recall that we track particular goal-independent factors like time of day, observer, location of intervention, and medication regimen because we already have good theoretical or empirical reasons for believing that those factors can affect performance in children with autism. In other words, our willingness to spend the time and energy tracking those factors is due to a tentative expectation (or hypothesis) that they could explain performance in this particular child as well. To test the hypothesis that Anton’s poor performance might be explained by a factor we are already tracking, we look for a factor that, like goal Y, has a value for Session 27 that is different from its typical values. We might find, for example, that the day of Session 27 was the only time in the “after” phase that Anton missed his medication or that he received intervention in the afternoon. If we did find that Session 27 was the only day that intervention was switched to the afternoon, our hypothesis would be supported. We might conclude that Anton is a “morning person,” at least with respect to the skills involved in goal Y. Then, having some justification, we may use Anton’s recomputed means (Figure 9.3) as the more realistic representation of his progress and plan the next phase of goal Y accordingly. In addition, the team now has some 135

FROM GOALS TO DATA AND BACK AGAIN

evidence for the usefulness of this goal-independent factor in predicting Anton’s behavior and might choose to spend some time and energy to understand it more fully in order to provide adaptations or intervention designed to extend his competency in the afternoons. Suppose, however, that there were days in the “after” phase when Anton had intervention in the afternoon and his performance on goal Y did not suffer. Such observations would violate our hypothesis that time of day might account for his performance in Session 27 and force us to explore alternatives. If we find that none of the factors we are tracking seems to offer much of an explanation for Anton’s bad day then we must consider whether there is some other goal-independent or goal-related factor that we should be measuring. In other words, an outlier can be seen as an opportunity to form a new hypothesis about factors that affect progress. In trying to form a new hypothesis it may be helpful to look at Anton’s performance on other goals during the same session. For example, if his performance seems to have suffered across the board on that day, then the data suggest that the relevant factor is goal independent. But if, for example, the scores for social goals were uniformly lower in Session 27, but those for communication goals were not, this suggests that Anton’s sensitivity might be to some aspect of the social activity that isn’t required by the communication skills. Another possibility is that goal Y contains the only clear outlier, suggesting that the difficulty is related to something specific to that goal. Additional information might be found in Anton’s data collection sheets in the form of a notation or comment that identifies an aspect of the skill or situation that was unusually problematic. Or it may be that team members who were present on that day can recall the events and identify a possibility. If a hypothesis can be formed, the team must consider next whether to track the new factor to test the hypothesis, or target the new factor intentionally in a revised version of the current goal. If no information is available, however, then the team must decide between two alternatives: to take the data as is, continue intervention for a while and then re-evaluate, or to discount the outlier and simply move the goal to its next phase. Remember that an outlier is just a single value that stands out against an otherwise regular pattern. Nothing in this definition requires the outlier to be a particularly low value for a goal. In fact, an unusually high value can also make progress look worse than it is if it occurs in the “before” portion of the data where it would increase both the mean and standard deviation in the “before” phase. The effect on the comparison of means, then, would be the same as that produced by an unusually low value in the “after” phase: there will be less of a difference between “before” and “after” due to a single value. An unusually high score may also exaggerate performance, making the child’s gains look better than they probably are. This happens when the high 136

EXCEPTIONS THAT PROVE THE RULE – FACTOR DATA

value occurs in the “after” data, pulling that mean up. You don’t want to let exaggerated progress mislead you into refocusing intervention too soon. Of course, paying attention to an outlier that is uncharacteristically high may be even more important than paying attention to an outlier that is uncharacteristically low because explaining an unusually high value may lead you to identify a factor that improves performance. Such a factor can give you insight into the child’s strengths that can, in turn, lead to more effective adaptations and intervention. What if there is more than one outlier in the graph? With large “before” and “after” data sets you might see more than one point that stands out against an otherwise clear pattern of behavior. If you find you need to explain each outlier with a different factor, be wary. Every new explanation you must bring to the analysis should make you that much less certain that you’re looking at a regular pattern in the child’s behavior. If, on the other hand, the multiple outliers seem to be explained by the same factor—that is, the values of the factor are the same on the outlier days and different from the values of the factor on non-outlier days—then you’re not really looking at multiple outliers, you’re looking at a consistent sub-pattern within the data.

Predicting sub-patterns of behavior from factors By definition an outlier is identifiable because it isn’t part of a pattern. Sometimes, however, a factor has influenced behavior in so many sessions that it obscures the pattern of change we are looking for by weaving a pattern of its own into the data. The statistic for detecting when this happens is correlation. The basis of this statistical method is very similar to the reasoning we went through in the previous section: we say that two variables are correlated when variation in the value of 1 variable seems to predict variation in the value of the other variable. Although we will usually be looking for a correlation between factor data and goal data, in general it is possible for two factors or even two goals to be correlated. Correlation is a useful tool because of its predictive power. We take advantage of the predictive aspect of correlation every day when, for example, we choose a route to take to work or a movie to see at the theater. How do these decisions rely on correlation? Suppose there are a few different ways to drive from home to work all covering more or less the same distance according to the odometer. Nevertheless, over time we notice that which route we take makes a big difference in how long it takes to get there. What we’ve discovered is a correlation between route and travel time. The correlation isn’t perfect—we can’t predict with certainty that on a given day the “best” route will be the fastest. There may be a detour or an accident or especially heavy traffic. But we assume, all other things being equal, that variation in route predicts variation in arrival time in that one route gets us to work earlier than the others.

137

FROM GOALS TO DATA AND BACK AGAIN

Similarly, our experience may lead us to notice that there is a high correlation between an actor and our enjoyment or dislike of a movie. Again, the correlation isn’t likely to be perfect—not even the casting of a favorite actor can save some movies and sometimes actors we dislike occasionally appear in great films. All other things being equal, however, we assume that the particular value of one variable (actor) predicts a better or worse outcome in another variable (our movie-going experience) when we make our choice. The phrase “all other things being equal” is an important aspect of the examples given above. We don’t, in the driving example, use correlation to choose between highway roads and surface roads. Highways and surface streets are in different categories with respect to our expectations and predictions. We know that highways are generally faster than surface streets for medium or long distances and can rely on that knowledge independent of our experience with particular routes of either type. In the same way, we don’t typically decide among movies on the basis of actor when one movie is a drama, one is a comedy, and the third is a horror film. Again, the differences among the categories seem to be more important in making the decision than a variable that might correlate within each category. This same reasoning applies to our developmental data. In general we look for a correlation between variables only within each phase of change because the phases themselves are assumed to define substantially different categories (“before” intervention, during learning, and “after” intervention).

Correlation is not cause It is extremely important to understand that the ability to predict behavior is not the same thing as understanding the cause of the behavior. It would never occur to you to think, for example, that the road itself is what causes you to be on time. You know from your commonsense understanding of the world that the real cause of when you arrive at work is a complex combination of factors like weather, traffic patterns, stop light timing, bus stops, etc. These factors are also the reason why there is a correlation between route and arrival time. Unfortunately, when we don’t have a good understanding of the causes behind a correlation, we often make the mistake of believing that the factor we use to predict behavior is also the cause of that behavior. Here’s an example we see far too often: a child (we’ll call her Kyra) who has a social goal (for example, to share toys with peers) is supposed to take medication every morning. When we look at Kyra’s data we find a high correlation between medication and sharing. In other words, when Kyra takes her medication she does well on the social goal and when she misses her medication she does poorly on the social goal. Given this result we would be correct if we predicted that on days when Kyra does not take her medication she will have difficulty sharing toys with her school peers. We would not be correct, however, if, solely on the basis of this data 138

EXCEPTIONS THAT PROVE THE RULE – FACTOR DATA

and correlation, we concluded that the lack of medication caused her poor performance (or that the medication caused her skilled performance on days when it was taken). We would even be incorrect if we concluded that the lack (or presence) of medication contributed to her poor (or good) performance. How can that be? Well, suppose, for example, that on days when Kyra wakes up feeling agitated and experiencing a lot of tactile sensitivity her tantrums are so severe at medication time that her parents decide not to fight that particular battle. When Kyra gets to school, the same tactile defensiveness makes it impossible for her to be near the other children, listen to their requests, and share her toys. In other words, the same difficulty that caused Kyra to miss her medication (her sensory system) also causes her poor social performance. So, even though missing her medication did not cause or contribute to her poor social performance, it will be correlated with it. Although the driving and medication examples look different on the surface, underneath they make the same point: there is always the possibility that the correlation between 2 variables reflects nothing more than a shared common cause. Keep these examples in mind the next time you find yourself tempted to claim a causal relationship between a factor and goal performance on the basis of the factor’s predictive value alone.

Translating factor data to numbers and computing correlation The statistical function we use for detecting whether there is a correlation between a factor and observed behavior is the Pearson r. There are other statistics that can be used depending upon whether the data you’ve collected is categorical, ordinal, or interval-based, but the PEARSON r is available in most spreadsheet programs and can be used for all these types (with some caution, as explained below). Exercise 9.2 takes you step-by-step through the use of this statistical function in Excel. Recall that when we converted our data from collection sheets to spreadsheets we delayed the translation of non-numeric factors into numbers. Before we can use the Pearson r we must complete that step. There are two cases: 1.

When the type of the factor is categorical (nominal) we typically assign 1 to the first category/name we come across in the data, two to the second category/name, and so on, as shown in Figure 9.4. (This straightforward approach works fine when there are only two categories but can lead to an unintended hypothesis when there are more than two categories, as discussed in the section “You get what you ask for,” below).

2.

When the type of a factor is ordinal or interval we assign numbers in a sequence that preserves the order information, just as we did when we 139

FROM GOALS TO DATA AND BACK AGAIN

converted our original qualitative scales to raw data (see Chapter 6). Even so, some factors can be translated in more than one way. Consider session date as an example. The date of each session can be translated into many different ordinal time scales. For example, we could translate session dates into session numbers with the earliest date being given the number 1, the next date being given the number 2, and so on. Or each session date might be translated into a number between 1 and 7 to represent its day of the week. We might even decide to use session date to distinguish weekdays from weekends by assigning 1 to Monday through Friday sessions and 2 to sessions that fall on Saturday or Sunday. Which of these choices we make will affect the degree of correlation, a fact we look at in more detail shortly.

Figure 9.4 Translation of the categorical factor specifying location into numbers

The result of computing the Pearson r is always a number between –1.00 and +1.00. Ignore the sign for the moment (we’ll discuss it in the next section). Just looking at the number, we use the following guideline to characterize the degree of correlation between 2 variables: 140

EXCEPTIONS THAT PROVE THE RULE – FACTOR DATA

· If the Pearson r is less than 0.55, there is no correlation. · If the Pearson r is between 0.55 and 0.74, there is a moderate

correlation.

5-point scale

· If the Pearson r is between 0.75 and 1.00, there is a strong correlation.

Comparison of Means, Becca, Goal Y

5-point scale

6.00 4.00 2.00 0.00 Before

After

(mean=2.0, sd=.55 mean=3.4, sd=1.06)

Figure 9.5 Is Becca still learning or is there something interfering systematically with her performance?

As with the boundary conditions in the comparison of means, these numbers should never be taken as absolutes. Other sources of information may suggest interpreting a value near a boundary as belonging to the weaker or stronger cor1 relation category. 1

These values were chosen assuming that the “before” or “after” data set has about 10 values in it. If you have fewer than 10 values, r must be higher to reach the same conclusion; if you have a lot more values, r can be a bit lower.

141

FROM GOALS TO DATA AND BACK AGAIN

All right, let’s look at some examples. If you open the ABC Workbook to the sheet titled “Becca Goal Y,” you will see the full data set for the graphs in Figure 9.5 as well as the location data specifying where Becca received intervention. This factor is categorical and the values for the factor are shown in their original and translated forms. Becca’s “before” data for goal Y looks pretty typical but her “after” data and the comparison of means seem to indicate that the goal requires continued intervention. We would like to understand if the irregular behavior in the period that we expected to be “after” learning is really due to inadequate practice or if it might have some other explanation. Since the line graph for the data in the “after” phase cannot be characterized as a pattern disturbed by an outlier, we must consider whether there could be some more systematic interference with Becca’s performance. Specifically, what happens if we consider the location factor? Might Becca’s performance on this goal be better on days when she receives intervention at home rather than at playgroup, or vice versa? Calculating the correlation between location and goal Y we find little evidence for a predictive relationship between location and behavior in the “before” data (r = 0.56), but a strong correlation in the “after” data (r = 0.82). In other words, it would seem that we can predict something about how Becca will do on goal Y at this point in her learning simply by knowing where intervention will occur. What is the specific prediction? Looking at Becca’s spreadsheet we see that most of the time her low scores on goal Y are paired with location values of 1. Since the categorical value of “home” was assigned to 1, the data is telling us that she does more poorly at home for this goal than she does when she is at playgroup. This is a fairly unusual outcome—we expect a child to perform well at home before she can move a skill into an outside environment. Becca’s example gives us the opportunity to point out, more concretely, why we calculate the correlation separately for the “before” and “after” data. Although the predictive value of location is strong for goal Y during the “after” phase of intervention, the same factor held little predictive value in the “before” phase. How can that be? A possible explanation is that the new goal was difficult enough that Becca’s struggle to perform during the initial phase of intervention didn’t depend on where the intervention was given. After Becca began to catch on, however, there was something about the environment that made a difference in her ability to perform the skill. Either location 1 (home) contains some element that is making it more difficult for Becca, or alternatively, there is something about location 2 (playgroup) that makes the skill more accessible or easier to perform. Note that if we had computed a single statistic over the combined “before” and “after” data the result would have been r = 0.64, a moderate value that only masks the real pattern in the data. What do the results of the correlation analysis mean for the team? If there were enough data points they could calculate the “after” means for each location 142

EXCEPTIONS THAT PROVE THE RULE – FACTOR DATA

separately and draw their conclusions based on the revised analysis. Even though there aren’t enough data points to do this (we’d like at least 10 points in each location) the trend still seems clear. So, the team might decide to break goal Y into 2 goals during the next period of intervention. The two new goals would be qualified with respect to location and worked on and measured independently. At home, the focus would be to try to identify factors that interfere with Becca’s performance in order to adapt the environment (or teach additional skills) to overcome the interference. At playgroup, where the skill looks solid, the focus would depend on the next stage for goal Y: possibly performance with less support, for longer duration, or some similar extension. The Pearson r can uncover fairly complex relationships. Consider Celeste’s data for goal Z. Figure 9.6 shows the graphs for the data found on the sheet titled “Celeste Goal Z” in the ABC Workbook. This is the first analysis of percentages we’ve shown, so it’s useful to note that the mean and standard deviation are computed in exactly the same way for interval data as for the ordinal data we’ve worked with so far. The comparison of means is also computed in the same way, except that we haven’t defined yet what a “category width” is for this type of data. We are in a position to do that now. Remember that the comparison of means uses the idea of category width to define how the behavioral change we are looking for would be reflected in the data. In ordinal data like our 4- and 5-point qualitative scales, the minimum category width required for reliable change is 1.00 because that is the value between steps that define categorically different skill levels. We need to be true to this principle when defining category width for interval data as well. In the case of percentages, then, the definition depends on the nature of the behavior over which the percentages are calculated. If there were about ten opportunities to observe performance on the goal every session then 10% would be the minimum category width because each success can affect the percentage score by not less than 10%. If there were only five opportunities in a session then 20% would be the minimum category width because the child’s scores could only have been 0% (0/5), 20% (1/5), 40% (2/5), 60% (3/5), 80% (4/5), or 100% (5/5). As noted on her data sheet, Celeste had about ten opportunities per session to practice her skills for goal Z. This gives a minimum category width of 10% so the comparison of means (24.67% versus 37.33%) passes the initial analytical test for change: a difference of at least one category. The standard deviations are less than a category width and the error bars barely overlap so we are fairly confident that Celeste has made reliable progress on this goal. When we look at the line graphs for her data, however, we see an interesting pattern. It looks as if Celeste is having a recurring problem: over each set of five sessions, she does progressively better on goal Z, only to suffer a small setback during the first session in each new set of five. How can we see if this hypothesis is right? 143

% successful

FROM GOALS TO DATA AND BACK AGAIN

% successful

50.00 40.00 30.00 20.00 10.00 0.00 Before

After

(mean=24.67, sd=5.81 mean=37.33, sd=7.04)

Figure 9.6 Celeste seems to show a systematic regression (followed by recoupment) after every 5 sessions

To test the hypothesis we want to compute the correlation between the position of the session in a five-day sequence and performance on goal Z. We begin by assigning each session a number between 1 and 5, as shown in column C of Celeste’s spreadsheet. As with Becca’s data, we then compute the Pearson r separately for the “before” and “after” data sets because the effect of session position could well be different at different points in the development of the new skill. The correlation between session number and goal Z is 0.76 in the “before” data set, and 0.80 in the “after” data set—pretty good indications that this sub-pattern in Celeste’s data (increasing performance over each set of five sessions with regression between sets) is predicted by this factor. We now have statistical evidence for her cyclic behavior to go along with our more intuitive analysis. Given this information, the team is likely to consider it worth the effort to try to uncover what might be producing the correlation.

144

EXCEPTIONS THAT PROVE THE RULE – FACTOR DATA

Negative correlation Recall that when we claim two variables are correlated we mean that variation in the value of one variable predicts variation in the value of the other variable. So far we’ve seen only cases in which an increase in one variable predicts an increase in the other. In Becca’s “after” data the higher value for location correlated with the higher values for goal Y, while in Celeste’s case rising scores accompanied rising session numbers. When increase predicts increase (or decrease predicts decrease) we call it a positive correlation. We made an arbitrary choice when we translated the location factor in Becca’s data into numbers. We could have chosen the opposite assignment (1 for playground and 2 for home) without changing the meaning of the factor. Similarly, when we created a factor reflecting position in a five-day sequence for Celeste’s data, we simply chose numbers that corresponded to the meaning of the words we used to describe the pattern we saw (1–5 for the first through the fifth position in the sequence). But as long as we were consistent and chose an ordinal numbering scheme we could have done that assignment differently as well. Although it would have seemed odd, we could have numbered the positions in the sequence 5, 4, 3, 2, and 1 without changing the meaning of the data or the hypothesis we wanted to test. Reversing the way each factor was assigned numbers would have left the hypothesis intact, but what about the degree of correlation between each factor and its goal? Wouldn’t that have changed? No, it wouldn’t. The only effect of reversing the assignments would have been to reverse the relationship between the variables. Where increase had predicted increase, increase would now predict decrease. When an increase in one variable predicts a decrease in the other variable (or decrease predicts increase) it is called negative correlation and it is signaled by a negative sign in the Pearson r statistic. The degree of the relationship—how well one variable predicts the other—remains the same, only the direction—increasing predicts decreasing—changes. You can explore this phenomenon further in Exercise 9.3 (Appendix C). As we mentioned in the introduction to this chapter, a strong correlation can hold between any two variables, not just a factor and a goal. An interesting example of negative correlation between goals was brought to our attention by a teacher who had a student named Alex. Alex was a non-verbal boy in the teacher’s classroom. Alex tended to scream a lot. The screaming bothered one of the other students and certainly bothered the staff. Not surprisingly, Alex’s goals included a decrease in the incidents of screaming. Since the teacher felt that Alex screamed to gain attention, the plan was to ignore all such outbursts in order to discourage this behavior. Another of Alex’s goals was an increase in his classroom engagement and participation, and an environmental adaptation was made in support of that goal. When the teacher reviewed 145

FROM GOALS TO DATA AND BACK AGAIN

Alex’s progress she noted that the number of periods in which Alex had sat without screaming had increased but Alex’s willingness to participate in classroom activities had decreased over the same time interval. His aide commented that she felt Alex was becoming more remote and removed from the social interaction in the room. The teacher asked us if we thought the extinction program for his screaming had anything to do with his decreasing availability for learning and socializing. When we examined the data we did, indeed, find a high correlation between performance on the two goals. On any given day, the higher Alex scored on his scale for sociability, the more he screamed, scoring poorly on the goal for reducing the screaming behavior. The lower Alex scored on sociability, the less he screamed, scoring high on the goal for reducing the screaming behavior. The correlation calculation for Alex’s data showed r = -0.89. The size of the value (0.89) told us there was a strong correlation between the two goals while the negative sign reflected the fact that as one variable increased the other decreased. After talking over the result with the team, the teacher decided that perhaps Alex’s screaming was a sign of happiness, joy, or eagerness to participate rather than a sign of acting out. While it had looked at first as if he was learning to control his screaming, he had, in fact, been learning that his attempts to communicate his interest in others were ineffective. It would make sense, then, that when his attempts to engage were ignored his motivation for interacting would subside and he would retreat into solitude again. To test this hypothesis the teacher began to respond to Alex’s screaming by quickly, joyfully, and emphatically modeling other behaviors within his repertoire that he could do instead. It was successful. The next time we visited the classroom, Alex’s overall engagement was higher and his screaming was rarely a problem.

You get what you ask for2 In the previous section we made an interesting statement about the translation of factors into numbers. Looking at the specific examples of Becca’s location factor and Celeste’s session factor, we observed that we could change the order of the numbers we had assigned without changing the meaning of the factor, the correlation hypothesis we were testing, or the size of the correlation we found. Although our statement was true in those two cases, it is not true in general. The purpose of this section is to give you a deeper understanding of the relationship between the choices available during translation and the differences in correlation that can result. In particular, we examine the circumstances under which Casual readers may want to skip this section. It contains a fairly technical discussion about the ways in which you can translate a factor into numbers and the effect of your choice on correlation. Readers who intend to use the Pearson r should not skip this section, however, because the issues discussed here can have a profound effect on the outcome of the analysis.

2

146

EXCEPTIONS THAT PROVE THE RULE – FACTOR DATA

varying the assignment of numbers to a factor does result in asking different questions and, in turn, receiving different answers. Once again we set our discussion in the context of Celeste’s data for goal Z. When we created the session factor for Celeste we were reacting to a pattern we saw in the line graph of her raw data. That information led us to group the observations into sets of five and question whether position in a five-session sequence strongly predicted performance on the goal. Since Celeste’s skill seemed to increase over the course of the sequence we assigned the lowest value of the factor (session number = 1) to the first observation, the next value (2) to the second observation, and so on. We began our numbering again for each new set of five sessions to reflect the regression of performance at the beginning of each set. Computing the Pearson r with this assignment of numbers is like asking the rather complicated question, “Does Celeste’s performance vary depending on where she is in a five-day cycle such that her worst days are always the first day of the cycle, her best days are at the end of each cycle, and her performance increases steadily from worst to best?” Of course, Celeste is a “variable” child so we were free to make the pattern in the data as clear as we wanted. If Celeste were a real child, however, there probably would have been more noise in the data and the visual suggestion of steady progress throughout the cycle might have been weaker. In addition, her data would have had real dates associated with it. Suppose, for example, that the first session fell on a Monday. Had we known that when we first noticed the pattern in her line graph, we might have computed the same correlation (five-session sequence), viewing it as a measurement of steady progress over the course of the school week. But having gone that far in our thinking, we might not have stopped to compute that correlation; instead, we might simply have gone a step farther, wondering whether Celeste is losing some of her skill over the weekend because she does not have the opportunity for practice. As a result, we might have formulated a different question: “Does Celeste’s performance vary depending on whether it is the beginning of the week or the end of the week?” Notice that this is a much more general question than the one we originally asked. Often a general question is a better explanation of the data than a more specific question, especially if the data is noisy. To test the new, more general hypothesis, we must create a session factor that ranges over only two values: one value for early in the week (1) and one value for late in the week (2). You can see just such an assignment if you look at “Celeste, Goal Z” in the ABC Workbook. Column D of the spreadsheet shows an early week/late week factor with (1) assigned to the first two goal observations out of every five and (2) assigned to the last three observations out of every five. When we compute the Pearson r for columns B and D, we are asking the early week/late week question. It is a different question from the first question we asked by computing the Pearson statistic for columns B and C. Different questions give different 147

FROM GOALS TO DATA AND BACK AGAIN

answers. The answer to the original question (five-day sequence, now known to be Monday through Friday) was r = 0.76 and r = 0.80, for the “before” and “after” data sets, respectively. The answer to the new question (early/late) is r = 0.68 both “before” and “after.” By testing both hypotheses we’ve found that variation over the course of the week is a better (more predictive) explanation of 3 the data than early week/late week, both “before” and “after” learning. In other words, Celeste doesn’t simply catch up by the end of the week in some unpredictable way; she advances steadily from Monday to Friday. Now let’s suppose that the first session was a day other than Monday (we’ll say it was Tuesday, just to be concrete). A five-day sequence that starts on Monday corresponds to a natural concept (weekdays), but a five-day sequence that starts on Tuesday does not. In an attempt to make sense of the data, then, the team might have focused on the reason for the regression, looking for other facts about Tuesdays to try to explain the periodic drop in scores. Perhaps Tuesday is the day Celeste rides the bus to school instead of being driven by a parent. Since Celeste is a child with autism and bus rides are notoriously chaotic, we might hypothesize that she needs additional time on bus days to organize herself after arrival. Under these circumstances we might formulate the question: “Is Celeste’s performance predictably worse on Tuesdays than on other days?” This is an even more general question than early week/late week. It would be a natural hypothesis if Tuesdays were a definite trouble spot, but Celeste’s performance the rest of the week looked fairly constant. As with early week/late week, to test the hypothesis we must create a factor that ranges over two values. This time, however, the value of 1 would be assigned only to Tuesdays and the value of 2 would be assigned to all the other days. The factor for this hypothesis is shown in column E of Celeste’s spreadsheet. When we compute the Pearson r for columns B and E, we find that this view of the data is also a less accurate explanation of Celeste’s pattern of behavior than our original hypothesis was. Looking back over all of the discussion of Celeste’s performance on goal Z, it may seem as if data analysis is a confusing and uncertain business. After all, the data itself might suggest a factor we couldn’t anticipate (for example, position in a five-session sequence). It’s even possible that a factor we collect in one form (as a date, for example) will contribute the best explanation only if we view it in a slightly different way (as early or late in the week, weekday versus weekend, etc.). Whatever the real explanation of Celeste’s difficulties is, the real lesson in her data is this: the power of an empirical approach comes as much from the ability to be

We’ve assigned the middle value of the sequence (the middle day of the week) to the late part of the week by giving it a value of (2). You may be wondering what would happen if we assigned the middle value to the early part of the week, or gave it its own value (2) and assigned the last two days of the week another value (3). Good questions. We suggest you try it for yourself and see.

3

148

EXCEPTIONS THAT PROVE THE RULE – FACTOR DATA

flexible in asking questions as it does from the statistical result. Flexibility is the place where theory, intuition, and data analysis meet. If Celeste’s goal Z was an exercise in flexibility, Anton’s goal Z is an exercise in stubbornness. Sometimes the Pearson r simply won’t let us ask the question we want to ask. To understand how this can occur, open “Anton Goal Z” in the ABC Workbook. A portion of this spreadsheet can also be seen in Figure 9.7. All we know about Anton’s version of goal Z is that it measures performance on a 5-point scale. The comparison of means shows no reliable change and we’d like to understand why. The location of intervention is the only factor we’ve tracked, so we must at least consider whether it could help explain Anton’s lack of progress. In short, we want to ask the question, “Does performance on goal Z vary as a function of the location of intervention?” As shown by the alternatives listed in column B, the location factor has three possible values (home, school, and playgroup) so we will need to use 1, 2, and 3 to translate the factor into numeric form. But if we assign values as in column D (school = 1, home = 2, and playgroup = 3), we will be formulating the hypothesis “Does Anton do better on goal Z in school than he does at home and better at home than he does in playgroup?” That’s a much more specific question than we had in mind. We can try the assignment in column E, but that corresponds to the question “Does Anton do better at school than at playgroup and better at playgroup than at home?” The choice in column F is no better. It asks “Does Anton do better at home than at school and better at school than at

Figure 9.7 There is no way to ask the general question, “Does performance vary with location?” Instead, the Pearson r forces us to test a set of more specific questions, each reflecting a different assignment of locations to numbers 149

FROM GOALS TO DATA AND BACK AGAIN

playgroup?” In each case, the translation to numeric form required for computing the Pearson r also forces us to ask a more specific version of the question than the one we really want to ask (“Is there a correlation between location and goal Z?”). This is true of the other statistics available for computing 4 correlation in Excel as well. What’s the lesson here? The fact that the available statistics ask us to formulate questions that are more specific than we want is both a blessing and a curse. The blessing comes both from the precision the statistic forces on us and from the resulting specificity of the response we get when a strong correlation is found. Look at the actual values given by the Pearson r in rows 19 and 38 of the spreadsheet. The assignment of 1 to home, 2 to school, and 3 to playgroup gives “before” and “after” values of r = 1.00 and r = 0.89, respectively. From this we not only know that location strongly predicts Anton’s performance on this goal during both phases of intervention, we also know which location is best, middle, and worst. More precise information can lead to more focused remediation. The curse, on the other hand, comes from the fact that we had to apply the statistic three times to be sure we understood whether there was a relationship between the factor and goal. Maybe that doesn’t seem too bad but remember that we may be tracking five or six factors for each child. Moreover, the number of possible assignments grows quickly as a function of the number of different 5 values in the factor. The real problem, though, is that the more statistical tests you perform on a data set, the more likely it is that you will detect a pattern in the data somewhere—even if it isn’t really there. Believe it or not, if you test it enough different ways you can even find a pattern in a randomly generated set of numbers. Don’t be alarmed. In general, the data itself combined with your experience with the child will suggest some of the possible assignments of numbers to a factor and not others. Having watched Anton or Becca or Celeste (whoever they may be) you will know, for example, that if location is going to be predictive for some goal then school will be at the bottom of the ordering for one child, at the top of the ordering for another. You do need to be aware that the correlation 4

With three values for location there are actually six possible ways you can assign the numbers 1, 2, and 3. We only need to test half of the assignments, however, because we know that reversing the order of the assignment will simply reverse the sign of the Pearson r value. For example, translation of 1 to school, 2 to home, and 3 to playgroup produces the same degree of correlation as the “reversed” assignment of 3 to school, 2 to home, and 1 to playgroup, but with the opposite sign. The same principle holds true for the pair of assignments (2,1,3) & (3,1,2) and the pair of assignments (2,3,1) & (1,3,2), so we need to test only one member of each pair.

5

For the mathematically minded, the number of assignments with unique degrees of correlation (ignoring sign) given a factor with n categories is n!/2.

150

EXCEPTIONS THAT PROVE THE RULE – FACTOR DATA

function interprets your assignments as specific instances of questions, but the other sources of information you have—about development, about cognition, about autism, and about the child—can help you decide which questions to ask.

Data analysis, revisited Over the last four chapters we have introduced a number of tools for data analysis: the line graph, the mean, the standard deviation, the comparison of means statistic, the bar graph, and correlation. We filled our statistical toolbox piece by piece, but in doing the work of interpreting the data in the examples we switched back and forth among them the way any good carpenter, plumber, or mechanic would. Each tool has something slightly different to offer. Does the line graph look confusing? Then check the means. Do the means really look different? Make sure with the standard deviation. Did the child progress less than we thought? Go back to the line graph and look for outliers or other patterns. See something unusual going on? Check the factor data for correlations. Find a strong correlation? Separate out the graphs for each factor and re-analyze the means. This pattern—asking questions, checking hypotheses against the data, revising the questions, and asking again—is how data analysis works. Think of the process as a kind of storytelling. In a good and satisfying story each chapter or character may reveal a different piece of what’s going on, but by the end they have presented a single, coherent view of events (and hinted at some of what might be coming in the sequel). In the next chapter we have the chance to apply the analytical tools we’ve developed to data sets for Joey, Tyler, and Mai Lin. Let’s tell some stories.

Things to remember T A single value that stands out against a background of otherwise regular

data is called an outlier.

• An outlier can be an unusually high value or an unusually low value in the context of its portion of the data. • A low outlier in the “before” data will pull the “before” mean down and increase the standard deviation of the “before” data set. A high outlier in the “after” data will pull the “after” mean up and increase the standard deviation of the “after” data set. Either of these events can exaggerate the child’s progress. • A low outlier in the “after” data will pull the “after” mean down and increase the standard deviation of the “after” data set. A high outlier in the “before” data will pull the “before” mean up and increase the standard deviation of the “before” 151

FROM GOALS TO DATA AND BACK AGAIN

data set. Either of these events can make the child’s progress look worse than it truly is.

• A high outlier might point toward a factor that is particularly beneficial for the child. A low outlier might point toward a factor that is particularly difficult for the child. When an outlier cannot be explained by a factor that is already being tracked it may suggest a new factor to be watched. T Two variables are correlated when variation in the value of one

variable predicts variation in the value of the other variable.

• Correlation allows us to make predictions but it is not the same thing as proving a causal relationship. • The Pearson r gives a correlation value between –1.00 and +1.00. The higher the number (ignoring sign) the stronger the correlation. The sign tells whether an increase in one value predicts an increase (+) or decrease (–) in the other value. • As with all of the other statistics we have discussed, the use of correlation must be guided by an understanding of the child and his or her circumstances, particularly when formulating the hypothesis to test. • Correlation helps us identify aspects of a skill or environment that are systematically helpful or harmful to the child. This information, in turn, allows for more specific recommendations, more focused design of intervention, clearer goals, and more accurate statistics in the future.

152

Chapter 10

Coming Full Circle — Joey, Tyler, and Mai Lin In Chapters 2 through 5 we demonstrated a method for collecting meaningful data for developmentally based goals. In Chapters 6 through 9 we presented a set of methods for analyzing that data. Because we were concentrating on one piece of the process in each chapter, it may have been difficult to see how the pieces work together. In this chapter, then, we want to bring The Big Picture back into focus. We’ll do this using the three case histories we started in Chapter 5, but this time we’ll take you from the goals to the data and back again.

Joey

1

The youngest and least able of the three children, Joey has communication skills at the focus of his intervention. His goals are all in Phase 1, requiring intense interaction, a great deal of support, and familiar people and environments. The data collection sheet we created for Joey in Chapter 5 is repeated in Figure 10.1. Joey receives services from an agency where progress and intervention plans are reviewed quarterly. This means that up to three months of data can be collected prior to analysis. The team feels that analysis prior to the three-month regulatory deadline would be inappropriate because they expect that progress will come slowly at this stage in Joey’s development. The sheet titled “Raw Data” in the file “Joey Jan-Mar2001.xls” contains the data for the three-month interval we analyze in this chapter. If you compare Joey’s data collection sheet to the spreadsheet you will notice that we left the “therapist” and “time” factors out of the translation to raw data. Since Joey receives intervention from a single therapist every weekday for the same 4-hour period these factors do not vary—if the values of the factor 1 don’t vary they can’t serve as an explanation for variation in Joey’s performance.

We also left out “therapist” and “time” to make the data sheet easier to read and the example easier to follow. In a real situation we would want the two factors represented if, for example, it were possible that this data might be combined with data from another therapist or if the current therapist might change Joey’s hours of intervention.

153

FROM GOALS TO DATA AND BACK AGAIN

Name: Joey Smith Name of therapist:____________ Date:______ Time:______ Location of intervention: q home q center On a scale of 1 to 5 (avoidant to sociable), please rate Joey’s overall mood for the day. 1 2 3 4 5 1. When offered a choice of food, or object with which to play or hold, was Joey able to indicate preference? (use same 2-hour period each day when choices are routinely offered)

# of opportunities

# of successes

2. When given an opportunity, was Joey able to match a photograph to a preferred item?

o o o o

3. After hearing the language modeled two times, was Joey able to fill in the word approximation “go” after “Ready, set…” to start a preferred activity?

# of opportunities

4. What level of prompt was needed to get Joey to pull an adult to an object of desire?

o multiple physical and verbal prompts o a verbal prompt (“Show me!”) with outstretched hand o independently (a few times) o independently (consistently)

5. Did Joey attempt to imitate your mouth movements?

o o o o

Calculate: ______ % (# of successes ÷ # of opportunities x 100) not today, even with modeling matched after modeling independently matched 1 or 2 times independently matched multiple times # of successes

Calculate: ______ % (# of successes ÷ # of opportunities x 100)

no attempts noted growing interest in mouth movements a few attempts multiple attempts

Figure 10.1 Joey’s data collection sheet, as created in Chapter 5

154

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

The location of intervention does vary, however, and is included in column B of the spreadsheet. On Mondays, Wednesdays, and Fridays the therapist comes to Joey’s home; on Tuesdays and Thursdays, Joey’s parents bring him to the center where the therapist is based. The remaining factor (overall mood) and Joey’s goals are shown in columns C through H. Before we make changes to or perform calculations on the raw data, let’s glance down the columns of the spreadsheet and see if any patterns jump out. The values for the goal “Show preference” in column D range between 11% and 56%. Since the column heading reminds us that Joey had about ten opportunities to practice this goal in each session, the percentages mean that his performance ranged between about 1/10 and 5/10 successes per session. We can also see that the beginning portion of the column contains mostly values in the 20%–30% range while values toward the end of the column seem to be in the 40% range. Even this brief glance, then, seems to promise that Joey has made gains on this goal. The next column shows the observations for “Match photo.” Again, the values seem to climb overall, but there are quite a few missing values (blank cells) and we’ll need to understand why. Moving right, a preliminary view of the percentage data for “Ready, set...” seems to show a pattern similar to the percentages for “Show preference.” The remaining two goals, “Prompt level” and “Imitate mouth,” look different from “Match photo,” however. Although all three goals measured progress on a 4-point scale, the observations for “Prompt level” and “Imitate mouth” stay in the 1–2 range over the entire time interval. What this means in each case depends, of course, on the scales themselves, but we can assume for the moment that less progress was made in these two areas. Since we are going to create graphs, compare means, and so on, why bother with this exercise? Having a general characterization in mind before you start a more formal analysis is important for three reasons: 1.

We’re about to eliminate some of our data when we “discard” the middle third of the interval. The justification for this step is that we expect cognitive change to unfold with a particular pattern. If, as we scan down a column, we see a pattern that seems to violate that expectation (for example, a month of 1s followed by a month of 4s followed by a month of 2s) we would need to find a different method of analysis. All methods have assumptions. If you see that the assumptions of your method do not hold for a data set, you cannot trust the outcome of the analysis.

2.

Some of the patterns that are in the data become less clear when we compute statistics. For example, the missing values in the “Match photo” goal are not reflected in the mean and standard 155

FROM GOALS TO DATA AND BACK AGAIN

deviation—these statistics are computed over the values that are there and have nothing to say about values that aren’t. If we look only at the results of the statistics, we are likely to miss whatever importance the missing values might have. 3.

The mechanics of copying, pasting, and clicking-and-dragging columns of values can sometimes go wrong, introducing mistakes into the data. Having a sense of where the analysis is likely to be going can help you detect when you’ve taken a misstep along the way.

With a feel for the data, we’re now ready to begin the comparison of means. Step 1 requires that we divide the data into thirds while good habits require that we leave the raw data untouched, so the next sheet in Joey’s workbook contains a copy of the raw data with the middle third dropped out. This gives a nice January versus March contrast for the “before” and “after” phases of intervention. Rows 25, 26, 53, and 54 in the spreadsheet show the results of step 2 of the comparison: the calculation of the means and standard deviations for each goal in each phase. This data is also shown in Table 10.1.

Table 10.1 The statistical information for Joey’s goals over a three-month period BEFORE

Preference Match (%) photo (1–4)

Ready, set, ... (%)

Prompt level (1–4)

Imitate mouth (1–4)

25.41

1.56

27.23

1.09

1.09

Standard deviation

6.93

0.51

8.90

0.29

0.29

Pearson r (location)

0.03

-0.03

-0.03

-0.06

-0.06

Pearson r (mood)

0.24

0.22

-0.15

-0.11

0.09

43.45

2.95

49.05

1.77

1.18

Standard deviation

5.88

0.23

9.05

0.43

0.39

Pearson r (location)

-0.08

-0.18

-0.02

-0.01

-0.09

Pearson r (mood)

-0.38

-0.36

-0.27

0.27

0.03

Mean

AFTER Mean

Because the number of factors is small and the categorical factor (location) has only two possible values, we’ve gone ahead and computed the correlation for each factor and goal pair. None of the correlations comes close to our boundary value of 0.55 (see Chapter 9), which tells us that Joey’s performance on these 156

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

goals is statistically independent of where intervention occurs and of his overall mood. Still, we don’t want to overgeneralize this result: Joey is receiving intervention in two highly controlled and familiar environments. Although location is not a factor during the emergence of these skills it could still become a factor when the skills are moved into more realistic settings. The lack of correlation between overall mood and performance is interesting because we tend to think of this factor as indicating availability to learn. Apparently, this is not the case for Joey at this stage in his development of these skills. The next step in the analysis is to compare the “before” and “after” means to see whether there is evidence of reliable progress. If you want to try the comparison using only numbers, contrast rows 25 and 53 on the spreadsheet or the two boldfaced rows in Table 10.1. If you prefer to work with graphs, they can be found on subsequent sheets in the workbook. They are also repeated as figures in the remainder of this section as we examine each goal in turn. Joey’s first goal (Figure 10.2) reflects his parents’ desire that he be able to “make his mark upon the world,” especially by communicating more effectively. In this period of intervention the team has focused on a developmentally appropriate instance of that goal: indicating preference when choices are offered. Joey had about 10 opportunities each session to demonstrate this skill, which means that there must be at least a 10% change in his means before we can believe there is any reliable change in his behavior. Joey’s “before” and “after” means are separated by 20%, twice the required difference. The standard deviation around each mean is smaller than the category width so we can assume that these percentages are good representative values. There is no overlap in the error bars. All in all, then, Joey’s skill seems to have improved significantly and reliably, at least during the same 2-hour period each day in which choices are routinely offered to him. So, there has been progress. Does that mean the goal should be changed? To say it another way: is the ability to indicate preference about half the time what the team is aiming for? Is this the level of responsiveness we would expect in a child of Joey’s developmental age? What is the percentage of success at which the skill should be considered “emerged?” In practical terms, do we keep doing what we are doing and wait for Joey’s mean to reach 60% (70%? 80%?) before we introduce additional sources of stress? Or do we begin now to build less rigid qualifications around the environment in which we want to elicit the behavior? The data can take us only so far—to answer questions like these, theory, intuition, and family concerns must also be brought into the discussion. Joey’s second goal requires him to match photos to preferred items such as favorite foods and prized spinning objects. The team is not only interested in the photo-matching task itself, but also in whether they might build on this skill to give Joey an effective form of communication. Is this possible for Joey? Does he get it? 157

FROM GOALS TO DATA AND BACK AGAIN

Joey, Showing Preferences, Jan-Mar '01

% of time showed preference w/in 2 hr period

60 50 40 30 20 10 0 1/2-1/31

3/1-3/30

Joey, Showing Preferences, Jan-Mar '01 Comparison of Means

% of time showed preference w/in 2 hr period

60.00 50.00 40.00 30.00 20.00 10.00 0.00 Before

After

Figure 10.2 Change in Joey’s ability to show preferences

In the beginning, Joey’s performance is not encouraging. Looking at the line graph in Figure 10.3 we notice a lot of seesawing in the “before” segment between the values for “Not today, even with modeling” and “Matched after modeling.” When we combine this evidence with the missing values that we noticed earlier (the photographs were not always available, perhaps), we are tempted to conclude that Joey isn’t responding well to this concept. But the “after” data shows us that with patient and consistent help Joey can learn to make an association between a photograph and a preferred item. By the end of the three-month period, Joey is independently matching the photo to the item one or two times a session. The difference between the “before” and “after” 158

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

Joey, Matching Item to Photo, Jan-Mar '01

4-point scale (1,2,3,4)

3.5 3 2.5 2 1.5 1 0.5 0 1/2-1/31

3/1-3/30

Joey, Matching Item to Photo, Jan-Mar '01 Comparison of Means

4-point scale (1,2,3,4)

3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 Before

After

Figure 10.3 Change in Joey’s ability to match a photo to an item

means (1.39) is more than a category width (1.00) even with the possible 2 outlier. The standard deviations are not large enough to cause concern and there is no overlap in the error bars. So, despite the initial worry that perhaps Joey isn’t “getting it,” the data says otherwise. The comparison of means shows reliable progress on this goal. The data indicates this is a good potential system of communication for Joey and the team should continue their efforts.

2

Recall from Chapter 9 that because this is a low-valued outlier in the “after” phase it can mask real change by making the difference between the means smaller than it would otherwise be.

159

FROM GOALS TO DATA AND BACK AGAIN

Joey is now at the point where he can match a photo to certain preferred items independently one or two times a session. Unlike Joey’s first goal, the level of skill he has reached in matching seems to indicate quite clearly that this skill is no longer “emerging” (Phase 1). As a result the team must consider how to work on the consistent use of this skill and how to expand matching photos to preferred items into a true communication system (Phase 2). The team may decide to expand Joey’s picture vocabulary next or to increase the frequency of using his known vocabulary of photos or both. Joey’s third goal attacks a different aspect of his communication problem. In “Ready, set…” Joey must say an approximation of “go.” The qualifiers on the goal provide Joey with two prompts and the additional motivation of a preferred activity to encourage his performance. When we examined the raw data briefly it seemed as if Joey’s progress on this goal was comparable with his progress on the goal for indicating preference. The graphs—in particular, the comparison of means graph in Figure 10.4—do nothing to change that impression. This is an example of how graphing can mislead. Joey had only five opportunities per session to demonstrate this skill. This means that a reliable change in behavior requires at least 20% difference in the means. The 20% difference we saw in the “Preference” goal represented two category widths, but here it represents only one. Although the data for this goal still meets our definition of reliable change, we should treat Joey’s progress a little more cautiously then we did with the previous goal. This makes sense—we have fewer observations to inform us. If Joey’s performance level of 50% is enough to convince the team that the skill is no longer emerging, how might they respond? Two straightforward ways to work on the consistent use of this skill would be to keep the situation the same (preferred activities) and fade the prompting, or to keep the prompt level the same and move to a wider range of activities. On an entirely different track, the team might look to expand the use of the word approximation to a different set of linguistic contexts for “go,” such as “Let’s go!” or “Here we go!” Moving on, we find that neither of Joey’s remaining two goals passes the first test for reliable change. In the fourth goal (Figure 10.5) the team wants Joey to pull an adult to an object he desires rather than standing at the location of the object and whining. His “before” data for the goal shows a regular pattern of 1s, indicating that he required multiple physical and verbal prompts in the beginning of intervention. His “after” data shows some progress in responding to more natural levels of prompting, but the pattern isn’t regular enough at this higher level to create a difference of 1.00 in the means. Indeed, Joey’s performance in the “after” interval looks exactly like what we would expect if he were in the learning phase of cognitive change, with the “after” pattern still to come.

160

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

Joey, Approximate "Go," Jan-Mar '01

Percentage with prompting

70 60 50 40 30 20 10 0 1/2-1/31

3/1-3/30

Joey, Approximate "Go," Jan-Mar '01 Comparison of Means

Percentage w/ prompting

70.00 60.00 50.00 40.00 30.00 20.00 10.00 0.00 Before

After

Figure 10.4 Apparent change in Joey’s ability to approximate the word “go”

Since there is no correlation between the goal and any of the goal-independent factors to give another explanation for the ups and downs, the team is likely to hypothesize that Joey is still in the learning phase for this goal. With this hypothesis they would extend measurement of the goal, as is, into the next period of intervention. How much longer they want to go before re-evaluating the goal is a question they must also decide. When they come to the re-evaluation, however, they will combine all of the current raw data for this goal—including the middle third—with the new observations and reapply the method to the data from the full time interval. 161

FROM GOALS TO DATA AND BACK AGAIN

Joey, Prompt Level to Pull Adult to Object Jan-Mar '01

4-point scale (1,2,3,4)

2.5 2 1.5 1 0.5 0 1/2-1/31

3/1-3/30

Joey, Prompt Level to Pull Adult to Object Comparison of Means Jan-Mar '01

4-point scale (1,2,3,4)

2.50 2.00 1.50 1.00 0.50 0.00 Before

After

Figure 10.5 Lack of change in Joey’s ability to pull an adult to a desired object

Joey’s final goal shows the least progress (Figure 10.6). In intervention for this goal the therapist tried to find ways to make Joey more attentive to and imitative of mouth movements. The idea that Joey might be in the learning stage for this goal seems less compelling than in the previous example because the values of 1 (“no attempts noted”) dominate even in the “after” phase. This might be a good time for the team to re-evaluate the importance of targeting this goal, or at least brainstorm new ideas for focusing Joey’s attention. As is usually the case, data analysis answers some questions even as it raises a host of new ones. Why was Joey successful in approximating “go” but uninterested in mouth movements? When he was prompted with “Ready, set…” was he 162

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

Joey, Imitate Mouth Movements, Jan-Mar '01

4-point scale (1,2,3,4)

2.5 2 1.5 1 0.5 0 1/2-1/31

3/1-3/30

Joey, Imitate Mouth Movements, Jan-Mar '01 Comparison of Means

4-point scale (1,2,3,4)

2.00 1.50 1.00 0.50 0.00 Before

After

Figure 10.6 Lack of change in Joey’s ability to imitate mouth movements

watching the therapist’s face or relying on only her voice? Were his successes with “go” dependent on whether he was watching her? And why did he make such great strides in indicating preference but slower progress in prompting others to help him get the things he desires? Questions like these lead the team into discussions and new hypotheses. Some of the new hypotheses will find their way into the language that defines new goals and factors and redefines ongoing goals as they change over time. Intervention must change as a function of the new knowledge we have about Joey; then Joey must be given the chance to change as a function of the new intervention. The story continues and if we were part of Joey’s team we would continue to observe, collect, and analyze the new data. But we’ve come as far as we can with Joey, so let’s turn our attention now to Tyler. 163

FROM GOALS TO DATA AND BACK AGAIN

Tyler Tyler is both older and more able than Joey. His data collection sheet (Figure 10.7) shows that he is learning new skills (goals 4 and 5), extending skills to new environments (goals 1 and 2), and developing independence (goal 3). In other words, Tyler’s case gives us the opportunity to look at goals that span the three phases of treatment. The family is providing both Tyler’s intervention and the data, which means that observations can be made seven days a week. Most of the data is collected when both parents are available; one parent acts as observer while the other is officially “on duty” to facilitate Tyler’s interactions and model intervention techniques for his siblings. Because they are providing most of his intervention, Tyler’s parents want feedback from the professionals they work with as soon as possible. As a result, the team makes the decision to limit data collection to 6 weeks before evaluating progress. As we did with Joey, we’ll begin the analysis of Tyler’s progress by glancing over the raw data sheet (see “Tyler Aug-Sept2000.xls”). Column B reveals that his parents have adopted a “two off, two on” pattern for trading duties so there’s plenty of data provided by each in case the identity of the “therapist” turns out to be a factor in Tyler’s performance. We also note that Tyler doesn’t take his allergy medication consistently (column C) which alerts us to watch for a relationship between this factor and his behavior. Column E of the raw data sheet has the observations relating to Tyler’s tendency to lapse into private reference. There is no clear pattern evident from a quick examination of the raw data; the difference in the means might be reliable but it might not. The same can be said for the goals “Secure listener attention,” “Eye contact before request,” and “Indicate disapproval or protest”—we’ll just have to wait for the statistical analysis. The remaining goal, “Number of verbal exchanges,” looks more promising. Division of the data gives 16 observations in the “before” and “after” subsets for each goal. The calculations of means, standard deviations, and correlations are given on the next sheet in Tyler’s workbook and in Table 10.2. All of Tyler’s goals except “Verbal exchange” were measured on a 4-point scale, so a difference in the means of 1.00 is necessary as a minimum indication that reliable progress has been made. None of them passes this preliminary test. “Verbal exchange” is our first example of a goal whose data comes from a tally. What is the category width for this kind of data? The minimum width must be 1.00, since that is the difference made by each additional turn in the exchange. Whether or not the team wants to consider 1.00 to be adequate progress is a separate consideration, and one we will discuss shortly. For now, we note that the difference in means for “Verbal exchange” comes close enough to the minimum category width of 1.00 that it is worth looking at that goal more 164

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

Name: Tyler Jones

Family member taking data:

Date:______

____________

Time:______ Did Tyler have his allergy medication today?

Yes

No

1 2 3 4 5

On a scale of 1 to 5 (self-absorbed to socially connected), please rate Tyler’s overall mood for the day. 1. Over a 1-hour period before bedtime when the family plays together, how many times did Tyler lapse into private reference?

o o o o

2. Over a 1-hour period before bedtime when the family plays together, how often did Tyler secure his listener’s attention before talking to him or her?

o rarely (needed prompts to do so) o only a few times o about half of the time o more than half the time

3. On the average, when asking for food or drink, did Tyler spontaneously look at you when making a request?

o o o o

frequently a few times once or twice not at all

needed multiple prompts needed a single prompt a few times without prompts consistently without prompts

4. During a single conversation in which a facilitative approach rather than a directive approach was used, how many verbal exchanges were achieved? (tally) 5. Was Tyler able to effectively indicate disapproval/protest?

o remained passive or got upset without explanation o indicated disapproval with coaching from family o beginning to protest or say “no!” independently o consistently indicated disapproval when appropriate

Figure 10.7 Tyler’s data collection sheet, as created in Chapter 5

165

FROM GOALS TO DATA AND BACK AGAIN

Table 10.2 The statistical information for Tyler’s goals BEFORE

Private reference (1–4)

Listener attention (1–4)

Eye contact (1–4)

Verbal Indicate exchange protest (tally) (1–4)

Mean

1.63

1.56

1.31

1.56

1.13

SD

0.62

0.51

0.48

0.51

0.34

Pearson r (parent)

-0.37

0.34

-0.09

0.49

0.03

Pearson r (meds)

0.30

0.10

-0.32

0.10

0.30

Pearson r (mood)

-0.76

0.24

0.19

0.37

-0.24

Mean

1.50

2.00

1.69

2.44

1.19

SD

0.63

0.63

0.60

0.51

0.40

Pearson r (parent)

0.27

0.42

0.37

0.07

-0.20

Pearson r (meds)

0.55

0.44

-0.10

0.49

0.02

Pearson r (mood)

-0.77

-0.28

-0.14

-0.12

0.34

AFTER

carefully. Remember: boundary values are just guidelines, not absolutes. The data set also shows that Tyler’s “Private reference” goal is strongly correlated with his overall mood so we need to understand the nature of this relationship better. The graphs for all the goals are provided for you in the subsequent sheets of Tyler’s workbook. In the remainder of this section we focus primarily on the two goals, “Private reference” and “Verbal exchange,” repeating those graphs in the discussion below. What’s the first thing we do to make sense of data? Go back to the meaning of the goal. In the case of Tyler’s “Private reference” goal, we want to make sense of two things: why there was no reliable progress and how his performance was affected by his overall mood. The phrase “Private reference” is shorthand for a deeper set of relationships. Those relationships are seen more clearly in the original goal statement than in the data question on the collection sheet: Goal: To help Tyler sustain connection to his social world, incidents of private reference occurring in the midst of social exchange will decrease in frequency.

We measured how often Tyler lapsed into private reference during the family time before bed because we believe that his tendency to do so interferes with his social connection. Seems reasonable—let’s see what the data says.

166

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

Figure 10.8 shows the line graphs for the “before” and “after” data and the comparison of means. We want to consider these graphs along with the fairly strong negative correlation between mood and goal performance in both phases. Tyler, Private Reference Aug-Sep '00

4-point scale (1,2,3,4)

3.5 3 2.5 2 1.5 1 0.5 0 8/2-8/16

8/31-9/15

Tyler, Private Reference Aug-Sep '00 Comparison of Means

4-point scale (1,2,3,4)

2.50 2.00 1.50 1.00 0.50 0.00 Before

After

Figure 10.8 A case of poorer performance after intervention?

Here is the first instance we have seen in which the means show a decrease in performance over time. Is Tyler really becoming more self-involved as a function of intervention? No, the rules for judging a reliable change for the worse in performance are the same as those for judging a reliable change for the better. The graph does not show a decrease in the means equal to the category width of 1.00. In addition, we can also look at the factor labeled “Overall mood,” which 167

FROM GOALS TO DATA AND BACK AGAIN

records his parents’ observations of Tyler’s level of social responsiveness each evening. We can calculate the mean and standard deviation of a factor, just as we do with the goals. When we perform those calculations we find that his mood averages 3.13 in the “before” phase (standard deviation of 1.02) and 3.81 (standard deviation of 0.75) in the “after” phase. His average level of social responsiveness has not decreased—he is not more self-involved. But we can say more than that. The strong negative correlation between mood and performance on this goal means that the more socially responsive Tyler was, the more frequently he lapsed into private reference. As we noted above, the rationale for this goal was that decreasing his private references would help Tyler sustain social interaction. It is certainly the case that decreasing Tyler’s private references would help his family members sustain social interaction. But Tyler seems to behave—quite consistently—in a manner that is different from our expectations. Perhaps, for Tyler, an increased desire to interact with his family leads to an increase in private reference because those bits of videos and songs are, currently, the only topics he has to share that are satisfying and pleasurable for him. Perhaps Tyler attempts to make his social connection through these private references. We’ll return to this idea shortly. Let’s switch our attention now to the “Number of verbal exchanges” goal. The rationale behind this goal was phrased: Goal: To help him sustain conversation, Tyler will be engaged in a daily verbal exchange of increasing length in which the adult partner avoids questioning and quizzing, and relies instead on commenting, narrating, rephrasing, and expressing opinions.

The data question for the goal required Tyler’s parents to tally the number of exchanges in a facilitative conversation each day. Table 10.2 shows the difference in the means is close to the minimal category width of 1.00 and the standard deviations are about a half-category width in size. The graphs of the data can be seen in Figure 10.9 with a now-familiar pattern in the line graph and only a small overlap in the error bars around the means. Certainly a case for progress can be made, even in this brief time interval. What do all these results mean for the next round of intervention? The 3 goals we did not examine in detail had small differences in means and large standard deviations indicating no evidence of change in Tyler’s underlying knowledge or skill in those areas at this point in time. The “after” data for securing his listener’s attention before speaking shows a trend in the right direction, however, and Tyler’s parents might feel confident that their strategies for encouraging this behavior will succeed given more time. The data for making eye contact before making a request and the data for indicating disapproval or protest in more effective and appropriate ways show no discernible change across the interval. Since “Indicate disapproval” is a Phase 1 goal, the team might decide to wait and see if the desired behavior emerges given more 168

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

time. The “Eye contact” goal is in Phase 2, and the team needs to consider what might be interfering with Tyler’s ability to develop consistency and independence in a skill he can reliably produce when prompted. Since none of the factors currently being recorded show a correlation to his performance, some creative thinking may be required. Perhaps the team will find a helpful notation made in the margin of Tyler’s data collection sheet on September 1st that hints at the reason for his unusually strong performance on that day. Tyler, Number of Exchanges Aug-Sep '00

Number of Exchanges

3.5 3 2.5 2 1.5 1 0.5 0 8/3-8/16

8/31-9/15

Tyler, Number of Exchanges, Aug-Sep '00 Comparison of Means

Number of Exchanges

3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 Before

After

Figure 10.9 Possible change in Tyler’s conversational ability

As for the “Verbal exchange” goal, a critical question when dealing with tally data is just how big a change is the team looking for. The arithmetic properties of the data say that the minimum category width is 1.00 but a reliable change 169

FROM GOALS TO DATA AND BACK AGAIN

from two turns to three turns may not correspond to what we think of as categorically different behavior. Concretely, the facilitative approach seems to be working. Do we think we’ve gone as far as we can or is it reasonable to expect longer conversations from Tyler at this point in his development? Perhaps the most interesting question the team must discuss is how they will respond to what the data analysis has told them about Tyler’s habit of lapsing into private reference. It seems as if Tyler does want to communicate, but, like all of us, he wants to communicate about what he is interested in—a limited repertoire of highly idiosyncratic topics. Also like the rest of us, he does a better job of communicating with a cooperative and facilitating conversational partner. So, on the one hand, the more sociable he is feeling the more enthusiastic his participation, but this enthusiasm has an unconventional expression. On the other hand, to the extent family members work hard to comment, narrate, and rephrase, social conversation can occur, and with longer duration than it would have with a more interrogatory style. The team might have to consider whether, at this stage in Tyler’s development, the burden should continue to rest on family members to adapt to Tyler’s idea of interesting conversation, commenting on and rephrasing his idiosyncratic expressions to turn them into exchanges within a more meaningful whole. The team may decide, in the future, that Tyler needs to learn to converse about a range of conventional topics, but for now such goals would be too costly. They might interfere, the data suggests, with his growing interest in being a social member of his family. So, are we done? Well, yes and no. It turns out that we missed something—a pretty big something, in fact. When we reviewed his data we noticed that Tyler doesn’t always take his medication so we dutifully checked for an interaction between this factor and his goals. Look back at Table 10.2 for the correlation between Tyler’s allergy medication and his skill at making eye contact before a request: r = -0.32 in the “before” phase and r = -0.10 in the “after” phase. No correlation, right? But the raw data records whether Tyler took his medication each day and the Pearson r looks for co-variation between taking the medication and Tyler’s performance on that same day. The problem is that Tyler takes his medication at bedtime so any correlation would show up in terms of his performance the next day. The sheet “Before & After (2)” gives the result of the Pearson r after shifting the factor data by one day. And there it is—r = 0.71 “before” and r = 0.63 “after”—a moderate correlation between taking his medication and better eye contact. We didn’t include this little bit of misdirection to fool you; we did it to make a point about data analysis. You’re not going to find everything. Data analysis has an unusual way of making its importance felt—we come to think that if a pattern is in the data we must be able to see it and if we can’t see it, it isn’t there. Nothing could be further from the truth. It’s always possible that there are patterns in a child’s behavior that the data questions don’t measure. It’s always possible that 170

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

there are patterns in the observations that the statistical methods can’t uncover. And it’s always possible that there are patterns in our own theories and expectations that keep us from seeing what the data has to say. These are not good reasons for not collecting data, but they are good reasons for combining a respect for your data with a healthy sense of its limitations.

Mai Lin Mai Lin is the oldest of the three children and the most “typical” in her skills and behavior. Her goals are focused on fine-tuning certain skills so that her communication becomes more socially appropriate and responsive to her partner. Mai Lin is now primarily in Phase 3, learning to extend her skills to multiple people in multiple environments, as shown in the data collection sheet in Figure 10.10. Data for these goals is collected at school, during two different periods and by two different adults. One is Miss A, a paraprofessional who is with Mai Lin during recess and lunch. The other is Mai Lin’s teacher, Mrs D, with whom she has a strong, supportive relationship. Because two individuals were making observations, it was decided that data would be collected only three days a week. Parents requested another meeting after two months of intervention to review the results. The raw data for Mai Lin can be seen on the first spreadsheet in the file, “MaiLin Feb-Mar2001.xls.” We note the following in our initial review of the data: · Mrs D’s data has no time of day associated with it because different

items were rated at different times. All of Miss A’s data was taken during recess.

· Like many children with ASD, Mai Lin is thrown off by change. So it is

good to see that her school day was pretty consistent over this two-month period with only six days on which a schedule change was required.

· Mrs D had limited opportunities to observe Mai Lin with an unfamiliar

person. As a result her data for the first two goals—facial orientation and voice control with an unfamiliar person—is sparse and it may be difficult to draw meaningful conclusions about Mai Lin’s progress. Despite the gaps, it does look as if her ability to select an appropriate volume is developing faster than her ability to face a stranger.

· The third goal, conversational repair, seems to be coming along nicely,

as does the fourth goal, topic switching. One shows the now-familiar pattern of increasing values while the other shows a not-so-familiar pattern of decreasing values. This makes sense, though, if we look back at the wording of the fourth goal which specifies that the number of 171

FROM GOALS TO DATA AND BACK AGAIN

Name: Mai Lin Wu

Teacher/Aide’s name:

Date:______ Time: ______

____________

Were there any unexpected changes in the school routine today?

Yes

No

If yes, please explain: 1. How much prompting was needed to help Mai Lin appropriately face new or unfamiliar people when speaking to them?

o unable to face unfamiliar person, even with prompt

o o o o

needed multiple prompts needed a single verbal prompt needed a single physical prompt no prompting needed

o no opportunity for observation 2. After a single prompt, if necessary, did Mai Lin use appropriate volume and inflection when speaking to unfamiliar people?

o o o o o

continued in quiet monotone even after prompt appropriate volume/inflection, 1 prompt, 1 or 2 x appropriate volume/inflection, 1 prompt, consistent appropriate volume/inflection, no prompts, 1 or 2 x appropriate volume/inflection, no prompts, consistent

o no opportunity for observation 3. Did Mai Lin effectively repair her communicative attempts after a peer said, “What?” or indicated in another way that Mai Lin was not understood?

o o o o o

skill emerging with assistance from adults skill emerging with multiple cues from peer skill emerging with 1 additional cue from peer repairs attempts herself about half the time consistently repairs communicative attempts

o no opportunity for observation 4. During “talk time” in which Mai Lin has to talk to a teacher for 5 minutes on a variety of common subjects, how many times did Mai Lin attempt to switch the conversation to a preferred topic? 5. What percentage of the time did Mai Lin make requests in the form of a question—as opposed to a statement—needing only a facial cue (expectant pause) as a reminder?

tally of occurrence: # of times: ______

o no opportunity for observation # of requests made:______ # made in correct form or repaired:______ Calculate: ______%

(# of successes ÷ # of opportunities x 100)

o no opportunity for observation

Figure 10.10 Mai Lin’s data collection sheet, as created in Chapter 5 172

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

times Mai Lin wanders off topic should decrease over time. Whether either of these goals shows reliable change must wait for the comparison of means, but the trend looks good. · The fifth goal, making a request using a question rather than a

statement, also looks promising although the small number of opportunities to measure this skill by each observer on each day is a little worrisome. Recall from our discussion of measurement using percentages (Chapter 3) that when there are only a few opportunities for observation, minor changes in behavior create large differences in percentage that may artificially inflate progress. We’ll see shortly that our method of calculating the category width helps compensate for this possibility.

Table 10.3 repeats the means, standard deviations and correlation computations found on the next sheet in Mai Lin’s workbook. Two of the “after” correlations (teacher versus “Face unfamiliar” and teacher versus “Voice unfamiliar”) and one “before” correlation (teacher versus “Face unfamiliar”) reached our boundary values. As we expected, the means for “Conversational repair” and “Topic switching” show the right trends (1 up, 1 down) but neither has changed enough for us to consider them achieved. The graphs for these two goals can be found on the appropriately labeled worksheets in Mai Lin’s workbook. In the remainder of this section we concentrate on goals 1, 2, and 5.

Table 10.3 The statistical information for Mai Lin’s goals BEFORE

Face Voice unfamiliar unfamiliar (1–5) (1–5)

Conv. repair (1–5)

Topic switch (tally)

Request as? (%)

Mean

1.92

1.92

1.56

2.71

33.25

Standard deviation

0.51

0.67

0.51

0.47

7.82

Pearson r (teacher/aide)

0.60

0.18

-0.13

0.23

-0.30

Pearson r (change)

0.08

0.06

-0.05

-0.19

0.17

Mean

2.27

3.07

2.22

2.19

58.44

Standard deviation

0.70

0.70

0.43

0.54

12.90

Pearson r (teacher/aide)

0.72

0.68

0.27

-0.07

0.30

Pearson r (change)

-0.15

-0.04

0.24

-0.09

-0.38

AFTER

173

FROM GOALS TO DATA AND BACK AGAIN

Mai Lin’s first two goals concern her behavior with unfamiliar people. From her case history we recall: One thing that concerns the teaching staff is that with unfamiliar people, Mai Lin seems to forget the body language and voice tone that normally accompany conversation. She turns away from the listener while talking, talks in an inappropriately quiet or low tone, and her inflection practically disappears. This makes it difficult for Mai Lin to interact with other children at lunch and recess.

It’s important to remember that these goals are in Phase 3; that is, Mai Lin has demonstrated consistent use of the required skills with her family and other

Mai Lin, Voice Volume (Feb-Mar '01)

5-point scale (1,2,3,4,5)

5 4 3 2 1 0 Before

After

Tyler, Number of Exchanges, Aug-Sep '00 Comparison of Means

Number of Exchanges

3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 Before

After

Figure 10.11 Possible change in Mai Lin’s ability to control her voice

174

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

familiar people but needs additional practice, redirection, and support with people she doesn’t frequently encounter. Let’s look at how Mai Lin is doing with voice control first (Figure 10.11). The line graph is not very helpful because of the missing values. The comparison of means is more useful, but neither captures the differences between Miss A’s observations and Mrs D’s observations, which we know are highly correlated to Mai Lin’s performance on this goal. The difference between the means is more than a category width but the overlap in the error bars says that we should do further analysis. Before we know what to do with the size of the standard deviation, we need to know how much of the standard deviation depends on each observer. The sheet titled “Voice Graphs(2)” in Mai Lin’s workbook makes the situation clearer. On this worksheet we have separated out each observer’s scores by date. Mrs D contributes fewer observations to both the “before” and “after” subsets of the data and her scores tend to be consistently lower in the “after” phase. They may actually be consistently lower in the “before” phase as well but we don’t really have enough of them to tell! If we consider only Miss A’s observations then a comparison of means shows reliable progress, with Mai Lin requiring fewer prompts to maintain appropriate volume and inflection over time. Deciding whether to believe this result is a bit tricky because there are only eight data points in each of Miss A’s “before” and “after” sets, not the ten that would make us feel comfortable. With only four data points in Mrs D’s “before” set we don’t even bother to perform a similar analysis on her observations—it can’t tell us anything we haven’t already observed informally from the values themselves. Every time Mai Lin is observed interacting with an unfamiliar individual her behavior is scored with respect to maintaining appropriate facial orientation as well as voice control. So before we draw any general conclusions about what the co-variation in observer and performance means, let’s turn our attention to that related goal. The graphs for the full set of “before” and “after” data for “Face unfamiliar person” can be seen in Figure 10.12. Again, the line graph is not very helpful but the comparison of means seems clear—no progress. And again, we have to worry about what effect the observer has had on these results because of the fairly strong correlation in the “after” data and the moderate correlation in the “before” data. The sheet titled “Face Graphs (2)” shows the same separation of observations for this goal that we made for the previous goal. There is not enough data to calculate means and standard deviations separately for Mrs D, but for Miss A, the “before” and “after” means are 2.13 and 2.67 respectively. So, even though Mrs D consistently rates Mai Lin lower on the scale for facial orientation than does Miss A, in this case her lower ratings are not interfering with a pattern of reliable progress. Miss A’s data shows

175

FROM GOALS TO DATA AND BACK AGAIN

some change over the two-month interval but not enough to stop intervention on this goal. Considering the information about the two goals together, what can we conclude and where do we go from here? The most liberal reading of the data suggests that when Mai Lin is with Miss A she has progressed to needing only a single prompt to remember to keep her voice level appropriate with an unfamiliar person but continues to need multiple prompts to face the person. Her data on these goals when Mrs D is the observer shows no reliable gains to date along either dimension. Mai Lin, Face Unfamiliar Person, Feb-Mar '01

5-point scale (1,2,3,4,5)

4 3 2 1 0 Before

After

Comparison of Means Mai Lin, Face Unfamiliar Person, Feb-Mar '01

5-point scale (1,2,3,4,5)

3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 Before

After

Figure 10.12 Lack of change in appropriate facial orientation

176

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

One possible reason for the co-variation of performance and observer in the voice goal is that Miss A and Mrs D could be interpreting the word “appropriate” differently. But another possible reason, one that explains the co-variation in both goals, is suggested by the fact that Miss A’s observations are always at recess. It is possible that Mai Lin’s gains actually depend on whether the unfamiliar person is a peer or an adult. Team discussion should reveal whether Miss A’s observations were almost always peer interactions and Mrs D’s observations were almost always child–adult interactions. It is certainly appropriate for a second grader to behave less confidently with an adult than a peer. Although Mai Lin’s behavior may still be more extreme than the typical signs of shyness in an 8-year-old, we shouldn’t expect more of her than we would of her shyest peers. So, if the hypothesis is valid, we should continue to expect progress to occur at different rates for each type of unfamiliar audience and we should separate the measurements accordingly (“Voice with adult, ”Face with peer," etc.). Even if the goals are split so that we are measuring Mai Lin’s performance on scales that more accurately reflect our expectations for peer or adult interactions, there is still work to be done. Mai Lin clearly needs to continue working on both the voice and facial orientation goals with adults and at least the facial orientation goal with peers. This means more data, although how much more depends upon the team’s best guess of an appropriate interval for Mai Lin to show progress on the more narrowly qualified goals. Once the additional data is collected, it would be combined with the current two months of raw data (separated into adult and peer interactions) and a comparison of means performed again. The last goal we will look at for Mai Lin concerns her use of the question form when making requests. As with her other goals, she has already demonstrated mastery of this skill in familiar surroundings. The team is interested in extending her skill using only the subtle cue of an expectant pause to remind her when she uses a statement instead of a question. Figure 10.13 shows the same graphs for this goal that are found on the last sheet of her workbook. The line graph has the familiar look of progress although there is enough variability in the “after” phase that we won’t be surprised to find a large standard deviation around that mean. That expectation is fulfilled when we look at the graph for the comparison of means. Nevertheless, the comparison looks exactly like what we have to come to expect for a solid, reliable jump forward in skill or underlying knowledge. This example is particularly instructive at this stage of the book exactly because it does look comfortably familiar. By now you have seen so many graphs like this that you probably feel confident assessing the situation with little more than a glance. And, indeed, Mai Lin’s “after” mean has jumped 25 points. But with only about 4 opportunities per score, this is the minimum difference we can consider indicative of change and so we must proceed, not confidently, but with 177

FROM GOALS TO DATA AND BACK AGAIN

Mai Lin, Requests as Question, Feb-Mar '01

% successful repairs

80 60 40 20 0 Before

After

Comparison of Means

% successful repairs

Mai Lin, Requests as Question, Feb-Mar '01

80.00 60.00 40.00 20.00 0.00 Before

After

Figure 10.13 Change in Mai Lin’s ability to form requests as questions

caution. In this case caution isn’t just an artifact of our method—it is truly warranted by the data. With only about four opportunities per session used to calculate her percentage of successful repairs, a single skilled or unskilled moment by Mai Lin carries a lot of weight. The team needs to remember this and look carefully at the variability in the “after” section of the line graph before concluding that Mai Lin has achieved this goal. If they are cautious they might ask what made the good days in that interval so good. If they are cautious they might even notice that, despite the fact that the correlation between observer and this goal did not reach criterion, all the best scores (the 75%s) occurred at recess. And if they are cautious they might notice that during those same recess 178

COMING FULL CIRCLE – JOEY, TYLER, AND MAI LIN

periods Mai Lin’s encounters with unfamiliar peers (goals 1 and 2) were most successful as well. Where caution ultimately leads we don’t know, but in this instance we may do Mai Lin more good by not assuming reliable progress than by simply marking the goal as achieved. In this chapter we wanted to accomplish two things. First, we wanted to give you the opportunity to follow the process of empirical developmental intervention through a whole loop: from goals to data then back to new goals. As important as it is to concentrate on each individual step along the way while you are becoming familiar with it, seeing all the steps together can add depth to your understanding. We also wanted the opportunity to introduce a small number of important ideas that didn’t fit into earlier chapters and stress some ideas that had been presented but were worth seeing again. The most important idea we wanted to stress was actually the catalyst for adding a surprise or two to the data. It is simply this: when all is said and done data analysis isn’t about graphs and numbers, as useful a shorthand as graphs and numbers can be for thinking about the messy stuff we call “behavior.” Data analysis is about—should be about— what’s behind the graphs and numbers: the expectations, the knowledge, the skills, the emotions, and the very real potential for change in any (or all) of these.

Things to remember T It’s important to get a feel for the data before you begin a methodical

analysis. That sense of what you’ve got and where things are heading helps to make sure you don’t violate the assumptions behind your method or forget important patterns that may be evident only in the raw data. It can also help you notice when the mechanics of working with a spreadsheet have introduced errors into a calculation.

T All analytical methods have assumptions. In the comparison of means,

the fundamental assumption comes from our understanding of cognitive change. If there has been change in the time interval as a result of intervention then we expect to see two different patterns of regular behavior separated by an irregular period of learning. Before you eliminate the middle third of the data in the comparison of means you must be sure that it is consistent with this assumption. Otherwise you may create an illusion of change by using a method whose assumptions do not hold.

T When all is said and done data analysis isn’t about graphs and numbers.

It’s about what’s behind the graphs and numbers: expectations, knowledge, skills, emotions, and the very real potential for change in any (or all) of these.

179

Chapter 11

Conclusions

The purpose of intervention is to help a child change. The process of intervention requires that you modify the underlying beliefs, knowledge, or emotions that cause the child’s current responses while at the same time modeling and teaching new behaviors that are more effective, appropriate, and adaptive. The purpose of this book is to bring about a kind of change as well. The process at work in this book is similar—we’ve tried to modify the underlying belief that developmental theory and empiricism are incompatible, while at the same time modeling an empirical form of intervention that we believe is more effective and appropriate than theory alone. The methods of data collection and analysis we have discussed were not developed as an afterthought. They have grown directly from practical experience with real children. At the same time, however, they are deeply tied to the particular view of cognitive growth that is at the heart of developmentalism. · Yes, a goal must be tied to an observable behavior that has been

qualified and quantified before we can consider it measurable. But the relationship of the behavior to the meaning of the goal cannot be forgotten and the goal itself must make sense in cognitive and developmental terms.

· Yes, a measurable goal must be put in a format that encourages clear and

consistent data collection. But we don’t observe just to produce observations; the data we collect is meaningful only insofar as it has the ability to reflect underlying change.

· Yes, data interpretation can be performed with simple statistical

functions available in almost any spreadsheet program. But we use the particular statistical functions we’ve described not because they are convenient, but because we expect that change takes the shape of two regular patterns of behavior separated by a period of learning.

· And, finally, yes, the results of the various statistical tests do make

specific recommendations for the next phase of intervention. But we 181

FROM GOALS TO DATA AND BACK AGAIN

acknowledge that those recommendations must be augmented with other sources of information and the needs of the unique and individual child. In the examples throughout the book we’ve tried to give you a sense of the sort of things you can do with data. You can’t do these things with data alone, but you can’t do them without data, either. You may find that once you start using data there are other things you want to do with it as well. Our focus has been on using data to measure individual change because that is what intervention is about. Yet there are other valid uses of the data, other things we can learn by comparing means and standard deviations. It is possible, for example, to compare different approaches for a single child or a single approach’s efficacy for different types of children. We need a few more tools, another concept or two, but you are well on your way to understanding what you need to in order to use data in those ways as well. For those of you who remain unconvinced about your own ability or desire to add an empirical component to your intervention, we hope that what you take away from this book is, at least, the ability to sit through the next team meeting or case review with an open mind. If you do, we think you’ll find it difficult not to wonder whether a therapist or teacher is remembering the way a child usually acts or just the way he or she acted most recently. We think it will be hard for you not to ask yourself if the change being discussed is reliable. We think you’ll be struck both by the idea that there are reasons behind the behavior and by the desire to understand those reasons. We think you’ll be back. On the other hand, some of you are already thinking about your kids, the patterns in their behavior, and the questions you want to ask. If we’ve done our job you now have the tools to go out and collect the data to answer those questions. We’ll still be here as a reminder and a reference when you need us. In the meantime, remember to listen to the theory, listen to the data, listen to the children, and listen to your own heart.

182

Appendix A

Basic Intervention Goals for Children with Autism

Attention and basic social relatedness Child will: · respond to the overtures of familiar/preferred adults with smile, frown,

reach, vocalization, or other intentional behavior.

· respond to the overtures of familiar/preferred adults with obvious

pleasure.

· demonstrate affection toward others. · seek comfort when hurt. · demonstrate awareness of others by seeking proximity. · stay engaged with familiar adult for increasing lengths of time. · become displeased when preferred adult is unresponsive during play for

30 seconds or more.

· spontaneously seek the company of his/her family members when

family is not attempting to engage him/her.

· show diminishing frequency of subvocalizations when engaged with a

family member/trusted adult.

· demonstrate awareness of others by showing some simple imitation. · acknowledge the comings and goings of familiar people. · call family members by name. · call family members and other familiar people by name. · focus attention on a social activity for ______minutes.

183

FROM GOALS TO DATA AND BACK AGAIN

Imitation Child will: · imitate with object after demonstration of use of object. · simultaneously imitate with objects. · imitate hand movements. · imitate body movements. · imitate mouth movements. · imitate sounds. · imitate words.

Affect Child will: · look up to caregiver using smile as a way of securing adult attention. · show positive emotional expressions in response to praise. · independently solicit praise upon the completion of a task. · label feeling states in self [begin with happy, sad, angry/mad, scared]. · identify emotions in family members/familiar adults/peers. · respond appropriately to emotions in family members/familiar

adults/peers.

· offer comfort to others in distress. · match spoken expressions of sadness, happiness, anger, and surprise with

facial expressions of the same emotions.

· accurately identify the feelings he/she has in a variety of settings and

will be able to explain the relationship of events to his/her feelings.

· use pretend play scenarios to explore negative affect and practice

appropriate responses.

· be tolerant of own mistakes and performances that were not perfect. · express precision and subtlety in the expression of emotion, through the

use of qualifiers to describe gradation of emotional experience (e.g., really disappointed, a little disappointed).

184

APPENDIX A BASIC INTERVENTION GOALS FOR CHILDREN WITH AUTISM

Self-regulation Child will: · recover from distress within ______minutes with help from familiar

adult.

· communicate through language when upset, rather than tantrum. · learn different strategies for self-calming during times of frustration,

anxiety, anger, or disappointment.

· use appropriate strategies for controlling his/her body when excited,

anxious, or angry.

· maintain a polite and/or tactful style of communication when letting

others know that something is bothering him/her.

· productively reflect upon the advantages and disadvantages of own

behavior.

Play

Increasing the play repertoire Child will:

· joyfully participate in sensory-motor play with a familiar adult. · participate in songs, finger-plays, and rhymes with familiar adults. · engage in parallel play. · engage in simple motor games with rules. · participate in turn-taking activities. · appropriately look at books with caregivers. · expand his/her play repertoire to include manipulation, sensory-motor,

art, music, building/construction, and early cognitive (sorting, matching, puzzles).

· participate in physical games with rules (e.g., “Duck, Duck, Goose”). · participate in non-physical games with rules (e.g., board games).

Pretend play

Child will: · develop interest in the content of pretend play as opposed to the simple

mechanics (i.e., interest will move from how the bottle fits in baby’s mouth to helping hungry baby).

185

FROM GOALS TO DATA AND BACK AGAIN

· use one object to represent another. · participate in pretend play involving concrete and familiar themes such

as self-care, daily activities, cars, and animals.

· develop nurturing play with baby dolls. · participate in increasingly elaborate make-believe, moving from early

concrete (episodes of eating/feeding, driving cars with noise, putting farm animals in barn) to more complex concrete (simple familiar stories).

· arrange doll furniture into meaningful groups and will use doll figures to

act out simple themes from own experience.

· participate in more elaborate play themes, moving from concrete themes

(involving everyday, common experience) to abstract themes (involving material never directly experienced).

· assume the role of another person (dress-up). · engage in role-playing using figures and puppets.

Drawing

Child will: · scribble with crayon. · imitate drawing of vertical line. · imitate drawing of circle. · add three parts to incomplete human drawing. · copy drawing of square. · draw unmistakable human with body, arms, legs, feet, nose, eyes, and

mouth.

Communication

Receptive communication (understanding language) Child will:

· respond to his/her name. · look for family members when asked, “Where is Mommy?” or “Where is

Daddy?”

· stop action in response to “No!” · appropriately respond to the command, “Stop!” · move body in response to a one-step direction. 186

APPENDIX A BASIC INTERVENTION GOALS FOR CHILDREN WITH AUTISM

· get familiar object or food that is requested. · take object or food to someone when requested. · follow two-step directions involving two different actions. · indicate approval when asked a “Do you want?” question. · appropriately respond to simple and familiar “Where” questions with

searching movements.

· point to eyes, nose, and mouth in self and others upon request. · identify all large body parts upon request. · point to pictures in a book or familiar objects as they are named. · follow a series of two to three simple related commands with the same

object.

· identify smaller body parts upon request (i.e., chin, knee, elbow, fingers,

and toes).

· follow a series of three unrelated commands.

Eye gaze

Child will: · look at person when given something. · look at person when giving him/her something. · follow someone’s point when object is in close proximity and can be

touched.

· point to desired object when object can be touched. · follow someone’s point when object is distant. · point to desired object when object is distant. · point to direct someone to look at object or event to share enjoyment

while looking back and forth to make sure adult sees what child sees.

· look toward adult to make sense of an ambiguous situation. · reference adult expression to guide own behavior. · look at person who is speaking to communicate interest/attention. · look at person to whom he/she is speaking to make sure person is

listening/attending.

187

FROM GOALS TO DATA AND BACK AGAIN

Expressive communication (body language and affect) Child will:

· respond to gestures with intentional gestures of his/her own (e.g.,

reaches out in response to outstretched arms).

· initiate interactions (e.g., reaches for toy). · look when name is called. · wave goodbye. · choose from two options using gestures and body language. · express desire for food using gestures and body language. · express desire for activity using gestures and body language. · express desire for toy/object using gestures and body language. · indicate disapproval using gestures and body language. · express wishes, intentions, and feelings using multiple gestures in a row. · find appropriate and effective ways to get attention. · participate in four reciprocal social interactions. · participate in eight reciprocal social interactions. · participate in 12 reciprocal social interactions.

Expressive communication (the use of symbols for communication) Child will:

· learn fill-in-the-blanks of familiar songs, rhymes, and or familiar verbal

routines (e.g., “Ready, set, go”).

· use word/sign/picture for “more.” · choose from two options using pictures/signs/words. · express desire for food using pictures/signs/words. · express desire for activity using pictures/signs/words. · express desire for toy/object using pictures/signs/words. · indicate disapproval using pictures/signs/words. · indicate that he/she is done with an activity by saying or signing, “All

done.”

· use pictures/signs/words for mother and father.

188

APPENDIX A BASIC INTERVENTION GOALS FOR CHILDREN WITH AUTISM

· develop consistent vocabulary of symbols used in the absence of

concrete gestures (e.g., child will come into the dining room and say “juice” to mother to request juice without needing to take mother to refrigerator and touch the juice bottle).

· respond to question, “What’s this?” · ask question, “What’s this?” · spontaneously add single words to play, beginning to narrate play

actions.

· use two-word combinations. · use “my” or “mine”. · refer to self by name. · ask questions by raising pitch at end of word or phrase. · ask for help. · say first and last name when asked. · use pronouns “I,” “me,” and “you.” · talk about an event that has just happened. · respond to “what” and “who” questions. · respond to “where” and “when” questions. · respond to “why” questions. · spontaneously ask “wh-” questions. · use complex language in imaginative play to narrate actions. · use prepositions “in,” “on” and “under.” · describe objects according to size, color and shape. · use pronouns “he,” “she,” “they,” “his,” “her,” “our,” and “their.” · use the terms “here,” “there,” “this,” and “that.” · ask meaning of new words. · retell a brief story. · tell home address. · talk about the future using “will.” · use pronouns “himself” and “herself.” · compare objects using “-er” and “-est” endings. 189

FROM GOALS TO DATA AND BACK AGAIN

Conversational skills/Pragmatics Child will:

· use attention-getting words such as “Hey!” · use appropriate volume with conversational partner. · use meaningful inflection with conversational partner. · use appropriate distance between self and conversational partner. · make appropriate adjustments when initiating conversation in order to

gain and keep partner’s attention (i.e., raising his/her voice, adding a gesture).

· attend to peers when they address her/him, responding appropriately. · say “What?” or “Excuse me, could you say it again?” or similar phrase

when he/she doesn’t understand question posed by an adult.

· will respond in appropriate, multi-word phrases when others initiate

conversation.

· use eye contact to signal conversational turn taking. · be able to engage in conversation over a broad range of topics. · add new, relevant information to previous comments in conversation. · ask questions that are related to topic to maintain conversational flow. · make transition statements to signify a change in conversational topic. · put his/her thoughts on pause so adult/peer can add to, or comment on,

the conversation.

· initiate conversation that is of interest to social partner. · change style of interaction when speaking with very young children. · change style of interaction when speaking with peers as opposed to

adults.

· use names of adults/siblings/peers when addressing them. · ask “how,” “why,” and “when” questions in order to obtain information. · provide relevant information to adult when it is requested. · provide relevant information to peers/sibling when it is requested. · share experiences through narration (describing connection between

settings, characters’ behavioral and emotional responses, and consequences).

190

APPENDIX A BASIC INTERVENTION GOALS FOR CHILDREN WITH AUTISM

Sensory issues Child will: · eat a greater variety of foods. · gain comfort with activities in which his/her feet are off the ground. · become sensitized to, and appropriately label, hot/cold/pain. · walk around toys, pets, and people on floor. · successfully avoid bumping into people. · develop compensatory strategies for feeling comfort while in large, open

spaces.

· employ appropriate strategies to reduce overwhelming stimuli in new

environments.

· become more comfortable with activities that involve hands and face. · become more comfortable with multiple voices singing. · tolerate proximity of other children. · remain socially engaged, as is typical for Child, in the midst of a group

of children.

· remain socially engaged, as is typical for Child, in new environments.

Restricted interests and perseverative behaviors · Instances of perseveration [specify types] will be successfully redirected. · Instances of idiosyncratic motor behaviors will decrease. · Playing with toys or objects in atypical/repetitive ways will decrease. · Reciting passages from books, videos, TV, and/or radio will decrease. · Child will stay focused on shared conversation with caregivers instead of

lapsing into private reference.

· Instances of perseveration around rules, when child appears bossy, will

decrease.

· Child will tolerate changes in routines. · Child will demonstrate interest and pleasure in a range of

developmentally appropriate play activities.

· Child will expand repertoire of social play activities.

191

FROM GOALS TO DATA AND BACK AGAIN

Concept development Child will: · demonstrate an understanding of self by pointing to self in mirror, in

photos, and/or by labeling self by name.

· demonstrate an understanding of ownership by word, sign, or gesture. · demonstrate understanding of function of familiar objects by selecting

correct item or insisting on correct item when “mistakenly” given wrong item.

· demonstrate knowledge of the spatial concepts in, on, and under. · demonstrate understanding of quantity concepts one, more, and all. · demonstrate knowledge of gender by pointing to boy/girl upon request. · demonstrate an understanding of the spatial concepts front and back by

moving his/her body or moving objects.

· demonstrate knowledge of front and back of clothes. · demonstrate spatial concepts above/below and top/bottom. · demonstrate understanding of same/different. · demonstrate understanding of first/middle/last. · demonstrate understanding of causality as demonstrated by appropriately

answering “why” questions.

· demonstrate a growing understanding of time and sequence by using

time markers in conversation [in the following order: now, later, soon, before, after, breakfast time, lunch time, dinnertime, morning, afternoon, night, yesterday, today, tomorrow, a long time ago, days of the week, months of the year].

· recall recent/familiar events with logical sequence. · demonstrate an understanding of locative state and prepositions by

answering “where” questions.

· comprehend the word “not” in sentences, such as “Which car is not in the

line?”

· be able to group items into the following categories: color, size, shape,

function, texture, taste, and temperature.

· practice sorting by one attribute. · practice sorting by more than one attribute at a time.

192

APPENDIX A BASIC INTERVENTION GOALS FOR CHILDREN WITH AUTISM

· accurately describe the relationship of both immediate and extended

family members using the appropriate labels for relatives.

· demonstrate an understanding of graduated size by stacking and nesting

blocks.

· use the prefix “-est” to demonstrate knowledge of relative size. · draw accurate inferences from auditory information, answering questions

such as “What do you think will happen next?” or “How do you think so-and-so might be feeling?”

· demonstrate the ability to guess, speculate, estimate, and imagine to

arrive at an answer or to solve a problem.

Increasing social awareness Child will: · watch what others are doing and shape his/her behavior accordingly. · be able to identify what another person is experiencing. · identify what another person knows. · predict what others might see or hear in a given situation. · demonstrate an awareness of the needs of others by spontaneously

offering help.

· receive a daily compliment for being considerate. · predict what others might think or feel in a given situation. · demonstrate concept that his/her actions have an effect on the way

other people feel.

· demonstrate the knowledge that other people do not know what Child

is thinking or feeling.

· demonstrate the ability to teach another person how to do something,

figuring out just what that other person needs to know.

Social skills with peers Child will: · successfully initiate conversation/play with peer. · be able to formulate a new plan of action when someone does not want

to play with him/her.

· appropriately respond to peers when they make social overtures. 193

FROM GOALS TO DATA AND BACK AGAIN

· decline an invitation to play or converse using appropriate

communication.

· develop tactful responses to describe dislikes and disagreements. · sustain interaction with peers. · be able to join others already engaged in a play activity (as opposed to

having a peer join her/him in their activity).

· tolerate and stay engaged in play with peer even when not in charge. · communicate with peers when ready to change activities. · demonstrate flexibility and the ability to adapt in social settings by

accommodating play suggestions from peers.

· sustain interaction on a playdate. · share toys when appropriate. · successfully negotiate over toys. · demonstrate appropriate responses to children who are mean or hurtful. · apologize if and when he/she bumps into someone. · apologize if and when he/she hurts someone’s feelings or body. · learn to talk on the phone in a developmentally appropriate manner.

Social norms Child will: · wait for his/her turn to talk, in an age-appropriate manner. · demonstrate an understanding of modesty and/or privacy by being fully

clothed when leaving the bathroom in public places.

· refrain from publicly touching private body parts. · wipe nose on tissue and throw tissue away. · demonstrate an understanding of ownership by refraining from taking

someone else’s food or belongings.

· demonstrate age-appropriate modesty. · demonstrate age-appropriate tact. · refrain from asking embarrassing or intrusive questions of conversational

partner.

· refrain from interrupting people who are talking on the phone.

194

APPENDIX A BASIC INTERVENTION GOALS FOR CHILDREN WITH AUTISM

School and camp skills Child will: · follow teacher’s instructions, in an age-appropriate fashion. · attend to verbal instructions, using compensatory strategies when

necessary.

· successfully transition between activities. · participate in large group activities. · participate in small group activities. · raise hand when wishing to speak and will wait until he/she is called on

before speaking out loud.

· tolerate the times when he/she is not picked for an activity. · listen to a book read aloud to the group and will give information about

the story when asked by the teacher.

· follow the typical schedule. · tolerate changes in the schedule when prepared for these changes. · stay in close proximity to the group when on field trip. · demonstrate the ability to use the teachers for needed help and support

with a decreasing reliance upon therapist/aide.

· tolerate being at the end or in the middle of the line. · tolerate not being first. · note what others are doing and shape behavior accordingly.

Leisure Child will: · increase repertoire of tolerable family outings. · keep caregivers informed of where he/she is going during outings. · join routine family activities, from start to finish, without needing to be

continually prompted to stay focused.

· join non-routine family activities, from start to finish, without needing to

be coerced or continually prompted to stay focused.

· demonstrate understanding of winning and losing. · be able to tolerate winning and losing. · demonstrate increasing comfort with participation in physical activities

with other children.

195

Appendix B

Charts and Handouts

The Three Phases of Intervention How Do We Know When Goals Are Complete? Writing Measurable Goals Detecting Change Using The Comparison of Means Sample Data Sheet — 2-year-old, more compromised Sample Data Sheet — 4-year-old, more compromised Sample Data Sheet — 6-year-old, moderately compromised Sample Data Sheet — school-aged child, more able Glossary Directions for Using Excel

198 199 200 201 202 203 204 205 206 211

197

198

Chart 3.1

frequent modeling, Johnny will wave goodbye.

T Example: With multiple prompts and

opportunities, the skill EMERGES in a single environment with consistent caregivers.

T With multiple and supported

PHASE 1

prompts, Johnny will wave goodbye to familiar adults and his siblings.

T Example: With a decreasing need for

target CONSISTENCY and INDEPENDENCE.

T Now that the skill has emerged, we

PHASE 2

will consistently wave goodbye when someone who is leaving waves to him or says “goodbye.”

T Example: In all environments, Johnny

certain conditions, expectations are EXTENDED to include multiple environments and numerous people.

PHASE 3 T Now that there is consistency under

Collecting data for children with autism changes over time. There are distinct phases.

The Three Phases of Intervention

FROM GOALS TO DATA AND BACK AGAIN

Copyright © 2001 Lehman and Klaw

APPENDIX B CHARTS AND HANDOUTS

How Do We Know When Goals Are Complete? Skill targeted in goal · Family input · Team input · Curriculum · Developmental guidelines

Skill emerges during 1:1 intervention in single environment · Learned incidentally · Taught through naturalistic means · Taught through drills

Skill improves during 1:1 intervention in single environment · Becomes developmentally appropriate · Becomes consistent · Becomes independent

Skill generalizes · Adults

siblings

peers

· Home

familiar environment

community

P Goal Complete

Copyright © 2001 Lehman and Klaw

Chart 3.2

199

200

Chart 3.3

QUANTIFY IT

QUALIFY IT

How we measure it: TAKE DESCRIPTION OF DESIRED BEHAVIOR

What we measure: CHANGE OVER TIME

T $ need for prompts

T # range of behavior

T 2 duration of behavior

T 2 frequency of behavior

QUANTIFY

T Type of measurement

T Qualitative scale

$ Need for prompts

T List mastered over time

# Range of behavior

T Quantitative time scale

2 Duration

T Qualitative scales

· Percentages

T With whom T Level of support

· Tally of occurrence

T Quantitative scales

2 Frequency

T When (how often)

T Where

QUALIFY

Writing Measurable Goals

FROM GOALS TO DATA AND BACK AGAIN

Copyright © 2001 Lehman and Klaw

Copyright © 2001 Lehman and Klaw

Chart 8.1

standard deviations.

T Calculate the overlap in the

between the “before” and “after” means.

T Calculate the difference

standard deviation for the “before” and “after” data sets separately.

T Compute the mean and

it “after.”

T Copy the last third and label

T Ignore the middle third.

it “before.”

T Copy the first third and label

T Divide data into thirds.

FIRST

but look for correlations to help understand why.

T …Conclude that there’s been no reliable change,

category width apart, then…

T If the “before” and “after” means are not a

no reliable change, but look for correlations to help understand why.

T If there’s a large overlap... conclude there’s been

correlations with other factors before you decide if there’s been progress.

T If there’s a small overlap... look for outliers and

been reliable change!

T If there’s no overlap... conclude that there has

check the reliability of the difference in the means.

T ...Use the overlap in the standard deviations to

category width apart, then...

T If the “before” and “after” means are at least

category width.

T Check the scale for this goal to determine its

NEXT

to plan the next round of intervention!

T Discuss the data with the team

LAST

Detecting Change Using The Comparison Of Means

APPENDIX B CHARTS AND HANDOUTS

201

FROM GOALS TO DATA AND BACK AGAIN

Sample Data Sheet — 2-year-old, more compromised Child: Danielle

Date:______

Time of day: Morning afternoon

evening

What was the longest amount of time Danielle stayed socially engaged with you during a preferred activity?

$1 min $3 min $5 min $7 min

When given 3 short breaks over the course of the therapy session, how many times did Danielle seek you out to begin playing again?

0/3 1/3 2/3 3/3

When you attempt to engage Danielle in some familiar play routine, how often did she immediately smile at you?

¨ ¨ ¨ ¨

Did Danielle look at you when requesting something?

¨ rarely, even when response withheld to encourage eye

When a pleasurable play routine was briefly interrupted, did Danielle use body language, affect, eye gaze and vocalizations to indicate that she wants to continue?

¨ ¨ ¨ ¨

rarely smiled at first smiled occasionally at your overtures smiled often at your overtures smiled consistently when approached in a playful manner

gaze ¨ about half the time when response withheld ¨ used eye gaze with requests some of the time ¨ frequently used eye gaze with requests

didn’t communicate a wish to continue used affect/body language used affect/body language, and eye gaze used affect/body language/eye gaze, and vocalizations

How many times did Danielle engage in early pretend play after you modeled this for her?

0 1 2 3

3

When Danielle needed help, did she ¨ even with prompting, unable to do this get someone’s hand and pull that ¨ did this once with multiple prompts person to the problem? ¨ did this more than once with prompts ¨ did this at least once on her own How long was Danielle able to look at a book with you?

1 min 2 min 3 min 4 min

Did Danielle vocalize today while playing?

202

¨ ¨ ¨ ¨

consistently quiet vocalized a little during certain activities vocalized a little throughout the session vocalized consistently throughout the session

Copyright © 2001 Lehman and Klaw

APPENDIX B CHARTS AND HANDOUTS

Sample Data Sheet — 4-year-old, more compromised Child: Patrick

Observer:____________

Date: ______ On a scale of 1 to 5, please rate Patrick’s day overall.

1 2 3 4 5 (Distressed/distracted) (content/alert)

1. How much time did it take for Patrick o to settle into the playgroup experience o and begin to participate? o

30 minutes 20 minutes 10 minutes

o 10 minutes 2. On average, how did Patrick respond to parallel play activities?

o o o o

3. On average, how long was Patrick able to participate in the parallel play activities with support as needed?

o participated only for a couple of minutes o participated for about half the expected length of time o participated most of the time o stayed with the group for the whole time

very resistant today somewhat resistant guarded participation willing participation

4. Please circle or list the sensory activities in which Patrick participated today: Play dough / fun dough; shaving cream; water play; sand; finger painting; pudding painting; jello play; _______________; _______________ 5. Please circle or list the common parallel play activities in which Patrick participated today: large blocks; little blocks; sticky blocks; cars; ball play; emptying and filling; early doll play; jello-in-the-bowl; _______________; _______________ 6. Did Patrick use contact gestures such as grabbing your hand or tapping you to get your attention when he communicates wants/needs?

o o o o

7. Did Patrick spontaneously look at you when making requests?

o needed to be prompted o occasionally o half the time o half the time

8. On the average, what level of prompt was needed to assist Patrick in signing “more” to request the continuation of a pleasurable activity?

o o o o

Copyright © 2001 Lehman and Klaw

only occasionally some of the time frequently consistently

hand-over-hand modeling with elbow prompt modeling only independent 203

FROM GOALS TO DATA AND BACK AGAIN

Sample Data Sheet — 6-year-old, moderately compromised Child: Kevin

Date: ______

Time in: ______ AM/PM Time out: ______ AM/PM

Place of Service:

¨ Classroom ¨ Resource room

How would you rate Kevin’s overall performance today?

1 2 3 4 5 (self-absorbed) (easy to engage)

Rate Kevin’s ability to recall daily events, without visual support:

1 2 3 4 5 (difficult) (easy)

Given 4 opportunities over the course of intervention, how often did Kevin independently respond to WHEN questions?

¨ ¨ ¨ ¨ ¨

0/4 1/4 2/4 3/4 4/4

Did Kevin participate in social pretend play with little figures?

¨ ¨ ¨ ¨ ¨

minimal participation, even with support improving participation, supported participating with minimal need for support some independent participation consistently independent

¨ no opportunity to observe When recalling daily events, how often was Kevin able to correctly classify these as having occurred in the morning, afternoon, or evening?

Number of times correct:______ Number of opportunities:______ ______%:

When Kevin’s friends were already engaged in play, was he able to join them?

¨ ¨ ¨ ¨ ¨

refused to join even with prompts joined friends with multiple prompts joined friends with just a single prompt spontaneously joined friends 1–2 times spontaneously joined friends many times

Did Kevin independently initiate conversation with peers?

¨ ¨ ¨ ¨

only when prompted once or twice a few times many times (quite chatty)

On the average, what level of support was needed to help Kevin sustain interaction with friends?

¨ ¨ ¨ ¨

was unable to sustain even with maximum support frequent verbal prompts/some physical support frequent verbal prompts occasional verbal prompts

Rate Kevin’s overall flexibility when interacting with peers.

1 2 3 4 5 (inflexible) (flexible)

On the average, how effectively was Kevin ¨ did not protest, even with modeling/prompting able to protest when play with peers ¨ protested with prompting from adults became too rough? ¨ protested independently but was ineffective ¨ protested immediately and effectively 204

Copyright © 2001 Lehman and Klaw

APPENDIX B CHARTS AND HANDOUTS

Sample Data Sheet — school-aged child, more able Child: Linda

Time in: ______AM/PM

Date:______

Time out: ______ AM/PM

Please indicate on a scale of 1 (self-absorbed/idiosyncratic) to 5 (easily engaged/easily integrated), an overall assessment to Linda’s day.

o o o o

home home w/ play date after-school program community outing

o o o o

refused to discuss discussed with much prompting discussed with minimal prompting discussed with single prompt only, such as “Let’s talk about...”

1 2 3 4 5 When provided with an opportunity for discussion, was Linda able to discuss the effect of her negative behavior on others?

o no opportunity to observe Given 4 opportunities to respond to you calling her name, what percentage of the time did Linda respond appropriately after only a single cue (e.g. “Linda, look at me a minute...” or “Linda, I need to talk to you”)

o o o o Cue used:_________________________ o Over the course of an hour play date, how many times did Linda wander way and take a break from the interaction?

o not applicable Did Linda demonstrate a willingness to do something her friend wanted to do even if she didn’t want to do it?

o no opportunity to observe How did Linda react when she was playing with peers and things didn’t go her way? (Note: When Linda has a tantrum, she screams, yells inappropriate things, may try to hit others and may throw things)

o no opportunity to observe

Copyright © 2001 Lehman and Klaw

0/4 = 0% 1/4 = 25% 2/4 = 50% 3/4 = 75% 4/4 = 100%

o needed multiple breaks of 5 or more minutes o needed multiple momentary breaks o needed only 1-2 breaks o needed no breaks o refused o agreed to do a non-preferred activity but attempted to change or control the activity once it began o negotiated a compromise o agreed to participate in non-preferred activity for the sake of friendship o tantrummed and had to be removed o tantrummed but with support could re-engage in activity o no tantrum but perseverated on disappointment throughout activity o expressed disappointment in appropriate manner and agreed to continue activity

205

FROM GOALS TO DATA AND BACK AGAIN

Glossary boundary condition: a number that is chosen to distinguish among alternatives when making a decision. For example, in the comparison of means there are two boundary conditions: the minimum category width and the overlap between the standard deviations. In each case there is a numeric value that determines whether change is or is not present. A boundary condition is typically somewhat arbitrary but must be chosen to make a procedure uniform and repeatable (Chapter 8). categorical data: data whose values can be only names or categories. Typical examples include where intervention takes place and the child’s gender. When data is categorical no value is better than any other value. Also called nominal data (Chapter 6). category width (category boundary): the numeric difference between scores that corresponds to a qualitative difference in behavior. In a 4-point qualitative scale, for example, a score of 1 corresponds to a different category of behavior than a score of 2, a score of 2 reflects different behavior than 3, and so on. So the minimum category width for a qualitative scale is 1.00, and a change in the means from 1.00 to 3.00 would reflect progress across two category boundaries (Chapter 7). change: the transformation of one regular pattern of behavior into another regular pattern of behavior over time as a result of learning or development (Chapter 7). See also: pre-transformation phase and post-transformation phase. consistency: a term that refers to frequent and repeated practice of behaviors or skills within specific settings. Consistency is the overall focus of Phase 2 of intervention (Chapter 3). See also: three phases of intervention. correlation: a relationship in which variation in the value of one factor or goal predicts variation in the value of a different factor or goal (Chapter 9). See also: positive correlation and negative correlation. data collection: the process that turns a desire for behavioral change stated in general terms into a set of questions that can be answered by observing a child’s actions (Chapter 6). data analysis: the process that turns observations back into meaningful statements about changes in the child’s behavior over a particular period of intervention (Chapter 6).

206

Copyright © 2001 Lehman and Klaw

APPENDIX B CHARTS AND HANDOUTS

data point: the number, checkmark, word, or phrase used to record a child’s performance on a data collection sheet. Also referred to as an observation, value, or score (Chapter 6). data set: a set of values (or data points, observations, or scores) that represent performance in a coherent way. For example, all the observations associated with one goal form a data set, but so do all the scores for a single child, all the data points taken in a particular location, and all the values collected during a single session (Chapter 6). data type: a formal description of the mathematical properties of a data set. The type of a data set determines what kind of statistical tests can be applied to the data set. See also: categorical data, interval data, ordinal data, and nominal data (Chapter 6). emergence: a term that refers to the appearance of a new behavior or skill given multiple and supported opportunities in a sheltered, consistent, and supportive therapeutic environment. Emergence is the overall focus of Phase 1 of intervention (Chapter 3). See also: three phases of intervention. empirical developmental intervention: an approach that incorporates data collection and analysis into intervention that is based on developmental and cognitive theory (Chapter 1). extension: a term that refers to the expansion of newly acquired behaviors or skills to multiple settings. Extension is the overall focus of Phase 3. (Chapter 3). See also: three phases of intervention. factor data: any set of values or observations that has been generated by a goal-independent factor. Typical examples of factor data include the dates of sessions held during the interval of time being analyzed, numeric evaluations of the child’s overall mood each session, and the name of the location where each session was held during the interval (Chapter 6). goal data: any set of observations that measure a child’s performance on a goal (Chapter 6). goal-independent factor (or goal factor): a variable that may affect a child’s performance even though it has nothing to do with the particular skill being taught. Typical examples of goal-independent factors are time of day, time of year, changes in routine, changes in caregivers, and changes in medication (Chapter 3). hypothesis: a tentative expectation about performance or its measurement that is based on theory or data (Chapter 9).

Copyright © 2001 Lehman and Klaw

207

FROM GOALS TO DATA AND BACK AGAIN

interval data: data whose values can be arranged in a meaningful order and in which the intervals between values are equal and obey the laws of arithmetic. Typical examples of interval data include tallies and percentages (Chapter 6). mean value: a data set’s arithmetic average. Alternatively, a representative value for a data set (Chapter 7). negative correlation: a relationship in which increase/decrease in the value of one factor or goal predicts decrease/increase in the value of a different factor or goal (Chapter 9). noise: values that contribute to an apparent lack of regularity in a pattern of data even when such regularity actually exists (Chapter 7). nominal data: data whose values can be only names or categories. Typical examples include a child’s name or gender. When data is nominal no value is better than any other value. Also called categorical data (Chapter 6). observation: the number, checkmark, word, or phrase used to record a child’s performance on a data collection sheet. Also referred to as a value, score, or data point (Chapter 6). ordinal data: data whose values can be arranged in a meaningful order. The intervals between the values are not known to be equal and cannot be combined arithmetically. The translation into numbers of observations that were recorded on a 4-point or 5-point qualitative scale always results in ordinal data (Chapter 6). outlier: a single value that stands out against an otherwise regular pattern of behavior (Chapter 9). percentage: a system of measurement that is used to evaluate progress when the frequency of opportunity for the behavior is known. Percentages are derived by dividing the number of times a behavior is observed by the number of opportunities for that behavior, then multiplying by 100 (Chapter 3). positive correlation: a relationship in which increase/decrease in the value of one factor or goal predicts increase/decrease in the value of a different factor or goal (Chapter 9). post-transformation phase: the time period during which the child’s regular pattern of behavior reflects learning of a new skill, emotion, or behavioral response. In the comparison of means method, the last third of the data collected during the interval being analyzed is considered to come from the post-transformation phase (Chapter 7). See also: pre-transformation phase and transformation phase.

208

Copyright © 2001 Lehman and Klaw

APPENDIX B CHARTS AND HANDOUTS

pre-transformation phase: the time period during which the child’s dominant pattern of behavior remains ineffective, inappropriate, or maladaptive despite intervention. In the comparison of means method, the first third of the data collected during the interval being analyzed is considered to come from the pre-transformation phase (Chapter 7). See also: post-transformation phase and transformation phase. qualifying a goal: a process in which you add text to a goal statement that is specific about the variables that surround the desired outcome. Where the intervention is being done, with whom, when, and with what level of support are all considerations when goals are qualified (Chapter 3). qualitative scale: a descriptive scale that measures change by advances along a behavioral progression (Chapter 3). quantifying a goal: a process in which you choose a system for measuring progress such as increasing/decreasing frequency, increasing/decreasing duration, increasing a range of behaviors, and/or decreasing a need for prompts (Chapter 3). quantitative scale: a numeric scale that measures progress by increasing or decreasing quantity (Chapter 3). raw data: observations as they appear on the data collection sheet or in the spreadsheet prior to the application of any statistical tests (Chapter 6). reliability: a property of change as measured by the relationship among the means and standard deviations “before” and “after” learning. If change is reliable then the “after” mean is predictive of the child’s ongoing level of performance without additional intervention. score: the number, checkmark, word, or phrase used to record a child’s performance on a data collection sheet. Also referred to as an observation, value, or data point (Chapter 6). standard deviation: a number that reflects our notion of representation as a function of distance from, or clustering around, the mean. As the standard deviation increases, the means ability to represent a data set decreases, as does the reliability of any change (Chapter 8). tally of occurrence: a system of measurement in which the observer counts every time a targeted behavior is performed. This system is used when there is a constant opportunity to perform the behavior (Chapter 3).

Copyright © 2001 Lehman and Klaw

209

FROM GOALS TO DATA AND BACK AGAIN

three phases of intervention: emergence, consistency, and extension. Every goal passes through these 3 phases on its way to mastery (Chapter 3). See also: emergence, consistency, and extension. transformation phase: the time period during which the child is learning a new skill, emotional response, or behavior. In the comparison of means method, the middle third of the data collected during the interval being analyzed is considered to come from the transformation phase (Chapter 7). See also: pre-transformation phase and post-transformation phase. type of data: a formal description of the mathematical properties of a data set. The type of a data set determines what kind of statistical tests can be applied to the data set. See also: categorical data, interval data, ordinal data, and nominal data (Chapter 6). value: the number, checkmark, word, or phrase used to record a child’s performance on a data collection sheet. Also referred to as an observation, score, or data point (Chapter 6).

210

Copyright © 2001 Lehman and Klaw

APPENDIX B CHARTS AND HANDOUTS

Directions for Using Excel A very quick reference guide

Please note: in these instructions we use the words “pull down” or “click” for mouse actions and the word “press” for keyboard actions.

Selecting To select a single cell: click on the cell you want to select. To select an entire column: click on the letter at the top of the column. To select an entire row: click on the number at the left of the row. To select a set of adjacent cells: click on the first cell. Then without letting your mouse button up, move the mouse to the last cell. This is called “dragging.” As you drag over a cell, it will be highlighted. When you release your mouse button after highlighting a set of cells, the entire highlighted area is selected. To select non-adjacent cells: select the first cell or range of cells and then, holding the CTRL key down, select each additional cell one at a time.

Deleting To delete a value or values: select a cell or range of cells and press DELETE. To delete a row or column: select the row or rows, column or columns. Pull down the Edit menu and click Delete, then entire row or entire column. To delete a sheet: pull down the Edit menu and click Delete sheet.

Inserting To insert a row or column: select the box below or to the right of the area where you want to insert a new row or column. Pull down the Insert menu then click Rows or Columns. All of the columns or rows will be renumbered automatically. Inserting a worksheet: pull down the Insert menu, then click Worksheet.

Copyright © 2001 Lehman and Klaw

211

FROM GOALS TO DATA AND BACK AGAIN

Changing or modifying To change the width of a column: drag the boundary line on the right side of the column heading until the column is the width you want. To change the column width for multiple columns, select the columns you want to change. Then drag a boundary at the right of any selected column heading. To make the column width fit the contents, double-click the boundary to the right of the column heading. To change the height of a row: drag on the boundary line below the row heading until the row is the height you want. For multiple rows, select all the rows and then adjust the boundary to one of them. To make the row height fit the contents, double-click the boundary below the row heading.

Copying and pasting To copy a cell: select the cell, pull down the Edit menu and click Copy (or use the Copy icon on the tool bar). To copy a column: click on the column heading to select the whole column. Then pull down the Edit menu and click Copy (or use the Copy icon on the tool bar). To copy a row: click on the number of the row to select the whole row. Pull down the Edit menu then click Copy (or use the Copy icon on the tool bar). To copy an entire data sheet: pull down the Edit menu then click Move or copy sheet. Click where you want this sheet to be in relation to the list of other sheets in the workbook. Click Create a copy, then OK. To paste a cell, column, or row: Move the cursor to the desired location. Pull down the Edit menu then click Paste (or use the Paste icon on the toolbar).

212

Copyright © 2001 Lehman and Klaw

APPENDIX B CHARTS AND HANDOUTS

Miscellaneous To move a row or column: select the whole row or column. Now click on the highlighted area and drag it where you want it to go. To rename your sheet: double-click on the tab containing the sheet name at the bottom of the screen (for example, Sheet 1) to select it. Now type the name you want. To save your data: pull down the File menu and click Save. To get help: pull down the Help menu and use the search mechanism available after clicking Contents and Index.

Copyright © 2001 Lehman and Klaw

213

Appendix C

Exercises

We recognize that some readers of this book want to learn a practical method of data collection and analysis that can be used in their everyday lives. These exercises can help you achieve that goal by giving you the chance to practice the different skills we discuss. When you begin to collect and analyze your own data you may find it convenient to pattern your worksheets on the ones we’ve created, even inserting your data into copies of the sheets you’ve practiced on. The number of each exercise reflects the chapter that discusses the skill you will practice. Exercises 3.1 through 5.8 give practice in the skills needed to write measurable goals. In order do these exercises online, you must copy the file “Goal Workbook.doc” from the CD-ROM to your hard drive. The remaining exercises (6.1 and higher) give practice in data analysis. To do these exercises you must copy the file “Analysis Workbook.xls” from the CD-ROM to your computer’s hard drive. Should you mistakenly modify or destroy any of the data in these files while working through the exercises, simply recopy the original files from your CD-ROM. Note that in the spreadsheet exercises (Exercise 6.1 and higher) all directions and figures are based on the PC version of Microsoft® Excel 97 using the Chart Wizard feature. You may find small differences between our instructions and your experiences if you are using a later version of Excel on a PC or any version of Excel on a Mac. The files in “Analysis Workbook.xls” can be opened in both PC and Macintosh versions of Excel 97 and should be compatible with later versions of Excel on both types of computer. The instructions for the spreadsheet exercises assume that you are already able to perform basic Excel actions such as selecting a column of data, copying the contents of a cell, using the on-line help facility, and so on. If you are unfamiliar with Excel at this level, you may want to keep a copy of “Directions for Using Excel” close at hand for easy reference. “Directions for Using Excel” can be printed from the CD-ROM or photocopied from Appendix B.

215

FROM GOALS TO DATA AND BACK AGAIN

Exercise 3.1 What’s wrong? Correct the following goals and data questions by finding errors, lack of clarity, or developmentally problematic expectations. The answers follow. 1.

Goal: With a decreasing need for assistance, Johnny will use picture exchange to request a drink 75% of the time during snack time at school.

2.

Goal: Sally will use eye gaze to bring attention to herself (as if to say “Look at me!”) 75% of the time.

3.

Goal: Sally will begin to sustain social interaction with adults, with siblings, or with peers in school for up to 10 minutes.

4.

Goal: Johnny will appropriately answer “wh-” questions 50% of the time. Data question: In your estimation, what percentage of the time was Johnny able to answer “what,” “who,” “when,” “where,” and “why” questions? <25% ~25% ~50% ~75%

5.

Goal: Sally will sort items by four single attributes. Data question: How many attributes did Sally use to sort or categorize today? 0 1 2 3 4

6.

Goal: With a decreasing need for prompts, Johnny will raise his hand and remain quiet until the teacher has called on him. Data question: What level of prompt was needed for Johnny to raise his hand and remain quiet until called on to speak in school? • verbal and physical prompts • verbal prompts • single verbal prompt • no prompt needed

216

APPENDIX C EXERCISES

Answers to Exercise 3.1 1.

Goal: With a decreasing need for assistance, Johnny will use picture exchange to request a drink 75% of the time during snack time at school.

You can’t measure both a “decreasing need for assistance” and percentage. Those are two different systems for measuring change over time. What would your data show? Would it show a decreasing need for assistance or an increasing percentage of time? You wouldn’t know. To correct this goal, you can do either of the following: With a single prompt, Johnny will use picture exchange to request a drink 75% of the time during snack time at school. OR: With a decreasing need for prompts, Johnny will use picture exchange to request something to drink during snack time at school.

2.

Goal: Sally will use eye gaze to bring attention to herself (as if to say “Look at me!”) 75% of the time.

You can’t quantify the number of opportunities Sally has to bring attention to herself. There is no way to count this—the opportunity is constant. Therefore, you cannot use percentages as a method for measuring progress for this goal. To correct this goal, you could measure the following: With increasing frequency, Sally will use eye gaze to bring attention to herself (as if to say “Look at me!”).

In this case, you would take a tally to establish progress over time. 3.

Goal: Sally will begin to sustain social interaction with adults, with siblings, or with peers at school for up to 10 minutes.

Sustaining social interaction with adults, with siblings, and with peers in school are three different goals that typically appear at three very different times during development. You shouldn’t try to measure all three in a single goal. 4.

Goal: Johnny will appropriately answer “wh-” questions 50% of the time. Data question: In your estimation, what percentage of the time was Johnny able to answer “what”, “who”, “when”, “where” and “why” questions? <25% ~25% ~50% ~75%

217

FROM GOALS TO DATA AND BACK AGAIN

5.

(a)

The “wh-” questions are mastered by typically developing children at very different times. If you want to target the “wh-” questions, you should write separate goals.

(b)

Estimating percentages is a shaky process. Remember: if you need to estimate because there are too many opportunities to count, then you should probably pick another method of measurement.

Goal: Sally will sort items by four single attributes. Data question: How many attributes did Sally use to sort today? 0 1 2 3 4

Let’s say the team wants Sally to be able to sort by color, shape, size, and function. Over time, they hope to teach her all of these methods for sorting. In setting up the data question this way however, they will never know when she has actually achieved all four skills. Maybe every day the data sheet comes back showing that she only sorted by one attribute. They can’t tell which attribute she is using. Perhaps she sorts by color every single day. But perhaps she sorts by a different attribute every day. If she is sorting by different attributes, she may have achieved the goal a long time ago. You could write the data question the following way: What attributes did Sally use for sorting today? o color o shape o size o function o other________

6.

Goal: With a decreasing need for prompts, Johnny will raise his hand and remain quiet until the teacher has called on him. Data question: What level of prompt was needed for Johnny to raise his hand and remain quiet until called on to speak in school? • verbal and physical prompts • verbal prompts • single verbal prompt • no prompt needed

It is important to design a prompt hierarchy so that it matches both the child’s need and the situation. In this case, the scale for prompting does not match a classroom environment. There are many ways to help a child raise his hand and remain quiet. Telling a child in a classroom to “Be quiet” or “Raise your hand” is a fairly intrusive prompt. It shouldn’t be the last step before independence. What about a gesture from the teacher? What about the use of visual reminders? What about the use of peer modeling? 218

APPENDIX C EXERCISES

Exercise 4.1 Creating data sheets using tables The easiest way to create data sheets like the ones in this book is to lay the information out in tables. In Microsoft® Word, table functions are found in the Table menu of the standard menu bar. Tables are complex objects that can be formatted in many different ways. In this exercise you will use some of the most basic commands to reproduce the following data sheet (a variant of Figure 4.1). Child’s name: ____________

Observer’s name:____________

Date:______

Time: ______

Please check the types of activities Peter participated in today during free play:

o o o o o o

What percentage of the time did Peter respond by saying “Hi” to familiar people who came to his classroom during his morning preschool session?

How many people came to his classroom? _____ How many times did Peter say “Hi”? ______

What level of prompt was needed to help Peter say goodbye to familiar adults?

o o o o

gross motor construction/building puzzles art music early pretend play

Calculate: ______% (# of greetings ÷ # of opportunities x 100) verbal and physical prompt (hand-over-hand) verbal with modeling verbal only no prompt needed

1.

Open a blank page in Word. Pull down the Table menu and select Insert Table. A dialog box asks you to specify the number of columns and rows in the table. Since we want a data sheet with three goals and a separate area for factor data we will need two columns and four rows. Fill in those values and click OK.

2.

To create a single box in the top row of the table we need to merge the two existing cells. The easiest way to do this is to select them by clicking and dragging across both cells. If you find this difficult, however, simply click in one cell then pull down the Table menu and click Select Row. Once the two cells have been highlighted (with a black bar) click Merge Cells in the Table menu.

3.

Each cell is its own small environment. Click in the left cell of the second row and type in some text. Notice the font size and font style of 219

FROM GOALS TO DATA AND BACK AGAIN

the text you just typed. Highlight the text and change the font size or font style. Now click in any other cell and type again. You should find that the text in the new cell reverts to the original font size and type. Type in the information for calculating the percentage in the second goal (the blank lines are made by repeated use of the underscore character, which is found above the hyphen on most keyboards). Highlight the text and choose Center from the formatting toolbar to complete this cell. Only this cell’s contents will be centered.

220

4.

In the right cell of the second row type the list of activities in the sample data sheet. Highlight the list and click the Itemize icon in the formatting toolbar (it’s the icon with the three squares next to three lines). The activities will be converted to an itemized list without disturbing any text in the other cells. If your default bullet character is not a square you may be able to change it. Select the list then click Bullets and Numbering in the Format menu. If you see squares as an option in the resulting dialog box, choose it. If you do not see squares, we suggest you read Word’s online documentation for customizing bullets.

5.

Notice that all the cells containing the scales for the data questions have the same width in your table. It is often useful to move the line that divides the data questions from the scales to use the space on the page more effectively. Move your cursor slowly across the dividing line until it becomes a symbol with two parallel lines and two arrows. Click and drag the cursor left or right to move the line. The whole line will move, simply resizing all the divided cells. To resize one cell highlight the whole cell by moving the cursor just inside the left border until you see the cursor change to a large white arrow then click. Now move the cursor over the dividing line until it becomes the parallel line symbol, click and drag. You should find that just the single cell resizes. If you cannot make this work in your file, you can specify an absolute size for the cell using Cell Height and Width in the Table menu.

6.

To get the double lined border click anywhere in the table then click Select Table in the Table menu. Pull down the Format menu and select Borders and Shading. Using the tab for Borders in the dialog box, select Grid for the Setting and then choose the double lines from the list of possible Styles.

APPENDIX C EXERCISES

Exercise 5.1 Writing goals for Joey Write five goals addressing communication needs for Joey. 1.

2.

3.

4.

5.

Copyright © 2001 Lehman and Klaw

221

FROM GOALS TO DATA AND BACK AGAIN

Exercise 5.2 Making Joey’s goals measurable Qualify and quantify the following goals: 1.

Joey will indicate preference when offered the choice of two options.

2.

Joey will associate photographs with preferred items.

3.

Joey will attempt to say “Go!” after being cued with “Ready, set...”

4.

Joey will begin to pull his parents to his object of desire.

5.

Joey will imitate mouth movements with increasing frequency.

1.

2.

3.

4.

5.

222

Copyright © 2001 Lehman and Klaw

APPENDIX C EXERCISES

Exercise 5.3 Writing data questions for Joey’s goals 1. During the same 2-hour period each day, Joey will indicate preference 75% of the time when offered the choice of food or an object with which to play or hold. 2. With increasing frequency, Joey will match photographs to preferred items when given the opportunity. 3. After hearing the phrase modeled twice, Joey will attempt to say “Go!” after hearing “Ready, set...” during a familiar and preferred activity. 4. With a decreasing need for assistance, Joey will begin to pull his parents and/or familiar adults to his object of desire. 5. Joey will imitate the mouth movements of familiar adults with increasing frequency.

Write data questions for the following goals: Data questions

Circle, check, or fill in information

1.

2.

3.

4.

5.

Copyright © 2001 Lehman and Klaw

223

FROM GOALS TO DATA AND BACK AGAIN

Exercise 5.4 Writing goals for Tyler Write five goals addressing Tyler’s communication needs at home. 1.

2.

3.

4.

5.

224

Copyright © 2001 Lehman and Klaw

APPENDIX C EXERCISES

Exercise 5.5 Making Tyler’s goals measurable Qualify and quantify the following goals: 1.

Incidents of private reference occurring in the midst of social exchange will decrease in frequency.

2.

Tyler will learn to secure his family’s attention before talking.

3.

Tyler will look at his communicative partner as he is talking or right before he begins to talk with increasing frequency.

4.

Tyler will be engaged in a daily verbal exchange of increasing duration in which the adult partner avoids questioning and quizzing, and relies instead on commenting, narrating, rephrasing, and expressing opinions.

5.

Tyler will learn to effectively and productively indicate disapproval or protest.

1.

2.

3.

4.

5.

Copyright © 2001 Lehman and Klaw

225

FROM GOALS TO DATA AND BACK AGAIN

Exercise 5.6 Writing data questions for Tyler’s goals Write data questions for Tyler using the following goals: 1. During the 1-hour period of time when the family plays together each evening, incidents of private reference occurring in the midst of social exchange will decrease in frequency. 2. During the 1-hour period of time when the family plays together each evening, Tyler will learn to secure his family’s attention before talking at least half of the time. 3. Without needing to be reminded, Tyler will consistently look at his communicative partner as he is requesting food or drink. 4. With an increasing number of verbal exchanges, Tyler will have one conversation a day in which the adults use a facilitative approach (avoiding questioning and quizzing, and relying on commenting, narrating, rephrasing, and expressing opinions). 5. Without being coached from family members, Tyler will learn to effectively and productively indicate disapproval or protest.

Data questions

Circle, check, or fill in information

1.

2.

3.

4.

5.

226

Copyright © 2001 Lehman and Klaw

APPENDIX C EXERCISES

Exercise 5.7 Writing measurable goals for Mai Lin Write five communication goals for Mai Lin in a school setting. As you write them, remember to qualify and quantify. 1.

2.

3.

4.

5.

Copyright © 2001 Lehman and Klaw

227

FROM GOALS TO DATA AND BACK AGAIN

Exercise 5.8 Writing data questions for Mai Lin’s goals Write data questions for Mai Lin based on the following goals. 1. With a decreasing need for reminders, Mai Lin will appropriately face unfamiliar people when she addresses them. 2. Needing no more than a single prompt, Mai Lin will increasingly use appropriate volume and inflection when speaking to unfamiliar people. 3. With consistency, Mai Lin will effectively repair her communicative attempts after a peer says “What?” or indicates in any other way that Mai Lin’s words were not understood. 4. Once a day, Mai Lin will sustain general conversation for 5 minutes without attempting to suddenly introduce a preferred topic. 5. Needing only an expectant pause as a reminder, Mai Lin will make requests using a question format rather than a statement 75% of the time.

Data questions

Circle, check, or fill in information

1.

2.

3.

4.

5.

228

Copyright © 2001 Lehman and Klaw

APPENDIX C EXERCISES

Exercise 6.1 How to produce a line graph This exercise takes you through the steps required to produce Figure 6.5 in Chapter 6. Don’t panic at the length of this exercise! Because this is the first of the spreadsheet exercises we discuss each step in great detail. As you become increasingly familiar with Excel by working through the remaining exercises, the directions will become briefer. In this and all the remaining exercises italics are used in the instructions for words that appear exactly as you will see them on-screen in Excel. 1.

Open the spreadsheet for Exercise 6.1 in the Analysis Workbook.

2.

Click on the graphing icon (circled in Figure C.1). This will start the Chart Wizard, a sequence of four dialog boxes that help you specify the graph.

Figure C.1 The icon to start the Chart Wizard in Excel 97 is circled in black

3.

Selecting the Chart Type. In Step 1 we choose the kind of graph we want to use to represent the data. First make sure that the Standard Types is showing in this dialog box. You can check this by seeing which tab is in front in the window (see Figure C.2 for the location of the tabs). If Standard Types is not in front, click it to bring it to the front. We want to create a line graph that shows lines with markers displayed at each data value. Choose Line from the Chart Type menu then click on the Chart sub-type darkened in Figure C.2. Click the Next button at the bottom of the dialog box to continue.

229

FROM GOALS TO DATA AND BACK AGAIN

Figure C.2 The options to select in the Chart Type dialog box. Also note the tabs circled at the top of the dialog box. Each tab gives you access to a different view of some or all of the information to be specified at this step

4.

Selecting the data range. In Step 2 of the Chart Wizard, we choose the data to be graphed. Make sure that the Data Range tab is in front in this dialog box. If not, click it to bring that view to the front, then click in the box to the right of the words Data Range. There are many ways to specify the data to be graphed but the easiest is to select rows 2 to 30 of column B in Anton’s spreadsheet. After you do this, notice how the Chart Wizard automatically fills in the formula for the data range and produces an initial version of the graph for your inspection. • Specifying the series name. If you examine the initial version of the graph in the dialog box, you’ll see that the line for the data is labeled with the uninformative name Series 1. Click on the tab labeled Series to bring a different view of the data range forward. You’ll see that Series 1 and the formula specifying its values have already been filled into Series and Values, respectively. To change the name of the series, just type “goal X” into the box for the Name. Notice

230

APPENDIX C EXERCISES

that the label changes in the legend on the graph. Also note how, if there is only one data set to be graphed, Excel also makes its label the default title of the graph (don’t worry, we’ll change that shortly).

• Specifying the horizontal axis. Still in the Series view, look at the horizontal axis (also called the x-axis) of the graph. Excel has given the axis a default labeling by creating a “tick mark” for each data point and numbering the tick marks sequentially. To recreate Figure 6.5, however, we want the tick marks to indicate the date on which each score was recorded. The date of each session is in column A of Anton’s spreadsheet. So, first click in the box to the right of Category (X) axis labels, then select cells A2 through A30 on Anton’s spreadsheet. Excel will update the information in the box and on the graph automatically (it doesn’t always show all the dates if the graph is too crowded to read). Click Next to advance to the next step in the Chart Wizard. 5.

Specifying the chart options. Step 3 of the Chart Wizard allows you to specify different characteristics of the graph. For better or worse, Excel makes many choices about how it displays the data automatically. Some choices are simple to change while others require that you understand a great deal about how Excel works. Luckily, we can compensate for some of Excel’s bad choices easily by placing clarifying titles on the axes of the graphs and on the graph itself. Before we start, make sure that the Titles tab is in front in this dialog box. If not, click it to bring that view to the front. • Chart title. The chart title should explain whose data and which goal we’re looking at. Excel has already filled in what it thinks is a good Chart title, “goal X,” based on the name of the single series in the data range. Replace this default title with “Anton, goal X.” • Category or x-axis title. This title explains the meaning of the measurement units on the horizontal axis. We could type something like “Session date” for this graph, but it would be redundant. To minimize visual clutter, we’ll leave the box blank and let the dates speak for themselves. • Value or y-axis title. This title explains the meaning of the measurement units on the vertical axis. Excel automatically chooses increments for this axis based on the values in the data. In this case, the units it has chosen are inappropriate because they imply that values could fall anywhere in the range 0 to 4.5. There is no easy way to change the increments and eliminate the misinformation so we’ll use the

231

FROM GOALS TO DATA AND BACK AGAIN

axis title to make the actual scale clear to anyone reading the graph. Type in “4-point scale (1,2,3,4).”

• Other options. There are many other features of the graph that can be modified at this step in the procedure but only one—removing the legend—is necessary to produce the graph exactly as it is seen in Figure 6.5. Exercise 6.2 teaches how to remove the legend and demonstrates the other options used to create the figures in this book. For now, just click Next to advance to the last dialog box. 6.

Selecting the chart location. The final step in the Chart Wizard is choosing where to put the graph. Excel gives you the choice of placing the finished graph on a new sheet in the current workbook or on the sheet from which the data was drawn. For the moment we’ll keep the graph with the data. Select As object in: Exercise 6.1 then click Finish.

7.

Compare your completed graph to Figure 6.5 in the book (it should differ only in the presence of the legend) or to the first graph on the sheet called “Answers 6.1.” To compare the graphs online, copy and paste the one you created next to Anton’s graph on the answer sheet.

8.

We encourage you to practice further by using the data provided for Becca and Celeste’s goals (columns D and E). The graphs you produce can be checked against Figure 6.8 and Figure 6.9 in the book or against the second and third graphs on the Answers 6.1 sheet in the Analysis Workbook.

IMPORTANT! If you want to put more than one graph on a sheet it is critical that a graph not be selected when you start the Chart Wizard. If a graph is selected, the Chart Wizard will overwrite the old graph with the new specification. If a graph is not selected, the Chart Wizard will create a new graph but will drop it right on top of the previous graph when it’s done. To keep this from happening, click on any empty cell before starting the Chart Wizard (or simply move the second graph around after it has been created by clicking-anddragging it to another part of the spreadsheet).

232

APPENDIX C EXERCISES

Exercise 6.2 Graphing shortcuts The purpose of this exercise is to show you some of the other features of graphing available in Excel. We assume that you have already done Exercise 6.1 and are now familiar with the basic use of the Chart Wizard. 1.

Open Exercise 6.2 in the Analysis Workbook.

2.

Pre-specifying the data range. Select Anton’s data for goal X (column B, rows 2 to 30). Now click on the graphing symbol, choose the appropriate line graph, and go to the Data Range dialog box. Notice that Excel has already filled in the range using the data that was selected when you started the Chart Wizard.

3.

Pre-specifying the series name. Select the formula for the data range in the box next to the words Data Range. We’re going to replace this formula by selecting rows 1 to 30 of column B. (If Excel gives you an error message when you try to do this, make sure you’ve selected the whole formula in the Data Range box then select rows 1 to 30 of column B again.) Notice that Excel considers text at the top of a column as the label for that column of data. If you pick up cell B1 at the same time you pre-specify the data (see above) the Chart Wizard will include it in its initial graph.

4.

Pre-specifying the horizontal axis. Just as it treats text in the first cell of a column as special, Excel also treats text in the first column of multi-column data as special. Click on Cancel to eliminate the current graph. Now select columns A and B from row 1 to 30 in Anton’s spreadsheet. With these cells highlighted, click the graphing icon and choose the line graph. The initial graph created by the Chart Wizard now has the data range, series name, and x-axis labels already filled in. Click Next to go to the Chart Options dialog box.

5.

Removing the legend. The legends for this graph and the graphs we produced in Exercise 6.1 are not particularly informative. To eliminate the legend and create more space for displaying the data, click on the Legend tab to bring the appropriate formatting information to the front. Now click on the check mark next to Show Legend. The legend will disappear and the data area of the graph will expand accordingly. To eliminate the legend on an existing graph, just click on the legend and press the DELETE key on your keyboard.

233

FROM GOALS TO DATA AND BACK AGAIN

234

6.

Removing the x-axis labels. Sometimes we want all the information about the x-axis to be given in the title for that axis (the comparison of means bar graphs are examples of when we’ve used this approach). To eliminate the labels while you are creating a graph, click on the Axes tab then click on the check mark next to Category (X) axis. To eliminate the labels on an existing graph, click on the x-axis and press the DELETE key on your keyboard.

7.

Other changes. Before you continue with the remainder of this exercise we suggest you click on some of the other tabs and “toggle” different options—that is, click the check marks on and off—to see what happens. Although the figures in this book use only the modifications to the basic graph that we’ve already discussed, you might find some of the other options useful in the future.

8.

Finish the graph by clicking on the Titles tab and titling the x-axis with “January 3 to March 10” and the y-axis with “4-point scale (1,2,3,4).” Place the finished graph in the Exercise 6.2 worksheet.

9.

Changing a title. There are two ways to change a title on an existing graph. The first is to select the graph by clicking once on the white area. This will add a pull-down menu for Chart to the other pull-down menus for the window (see Figure C.3). If you pull-down the Chart menu you will see Chart Options... Click on it and you will find yourself in front of the same set of tabs we experimented with previously in this exercise. If you click on the Titles tab, you can then change the text as if you were back at this step in the Chart Wizard (click ok when you’re done). Try changing the title for the x-axis to “January 3, 2001 to March 10, 2001” using this method. The second, and simpler, way to change a title, however, is to click once on the title itself in the graph. A gray box will appear around the text. Place the cursor where you want to type and click again, then make your changes. To indicate you are done, click somewhere outside the graph. Change the title for the y-axis to “5-point scale (1,2,3,4,5)” using this method. Compare your graph to the first graph on the Answers 6.2 spreadsheet.

APPENDIX C EXERCISES

Figure C.3 When you select the whole chart the data is highlighted on the spreadsheet and a pull-down menu for the chart appears

10. Multiple goals for one child on a single graph. There may be times when you want to put more than one data set on a single graph to make data presentation more efficient. Remember that it is legitimate to combine goals in a single graph only if the data was taken during the same time interval and the goals you want to combine have the same size scale (all 4-point, for example). To create a graph with multiple goals: (a)

Click on the graphing icon, choose the line graph, and continue to the Data Range dialog box.

(b)

In general, you will select the columns of data for all the goals you want to graph. Here, select columns A and B to graph Anton’s goals together.

(c)

Finish the graph with the appropriate titles, etc. In this case it is crucial that you leave the legend on the graph so that you know which data is for which goal. Compare your finished graph to the second graph on the Answers 6.2 worksheet.

11. Multiple children for the same goal on a single graph. There may also be times when you want to directly compare performance on the same goal by more than one child. Remember that it is legitimate to compare data across children only if they were exposed to the same group instruction on the same dates. To create a graph for one goal, multiple children: 235

FROM GOALS TO DATA AND BACK AGAIN

236

(a)

Choose the line graph and continue to the Data Range dialog box.

(b)

In general, you will select the columns of data for all the children you want to graph. Here, select columns A, C, and D to graph the data for Anton, Becca, and Celeste for goal X. To select non-adjacent cells, select the first cell or range of cells and then, holding the CTRL key down, select each additional cell one at a time.

(c)

Finish the graph with the appropriate titles, etc. Again, it is crucial that you leave the legend on the graph so that you know whose data is whose. Compare your finished graph to the third graph on the Answers 6.2 worksheet.

APPENDIX C EXERCISES

Exercise 7.1 Computing the mean 1.

Open Exercise 7.1 in the Analysis Workbook.

2.

Scroll to the bottom of the data, click on cell A32 and type in “Mean”. You may also want to select all of row 32 and make it boldface; this will help you find the mean more easily when you come back to this sheet in the future.

3.

Click on cell B32. This is where the value computed by the average function will be placed.

4.

In the tool bar, click on the symbol fx, as shown in Figure C.4. The Paste Function dialog box will appear.

Figure C.4 The locations of the function symbol (white circle), the rounding up symbol (black circle), and the formula bar (black rectangle)

5.

Choose Statistical from the Function Category menu then choose Average from the Function Name menu. Click OK to move to the next dialog box.

6.

Select the data for Anton’s goal X (cells B2 through B30). If you can’t get at column B, move the dialog box out of the way by clicking on it and dragging it across the screen first.

7.

Click OK. The dialog box will disappear and a value will appear in cell B32. The value will depend on how many decimal places are showing. We want to round to 2 decimal places. To do this click on the rounding 237

FROM GOALS TO DATA AND BACK AGAIN

up symbol (.00 - >.0) in the tool bar (see Figure C.4) until only 2 numbers follow the decimal point. The number 2.55 should now be in cell B32. 8.

Right below the second row of the tool bar, to the right of an =, is an area called the formula bar. It is outlined with a black rectangle in Figure C.4. Typing in this area is one way to change the contents of a cell. It is also the place to look if you want to understand what kind of information Excel thinks is in a cell. For example, click on cell B30 and notice that the data value, 4, appears in the formula bar. Excel defines the formula for a number to be the number itself. Now click B32—you’ll see that the function AVERAGE(B2:B30) appears. In other words, when the contents of a cell has been computed by a formula, the value of the computation appears in the cell but the formula that computed that value appears in the formula bar.

9.

To change the value in B30 you just type a new value into B30 (or into the formula bar when B30 is selected). Go ahead—click B30, type 1 but don’t press the ENTER key on your keyboard. Now, as you press ENTER, watch cell B32 and note that the value changes. Click on B32 and look in the value box. The formula for computing the value hasn’t changed even though the value has. When you change the data that a function has used to compute a value, Excel automatically recomputes that value with the new data.

10. We suggest you repeat the mean calculation for goal X for Becca and for Celeste. When you’ve finished, compare your sheet to Answers 7.1 in the Analysis Workbook.

238

APPENDIX C EXERCISES

Exercise 7.2 Computing multiple means on a single sheet 1.

Open Exercise 7.2 in the Analysis Workbook.

2.

Click on B32 (the cell containing the mean for Anton’s data). Note that the formula for the average of the data in column B appears in the formula bar.

3.

Once you have computed one average on a data sheet there is a shortcut for computing additional averages. Copy B32; this copies the formula not the value.

4.

Paste the formula into C32, thereby computing the average for Becca’s data. Look at the value of C32 in the formula bar—it contains a formula that computes the average over column C. In addition to automatically adjusting the data range for you, Excel also automatically displays the mean with just 2 digits following the decimal point. In other words, Excel copies the format of the data as well as the formula for the data.

5.

Paste the formula again, this time into D32. (As long as there is still a striped box around B32 it can still be pasted. If the box has disappeared recopy either B32 or C32.) Once again, Excel automatically changes the data range to take the average of Celeste’s data.

6.

Delete the contents of C32 and D32. Now select both C32 and D32 at the same time. Paste the formula you copied—both means are computed automatically, each for the correct column of data. This is an extremely useful feature of Excel—a single copy and paste is much more efficient than stepping through a function’s dialog boxes over and over again.

7.

Compare your sheet to Answers 7.2 in the Analysis Workbook. Note that although all the means have the same value (2.55), each mean has been computed over a different data set.

239

FROM GOALS TO DATA AND BACK AGAIN

Exercise 7.3 Computing “before” and “after” means

240

1.

Open Exercise 7.1 in the Analysis Workbook.

2.

Copy the data for goal X for Anton, Becca, and Celeste (columns A–D and rows 1–30, not the means you calculated). Paste the copy into the Exercise 7.3 worksheet (which is blank).

3.

To lose the minimum amount of information we will divide the data for each child into (approximate) thirds of ten values, nine values, and ten values. Highlight the middle nine rows (rows 12 to 20) then choose Delete from the Edit menu. Now highlight rows 12 and 13 and choose Insert from the Edit menu to add two blank lines between the “before” and “after” data sets. Label the row at the bottom of each data set (cell A12 and cell A24) to indicate that the row contains means over the appropriate dates. You may want to check your progress against the Answers 7.3 worksheet at this point.

4.

Using the average function as in Exercise 7.1, above, compute the “before” mean for Anton.

5.

Using the same trick you practiced in Exercise 7.2, copy and paste the formula that computes Anton’s mean in order to compute the “before” means for Becca and Celeste.

6.

Now select the three cells that correspond to the children’s “after” means and paste the same formula. Click on each mean and check the data range in the formula bar. Even with the other mean and the blank line in the way Excel got the data ranges right. Pretty neat, huh? (Unfortunately Excel doesn’t always do what you intend it to, so it is a very good idea to check the data ranges in at least one of the cells whenever you cut and paste a formula.)

7.

Compare your work to Answers 7.3.

APPENDIX C EXERCISES

Exercise 7.4 Producing a bar graph from data for a single goal 1.

Open Exercise 7.4 in the Analysis Workbook.

2.

Select the graphing icon from the toolbar. Choose Column for the Chart type (not Bar) and the simplest of the graphs (upper left corner) from the Chart sub-type options. Click Next.

3.

Now you must specify the data range. Unlike the previous exercises, this time the data is in rows rather than columns. In the Data Range view, click Rows for Series In. Click in the Data Range box. Select the cell containing the label for the “before” mean and the cell with Anton’s mean in it. Excel will put a striped box around your selection and draw the first bar whose height equals the mean of the “before” data. Now select the cell containing the label of the “after” mean and the cell with the value of Anton’s “after” mean in it. A total of four cells should be selected. If this is not true, delete the text in the Data Range box, check “Excel Basics” for how to select non-adjacent cells and try again. When you have only those four cells selected, click Next.

4.

Label the graph “Comparison of Means, Anton, Goal X.” Label the x-axis “Before After” and the y-axis with our usual reminder of the scale for this goal. Click Next. Place the graph in the current worksheet.

5.

Use the shortcuts you practiced in Exercise 6.2 to eliminate the labels on the x-axis and clean up the titles. Compare your graph to the top graph in Answers 7.4.

6.

For additional practice, create the Comparison of Means graphs for Becca and Celeste. Don’t worry if Excel doesn’t show you an initial graph in the Data Range dialog box—just click Next and make sure you got the right 4 cells by looking at the graph in the Chart Options dialog box (go Back if you didn’t). Check your graphs against our versions in Answers 7.4.

241

FROM GOALS TO DATA AND BACK AGAIN

Exercise 7.5 Producing a bar graph with multiple goals for one child

242

1.

Open Exercise 7.5 in the Analysis Workbook.

2.

Remember that it is legitimate to combine goals in a single graph only if the data was taken during the same time interval and the goals used the same type of scale (all 4-point, for example).

3.

Choose the simple form of the Column bar graph.

4.

Specify the Data Range as you did in Exercise 7.4, this time using Anton’s data for goals X and Y.

5.

Label the data and click Finish, skipping the last dialog box. Excel defaults to putting the graph in the current worksheet.

6.

Compare your graph to the one on Answers 7.5.

APPENDIX C EXERCISES

Exercise 8.1 Computing the standard deviation 1.

Open Exercise 8.1 in the Analysis Workbook.

2.

We’ll add the standard deviation for each data set right below the mean. Label each row appropriately. You may want to make this row boldface as you did for the mean. You may also want to insert a blank row between the “before” and “after” sets to keep things easy to read.

3.

Click on the cell for the standard deviation for Anton’s data for goal X. In the tool bar, click on the symbol fx. Choose Statistical from the Function Category menu then scroll down the Function Name menu until you can select STDEV.

4.

In the Data Range dialog, select Anton’s “before” data for goal X. Or, if you prefer, you may simply type the range—B2:B11—into the box labeled Number 1. Whatever technique you use to specify the range be careful not to include the mean value as part of the data. Click OK and then round the value to 2 decimal places.

5.

Compute the standard deviation for the other data sets on this sheet using the same copy-and-paste technique you learned in Exercise 7.2.

6.

Compare your sheet to Answers 8.1.

243

FROM GOALS TO DATA AND BACK AGAIN

Exercise 8.2 Adding standard deviation information to a bar graph

244

1.

Open Exercise 8.2 in the Analysis Workbook where you will see data and bar graphs for Anton, Becca, and Celeste. If you are not sure how to produce the bar graphs, review Exercise 7.4.

2.

We add standard deviation information to each graph by putting error bars around each mean. In Anton’s graph, click on the bar representing the “before” mean. A black square will appear on the bar to show it has been selected (a box will also appear around the mean in the data).

3.

Click Selected Data Series in the pull-down menu for Format. A window called Format Data Series will pop up.

4.

Click on the tab labeled Y Error Bars then select Both from the Display.

5.

Click the circle next to Custom at the bottom of the list of Error Amounts.

6.

Now you have a choice of how to fill in the + and - values for the error. You can click on each data box in turn and type in the standard deviation for Anton’s “before” data (0.52) or you can click on each data box in turn and then click on the cell containing the standard deviation in the spreadsheet. Either works for our purposes, but the latter method causes the graph to be updated automatically if Anton’s data changes.

7.

When you click OK the Format Data Series will disappear and the error bar will appear on the graph. Now click on the error bar in the graph then pull down the Format menu again. Note that Excel now offers the option of reformatting the Selected Error Bars. Another way of saying this is that Excel keeps track of the context created by your selections and offers you options that depend on that context.

8.

Repeat steps 2 through 7 for the bar representing the “after” mean for Anton’s data. Be sure to use the “after” value of the standard deviation in step 6.

9.

If you want more practice repeat this exercise for Becca and Celeste’s graphs. When you are finished compare your work to Answers 8.2.

APPENDIX C EXERCISES

Exercise 8.3 Experimenting with raw data, mean, and standard deviation in the Statistics Lab 1.

Open Exercise 8.3 in the Analysis workbook to enter the Statistics Lab. In it you’ll see data for a single child, separated into “before” and “after” subsets and graphed in ways that should be familiar to you. Since you will be changing the data, you may want to make a copy of this sheet to work with (that way you won’t have to go to the CD-ROM if you want to get back to this state). Use Move or Copy Sheet from the Edit menu, making sure to click Create a copy in the dialog box.

2.

Notice the values in cells B2 through B11—the “before” data is perfectly regular. Its mean is a perfect representation of the observations so the standard deviation is 0. Because of these values, you cannot see an error bar around the “before” mean on the graph showing the comparison of means. The values in column C are the repetition of the mean value required to draw the mean line in the “before” and “after” line graphs.

3.

Change the value in cell B2 from 2 to 1. Be sure to press the ENTER key on your keyboard (or click another cell) so that the change takes effect and all the values that depend on B2 are updated. Notice that the mean and standard deviation for the “before” data set changes and that those changes are reflected automatically in the graphs.

4.

Continue to replace 2s in the “before” data set with 1s and watch what happens to the mean and the standard deviation for the “before” data as well as what happens to the error bars on the comparison of means graph. Notice that even though the standard deviation has gotten bigger, there is no overlap in the error bars because the data sets remain distinct.

5.

Change the “before” observations back to all 2s. Now replace the 2s with 3s one at a time. Again, watch how the mean and standard deviation change for the “before” data set as well as how the error bars overlap. Notice that this time, as the “before” and “after” sets have a larger percentage of their values in common, the means grow together until the difference in the means is first overshadowed by the overlap in the error bars and then falls below the category width of 1.00.

245

FROM GOALS TO DATA AND BACK AGAIN

246

6.

Start again with all 2s in B2 to B11. Now replace the first five values one at a time with 1s and then the last five values with 3s. As always, watch the clustering around the mean, the standard deviation, the error bars, and the overlap in the “before” and “after” data sets.

7.

Try other changes in the “before” and “after” data sets to see what sorts of patterns result. Change the values one at a time and see if you can predict what the graphs will look like before you press the ENTER key or click another cell.

APPENDIX C EXERCISES

Exercise 9.1 Experimenting with outliers in the Statistics Lab 1.

Open Exercise 9.1 in the Analysis workbook to re-enter the Statistics Lab. This sheet works just like the one you explored in Exercise 8.3 but begins in a different configuration and contains two data sets: one for a qualitative scale and one for percent data. There is one value missing in each of the “before” and “after” sets of observations. These are the values we’re going to explore. Notice that without the 4 missing values the initial configuration of both sets shows a pattern of change: means are more than a category width apart and there is no overlap in the error bars.

2.

We’ll work with the qualitative scale first. Put the number 1 in cell B21, introducing a low outlier value in the “after” data (don’t forget to press ENTER). Notice that the mean changes very little but the standard deviation increases. Keeping your eye on the Comparison of Means graph, delete the outlier (B21) and watch what happens, then add it back in. Do this a few times until you see the pattern clearly.

3.

Change B21 back to an empty cell. Now we’re going to look at what happens when the outlier is a high value in the “before” data. Put the number 4 in cell B8 (and press ENTER). Notice the way the “before” standard deviation increases and the error bars overlap. Keeping your eye on the Comparison of Means graph, delete the outlier (B8) and watch what happens. Do this a few times until you see the pattern clearly.

4.

Scroll down the sheet to work with the percent data that starts in row 35. The wider range of possible values on this scale (0 to 100%) makes it easier to explore the remaining two cases.

5.

Place a 5 in cell B41, introducing a low-valued outlier in the “before” data. This case sends a mixed message. Although the outlier pulls the “before” mean down, increasing the difference between the means, it also increases the “before” standard deviation. Delete the value in B41.

6.

Place a 55 in cell B56, introducing a high-valued outlier in the “after” data. Again, the difference in the means grew, exaggerating progress. Even though the “after” standard deviation also grows, there is still no overlap in the error bars. Now try the values 60, 65, and 75 in B56. As the outlier’s value increases, the difference in the means increases as well but eventually the size of the “after” standard deviation grows too large to ignore. 247

FROM GOALS TO DATA AND BACK AGAIN

Exercise 9.2 Computing correlations with the Pearson r In this exercise we want to know if the difference in teacher is correlated with the performance of any of the children in a group instruction environment.

248

1.

Open Exercise 9.2 in the Analysis Workbook.

2.

Insert a new column between columns B and C and translate the teacher factor into numeric form. If your numbers look odd, select the cells in the new column then use the Format pull-down menu for cells to specify that the data in the cells will be Numbers with 0 decimal places. Check your answer on the next sheet before continuing.

3.

We will compute the correlation between teacher and each child in the “before” and “after” data separately. Insert rows into your spreadsheet to make room for the correlation values and label the rows appropriately.

4.

Click on the cell where you want to place the value for the correlation between teacher and Anton’s performance for this goal. Now select the PEARSON function just as you have selected AVERAGE and STDEV in previous exercises.

5.

In the Data Range dialog box, specify the teacher’s numeric “before” data as Array1 and Anton’s “before” data as Array2. Click OK and then round the value to 2 decimal places if necessary.

6.

Copy the formula for Teacher/Anton and paste it into the cell for Teacher/Becca. Look up at the formula bar: this is one of those times we warned you about in Exercise 7.3: Excel did not do what you wanted it to (it increased the index of both ranges not just one). There are many ways to correct the Teacher/Becca correlation value. The simplest one is to just edit the first data range in the formula bar to compute over the values in column C.

7.

Compute the remaining “before” correlations. Now select the three cells that contain the “before” correlations and paste them into the three cells that are meant to contain the “after” correlations. Note that, this time, Excel does exactly what we want.

APPENDIX C EXERCISES

8.

Compare your sheet to Answers 9.2. Notice that we put only the values that were close to or greater than the boundary value of 0.65 into boldface. This makes it easy to ignore the meaningless values and concentrate our attention on the ones of interest. The only value of interest here is Becca’s in the “after” phase; her performance on this goal is better when Ms B is teaching. Does this mean Ms B is a better teacher? No, it just means that we can predict a difference in Becca’s performance depending upon who is the instructor.

249

FROM GOALS TO DATA AND BACK AGAIN

Exercise 9.3 Understanding negative correlation

250

1.

Open Exercise 9.3 in the Analysis Workbook.

2.

Insert a column between the numeric values for the teacher factor and Anton’s data. In the new column reverse the assignment of numbers to teacher. In other words, assign the value of 1 to Mr L and 2 to Ms B.

3.

Insert an extra row below the correlations for the “before” data.

4.

Now calculate the correlation between the new values of the teacher factor and each child, “before” and “after.”

5.

If you reversed the assignment of numbers without making any mistakes, the new Pearson r values will be the same as the Pearson r values you calculated in Exercise 9.2 but with the sign reversed. The value is the same because the degree of correlation hasn’t changed; the sign is reversed because the lower/higher value of the teacher factor now predicts higher/lower values for performance. Note that in the case of the “after” data for Becca—the only instance that reaches criterion—we recover the same conclusion we came to in Exercise 9.2: in the “after” phase of intervention, Becca performs better when Ms B is teaching.

6.

Compare your sheet to Answers 9.3.

X (4-point scale) Date 3-Jan 1 5-Jan 1 7-Jan 2 10-Jan 1 12-Jan 1 14-Jan 2 19-Jan 1 21-Jan 2 24-Jan 1 26-Jan 2 28-Jan 3 31-Jan 3 2-Feb 2 4-Feb 3 7-Feb 2 9-Feb 3 11-Feb 3 14-Feb 2 16-Feb 3 18-Feb 4 21-Feb 3 23-Feb 4 25-Feb 3 28-Feb 4 1-Mar 4 3-Mar 3 6-Mar 4 8-Mar 3 10-Mar 4

X (4-point scale) Date 3-Jan 2 5-Jan 1 7-Jan 2 10-Jan 1 12-Jan 3 14-Jan 2 19-Jan 2 21-Jan 3 24-Jan 3 26-Jan 2 28-Jan 3 31-Jan 1 2-Feb 2 4-Feb 2 7-Feb 3 9-Feb 3 11-Feb 3 14-Feb 2 16-Feb 3 18-Feb 3 21-Feb 4 23-Feb 1 25-Feb 3 28-Feb 4 1-Mar 3 3-Mar 2 6-Mar 3 8-Mar 4 10-Mar 4

X Date 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan 28-Jan 31-Jan 2-Feb 4-Feb 7-Feb 9-Feb 11-Feb 14-Feb 16-Feb 18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar

(4point scale) 1 2 3 2 2 2 3 2 3 4 4 2 1 1 2 2 3 3 2 4 4 2 1 3 4 2 3 3 4

1/ 3/ 20 00 1/ 10 /2 00 0 1/ 17 /2 00 0 1/ 24 /2 00 0 1/ 31 /2 00 0 2/ 7/ 20 00 2/ 14 /2 00 0 2/ 21 /2 00 0 2/ 28 /2 00 0 3/ 6/ 20 00

4-point scale (1,2,3,4) 1/ 3/ 20 00 1/ 10 /2 00 0 1/ 17 /2 00 0 1/ 24 /2 00 0 1/ 31 /2 00 0 2/ 7/ 20 00 2/ 14 /2 00 0 2/ 21 /2 00 0 2/ 28 /2 00 0 3/ 6/ 20 00

4-point scale (1,2,3,4)

Anton, Goal X

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0

Becca, Goal X

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0

1/ 3/ 20 00 1/ 10 /2 00 0 1/ 17 /2 00 0 1/ 24 /2 00 0 1/ 31 /2 00 0 2/ 7/ 20 00 2/ 14 /2 00 0 2/ 21 /2 00 0 2/ 28 /2 00 0 3/ 6/ 20 00

4-point scale (1,2,3,4)

Celeste, Goal X

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0

Becca 1 1 2 1 1 2 1 2 1 2 3 3 2 3 2 3 3 2 3 4 3 4 3 4 4 3 4 3 4

2 1 2 1 3 2 2 3 3 2 3 1 2 2 3 3 3 2 3 3 4 1 3 4 3 2 3 4 4

Celeste 1 2 3 2 2 2 3 2 3 4 4 2 1 1 2 2 3 3 2 4 4 2 1 3 4 2 3 3 4

Comparison of Progress on Goal X 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 1/ 3/ 2 1/ 000 10 /2 1/ 000 17 /2 00 1/ 24 0 /2 1/ 000 31 /2 0 2/ 00 7/ 20 0 2/ 14 0 /2 2/ 000 21 /2 2/ 000 28 /2 0 3/ 00 6/ 20 00

Anton

4-point scale (1,2,3,4)

Date 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan 28-Jan 31-Jan 2-Feb 4-Feb 7-Feb 9-Feb 11-Feb 14-Feb 16-Feb 18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar

Anton Becca Celeste

Date 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar

X (4-point scale) 1 1 2 1 1 2 1 2 1 2

4 3 4 3 4 4 3 4 3 4

Date 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar

X (4-point scale) 2 1 2 1 3 2 2 3 3 2

3 4 1 3 4 3 2 3 4 4

Date 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan

X (4-point scale) 1 2 3 2 2 2 3 2 3 4

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar

4 4 2 1 3 4 2 3 3 4

3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean, 1/3-1/26

Anton 1 1 2 1 1 2 1 2 1 2 1.4

Becca 2 1 2 1 3 2 2 3 3 2 2.1

Celeste 1 2 3 2 2 2 3 2 3 4 2.4

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean, 2/18-3/10

4 3 4 3 4 4 3 4 3 4 3.6

3 4 1 3 4 3 2 3 4 4 3.1

4 4 2 1 3 4 2 3 3 4 3.0

4-point scale (1,2,3,4)

Comparison of Means, Anton, Goal X 4 3 Mean, 1/3-1/26

2

Mean, 2/18-3/10

1 0 Before versus After

4-point scale (1,2,3,4)

Comparison of Means, Becca, Goal X 3.5 3 2.5 2

Mean, 1/3-1/26

1.5

Mean, 2/18-3/10

1 0.5 0 Before versus After

4-point scale (1,2,3,4)

Comparison of Means, Celeste, Goal X 3.5 3 2.5 2

Mean, 1/3-1/26

1.5

Mean, 2/18-3/10

1 0.5 0 Before versus After

3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean, before Std. Dev., before

Anton 1 1 2 1 1 2 1 2 1 2 1.4 0.52

2 1 2 1 3 2 2 3 3 2 2.1 0.74

Celeste 1 2 3 2 2 2 3 2 3 4 2.4 0.84

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar

4 3 4 3 4 4 3 4 3 4 3.6 0.52

3 4 1 3 4 3 2 3 4 4 3.1 0.99

4 4 2 1 3 4 2 3 3 4 3.0 1.05

Mean, after Std. Dev., after

Becca

Comparison of Means, Anton, Goal X 4.5 4-point scale (1,2,3,4)

4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 1/3-1/26 versus 2/18-3/10

Comparison of Means, Becca, Goal X 4.5 4-point scale (1,2,3,4)

4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 1/3-1/26 versus 2/18-3/10

Comparison of Means, Celeste, Goal X

4-point scale (1,2,3,4)

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 1/3-1/26 versus 2/18-3/10

Y (4-point scale) Original Revised 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 1 1 2 2 2 2 2 2 3 3 3 1 1.9 1.9 3 0.57 0.57 2 3 2 4 3 2 3 3 3 3 4 4 4 4 4 4 3 3 3 4 4 4 4 4 4 1 1 4 4 4 3 3 3 4 4 4 3.4 3.67 0.97 0.50

Sessions

29

27

25

23

21

19

17

15

13

11

9

7

5

3

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 1

4-point scale (1,2,3,4)

Anton, Goal Y

4-point scale (1,2,3,4)

Comparison of Means, Anton, Goal Y 5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 Before

After

Comparison of Means, Anton, Goal Y, Revised

4-point scale (1,2,3,4)

4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 Before

After

Location home school school home school playgroup home home school school school school playgroup school playgroup Before Mean Before Std. Dev. Before Pearson playgroup playgroup playgroup playgroup school school playgroup playgroup school playgroup playgroup playgroup home school playgroup After Mean After Std. Dev. After Pearson

Z 1=school 1=school 1=home (5-point 2=home 2=playgroup 2=school scale) 3=playgroup 3=home 3=playgroup 1 2 3 1 2 1 1 2 2 3 1 2 1 2 3 1 2 1 1 2 3 3 2 3 1 2 3 1 1 1 3 1 2 3 1 2 2 2 1 2 2 1 1 2 2 3 1 2 3 2 2 3 2 1 1 2 3 3 2 3 1.93 0.70 0.36 -0.49 1.00 4 4 3 4 2 2 3 3 2 3 3 4 1 2 3 2.60 0.63

1 1 3 1 2 2 3 3 2 3 3 2 2 3 1

2 2 2 2 1 1 2 2 1 2 2 2 3 1 2

3 3 3 3 2 2 3 3 2 3 3 3 1 2 3

-0.35

0.22

0.89

5-point scale (1,2,3,4,5)

Comparison of Means, Anton, Goal Z 3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 Before

After

5-point scale (1,2,3,4,5)

Anton, Goal Z 5 4 3 2 1 0 Before

After

Location home home home home playgroup playgroup home playgroup home playgroup home playgroup home home home Before Mean Before Std. Dev. Before Pearson

1 2 2 2 2 2 1 3 2 2 2 3 2 2 2 2.00 0.55

Location 1 1 1 1 2 2 1 2 1 2 1 2 1 1 1

0.56 playgroup home home playgroup home playgroup playgroup playgroup home playgroup home playgroup home playgroup home

After Mean After Std. Dev. After Pearson

Y (4-point scale)

4 3 2 5 3 4 5 3 3 4 2 4 2 5 3 3.47 1.06

2 1 1 2 1 2 2 2 1 2 1 2 1 2 1

0.82

Becca, Goal Y

5-point scale (1,2,3,4,5)

6 4 2 0 Before

After

5-point scale (1,2,3,4,5)

Comparison of Means, Becca Goal Y 5.00 4.00 3.00 2.00 1.00 0.00 Before

After

Before Mean Before Std. Dev. Before Pearson

After Mean After Std. Dev. After Pearson

Z % success (~10 opps) 20 25 25 30 30 15 15 20 25 25 20 25 30 30 35 24.67 5.81

25 35 35 35 40 30 30 35 40 45 30 40 45 45 50 37.33 7.04

Position in the 5-day sequence 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

Early Tuesday Week vs vs other Late Week days 1 1 1 2 2 2 2 2 2 2 1 1 1 2 2 2 2 2 2 2 1 1 1 2 2 2 2 2 2 2

0.76

0.68

0.56

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

1 1 2 2 2 1 1 2 2 2 1 1 2 2 2

1 2 2 2 2 1 2 2 2 2 1 2 2 2 2

0.80

0.68

0.66

Celeste, Goal Z

% successful

60 50 40 30 20 10 0 Before

After

Comparison of Means, Celeste, Goal Z

% successful

50.00 40.00 30.00 20.00 10.00 0.00 Before

After

Anton, X (1-4) 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan 28-Jan 31-Jan 2-Feb 4-Feb 7-Feb 9-Feb 11-Feb 14-Feb 16-Feb 18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar

1 1 2 1 1 2 1 2 1 2 3 3 2 3 2 3 3 2 3 4 3 4 3 4 4 3 4 3 4

Anton, Y (1- Becca, X 5) 4) 4 2 2 2 2 2 1 2 1 2 1 3 2 3 2 4 3 1 3 4 4 4 3 4 4 3 4 3 4

(1- Celeste, X 4) 2 1 2 1 3 2 2 3 3 2 3 1 2 2 3 3 3 2 3 3 4 1 3 4 3 2 3 4 4

(11 2 3 2 2 2 3 2 3 4 4 2 1 1 2 2 3 3 2 4 4 2 1 3 4 2 3 3 4

1/ 10 /0 0 1/ 17 /0 0 1/ 24 /0 0 1/ 31 /0 0 2/ 7/ 00 2/ 14 /0 0 2/ 21 /0 0 2/ 28 /0 0 3/ 6/ 00

1/ 3/ 00

4-point scale (1,2,3,4)

1/ 10 /0 0 1/ 17 /0 0 1/ 24 /0 0 1/ 31 /0 0 2/ 7/ 00 2/ 14 /0 0 2/ 21 /0 0 2/ 28 /0 0 3/ 6/ 00

1/ 3/ 00

4-point scale (1,2,3,4)

1/ 10 /0 0 1/ 17 /0 0 1/ 24 /0 0 1/ 31 /0 0 2/ 7/ 00 2/ 14 /0 0 2/ 21 /0 0 2/ 28 /0 0 3/ 6/ 00

1/ 3/ 00

4-point scale (1,2,3,4)

Anton, Goal X

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 goal x

Becca, Goal X

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 goal X

Celeste, Goal X

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 goal X

Anton, X 4)

(11 1 2 1 1 2 1 2 1 2 3 3 2 3 2 3 3 2 3 4 3 4 3 4 4 3 4 3 4

Anton, Y (1-5) 4 2 2 2 2 2 1 2 1 2 1 3 2 3 2 4 3 1 3 4 4 4 3 4 4 3 4 3 4

Becca, X (1-4)

Celeste, X (1-4) 2 1 2 1 3 2 2 3 3 2 3 1 2 2 3 3 3 2 3 3 4 1 3 4 3 2 3 4 4

1 2 3 2 2 2 3 2 3 4 4 2 1 1 2 2 3 3 2 4 4 2 1 3 4 2 3 3 4

5-point scale (1,2,3,4,5)

Anton, X

(1-4)

5 4 3 2 1 0

January 3, 2001 to March 10, 2001

4-point scale (1,2,3,4)

Anton, Goals X and Y 5 4 3

Anton, X (1-4) Anton, Y (1-5)

2 1 0

January 3 to March 10

Performance on Goal X for Anton, Becca & Celeste

4-point scale (1,2,3,4)

5 4

Anton, X (1-4) Becca, X (1-4) Celeste, X (1-4)

3 2 1 0

January 3 through March 10

Anton, X (1-4) 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan 28-Jan 31-Jan 2-Feb 4-Feb 7-Feb 9-Feb 11-Feb 14-Feb 16-Feb 18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar

Becca, X (1-4) 1 1 2 1 1 2 1 2 1 2 3 3 2 3 2 3 3 2 3 4 3 4 3 4 4 3 4 3 4

Celeste, X (1-4) 2 1 2 1 3 2 2 3 3 2 3 1 2 2 3 3 3 2 3 3 4 1 3 4 3 2 3 4 4

1 2 3 2 2 2 3 2 3 4 4 2 1 1 2 2 3 3 2 4 4 2 1 3 4 2 3 3 4

Anton, X (1-4) 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan 28-Jan 31-Jan 2-Feb 4-Feb 7-Feb 9-Feb 11-Feb 14-Feb 16-Feb 18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean

Becca, X (1-4)

Celeste, X (1-4)

1 1 2 1 1 2 1 2 1 2 3 3 2 3 2 3 3 2 3 4 3 4 3 4 4 3 4 3 4

2 1 2 1 3 2 2 3 3 2 3 1 2 2 3 3 3 2 3 3 4 1 3 4 3 2 3 4 4

1 2 3 2 2 2 3 2 3 4 4 2 1 1 2 2 3 3 2 4 4 2 1 3 4 2 3 3 4

2.55

2.55

2.55

Anton, X (1-4) 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan 28-Jan 31-Jan 2-Feb 4-Feb 7-Feb 9-Feb 11-Feb 14-Feb 16-Feb 18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean

Becca, X 4) 1 1 2 1 1 2 1 2 1 2 3 3 2 3 2 3 3 2 3 4 3 4 3 4 4 3 4 3 4

2.55

(1- Celeste, X (1-4) 2 1 1 2 2 3 1 2 3 2 2 2 2 3 3 2 3 3 2 4 3 4 1 2 2 1 2 1 3 2 3 2 3 3 2 3 3 2 3 4 4 4 1 2 3 1 4 3 3 4 2 2 3 3 4 3 4 4

Anton, X (1-4) 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan 28-Jan 31-Jan 2-Feb 4-Feb 7-Feb 9-Feb 11-Feb 14-Feb 16-Feb 18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean

Becca, X (1-4)

Celeste, X (1-4)

1 1 2 1 1 2 1 2 1 2 3 3 2 3 2 3 3 2 3 4 3 4 3 4 4 3 4 3 4

2 1 2 1 3 2 2 3 3 2 3 1 2 2 3 3 3 2 3 3 4 1 3 4 3 2 3 4 4

1 2 3 2 2 2 3 2 3 4 4 2 1 1 2 2 3 3 2 4 4 2 1 3 4 2 3 3 4

2.55

2.55

2.55

3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean 1/3-1

(1- Becca, X (1-4) 1 2 1 1 2 2 1 1 1 3 2 2 1 2 2 3 1 3 2 2 1.4

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean 2/18-

4 3 4 3 4 4 3 4 3 4 3.6

Anton, X 4)

3 4 1 3 4 3 2 3 4 4

Celeste, X (1-4) 1 2 3 2 2 2 3 2 3 4

4 4 2 1 3 4 2 3 3 4

Anton, X (1-4)

Becca, X (1-4)

Celeste, X (1-4)

3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean 1/3-1/26

1 1 2 1 1 2 1 2 1 2 1.4

2 1 2 1 3 2 2 3 3 2 2.1

1 2 3 2 2 2 3 2 3 4 2.4

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean 2/18-3/10

4 3 4 3 4 4 3 4 3 4 3.6

3 4 1 3 4 3 2 3 4 4 3.1

4 4 2 1 3 4 2 3 3 4 3.0

Anton, X (1-4)

Becca, X (1-4)

Celeste, X (1-4)

3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean 1/3-1/26

1 1 2 1 1 2 1 2 1 2 1.4

2 1 2 1 3 2 2 3 3 2 2.1

1 2 3 2 2 2 3 2 3 4 2.4

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean 2/18-3/10

4 3 4 3 4 4 3 4 3 4 3.6

3 4 1 3 4 3 2 3 4 4 3.1

4 4 2 1 3 4 2 3 3 4 3.0

Comparison of Means, Anton, Goal X 4-point scale (1,2,3,4)

4 3 Mean 1/3-1/26 Mean 2/18-3/10

2 1 0 Before After

4-point scale (1,2,3,4)

Comparison of Means, Becca, Goal X 3.5 3 2.5 2 1.5 1 0.5 0

Mean 1/3-1/26 Mean 2/18-3/10

Before After

Comparison of Means, Celeste, Goal X

4-point scale (1,2,3,4)

3.5 3 2.5 2

Mean 1/3-1/26 Mean 2/18-3/10

1.5 1 0.5 0 Before After

Anton, X (1-4) 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean 1/3-1/26

1 1 2 1 1 2 1 2 1 2 1.4

Anton, Y (1-5) 4 2 2 2 2 2 1 2 1 2 2.0

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean 2/18-3/10

4 3 4 3 4 4 3 4 3 4 3.6

4 4 4 3 4 4 3 4 3 4 3.7

Becca, X (1-4)

Celeste, X 4)

(1-

2 1 2 1 3 2 2 3 3 2 2.1

1 2 3 2 2 2 3 2 3 4 2.4

3 4 1 3 4 3 2 3 4 4 3.1

4 4 2 1 3 4 2 3 3 4 3.0

Comparison of Means for Anton

4-point scale (1,2,3,4)

4 3.5 3 2.5

Mean 1/3-1/26

2

Mean 2/18-3/10

1.5 1 0.5 0 Goal X

Goal Y

Anton, X (1-4) 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean 1/3-1/26

1 1 2 1 1 2 1 2 1 2 1.4

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean 2/18-3/10

4 3 4 3 4 4 3 4 3 4 3.6

Anton, Y (14) 4 2 2 2 2 2 1 2 1 2 2.0 4 4 4 3 4 4 3 4 3 4 3.7

Becca, X (1-4)

Celeste, X (1-4)

2 1 2 1 3 2 2 3 3 2 2.1

1 2 3 2 2 2 3 2 3 4 2.4

3 4 1 3 4 3 2 3 4 4 3.1

4 4 2 1 3 4 2 3 3 4 3.0

Anton, X (1-4) 3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean 1/3-1/26 Std. Dev. 1/3-1/26

1 1 2 1 1 2 1 2 1 2 1.4 0.52

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean 2/18-3/10 Std. Dev. 2/18-3/10

4 3 4 3 4 4 3 4 3 4 3.6 0.52

Anton, Y (1- Becca, X 4) 4) 4 2 2 2 2 2 1 2 1 2 2.0 0.82 4 4 4 3 4 4 3 4 3 4 3.7 0.48

(1-

Celeste, X (1-4)

2 1 2 1 3 2 2 3 3 2 2.1 0.74

1 2 3 2 2 2 3 2 3 4 2.4 0.84

3 4 1 3 4 3 2 3 4 4 3.1 0.99

4 4 2 1 3 4 2 3 3 4 3.0 1.05

Anton, X 4)

(1-

Becca, X (1-4)

Celeste, X (1-4)

3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean 1/3-1/26 Std. Dev. 1/3-1/26

1 1 2 1 1 2 1 2 1 2 1.4 0.52

2 1 2 1 3 2 2 3 3 2 2.1 0.74

1 2 3 2 2 2 3 2 3 4 2.4 0.84

18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean 2/18-3/10 Std. Dev. 2/18-3/10

4 3 4 3 4 4 3 4 3 4 3.6 0.52

3 4 1 3 4 3 2 3 4 4 3.1 0.99

4 4 2 1 3 4 2 3 3 4 3.0 1.05

4-point scale (1,2,3,4)

Comparison of Means, Anton, Goal X 4 3 Mean 1/3-1/26 Mean 2/18-3/10

2 1 0 Before

After

Comparison of Means, Becca, Goal X

4-point scales (1,2,3,4)

4 3 Mean 1/3-1/26 Mean 2/18-3/10

2 1 0 Before

After

4-point scale (1,2,3,4)

Comparison of Means, Celeste, Goal X 4 3 Mean 1/3-1/26 Mean 2/18-3/10

2 1 0 Before

After

4-point scale (1,2,3,4)

Comparison of Means, Anton, Goal X 5 4 3

Mean 1/3-1/26 Mean 2/18-3/10

2 1 0 Before

After

4-point scales (1,2,3,4)

Comparison of Means, Becca, Goal X 5 4 3

Mean 1/3-1/26 Mean 2/18-3/10

2 1 0 Before

After

Comparison of Means, Celeste, Goal X

4-point scale (1,2,3,4)

5 4 3

Mean 1/3-1/26 Mean 2/18-3/10

2 1 0 Before

After

Observations BEFORE

Mean Value 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0 2.0

2 2 2 2 2 2 2 2 2 2 2.0 0.00

Before Mean Before Std. Dev.

3 3 3 3 3 4 4 4 4 4 3.5 0.53

AFTER

After Mean After Std. Dev.

3.5 3.5 3.5 3.5 3.5 3.5 3.5 3.5 3.5 3.5

Before Observations and Means 5 4 3

Data Mean Value

2 1 0 1

2

3

4

5

6

7

8

9

10

After Observations and Means 5 4 3

Data Mean Value

2 1 0 1

2

3

4

5

6

7

8

9

10

Before versus After Mean and Standard Deviation 5.0 4.0 3.0 2.0 1.0 0.0 Before versus After

BEFORE

Before Mean Before Std. Dev.

Observations (5-point scale) Mean Value 1 1.7 2 1.7 2 1.7 2 1.7 1 1.7 2 1.7 1.7 2 1.7 1 1.7 2 1.7 1.7 0.50 3 3 3 3 3 4

AFTER

3.3 3.3 3.3 3.3 3.3 3.3 3.3 3.3 3.3 3.3

3 4 4 3.3 0.50

After Mean After Std. Dev.

Before Observations and Means 5 4 3

Data Mean Value

2 1 0 1

2

3

4

5

6

7

8

9

10

After Observations and Means 5 4 3

Data Mean Value

2 1 0 1

2

3

4

5

6

7

8

9

10

Before versus After Mean and Standard Deviation 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 Before versus After

BEFORE

Before Mean Before Std. Dev.

Observations (%) Mean Value 30 26.1 25 26.1 25 26.1 30 26.1 25 26.1 26.1 20 26.1 30 26.1 25 26.1 25 26.1 26.1 3.33 35 30 35 40 35 40 45

AFTER

37.8 37.8 37.8 37.8 37.8 37.8 37.8 37.8 37.8 37.8

40 40 37.8 4.41

After Mean After Std. Dev.

Before Observations and Means 100 90 80 70 60 50 40 30 20 10 0

Data Mean Value

1

2

3

4

5

6

7

8

9

10

After Observations and Means 100 90 80 70 60 50 40 30 20 10 0

Data Mean Value

1

2

3

4

5

6

7

8

9

10

Before versus After Mean and Standard Deviation 45.0 40.0 35.0 30.0 25.0 20.0 15.0 10.0 5.0 0.0 Before versus After

3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean 1/3-1/26 Std. Dev. 1/3-1/26 18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean 2/18-3/10 Std. Dev. 2/18-3/10

Teacher Ms. B Mr. L Ms. B Mr. L Mr. L Mr. L Ms. B Ms. B Mr. L Ms. B

Ms. B Mr. L Ms. B Mr. L Mr. L Mr. L Ms. B Ms. B Mr. L Ms. B

Anton, X Becca, X Celeste, X (1-4) (1-4) (1-4) 1 2 1 1 1 2 2 2 3 1 1 2 1 3 2 2 2 2 1 2 3 2 3 2 1 3 3 2 2 4 1.4 2.1 2.4 0.52 0.74 0.84 4 3 4 3 4 4 3 4 3 4 3.6 0.52

3 4 1 3 4 3 2 3 4 4 3.1 0.99

4 4 2 1 3 4 2 3 3 4 3 1.05

3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean 1/3-1/26 Std. Dev. 1/3-1/26 Pearson(Teacher, Child) 18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean 2/18-3/10 Std. Dev. 2/18-3/10 Pearson(Teacher, Child)

Teacher Ms. B Mr. L Ms. B Mr. L Mr. L Mr. L Ms. B Ms. B Mr. L Ms. B

Teacher 1 2 1 2 2 2 1 1 2 1

Ms. B Mr. L Ms. B Mr. L Mr. L Mr. L Ms. B Ms. B Mr. L Ms. B

1 2 1 1 2 1 1 1 2 2

Anton, X Becca, X Celeste, X (1-4) (1-4) (1-4) 1 2 1 1 1 2 2 2 3 1 1 2 1 3 2 2 2 2 1 2 3 2 3 2 1 3 3 2 2 4 1.4 2.1 2.4 0.52 0.74 0.84 -0.41 -0.14 -0.25 4 3 4 3 4 4 3 4 3 4 3.6 0.52 -0.17

3 4 1 3 4 3 2 3 4 4 3.1 0.99 0.78

4 4 2 1 3 4 2 3 3 4 3 1.05 0.41

3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean 1/3-1/26 Std. Dev. 1/3-1/26 Pearson(Teacher, Child) 18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean 2/18-3/10 Std. Dev. 2/18-3/10 Pearson(Teacher, Child)

Teacher Ms. B Mr. L Ms. B Mr. L Mr. L Mr. L Ms. B Ms. B Mr. L Ms. B

Teacher 1 2 1 2 2 2 1 1 2 1

Ms. B Mr. L Ms. B Ms. B Mr. L Ms. B Ms. B Ms. B Mr. L Mr. L

1 2 1 1 2 1 1 1 2 2

Anton, X Becca, X Celeste, X (1-4) (1-4) (1-4) 1 2 1 1 1 2 2 2 3 1 1 2 1 3 2 2 2 2 1 2 3 2 3 2 1 3 3 2 2 4 1.4 2.1 2.4 0.52 0.74 0.84 -0.41 -0.14 -0.25 4 3 4 3 4 4 3 4 3 4 3.6 0.52 -0.17

3 4 1 3 4 3 2 3 4 4 3.1 0.99 0.78

4 4 2 1 3 4 2 3 3 4 3 1.05 0.41

3-Jan 5-Jan 7-Jan 10-Jan 12-Jan 14-Jan 19-Jan 21-Jan 24-Jan 26-Jan Mean 1/3-1/26 Std. Dev. 1/3-1/26 Pearson(Teacher, Child) Pearson(Teacher2, Child) 18-Feb 21-Feb 23-Feb 25-Feb 28-Feb 1-Mar 3-Mar 6-Mar 8-Mar 10-Mar Mean 2/18-3/10 Std. Dev. 2/18-3/10 Pearson(Teacher, Child) Pearson(Teacher2, Child)

Teacher Ms. B Mr. L Ms. B Mr. L Mr. L Mr. L Ms. B Ms. B Mr. L Ms. B

Ms. B Mr. L Ms. B Ms. B Mr. L Ms. B Ms. B Ms. B Mr. L Mr. L

Teacher Teacher2 Anton, X Becca, X Celeste, X 1=B,2=L 1=L,2=B (1-4) (1-4) (1-4) 1 2 1 2 1 2 1 1 1 2 1 2 2 2 3 2 1 1 1 2 2 1 1 3 2 2 1 2 2 2 1 2 1 2 3 1 2 2 3 2 2 1 1 3 3 1 2 2 2 4 1.4 2.1 2.4 0.52 0.74 0.84 -0.41 -0.14 -0.25 0.41 0.14 0.25 1 2 1 1 2 1 1 1 2 2

2 1 2 2 1 2 2 2 1 1

4 3 4 3 4 4 3 4 3 4 3.6 0.52 -0.17 0.17

3 4 1 3 4 3 2 3 4 4 3.1 0.99 0.78 -0.78

4 4 2 1 3 4 2 3 3 4 3 1.05 0.41 -0.41

Date Location 2-Jan center 3-Jan home 4-Jan center 5-Jan home 8-Jan home 9-Jan center 10-Jan home 11-Jan center 12-Jan home 15-Jan home 16-Jan center 17-Jan home 18-Jan center 19-Jan home 22-Jan home 23-Jan center 24-Jan home 25-Jan center 26-Jan home 29-Jan home 30-Jan center 31-Jan home 1-Feb center 2-Feb home 5-Feb home 6-Feb center 7-Feb home 8-Feb center 9-Feb home 12-Feb home 13-Feb center 14-Feb home 15-Feb center 16-Feb home 19-Feb home 20-Feb center 21-Feb home 22-Feb center

Overall mood (1-5) 1 3 2 4 2 3 4 2 2 3 3 4 3 3 2 2 3 4 3 2 3 3 3 4 3 1 2 3 3 3 2 4 3 4 5 3 2 3

Show preference (~10 opps per session)

20 20 25 11 27 20 30 25 18 30 20 22 22 33 20 25 27 40 36 22 30 36 40 45 50 36 33 20 25 33 30 40 25 20 22 20 30 27

Match photo (1-4)

Ready, Set… Prompt Imitate (~5 opps per level (1- mouth (1session) 4) 4)

1 1 1 2 1 2 1 1 2 2 2 1 2 2 2 1 2 2 2 2 3 2 3 3 2 3 3 3 2 3 3 3

20 17 20 17 20 29 29 33 40 29 25 20 40 33 33 40 40 20 17 17 20 40 33 40 50 33 17 40 33 25 40 40 33 40 43 40 33 60

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 1 1 2 1 2 2 1 2 1 2 1 2 1 2 2 2 2

1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 2 1 1 1 1 1 2 1 1 1 2 1 2 1 1 1 1 2 1 1 1 1

23-Feb 26-Feb 27-Feb 28-Feb 1-Mar 2-Mar 5-Mar 6-Mar 7-Mar 8-Mar 9-Mar 12-Mar 13-Mar 14-Mar 15-Mar 16-Mar 19-Mar 20-Mar 21-Mar 22-Mar 23-Mar 26-Mar 27-Mar 28-Mar 29-Mar 30-Mar

home home center home center home home center home center home home center home center home home center home center home home center home center home

3 2 3 3 4 3 4 2 3 3 3 2 3 3 3 2 2 4 4 3 4 2 3 2 3 3

33 40 36 33 42 36 30 40 50 45 44 44 50 45 50 56 44 40 36 50 44 45 44 40 36 45

2 3 2 3 3 3 3 3 3 3 3 3 3 3 3 3 2 3 3 3 3 3

60 33 40 60 50 50 40 40 60 60 40 50 40 60 50 40 50 43 50 60 29 60 60 57 40 50

1 1 2 2 2 1 2 2 2 1 2 2 2 2 1 2 2 2 2 2 2 1 2 1 2 2

2 1 1 1 1 2 1 1 1 1 1 2 1 1 1 1 1 2 1 1 1 1 2 1 1 1

Date 2-Jan 3-Jan 4-Jan 5-Jan 8-Jan 9-Jan 10-Jan 11-Jan 12-Jan 15-Jan 16-Jan 17-Jan 18-Jan 19-Jan 22-Jan 23-Jan 24-Jan 25-Jan 26-Jan 29-Jan 30-Jan 31-Jan

Location center home center home home center home center home home center home center home home center home center home home center home

Location 1 2 1 2 2 1 2 1 2 2 1 2 1 2 2 1 2 1 2 2 1 2

Overall mood (1-5) 1 3 2 4 2 3 4 2 2 3 3 4 3 3 2 2 3 4 3 2 3 3

Mean SD Pearson r (loc) Pearson r (mood) 1-Mar 2-Mar 5-Mar 6-Mar 7-Mar 8-Mar 9-Mar 12-Mar 13-Mar 14-Mar

center home home center home center home home center home

1 2 2 1 2 1 2 2 1 2

4 3 4 2 3 3 3 2 3 3

Show preference (~10 opps per session)

20 20 25 11 27 20 30 25 18 30 20 22 22 33 20 25 27 40 36 22 30 36

Match photo (1-4)

1 1 1 2 1 2 1 1 2 2 2 1 2 2 2 1 2 2

25.41 6.93 0.02 0.24

1.56 0.51 -0.03 0.22

42 36 30 40 50 45 44 44 50 45

3 3 3 3 3 3 3 3 3

15-Mar 16-Mar 19-Mar 20-Mar 21-Mar 22-Mar 23-Mar 26-Mar 27-Mar 28-Mar 29-Mar 30-Mar Mean SD Pearson r (loc) Pearson r (mood)

center home home center home center home home center home center home

1 2 2 1 2 1 2 2 1 2 1 2

3 2 2 4 4 3 4 2 3 2 3 3

50 56 44 40 36 50 44 45 44 40 36 45

3 3 3

43.45 5.88 -0.10 -0.38

2.95 0.23 -0.18 -0.36

3 2 3 3 3 3 3

Ready, Set… Prompt Imitate (~5 opps per level (1- mouth (1session) 4) 4)

20 17 20 17 20 29 29 33 40 29 25 20 40 33 33 40 40 20 17 17 20 40

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 1

1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 2 1 1 1 1 1

27.23 8.90 -0.02 -0.14

1.09 0.29 -0.06 -0.11

1.09 0.29 -0.06 0.09

50 50 40 40 60 60 40 50 40 60

2 1 2 2 2 1 2 2 2 2

1 2 1 1 1 1 1 2 1 1

50 40 50 43 50 60 29 60 60 57 40 50

1 2 2 2 2 2 2 1 2 1 2 2

1 1 1 2 1 1 1 1 2 1 1 1

49.05 9.05 -0.02 -0.26

1.77 0.43 -0.01 0.27

1.18 0.39 -0.09 0.03

20 Mean 20 SD 25 11 27 20 30 25 18 30 20 22 22 33 20 25 27 40 36 22 30 36

25.41 6.93

Joey, Showing Preferences, Jan-Mar '01 Percentage of time indicated preference w/in 2 hour period

Date 2-Jan 3-Jan 4-Jan 5-Jan 8-Jan 9-Jan 10-Jan 11-Jan 12-Jan 15-Jan 16-Jan 17-Jan 18-Jan 19-Jan 22-Jan 23-Jan 24-Jan 25-Jan 26-Jan 29-Jan 30-Jan 31-Jan

Show preference (~10 opps per session)

60 50 40 30 20 10 0 1/2-1/31

3/1-3/30

42 Mean 36 SD 30 40 50 45 44 44 50 45 50 56 44 40 36 50 44 45 44 40 36 45

Comparison of Means Joey, Showing Preferences, Jan-Mar '01 60.00

43.45 5.88 Percentage of time indicated preference w/in 2 hour period

Date 1-Mar 2-Mar 5-Mar 6-Mar 7-Mar 8-Mar 9-Mar 12-Mar 13-Mar 14-Mar 15-Mar 16-Mar 19-Mar 20-Mar 21-Mar 22-Mar 23-Mar 26-Mar 27-Mar 28-Mar 29-Mar 30-Mar

Show preference (~10 opps per session)

50.00 40.00 30.00 20.00 10.00 0.00 Before

After

1 Mean 1 SD 1 2 1 2 1 1 2 2 2 1 2 2 2 1 2 2

1.56 0.52

Joey, Matching Item to Photo, Jan-Mar '01 3.5 3 4-point scale (1,2,3,4)

Date 2-Jan 3-Jan 4-Jan 5-Jan 8-Jan 9-Jan 10-Jan 11-Jan 12-Jan 15-Jan 16-Jan 17-Jan 18-Jan 19-Jan 22-Jan 23-Jan 24-Jan 25-Jan 26-Jan 29-Jan 30-Jan 31-Jan

Match photo (1-4)

2.5 2 1.5 1 0.5 0 1/2-1/31

3/1-3/30

Joey, Matching Item to Photo, Jan-Mar '01 Comparison of Means

3 Mean 3 SD 3 3 3 3 3 3 3 3 3 3 3 2 3 3 3 3 3

3.50

2.95 0.24

3.00 4-point scale (1,2,3,4)

Date 1-Mar 2-Mar 5-Mar 6-Mar 7-Mar 8-Mar 9-Mar 12-Mar 13-Mar 14-Mar 15-Mar 16-Mar 19-Mar 20-Mar 21-Mar 22-Mar 23-Mar 26-Mar 27-Mar 28-Mar 29-Mar 30-Mar

Match photo (1-4)

2.50 2.00 1.50 1.00 0.50 0.00 Before

After

20 Mean 17 SD 20 17 20 29 29 33 40 29 25 20 40 33 33 40 40 20 17 17 20 40

27.23 8.90

Joey, Approximate "Go," Jan-Mar '01 70 Percentage with prompting

Date 2-Jan 3-Jan 4-Jan 5-Jan 8-Jan 9-Jan 10-Jan 11-Jan 12-Jan 15-Jan 16-Jan 17-Jan 18-Jan 19-Jan 22-Jan 23-Jan 24-Jan 25-Jan 26-Jan 29-Jan 30-Jan 31-Jan

Ready, Set… (~5 opps per session)

60 50 40 30 20 10 0 1/2-1/31

3/1-3/30

50 Mean 50 SD 40 40 60 60 40 50 40 60 50 40 50 43 50 60 29 60 60 57 40 50

49.05 9.05

Joey, Approximate "Go," Jan-Mar '01 Comparison of Means 70.00 Percentage w/ prompting

Date 1-Mar 2-Mar 5-Mar 6-Mar 7-Mar 8-Mar 9-Mar 12-Mar 13-Mar 14-Mar 15-Mar 16-Mar 19-Mar 20-Mar 21-Mar 22-Mar 23-Mar 26-Mar 27-Mar 28-Mar 29-Mar 30-Mar

Ready, Set… (~5 opps per session)

60.00 50.00 40.00 30.00 20.00 10.00 0.00 Before

After

1 Mean 1 SD 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 1

1.09 0.29

Joey, Prompt Level to Pull Adult to Object Jan-Mar '01 2.5 4-point scale (1,2,3,4)

Date 2-Jan 3-Jan 4-Jan 5-Jan 8-Jan 9-Jan 10-Jan 11-Jan 12-Jan 15-Jan 16-Jan 17-Jan 18-Jan 19-Jan 22-Jan 23-Jan 24-Jan 25-Jan 26-Jan 29-Jan 30-Jan 31-Jan

Prompt level (14)

2 1.5 1 0.5 0 1/2-1/31

3/1-3/30

2 Mean 1 SD 2 2 2 1 2 2 2 2 1 2 2 2 2 2 2 1 2 1 2 2

Joey, Prompt Level to Pull Adult to Object Comparison of Means Jan-Mar '01

1.77 0.43

2.50

4-point scale (1,2,3,4)

Date 1-Mar 2-Mar 5-Mar 6-Mar 7-Mar 8-Mar 9-Mar 12-Mar 13-Mar 14-Mar 15-Mar 16-Mar 19-Mar 20-Mar 21-Mar 22-Mar 23-Mar 26-Mar 27-Mar 28-Mar 29-Mar 30-Mar

Prompt level (14)

2.00 1.50 1.00 0.50 0.00 Before

After

1 Mean 1 SD 1 1 1 1 1 1 1 1 1 1 2 1 1 1 2 1 1 1 1 1

1.09 0.29

Joey, Imitate Mouth Movements Jan-Mar '01 2.5 4-point scale (1,2,3,4)

Date 2-Jan 3-Jan 4-Jan 5-Jan 8-Jan 9-Jan 10-Jan 11-Jan 12-Jan 15-Jan 16-Jan 17-Jan 18-Jan 19-Jan 22-Jan 23-Jan 24-Jan 25-Jan 26-Jan 29-Jan 30-Jan 31-Jan

Imitate mouth (14)

2 1.5 1 0.5 0 1/2-1/31

3/1-3/30

1 Mean 2 SD 1 1 1 1 1 2 1 1 1 1 1 2 1 1 1 1 2 1 1 1

Joey, Imitate Mouth Movements Jan-Mar '01 Comparison of Means

1.18 0.39 4-point scale (1,2,3,4)

Date 1-Mar 2-Mar 5-Mar 6-Mar 7-Mar 8-Mar 9-Mar 12-Mar 13-Mar 14-Mar 15-Mar 16-Mar 19-Mar 20-Mar 21-Mar 22-Mar 23-Mar 26-Mar 27-Mar 28-Mar 29-Mar 30-Mar

Imitate mouth (14) 1.80 1.60 1.40 1.20 1.00 0.80 0.60 0.40 0.20 0.00

Before

After

Date

2-Feb 2-Feb 5-Feb 5-Feb 7-Feb 7-Feb 9-Feb 9-Feb 12-Feb 12-Feb 14-Feb 14-Feb 16-Feb 16-Feb 19-Feb 19-Feb 21-Feb 21-Feb 23-Feb 23-Feb 26-Feb 26-Feb 28-Feb 28-Feb 2-Mar 2-Mar 5-Mar

Time of day

1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15

Teacher or Aide

Change

D C D C D C D C D C D C D C D C D C D C D C D C D C D

no no no yes no no no no no no no no yes no no no no no yes no no no no yes no no no

Face unfamiliar person (1-5)

Volume and inflection (1-5)

1 2

2 2

2

2

2 1 2

1 2 2

2 2 2 2 3

2 1 1 2 3

2 2 2 1 3 1 2

3 2 3 2 3 2 2

3 2 1 2

3 3 2 3

Conversation repair (1-5)

Topic switches (tally)

1 1 2 2 1 1 2 1 2 2 2 2 1 2 2 1 2 3 2 3 2 2 2 2 2 3 2

3 3 3 3 2 3 3 2 2 3 3 2 3 3 3 2 3 3 2 2 3 3 2 3

% request via question (~4 opps per score)

25 25 50 33 25 40 40 25 25 33 40 33 40 25 40 33 33 40 25 50 20 40 50 50 33 40 40

5-Mar 7-Mar 7-Mar 9-Mar 9-Mar 12-Mar 12-Mar 14-Mar 14-Mar 16-Mar 16-Mar 19-Mar 19-Mar 21-Mar 21-Mar 23-Mar 23-Mar 26-Mar 26-Mar 28-Mar 28-Mar 30-Mar 30-Mar

1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15

C D C D C D C D C D C D C D C D C D C D C D C

no no no no no no yes no no no no no no no no no no no yes no no no no

3

4

3 2 2 2 2

3 2 3 2 3

3 1 2 2 3 1 3

4 2 3 3 3 2 4

3 2 2

3 3 3

3 2 3

4 3 4

2 3 3 2 2 2 2 2 2 2 2 2 2 2 3 3 2 2 3 2 3 2 2

3 3 3 2 3 2 3 2 2 3 3 3 2 2 3 3 2 3 2 2 2

75 50 33 67 75 75 50 40 75 40 50 60 60 50 75 50 60 67 40 50 75 60 75

Date 2-Feb 2-Feb 5-Feb 5-Feb 7-Feb 7-Feb 9-Feb 9-Feb 12-Feb 12-Feb 14-Feb 14-Feb 16-Feb 16-Feb 19-Feb 19-Feb

Time of day 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15

Teacher or Aide D C D C D C D C D C D C D C D C

Mean SD Pearson r (teacher/aide) Pearson r (change)

Teacher or Aide 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2

Change no no no yes no no no no no no no no yes no no no

Change 1 1 1 2 1 1 1 1 1 1 1 1 2 1 1 1

Face unfamiliar person (1-5) 1 2

Volume and inflection (1-5) 2 2

2

2

2 1 2

1 2 2

2 2 2 2 3

2 1 1 2 3

2

3

Conversation repair (1-5) 1 1 2 2 1 1 2 1 2 2 2 2 1 2 2 1

1.92 0.51 0.60 0.08

1.92 0.67 0.18 0.06

1.56 0.51 -0.13 -0.05

Topic switches (tally) 3 3 3 3 2 3 3

2.71 0.47 0.23 -0.19

2 2 3 3 2 3 3

% request via question (~4 opps per score) 25 25 50 33 25 40 40 25 25 33 40 33 40 25 40 33 33.25 7.82 -0.31 0.16

Date 12-Mar 12-Mar 14-Mar 14-Mar 16-Mar 16-Mar 19-Mar 19-Mar 21-Mar 21-Mar 23-Mar 23-Mar 26-Mar 26-Mar 28-Mar 28-Mar 30-Mar 30-Mar

Time of day 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15 1:30-2:15

Teacher or Aide D C D C D C D C D C D C D C D C D C

Mean SD Pearson r (teacher/aide) Pearson r (change)

Teacher or Aide 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2

Change no yes no no no no no no no no no no no yes no no no no

Change 1 2 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1

Face unfamiliar person (1-5) 2 2

Volume and inflection (1-5) 2 3

3 1 2 2 3 1 3

4 2 3 3 3 2 4

3 2 2

3 3 3

3 2 3

4 3 4

Conversation repair (1-5) 2 2 2 2 2 2 2 2 2 3 3 2 2 3 2 3 2 2

2.27 0.70 0.72 -0.15

3.07 0.70 0.68 -0.04

2.22 0.43 0.27 0.24

Topic switches (tally) 3

2 2 2 2 3 2 1 2

% request via question (~4 opps per score) 75 50 40 75 40 50 60 60 50 75 50 60 67 40 50 75 60 75

2.19 0.54 -0.07 -0.09

58.44 12.90 0.30 -0.38

3 2 2 2 2 3 2

2-Feb 2-Feb 5-Feb 5-Feb 7-Feb 7-Feb 9-Feb 9-Feb 12-Feb 12-Feb 14-Feb 14-Feb 16-Feb 16-Feb 19-Feb 19-Feb

1 Mean 2 SD

1.92 0.51

Mai Lin, Face Unfamiliar Person (Feb-Mar '01)

2 2 1 2 2 2 2 2 3 2

5-point scale (1,2,3,4,5)

Date

Face unfamiliar person (1-5)

3.5 3 2.5 2 1.5 1 0.5 0 Before

After

12-Mar 12-Mar 14-Mar 14-Mar 16-Mar 16-Mar 19-Mar 19-Mar 21-Mar 21-Mar 23-Mar 23-Mar 26-Mar 26-Mar 28-Mar 28-Mar 30-Mar 30-Mar

2 2 3 1 2 2 3 1 3 3 2 2 3 2 3

Comparison of Means Mai Lin, Face Unfamiliar Person (Feb-Mar '01) Mean SD

2.27 0.70

3.50 5-point scale (1,2,3,4,5)

Date

Face unfamiliar person (1-5) 3.00 2.50 2.00 1.50 1.00 0.50 0.00

Before

After

Face Miss D (1-5)

2-Feb 5-Feb 7-Feb 9-Feb 12-Feb 14-Feb 16-Feb 19-Feb

2 2 2 2 2 2 3 2

1

12-Mar 14-Mar 16-Mar 19-Mar 21-Mar 23-Mar 26-Mar 28-Mar 30-Mar

2 3 2 3 3 3 2 3 3

Mai Lin, Face Unfamiliar Person (Feb-Mar '01) Mean SD

2.13 0.35

Mean SD

2.67 0.50

5-point scale (1,2,3,4,5)

Date

Face Miss C (1-5)

1 2 2

2

3.5 3 2.5 2 1.5 1 0.5 0 Before

1 2 1

After

Comparison of Means Mai Lin, Face Unfamiliar Person (Feb-Mar '01)

2

5-point scale (1,2,3,4,5)

2 3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 Before

After

2-Feb 2-Feb 5-Feb 5-Feb 7-Feb 7-Feb 9-Feb 9-Feb 12-Feb 12-Feb 14-Feb 14-Feb 16-Feb 16-Feb 19-Feb 19-Feb

2 2

Mean SD

1.92 0.67

Mai Lin, Voice Volume (Feb-Mar '01)

2 1 2 2 2 1 1 2 3

5-point scale (1,2,3,4,5)

Date

Volume and inflection (1-5)

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Before

3

After

12-Mar 12-Mar 14-Mar 14-Mar 16-Mar 16-Mar 19-Mar 19-Mar 21-Mar 21-Mar 23-Mar 23-Mar 26-Mar 26-Mar 28-Mar 28-Mar 30-Mar 30-Mar

2 3 4 2 3 3 3 2 4 3 3 3 4 3 4

Comparison of Means Mai Lin, Voice Volume (Feb-Mar '01) Mean SD

3.07 0.70

4.00 5-point scale (1,2,3,4,5)

Date

Volume and inflection (1-5) 3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00

Before

After

Date

Volume Miss C (1-5)

Volume Mrs D (1-5)

2-Feb 5-Feb 7-Feb 9-Feb 12-Feb 14-Feb 16-Feb 19-Feb

2 2 1 2 2 1 3 3

2

12-Mar 14-Mar 16-Mar 19-Mar 21-Mar 23-Mar 26-Mar 28-Mar 30-Mar

3 4 3 3 4 3 3 4 4

Mean(Miss C) SD

2.00 0.76

Mean(Miss C) SD

3.44 0.53

2 1 2

2 2 3 2 3 3

5-point scale (1,2,3,4,5)

Mai Lin, Voice Volume (Feb-Mar '01) Observations of Miss C only 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Before

After

5-point scale (1,2,3,4,5)

Comparison of Means Mai Lin, Voice Volume (Feb-Mar '01) Observations of Miss C only 4.50 4.00 3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 Before

After

Date

Conversation repair (1-5)

2-Feb 2-Feb 5-Feb 5-Feb 7-Feb 7-Feb 9-Feb 9-Feb 12-Feb 12-Feb 14-Feb 14-Feb 16-Feb 16-Feb 19-Feb 19-Feb

1 1 2 2 1 1 2 1 2 2 2 2 1 2 2 1

Mean SD

1.56 0.51

Mai Lin, Conversational Repair (Feb-Mar '01)

5-point scale (1,2,3,4,5)

3.5 3 2.5 2 1.5 1 0.5 0 Before

After

Date

Conversation repair (1-5)

12-Mar 12-Mar 14-Mar 14-Mar 16-Mar 16-Mar 19-Mar 19-Mar 21-Mar 21-Mar 23-Mar 23-Mar 26-Mar 26-Mar 28-Mar 28-Mar 30-Mar 30-Mar

2 2 2 2 2 2 2 2 2 3 3 2 2 3 2 3 2 2

Mean SD

2.22 0.43

Mai Lin, Conversational Repair (Feb-Mar '01) Comparison of Means

5-point scale (1,2,3,4,5)

3.00 2.50 2.00 1.50 1.00 0.50 0.00 Before

After

Topic switches (tally)

Date

2-Feb 2-Feb 5-Feb 5-Feb 7-Feb 7-Feb 9-Feb 9-Feb 12-Feb 12-Feb 14-Feb 14-Feb 16-Feb 16-Feb 19-Feb 19-Feb

3 3 3 3 2 3 3

Mean SD

2.71 0.47

2 2 3 3 2 3 3

Mai Lin, Topic Switching (Feb-March '01)

Number of topic switches

3.5 3 2.5 2 1.5 1 0.5 0 2/2-2/19

3/12-3/30

Date

Topic switches (tally)

12-Mar 12-Mar 14-Mar 14-Mar 16-Mar 16-Mar 19-Mar 19-Mar 21-Mar 21-Mar 23-Mar 23-Mar 26-Mar 26-Mar 28-Mar 28-Mar 30-Mar 30-Mar

3

Mean SD

2.19 0.54

3 2 2 2 2 3 2 2 2 2 2 3 2 1 2

Mai Lin, Topic Switching Comparison of Means

Number of topic switches

3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 Before

After

Date

% request via question (~4 opps per score)

2-Feb 2-Feb 5-Feb 5-Feb 7-Feb 7-Feb 9-Feb 9-Feb 12-Feb 12-Feb 14-Feb 14-Feb 16-Feb 16-Feb 19-Feb 19-Feb

25 25 50 33 25 40 40 25 25 33 40 33 40 25 40 33

Mean SD

33.25 7.82

Mai Lin, Requests as Question (Feb-Mar '01)

Percentage successful repairs

80 70 60 50 40 30 20 10 0 2/2-2/19

3/12-3/30

Date

% request via question (~4 opps per score)

12-Mar 12-Mar 14-Mar 14-Mar 16-Mar 16-Mar 19-Mar 19-Mar 21-Mar 21-Mar 23-Mar 23-Mar 26-Mar 26-Mar 28-Mar 28-Mar 30-Mar 30-Mar

75 50 40 75 40 50 60 60 50 75 50 60 67 40 50 75 60 75

Mean SD

58.44 12.90

Mai Lin, Request as Question (Feb-Mar '01) Comparison of Means

Percentage successful repairs

80.00 70.00 60.00 50.00 40.00 30.00 20.00 10.00 0.00 Before

After

Date

Family member

Allergy meds

1-Aug 2-Aug 3-Aug 4-Aug 5-Aug 6-Aug 7-Aug 8-Aug 9-Aug 10-Aug 11-Aug 12-Aug 13-Aug 14-Aug 15-Aug 16-Aug 17-Aug 18-Aug 19-Aug 20-Aug 21-Aug 22-Aug 23-Aug 24-Aug 25-Aug 26-Aug

mom mom dad dad mom mom dad dad mom mom dad dad mom mom dad dad mom mom dad dad mom mom dad dad mom mom

y y y y n y n y y y y y n y y y n y n y y n y y y n

Overall mood (1-5)

Private reference (1-4)

Secure listener attention (1-4)

Eye contact before request (1-4)

Number of verbal exchanges (tally)

Indicate disapproval or protest (1-4)

1 4 3 4 2 5 4 3 2 3 3 4 2 3 3 4 4 3 4 2 3 4 4 3 2 3

2 1 1 1 2 1 1 2 2 2 2 1 3 2 2 1 1 2 1 3 2 1 1 2 3 2

1 2 2 1 1 1 2 2 1 2 1 2 2 1 2 2 1 3 1 2 3 1 2 2 1 2

1 1 1 2 1 2 1 2 2 1 1 1 1 2 1 1 2 2 1 2 1 2 3 1 2 2

1 1 1 2 1 2 2 2 1 2 2 2 2 1 2 1 2 1 3 2 2 2 2 3 3 2

1 1 1 1 1 1 1 2 1 1 1 1 2 1 1 1 2 1 1 1 2 2 1 1 1 1

27-Aug 28-Aug 29-Aug 30-Aug 31-Aug 1-Sep 2-Sep 3-Sep 4-Sep 5-Sep 6-Sep 7-Sep 8-Sep 9-Sep 10-Sep 11-Sep 12-Sep 13-Sep 14-Sep 15-Sep

dad dad dad mom mom dad dad mom mom dad dad mom mom dad dad mom mom dad dad mom

y n y n n y y y n y y y n n y y y n y y

3 4 4 4 3 4 5 4 3 4 4 4 3 2 4 4 4 4 5 4

2 1 1 2 2 1 1 1 2 1 2 1 2 3 1 2 1 1 1 2

2 1 2 1 2 3 2 1 2 2 2 1 2 3 2 2 1 3 2 2

2 2 2 1 2 3 1 1 1 2 1 1 2 2 2 2 2 1 2 2

3 2 2 2 3 2 3 2 3 2 2 3 2 3 2 2 2 3 3 2

2 1 1 1 1 1 2 1 1 1 1 1 2 1 1 1 1 1 2 1

Date

Family member

Family member

Allergy meds

Allergy meds

1-Aug 2-Aug 3-Aug 4-Aug 5-Aug 6-Aug 7-Aug 8-Aug 9-Aug 10-Aug 11-Aug 12-Aug 13-Aug 14-Aug 15-Aug 16-Aug

mom mom dad dad mom mom dad dad mom mom dad dad mom mom dad dad

1 1 2 2 1 1 2 2 1 1 2 2 1 1 2 2

y y y y n y n y y y y y n y y y

1 1 1 1 2 1 2 1 1 1 1 1 2 1 1 1

Overall mood (1-5)

Private reference (1-4)

Secure listener attention (1-4)

1 4 3 4 2 5 4 3 2 3 3 4 2 3 3 4

2 1 1 1 2 1 1 2 2 2 2 1 3 2 2 1

1 2 2 1 1 1 2 2 1 2 1 2 2 1 2 2

1 1 1 2 1 2 1 2 2 1 1 1 1 2 1 1

1 1 1 2 1 2 2 2 1 2 2 2 2 1 2 1

1 1 1 1 1 1 1 2 1 1 1 1 2 1 1 1

1.63 0.62 -0.37 0.30 -0.76

1.56 0.51 0.34 0.10 0.24

1.31 0.48 -0.09 -0.32 0.19

1.56 0.51 0.49 0.10 0.37

1.13 0.34 0.03 0.30 -0.24

2 1 1 1

2 3 2 1

2 3 1 1

3 2 3 2

1 1 2 1

Mean SD Pearson r (parent) Pearson r (meds) Pearson r (mood) 31-Aug 1-Sep 2-Sep 3-Sep

mom dad dad mom

1 2 2 1

n y y y

2 1 1 1

3 4 5 4

Eye contact before request (1-4)

Number of verbal exchanges (1-4)

Indicate disapproval or protest (1-4)

4-Sep 5-Sep 6-Sep 7-Sep 8-Sep 9-Sep 10-Sep 11-Sep 12-Sep 13-Sep 14-Sep 15-Sep

mom dad dad mom mom dad dad mom mom dad dad mom

Mean SD Pearson r (parent) Pearson r (meds) Pearson r (mood)

1 2 2 1 1 2 2 1 1 2 2 1

n y y y n n y y y n y y

2 1 1 1 2 2 1 1 1 2 1 1

3 4 4 4 3 2 4 4 4 4 5 4

2 1 2 1 2 3 1 2 1 1 1 2

2 2 2 1 2 3 2 2 1 3 2 2

1 2 1 1 2 2 2 2 2 1 2 2

3 2 2 3 2 3 2 2 2 3 3 2

1 1 1 1 2 1 1 1 1 1 2 1

1.50 0.63 0.27 0.55 -0.77

2.00 0.63 0.42 0.44 -0.28

1.69 0.60 0.37 -0.10 -0.14

2.44 0.51 0.07 0.49 -0.12

1.19 0.40 -0.20 0.02 0.34

Date

Allergy meds

Allergy meds

1-Aug 2-Aug 3-Aug 4-Aug 5-Aug 6-Aug 7-Aug 8-Aug 9-Aug 10-Aug 11-Aug 12-Aug 13-Aug 14-Aug 15-Aug 16-Aug

y y y y n y n y y y y y n y y y

1 1 1 1 2 1 2 1 1 1 1 1 2 1 1 1

Mean SD Pearson r (meds) Pearson r (meds')

Allergy meds, shifted*

1 1 1 1 2 1 2 1 1 1 1 1 2 1 1

Eye contact before request (1-4)

1 1 1 2 1 2 1 2 2 1 1 1 1 2 1 1 1.31 0.48 -0.32 0.71

Date

Allergy meds

Allergy meds

Allergy meds, shifted*

31-Aug 1-Sep 2-Sep 3-Sep 4-Sep 5-Sep 6-Sep 7-Sep 8-Sep 9-Sep 10-Sep 11-Sep 12-Sep 13-Sep 14-Sep 15-Sep

n y y y n y y y n n y y y n y y

2 1 1 1 2 1 1 1 2 2 1 1 1 2 1 1

2* 2 1 1 1 2 1 1 1 2 2 1 1 1 2 1

Mean SD Pearson r (meds) Pearson r (meds') *value pulled from Raw Data for 8/30/00

Eye contact before request (1-4)

2 3 1 1 1 2 1 1 2 2 2 2 2 1 2 2 1.69 0.60 -0.10 0.63

2 1 1 1 2 1 1 2 2 2 2 1 3 2 2 1

Mean SD

1.63 0.62

Tyler, Private Reference Aug-Sep '00 3.5 4-point scale (1,2,3,4)

Date

1-Aug 2-Aug 3-Aug 4-Aug 5-Aug 6-Aug 7-Aug 8-Aug 9-Aug 10-Aug 11-Aug 12-Aug 13-Aug 14-Aug 15-Aug 16-Aug

Private reference (1-4)

3 2.5 2 1.5 1 0.5 0 8/2-8/16

8/31-9/15

2 1 1 1 2 1 2 1 2 3 1 2 1 1 1 2

Mean SD

1.50 0.63

Tyler, Private Reference Aug-Sep '00 of Means

Comparison

2.50 4-point scale (1,2,3,4)

Date

31-Aug 1-Sep 2-Sep 3-Sep 4-Sep 5-Sep 6-Sep 7-Sep 8-Sep 9-Sep 10-Sep 11-Sep 12-Sep 13-Sep 14-Sep 15-Sep

Private reference (1-4)

2.00 1.50 1.00 0.50 0.00 Before

After

1-Aug 2-Aug 3-Aug 4-Aug 5-Aug 6-Aug 7-Aug 8-Aug 9-Aug 10-Aug 11-Aug 12-Aug 13-Aug 14-Aug 15-Aug 16-Aug

1 2 2 1 1 1 2 2 1 2 1 2 2 1 2 2

Mean SD

1.56 0.51

Tyler, Secure Listener's Attention Aug-Sep '00 3.5 3 4-point scale (1,2,3,4)

Date

Secure listener attention (1-4)

2.5 2 1.5 1 0.5 0 8/2-8/16

8/31-9/15

31-Aug 1-Sep 2-Sep 3-Sep 4-Sep 5-Sep 6-Sep 7-Sep 8-Sep 9-Sep 10-Sep 11-Sep 12-Sep 13-Sep 14-Sep 15-Sep

2 3 2 1 2 2 2 1 2 3 2 2 1 3 2 2

Mean SD

2.00 0.63

Tyler, Secure Listener's Attention Aug-Sep '00 Comparison of Means 3.00 4-point scale (1,2,3,4)

Date

Secure listener attention (1-4)

2.50 2.00 1.50 1.00 0.50 0.00 Before

After

1-Aug 2-Aug 3-Aug 4-Aug 5-Aug 6-Aug 7-Aug 8-Aug 9-Aug 10-Aug 11-Aug 12-Aug 13-Aug 14-Aug 15-Aug 16-Aug

1 1 1 2 1 2 1 2 2 1 1 1 1 2 1 1

Mean SD

1.31 0.48

Tyler, Eye Contact Before Request Aug-Sep '00 3.5 3 4-point scale (1,2,3,4)

Date

Eye contact before request (1-4)

2.5 2 1.5 1 0.5 0 8/3-8/16

8/31-9/15

31-Aug 1-Sep 2-Sep 3-Sep 4-Sep 5-Sep 6-Sep 7-Sep 8-Sep 9-Sep 10-Sep 11-Sep 12-Sep 13-Sep 14-Sep 15-Sep

2 3 1 1 1 2 1 1 2 2 2 2 2 1 2 2

Mean SD

1.69 0.60

Tyler, Eye Contact Before Request, Aug-Sep '00 Comparison of Means 2.50 4-point scale (1,2,3,4)

Date

Eye contact before request (1-4)

2.00 1.50 1.00 0.50 0.00 Before

After

1-Aug 2-Aug 3-Aug 4-Aug 5-Aug 6-Aug 7-Aug 8-Aug 9-Aug 10-Aug 11-Aug 12-Aug 13-Aug 14-Aug 15-Aug 16-Aug

1 1 1 2 1 2 2 2 1 2 2 2 2 1 2 1

Mean SD

1.56 0.51

Tyler, Number of Exchanges Aug-Sep '00 3.5 Number of Exchanges

Date

Number of verbal exchanges (tally)

3 2.5 2 1.5 1 0.5 0 8/3-8/16

8/31-9/15

31-Aug 1-Sep 2-Sep 3-Sep 4-Sep 5-Sep 6-Sep 7-Sep 8-Sep 9-Sep 10-Sep 11-Sep 12-Sep 13-Sep 14-Sep 15-Sep

3 2 3 2 3 2 2 3 2 3 2 2 2 3 3 2

Mean SD

2.44 0.51

Tyler, Number of Exchanges Aug-Sep '00 Comparison of Means 3.50 Number of Exchanges

Date

Number of verbal exchanges (tally)

3.00 2.50 2.00 1.50 1.00 0.50 0.00 Before

After

1-Aug 2-Aug 3-Aug 4-Aug 5-Aug 6-Aug 7-Aug 8-Aug 9-Aug 10-Aug 11-Aug 12-Aug 13-Aug 14-Aug 15-Aug 16-Aug

1 1 1 1 1 1 1 2 1 1 1 1 2 1 1 1

Mean SD

1.13 0.34

Tyler, Indicate Disapproval or Protest Aug-Sep '00 2.5 4-point scale (1,2,3,4)

Date

Indicate disapproval or protest (1-4)

2 1.5 1 0.5 0 8/1-8/16

8/31-9/15

31-Aug 1-Sep 2-Sep 3-Sep 4-Sep 5-Sep 6-Sep 7-Sep 8-Sep 9-Sep 10-Sep 11-Sep 12-Sep 13-Sep 14-Sep 15-Sep

1 1 2 1 1 1 1 1 2 1 1 1 1 1 2 1

Mean SD

1.19 0.40

Tyler, Indicate Disapproval or Protest Aug-Sep '00 Comparison of Means 1.80 4-point scale (1,2,3,4)

Date

Indicate disapproval or protest (1-4)

1.60 1.40 1.20 1.00 0.80 0.60 0.40 0.20 0.00 Before

After

Appendix A: Basic Intervention Goals for Children with Autism

Appendix A Basic Intervention Goals for Children with Autism Attention and basic social relatedness • • • • • • • • • • • • • •

Child will respond to the overtures of familiar/preferred adults with smile, frown, reach, vocalization, or other intentional behavior. Child will respond to the overtures of familiar/preferred adults with obvious pleasure. Child will demonstrate affection towards others. Child will seek comfort when hurt. Child will demonstrate awareness of others by seeking proximity. Child will stay engaged with familiar adult for increasing lengths of time. Child will become displeased when preferred adult is unresponsive during play for 30 seconds or more. Child will spontaneously seek the company of his/her family members when family is not attempting to engage him/her. When engaged with a family member/trusted adult, frequency of subvocalizations will diminish. Child will demonstrate awareness of others by showing some simple imitation. Child will acknowledge the comings and goings of familiar people. Child will call family members by name. Child will call family members and other familiar people by name. Child will focus attention on a social activity for minutes.

Imitation • • • • • • •

Child will imitate with object after demonstration of use of object. Child will simultaneously imitate with objects. Child will imitate hand movements. Child will imitate body movements. Child will imitate mouth movements. Child will imitate sounds. Child will imitate words.

Affect • •

Child will look up to caregiver using smile as a way of securing adult attention. Child will show positive emotional expressions in response to praise.

From Goals to Data and Back Again

A-1

Copyright © 2001-02 Lehman and Klaw

Appendix A: Basic Intervention Goals for Children with Autism

• • • • • • • • • •

Child will independently solicit praise upon the completion of a task. Child will label feeling states in self [begin with happy, sad, angry/mad, scared]. Child will identify emotions in family members/familiar adults/peers. Child will respond appropriately to emotions in family members/familiar adults/peers. Child will offer comfort to others in distress. Child will match spoken expressions of sadness, happiness, anger, and surprise with facial expressions of the same emotions. Child will accurately identify the feelings he/she has in a variety of settings and will be able to explain the relationship of events to his/her feelings. Child will use pretend play scenarios to explore negative affect and practice appropriate responses. Child will be tolerant of own mistakes and performances that were not perfect. To express precision and subtlety in the expression of emotion, child will use qualifiers to describe gradation of emotional experience (e.g., really disappointed, a little disappointed).

Self-regulation • • • • • •

Child will recover from distress within minutes with help from familiar adult. Child will communicate through language when upset, rather than tantrum. Child will learn different strategies for self-calming during times of frustration, anxiety, anger, or disappointment. Child will use appropriate strategies for controlling his/her body when excited, anxious, or angry. Child will maintain a polite and/or tactful style of communication when letting others know that something is bothering them. Child will productively reflect upon the advantages and disadvantages of own behavior.

Play Increasing the play repertoire • • • • • •

Child will joyfully participate in sensory-motor play with a familiar adult. Child will participate in songs, finger-plays, and rhymes with familiar adults. Child will engage in parallel play. Child will engage in simple motor games with rules. Child will participate in turn taking activities. Child will appropriately look at books with caregivers.

From Goals to Data and Back Again

A-2

Copyright © 2001-02 Lehman and Klaw

Appendix A: Basic Intervention Goals for Children with Autism

• • •

Child will expand his/her play repertoire to include manipulation, sensorymotor, art, music, building/construction, and early cognitive (sorting, matching, puzzles). Child will participate in physical games with rules (e.g., “Duck, Duck, Goose”). Child will participate in non-physical games with rule (e.g., board games).

Pretend Play • • • • • • • • •

Child will develop interest in the content of pretend play as opposed to the simple mechanics (i.e., interest will move from how the bottle fits in baby’s mouth to helping hungry baby). Child will use one object to represent another. Child will participate in pretend play involving concrete and familiar themes such as self-care, daily activities, cars, and animals. Child will develop nurturing play with baby dolls. Child will participate in increasingly elaborate make-believe, moving from early concrete (episodes of eating/feeding, driving cars with noise, putting farm animals in barn) to more complex concrete (simple familiar stories). Child will arrange doll furniture into meaningful groups and will use doll figures to act out simple themes from own experience. Child will participate in more elaborate play themes, moving from concrete themes (involving everyday, common experience) to abstract themes (involving material never directly experienced). Child will assume the role of another person (dress-up). Child will engage in role-playing using figures and puppets.

Drawing • • • • • •

Child will scribble with crayon. Child will imitate drawing of vertical line. Child will imitate drawing of circle. Child will add 3 parts to incomplete human drawing. Child will copy drawing of square. Child will draw unmistakable human with body, arms, legs, feet, nose, eyes, and mouth.

From Goals to Data and Back Again

A-3

Copyright © 2001-02 Lehman and Klaw

Appendix A: Basic Intervention Goals for Children with Autism

Communication Receptive communication (understanding language) • • • • • • • • • • • • • • • •

Child will respond to his/her name. Child will look for family members when asked, “Where is Mommy?” or “Where is Daddy?” Child will stop action in response to “No!” Child will appropriately respond to the command, “Stop!” Child will move body in response to a one-step direction. Child will get familiar object or food that is requested. Child will take object or food to someone when requested. Child will follow two-step directions involving two different actions. Child will indicate approval when asked a “Do you want?” question. Child will appropriately respond to simple and familiar “Where” questions with searching movements. Child points to eyes, nose, and mouth in self and others upon request. Child identifies all large body parts upon request. Child will point to pictures in a book or familiar objects as they are named. Child will follow a series of 2-3 simple related commands with the same object. Child will identify smaller body parts upon request (i.e., chin, knee, elbow, fingers, and toes). Child will follow a series of three unrelated commands. Eye Gaze

• • • • • • • • • • •

Child will look at person when given something. Child will look at person when giving them something. Child will follow someone’s point when object is in close proximity and can be touched. Child will point to desired object when object can be touched. Child will follow someone’s point when object is distant. Child will point to desired object when object is distant. Child points to direct someone to look at object or event to share enjoyment while looking back and forth to make sure adult sees what child sees. Child will look towards adult to make sense of an ambiguous situation. Child will reference adult expression to guide own behavior. Child will look at person who is speaking to communicate interest/attention. Child will look at person to whom he/she is speaking to make sure person is listening/attending.

From Goals to Data and Back Again

A-4

Copyright © 2001-02 Lehman and Klaw

Appendix A: Basic Intervention Goals for Children with Autism

Expressive communication (body language and affect) • • • • • • • • • • • • • •

Child will respond to gestures with intentional gestures of his/her own (e.g., reaches out in response to outstretched arms). Child will initiate interactions (e.g., reaches for toy). Child will look when name is called. Child will wave goodbye. Child will choose from two options using gestures and body language. Child will express desire for food using gestures and body language. Child will express desire for activity using gestures and body language. Child will express desire for toy/object using gestures and body language. Child will indicate disapproval using gestures and body language. Child will express wishes, intentions, and feelings using multiple gestures in a row. Child will find appropriate and effective ways to get attention. Child will participate in 4 reciprocal social interactions. Child will participate in 8 reciprocal social interactions. Child will participate in 12 reciprocal social interactions. Expressive communication (the use of symbols for communication)

• • • • • • • • • •

• • • • • •

Child will learn fill-in-the-blanks of familiar songs, rhymes, and or familiar verbal routines (e.g., “Ready, set, go”). Child will use word/sign/picture for “more.” Child will choose from two options using pictures/signs/words Child will express desire for food using pictures/signs/words. Child will express desire for activity using pictures/signs/words. Child will express desire for toy/object using pictures/signs/words. Child will indicate disapproval using pictures/signs/words. Child will indicate that he is done with an activity by saying or signing, “All done.” Child will use pictures/signs/words for mother and father. Child will develop consistent vocabulary of symbols used in the absence of concrete gestures (e.g., child will come into the dining room and say “juice” to mother to request juice without needing to take mother to refrigerator and touch the juice bottle). Child will respond to question, “What’s this?” Child will ask question, “What’s this?” Child will spontaneously add single words to play, beginning to narrate play actions. Child will use two-word combinations. Child will use “my” or “mine”. Child will refer to self by name.

From Goals to Data and Back Again

A-5

Copyright © 2001-02 Lehman and Klaw

Appendix A: Basic Intervention Goals for Children with Autism

• • • • • • • • • • • • • • • • • • • •

Child will ask questions by raising pitch at end of word or phrase. Child will ask for help. Child will say first and last name when asked. Child will use pronouns “I,” “me,” and “you.” Child will talk about an event that has just happened. Child will respond to “What” and “Who” questions. Child will respond to “Where” and “When” questions. Child will respond to “Why” questions. Child will spontaneously ask “wh-“ questions. Child will use complex language in imaginative play to narrate actions. Child will use prepositions “in,” “on,” and “under.” Child will describe objects according to size, color, and shape. Child will use pronouns “he,” “she,” “they,” “his,” “her,“ “our,” and “their.” Child will use the terms “here,” “there,” “this,” and “that.” Child will ask meaning of new words. Child will retell a brief story. Child will tell home address. Child will talk about the future using “will.” Child will use pronouns “himself” and “herself.” Child will compare objects using “-er” and “-est” endings. Conversational Skills/Pragmatics

• • • • • • • • • • • • •

Child will use attention-getting words such as “Hey!” Child will use appropriate volume with conversational partner. Child will use meaningful inflection with conversational partner. Child will use appropriate distance between self and conversational partner. Child will make appropriate adjustments when initiating conversation in order to gain and keep partner’s attention (i.e., raising her voice, adding a gesture). Child will attend to peers when they address her/him, responding appropriately. Child will say “What?” or “Excuse me, could you say it again?” or similar phrase when he/she doesn’t understand question posed by an adult. When others initiate conversation, child will respond in appropriate, multiword phrases. Child will use eye contact to signal conversational turn taking. Child will be able to engage in conversation over a broad range of topics. Child will add new, relevant information to previous comments in conversation. Child will ask questions that are related to topic to maintain conversational flow. Child will make transition statements to signify a change in conversational topic.

From Goals to Data and Back Again

A-6

Copyright © 2001-02 Lehman and Klaw

Appendix A: Basic Intervention Goals for Children with Autism

• • • • • • • • •

Child will put his/her thoughts on pause so adult/peer can add to, or comment on, the conversation. Child will initiate conversation that is of interest to social partner. Child will change style of interaction when speaking with very young children. Child will change style of interaction when speaking with peers as opposed to adults. Child will use names of adults/siblings/peers when addressing them. Child will ask “how,” “why,” and “when” questions in order to obtain information. Child will provide relevant information to adult when it is requested. Child will provide relevant information to peers/sibling when it is requested. Child will share experiences through narration (describing connection between settings, characters’ behavioral and emotional responses, and consequences).

Sensory issues • • • • • • • • • • • •

Child will eat a greater variety of foods. Child will gain comfort with activities in which his/her feet are off the ground. Child will become sensitized to, and appropriately label, hot/cold/pain. Child will walk around toys, pets, and people on floor. Child will successfully avoid bumping into people. Child will develop compensatory strategies for feeling comfort while in large, open spaces. Child will employ appropriate strategies to reduce overwhelming stimuli in new environments. Child will become more comfortable with activities that involve hands and face. Child will become more comfortable with multiple voices singing. Child will tolerate proximity of other children. Child will remain socially engaged, as is typical for Child, in the midst of a group of children. Child will remain socially engaged, as is typical for Child, in new environments.

Restricted interests and perseverative behaviors • • • •

Instances of perseveration [specify types] will be successfully redirected. Instances of idiosyncratic motor behaviors will decrease. Playing with toys or objects in atypical/repetitive ways will decrease. Reciting passages from books, videos, TV, and/or radio will decrease.

From Goals to Data and Back Again

A-7

Copyright © 2001-02 Lehman and Klaw

Appendix A: Basic Intervention Goals for Children with Autism

• • • • •

Child will stay focused on shared conversation with caregivers instead of lapsing into private reference. Instances of perseveration around rules, when child appears bossy, will decrease. Child will tolerate changes in routines. Child will demonstrate interest and pleasure in a range of developmentally appropriate play activities. Child will expand repertoire of social play activities.

Concept development • • • • • • • • • • • • •

• • • •

Child will demonstrate an understanding of self by pointing to self in mirror, in photos, and/or by labeling self by name. Child will demonstrate an understanding of ownership by word, sign, or gesture. Child will demonstrate understanding of function of familiar objects by selecting correct item or insisting on correct item when “mistakenly” given wrong item. Child will demonstrate knowledge of the spatial concepts IN, ON, and UNDER. Child will demonstrate understanding of quantity concepts ONE, MORE, and ALL. Child will demonstrate knowledge of gender by pointing to boy/girl upon request. Child will demonstrate an understanding of the spatial concepts FRONT and BACK by moving his/her body or moving objects. Child will demonstrate knowledge of FRONT and BACK of clothes. Child will demonstrate spatial concepts ABOVE/BELOW and TOP/BOTTOM. Child will demonstrate understanding of same/different. Child will demonstrate understanding of first/middle/last. Child will demonstrate understanding of causality as demonstrated by appropriately answering “Why” questions. To demonstrate a growing understanding of time and sequence, child will spontaneously use time markers in conversation [in the following order: now, later, soon, before, after, breakfast time, lunch time, dinnertime, morning, afternoon, night, yesterday, today, tomorrow, a long time ago, days of the week, months of the year]. Child will recall recent/familiar events with logical sequence. To demonstrate an understanding of locative state and prepositions, child will be able to answer “where” questions. Child will comprehend the word “not” in sentences, such as “Which car is not in the line?” Child will be able to group items into the following categories: color, size, shape, function, texture, taste, and temperature.

From Goals to Data and Back Again

A-8

Copyright © 2001-02 Lehman and Klaw

Appendix A: Basic Intervention Goals for Children with Autism

• • • • • • •

Child will practice sorting by one attribute. Child will practice sorting by more than one attribute at a time. Child will accurately describe the relationship of both immediate and extended family members using the appropriate labels for relatives. Child will demonstrate an understanding of graduated size by stacking and nesting blocks. Child will use the prefix “-est” to demonstrate knowledge of relative size Child will draw accurate inferences from auditory information, answering questions such as “What do you think will happen next?” or “How do you think so-and-so might be feeling?” Child will demonstrate the ability to guess, speculate, estimate, and imagine to arrive at an answer or to solve a problem.

Increasing social awareness • • • • • • • • • •

Child will watch what others are doing and shape his/her behavior accordingly. Child will be able to identify what another person is experiencing. Child will identify what another person knows. Child will predict what others might see or hear in a given situation. Child will demonstrate an awareness of the needs of others by spontaneously offering help. Child will receive a daily compliment for being considerate. Child will predict what others might think or feel in a given situation. Child will demonstrate concept that his/her actions have an effect on the way other people feel. Child will demonstrate the knowledge that other people do not know what Child is thinking or feeling. Child will demonstrate the ability to teach another person how to do something, figuring out just what that other person needs to know.

Social skills with peers • • • • • • •

Child will successfully initiate conversation/play with peer. When someone does not want to play with Child, he/she will be able to formulate a new plan of action. Child will appropriately respond to peers when they make social overtures. Child will decline an invitation to play or converse using appropriate communication. Child will develop tactful responses to describe dislikes and disagreements. Child will sustain interaction with peers. Child will be able to join others already engaged in a play activity (as opposed to having a peer join her/him in their activity).

From Goals to Data and Back Again

A-9

Copyright © 2001-02 Lehman and Klaw

Appendix A: Basic Intervention Goals for Children with Autism

• • • • • • • • • •

Child will tolerate and stay engaged in play with peer even when not in charge. Child will communicate with peers when ready to change activities. Child will demonstrate flexibility and the ability to adapt in social settings by accommodating play suggestions from peers. Child will sustain interaction on a playdate. Child will share toys when appropriate. Child will successfully negotiate over toys. Child will demonstrate appropriate responses to children who are mean or hurtful. Child will apologize if and when he/she bumps into someone. Child will apologize if and when he/she hurts someone’s feelings or body Child will learn to talk on the phone in a developmentally appropriate manner.

Social norms • • • • • • • • •

In an age appropriate manner, Child will wait for his/her turn to talk. Child will demonstrate an understanding of modesty and/or privacy by being fully clothed when leaving the bathroom in public places. Child will refrain from publicly touching private body parts. Child will wipe nose on tissue and throw tissue away. Child will demonstrate an understanding of ownership by refraining from taking someone else’s food or belongings. Child will demonstrate age-appropriate modesty. Child will demonstrate age-appropriate tact. Child will refrain from asking embarrassing or intrusive questions of conversational partner. Child will refrain from interrupting people who are talking on the phone.

School and camp skills • • • • • • • •

In an age-appropriate fashion, child will follow teacher’s instructions. Child will attend to verbal instructions, using compensatory strategies when necessary. Child will successfully transition between activities. Child will participate in large group activities. Child will participate in small group activities. Child will raise hand when wishing to speak and will wait until he/she is called on before speaking out loud. Child will tolerate the times when he/she is not picked for an activity. Child will listen to a book read aloud to the group and will give information about the story when asked by the teacher.

From Goals to Data and Back Again

A-10 Copyright © 2001-02 Lehman and Klaw

Appendix A: Basic Intervention Goals for Children with Autism

• • • • • • •

Child will follow the typical schedule. Child will tolerate changes in the schedule when prepared for these changes. Child will stay in close proximity to the group when on field trip. Child will demonstrate the ability to use the teachers for needed help and support with a decreasing reliance upon therapist/aide. Child will tolerate being at the end or in the middle of the line. Child will tolerate not being first. Child will note what others are doing and shape behavior accordingly.

Leisure • • • • • • •

Child will increase repertoire of tolerable family outings. Child will keep caregivers informed of where he/she is going during outings. Child will join routine family activities, from start to finish, without needing to be continually prompted to stay focused. Child will join non-routine family activities, from start to finish, without needing to be coerced or continually prompted to stay focused. Child will demonstrate understanding of winning and losing. Child will be able to tolerate winning and losing. Child will demonstrate increasing comfort with participation in physical activities with other children.

ADDITONAL SOURCES USED FOR GOAL LIST CONNECT, “Getting from Here to There: Patterns of Growth,” (Developmental Milestones for Communication, Cognition, Social Development, Motor Development – 0-5 years). Dalrymple & Ruble, “Specialized Areas of Assessment and Learning for Young Children with Autism,” 1998. Gard, Gilman & Gorman, “Speech and Language Development Chart.” Garnett & Attwood, “Australian Scale for Asperger’s Syndrome,” 1995. Greenspan & Wieder, “Observation of Milestones Checklist,” 1997. Klaw, ”PAR – Preschool Autism Rating Scale,” 1999. Robertson et al, “Domains of Social Communication Handicap in Autism Spectrum Disorder,” 1999. Sigman & Capps, Children with Autism – A Developmental Perspective, 1997. Sparrow, Bella, Cichetti, “Vineland Adaptive Behavior Scale.” Wetherby, “Screening for Social and Communication Disorder in the First 2 Years,” 1999.

From Goals to Data and Back Again

A-11

Copyright © 2001-02 Lehman and Klaw

Appendix B: Charts and Handouts

Appendix B Charts and Handouts

Please note: because different versions of Microsoft® Word are sometimes inconsistent in the way they encode certain characters, the charts in this appendix may not look like they do in Appendix B of the book. We regret this unavoidable inconvenience but remind you that you are free to photocopy the charts from Appendix B for educational purposes.

From Goals to Data and Back Again

B-1

Copyright © 2001-02 Lehman and Klaw

Appendix B: Charts and Handouts

The 3 Phases of Intervention Collecting data for children with autism changes over time. There are distinct phases.

PHASE 1: With multiple and supported opportunities, the skill EMERGES in a single environment with consistent caregivers. Example: With multiple prompts and frequent modeling, Johnny will wave goodbye.

From Goals to Data and Back Again

PHASE 2:

PHASE 3:

Now that the skill has emerged, we target CONSISTENCY and INDEPENDENCE.

Now that there is consistency under certain conditions, expectations are EXTENDED to include multiple environments and numerous people.

Example: With a decreasing need for prompts, Johnny will wave goodbye to familiar adults and his siblings.

Example: In all environments, Johnny will consistently wave goodbye when someone who is leaving waves to him or says “goodbye.”

Chart 3.1

Copyright © 2001-02 Lehman and Klaw

HOW DO WE KNOW WHEN GOALS ARE COMPLETE?

Skill targeted in goal ♦; ♦ ♦ ♦

Family input Team input Curriculum Developmental guidelines

Skill emerges during 1:1 intervention in single environment ♦ Learned incidentally ♦ Taught through naturalistic means ♦ Taught through drills

Skill improves during 1:1 intervention in single environment ♦ Becomes developmentally appropriate ♦ Becomes consistent ♦ Becomes independent

Skill generalizes ♦ Adults ♦ Home

siblings peers familiar environment

From Goals to Data and Back Again

Chart 3.2

community

Copyright © 2001-02 Lehman and Klaw

WRITING MEASURABLE GOALS

↕ Frequency:

What we measure: QUALIFY:

CHANGE OVER TIME How we measure it: TAKE DESCRIPTION OF DESIRED BEHAVIOR,

• • • • •

Where When (how often) With whom Level of support Type of measurement

• Quantitative scales Tally of occurrence Percentages • Qualitative scales ↕ Duration:

QUALIFY IT QUANTIFY:

QUANTIFY IT

• • • •

↕ frequency of behavior ↕ duration of behavior ↑ range of behavior ↓ need for prompts

• Quantitative time scale ↑ Range of behavior: • List mastered over time ↓ Need for Prompt Qualitative scale

From Goals to Data and Back Again

Chart 3.3

Copyright © 2001-02 Lehman and Klaw

DETECTING CHANGE USING THE COMPARISON OF MEANS

First: Divide data into thirds Copy the first third and label it “before” Ignore the middle third Copy the last third and label it “after”

Next: Check the scale for this goal to determine its category width ➲ If the “before” and “after” means are at least category width apart, then…

➲ If the “before” and “after”

…Use the overlap in the standard deviations to check the reliability of the difference in the means.

…Conclude that there’s been no reliable change, but look for correlations to help understand why.

Compute the mean and standard deviation for the “before” and “after” data sets separately

➲ If there’s no overlap… conclude that there has been reliable change!

Calculate the difference between the “before” and “after” means

➲ If there’s a small overlap… look for outliers & correlations with other factors before you decide if there’s been progress. ➲ If there’s a large overlap… conclude there’s been no reliable change, but look for correlations to help understand why.

From Goals to Data and Back Again

Chart 8.1

means are not a category width apart, then…

Last: Discuss the data with the team to plan the next round of intervention!

Copyright © 2001-02 Lehman and Klaw

Appendix B: Charts and Handouts

Sample Data Sheet—2 year old, more compromised Child: Bailey Date:

Time of day: Morning afternoon

What was the longest amount of time Bailey stayed socially engaged with you during a preferred activity?

When given three short breaks over the course of the therapy session, how many times did Bailey seek you out to begin playing again?

evening ≥1 min. ≥3 min. ≥5 min. ≥7 min. 0/3 1/3 2/3 3/3

When you attempt to engage Bailey in some familiar play routine, how often did she immediately smile at you?

Rarely smiled at first Smiled occasionally at your overtures Smiled often at your overtures Smiled consistently when approached in a playful manner

Did Bailey look at you when requesting something?

Rarely, even when response withheld to encourage eye gaze About half the time when response withheld Used eye gaze with requests some of the time Frequently used eye gaze with requests

When a pleasurable play routine is briefly interrupted, did Bailey use body language, affect, eye gaze and vocalizations to indicate that she wants to continue?

Didn’t communicate a wish to continue Used affect/body language Used affect/body language and eye gaze Used affect/body language/eye gaze and vocalizations

How many times did Bailey engage in early pretend play after you modeled this for her?

0 1 2 3 >3

When Bailey needed help, did she get someone’s hand and pull that person to the problem?

Even with prompting, unable to do this Did this once with multiple prompts Did this more than once with prompts Did this at least once on her own <1 min. <2 min <3 min <4 min

How long was Bailey able to look at a book with you? Did Bailey vocalize today while playing?

Consistently quiet Vocalized a little during certain activities Vocalized a little throughout the session Vocalized consistently throughout the session.

From Goals to Data and Back Again

B-6

Copyright © 2001-02 Lehman and Klaw

Sample Data Sheet—4 year old, more compromised Child: Matthew Date:___________________

Observer:_______________________

On a scale of 1 to 5, please rate Matthew’s day overall.

1 2 3 (distressed/distracted)

4

5 (content/alert)

1. How much time did it take for Matthew to settle into the playgroup experience and begin to participate?

>30 minutes >20 minutes >10 minutes <10 minutes

2. On average, how did Matthew respond to parallel play activities?

Very resistant today Somewhat resistant Guarded participation Willing participation

3. On average, how long was Matthew able to participate in the parallel play activities with support as needed?

Participated only for a couple of minutes Participated for about half the expected length of time Participated most of the time Stayed with the group for the whole time

4. Please circle or list the sensory activities in which Matthew participated today: Play dough / fun dough; shaving cream; water play; sand; finger painting; pudding painting; jello play; ________________; __________________. 5. Please circle or list the common parallel play activities in which Matthew participated today: large blocks; little blocks; sticky blocks; cars; ball play; emptying and filling; early doll play; jello-in-the-bowl; _________________; ______________. 6. Did Matthew use contact gestures such as grabbing your hand or tapping you to get your attention when he communicates wants/needs?

Only occasionally Some of the time Frequently Consistently

7. Did Matthew spontaneously look at you when making requests?

Needed to be prompted Occasionally < half the time > half the time

8. On the average, what level of prompt was needed to assist Matthew in signing “more” to request the continuation of a pleasurable activity?

Hand-over-hand Modeling with elbow prompt Modeling only Independent

From Goals to Data and Back Again

B-7

Copyright © 2001-02 Lehman and Klaw

Appendix B: Charts and Handouts

Sample Data Sheet—6 year old, moderately compromised Child: John Patrick Date: ________________

Time in: _________AM/PM Time out: _______AM/PM

How would you rate John Patrick’s overall performance today? Rate John Patrick’s ability to recall daily events, without visual support: Given 4 opportunities over the course of intervention, how often did John Patrick independently respond to WHEN questions?

Did John Patrick participate in social pretend play with little figures? No opportunity to observe

Place of Service: Classroom Resource room

1 2 (self-absorbed)

3

1 2 (difficult)

3

4 5 (easy to engage) 4 5 (easy)

0/4 1/4 2/4 3/4 4/4 Minimal participation, even with support Improving participation, supported Participating with minimal need for support Some independent participation Consistently independent

When recalling daily events, how often was John Patrick able to correctly classify these as having occurred in the morning, afternoon, or evening?

Number of times correct: ______ Number of opportunities: ______ %: ______

When John Patrick’s friends were already engaged in play, was he able to join them?

Refused to join even with prompts Joined friends with multiple prompts Joined friends with just a single prompt Spontaneously joined friends 1-2 times Spontaneously joined friends many times

Did John Patrick independently initiate conversation with peers?

Only when prompted Once or twice A few times Many times (quite chatty)

On the average, what level of support was needed to help John Patrick sustain interaction with friends?

Was unable to sustain even with maximum support Frequent verbal prompts/some physical support Frequent verbal prompts Occasional verbal prompts

Rate John Patrick’s overall flexibility when interacting with peers. On the average, how effectively was John Patrick able to protest when play with peers became too rough?

From Goals to Data and Back Again

1 2 3 (inflexible)

4 5 (flexible)

Did not protest, even with modeling/prompting Protested with prompting from adults Protested independently but was ineffective Protested immediately and effectively

B-8

Copyright © 2001-02 Lehman and Klaw

Sample Data Sheet—school-aged child, more able Child: Sarah Date: __________

Time In: __________AM/PM Time Out:_________AM/PM

Please indicate on a scale of 1 (selfabsorbed/idiosyncratic) to 5 (easily engaged/easily integrated), an overall assessment to Sarah’s day. 1 2 3 4 5

Home Home w/ play date After-school program Community outing

When provided with an opportunity for discussion, was Sarah able to discuss the effect of her negative behavior on others?

Refused to discuss Discussed with much prompting Discussed with minimal prompting Discussed with single prompt only, such as “Let’s talk about…”

No opportunity to observe Given 4 opportunities to respond to you calling her name, what percentage of the time did Sarah respond appropriately after only a single cue (e.g. “Sarah, look at me a minute…” or “Sarah, I need to talk to you”) Cue used:_______________________

0/4 = 0% 1/4 = 25% 2/4 = 50% 3/4 = 75% 4/4 = 100%

Over the course of an hour play date, how many times did Sarah wander way and take a break from the interaction?

Needed multiple breaks of 5 or more minutes Needed multiple momentary breaks Needed only 1-2 breaks Needed no breaks

Not applicable Did Sarah demonstrate a willingness to do something her friend wanted to do even if she didn’t want to do it?

No opportunity to observe How did Sarah react when she was playing with peers and things didn’t go her way? (Note: When Sarah has a tantrum, she screams, yells inappropriate things, may try to hit others and may throw things) No opportunity to observe From Goals to Data and Back Again

B-9

Refused Agreed to do a non-preferred activity but attempted to change or control the activity once it began Negotiated a compromise Agreed to participate in non-preferred activity for the sake of friendship Tantrummed and had to be removed Tantrummed but with support could reengage in activity No tantrum but perseverated on disappointment throughout activity Expressed disappointment in appropriate manner and agreed to continue activity

Copyright © 2001-02 Lehman and Klaw

Appendix B: Charts and Handouts

Glossary boundary condition: a number that is chosen to distinguish among alternatives when making a decision. For example, in the comparison of means there are two boundary conditions: the minimum category width and the overlap between the standard deviations. In each case there is a numeric value that determines whether change is or is not present. A boundary condition is typically somewhat arbitrary but must be chosen to make a procedure uniform and repeatable (Chapter 8). categorical data: data whose values can be only names or categories. Typical examples include where intervention takes place and the child’s gender. When data is categorical no value is better than any other value. Also called nominal data (Chapter 6). category width (category boundary): the numeric difference between scores that corresponds to a qualitative difference in behavior. In a 4-point qualitative scale, for example, a score of 1 corresponds to a different category of behavior than a score of 2, a score of 2 reflects different behavior than 3, and so on. So the minimum category width for a qualitative scale is 1.00, and a change in the means from 1.00 to 3.00 would reflect progress across two category boundaries (Chapter 7). change: the transformation of one regular pattern of behavior into another regular pattern of behavior over time as a result of learning or development (Chapter 7). See also, pre-transformation phase and post-transformation phase. consistency: a term that refers to frequent and repeated practice of behaviors or skills within specific settings. Consistency is the overall focus of Phase 2 of intervention (Chapter 3). See also, three phases of intervention. correlation: a relationship in which variation in the value of one factor or goal predicts variation in the value of a different factor or goal (Chapter 9). See also, positive correlation and negative correlation. data collection: the process that turns a desire for behavioral change stated in general terms into a set of questions that can be answered by observing a child’s actions (Chapter 6). data analysis: the process that turns those observations back into meaningful statements about changes in the child’s behavior over a particular period of intervention (Chapter 6).

From Goals to Data and Back Again

B-10 Copyright © 2001-02 Lehman and Klaw

data point: the number, checkmark, word, or phrase used to record a child’s performance on a data collection sheet. Also referred to as an observation, value, or score (Chapter 6). data set: a set of values (or data points, observations, or scores) that represent performance in a coherent way. For example, all the observations associated with one goal form a data set, but so do all the scores for a single child, all the data points taken in a particular location, and all the values collected during a single session (Chapter 6). data type: a formal description of the mathematical properties of a data set. The type of a data set determines what kind of statistical tests can be applied to the data set. See also categorical data, interval data, ordinal data, and nominal data (Chapter 6). emergence: a term that refers to the appearance of a new behavior or skill given multiple and supported opportunities in a sheltered, consistent and supportive therapeutic environment. Emergence is the overall focus of Phase 1 of intervention (Chapter 3). See also, three phases of intervention. empirical developmental intervention: an approach that incorporates data collection and analysis into intervention that is based on developmental and cognitive theory (Chapter 1). extension: a term that refers to the expansion of newly acquired behaviors or skills to multiple settings. Extension is the overall focus of Phase 3 (Chapter 3). See also, three phases of intervention. factor data: any set of values or observations that has been generated by a goalindependent factor. Typical examples of factor data include the dates of sessions held during the interval of time being analyzed, numeric evaluations of the child’s overall mood each session, and the name of the location where each session was held during the interval (Chapter 6). goal data: any set of observations that measure a child’s performance on a goal (Chapter 6). goal-independent factor (or goal factor): a variable that may affect a child’s performance even though it has nothing to do with the particular skill being taught. Typical examples of goal-independent factors are time of day, time of year, changes in routine, changes in caregivers, and changes in medication (Chapter 3). hypothesis: a tentative expectation about performance or its measurement that is based on theory or data (Chapter 9).

From Goals to Data and Back Again

B-11 Copyright © 2001-02 Lehman and Klaw

Appendix B: Charts and Handouts

interval data: data whose values can be arranged in a meaningful order and in which the intervals between values are equal and obey the laws of arithmetic. Typical examples of interval data include tallies and percentages (Chapter 6). mean value: a data set’s arithmetic average. Alternatively, a representative value for a data set (Chapter 7). negative correlation: a relationship in which increase/decrease in the value of one factor or goal predicts decrease/increase in the value of a different factor or goal (Chapter 9). noise: values that contribute to an apparent lack of regularity in a pattern of data even when such regularity actually exists (Chapter 7). nominal data: data whose values can be only names or categories. Typical examples include a child’s name or gender. When data is nominal no value is better than any other value. Also called categorical data (Chapter 6). observation: the number, checkmark, word, or phrase used to record a child’s performance on a data collection sheet. Also referred to as a value, score or data point (Chapter 6). ordinal data: data whose values can be arranged in a meaningful order. The intervals between the values are not known to be equal and cannot be combined arithmetically. The translation into numbers of observations that were recorded on a 4-point or 5-point qualitative scale always results in ordinal data (Chapter 6). outlier: a single value that stands out against an otherwise regular pattern of behavior (Chapter 9). percentage: a system of measurement that is used to evaluate progress when the frequency of opportunity for the behavior is known. Percentages are derived by dividing the number of times a behavior is observed by the number of opportunities for that behavior, then multiplying by 100 (Chapter 3). positive correlation: a relationship in which increase/decrease in the value of one factor or goal predicts increase/decrease in the value of a different factor or goal (Chapter 9). post-transformation phase: the time period during which the child’s regular pattern of behavior reflects learning of a new skill, emotion, or behavioral response. In the comparison of means method, the last third of the data collected during the interval being analyzed is considered to come from the posttransformation phase (Chapter 7). See also, pre-transformation phase and transformation phase.

From Goals to Data and Back Again

B-12 Copyright © 2001-02 Lehman and Klaw

pre-transformation phase: the time period during which the child’s dominant pattern of behavior remains ineffective, inappropriate, or maladaptive despite intervention. In the comparison of means method, the first third of the data collected during the interval being analyzed is considered to come from the pretransformation phase (Chapter 7). See also, post-transformation phase and transformation phase. qualifying a goal: a process in which you add text to a goal statement that is specific about the variables that surround the desired outcome. Where the intervention is being done, with whom, when, and with what level of support are all considerations when goals are qualified (Chapter 3). qualitative scale: a descriptive scale that measures change by advances along a behavioral progression (Chapter 3). quantifying a goal: a process in which you chose a system for measuring progress such as increasing/decreasing frequency, increasing/decreasing duration, increasing a range of behaviors and/or decreasing a need for prompts (Chapter 3). quantitative scale: a numeric scale that measures progress by increasing or decreasing quantity (Chapter 3). raw data: observations as they appear on the data collection sheet or in the spreadsheet prior to the application of any statistical tests (Chapter 6). reliability: a property of change as measured by the relationship among the means and standard deviations “before” and “after” learning. If change is reliable then the “after” mean is predictive of the child’s ongoing level of performance without additional intervention. score: the number, checkmark, word, or phrase used to record a child’s performance on a data collection sheet. Also referred to as an observation, value, or data point (Chapter 6). standard deviation: a number that reflects our notion of representation as a function of distance from, or clustering around, the mean. As the standard deviation increases, there is a decrease in both the reliability of the mean as a representative and the difference between the means as an indication of progress (Chapter 8). tally of occurrence: a system of measurement in which the observer counts every time a targeted behavior is performed. This system is used when there is a constant opportunity to perform the behavior (Chapter 3).

From Goals to Data and Back Again

B-13 Copyright © 2001-02 Lehman and Klaw

Appendix B: Charts and Handouts

three phases of intervention: emergence, consistency, and extension. Every goal passes through these three phrases on its way to mastery (Chapter 3). See also, emergence, consistency, and extension. transformation phase: the time period during which the child is learning a new skill, emotional response, or behavior. In the comparison of means method, the middle third of the data collected during the interval being analyzed is considered to come from the transformation phase (Chapter 7). See also, pre-transformation phase and post-transformation phase. type of data: a formal description of the mathematical properties of a data set. The type of a data set determines what kind of statistical tests can be applied to the data set. See also categorical data, interval data, ordinal data, and nominal data (Chapter 6). value: the number, checkmark, word, or phrase used to record a child’s performance on a data collection sheet. Also referred to as an observation, score, or data point (Chapter 6).

From Goals to Data and Back Again

B-14 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Appendix C Exercises We recognize that some readers of this book want to learn a practical method of data collection and analysis that can be used in their everyday lives. These exercises can help you achieve that goal by giving you the chance to practice the different skills we discuss. When you begin to collect and analyze your own data you may find it convenient to pattern your worksheets on the ones we’ve created, even inserting your data into copies of the sheets you’ve practiced on. The number of each exercise reflects the chapter that discusses the skill you will practice. Exercises 3-1 through 5-8 give practice in the skills needed to write measurable goals. In order do these exercises on-line, you must copy the file “Goal Workbook.doc“ from the CD-ROM to your hard drive. The remaining exercises (6-1 and higher) give practice in data analysis. To do these exercises you must copy the file “Analysis Workbook.xls” from the CD-ROM to your computer’s hard drive. Should you mistakenly modify or destroy any of the data in these files while working through the exercises, simply recopy the original files from your CD-ROM. Note that in the spreadsheet exercises (Exercise 6-1 and higher) all directions and figures are based on the PC version of Microsoft® Excel ‘97 using the Chart Wizard feature. You may find small differences between our instructions and your experiences if you are using a later version of Excel on a PC or any version of Excel on a Mac. The files in “Analysis Workbook.xls” can be opened in both PC and Macintosh versions of Excel ’97 and should be compatible with later versions of Excel on both types of computer. The instructions for the spreadsheet exercises assume that you are already able to perform basic Excel actions such as selecting a column of data, copying the contents of a cell, using the on-line help facility, and so on. If you are unfamiliar with Excel at this level, you may want to print out the file on the CD-ROM called “Directions for Using Excel” and keep it close at hand for easy reference. “Directions for Using Excel” can be printed from the CD-ROM or photocopied from Appendix B.

From Goals to Data and Back Again

C-1

Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 3-1: What’s wrong? Correct the following by finding errors, lack of clarity, or developmentally problematic expectations. Answers to this exercise follow. 1. Goal: With a decreasing need for assistance, Johnny will use picture exchange to request a drink 75% of the time during snack time at school. 2. Goal: Sally will use eye gaze to bring attention to herself (as if to say “Look at me!”) 75% of the time. 3. Goal: Sally will begin to sustain social interaction with adults, with siblings, or with peers in school for up to 10 minutes. 4. Goal: Johnny will appropriately answer “Wh-“ questions 50% of the time. Data question: In your estimation, what percentage of the time was Johnny able to answer “What,” “Who,” “When,” “Where,” and “Why” questions? <25% ~25% ~50% ~75% 5. Goal: Sally will sort items by 4 single attributes. Data Question: How many attributes did Sally use to sort or categorize today? 0 1 2 3 4 6. Goal: With a decreasing need for prompts, Johnny will raise his hand and remain quiet until the teacher has called on him. Data Question: What level of prompt was needed for Johnny to raise his hand and remain quiet until called on to speak in school? Verbal and physical prompts Verbal prompts Single verbal prompt No prompt needed

From Goals to Data and Back Again

C-2

Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Answers to Exercise 3-1 1. Goal: With a decreasing need for assistance, Johnny will use picture exchange to request a drink 75% of the time during snack time at school. You can’t measure both a “decreasing need for assistance” and percentage. Those are two different systems for measuring change over time. What would your data show? Would it show a decreasing need for assistance or an increasing percentage of time? You wouldn’t know. To correct this goal, you can do either of the following: With a single prompt, Johnny will use picture exchange to request a drink 75% of the time during snack time at school. Or: With a decreasing need for prompts, Johnny will use picture exchange to request something to drink during snack time at school. 2. Goal: Sally will use eye gaze to bring attention to herself (as if to say “Look at me!”) 75% of the time. You can’t quantify the number of opportunities Sally has to bring attention to herself. There is no way to count this—the opportunity is constant. Therefore, you cannot use percentages as a method for measuring progress for this goal. To correct this goal, you could measure the following: With increasing frequency, Sally will use eye gaze to bring attention to herself (as if to say “Look at me!”). In this case, you would take a tally to establish progress over time. 3. Goal: Sally will begin to sustain social interaction with adults, with siblings, or with peers at school for up to 10 minutes. Sustaining social interaction with adults, with siblings, and with peers in school are three different goals that typically appear at three very different times during development. You shouldn’t try to measure all three in a single goal. 4. Goal: Johnny will appropriately answer “Wh-“ questions 50% of the time. Data question: In your estimation, what percentage of the time was Johnny able to answer WHAT, WHO, WHEN, WHERE and WHY questions? <25% ~25% ~50% ~75%

From Goals to Data and Back Again

C-3

Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

1) The WH questions are mastered by typically developing children at very different times. If you want to target the WH questions, you should write separate goals. 2) Estimating percentages is a shaky process. Remember: if you need to estimate because there are too many opportunities to count, then you should probably pick another method of measurement. 5. Goal: Sally will sort items by 4 single attributes. Data Question: How many attributes did Sally use to sort today? 0 1 2 3 4 Let’s say the team wants Sally to be able to sort by color, shape, size, and function. Over time, they hope to teach her all of these methods for sorting. In setting up the data question this way however, they will never know when she has actually achieved all four skills. Maybe every day the data sheet comes back showing that she only sorted by 1 attribute. They can’t tell which attribute she is using. Perhaps she sorts by color every single day. But perhaps she sorts by a different attribute every day. If she is sorting by different attributes, she may have achieved the goal a long time ago. You could write the data question the following way: What attributes did Sally use for sorting today? color shape size function other:__________ 6. Goal: With a decreasing need for prompts, Johnny will raise his hand and remain quiet until the teacher has called on him. Data Question: What level of prompt was needed for Johnny to raise his hand and remain quiet until called on to speak in school? Verbal and physical prompts Verbal prompts Single verbal prompt No prompt needed It is important to design a prompt hierarchy so that it matches both the child’s need and the situation. In this case, the scale for prompting does not match a classroom environment. There are many ways to help a child raise his hand and remain quiet. Telling a child in a classroom to “Be quiet” or “Raise your hand” is a fairly intrusive prompt. It shouldn’t be the last step before independence. What about a gesture from the teacher? What about the use of visual reminders? What about the use of peer modeling?

From Goals to Data and Back Again

C-4

Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 4-1: Creating data sheets using tables The easiest way to create data sheets like the ones in this book is to lay the information out in tables. In Microsoft® Word, table functions are found in the Table menu of the standard menu bar. Tables are complex objects that can be formatted in many different ways. In this exercise you will use some of the most basic commands to reproduce the following data sheet (a variant of Figure 4-1). Child’s name: Date:_________

Observer’s name:__________________ Time:_________

Please check the types of activities Peter participated in today during free play:

What percentage of the time did Peter respond by saying “Hi” to familiar people who came to his classroom during his morning preschool session?

gross motor construction/building puzzles art music early pretend play How many people came to his classroom? ______ How many times did Peter say “Hi”? ______

What level of prompt was needed to help Peter say goodbye to familiar adults?

Calculate: _______% (# of greetings ÷ # of opportunities x 100) Verbal & physical prompt (hand-over-hand) Verbal with modeling Verbal only No prompt needed

1. Open a blank page in Word. Pull down the Table menu and select Insert Table. A dialog box asks you to specify the number of columns and rows in the table. Since we want a data sheet with three goals and a separate area for factor data we will need two columns and four rows. Fill in those values and click ok. 2. To create a single box in the top row of the table we need to merge the two existing cells. The easiest way to do this is to select them by clicking and dragging across both cells. If you find this difficult, however, simply click in one cell then pull down the Table menu and click Select Row. Once the two cells have been highlighted (with a black bar) click Merge Cells in the Table menu.

From Goals to Data and Back Again

C-5

Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

3. Each cell is it’s own small environment. Click in the left cell of the second row and type in some text. Notice the font size and font style of the text you just typed. Highlight the text and change the font size or font style. Now click in any other cell and type again. You should find that the text in the new cell reverts to the original font size and type. Type in the information for calculating the percentage in the second goal (the blank lines are made by repeated use of the underscore character, which is found above the hyphen on most keyboards). Highlight the text and choose center from the formatting toolbar to complete this cell. Only this cell’s contents will be centered. 4. In the right cell of the second row type the list of activities in the sample data sheet. Highlight the list and click the itemize icon in the formatting toolbar (it’s the icon with the three squares next to three lines). The activities will be converted to an itemized list without disturbing any text in the other cells. If your default bullet character is not a square you may be able to change it. Select the list then click Bullets and Numbering in the Format menu. If you see squares as an option in the resulting dialog box choose it. If you do not see squares, we suggest you read Word’s on-line documentation for customizing bullets. 5. Notice that all the cells containing the scales for the data questions have the same width in your table. It is often useful to move the line that divides the data questions from the scales to use the space on the page more effectively. Move your cursor slowly across the dividing line until it becomes a symbol with two parallel lines and two arrows. Click and drag the cursor left or right to move the line. The whole line will move, simply resizing all the divided cells. To resize one cell highlight the whole cell by moving the cursor just inside the left border until you see the cursor change to a large white arrow then click. Now move the cursor over the dividing line until it becomes the parallel line symbol, click and drag. You should find that just the single cell resizes. If you cannot make this work in your file, you can specify an absolute size for the cell using Cell Height and Width in the Table menu. 6. To get the double lined border click anywhere in the table then click Select Table in the Table menu. Pull down the Format menu and select Borders and Shading. Using the tab for Borders in the dialog box, select Grid for the Setting and then choose the double lines from the list of possible Styles.

From Goals to Data and Back Again

C-6

Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 5-1: Writing goals for Joey Write 5 goals addressing communication needs for Joey. 1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-7

Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 5-2: Making Joey’s goals measurable Qualify and quantify the following goals: •

Joey will indicate preference when offered the choice of two options.

•

Joey will associate photographs with preferred items.

•

Joey will attempt to say “Go!” after being cued with “Ready, set….”

•

Joey will begin to pull his parents to his object of desire.

•

Joey will imitate mouth movements with increasing frequency.

1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-8

Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 5-3: Writing data questions for Joey’s goals Write data questions for the following goals: 1. During the same two hour period each day, Joey will indicate preference 75% of the time when offered the choice of food or an object with which to play or hold. 2. With increasing frequency, Joey will match photographs to preferred items when given the opportunity. 3. After hearing the phrase modeled twice, Joey will attempt to say “Go!” after hearing “Ready, set…” during a familiar and preferred activity. 4. With a decreasing need for assistance, Joey will begin to pull his parents and/or familiar adults to his object of desire. 5. Joey will imitate the mouth movements of familiar adults with increasing frequency.

Data questions

Circle, check or fill in information

1. 2. 3. 4. 5.

From Goals to Data and Back Again

C-9

Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 5-4: Writing goals for Tyler Write five goals addressing Tyler’s communication needs at home. 1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-10 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 5-5: Making Tyler’s goals measurable Qualify and quantify the following goals: •

Incidents of private reference occurring in the midst of social exchange will decrease in frequency.

•

Tyler will learn to secure his family’s attention before talking.

•

Tyler will look at his communicative partner as he is talking or right before he begins to talk with increasing frequency.

•

Tyler will be engaged in a daily verbal exchange of increasing duration in which the adult partner avoids questioning and quizzing, and relies instead on commenting, narrating, rephrasing, and expressing opinions.

•

Tyler will learn to effectively and productively indicate disapproval or protest.

1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-11 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 5-6: Writing data questions for Tyler’s goals Write data questions for Tyler using the following goals: 1. During the one-hour period of time when the family plays together each evening, incidents of private reference occurring in the midst of social exchange will decrease in frequency. 2. During the one-hour period of time when the family plays together each evening, Tyler will learn to secure his family’s attention before talking at least half of the time. 3. Without needing to be reminded, Tyler will consistently look at his communicative partner as he is requesting food or drink. 4. With an increasing number of verbal exchanges, Tyler will have one conversation a day in which the adults use a facilitative approach (avoiding questioning and quizzing, and relying on commenting, narrating, rephrasing, and expressing opinions). 5. Without being coached from family members, Tyler will learn to effectively and productively indicate disapproval or protest.

Data questions

Circle, check or fill in information

1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-12 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 5-7 Writing measurable goals for Mai Lin Write 5 communication goals for Mai Lin in a school setting. As you write them, remember to qualify and quantify. 1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-13 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 5-8: Writing data questions for Mai Lin’s goals Write data questions for Mai Lin based on the following goals. 1. With a decreasing need for reminders, Mai Lin will appropriately face unfamiliar people when she addresses them. 2. Needing no more than a single prompt, Mai Lin will increasingly use appropriate volume and inflection when speaking to unfamiliar people. 3. With consistency, Mai Lin will effectively repair her communicative attempts after a peer says “What?” or indicates in any other way that Mai Lin’s words were not understood. 4. Once a day, Mai Lin will sustain general conversation for 5 minutes without attempting to suddenly introduce a preferred topic. 5. Needing only an expectant pause as a reminder, Mai Lin will make requests using a question format rather than a statement >75% of the time.

Data questions

Circle, check or fill in information

1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-14 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 6-1: How to produce a line graph This exercise takes you through the steps required to produce Figure 6-5 in Chapter 6. Don’t panic at the length of this exercise! Because this is the first of the spreadsheet exercises we discuss each step in great detail. As you become increasingly familiar with Excel by working through the remaining exercises, the directions will become briefer. In this and all the remaining exercises italics are used in the instructions for words that appear exactly as you will see them on-screen in Excel. 1. Open the spreadsheet for Exercise 6.1 in the Analysis Workbook. 2. Click on the graphing icon (circled in Figure C-1). This will start the Chart Wizard, a sequence of four dialog boxes that help you specify the graph.

Figure C-1 The icon to start the Chart Wizard in Excel '97 is circled in black.

3. Selecting the Chart Type. In Step 1 we choose the kind of graph we want to use to represent the data. First make sure that the Standard Types are showing in this dialog box. You can check this by seeing which tab is in front in the window (see Figure C-2 for the location of the tabs). If Standard Types is not in front, click it to bring it to the front. We want to create a line graph that shows lines with markers displayed at each data value. Choose Line from the Chart type menu then click on the Chart subtype darkened in Figure C-2. Click the Next button at the bottom of the dialog box to continue.

From Goals to Data and Back Again

C-15 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Figure C-2 The options to select in the Chart Type dialog box. Also note the tabs circled at the top of the dialog box. Each tab gives you access to a different view of some or all of the information to be specified at this step.

4. Selecting the data range. In Step 2 of the Chart Wizard, we choose the data to be graphed. Make sure that the Data Range tab is in front in this dialog box. If not, click it to bring that view to the front, then click in the box to the right of the words Data range. There are many ways to specify the data to be graphed but the easiest is to select rows 2 to 30 of column B in Anton’s spreadsheet. After you do this, notice how the Chart Wizard automatically fills in the formula for the data range and produces an initial version of the graph for your inspection. •

Specifying the series name. If you examine the initial version of the graph in the dialog box, you’ll see that the line for the data is labeled with the uninformative name Series 1. Click on the tab labeled Series to bring a different view of the data range forward. You’ll see that Series 1 and the formula specifying its values have already been filled into Series and Values, respectively. To change the name of the series, just type “goal X” into the box for the Name. Notice that the label changes in the legend on the graph. Also note how, if there is only one data set to be graphed, Excel also makes its label the default title of the graph (don’t worry, we’ll change that shortly).

•

Specifying the horizontal axis. Still in the Series view, look at the horizontal axis (also called the x-axis) of the graph. Excel has given the axis a default labeling by creating a “tick mark” for each data point and numbering the tick marks sequentially. To recreate Figure 6-5, however, we want the tick marks to indicate the date on which each score was recorded. The date of each session is in column A of Anton’s spreadsheet. So, first click in the box to the right of Category (X) axis labels, then select cells A2 through A30 on Anton’s spreadsheet. Excel will update the information in the box and on the graph

From Goals to Data and Back Again

C-16 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

automatically (it doesn’t always show all the dates if the graph is too crowded to read). Click Next to advance to the next step in the Chart Wizard. 5. Specifying the Chart Options. Step 3 of the Chart Wizard allows you to specify different characteristics of the graph. For better or worse, Excel makes many choices about how it displays the data automatically. Some choices are simple to change while others require that you understand a great deal about how Excel works. Luckily, we can compensate for some of Excel’s bad choices easily by placing clarifying titles on the axes of the graphs and on the graph itself. Before we start, make sure that the Titles tab is in front in this dialog box. If not, click it to bring that view to the front. •

Chart title. The chart title should explain whose data and which goal we’re looking at. Excel has already filled in what it thinks is a good Chart title, “goal X,” based on the name of the single series in the data range. Replace this default title with “Anton, Goal X.”

•

Category or x-axis title. This title explains the meaning of the measurement units on the horizontal axis. We could type something like “Session Date” for this graph, but it would be redundant. To minimize visual clutter, we’ll leave the box blank and let the dates speak for themselves.

•

Value or y-axis title. This title explains the meaning of the measurement units on the vertical axis. Excel automatically chooses increments for this axis based on the values in the data. In this case, the units it has chosen are inappropriate because they imply that values could fall anywhere in the range 0 to 4.5. There is no easy way to change the increments and eliminate the misinformation so we’ll use the axis title to make the actual scale clear to anyone reading the graph. Type in “4-point scale (1,2,3,4).”

•

Other options. There are many other features of the graph that can be modified at this step in the procedure but only one—removing the legend—is necessary to produce the graph exactly as it is seen in Figure 6-5. Exercise 6-2 teaches how to remove the legend and demonstrates the other options used to create the figures in this book. For now, just click Next to advance to the last dialog box.

6. Selecting the Chart location. The final step in the Chart Wizard is choosing where to put the graph. Excel gives you the choice of placing the finished graph on a new sheet in the current workbook or on the sheet from which the data was drawn. For the moment we’ll keep the graph with the data. Select As object in: Exercise 6-1 then click Finish. 7. Compare your completed graph to Figure 6-5 in the book (it should differ only in the presence of the legend) or to the first graph on the sheet called “Answers

From Goals to Data and Back Again

C-17 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

6-1.” To compare the graphs on-line, copy and paste the one you created next to Anton’s graph on the answer sheet. 8. We encourage you to practice further by using the data provided for Becca and Celeste’s goals (columns D and E). The graphs you produce can be checked against Figure 6-8 and Figure 6-9 in the book or against the second and third graphs on the Answers 6-1 sheet in the Analysis Workbook. IMPORTANT! If you want to put more than one graph on a sheet it is critical that a graph not be selected when you start the Chart Wizard. If a graph is selected, the Chart Wizard will overwrite the old graph with the new specification. If a graph is not selected, the Chart Wizard will create a new graph but will drop it right on top of the previous graph when it’s done. To keep this from happening, click on any empty cell before starting the Chart Wizard (or simply move the second graph around after it has been created by clicking-and-dragging it to another part of the spreadsheet).

From Goals to Data and Back Again

C-18 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 6-2: Graphing shortcuts The purpose of this exercise is to show you some of the other features of graphing available in Excel. We assume that you have already done Exercise 6-1 and are now familiar with the basic use of the Chart Wizard. 1. Open Exercise 6-2 in the Analysis Workbook. 2. Pre-specifying the data range. Select Anton’s data for goal X (column B, rows 2 to 30). Now click on the graphing symbol, choose the appropriate line graph, and go to the Data Range dialog box. Notice that Excel has already filled in the range using the data that was selected when you started the Chart Wizard. 3. Pre-specifying the series name. Select the formula for the data range in the box next to the words Data range. We’re going to replace this formula by selecting rows 1 to 30 of column B. (If Excel gives you an error message when you try to do this, make sure you’ve selected the whole formula in the Data range box then select rows 1 to 30 of column B again.) Notice that Excel considers text at the top of a column as the label for that column of data. If you pick up cell B1 at the same time you pre-specify the data (see above) the Chart Wizard will include it in its initial graph. 4. Pre-specifying the horizontal axis. Just as it treats text in the first cell of a column as special, Excel also treats text in the first column of multi-column data as special. Click on Cancel to eliminate the current graph. Now select columns A and B from row 1 to 30 in Anton’s spreadsheet. With these cells highlighted, click the graphing icon and choose the line graph. The initial graph created by the Chart Wizard now has the data range, series name, and x-axis labels already filled in. Click Next to go to the Chart Options dialog box. 5. Removing the legend. The legends for this graph and the graphs we produced in Exercise 6-1 are not particularly informative. To eliminate the legend and create more space for displaying the data, click on the Legend tab to bring the appropriate formatting information to the front. Now click on the check mark next to Show legend. The legend will disappear and the data area of the graph will expand accordingly. To eliminate the legend on an existing graph, just click on the legend and press the DELETE key on your keyboard. 6. Removing the x-axis labels. Sometimes we want all the information about the x-axis to be given in the title for that axis (the comparison of means bar graphs are examples of when we’ve used this approach). To eliminate the labels while you are creating a graph, click on the Axes tab then click on the check mark next to Category (X) axis. To eliminate the labels on an existing graph, click on the x-axis and press the DELETE key on your keyboard.

From Goals to Data and Back Again

C-19 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

7. Other changes. Before you continue with the remainder of this exercise we suggest you click on some of the other tabs and “toggle” different options—that is, click the check marks on and off—to see what happens. Although the figures in this book use only the modifications to the basic graph that we’ve already discussed, you might find some of the other options useful in the future. 8. Finish the graph by clicking on the Titles tab and titling the x-axis with “January 3 to March 10” and the y-axis with “4-point scale (1,2,3,4).” Place the finished graph in the Exercise 6-2 worksheet. 9. Changing a title. There are two ways to change a title on an existing graph. The first is to select the graph by clicking once on the white area. This will add a pull-down menu for Chart to the other pull-down menus for the window (see Figure C-3). If you pull-down the Chart menu you will see Chart Options… Click on it and you will find yourself in front of the same set of tabs we experimented with previously in this exercise. If you click on the Titles tab, you can then change the text as if you were back at this step in the Chart Wizard (click OK when you’re done). Try changing the title for the x-axis to “January 3, 2001 to March 10, 2001” using this method.

Figure C-3 When you select the whole chart the data is highlighted on the spread sheet and a pull-down menu for the Chart appears.

The second, and simpler, way to change a title, however, is to click once on the title itself in the graph. A gray box will appear around the text. Place the cursor where you want to type and click again then make your changes. To indicate you are done, click somewhere outside the graph. Change the title for the y-axis to “5-point scale (1,2,3,4,5)” using this method. Compare your graph to the first graph on the Answers 6-2 spreadsheet.

From Goals to Data and Back Again

C-20 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

10. Multiple goals for one child on a single graph. There may be times when you want to put more than one data set on a single graph to make data presentation more efficient. Remember that it is legitimate to combine goals in a single graph only if the data was taken during the same time interval and the goals you want to combine have the same size scale (all 4-point, for example). To create a graph with multiple goals: a) Click on the graphing icon, choose the line graph, and continue to the Data Range dialog box. b) In general, you will select the columns of data for all the goals you want to graph. Here, select columns A and B to graph Anton’s goals together. c) Finish the graph with the appropriate titles, etc. In this case it is crucial that you leave the legend on the graph so that you know which data is for which goal. Compare your finished graph to the second graph on the Answers 6-2 worksheet. 11. Multiple children for the same goal on a single graph. There may also be times when you want to directly compare performance on the same goal by more than one child. Remember that it is legitimate to compare data across children only if they were exposed to the same group instruction on the same dates. To create a graph for one goal, multiple children: a) Choose the line graph and continue to the Data Range dialog box. b) In general, you will select the columns of data for all the children you want to graph. Here, select columns A, C and D to graph the data for Anton, Becca and Celeste for goal X. To select non-adjacent cells, select the first cell or range of cells and then, holding the CTRL key down, select each additional cell one at a time. c) Finish the graph with the appropriate titles, etc. Again, it is crucial that you leave the legend on the graph so that you know whose data is whose. Compare your finished graph to the third graph on the Answers 6-2 worksheet.

From Goals to Data and Back Again

C-21 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 7-1: Computing the mean 1. Open Exercise 7-1 in the Analysis Workbook. 2. Scroll to the bottom of the data, click on cell A32 and type in “Mean”. You may also want to select all of row 32 and make it boldface; this will help you find the mean more easily when you come back to this sheet in the future. 3. Click on cell B32. This is where the value computed by the average function will be placed. 4. In the tool bar, click on the symbol fx, as shown in Figure C-4. The Paste Function dialog box will appear.

Figure C-4 The locations of the function symbol (white circle), the rounding up symbol (black circle), and the function bar (black rectangle)

5. Choose Statistical from the Function Category menu then choose Average from the Function Name menu. Click OK to move to the next dialog box. 6. Select the data for Anton’s goal X (cells B2 through B30). If you can’t get at column B, move the dialog box out of the way by clicking on it and dragging it across the screen first. 7. Click OK. The dialog box will disappear and a value will appear in cell B32. The value will depend on how many decimal places are showing. We want to round to 2 decimal places. To do this click on the rounding up symbol (.00 -> .0) From Goals to Data and Back Again

C-22 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

in the tool bar (see Figure C-4) until only 2 numbers follow the decimal point. The number 2.55 should now be in cell B32. 8. Right below the second row of the tool bar, to the right of an =, is an area called the formula bar. It is outlined with a black rectangle in Figure C-4. Typing in this area is one way to change the contents of a cell. It is also the place to look if you want to understand what kind of information Excel thinks is in a cell. For example, click on cell B30 and notice that the data value, 4, appears in the formula bar. Excel defines the formula for a number to be the number itself. Now click B32—you’ll see that the function AVERAGE(B2:B30) appears. In other words, when the contents of a cell has been computed by a formula, the value of the computation appears in the cell but the formula that computed that value appears in the formula bar. 9. To change the value in B30 you just type a new value into B30 (or into the formula bar when B30 is selected). Go ahead—click B30, type 1 but don’t press the ENTER key on your keyboard. Now, as you press ENTER, watch cell B32 and note that the value changes. Click on B32 and look in the value box. The formula for computing the value hasn’t changed even though the value has. When you change the data that a function has used to compute a value, Excel automatically recomputes that value with the new data. 10. We suggest you repeat the mean calculation for goal X for Becca and for Celeste. When you’ve finished, compare your sheet to Answers 7-1 in the Analysis Workbook.

From Goals to Data and Back Again

C-23 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 7-2: Computing multiple means on a single sheet 1. Open Exercise 7-2 in the Analysis Workbook. 2. Click on B32 (the cell containing the mean for Anton’s data). Note that the formula for the average of the data in column B appears in the formula bar. 3. Once you have computed one average on a data sheet there is a shortcut for computing additional averages. Copy B32; this copies the formula not the value. 4. Paste the formula into C32, thereby computing the average for Becca’s data. Look at the value of C32 in the formula bar—it contains a formula that computes the average over column C. In addition to automatically adjusting the data range for you, Excel also automatically displays the mean with just two digits following the decimal point. In other words, Excel copies the format of the data as well as the formula for the data. 5. Paste the formula again, this time into D32 (as long as there is still a striped box around B32 it can still be pasted. If the box has disappeared recopy either B32 or C32). Once again, Excel automatically changes the data range to take the average of Celeste’s data. 6. Delete the contents of C32 and D32. Now select both C32 and D32 at the same time. Paste the formula you copied—both means are computed automatically, each for the correct column of data. This is an extremely useful feature of Excel—a single copy and paste is much more efficient than stepping through a function’s dialog boxes over and over again. 7. Compare your sheet to Answers 7-2 in the Analysis Workbook. Note that although all the means have the same value (2.55), each mean has been computed over a different data set.

From Goals to Data and Back Again

C-24 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 7-3: Computing “before” and “after” means 1. Open Exercise 7-1 in the Analysis Workbook. 2. Copy the data for goal X for Anton, Becca and Celeste (columns A-D and rows 1-30, not the means you calculated). Paste the copy into the Exercise 7-3 worksheet (which is blank). 3. To lose the minimum amount of information we will divide the data for each child into (approximate) thirds of 10 values, 9 values, and 10 values. Highlight the middle 9 rows (rows 12 to 20) then choose Delete from the Edit menu. Now highlight rows 12 and 13 and choose Insert from the Edit menu to add two blank lines between the “before” and “after” data sets. Label the row at the bottom of each data set (cell A12 and cell A24) to indicate that the row contains means over the appropriate dates. You may want to check your progress against the Answers 7-3 worksheet at this point. 4. Using the average function as in Exercise 7-1, above, compute the “before” mean for Anton. 5. Using the same trick you practiced in Exercise 7-2, copy and paste the formula that computes Anton’s mean in order to compute the “before” means for Becca and Celeste. 6. Now select the three cells that correspond to the children’s “after” means and paste the same formula. Click on each mean and check the data range in the formula bar. Even with the other mean and the blank line in the way Excel got the data ranges right. Pretty neat, huh? (Unfortunately Excel doesn’t always do what you intend it to, so it is a very good idea to check the data ranges in at least one of the cells whenever you cut and paste a formula). 7. Compare your work to Answers 7-3.

From Goals to Data and Back Again

C-25 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 7-4: Producing a bar graph from data for a single goal 1. Open Exercise 7-4 in the Analysis Workbook. 2. Select the graphing icon from the toolbar. Choose Column for the Chart type (not Bar) and the simplest of the graphs (upper left corner) from the Chart subtype options. Click Next. 3. Now you must specify the data range. Unlike the previous exercises, this time the data is in rows rather than columns. In the Data Range view, click Rows for Series In. Click in the Data range box. Select the cell containing the label for the “before” mean and the cell with Anton’s mean in it. Excel will put a striped box around your selection and draw the first bar whose height equals the mean of the “before” data. Now select the cell containing the label of the “after” mean and the cell with the value of Anton’s “after” mean in it. A total of 4 cells should be selected. If this is not true, delete the text in the Data range box, check “Excel Basics” for how to select non-adjacent cells and try again. When you have only those four cells selected, click Next. 4. Label the graph “Comparison of Means, Anton, Goal X.” Label the x-axis “Before After” and the y-axis with our usual reminder of the scale for this goal. Click Next. Place the graph in the current worksheet. 5. Use the shortcuts you practiced in Exercise 6-2 to eliminate the labels on the x-axis and clean up the titles. Compare your graph to the top graph in Answers 74. 6. For additional practice, create the Comparison of Means graphs for Becca and Celeste. Don’t worry if Excel doesn’t show you an initial graph in the Data Range dialog box—just click Next and make sure you got the right 4 cells by looking at the graph in the Chart options dialog box (go Back if you didn’t). Check your graphs against our versions in Answers 7-4.

From Goals to Data and Back Again

C-26 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 7-5: Producing a bar graph with multiple goals for one child 1. Open Exercise 7-5 in the Analysis Workbook. 2. Remember that it is legitimate to combine goals in a single graph only if the data was taken during the same time interval and the goals used the same type of scale (all 4-point, for example). 3. Choose the simple form of the Column bar graph. 4. Specify the Data Range as you did in Exercise 7-4, this time using Anton’s data for goals X and Y. 5. Label the data and click Finish, skipping the last dialog box. Excel defaults to putting the graph in the current worksheet. 6. Compare your graph to the one on Answers 7-5.

From Goals to Data and Back Again

C-27 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 8-1: Computing the standard deviation 1. Open Exercise 8-1 in the Analysis Workbook. 2. We’ll add the standard deviation for each data set right below the mean. Label each row appropriately. You may want to make this row boldface as you did for the mean. You may also want to insert a blank row between the “before” and “after” sets to keep things easy to read. 3. Click on the cell for the standard deviation for Anton’s data for goal X. In the tool bar, click on the symbol fx. Choose Statistical from the Function Category menu then scroll down the Function Name menu until you can select STDEV. 4. In the Data Range dialog, select Anton’s “before” data for goal X. Or, if you prefer, you may simply type the range—B2:B11—into the box labeled Number 1. Whatever technique you use to specify the range be careful not to include the mean value as part of the data. Click OK and then round the value to 2 decimal places. 5. Compute the standard deviation for the other data sets on this sheet using the same copy-and-paste technique you learned in Exercise 7-2. 6. Compare your sheet to Answers 8-1.

From Goals to Data and Back Again

C-28 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 8-2: Adding standard deviation information to a bar graph 1. Open Exercise 8-2 in the Analysis Workbook where you will see data and bar graphs for Anton, Becca, and Celeste. If you are not sure how to produce the bar graphs, review Exercise 7-4. 2. We add standard deviation information to each graph by putting error bars around each mean. In Anton’s graph, click on the bar representing the “before” mean. A black square will appear on the bar to show it has been selected (a box will also appear around the mean in the data). 3. Click Selected Data Series in the pull-down menu for Format. A window called Format Data Series will pop up. 4. Click on the tab labeled Y Error Bars then select Both from the Display. 5. Click the circle next to Custom at the bottom of the list of Error Amounts. 6. Now you have a choice of how to fill in the + and – values for the error. You can click on each data box in turn and type in the standard deviation for Anton’s “before” data (0.52) or you can click on each data box in turn and then click on the cell containing the standard deviation in the spreadsheet. Either works for our purposes, but the latter method causes the graph to be updated automatically if Anton’s data changes. 7. When you click OK the Format Data Series will disappear and the error bar will appear on the graph. Now click on the error bar in the graph then pull down the Format menu again. Note that Excel now offers the option of reformatting the Selected Error Bars. Another way of saying this is that Excel keeps track of the context created by your selections and offers you options that depend on that context. 8. Repeat steps 2 through 7 for the bar representing the “after” mean for Anton’s data. Be sure to use the “after” value of the standard deviation in step 6. 9. If you want more practice repeat this exercise for Becca and Celeste’s graphs. When you are finished compare your work to Answers 8-2.

From Goals to Data and Back Again

C-29 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 8-3: Experimenting with raw data, mean, and standard deviation in the Statistics Lab 1. Open Exercise 8-3 in the Analysis workbook to enter the Statistics Lab. In it you’ll see data for a single child, separated into “before” and “after” subsets and graphed in ways that should be familiar to you. Since you will be changing the data, you may want to make a copy of this sheet to work with (that way you won’t have to go to the CD-ROM if you want to get back to this state). Use Move or Copy Sheet from the Edit menu, making sure to click Create a copy in the dialog box. 2. Notice the values in cells B2 through B11—the “before” data is perfectly regular. Its mean is a perfect representation of the observations so the standard deviation is 0. Because of these values, you cannot see an error bar around the “before” mean on the graph showing the comparison of means. The values in column C are the repetition of the mean value required to draw the mean line in the “before” and “after” line graphs. 3. Change the value in cell B2 from 2 to 1. Be sure to press the ENTER key on your keyboard (or click another cell) so that the change takes effect and all the values that depend on B2 are updated. Notice that the mean and standard deviation for the “before” data set changes and that those changes are reflected automatically in the graphs. 4. Continue to replace 2’s in the “before” data set with 1’s and watch what happens to the mean and the standard deviation for the “before” data as well as what happens to the error bars on the comparison of means graph. Notice that even though the standard deviation has gotten bigger, there is no overlap in the error bars because the data sets remain distinct. 5. Change the “before” observations back to all 2’s. Now replace the 2’s with 3’s one at a time. Again, watch how the mean and standard deviation change for the “before” data set as well as how the error bars overlap. Notice that this time, as the “before” and “after” sets have a larger percentage of their values in common, the means grow together until the difference in the means is first overshadowed by the overlap in the error bars and then falls below the category width of 1.00. 6. Start again with all 2’s in B2 to B11. Now replace the first five values one at a time with 1’s and then the last five values with 3’s. As always, watch the clustering around the mean, the standard deviation, the error bars, and the overlap in the “before” and “after” data sets. 7. Try other changes in the “before” and “after” data sets to see what sorts of patterns result. Change the values one at a time and see if you can predict what the graphs will look like before you press the ENTER key or click another cell. From Goals to Data and Back Again

C-30 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 9-1: Experimenting with outliers in the Statistics Lab 1. Open Exercise 9-1 in the Analysis workbook to re-enter the Statistics Lab. This sheet works just like the one you explored in Exercise 8-3 but begins in a different configuration and contains two data sets: one for a qualitative scale and one for percent data. There is one value missing in each of the “before” and “after” sets of observations. These are the values we’re going to explore. Notice that without the four missing values the initial configuration of both sets shows a pattern of change: means are more than a category width apart and there is no overlap in the error bars. 2. We’ll work with the qualitative scale first. Put the number 1 in cell B21, introducing a low outlier value in the “after” data (don’t forget to press ENTER). Notice that the mean changes very little but the standard deviation increases. Keeping your eye on the Comparison of Means graph, delete the outlier (B21) and watch what happens, then add it back in. Do this a few times until you see the pattern clearly. 3. Change B21 back to an empty cell. Now we’re going to look at what happens when the outlier is a high value in the “before” data. Put the number 4 in cell B8 (and press ENTER). Notice the way the “before” standard deviation increases and the error bars overlap. Keeping your eye on the Comparison of Means graph, delete the outlier (B8) and watch what happens. Do this a few times until you see the pattern clearly. 4. Scroll down the sheet to work with the percent data that starts in row 35. The wider range of possible values on this scale (0 to 100 percent) makes it easier to explore the remaining two cases. 5. Place a 5 in cell B41, introducing a low-valued outlier in the “before” data. This case sends a mixed message. Although the outlier pulls the “before” mean down, increasing the difference between the means, it also increases the “before” standard deviation. Delete the value in B41. 6. Place a 55 in cell B56, introducing a high-valued outlier in the “after” data. Again, the difference in the means grew, exaggerating progress. Even though the “after” standard deviation also grows, there is still no overlap in the error bars. Now try the values 60, 65, and 75 in B56. As the outlier’s value increases, the difference in the means increases as well but eventually the size of the “after” standard deviation grows too large to ignore.

From Goals to Data and Back Again

C-31 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 9-2: Computing correlations with the Pearson r In this exercise we want to know if the difference in teacher is correlated with the performance of any of the children in a group instruction environment. 1. Open Exercise 9-2 in the Analysis Workbook. 2. Insert a new column between columns B and C and translate the teacher factor into numeric form. If your numbers look odd, select the cells in the new column then use the Format pull-down menu for cells to specify that the data in the cells will be Numbers with 0 decimal places. Check your answer on the next sheet before continuing. 3. We will compute the correlation between teacher and each child in the “before” and “after” data separately. Insert rows into your spreadsheet to make room for the correlation values and label the rows appropriately. 4. Click on the cell where you want to place the value for the correlation between teacher and Anton’s performance for this goal. Now select the PEARSON function just as you have selected AVERAGE and STDEV in previous exercises. 5. In the Data Range dialog box, specify the teacher’s numeric “before” data as Array1 and Anton’s “before” data as Array2. Click OK and then round the value to 2 decimal places if necessary. 6. Copy the formula for Teacher/Anton and paste it into the cell for Teacher/Becca. Look up at the formula bar: this is one of those times we warned you about in Exercise 7-3: Excel did not do what you wanted it to (it increased the index of both ranges not just one). There are many ways to correct the Teacher/Becca correlation value. The simplest one is to just edit the first data range in the formula bar to compute over the values in column C. 7. Compute the remaining “before” correlations. Now select the three cells that contain the “before” correlations and paste them into the three cells that are meant to contain the “after” correlations. Note that, this time, Excel does exactly what we want. 8. Compare your sheet to Answers 9-2. Notice that we put only the values that were close to or greater than the boundary value of .65 into boldface. This makes it easy to ignore the meaningless values and concentrate our attention on the ones of interest. The only value of interest here is Becca’s in the “after” phase; her performance on this goal is better when Ms. B is teaching. Does this mean Ms. B is a better teacher? No, it just means that we can predict a difference in Becca’s performance depending upon who is the instructor. From Goals to Data and Back Again

C-32 Copyright © 2001-02 Lehman and Klaw

Appendix C: Exercises

Exercise 9-3: Understanding negative correlation 1. Open Exercise 9-3 in the Analysis Workbook. 2. Insert a column between the numeric values for the teacher factor and Anton’s data. In the new column reverse the assignment of numbers to teacher. In other words, assign the value of 1 to Mr. L and 2 to Ms. B. 3. Insert an extra row below the correlations for the “before” data. 4. Now calculate the correlation between the new values of the teacher factor and each child, “before” and “after.” 5. If you reversed the assignment of numbers without making any mistakes, the new Pearson r values will be the same as the Pearson r values you calculated in Exercise 9-2 but with the sign reversed. The value is the same because the degree of correlation hasn’t changed; the sign is reversed because the lower/higher value of the teacher factor now predicts higher/lower values for performance. Note that in the case of the “after” data for Becca—the only instance that reaches criterion—we recover the same conclusion we came to in Exercise 9-2: in the “after” phase of intervention, Becca performs better when Ms. B is teaching. 6. Compare your sheet to Answers 9-3.

From Goals to Data and Back Again

C-33 Copyright © 2001-02 Lehman and Klaw

Appendix B: Charts and Handouts

Directions for Using Excel A Very Quick Reference Guide Please note: in these instructions we use the words “pull down” or “click” for mouse actions and the word “press” for keyboard actions. Selecting To select a single cell: click on the cell you want to select. To select an entire column: click on the letter at the top of the column. To select an entire row: click on the number at the left of the row. To select a set of adjacent cells: click on the first cell. Then without letting your left-click up, move the mouse to the last cell. This is called “dragging.” As you drag over a cell, it will be highlighted. When you release your click after highlighting a set of cells, the entire highlighted area is selected. To select non-adjacent cells: select the first cell or range of cells and then, holding the CTRL key down, select each additional cell one at a time. Deleting To delete a value or values: select a cell or range of cells and press DELETE. To delete a row or column: select the row or rows, column or columns. Pull down the Edit menu and click ”Delete,” then “entire row” or “entire column.” To delete a sheet: pull down the Edit menu and click “Delete sheet.” Inserting To insert a row or column: select the box below or to the right of the area where you want to insert a new row or column. Pull down the Insert menu then click “Rows” or “Columns.” All of the columns or rows will be renumbered automatically. Inserting a worksheet: pull down the Insert menu, then click “Worksheet.”

From Goals to Data and Back Again

B-1

Copyright © 2001 Lehman and Klaw

Appendix B: Charts and Handouts

Changing or Modifying To change the width of a column: drag the boundary line on the right side of the column heading until the column is the width you want. To change the column width for multiple columns, select the columns you want to change. Then drag a boundary at the right of any selected column heading. To make the column width fit the contents, double-click the boundary to the right of the column heading. To change the height of a row: drag on the boundary line below the row heading until the row is the height you want. For multiple rows, select all the rows and then adjust the boundary to one of them. To make the row height fit the contents, double-click the boundary below the row heading. Copying and Pasting To copy a cell: select the cell, pull down the Edit menu and click “copy” (or use the copy icon on the tool bar). To copy a column: click on the column heading to select the whole column. Then pull down the Edit menu and click “copy” (or use the copy icon on the tool bar). To copy a row: click on the number of the row to select the whole row. Pull down the Edit menu then click “copy” (or use the copy icon tool bar). To copy an entire data sheet: pull down the Edit menu then click “Move or copy sheet.” Click where you want this sheet to be in relation to the list of other sheets in the workbook. Click “Create a copy,” then “OK.” To paste a cell, column, or row: Move the cursor to the desired location. Pull down the Edit menu then click “Paste” (or use the Paste icon on the toolbar). Miscellaneous To move a row or column: select the whole row or column. Now click on the highlighted area and drag it where you want it to go. To rename your sheet: double click on the tab containing the sheet name at the bottom of the screen (for example, Sheet 1) to select it. Now type the name you want. To save your data: pull down the File menu and click “Save.” To get help: pull down the Help menu and use the search mechanism available after clicking “Contents and Index.” From Goals to Data and Back Again

B-2

Copyright © 2001 Lehman and Klaw

Goal Workbook

Exercise 5-1: Writing goals for Joey Write 5 goals addressing communication needs for Joey. 1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-1

Copyright © 2001 Lehman and Klaw

Goal Workbook

Exercise 5-2: Making Joey’s goals measurable Qualify and quantify the following goals: •

Joey will indicate preference when offered the choice of two options.

•

Joey will associate photographs with preferred items.

•

Joey will attempt to say “Go!” after being cued with “Ready, set….”

•

Joey will begin to pull his parents to his object of desire.

•

Joey will imitate mouth movements with increasing frequency.

1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-2

Copyright © 2001 Lehman and Klaw

Goal Workbook

Exercise 5-3: Writing data questions for Joey’s goals Write data questions for the following goals: 1. During the same two hour period each day, Joey will indicate preference 75% of the time when offered the choice of food or an object with which to play or hold. 2. With increasing frequency, Joey will match photographs to preferred items when given the opportunity. 3. After hearing the phrase modeled twice, Joey will attempt to say “Go!” after hearing “Ready, set…” during a familiar and preferred activity. 4. With a decreasing need for assistance, Joey will begin to pull his parents and/or familiar adults to his object of desire. 5. Joey will imitate the mouth movements of familiar adults with increasing frequency.

Data questions

Circle, check or fill in information

1. 2. 3. 4. 5.

From Goals to Data and Back Again

C-3

Copyright © 2001 Lehman and Klaw

Goal Workbook

Exercise 5-4: Writing goals for Tyler Write five goals addressing Tyler’s communication needs at home. 1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-4

Copyright © 2001 Lehman and Klaw

Goal Workbook

Exercise 5-5: Making Tyler’s goals measurable Qualify and quantify the following goals: •

Incidents of private reference occurring in the midst of social exchange will decrease in frequency.

•

Tyler will learn to secure his family’s attention before talking.

•

Tyler will look at his communicative partner as he is talking or right before he begins to talk with increasing frequency.

•

Tyler will be engaged in a daily verbal exchange of increasing duration in which the adult partner avoids questioning and quizzing, and relies instead on commenting, narrating, rephrasing, and expressing opinions.

•

Tyler will learn to effectively and productively indicate disapproval or protest.

1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-5

Copyright © 2001 Lehman and Klaw

Goal Workbook

Exercise 5-6: Writing data questions for Tyler’s goals Write data questions for Tyler using the following goals: 1. During the one-hour period of time when the family plays together each evening, incidents of private reference occurring in the midst of social exchange will decrease in frequency. 2. During the one-hour period of time when the family plays together each evening, Tyler will learn to secure his family’s attention before talking at least half of the time. 3. Without needing to be reminded, Tyler will consistently look at his communicative partner as he is requesting food or drink. 4. With an increasing number of verbal exchanges, Tyler will have one conversation a day in which the adults use a facilitative approach (avoiding questioning and quizzing, and relying on commenting, narrating, rephrasing, and expressing opinions). 5. Without being coached from family members, Tyler will learn to effectively and productively indicate disapproval or protest.

Data questions

Circle, check or fill in information

1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-6

Copyright © 2001 Lehman and Klaw

Goal Workbook

Exercise 5-7 Writing measurable goals for Mai Lin Write 5 communication goals for Mai Lin in a school setting. As you write them, remember to qualify and quantify. 1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-7

Copyright © 2001 Lehman and Klaw

Goal Workbook

Exercise 5-8: Writing data questions for Mai Lin’s goals Write data questions for Mai Lin based on the following goals. 1. With a decreasing need for reminders, Mai Lin will appropriately face unfamiliar people when she addresses them. 2. Needing no more than a single prompt, Mai Lin will increasingly use appropriate volume and inflection when speaking to unfamiliar people. 3. With consistency, Mai Lin will effectively repair her communicative attempts after a peer says “What?” or indicates in any other way that Mai Lin’s words were not understood. 4. Once a day, Mai Lin will sustain general conversation for 5 minutes without attempting to suddenly introduce a preferred topic. 5. Needing only an expectant pause as a reminder, Mai Lin will make requests using a question format rather than a statement >75% of the time.

Data questions

Circle, check or fill in information

1.

2.

3.

4.

5.

From Goals to Data and Back Again

C-8

Copyright © 2001 Lehman and Klaw