Collecting data by Esmat niazi
[email protected] http://www.facebook.com/Esmat niazi +93786482395 +93799562925 All the best
The hard copy is in my hand bellow
Introduction Evaluation is a systematic determination of a subject's merit, worth and significance, using criteria governed by a set of standards. It can assist an organization, program, project or any other intervention or initiative to assess any aim, realizable concept/proposal, or any alternative, to help in decision-making; or to ascertain the degree of achievement or value in regard to the aim and objectives and results of any such action that has been completed. The primary purpose of evaluation, in addition to gaining insight into prior or existing initiatives, is to enable reflection and assist in the identification of future change. Evaluation is often used to characterize and appraise subjects of interest in a wide range of human enterprises, including the arts, criminal justice, foundations, non-profit organizations, government, health care, and other human services. A class evaluation is a paper or electronic questionnaire, which requires a written or selected response answer to a series of questions in order to evaluate the instruction of a given course. The term may also refer to the completed survey form or a summary of responses to questionnaires. They are a means to produce useful feedback which the teacher and school can use to improve their quality of instruction. The process of (a) gathering information about the impact of learning and of teaching practice on student learning, (b) analyzing and interpreting this information, and (c) responding to and acting on the results, is valuable for several reasons. They are beneficial because instructors can review how others interpret their teaching methods, thereby improving their instruction. The information can be also used by administrators, along with other input, to make summative decisions (e.g., decisions about promotion, tenure, salary increases, etc.) and make formative recommendations (e.g., identify areas where a faculty member needs to improve). Typically, these evaluations are combined with peer evaluations, supervisor evaluations, and results of student’s test scores to create an overall picture of teaching performance.
Course
evaluations
either summative or formative.
are
implemented
in
one
of
two
ways,
There are two common purposes in educational evaluation which are, at times, in conflict with one another. Educational institutions usually require evaluation data to demonstrate effectiveness to funders and other stakeholders, and to provide a measure of performance for marketing purposes. Educational evaluation is also a professional activity that individual educators need to undertake if they intend to continuously review and enhance the learning they are endeavoring to facilitate.
Definition Evaluation is the structured interpretation and giving of meaning to predict or actual impacts of proposals or results. It looks at original objectives, and at what is either predicted or what was accomplished and how it was accomplished. So evaluation can be formative that is taking place during the development of a concept or proposal, project or organization, with the intention of improving the value or effectiveness of the proposal, project, or organization. It can also be assumptive, drawing lessons from a completed action or project or an organization at a later point in time or circumstance. Evaluation is inherently a theoretically informed approach (whether explicitly or not), and consequently any particular definition of evaluation would have be tailored to its context – the theory, needs, purpose, and methodology of the evaluation process itself. Having said this, evaluation has been defined as:
A systematic, rigorous, and meticulous application of scientific methods to assess the design, implementation, improvement, or outcomes of a program. It is a resourceintensive process, frequently requiring resources, such as, evaluate expertise, labor, time, and a sizable budget
"The critical assessment, in as objective a manner as possible, of the degree to which a service or its component parts fulfills stated goals" (St Leger and WordsworthBell). The focus of this definition is on attaining objective knowledge, and scientifically or quantitatively measuring predetermined and external concepts.
"A study designed to assist some audience to assess an object's merit and worth" (Shuffleboard).[4] In this definition the focus is on facts as well as value laden judgments of the programs outcomes and worth.
Purpose The main purpose of a program evaluation can be to "determine the quality of a program by formulating a judgment" Marthe Hurteau, Sylvain Houle, Stéphanie Mongiat (2009). An alternative view is that "projects, evaluators, and other stakeholders (including funders) will all have potentially different ideas about how best to evaluate a project since each may have a different definition of 'merit'. The core of the problem is thus about defining what
is of value." From this perspective, evaluation "is a contested term", as "evaluators" use the term evaluation to describe an assessment, or investigation of a program whilst others simply understand evaluation as being synonymous with applied research. There are two function considering to the evaluation purpose Formative Evaluations provide the information on the improving a product or a process Summative Evaluations provide information of short-term effectiveness or long-term impact to deciding the adoption of a product or process. Not all evaluations serve the same purpose some evaluations serve a monitoring function rather than focusing solely on measurable program outcomes or evaluation findings and a full list of types of evaluations would be difficult to compile. This is because evaluation is not part of a unified theoretical framework, drawing on a number of disciplines, which include management and organizational theory, policy analysis, education, sociology, social anthropology, and social change.
Discussion Within the last three decades there have been tremendous theoretical and methodological developments within the field of evaluation.[5] Despite its progress, there are still many fundamental problems faced by this field as "unlike medicine, evaluation is not a discipline that has been developed by practicing professionals over thousands of years, so we are not yet at the stage where we have huge encyclopedias that will walk us through any evaluation step-by-step", or provide a clear definition of what evaluation entails (Davidson, 2005).[5] It could therefore be argued that a key problem that evaluators face is the lack of a clear definition of evaluation, which may "underline why program evaluation is periodically called into question as an original process, whose primary function is the production of legitimate
and
justified
judgments
which
serve
as
the
bases
for
relevant
recommendations."[5] However, the strict adherence to a set of methodological assumptions may make the field of evaluation more acceptable to a mainstream audience but this adherence will work towards preventing evaluators from developing new strategies for dealing with the myriad problems that programs face. Is claimed that only a minority of evaluation reports are used by the devalued (client) (Datta, 2006). One justification of this is that "when evaluation findings are challenged or
utilization has failed, it was because stakeholders and clients found the inferences weak or the warrants unconvincing" (Fournier and Smith, 1993). Some reasons for this situation may be the failure of the evaluator to establish a set of shared aims with the devalued, or creating overly ambitious aims, as well as failing to compromise and incorporate the cultural differences of individuals and programs within the evaluation aims and process. None of these problems are due to a lack of a definition of evaluation but are rather due to evaluators attempting to impose predisposed notions and definitions of evaluations on clients. The central reason for the poor utilization of evaluations is arguably due to the lack of tailoring of evaluations to suit the needs of the client, due to a predefined idea (or definition) of what an evaluation is rather than what the client needs are (House, 1980). The development of a standard methodology for evaluation will require arriving at applicable ways of asking and stating the results of questions about ethics such as agentprincipal, privacy, stakeholder definition, limited liability; and could-the-money-be-spentmore-wisely issues.
Standards Depending on the topic of interest, there are professional groups that review the quality and rigor of evaluation processes. Evaluating programs and projects, regarding their value and impact within the context they are implemented, can be ethically challenging. Evaluators may encounter complex, culturally specific systems resistant to external evaluation. Furthermore, the project organization or other stakeholders may be invested in a particular evaluation outcome. Finally, evaluators themselves may encounter "conflict of interest (COI)" issues, or experience interference or pressure to present findings that support a particular assessment. General professional codes of conduct, as determined by the employing organization, usually cover three broad aspects of behavioral standards, and include inter-collegial relations (such as respect for diversity and privacy), operational issues (due competence, documentation accuracy and appropriate use of resources), and conflicts of interest (nepotism, accepting gifts and other kinds of favoritism).
However, specific guidelines particular to the evaluator's role that can be utilized in the management of unique ethical challenges are required. The Joint Committee on Standards for Educational Evaluation has developed standards for program, personnel, and student evaluation. The Joint Committee standards are broken into four sections: Utility, Feasibility, Propriety, and Accuracy. Various European institutions have also prepared their own standards, more or less related to those produced by the Joint Committee. They provide guidelines about basing value judgments on systematic inquiry, evaluator competence and integrity, respect for people, and regard for the general and public welfare.
The American Evaluation Association has created a set of Guiding Principles for evaluators. The order of these principles does not imply priority among them; priority will vary by situation and evaluator role. The principles run as follows:
Systematic Inquiry: evaluators conduct systematic, data-based inquiries about whatever is being evaluated. This requires quality data collection, including a defensible choice of indicators, which lends credibility to findings.
Findings are credible when they are demonstrably evidence-based, reliable and valid. This also pertains to the choice of methodology employed, such that it is consistent with the aims of the evaluation and provides dependable data. Furthermore, utility of findings is critical such that the information obtained by evaluation is comprehensive and timely, and thus serves to provide maximal benefit and use to stakeholders.
Competence: evaluators provide competent performance to stakeholders. This requires that evaluation teams comprise an appropriate combination of competencies, such that varied and appropriate expertise is available for the evaluation process, and that evaluators work within their scope of capability.
Integrity/Honesty: evaluators ensure the honesty and integrity of the entire evaluation process. A key element of this principle is freedom from bias in evaluation and this is underscored by three principles: impartiality, independence, and transparency. Independence is attained through ensuring independence of judgment is upheld such
that evaluation conclusions are not influenced or pressured by another party, and avoidance of
conflict of interest, such that the evaluator does not have a stake in a particular conclusion. Conflict of interest is at issue particularly where funding of evaluations is provided by particular bodies with a stake in conclusions of the evaluation, and this is seen as potentially compromising the independence of the evaluator. Whilst it is acknowledged that evaluators may be familiar with agencies or projects that they are required to evaluate, independence requires that they not have been involved in the planning or implementation of the project. A declaration of interest should be made where any benefits or association with project are stated. Independence of judgment is required to be maintained against any pressures brought to bear on evaluators, for example, by project funders wishing to modify evaluations such that the project appears more effective than.
Findings can verify Impartiality pertains to findings being a fair and thorough assessment of strengths and weaknesses of a project or program. This requires taking due input from all stakeholders involved and findings presented without bias and with a transparent, proportionate, and persuasive link between findings and recommendations. Thus evaluators are required to delimit their findings to evidence. A mechanism to ensure impartiality is external and internal review. Such review is required of significant (determined in terms of cost or sensitivity) evaluations. The review is based on quality of work and the degree to which a demonstrable link is provided between findings. Transparency requires that stakeholders are aware of the reason for the evaluation, the criteria by which evaluation occurs and the purposes to which the findings will be applied. Access to the evaluation document should be facilitated through findings being easily readable, with clear explanations of evaluation methodologies, approaches, sources of information, and costs.
Incurred
Respect for People: Evaluators respect the security, dignity and self-worth of the respondents, program participants, clients, and other stakeholders with whom they interact. This is particularly pertinent with regards to those who will be impacted upon by the evaluation findings. Protection of people includes ensuring informed consent from those involved in the evaluation, upholding confidentiality, and ensuring that the identity of those who may provide sensitive information towards the program evaluation is protected. Evaluators are ethically required to respect the customs and beliefs of those who are impacted upon by the evaluation or program activities. Examples of how such respect is demonstrated is through respecting local customs e.g. dress codes, respecting people’s privacy, and minimizing demands on others' time. Where stakeholders wish to place objections to evaluation findings, such a process should be facilitated through the local office of the evaluation organization, and procedures for lodging complaints or queries should be accessible and clear.
Responsibilities for General and Public Welfare: Evaluators articulate and take into account the diversity of interests and values that may be related to the general and public welfare. Access to evaluation documents by the wider public should be facilitated such that discussion and feedback is enabled. Furthermore, the international organizations such as the I.M.F. and the World Bank
have independent evaluation functions. The various funds, programmers, and agencies of the United Nations has a mix of independent, semi-independent and self-evaluation functions, which have organized themselves as a system-wide UN Evaluation Group (UNEG), that works together to strengthen the function, and to establish UN norms and standards for evaluation. There is also an evaluation group within the OECD-DAC, which endeavors to improve development evaluation standards.[14] The independent evaluation units of the major multinational development banks (MDBs) have also created the Evaluation Cooperation Group to strengthen the use of evaluation for greater MDB effectiveness and accountability, share lessons from MDB evaluations, and promote evaluation harmonization and collaboration.
Approaches Evaluation approaches are conceptually distinct ways of thinking about, designing, and conducting evaluation efforts. Many of the evaluation approaches in use today make truly unique contributions to solving important problems, while others refine existing approaches in some way.
Classification of approaches Two classifications of evaluation approaches by House and Stufflebeam and Webster can be combined into a manageable number of approaches in terms of their unique and important underlying principles. House
considers
all
major
evaluation
approaches
to
be
based
on
a
common ideology entitled liberal democracy. Important principles of this ideology include freedom of choice, the uniqueness of the individual and empirical inquiry grounded in objectivity. He also contends that they are all based on subjectivist ethics, in which ethical conduct is based on the subjective or intuitive experience of an individual or group. One form of subjectivist ethics is utilitarian, in which "the good" is determined by what maximizes a single, explicit interpretation of happiness for society as a whole. Another form of subjectivist ethics is intuitionist/pluralist, in which no single interpretation of "the good" is assumed and such interpretations need not be explicitly stated nor justified. These ethical positions have corresponding epistemologies—philosophies for obtaining knowledge. The objectivist epistemology is associated with the utilitarian ethic; in general, it is used to acquire knowledge that can be externally verified (inter subjective agreement) through publicly exposed methods and data. The subjectivist epistemology is associated with the intuitionist/pluralist ethic and is used to acquire new knowledge based on existing personal knowledge, as well as experiences that are (explicit) or are not (tacit) available for public inspection. House then divides each epistemological approach into two main political perspectives. Firstly, approaches can take an elite perspective, focusing on the interests of
managers and professionals; or they also can take a mass perspective, focusing on consumers and participatory approaches. Stufflebeam and Webster place approaches into one of three groups, according to their orientation toward the role of values and ethical consideration. The political orientation promotes a positive or negative view of an object regardless of what its value actually is and might be—they call this pseudo-evaluation. The questions orientation includes approaches that might or might not provide answers specifically related to the value of an object—they call this quasi-evaluation. The values orientation includes approaches primarily intended to determine the value of an object—they call this true evaluation. When the above concepts are considered simultaneously, fifteen evaluation approaches can be identified in terms of epistemology, major perspective (from House), and orientation. Two pseudo-evaluation approaches, politically controlled and public relations studies, are represented. They are based on an objectivist epistemology from an elite perspective. Six quasi-evaluation approaches use an objectivist epistemology. Five of them—experimental research, management information systems, testing programs, objectives-based studies, and content analysis—take an elite perspective. Accountability takes a mass perspective. Seven true evaluation approaches are included. Two approaches, decision-oriented and policy studies, are based on an objectivist epistemology from an elite perspective. Consumer-oriented studies are based on an objectivist epistemology from a mass perspective. Two approaches—accreditation/certification and connoisseur studies—are based on a subjectivist epistemology from an elite perspective. Finally, adversary and centered studies are based on a subjectivist epistemology from a mass perspective.
Summary of approaches The following table is used to summarize each approach in terms of four attributes— organizer, purpose, strengths, and weaknesses. The organizer represents the main considerations or cues practitioners use to organize a study. The purpose represents the desired outcome for a study at a very general level. Strengths and weaknesses represent other
attributes that should be considered when deciding whether to use the approach for a particular study. The following narrative highlights differences between approaches grouped together.
Educational evaluation Educational evaluation is the evaluation process of characterizing and appraising some aspect/s of an educational process. There are two common purposes in educational evaluation which are, at times, in conflict with one another. Educational institutions usually require evaluation data to demonstrate effectiveness to funders and other stakeholders, and to provide a measure of performance for marketing purposes. Educational evaluation is also a professional activity that individual educators need to undertake if they intend to continuously review and enhance the learning they are endeavoring to facilitate.
Standards for educational evaluation The Joint Committee on Standards for Educational Evaluation published three sets of standards for educational evaluations. The Personnel Evaluation Standards was published in 1988, The Program Evaluation Standards (2nd edition) was published in 1994, and The Student Evaluations Standards was published in 2003. Each publication presents and elaborates a set of standards for use in a variety of educational settings. The standards provide guidelines for designing, implementing, assessing and improving the identified form of evaluation. Each of the standards has been placed in one of four fundamental categories to promote evaluations that are proper, useful, feasible, and accurate.
The Personnel Evaluation Standards
The propriety standards require that evaluations be conducted legally, ethically, and with due regard for the welfare of evaluates and clients involved in.
The utility standards are intended to guide evaluations so that they will be informative, timely, and influential.
The feasibility standards call for evaluation systems that are as easy to implement as possible, efficient in their use of time and resources, adequately funded, and viable from a number of other standpoints.
The accuracy standards require that the obtained information be technically accurate and that conclusions be linked logically to the data.
The Program Evaluation Standards
The utility standards are intended to ensure that an evaluation will serve the information needs of intended users.
The feasibility standards are intended to ensure that an evaluation will be realistic, prudent, diplomatic, and frugal.
The propriety standards are intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results.
The accuracy standards are intended to ensure that an evaluation will reveal and convey technically adequate information about the features that determine worth or merit of the program being evaluated.
The Student Evaluation Standards
The Propriety standards help and ensure that student evaluations are conducted lawfully, ethically, and with regard to the rights of students and other persons affected by student evaluation.
The Utility standards promote the design and implementation of informative, timely, and useful student evaluations.
The Feasibility standards help ensure that student evaluations are practical; viable; costeffective; and culturally, socially, and politically appropriate.
The Accuracy standards help ensure that student evaluations will provide sound, accurate, and credible information about student learning and performance.
Criticism of educational evaluation Evaluation in a democratic school Sudbury model of democratic education schools do not perform and do not offer evaluations, assessments, transcripts, or recommendations, asserting that they do not rate people, and that school is not a judge; comparing students to each other, or to some standard that has been set is for them a violation of the student's right to privacy and to self-
determination. Students decide for themselves how to measure their progress as self-starting learners as a process of self-evaluation: real lifelong learning and the proper educational evaluation for the 21st Century, they adduce. According to Sudbury schools (Riaz Institute of education and research.)..., this policy does not cause harm to their students as they move on to life outside the school. However, they admit it makes the process more difficult, but that such hardship is part of the students learning to make their own way, set their own standards and meet their own goals. The no-grading and no-rating policy helps to create an atmosphere free of competition among students or battles for adult approval, and encourages a positive co-operative environment amongst the student body. The final stage of a Sudbury education, should the student choose to take it, is the graduation thesis. Each student writes on the topic of how they have prepared themselves for adulthood and entering the community at large. This thesis is submitted to the Assembly, who reviews it. The final stage of the thesis process is an oral defense given by the student in which they open the floor for questions, challenges and comments from all Assembly members. At the end, the Assembly votes by secret ballot on whether or not to award a diploma.
Pseudo-evaluation Politically controlled and public relations studies are based on an objectivist epistemology from an elite perspective. Although both of these approaches seek to misrepresent value interpretations about an object, they function differently from each other. Information obtained through politically controlled studies is released or withheld to meet the special interests of the holder, whereas public relations information creates a positive image of an object regardless of the actual situation. Despite the application of both studies in real scenarios, neither of these approaches is acceptable evaluation practice.
Objectivist, elite, quasi-evaluation A group, these five approaches represent a highly respected collection of disciplined inquiry approaches. They are considered quasi-evaluation approaches because particular studies legitimately can focus only on questions of knowledge without addressing any questions of value. Such studies are, by definition, not evaluations. These approaches can
produce characterizations without producing appraisals, although specific studies can produce both. Each of these approaches serves its intended purpose well. They are discussed roughly in order of the extent to which they approach the objectivist ideal.
Experimental research is the best approach for determining causal relationships between variables. The potential problem with using this as an evaluation approach is that its highly controlled and stylized methodology may not be sufficiently responsive to the dynamically changing needs of most human service programs.
Management information systems (MISs) can give detailed information about the dynamic operations of complex programs. However, this information is restricted to readily quantifiable data usually available at regular intervals.
Testing programs are familiar to just about anyone who has attended school, served in the military, or worked for a large company. These programs are good at comparing individuals or groups to selected norms in a number of subject areas or to a set of standards of performance. However, they only focus on tested performance and they might not adequately sample what is taught or expected.
Objectives-based approaches relate outcomes to pre specified objectives, allowing judgments to be made about their level of attainment. Unfortunately, the objectives are often not proven to be important or they focus on outcomes too narrow to provide the basis for determining the value of an object.
Content analysis is a quasi-evaluation approach because content analysis judgments need not be based on value statements. Instead, they can be based on knowledge. Such content analyses are not evaluations. On the other hand, when content analysis judgments are based on values, such studies are evaluations.
Objectivist, mass, quasi-evaluation Accountability is popular with constituents because it is intended to provide an accurate accounting of results that can improve the quality of products and services. However, this approach quickly can turn practitioners and consumers into adversaries when implemented in a heavy-handed fashion.
Objectivist, elite, true evaluation
Decision-oriented studies are designed to provide a knowledge base for making and defending decisions. This approach usually requires the close collaboration between an evaluator and decision-maker, allowing it to be susceptible to corruption and bias.
Policy studies provide general guidance and direction on broad issues by identifying and assessing potential costs and benefits of competing policies. The drawback is these studies can be corrupted or subverted by the politically motivated actions of the participants.
Objectivist, mass, true evaluation
Consumer-oriented studies are used to judge the relative merits of goods and services based on generalized needs and values, along with a comprehensive range of effects. However, this approach does not necessarily help practitioners improve their work, and it requires a very good and credible evaluator to do it well.
Subjectivist, elite, true evaluation
Accreditation / certification programs are based on self-study and peer review of organizations, programs, and personnel. They draw on the insights, experience, and expertise of qualified individuals who use established guidelines to determine if the applicant should be approved to perform specified functions. However, unless performance-based standards are used, attributes of applicants and the processes they perform often are overemphasized in relation to measures of outcomes or effects.
Connoisseur studies use the highly refined skills of individuals intimately familiar with the subject of the evaluation to critically characterize and appraise it. This approach can help others see programs in a new light, but it is difficult to find a qualified and unbiased connoisseur.
Subjectivist, mass, true evaluation
The adversary approach focuses on drawing out the pros and cons of controversial issues through quasi-legal proceedings. This helps ensure a balanced presentation of different perspectives on the issues, but it is also likely to discourage
later cooperation and heighten animosities between contesting parties if "winners" and "losers" emerge.
Client-centered
Client-centered studies address specific concerns and issues of practitioners and other clients of the study in a particular setting. These studies help people understand the activities and values involved from a variety of perspectives. However, this responsive approach can lead to low external credibility and a favorable bias toward those who participated in the study.
Methods and techniques Evaluation is methodologically diverse. Methods may be qualitative or quantitative, and include case studies, survey research, statistical analysis, model building, and many more such as:
Accelerated aging Action research Advanced product quality planning Alternative assessment Appreciative Inquiry Assessment Axiomatic design Benchmarking Case study Change management Clinical trial Cohort study Competitor analysis Consensus decision-making Consensus-seeking decision-making Content analysis
Joint Committee on Standards for Educational Evaluation
The
Joint
Committee
on
Standards
for
Educational
Evaluation
is
an American/Canadian based Standards Developer Organization (SDO). The Joint Committee represents a coalition of major professional associations formed in 1975 to help improve the quality of standardized evaluation. The Committee has thus far published three sets of standards for evaluations. The Personnel Evaluation Standards (2nd edition) was published in 1988 and updated in 2008, The Program Evaluation Standards (2nd edition) was published in 1994 (the third edition of which is in draft form as of 2008), and The Student Evaluation Standards was published in 2003. The Joint Committee is a private nonprofit organization. It is accredited by the
American
National
Standards
Institute
(ANSI).
Standards
approved
by ANSI become American National Standards. In addition to setting standards in evaluation, it also is involved in reviewing and updating its published standards (every five years); training policymakers, evaluators, and educators in the use of the standards; and serving as a clearinghouse on evaluation standards literature. Each publication presents and elaborates a set of standards for use in a variety of educational settings. The standards provide guidelines for designing, implementing, assessing and improving the identified form of evaluation. Each of the standards has been placed in one of four fundamental categories to promote educational evaluations that are proper, useful, feasible, and accurate.
The Personnel Evaluation Standards The second edition of the Personnel Evaluation Standards (2008) is based on knowledge about personnel evaluation gained from the professional literature and research/development since 1988. In this edition, six new standards were added to the original 21 of the first edition. The Joint Committee on Standards for Educational Evaluation requires that personnel evaluations be ethical, fair, useful, feasible, and accurate. The standards also provide special consideration to issues of diversity.
It is not the intent of these standards to design or promote specific systems of evaluation, rather to ensure that whatever system is in place provides a sound process most likely to produce the desired results. The four attributes of sound educational evaluation practices are:
The propriety standards require that evaluations be conducted legally, ethically, and with due regard for the welfare of evaluates and clients involved in. There are seven standards under this attribute which include service orientation, appropriate policies and procedures, access to evaluation information, interactions with evaluates, comprehensive evaluation, conflict of interest, and legal viability.
The utility standards are intended to guide evaluations so that they will be informative, timely, and influential. There are six standards under this attribute which include constructive orientation, defined uses, evaluator qualifications, explicit criteria, functional reporting, and follow-up/professional development.
The feasibility standards call for evaluation systems that are as easy to implement as possible, efficient in their use of time and resources, adequately funded, and viable from a number of other standpoints. There are three standards under this attribute including practical procedures, political viability, and fiscal viability.
The accuracy standards require that the obtained information be technically accurate and that conclusions be linked logically to the data. There are eleven standards under this attribute including validity orientation, defined expectations, analysis of context, documented purposes and procedures, defensible information, systemic data control, bias identification and management, analysis of information, justified conclusions, and met evaluation.
The Program Evaluation Standards The utility standards are intended to ensure that an evaluation will serve the information needs of intended users. The utility standards for program evaluation incorporate the following: Stakeholder Identification: the people involved in the evaluation and those who will be affected by the evaluation must be identified so that their needs can be addressed.
Evaluator Credibility: the people conducting the evaluation must be trustworthy and competent to perform the evaluation in order for the evaluation's findings to achieve maximum credibility and acceptance.
Information Scope and Selection: the collected information must be broadly selected so that it addresses pertinent questions about the program and is able to be responsive to the needs and interests of clients and other specified stakeholders.
Values Identification: the perspectives, procedures and rationale used to interpret the findings of the evaluation should be carefully described so that the bases for value judgments are clear.
Report Clarity: an evaluation report must precisely describe the program being evaluated, including its context, purposes, procedures and findings so that the essential information is provided and easy to understand.
Report Timelines and Dissemination: evaluation reports and any significant interim findings should be disseminated to intended users so that they may be used in a timely fashion.
Evaluation Impact: the way an evaluation is planned, conducted and reported should encourage follow-through by stakeholders in order to increase the likelihood that the evaluation will be used.
The feasibility standards are intended to ensure that an evaluation will be realistic, prudent, diplomatic, and frugal. The feasibility standards for program evaluation incorporate the following.
Practical Procedures: evaluation procedures should be practical in order to keep disruption to a minimum while relevant and needed information is obtained.
Political Viability: whilst planning and conducting the evaluation one must anticipate the different positions of various interest groups so that their cooperation may be obtained. This will also allow one to avert or counteract any possible attempts by these groups to obstruct evaluation operations or to bias or misapply the evaluation's results.
Cost Effectiveness: a good evaluation should be efficient and produce information of sufficient value to justify the use of available resources.
The propriety standards are intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results. The propriety standards for program evaluation incorporate the following:
Service Orientation
Formal Agreement
Rights of Human Subjects
Human Interactions
Complete and Fair Assessment
Disclosure of Findings
Conflict of Interest
Fiscal Responsibility
The accuracy standards are intended to ensure that an evaluation will reveal and convey technically adequate information about the features that determine worth or merit of the program being evaluated. The accuracy standards for program evaluation incorporate the following:
Program Documentation
Context Analysis
Described Purposes and Procedures
Defensible Information Sources
Valid Information
Reliable Information
Systematic Information
Analysis of Quantitative Information
Analysis of Qualitative Information
Justified Conclusions
Impartial Reporting
Meta-Evaluation
The Student Evaluation Standards
The Propriety standards help ensure that student evaluations are conducted law fully, ethically, and with regard to the rights of students and other persons affected by student evaluation.
The Utility standards promote the design and implementation of informative, timely, and useful student evaluations.
The Feasibility standards help ensure that student evaluations are practical; viable; costeffective; and culturally, socially, and politically appropriate.
The Accuracy standards help ensure that student evaluations will provide sound, accurate, and credible information about student learning and performance.
Questionnaire A questionnaire is a research instrument consisting of a series of questions and other prompts for the purpose of gathering information from respondents. Although they are often designed for statistical analysis of the responses, this is not always the case. The questionnaire was invented by Sir Francis Galton. Questionnaires have advantages over some other types of surveys in that they are cheap, do not require as much effort from the questioner as verbal or telephone surveys, and often have standardized answers that make it simple to compile data. However, such standardized answers may frustrate users. Questionnaires are also sharply limited by the fact that respondents must be able to read the questions and respond to them. Thus, for some demographic groups conducting a survey by questionnaire may not be concrete.
Types A distinction can be made between questionnaires with questions that measure separate variables, and questionnaires with questions that are aggregated into either a scale or index. Questionnaires within the former category are commonly part of surveys, whereas questionnaires in the latter category are commonly part of tests. Questionnaires with questions that measure separate variables, could for instance include questions on:
preferences (e.g. political party)
behaviors (e.g. food consumption)
facts (e.g. gender)
Questionnaires with questions that are aggregated into either a scale or index, include for instance questions that measure:
latent traits (e.g. personality traits such as extroversion)
attitudes (e.g. towards immigration)
an index (e.g. Social Economic Status)
Questionnaire construction Question types Usually, a questionnaire consists of a number of questions that the respondent has to answer in a set format. A distinction is made between open-ended and closed-ended questions. An open-ended question asks the respondent to formulate his own answer, whereas a closedended question has the respondent pick an answer from a given number of options. The response options for a closed-ended question should be exhaustive and mutually exclusive. Four types of response scales for closed-ended questions are distinguished:
Dichotomous, where the respondent has two options
Nominal-polychromous, where the respondent has more than two unordered options
Ordinal-polychromous, where the respondent has more than two ordered options
(Bounded)Continuous, where the respondent is presented with a continuous scale.
Question sequence: In general, questions should flow logically from one to the next. To achieve the best response rates, questions should flow from the least sensitive to the most sensitive, from the factual and behavioral to the attitudinal, and from the more general to the more specific. There typically is a flow that should be followed when constructing a questionnaire in regards to the order that the questions are asked. The order is as follows: 1. Screens 2. Warm-ups
3. Transitions 4. Skips 5. Difficult 6. Changing Formula Screens are used as a screening method to find out early whether or not someone should complete the questionnaire. Warm-ups are simple to answer, help capture interest in the survey, and may not even pertain to research objectives. Transition questions are used to make different areas flow well together. Skips include questions similar to "If yes, then answer question 3. If no, then continue to question 5." Difficult questions are towards the end because the respondent is in "response mode." Also, when completing an online questionnaire, the progress bars lets the respondent know that they are almost done so they are more willing to answer more difficult questions. Classification, or demographic question should be at the end because typically they can feel like personal questions which will make respondents uncomfortable and not willing to finish survey.
Basic rules for questionnaire item construction
Use statements which are interpreted in the same way by members of different subpopulations of the population of interest.
Use statements where persons that have different opinions or traits will give different answers.
Think of having an "open" answer category after a list of possible answers.
Use only one aspect of the construct you are interested in per item.
Use positive statements and avoid negatives or double negatives.
Do not make assumptions about the respondent.
Use clear and comprehensible wording, easily understandable for all educational levels
Use correct spelling, grammar and punctuation.
Avoid items that contain more than one question per item (e.g. Do you like strawberries and potatoes?).
Question should not be biased or even leading the participant towards an answer.
Questionnaire administration modes Main modes of questionnaire administration are:
Face-to-face questionnaire administration, where an interviewer presents the items orally.
Paper-and-pencil questionnaire administration, where the items are presented on paper.
Computerized questionnaire administration, where the items are presented on the computer.
Adaptive computerized questionnaire administration, where a selection of items is presented on the computer, and based on the answers on those items, the computer selects following items optimized for the taste’s estimated ability or trait.
Concerns with questionnaires While questionnaires are inexpensive, quick, and easy to analyze, often the questionnaire can have more problems than benefits. For example, unlike interviews, the people conducting the research may never know if the respondent understood the question that was being asked. Also, because the questions are so specific to what the researchers are asking, the information gained can be minimal. Often, questionnaires such as the Myers-Briggs Type Indicator, give too few options to answer; respondents can answer either option but must choose only one response. Questionnaires also produce very low return rates, whether they are mail or online questionnaires. The other problem associated with return rates is that often the people that do return the questionnaire are those that have a really positive or a really negative viewpoint and want their opinion heard. The people that are most likely unbiased either way typically don't respond because it is not worth their time. Some questionnaires have questions addressing the participant’s gender. Seeing someone as male or female is something we all do unconsciously, we don’t give much important to one’s sex or gender as most people use the terms ‘sex’ and ‘gender’ interchangeably, unaware that they are not synonyms. Gender is a term to exemplify the attributes that a society or culture constitutes as masculine or feminine. Although your sex as male or female stands at a biological fact that is identical in any culture, what that specific sex
means in reference to your gender role as a ‘woman’ or ‘man’ in society varies cross culturally according to what things are considered to be masculine or feminine. The survey question should really be what your sex is. Sex is traditionally split into two categories, which we typically don’t have control over, you were either born a girl or born a boy and that’s decided by nature. There's also the intersex population which is disregarded in the North American society as a sex. Not many questionnaires have a box for people that fall under Intersex. These are some small things that can be misinterpreted or ignored in questionnaires.
Methods of instruction Explaining, or lecturing, is the process of teaching by giving spoken explanations of the subject that is to be learned. Lecturing is often accompanied by visual aids to help students visualize an object or problem. Explaining may meet the needs of auditory or visual learning preferences . But often fails to meet the needs of individuals with other learning preferences , such as kinesthetic or social learners.
Demonstrating Demonstrating is the process of teaching through examples or experiments. For example, a science teacher may teach an idea by performing an experiment for students. A demonstration may be used to prove a fact through a combination of visual evidence and associated reasoning. Demonstrations are similar to written storytelling and examples in that they allow students to personally relate to the presented information. Memorization of a list of facts is a detached and impersonal experience, whereas the same information, conveyed through demonstration, becomes personally relatable. Demonstrations help to raise student interest and reinforce memory retention because they provide connections between facts and real-world applications of those facts. Lectures, on the other hand, are often geared more towards factual presentation than connective learning.
Collaborating Collaboration allows students to actively participate in the learning process by talking with each other and listening to other points of view. Collaboration establishes a personal connection between students and the topic of study and it helps students think in a less personally biased way. Group projects and discussions are examples of this teaching method. Teachers may employ collaboration to assess student's abilities to work as a team, leadership skills, or presentation abilities. Collaborative discussions can take a variety of forms, such as fishbowl discussions. After some preparation and with clearly defined roles, a discussion may constitute most of a lesson, with the teacher only giving short feedback at the end or in the following lesson.
Learning by teaching In this teaching method, students assume the role of teacher and teach their peers. Students who teach others as a group or as individuals must study and understand a topic well enough to teach it to their peers. By having students participate in the teaching process, they gain self-confidence and strengthen their speaking and communication skills.
Evolution of teaching methods Ancient education About 3000 BC, with the advent of writing, education became more conscious or selfreflecting, with specialized occupations such as scribe and astronomer requiring particular skills and knowledge. Philosophy in ancient led to questions of educational method entering national discourse. In his literary work The Republic, Plato described a system of instruction that he felt would lead to an ideal state. In his dialogues, Plato described the Socratic Method, a form of inquiry and debate intended to stimulate critical thinking and illuminate ideas.
It has been the intent of many educators since, such as the Roman educator Quintilian, to find specific, interesting ways to encourage students to use their intelligence and to help them to learn.
Medieval education Comenius, in Bohemia, wanted all children to learn. In his The World in Pictures, he created an illustrated textbook of things children would be familiar with in everyday life and used it to teach children. Rabelais described how the student Gargantuan learned about the world, and what is in it. Much later, Jean-Jacques Rousseau in his Emile, presented methodology to teach children the elements of science and other subjects. During Napoleonic warfare, the teaching methodology of Pestalozzi of Switzerland enabled refugee children, of a class believed to be unteachable, to learn. He described this in his account of an educational experiment at Stanza. He felt the key to have children learn is for them to be loved. The Prussian education system was a system of mandatory education dating to the early 19th century. Parts of the Prussian education system have served as models for the education systems in a number of other countries, including Japan and the United States. The Prussian model required classroom management skills to be incorporated into the teaching process.
20th century Newer teaching methods may incorporate television, radio, internet, multimedia and other modern devices. Some educators believe that the use of technology, while facilitating learning to some degree, is not a substitute for educational methods that encourage critical thinking and a desire to learn. Inquiry learning is another modern teaching method.
Conclusion Evaluation is often used to characterize and appraise subjects of interest in a wide range of human enterprises, including the arts, criminal justice, foundations, non-profit organizations, government, health care, and other human services. A class evaluation is a paper or electronic questionnaire, which requires a written or selected response answer to a series of questions in order to evaluate the instruction of a given course. The term may also refer to the completed survey form or a summary of responses to questionnaires. They are a means to produce useful feedback which the teacher and school can use to improve their quality of instruction. The process of (a) gathering information about the impact of learning and of teaching practice on student learning, (b) analyzing and interpreting this information, and (c) responding to and acting on the results, is valuable for several reasons. They are beneficial because instructors can review how others interpret their teaching methods, thereby improving their instruction. The information can be also used by administrators, along with other input, to make summative decisions (e.g., decisions about promotion, tenure, salary increases, etc.) and make formative recommendations (e.g., identify areas where a faculty member needs to improve). Summative student evaluations of teaching (SETs) have been widely criticized, especially by teachers, for not being accurate measures of teaching effectiveness. Surveys have shown that a majority of teachers believe that a teacher's raising the level of standards and/or content would result in worse SETs for the teacher, and that students in filling out SETs are biased in favor of certain teachers' personalities, looks, disabilities, gender and ethnicity. The evidence that some of these critics cite indicates that factors other than effective teaching are more predictive of favorable ratings. In order to get favorable ratings, teachers are likely to present the content which can be understood by the slowest student. Consequently, the content has been affected. Many of those who are critical of SETs have suggested that they should not be used in decisions regarding faculty hires, retentions, promotions, and tenure. Some have suggested that using them for such purposes leads to the dumbing down of educational standards. Others have said
that the typical way SETs are now used at most universities is demeaning to instructors and has a corrupting effect on students' attitudes toward their teachers and higher education in general. The economics of education literature and the economic education literature is especially critical. For example, Weinberg et al. (2009) finds SET scores in first-year economics courses at Ohio State University are positively related to the grades instructors assign but are unrelated to learning outcomes once grades are controlled for. Others have also found a positive relationship between grades and SET scores but unlike Weinberg et al. (2009) do not directly address the relationship between SET scores and learning outcomes. A paper by Kaufmann and Sander (1999) find that the grades students expect to receive in a course are positively related to SET scores. Wisely and Singh (2005) find it is the difference between the grades students expect to receive and their cumulative GPA that is the relevant variable for obtaining favorable course evaluations. Another paper by Carrel and West (2010) use a data set from the U.S. Air Force Academy where students are randomly assigned to course sections (reducing selection problems). It found that calculus students got higher marks on common course examinations when they had instructors with high SET scores but did worse when they took later courses requiring calculus. The authors discuss a number of possible explanations for this finding, including that instructors with higher SET scores may have concentrated their teaching on the common examinations in the course rather than giving students a deeper understanding for later courses. Hammerers and West (2005) find that students at the University of Texas at Austin gave attractive instructors higher SET scores than less attractive instructors. However, the authors conclude that it may not be possible to determine if attractiveness increases the effectiveness of an instructor, possibly resulting in better learning outcomes. It may be the case that students pay more attention to attractive instructors.
