When do we evaluate the effectiveness of instruction

If this method is used, students should clearly understand that the written summary is not a quiz and will not be graded. Student responses to the second and third questions also can help instructors select and structure topics for the next class meeting. Large numbers of such short papers can be read quickly, and a review of unclear concepts can take place at the next class meeting Angelo and Cross, ; Schwartz, Students are asked to respond to one or two short questions posed by the instructor the day before a subject is to be taught.

They submit their responses via e-mail or to a website. These responses give the instructor a good idea of what the students do and do not understand about the concepts to be considered. The instructor can then adjust the amount of time spent on explaining the concepts, working through problems, or providing examples that will help the students learn and understand the concepts.

Another documented approach involves asking a team of students to work throughout the term on continuous course evaluation Baugher, ; Greene, ; Wright et al. The team members are encouraged to administer questionnaires and interview their peers about how the instructor is or is not promoting learning. For larger classes, a liaison committee of two to four students can be established that meets periodically with the instructor to discuss difficulties or dissatisfactions.

Membership on the committee can be rotated from a list of volunteers as long as the entire class knows who the liaisons are at any given. Alternatively, students who are not enrolled in a course can be hired to attend the class and offer ongoing feedback to the instructor e. This technique allows teachers to see what students consider to be the main points presented and whether there is misinformation or confusion about various topics.

Alternatively, to ensure student anonymity, students can be asked to photocopy selected portions of their notes and submit them to the instructor without identifying information Davis, In small classes, it may be possible to pass around a piece of paper midway through a session and ask students to jot down the main point of what is being discussed at that moment.

The instructor then has a listing of what students consider to be the key concepts discussed in that class period, which can be used Angelo and Cross, Students can be encouraged to form small study groups and to send representatives to discuss any difficulties or questions with the instructor. Study groups provide students with opportunities to learn from one another, and a group may find it easier to seek assistance from the instructor.

In turn, having group representatives rather than individual students approach the instructor can reduce the amount of time required to answer repetitive questions, especially in larger classes. Instructors can seek feedback through informal conversations with students during office hours, before or after class, or through e-mail.

They can ask students about what has been working well or what is problematic. Instructors should not pose these questions to students in ways or at times that might force them to answer quickly. Questions should be directed to those students the teacher thinks would be most likely to respond candidly.

Whenever this kind of feedback is solicited, instructors should keep in mind that such evidence is anecdotal and may not be representative of the entire class. However, informal responses from individual students can serve as the basis for index card questions to the entire class discussed next. Asking such questions based on informal conversations with students can also reinforce the message that the instructor is listening to students and takes input from them seriously.

Several times during the term, an instructor can pass out index cards to students and ask them to respond to two questions, one on the front of the card, the other on the back. Midway through the term, an instructor can invite an instructional improvement specialist from the campus-wide or discipline-based teaching and learning center or a departmental colleague to conduct an oral evaluation with his or her students. At the beginning of the class, the teacher introduces the guest evaluator and then leaves the room for 20 minutes.

After the groups have completed their work, the evaluator asks the spokesperson from each group to report. The evaluator summarizes the points of consensus for the entire class and also clarifies points of disagreement.

The evaluator then provides an oral or written summary for the instructor Clark and Redmond, This technique also known by its abbreviation, SGID originated at the University of Washington and is now promoted by teaching and learning centers on a variety of types of cam-. The description of small group instruction diagnosis presented here is based on information taken from the websites of several campus centers for teaching and learning.

The goal of SGID is to align expectations for improving teaching and learning. Consultants are employed to gather information directly from students and instructors. The technique provides feedback to instructors, including suggestions for strengthening their courses, and generally increases communication between students and instructors. The consultative process takes anywhere from 15 to 60 minutes and is most effective when conducted near midsemester so the faculty member will have sufficient time to amend the course.

Research at the University of Washington on the use of class interviews indicates that students appreciate the opportunity to provide feedback to their instructor before the end of the quarter. It is important that the issues posed in a midsemester evaluation be ones to which the instructor will be able to respond during the term.

Otherwise, students may develop false expectations about the remainder of the course. Instructors should emphasize to students that they would like to receive candid, constructive responses that will help them improve the course.

Davis discusses strategies for responding to student feedback. She sorts student suggestions for improvement into three categories: 1 improvements that can be instituted immediately during the current semester e.

The instructor should then indicate which suggestions would be implemented this term, those that must wait until the course is next offered, and those on which action cannot or will not be taken. In the third case, it would be helpful to explain briefly the reasons for this decision. Teaching assistants can be an invaluable source of feedback for faculty members about successes and problems that are occurring in classes, discussion sections, and teaching laboratories. Such feedback can be especially illuminating if teaching assistants are encouraged to attend class sessions regularly and to meet with the faculty member in charge of the course and with each other.

Ways in which teaching assistants can provide appropriate feedback to individual faculty and to their academic department include the following:. Encouraging teaching assistants to provide information throughout the term about the difficulties students may be having in the courses with which the teaching assistants are involved.

Through conversations with and direct observation of students in the course, teaching assistants can tell an instructor what aspects of the course readings, assignments, and presentations are causing problems for students.

Some faculty ask teaching assistants to give them brief weekly reports on the one or two things that cause students the most difficulty. Asking teaching assistants to review examinations and quizzes before they are given to students. Having participated in the course, teaching assistants can identify ambiguous or unclear exam items before the tests are administered.

After midterms or quizzes have been graded, teaching assistants can provide detailed information about patterns of error or misunderstanding. Collecting this kind of information from a number of teaching assistants from different courses, from sections within a course, and over an extended period of time can also enable departments to determine which concepts need to be reinforced in several courses or which misconceptions persist as students advance through the curriculum.

Under the right circumstances, these judgments can be used to assist in summative evaluations of faculty see also Chapter 4. Similar judgments from colleagues also can be useful in formative evaluations for professional development of faculty. At small institutions or in very small departments, a lack of resources or limited numbers of faculty may make faculty input more difficult to obtain than in larger institutions or departments.

In addition, friendships or rivalries that arise within any department may be amplified in smaller departments. In such cases, balanced and objective evaluations of teaching colleagues may be achieved only by including in the evaluation process additional faculty from outside the academic unit of the person being evaluated. Instructors who are being evaluated can ask a mentor, colleague, or instructional improvement specialist at the campus or discipline-based teaching and learning center to visit their classes and provide feedback on their teaching.

Prior to each visit, instructors can discuss with observers the specific classroom issues or techniques on which the observers should focus e. Faculty also can ask colleagues, particularly those known to be excellent teachers, for permission to visit their courses.

Visitors can note the specific techniques used by the colleague in leading discussions, conducting teaching laboratories, and so on. If time permits after class, the observing and observed faculty members can discuss their respective teaching philosophies, goals, instructional methods, out-of-class preparation, and similar matters.

It is usually most helpful for a faculty member to attend a series of classes say, all classes dealing with a specific topic or issue to obtain a broad per-. Informal discussions and efforts to improve instruction among faculty members take place daily, but some departments and institutions employ more systematic and formal efforts to assist in the improvement of teaching through formative evaluation. In addition to the evaluation questionnaires reprinted in Appendix C , the following approaches to formative evaluation can be especially useful for the purposes of faculty professional development.

Faculty mentoring faculty. Increasingly, departments are assigning senior faculty as mentors to untenured faculty. Boice found that it was not necessary for successful mentors to be from the same department. At a variety of higher education institutions, Katz and Henry developed a strategy of transdisciplinary mentoring based on faculty working together to understand both how students learn and how to improve their teaching. Referred to as the Master Faculty Program, this initiative involves faculty working together in pairs or in triads.

With these observations in hand, the faculty participating in the program meet periodically to discuss candidly, and confidentially, how each participant has or has not fostered student learning.

Chandler has documented the generally positive results of this type of program involving some faculty at 21 different colleges and universities. Formative evaluation by faculty colleagues from other institutions.

Faculty at higher education institutions across the country and around the world can provide formative evaluation to colleagues via the Internet.

This kind of input from colleagues at other institutions could be included as part of a teaching portfolio or dossier for. Projects of the American Association for Higher Education. One such project, conducted in the mids, involved 12 universities and stressed peer review as a means of formative evaluation.

In this project, participants monitored their progress in improving student learning. More recently, AAHE, the Carnegie Foundation for the Advancement of Teaching, and the Carnegie Academy for the Scholarship of Teaching and Learning jointly developed a program for peer collaboration based on ideas and criteria advanced by Boyer and Glassick and colleagues The goals of the program are to support the development of a scholarship of teaching and learning that will foster significant, long-lasting learning for all students.

The program also seeks to enhance the practice and profession of teaching and bring to the scholarship of teaching the same kinds of recognition and reward afforded for other forms of scholarly work Hutchings, Centra has extended these criteria to allow for evaluation of the scholarship of teaching and learning as practiced by academic departments and institutions see Box These self-reports, which may be part of a required annual report or a teaching portfolio, are more useful and appropriate for formative or professional development purposes than for summative personnel decisions.

Faculty who have not previously performed self-evaluation may require assistance from teaching and learning centers. Box Evaluating the Scholarship of Teaching. Clear Goals: Does the scholar state the basic purposes of his or her work clearly? Does the scholar define objectives that are realistic and achievable?

Does the scholar identify important questions in the field? Adequate Preparation: Does the scholar show an understanding of existing scholarship in the field? Does the scholar bring the necessary skills to his or her work?

Does the scholar bring together the resources necessary to move the project forward? Appropriate Methods: Does the scholar use methods appropriate to the goals? Does the scholar apply effectively the methods selected? Does the scholar modify procedures in response to changing circumstances?

Significant Results: Does the scholar achieve the goals? Effective Presentation: Does the scholar use a suitable style and effective organization to present his or her work? Does the scholar use appropriate forums for communicating work to its intended audiences? Does the scholar present his or her message with clarity and integrity? Reflective Critique: Does the scholar critically evaluate his or her own work?

Does the scholar bring an appropriate breadth of evidence to his or her critique? Does the scholar use evaluation to improve the quality of future work? Products of good teaching for example, student workbooks or logs, student pre- and post-examination results, graded student essays.

Material developed by the individual course and curriculum development materials, syllabi, descriptions of how various materials were used in teaching, innovations the instructor has attempted and an evaluation of their success, videotapes of teaching. Material or assessments from others student work and evaluations, input from colleagues or alumni. Descriptions of how the individual has remained current in the field, such as using knowledge gained from attending professional conferences Edgerton et al.

External support obtained for such purposes as improving teaching or purchasing instrumentation for teaching laboratories. Videotaping is a useful strategy that enables instructors to see what they do well and what needs to be improved. Faculty who have been videotaped find the experience extremely helpful, especially if they discuss the analysis with someone having expertise in classroom behavior.

Videotaping is best used for formative evaluation. Faculty members can use before-and-after self-assessment to determine whether course outcomes meet their expectations. Before a course begins, the instructor writes brief comments about the types of students for whom the course is intended. Given that audience, the instructor lists the most important course and learning goals and the teaching strategies she or he will design to achieve them.

These self-assessments can become part of a teaching portfolio that can later be used for more summative types of evaluation. Another form of before-and-after assessment may help instructors who are interested in examining their teaching behaviors and effectiveness rather than course outcomes.

For this technique, instructors use the end-of-course evaluation form, but complete the questionnaire before their course begins predicting how they think they will do and again at the end of the semester how they believe they did. They also may wish to fill out a questionnaire at the end of the term based on what they expect, on average, their students will say about their teaching.

In most cases, such self-evaluations are. In looking at the results, instructors may wish to focus on any deficiencies noted in the self-evaluation or on discrepancies between their own evaluations and those of their students. Questionnaires are most commonly used for summative student evaluations of teaching. The questionnaires can be machine-scored and fall into two categories: those developed locally by campus teaching and learning centers by consulting the literature or adapting forms used elsewhere, and those developed by other institutions or organizations and made available for a fee.

Developing and implementing such an evaluation system has many benefits including helping you to:. As you set goals, objectives, and a desired vision of the future for your program, identify ways to measure these goals and objectives and how you might collect, analyze, and use this information.

This process will help ensure that your objectives are measurable and that you are collecting information that you will use. Strategic planning is also a good time to create a list of questions you would like your evaluation to answer. See Step 2 to make sure you are on track. Update these documents on a regular basis, adding new strategies, changing unsuccessful strategies, revising relationships in the model, and adding unforeseen impacts of an activity EMI, It describes features of an organizational culture, and explains how to build teamwork, administrative support and leadership for evaluation.

It discusses the importance of developing organizational capacity for evaluation, linking evaluation to organizational planning and performance reviews, and unexpected benefits of evaluation to organizational culture.

If you want to learn more about how to institutionalize evaluation, check out the following resources on adaptive management. Adaptive management is an approach to conservation management that is based on learning from systematic, on-going monitoring and evaluation, and involves adapting and improving programs based on the findings from monitoring and evaluation.

Downloaded September 20, from: www. Patton, M. Qualitative Research Evaluation Methods. Thomson, G. Measuring the success of EE programs. Canadian Parks and Wilderness Society.

Skip to main content. Evaluation: What is it and why do it? Table of Contents What is evaluation? Should I evaluate my program?

What type of evaluation should I conduct and when? What makes a good evaluation? How do I make evaluation an integral part of my program? How can I learn more?

What is evaluation? Experts stress that evaluation can: Improve program design and implementation. Demonstrate program impact. Within the categories of formative and summative, there are different types of evaluation. Which of these evaluations is most appropriate depends on the stage of your program: Type of Evaluation Purpose Formative 1.

Needs Assessment Determines who needs the program, how great the need is, and what can be done to best meet the need. For more information, Needs Assessment Training uses a practical training module to lead you through a series of interactive pages about needs assessment. Process or Implementation Evaluation Examines the process of implementing the program and determines whether the program is operating as planned. Test the questions with colleagues first. Decide whether you are likely to get better responses by asking students to complete the questionnaire in class or in their spare time.

Consider using a combination of a Likert-scale e. Likert-scale answers can make issues easier to identify, whereas free-text responses can provide richer content, explaining why students felt a particular problem occurred.

Remember to close the feedback loop: feed back to the students what was raised in the evaluation and tell them what you will be doing to address potential issues. If you cannot address something, it is often still worth explaining why to students so they know you are listening to them. You are welcome to use this guide if you are from another educational facility, but you must credit the UCL Arena Centre.

More teaching toolkits - back to the toolkits menu. UCL Education Strategy — Learning and Development at UCL. Academic Careers Framework. Download a printable copy of this guide. Gain recognition for your role in education at UCL.

There are pathways for teaching staff, researchers, postgraduate teaching assistantsand professional services staff:.