Bb World 2011 — Go Rubrics! Creating A+ Rubrics

bbworld-2011-sm.png

PRESENTERS:

Peggy Brown — Director of Instructional Design / Adjunct  Faculty, Syracuse University
Michael Fudge — Senior Systems and Network Support Administrator, Syracuse University

A few select members of EdTech had the opportunity to attend Blackboard World 2011 in Las Vegas (July 12-14). Bb World is a fairly large conference hosted by (of course) Blackboard. However, even though many of the sessions touched upon Blackboard, either directly or indirectly, others covered topics that are applicable to a wide range of teaching situations, where which LMS instructors use is not relevant to the discussion.

One such session was called: Go Rubrics! Creating A+ Rubrics.

The basic idea behind a rubric is that it is a way of evaluating and measuring student work according to some pre-established criteria. In other words, it is one more performance assessment, like tests or quizzes. However, unlike a multiple-choice test where students are either right or wrong, it is NOT simply a checklist and is often highly subjective. But it is still possible to use a rubric to set clear expectations between students and instructors. Students are often frustrated with instructors because they don’t know exactly what an instructor is looking for in a paper. Therefore, the feedback they receive on their first paper is used as the
“baseline” for future assignments. Using a rubric, especially one that is available for students to view, gives students a much better idea of the standards that will be used to gauge performance in the class.


Using a rubric also gives students the opportunity to judge their own
work, or, if peer-review workshops are part of the class, the work of
other students. Students can focus their efforts on relevance and
quality within the assignment, if they are able to see how much each
criteria is worth for the assignment. For instance, content and
relevance might be worth most of the points on the assignment, while
fewer points are assigned to grammar and style. Students will also
receive consistent feedback from one assignment to the next (if they are
weak on grammar/style, or don’t provide sufficient detail in their
responses, the instructor can note this in repeated assignments, giving
the student more opportunities to correct for their deficiencies in one
area while remaining strong in another).

From an instructor’s
point of view, using a rubric has many benefits. Instructors can use the
rubric to outline learning objectives in the course. Are the students
“getting it”? Rubrics also make it easier to evaluate students more
equitably by using an established guideline for quality. Finally,
rubrics allow for much quicker grading through formalized guidelines.
Blackboard 9.1 SP 6 (which is NOT released on campus at the time of this
blog post) includes a handy rubric feature that can be used in blogs,
wikis, and other assignments within Blackboard for very, very quick
evaluation and grading of student posts.

Creating a “good” rubric
is often an iterative process. Once the initial rubric is created, it
is then refined over time to become more effective. Student input often
helps in this process. Rubric criteria should be tied directly to the
learning outcomes for the assignment and for the course. Also, the
rubric should contain numerous definitions as well as some examples of
quality work to illustrate the expectations from the instructor.

Rubrics
have been criticized for a variety of reasons. Some instructors believe
they take too much effort to create. There is definitely some truth in
this in that a good rubric takes a lot of thought and energy. However,
as with many tasks associated with quality instruction, an initial
investment before a class starts can lead to significant savings in time
and energy later when it is time to grade assignments. Rubrics can be
difficult to get just right, but again, this is often an iterative
process that may take several tries and refinements. Another criticism
is that a rubric may shape student work into a certain format. This is
not necessarily a bad thing, especially if an instructor is looking for
students to grasp concepts in a certain way. The rubric should
illustrate the “baseline” for acceptable work in a class. Students who
consistently go above and beyond the confines of the rubric through
their own innovation and initiative should be rewarded for their hard
work.

There are two basic styles of rubrics: holistic and analytical.

A
holistic rubric is simple, one-dimensional, and has a single
description scale. For example, a holistic rubric for a discussion board
might have five criteria: relevance, conciseness, engagement, clarity,
and timeliness. Students must meet all five criteria to get full credit
on a discussion board posting. Otherwise, they only get credit for the
criteria they actually meet (a post might be timely and concise, but
lack relevance, be unengaging, and somewhat difficult to understand).
Holistic rubrics are fairly quick and easy to create, but lack the depth
needed to fully rate student performance. They are best used for things
like discussion board postings and other short assignments where the
instructor is just looking for some basic understanding from the
student.

Analytical rubrics are two-dimensional matrices with a
full range of criteria and performance levels. This is what we typically
think of when we talk about rubrics. Criteria for an analytical rubric
might include relevance, clarity, grammar/style, direction-following,
and so forth. It is important to define what a “0” means within the
context of this rubric as sometimes it is not very clear. Also, examples
of the different performance levels are often very helpful to students.
They do not have to be long and complex examples, but should be
representative of what the instructor is looking for on the assignment.
One of the nice things about an analytical rubric is that the different
criteria can be “weighted” so that students who meet the higher-weighted
criteria while missing some of the lower-weighted criteria can still
demonstrate proficiency in the learning outcomes. Analytical rubrics are
definitely more complicated to produce, but the benefits can absolutely
be worth it.

Rubrics are NOT the only way to measure academic performance in a
course, and may not lend themselves to every discipline. For instance, a
rubric doesn’t make a whole lot of sense for an introductory calculus
class where students spend most of their time solving math problems.
However, they can be a VERY effective tool in more subjective
disciplines where students must demonstrate mastery of concepts through
writing exercises and other activities (e.g. presentations). Students
generally respond very favorably to rubrics because they have a clearer
picture of the expectations for quality work in a course and can
therefore dedicate their time and energy to meeting the criteria.
Instructors who have used rubrics for a while also tend to like them as
they DO speed up the grading process considerably.

EdTech uses the Quality Matters rubric, which is a weighted analytical
rubric, to evaluate the quality of online courses. We can provide
detailed feedback for the critical items required to meet the QM
standard, so it definitely saves us time and energy.