Online Assessment: How do I know my students are learning?

Assessments that are aligned with your learning outcomes provide reliable feedback about student learning and reinforce to students what needs to be mastered. A balanced assessment strategy in online courses can prevent surprise grades and the drama that often accompanies them.

Learning Objectives

In this module, we discuss the "why" of aligning assessments with learning outcomes, the "what" of types of assessments to gather valid evidence that your students are learning what you intend for them to learn, and the "how" of developing assessments for online learning.  At the end of this module, we hope that you will be able to: 

3. Differentiate between authentic and traditional assessments and between formative and summative assessments
4. Identify the components of a balanced, comprehensive assessment strategy
5. Create appropriate ways to assess whether students have achieved the learning outcomes for your course

Table of Contents

• Aligning and Developing Assessments
• Planning and Developing Assessments
• Practice Developing Assessments
• Online Assessment: Resources 

Aligning and Developing Assessments

Why is it important to align assessments with learning outcomes?

Alignment between assessments and desired learning outcomes is foundational if your assessments are to be valid.  Just like in a research study where you want to make sure that your research instrument is measuring what you want it to measure, by aligning your assessments to your learning outcomes you are making sure you are assessing what you want to assess.  As we mentioned in the Course Planning with Backward Design, Biggs (2003) describes the constructive alignment of three components: (a) measurable, clearly-stated learning outcomes, (b) assessment tasks that allow students to show to what extent they have reached the learning outcomes, and (c) activities (including content and practice) that help students reach the learning outcomes.  Assessments that are aligned with your learning outcomes provide dependable evidence as to how well students are reaching the desired outcomes. 

Clearly aligning assessments to desired learning outcomes also reinforces to students what needs to be mastered and helps them track their progress in the course. Students pay attention to what you test.  For example, If your intent is for students to be able to apply, critique, or evaluate, but your assignments and exams ask students to remember, identify, and describe, then your assessments aren't aligned with your desired learning outcomes.  Asking students to describe a concept doesn't encourage them to evaluate the concept in context and doesn't provide evidence that they can evaluate the concept.

The intent of Backward Design is that assignments (and everything else) are aligned to desired learning outcomes instead of creating learning outcomes based on what you are assessing.  Starting with assessments and extrapolating learning outcomes from them is the definition of "teaching to the test."  This may be necessary if your course is preparing students to sit for a licensure or registry exam, but those cases are the exception more than the rule.  

What does assessment look like in an online context?

Just as you can't confirm a hypothesis without testing it, so, too, you can't confirm whether your students have achieved the course learning outcomes without some form of assessment. This is why assessment is the second stage of backward design - if you know where you want students to go (learning outcomes), you next need to decide how you'll know if they've gotten there (assessment). Assessment is that evidence.

Although the types of assessments that often first come to mind are a test, paper, or lab exercise, many other activities can be used for assessment, including portfolios, discussion forums, concept maps, diagrams, and presentations. Any tangible output from a learning activity can be assessed. Your choice of output—and the activity designed to generate that output—should be determined by your learning outcomes; this is just as true in the online environment as it is in the traditional classroom.

First let's look at four qualities of assessments: Formative, Summative, Authentic, and Traditional; and how those qualities can be leveraged in different ways online. Below, we see how mixing these qualities can create a balanced, comprehensive assessment strategy that is aligned with learning outcomes for an entire course.

Formative Assessment

Formative assessment is designed to provide feedback to both student and instructors about how well the learning process is going. Examples of formative assessment include self-tests, think-pair-share activities, and other low-risk assignments that allow students to demonstrate their knowledge, skills, and abilities.

Another option for formative assessment is to develop a larger, summative assessment and break it into smaller components that can be turned in throughout the semester. This allows you to catch and address misconceptions, challenge students’ early analyses, and provide the opportunity for them to revise and resubmit each piece in a unified whole at the end of the semester or unit.

Formative assessment options such as ungraded self-tests using the Blackboard Quizzes tool, think-pair-share activities in discussion forums or group spaces offer ways for students assess their own understanding of course concepts. If you are interested in embedding some understanding checks in your Blackboard course, try Quizlet (Links to an external site.) which can be added by going to Settings and then Apps in your course.

An often overlooked option for formative assessment is the Blackboard Self and Peer Assessment (Links to an external site.). When adequately scaffolded, peer review and critique can be a learning activity for both the student giving and the student receiving the peer review. The Assignments tool in Blackboard provides options for blind peer review, or you can set up a Discussion where students post their thoughts or explanations or examples and then provide feedback to the person posting immediately above them.  Students can be split into small groups where they can share an initial draft of a paper or project, each student gives feedback to all the other group members, and then they work together to synthesize their best efforts into a group report. Using the Group spaces in Blackboard allows the instructor to see all of the initial drafts and student discussions while keeping each group separated from the other groups.

(Go back to top)

Summative Assessment

Summative assessment is designed to provide evidence that students have achieved a learning outcome or otherwise gained skills or knowledge throughout the course. End-of-semester exams, projects, portfolios, and presentations are often used to summatively assess students' knowledge and skills. Courses that use a blend of summative and formative assessments provide more consistent support for learning than relying exclusively on a midterm and a final exam.

Final papers, projects, and portfolios have a variety of options in an online class.  It's easy to incorporate media into Assessments, Discussions, and Content Areas - both in project instructions such as presenting a video case for analysis, and in student work such as recorded presentations, interviews, and demonstrations.  A videoconferencing tool like Blackboard Collaborate can be used for synchronous assessments such as oral exams in languages. Other tools such as WebEx or MS Teams record individual video presentations or interactions such as mock counseling sessions and other role-play scenarios which students can submit to an assignment or share in a discussion.

Authentic Assessment

Authentic assessment asks students to demonstrate skills and knowledge by performing realistic tasks within the discipline. It provides opportunities to practice, consult resources, get feedback, and refine performances and products. Well-designed authentic assessments: 

• are realistic, using real-life situations with constraints, purposes, and audiences that impact what needs to be done.
• require judgment and innovation to effectively solve unstructured problems
• assess students' ability to use their knowledge and skills to negotiate a complex task as a whole. 

Authentic assessment commonly uses strategies such as case studies, simulations, consulting (where students work with real organization to explore a problem and recommend solutions that are evaluated by both the instructor and the organizational partner), internships, and service-learning. However, depending on the discipline, authentic assessment can leverage simpler tools.  For example,

• situating statistical questions in the context of fantasy sports over several weeks of a season,
• sending emails in a foreign language to request information about a travel destination and working with a small group to determine which destination would be the best vacation spot.

Traditional Assessment

Traditional assessment (defined mainly as discrete-item testing) tends to emphasize the development of a body of knowledge or skill. Does a student know the who, what, when, and where? Traditional assessment strategies are helpful when you want students to identify one best answer and/or target isolated skills in a concrete fashion.
Something to keep in mind is that assessment methods do not have to line up with assessment approaches. For example, multiple-choice test items can be developed to draw attention to contextual factors in an authentic case. In the same way, artificial and minimally contextualized cases can be used to identify who, what, when, and where without asking students two work with holistic, complex problems.

Online testing using the Blackboard Quizzes tool provides auto-grading and auto-feedback features with a wider range of options than blue book or scantron testing. You can provide video, audio, and images as part of a question, and students can record or upload video, audio, and images as part of their answers. You can pre-set different feedback for different incorrect answers and even re-route students to review content area explaining the question in more depth. 

However, there are also drawbacks. Many faculty express concerns about the potential for cheating in an online class. Where a faculty member might make one test and deliver it once in a proctored room for a face-to-face course, a similar fully online test may be delivered over time. 

If you do use online testing features, here are some options to consider:

• Create multiple question banks for a single test - break down the questions that are delivered to students, so no one gets the exact same examination, but all are tested on similar concepts and at similar levels of difficulty.
• Randomize the order of questions to discourage cheating
• Randomize the answers and distractors to discourage cheating
• Use an honor statement at the beginning of each test, forcing the students to check "yes" or "no" that they understand the academic dishonesty policy, and they have not received assistance.
• For more options specific to the Blackboard Test tool option, please see the Blackboard Creating Tests Instructor Guide. 

(Go back to top)

Planning and Developing Assessments

A Balanced Assessment Strategy

In an online course it is important for students to get frequent feedback on how they are doing. Are they learning what they are supposed to be learning? Are they achieving the learning outcomes? The most effective way to ensure that students get the feedback they need to stay on track is through a comprehensive, balanced assessment strategy that includes both formative and summative assessments.

Keep in mind that summative assessment doesn’t necessarily mean “graded” nor does formative assessment necessarily mean “non-graded.” For example, a mastery quiz on reference formatting may not provide a grade that counts toward the course final grade, but passing it may be required before the student can turn in a first paper. Alternatively, a first draft of a paper may count toward the final grade in the course, but the formative feedback on the draft is used to improve the final draft later in the semester.  

Online courses also lend themselves to the use of automatically graded multiple-choice or short-answer "Understanding Checks."  After completing one, students can receive feedback based on the answer they chose in a multiple-choice section or compare their answers to those of an expert in a short answer section. Although a grade may or may not be recorded in a grade book, such activities provide students with feedback on how well they understand course concepts. For more on setting up these types of assessment see the Blackboard Guide on the different question types. (Links to an external site.) 

These sorts of frequent, low-stakes assessment opportunities where students self-test their knowledge and understanding of concepts can be very helpful, especially in situations where mastery is the desired outcome. They allow both instructors and students to catch misunderstandings and misconceptions early before other learning is built on a shaky foundation.

For more tips on online assessment you may be interested in Multiple Choice Test Ideas For College Classrooms (Links to an external site.).

What should I consider when I create assessments?

As part of a comprehensive and balanced assessment strategy, you will want to develop both formative and summative assessments.  Smaller, lower-stakes assessments are good opportunities to provide formative feedback for students as they work through the course.  In an online course you can structure these types of assessments in many of the same ways as you would in your face-to-face class.

In addition to your "homework" and "in-class" activities Classroom assessment techniques (Links to an external site.) (CATs) such as

• a poll or survey
• muddiest point,
• pro/con grid,
• focused paraphrasing, or
• a concept map

translate well to online tools both inside and outside your learning management system and provide opportunities for gathering formative assessment data. 

As you incorporate these techniques, it's important to ensure that you're assessing more than whether or not your students remember individual facts.  Many of these CATs ask students to use what they remember by applying it, evaluating it, or creating something new with it.  For more on online CATs you may be interested in the white paper Classroom Assessment Techniques (CATs) for Online Instruction from the University of Kentucky, (Links to an external site.). 

Student presentations (whether regarding cases or other projects) are another way to leverage online multimedia resources to support student learning and motivation.  Students can record and share video or narrated presentations asynchronously using a tool like VoiceThread (Links to an external site.) or you can gather students together in a video conference and allow them to present live to the class.

When having students turn in papers or analyses or similar items in Assignments, having a rubric for those assignments provides students an opportunity to self-assess their work in a formative way. For more on rubrics and the Blackboard Rubrics Tool see the Rubric section in Teaching Online Module. 

If you do use more traditional online testing, testing tools - such as those built into a learning management system - also allow you to create feedback that students can see after they take the test. It's sort of like an automated review. As the test creator, you have the option of when you want this automated feedback to become available. Most faculty make the feedback available only after everyone has taken the test, as a review. As soon as the students finish the test, they can see the answers and the feedback to check their own learning. 

Assessments that are aligned with your learning outcomes provide reliable feedback about student learning. Clearly aligning assessments to desired learning outcomes also reinforces to students what needs to be mastered and helps them track their progress in the course.

(Go back to top)

Practice Developing Assessments

How will you assess to what extent your students have met the learning outcomes?

Describe a potential assessment to measure 1-2 of your learning outcomes.

How does your assessment meet the criteria on the rubric below?  Do you need to revise your assessment or your learning outcome(s) It's not uncommon to refine learning outcomes during this process as you may find measuring your outcomes is not a clear-cut as it appears at the beginning.

Assessment Rubric Transcript (File can also be downloaded under Attachments at the bottom of this article)

(Go back to top)

Online Assessment Resources

Additional Information

A Model of Learning Objectives: (Links to an external site.) Iowa State University - This includes a great interactive tool for exploring the different dimensions of the revised Bloom's Taxonomy. They also have the same content in a pdf handout (1.64MB). (Links to an external site.)

One alternative taxonomy to Bloom is Dee Fink's Taxonomy of Significant Learning (pdf, 41k) (Links to an external site.). This is a non-hierarchical taxonomy that focuses on the interaction of 6 types of significant learning.

50 CATS by Angelo and Cross (pdf, 126k) (Links to an external site.) (Compiled by Cunningham and Moore)

Flint, W. Classroom Assessment Techniques (CATs) for Online Instruction (pdf, 44k) (Links to an external site.)

Alternative Assessment Methods for the Online Classroom (Links to an external site.) (Faculty Focus)

(Go back to top)