How Does Backward Design Provide For A Way To See If Students Are Actually Learning What You Want To Teach Then By Looking At The Assessment Results?

Let's look back on what a backward design looks like and how is it implemented. There are 5 steps to this lesson plan and to be carried out in the following order:
     Step 1: Choose the content - Subject or topic the eLearner is to learn
     Step 2: Learning Outcomes - What eLearners actually learned
     Step 3: Performance Objectives -  What eLearners do to acknowledge what they have learned 
     Step 4: Assessment - Proof of what eLearners actually learned (formative or summative) 
     Step 5: Activities/Materials - eLearners instruction and practice time

Taking this approach prompts instructors to think about the learning outcomes before they begin teaching  the material. Instructors then must begin with the end in mind when using this approach. With this design the learning outcomes are engrained in the brain, a constant reminder perpetuated throughout the entire learning process of what your students need to be learning. From the get go, knowing the learning outcomes and performance objectives allows instructors to plan activities more easily and appropriately thus continually assessing each and every student from start to finish. 
Consider this example of a Backward Design in progress:

Step 1:   Content is "Introductions in Spanish"
Step 2:   Learning outcomes:

• eLeaners will accurately be able to meet and introduce themselves verbally in Spanish 
• eLearners will be able to construct in writing a conversational dialog using Spanish vocabulary words and grammar relating to greetings and introductions with 90% accuracy.

Step 3:  Performance Objectives: 

• eLearners eLearners will correctly use Spanish vocabulary words and grammar as they relate to greetings and introductions.
• eLearners will construct a conversational dialog using Spanish vocabulary words and grammar relating to greetings and introductions with 90% accuracy. 

Step 4: One type of assessment, a performance assessment, might be to have eLearners create a conversation/dialog using the makebeliefscomix site as well as produce an audio voiceover to go along with the comic strip for assessing correct pronunciation. Scoring would involve a rubric that could both guide eLearners while doing the assessment as well as supply the scoring criteria for grading the assessment. 

Step 5:  This final step in the planning stage involves getting materials together and planning the activities that will ultimately take eLearners to the learning outcome.  This may consist of listening and watching an instructional video, completing a game like activity or perhaps IM chatting with other eLearners learning the same content material. 

Performance Assessments... What Are They And How Do They Benefit Or Not Benefit eLearning?

What is a Performance Assessment?

Basically, it is assessing a performance.  It can be either a direct observation of real-time learner behavior or result of a project that comes close to simulating the learner's behavior. In addition, performance assessments do something else that a typical written assessment does not, that is it measures 'problem-solving' skills.

(Oosterhof, Conrad, Ely, 2008, pp. 142-143 )

  

How can this type of assessment be used in eLearning?

Performance assessments can be ideal in eLearning situations because eLearners are able to demonstrate what they actually have learned about a topic by the process of creating a product or expressing themselves through the use of video, podcast, blog, wiki or other online delivery modality. Performance assessments can be used to measure any type of capability. In other words determine the type of behavior that is being observed and the product being a tangible outcome as a result of completing the process. 

Establishing a scoring plan is also an important part when using performance based assessments in eLearning situations. Rubrics are typically developed and used and must specify the essential qualities of both an unsatifactory performance and satisfactory performance.  In addition, both qualities must be directly observable and attain similar scores from different observers.

(Oosterhof, Conrad, Ely, 2008, pp.158-159; pp. 165-167 ); (Rio Salado College, 2011, lesson 7 lecture notes) 

  

What makes performance assessments truly worthwhile in eLearning? They can...

• Be Interactive and engaging for both the eLearner and the eInstructor.  
• Authentic to life tasks or simulations and incorporate a variety of learning styles that are multi-sensory 
• Be used repeatedly once created or developed.  
• Provide good insight into a learner's knowledge of a concept or subject
• Influence what is taught and learned by examining what is assessed.  
• Be preferable to written assessments for security reasons.

(Oosterhof, Conrad, Ely, 2008, pp. 145-146 )  

What could be possible pitfalls and problems with performance assessments in the eLearning environment?

• Using new technologies for the sake of trying it without considering what you need the real learning outcome to be.
• Observing and evaluating performance assessments can be time consuming.
• Not having the available or correct hardware connections or software necessary to do the performance assessment. 
• Performance assessments may be less efficient in that they require more time to produce, administer and score.
• Scoring is subjective and instructors may not interpret the same observation the same way.

(Oosterhof, Conrad, Ely, 2008, p. 147 )  

What Are The Benefits Of Using Both Constructed-Response And Fixed-Response Written Assessments For eLearning? What Is The Necessity For A Balance Between Teacher-Graded And Computer-Graded Assessment Items?

Let's take a quick look again at the difference between both constructed-response questions and fixed-response questions using the following chart, which also was presented in the previous posted blog on November 12, 2011.

Completion-Response	Fixed-Response
Limited sampling of content material	Adequate sampling of content material
Longer to grade although easier to create	Easier to grade although longer to create
Assesses more declarative knowledge	Assesses more procedural knowledge
Have to know the answer	Able to choose the answer
Directly measures targeted behaviors	Indirectly measures targeted behaviors
Fairly easy to construct	Time consuming to construct

What Are The Benefits Of Using Both Constructed-Response And Fixed-Response Written Assessments In eLearning? 

• Both are able to incorporate into the assessment design such external applications such as photos, sound, links, movies and video. Also, charts, tables, symbols and clip art. 
• Both are able to provide immediate feedback to eLearners on various objective type questions that are in an assessment such as multiple choice, T/F, matching and fill-in-the-blank. 
• Both make recording the outcomes of the eLearner's assessment easier by setting up and maintaining electronic gradebooks that the instructor and eStudent can access and view.   
• Both seek to decrease the workload for the instructor by using online computer software programs to help to develop, deliver and score the assessments. 
• Both allow instructors to have full view of content items and edit items using bold, italics, underlining, for example.
• Both allow an instructor access to a spell check, to preview the assessment and provide a description and/or directions for the assessment. 
• Both are designed and displayed in a way that is in an easy-to-read format for eStudents to navigate through and be able to scroll through answers changing responses if necessary before finally submitting.

(Oosterhof, Conrad, Ely, 2008, Pgs 124-140)  

What Is The Necessity For A Balance Between Teacher-Graded And Computer-Graded Assessment Items?

Establishing a balance between teacher-graded and computer-graded assessements is important for several reasons. Having one type without the other may short change what we, as eTeachers, learn about how and what an eStudent's is learning or has learned about a particular subject matter.  For example, although quick feedback from a computer graded multiple choice or true/false assessment provides immediate answers for both eStudents and eTeachers as to what is learned and what might need to be retaught, teacher-graded assessments, although slower to grade, subsequently allow eTeachers to analyze the more higher level thinking skills of their eLearners. Incorporating both teacher-graded and computer-graded assessments also makes it easier and more probable that all the learning styles of eLearners will be recognized and accounted for both the grading system as well as getting to know the capabilities of each eStudents so that planning for instruction can be relevant and effective for all concerned.  

What Are The Differences Between Constructed-Response And Fixed-Response Assessment And When Should You Use Each Type Of Assessment In eLearning And Why?

A completion-response question is one where the test-taker is required to supply the answer whether by completing a sentence with the correct word or completing an essay. A fixed-response question is one where the test-taker chooses an answer from various choices presented that best fits the question. Both can be used as a formative assessment although more likely seen in a fixed-response version. If you look at the chart below you notice that the two assessments seem to be quite a contrast in what each one delivers and possibly measures, as well. However, both type of assessments can be and should be created for online learning. The following chart reveals even more differences between two: 

Completion-Response	Fixed-Response
Limited sampling of content material	Adequate sampling of content material
Longer to grade although easier to create	Easier to grade although longer to create
Assesses more declarative knowledge	Assesses more procedural knowledge
Have to know the answer	Able to choose the answer
Directly measures targeted behaviors	Indirectly measures targeted behaviors
Fairly easy to construct	Time consuming to construct

When to use or create an appropriate assessment to fit your needs as an eTeacher is another question? What should this depend on? What factors must you, as an eTeacher consider? One factor is time.  How much time you have to either create or grade the assessments has to be a consideration no matter the extent you want to know what your eLearners have mastered so far. It takes time to create fixed-resoponse but because you can include a larger sampling of the content material it may be worthwhile. On the other hand, although easier to create and consists of a smaller sample of content material, a completion-response or essay question/answer can pinpoint more clearly whether or not an eLearner has truly mastered a specific content area because the student must 'know' or convey in writing knowledge of the subject sometimes in detail. Fixed-response questions can be more readily designed to measure procedural knowledge than do completion-response questions. Finally, there is always the issue of the programs that are used to grade various assessment that you create.  Are they compatible with the computer hardware and software you have available? How about grading essay questions? eTeachers spend countless hours grading essay questions it seems. Although there appears to be some new programs emerging claiming to be able to resolve those essay grading complications only time will tell.   

What Are The Advantages And Disadvantages Of Completion Style And Essay Formatted Constructed-Response Assessments?

A Constructed-Response assessment means what is implies, that eLearners must construct a response to a question as opposed to choose a response such as the case in multiple choice, T/F and matching tests. The following two sections describe two types of Constructed-Response assessments along with their advantages and disadvantages and grading criteria. 

Completion Assessments 

Advantages: 

1. The eLearner supplies the answer or solves a problem which signifies true knowledge or comprehension of the concept.

2. Completion questions are relatively easy to construct and do not require detailed scoring.

3. Completion type tests can include an adequate sampling of content material which is therefore likely to have more validity. 

Disadvantages:

1. Your assessment measures are restricted to your eLearner's recall of information. 

2. There is a possibility of more errors in scoring especially if using software that is somewhat restrictive.

When making and grading these type assessments and using the performance objectives as a guide in doing so, eTeachers need to make sure that the completion items include the following components or criteria: They must be...

• Designed to measure specific skills 
• Designed to read below an eLearner's ability level
• Designed so that the blank word is a "Key" word
• Designed so that the blank word or words are on the end or close to the end of the sentence
• Designed so that the number of blanks are limited

Sample Completion Question:
The type of assessments that require eLearners to fill-in-the-blank are called _________________ assessments.

Reference: (Oosterhof, Conrad, Ely, 2008)  

******************************************************************************

Essay Question Assessments 

Advantages:

1. Measures more directly the behaviors specific to the instructional objectives. 

2. eLearner's ideas are communicated in writing to demonstrate actual knowledge and comprehension of material as opposed to mere memorization. 

3. eLearner's can explain their logic and reasoning.

Disadvantages:

1. There is less adequate sampling of the content being tested.

2. Scoring runs the risk of being less reliable do to subjectivity. 

3. Essay questions take longer to score although developing and using rubrics can help. 

When making and grading these type assessments and using the performance objectives as a guide in doing so, eTeachers need to make sure that the completion items include the following components or criteria: They must be...

• Designed to measure specific skills
• Designed to read below an eLearner's ability level
• Designed to be able to answer in less than 10 minutes
• Designed so that scoring will be is relatively the same regardless of how many different readers are scoring it
• Designed so that the scoring plan is obvious and understandable to eLearners 

Sample Essay Question:
Explain how a blog is helpful in educating others of the same profession.

Reference: (Oosterhof, Conrad, Ely, 2008)  

What Is The Best Assessment For eLearners And Why?

I would have to say it depends on subject content and what performance objectives you are attempting to assess.  Since my area of teaching relates primarily to a K-12 eLearning curriculum, it's more likely that I will be concerned with finding out just how much each of my students have learned about a specific topic of study. Therefore a formative assessment that emphasizes criterion-referenced interpretations, that is, comparing an eLearner's performance to a well-defined domain can indicate an accurate assessment of what my eLearners can and cannot do; do know or does not know. It also means that no matter how an assessment is carried out it must ask questions specific to the content domain that you, as a teacher, want your  eLearners to be learning. It is essential to base formative assessments on lesson objectives and to use to monitor progress to validate  instruction as well as dictate whether or not instruction needs to be modified. Most often these type of assessments are performance or behavior driven not just paper and pencil multiple choice tests.

Specific questions asked within the framework of a formative assessment can be asked and answered by having eLearners participate in a game activity or in creating a game activity, project presentation, video screencast, short essay or even taking a short online T/F quiz. Utilizing audio and video technology is a great way to implement a formative assessment and for interpreting criterion-referenced questions/answers that establish what an eLearner can and cannot do.    

For example, if the topic being taught were knights in the Middle Ages, an eAssessment might be in the form of audio/visual project where an eLearner displays a picture of armor and labels it and describes it by creating a podcast to deliver that information. Or, an eLearner could be asked to write a blog from the point of view of a knight while in battle or perhaps role play an interview with a knight. These three examples demonstrate a performance driven formative assessment.

Another example may invlove having eLearners order a list of spelling words according to their difficulty or solve algebraic equations involving two unknowns. (Oosterhof, Conrad, Ely, 2008, pp. 74-75)  

References

Authorstream (2005). Assessing students online. Retrieved from
      http://www.authorstream.com/Presentation/Berenger-33823-Assessing-Students-Online-Why-Assess-Uses-Assessment-Tools-Modes-Knowledge-Based-as-Entertainment-ppt-powerpoint/.
Classroom Assessments. (n.d.). Formative and summative assessments. Retrieved from
    http://fcit.usf.edu/assessment/basic/basica.html.

      

What Is The Difference Between Training And Education As It Pertains To Assessment?

When a person receives training it is usually for a very specific task. Assessments seek to measure  proficiency and consistency with the performance of the task. Formative assessments are used more often as well as a set of criteria set for measuring the performance outcome of the training. It is also more likely to be a hands-on type assessment or for testing motor skills of some kind. Some examples include driving a car, becoming a bartender, or even playing a musical instrument.  

When a person receives an education the content to be learned is usually more broad and provides a person with a framework of knowledge that will also help them solve unanticipated problems later in life. It is the foundation for further learning. Both summative and formative assessments are used, however, with attention given to only a sampling of the content learned because the areas of knowledge cover are so much broader. Example of a person with an education would be teachers, psychologists and scientists. 

Although the two are quite different more often than not they are both involved in learning a certain concept or skill. Learning to read might be one example. Learning to decode words requires training and repetition in order to become proficient. However, once those strategies are known and applied consistently, they lay the foundation for more education and learning.