Essay / Fill Blank Offline Design

Back - Multiple Choice Up - Question Design Next - Check All

Each Test Pilot question type has a number of configurable options. Clicking on the Response Handling tab near the top of the database window bring up the first set of these options. If you wish to return to editing the question's text or associated media, click on the Text and Style tab.

columns and lines - this controls the size of the text entry area presented to the user by Test Pilot's web server extension. In the example above and shown as presented below, a single line entry area, 20 characters wide is provided. Test Pilot only automatically scores single line text entries. Larger responses for essay questions and the like may be scored online after they have been recorded. To read more about online scoring, consult the owner options section of our manual.

• default response - this allow you to enter a preselected response on the web page generated by Test Pilot's web server extension.
  
  
• correct answers - This is a list of answers considered to be correct. Each correct answer should be entered on a line by itself. Test Pilot makes comparisons ignoring character case between the user's response and the entries in this list. If you want exact character matching including upper and lower case sensitivity, be sure to check the case sensitive checkbox.
  
  
• feedback - Test Pilot can optionally return feedback based on the user's choice of response. You may enter your customized feedback in the space provided. Test Pilot will not issue this feedback unless you choose to Return Feedback to User in the Submission Handling section.
  
  
• points - you may enter the number of points you wish to award a particular answer. Test Pilot considers any positive non-zero value as a correct answer to a question. You may leave the points fields empty or zero if, as in the case of survey questions, there is no correct answer to the question. When left empty or zero, Test Pilot will refrain from automatically scoring the question. There are two convenience buttons to help you quickly assign 1 point to a particular correct answer and zero to the incorrect response.
  
  
• help - as always, you can click on the help message to view a help screen tailored to this particular database screen and it options.
  
  
• branching - if you have elected to present this assessment one question at a time, you may also enable branching which can present the next question based on the participant's response to the current question. For text questions, this is implemented as a correct/incorrect decision. That is, if the response is correct, one question is shown, while otherwise another question is shown. If either entry is left blank, the next logical question in the assessment is shown when the appropriate condition is met.

Mathematical Formula Handling

Test Pilot's most powerful features include its ability to handle questions with numeric answers and its ability to generate numeric questions whose answer is based on an evaluated mathematical expression whose terms may include randomly generated values. That is, you can author mathematical questions which change each time the Test Pilot web server generates a test. Imagine a computer lab where students all take the same Test Pilot assessment but each get different questions.

To access the formula creation options which encompass random variables, click on the Random Variables tab near the top of the database window. You can always return to the response handling or text and style options by clicking on the appropriate tab.

• random variables - each question may define up to ten random variables. These variables may only be used in the current question. Test Pilot also provides up to ten random variables that may be shared by all questions. They are defined using the Look and Feel controls. Random variables are randomized every time a Test Pilot assessment is generated. Each variable has four parameters:
  
  

1. name - this should be a unique identifier for the variable
   
   
2. start - this is a real number indicating what is the lowest value possible for the variable.
   
   
3. end - this is a real number indicating the highest value possible for the variable.
   
   
4. increment - this is a real number specifying the increment between each possible generation of random values.

In the above example a variable named r has been defined to range from 1 to 5 by 1's. That is, the possible set of values for r is the set {1,2,3,4,5}.

• text - you may edit the text of your question to include the value of any random variable in the text. To do so, you simply enter the name of the random variable surrounded by [brackets].
  
  
• evaluate this formula - to have Test Pilot evaluate a fill in the blank response as numeric data, you must check this checkbox and enter a value or formula in the space provided. In the case of a value, simply enter an integer ot real number value. In the case of a formula, you may enter a mathematical formula using the functions shown. Standard mathematical syntax and precedence rules apply. To include the value of a random variable in your expression, simply enter the variable's name surrounded by [brackets].
  
  In the example above, the formula entered is for the circumference of a circle based on a random radius variable r.
  
  
• precision - quite often the evaluation of a mathematical expression, done by Test Pilot with double precision, results in a value that is rounded by the user. Test Pilot permits you to specify a range of values that will be scored as correct. This value can be specified absolutely (e.g. plus of minus 0.001) or as a percentage error (e.g. plus of minus 1%, as shown above). Simply enter the desired precision in the space provided and use the pop-up menu shown below the value to select to use absolute or percentage-based error.
  
  
• units - if you wish to require the user to include units with their response (e.g. centimeters as in the example above), you may mark the require units checkbox and enter unit labels that are to be considered correct in the box provided. You'll need to enter each correct unit label value on a line by itself. Comparison is done ignoring case. You can enter , in the space provided, a point value to award when correct units are entered by the user. You may also specify that user responses with incorrect or missing units are to be considered incorrect regardless of their numeric values.
  

These options combine to provide the most powerful mathematical and scientific question handling capabilities of any computer-based assessment software available. To the best of our knowledge, there is no other package that offers all of these capabilities for quantitative assessment.