How can I time a participant’s progress through portions of my survey?

The problem: You want to know how long it takes participants to complete a specific section of your survey. The solution: Use calculated variables to create timestamps at selected points in the survey. This solution is fairly detailed, and a knowledge of JavaScript or JScript is helpful.

NOTE: Illume automatically records the elapsed time for all surveys. This value is stored in the variable DATSTAT.ELAPSEDTIME and appears in the data dictionary for all surveys in a collection called DATSTAT.INTERNAL. The variable does not show up in the Survey Designer, but it does show up in the Data Manager, where you can easily run queries on it.   DATSTAT.ELAPSEDTIME shows the elapsed time for the entire survey, not for individual components of the survey.

You should also keep in mind that the elapsed time may not always be accurate, because participants may stop work on a survey for 10 or 20 minutes at a time. They may even start a survey one day and then log back in three days later to complete it. In this case, the elapsed time will be three days!

Overview

Assume your survey contains three collections of questions called SECTION_ONE, SECTION_TWO and SECTION_THREE. You want to know how long it takes participants to complete SECTION_TWO. You can measure the time interval by creating three calculated variables. The first records the time at which Illume received a response to the last question in SECTION_ONE. The second records the time at which Illume received a response to the last question in SECTION_TWO. The third calculates the difference between the first two variables. In order for this work, the last questions in sections one and two must be required, and there must be at least one page break between the two questions to ensure that Illume does not receive both responses at the same time.

Creating a Timestamp

To create a timestamp that records when a question was answered, follow these steps:

1. Choose Survey / Calculations from the DatStat Illume Survey Designer menu.
2. Click the Add button to add a new calculated variable.
3. Type a descriptive name into the Unique Name field. In this example, the variable is called SECTION_ONE_END.
4. Click OK to save the calculated variable.
5. In the survey calculations editor, double-click on the name of the variable you just created. This re-opens the calculations editor. (Note: Saving and re-opening the variable may seem redundant, but is it necessary because this calculation will refer to itself.)
6. Enter the formula that appears in the image below, substituting the name of the variable you want to timestamp for the variable GENDER and the name of your calculated variable for SECTION_ONE_END. Be sure the datatype for this variable is Decimal Numbers and default value is 0 (zero).

The formula here (for cutting and pasting) is: ({Value:GENDER} != null && {Value:SECTION_ONE_END} ==
0) ? new Date().getTime() : {Value:SECTION_ONE_END};

7. Click OK to save the calculation.

Repeat steps 2-7 to create a second calculated variable to record the timestamp at the next milestone question.

Calculating the Difference Between Timestamps

You now have two timestamps: one showing when Illume received the answer to the last question in SECTION_ONE and another showing when Illume received the answer to the last question in SECTION_TWO. Follow these steps to create a third variable that calculates the difference between the two.

1. Choose Survey Add/Edit Survey Calculations… from the Survey Designer menu.
2. Click the Add… button to add a new calculated variable.
3. Type a descriptive name into the Unique Name field. In this example, the variable is called SECTION_TWO_TIME.
4. Set the data type to Decimal Numbers.
5. Use the calculation shown in the image below, substituting the name of your later timestamp variable for SECTION_TWO_END and your earlier timestamp variable for SECTION_ONE_END ({Value:SECTION_TWO_END} – {Value:SECTION_ONE_END}) / 60000

6. Click OK to save the calculation.

Details of the Formulas

The formula used to generate the timestamps above does the following:

1. The code that precedes the question mark tests to see whether the question GENDER was answered, and whether a value has been set for the calculated variable SECTION_ONE_END ({Value:GENDER} != null && {Value:SECTION_ONE_END} == 0)

2. If GENDER was answered, and SECTION_ONE_END is still equal to zero, then it’s time to set a real value for SECTION_ONE_END. Because the expression to the left of the question mark is true, the value of SECTION_ONE_END will be set to the expression between the question mark and the colon. new Date().getTime() Here, that value is new Date().getTime(), which returns the number of milliseconds between January 1, 1970 and now. Because Illume’s calculations provide full access to built-in JScript objects, you can create Date objects and call any of their methods. For information about what objects are available in JScript, see Microsoft’s JScript Language Reference. For information about the Date object in particular, see Microsoft’s documenation on the JScript Date Object.

3. If the expression before the question mark is false, which it will be before GENDER has been answered and after SECTION_ONE_TIME has been set, then SECTION_ONE_TIME is set to the value after the colon: {Value:SECTION_ONE_END} That is, it is re-set to it’s current value. This is important because calculated variables are re-calculated every time a participant moves from page to page. We do not want the timestamp to change once it has been set.

The formula used to calculate the difference between the two timestamps simply subtracts the earlier time from the later time and divides by 60000. Because the times are measured in milliseconds, the difference between the two times will be in milliseconds. To get the number of minutes, divide the number of milliseconds by 60000.