Using Calculated Variables

Illume calculations are algorithms that are run within a survey, and persist within the survey data set.  Calculations are run on every page, i.e. every time the respondent clicks the Next button, all calculations are re-run.  A survey may contain multiple calculations, and calculations can include other calculations.  Calculations are processed in the order that they appear in the calculations editor, which is particularly relevant when calculations are based on the results of other calculations.

There are a number of valuable uses for calculations.  Calculations can be used to do the following (this list is not all inclusive):

1. Create scores
2. Grab the current date, time, year, or other local information from the respondent’s device
3. Calculate new variables from survey variables, e.g. calculating a participant’s age from their entered Date of Birth
4. Output text.  For example, a survey may ask for the respondent’s child’s gender.  Throughout the survey you may want to include questions such as “how old was he/she when he/she learned to walk”.  Rather than saying “he/she”, you can create a calculation that outputs the word “he” if Gender=male, and outputs “she” if Gender equals female.
5. Set the value of a Preload/Hidden variable or other survey variable

For each calculation, Illume creates a variable with the same name as the calculation. Illume attempts to evaluate all of the calculations in the survey each time a participant moves from one page to the next.   Illume relies on piping to grab the value of variables used in the calculation.  For example, if a participant entered “200” in response to the question named WEIGHT that asked “What is your weight (in pounds)?”, Illume substitutes 200 for the tag {Value:WEIGHT}.

Illume will perform value substitutions (aka piping) on any variables for which it has data. It does not matter whether that data came from a participant response, from a variable appended to the survey URL, or from data that was pre-loaded from the participant list.  If Illume cannot perform the calculation, the value of the calculation will be the default value set for the calculation in the Survey Calculations Editor.  If no default value is set, and Illume cannot perform the calculation, the value for the calculation will be null (empty).

There are three conditions under which Illume cannot perform a calculation:

1. The expression is invalid. The Survey Calculations Editor will tell you if your calculation is invalid before you save it.
2. The expression includes variables whose values are unanswered or unknown.
3. The calculation would result in an error. The most common causes of this are attempting divide by zero and supplying a value of the wrong data type. For example, if your calculation is {Value:HEIGHT} / {Value:WEIGHT}, design the survey to allow only numeric values for the height and weight questions. Illume can divide by the numeric value “150” but it cannot divide by the text value “one hundred and fifty.”

The calculation itself is a JScript.NET expression, or a series of JScript.NET statements. Illume evaluates the JScript.NET using the eval() function and sets the value of the calculated variable to whatever eval() returns.

There are two things to note about eval() in JScript.NET:

1. eval() returns the value of the last expression in the script.
2. eval() does not permit a return statement.

EXAMPLE:  The following calculation determines the total price for a quantity of some material, then applies a discount of 5%:

var weightInGrams = {Value:KILOGRAMS} * 1000;

var pricePerGram = 0.13;

var totalPrice = weightInGrams * pricePerGram;

var discount = 0.05;

totalPrice * (1 – discount);

Notice that the value to be returned is expressed on the last line as totalPrice * (1 – discount), and that there is no return statement. Because JScript’s eval() returns the value of the last expression, the result of the calculation will be the value of the last expression in the calculation.

Calculation Operators

The following operators are available for calculated variables:

Arithmetic Operators

• + (Plus Sign) Used for addition.
• – (Minus Sign) Used for subtraction.
• * (Asterisk) Used for multiplication.
• / (Forward Slash) Used for division.
• ^ (Carat) Raises a number to a power. E.g. 5^3 is 5 to the third power, or 125.
• % (Percent Sign) Modulo operator, used to find the remainder of a division operation

Comparison Operators

• == Is equal.
• != Is not equal.
• < Is less than.
• Is greater than
• <= Is less than or equal to
• = Is greater than or equal to

Logical Operators

• && And
• || Or
• ?: If/then/else

To define a calculated variable, follow these steps:

1. Choose Survey / Calculations from the DatStat Illume Survey Designer menu
2. Click the Add button to add a new calculated variable, or double click on the name of an existing calculation to edit it.  You may also copy and paste an existing calculation.

3. In the Calculation Editor, give the calculated variable a unique name if it does not already have one. Provide a description to help users remember what this variable calculates.
4. Choose a data type. The data type to choose depends on what result is expected from the expression. Mathematical operations and any true/false expression produce numbers, so these require a numeric data type. (True expressions are equal to 1; false expressions are equal to 0.) Expressions that return text data require a Text data type.
5. Assign an optional default value. This will be the value of the calculation when there are not enough data to perform the actual calculation. For example, if the calculation is the sum of A, B and C, the calculation will contain the default value until the participant has answered questions A, B and C.
6. Click Insert Expression to select survey variables to include in the calculation.
   • NOTE:  Whereas in other places in a survey you can pipe the value of a variable using a single letter tag such as “V” for “Value”, in the Calculations editor you must write out the tag fully, thus Value is acceptable but V is not.
7. In the Expression Calculator, choose the variables required to perform the calculation. To select more than one variable, hold down the Control key while you click on each variable to include. Note that the selected variables appear in an expression below the variable list. The expression uses only addition. This can be changed going forward.

8. After choosing all of the variables to include in the calculation, click Paste. The expression will be pasted into the Calculation Editor. Beneath the Insert Expression button, the message “Calculation is valid and evaluated successfully.” should be visible.
9. Now edit the calculation by hand. Illume will evaluate any simple and complex JScript.NET expressions. Expressions cannot include user-defined objects or functions, but they can include built-in JScript.NET objects, datatypes, methods and operators. Note JScript.NET is the version of JavaScript used by Microsoft Windows and Internet Explorer. It is essentially a superset of standard JavaScript (a.k.a. ECMAScript). See Microsoft’s JScript.NET Language Reference for more information about JScript.NET. If there is a problem with the calculation, a message describing the problem will appear below the Insert Expression button, and the offending part of the calculation will be highlighted in red. If the calculation is valid JScript.NET but the calculation editor does not have enough information to evaluate it, you’ll see the message Calculation is valid but failed to evaluate. The calculation will be evaluated while the survey is running, if there is enough information to perform the evaluation. (I.e., if the participant answered the questions to which the expression refers.)

10. Click OK .

NOTE: Illume uses parentheses to determine precedence when evaluating calculations.

Illume supports complex expressions that use the ?: operator.

This operator tests an expression and returns one of two values, based on whether the expression is true or false. If the expression before the question mark is true, the expression returns the value before the colon. If it is false, it returns the value after the colon.

A simple way to remember this syntax is: Is this true ? “Yes Value” : “No Value”;

Multiple complex expressions can be included within a single calculation:

For example, this calculation which calculates the cost of an order as the number of items times the cost per item, plus a fixed rate of either $20 or $12 or $8, based on the type of shipping: {Value:NumberOfItems} * {Value:CostPerItem} + ({Value:Shipping} == “Overnight” ? 20 : 0) + ({Value:Shipping} == “3-day” ? 12 : 0) + ({Value:Shipping} == “USPS” ? 8 : 0) Assuming that the Shipping question is required and that participants can choose only one shipping option, one of the three complex expressions will be true, and the other two will be false. The true expression will add either 20 or 12 or 8 to the value of the calculation. The two false expressions will both add 0 to the calculation. Complex expressions are especially useful in calculations that must be applied differently to different participants. For example, calculations of lean body weight and blood alcohol content use different constants for men and women.

See Microsoft’s JScript Language Reference for more information about JScript.

Arithmetic Operators

+ (Plus Sign) Used for addition.

– (Minus Sign) Used for subtraction.

* (Asterisk) Used for multiplication.

/ (Forward Slash) Used for division.

Raising to a Power

math.pow({value:VARNAME}, power);

Comparison Operators

== Is equal.

!= Is not equal.

< Is less than.

> Is greater than

<= Is less than or equal to

>= Is greater than or equal to

Logical Operators

&& And

|| Or

?: If/then/else

Rscore and Score tags

While the {Value} tag yields a question’s selected response value, the {RScore} tag yields the reverse score.

To get a question’s reverse score, use the {RScore} with the question id. For example, to get the reverse score of the question SATISFACTION, use the tag {RScore:Satisfaction}.

Reverse scoring is generally used on questions that have both negative and positive items in the scale, though Illume permits reverse scoring of any items.

For questions with numeric scales, the reverse score is the score in the same position at the opposite end of the scale. The following tables illustrate reverse scores for items with numeric scales.

For Check all that apply questions, the {Score} or {Value} tag gives the number of items checked. The {RScore} tag gives the number of items NOT checked.

For individual checkboxes, the {Score} tag yields a value of 1 if the item is checked and 0 if the item is not checked. The {RScore} yields the opposite values.

For non-question items (such as calculations and pre-loaded variables), the {Score} tag yields the item’s value. The {RScore} tag always yields one of the “not set” values described below.

Reverse Scoring Values for Unanswered/Unset Items

If a question has not been answered, or has no value, the {RScore} will return one of the following values, depending on the item’s data type:

• For all Numeric items that are unanswered or have no value, the {RScore} will be 0 (zero).
• For all Yes/No items that are unanswered or have no value, the {RScore} will be 0 (zero). (Note that zero is equivalent to an answer of False or No in most computer systems and applications.)
• For all Date, Time and Date/Time items that are unanswered or have no value, the {RScore} will be the current date with hours, minutes and seconds set to zero. (E.g. June 23, 2005 00:00:00.)
• For all Text types that are unanswered or have no value, the {RScore} will be an empty string (“”)– that is, a text value with a length of zero.

Calculating and Outputting Text

When text is referenced in a calculation, it is put in parenthesis, as in the examples below.

({Value:Q1}==”California”)?1:0

({Value:gender})==1)?”he”:”she”

Complex Calcs

More complex Jscript objects can be used within these calculations. A good resource for information regarding this can be found at the following URL:

http://msdn.microsoft.com/en-us/library/ye921ye4%28v=VS.80%29.aspx

Though JScript.NET can be used in date object calculations, date and time calculations can be tricky. Calculations that use the current time must take into account the fact that Illume re-calculates calculated variables every time participants move from one page to the next.

Date calculations are tricky because programmers often make incorrect assumptions about units of time. For example, because the JScript date object uses milliseconds as its unit of time, you might think it’s easy to measure a span of years by converting years to milliseconds and performing simple arithmetic on the milliseconds. This may lead to incorrect calculations, since leap years are longer than standard years.

Months are notoriously irregular. They can have 30 days, or 31, or 28, or in some cases, 29.  Days irregular as well. In most parts of the US, the first Sunday in April is a 23-hour day, and the last Sunday in October is a 25-hour day.

Below are some examples utilizing Dates.  The following JScript calculates a participant’s age on the date of the survey.   This calculation assumes that the variable BIRTHDATE is a variable in the survey that asks for the participant’s date of birth, and that variable is of data type Date:

var today = new Date();

var dob = new Date({Value:BIRTHDATE});

var yearDiff = today.getFullYear() – dob.getFullYear(); var monthDiff = today.getMonth() – dob.getMonth();

var dayDiff = today.getDate() – dob.getDate();

if(monthDiff < 0 || (monthDiff == 0 && dayDiff < 0))

 yearDiff–;

yearDiff;

If BIRTHDATE were a text variable, not a Date data type variable, and had the format mm/dd/yyyy, the calculation would be as follows:

var today = new Date();

var dob = {Response:BIRTHDATE} + “”;

var dateParts = dob.split(/\D/);

var month = parseInt(dateParts[0], 10);

var day = parseInt(dateParts[1], 10);

var year = parseInt(dateParts[2], 10);

if(year < 100)

 year += 1900;

var yearDiff = today.getFullYear() – year;

var monthDiff = today.getMonth() – ( month – 1);

var dayDiff = today.getDate() – day;

if(monthDiff < 0 || (monthDiff == 0 && dayDiff < 0))

 yearDiff–;

yearDiff;

NOTE: The value of the value of the calculated variable is set to the value of the last expression evaluated in the JScript. In the examples above, that is the variable yeardiff. Keep in mind that because Illume uses eval() to evaluate the JScript in calculated variables, a return at the end of the script is not needed.

Illume calculations use parentheses to determine which parts of a calculation should be evaluated first. If the calculation uses only addition and subtraction, parentheses do not matter. If the calculation uses multiplication or division, parentheses matter very much.

Illume will scan a calculation from right to left, evaluating all expressions in parentheses, followed by all multiplication operations, then all division operations, then all addition operations, and finally all subtraction operations.

Note the following examples:

• ({Value:SALARY} + {Value:BONUS} + {Value:OTHERINCOME})
  • This calculation simply adds together a participant’s salary, bonus, and other income.
• ({Value:SALARY} + {Value:BONUS} * {Value:TAXRATE})
  • This would be an incorrect calculation for the amount of income tax a person pays. If salary is 40000 and bonus is 2000 and tax rate is 0.28, this calculation will produce the following result:

40000 + (2000 * 0.28) = 40000 + 560 = 40560

The correct calculation uses parentheses to add salary and bonus before applying the tax rate.

(({Value:SALARY} + {Value:BONUS}) * {Value:TAXRATE})

Which works out as follows:

(40000 + 2000) * 0.28 = 42000 * 0.28 = 11760

Calculations can be used to test and assemble text values. The following expression tests whether someone typed “California” in response to question Q2:

{Value:Q2} == “California”?1:0

This expression returns a whole number: 1 if the value does equal “California” and 0 if the value does not equal California. Text calculations are not case sensitive.  This means the calculation above will be true whether the participant typed California, california, or CALIFORNIA.

The expression below creates an email address by adding @company.com to whatever the user typed in response to the LASTNAME question:

({Value:LASTNAME} + “@company.com”)

The survey calculation DatStat Object consists of many very useful methods that can be called from survey calculations.  These methods allow for more capability and flexibility within a survey calculation.  Note: The survey calculation editor does not currently color code any of the DatStat object methods.  Variable name references used in the DatStat object methods will not be updated when using the survey rename capability unlike the tags (e.g. {Value:GENDER}).

How to Use

Almost all of the DatStat object methods share the same signatures as those used by our runtime SDK.  To make a call just prefix the method or property with “DatStat.” (case sensitive).  The methods are also data-type aware so make sure to pass variables of the proper type to them (eg if the method is expecting an integer, do not pass a string to it).

Reference Guide

The problem: You want to know how long it takes participants to complete a specific section of your survey. The solution: Use calculated variables to create timestamps at selected points in the survey. This solution is fairly detailed, and a knowledge of JavaScript or JScript is helpful.

NOTE: Illume automatically records the elapsed time for all surveys. This value is stored in the variable DATSTAT.ELAPSEDTIME and appears in the data dictionary for all surveys in a collection called DATSTAT.INTERNAL. The variable does not show up in the Survey Designer, but it does show up in the Data Manager, where you can easily run queries on it.   DATSTAT.ELAPSEDTIME shows the elapsed time for the entire survey, not for individual components of the survey.

You should also keep in mind that the elapsed time may not always be accurate, because participants may stop work on a survey for 10 or 20 minutes at a time. They may even start a survey one day and then log back in three days later to complete it. In this case, the elapsed time will be three days!

Overview

Assume your survey contains three collections of questions called SECTION_ONE, SECTION_TWO and SECTION_THREE. You want to know how long it takes participants to complete SECTION_TWO. You can measure the time interval by creating three calculated variables. The first records the time at which Illume received a response to the last question in SECTION_ONE. The second records the time at which Illume received a response to the last question in SECTION_TWO. The third calculates the difference between the first two variables. In order for this work, the last questions in sections one and two must be required, and there must be at least one page break between the two questions to ensure that Illume does not receive both responses at the same time.

Creating a Timestamp

To create a timestamp that records when a question was answered, follow these steps:

1. Choose Survey / Calculations from the DatStat Illume Survey Designer menu.
2. Click the Add button to add a new calculated variable.
3. Type a descriptive name into the Unique Name field. In this example, the variable is called SECTION_ONE_END.
4. Click OK to save the calculated variable.
5. In the survey calculations editor, double-click on the name of the variable you just created. This re-opens the calculations editor. (Note: Saving and re-opening the variable may seem redundant, but is it necessary because this calculation will refer to itself.)
6. Enter the formula that appears in the image below, substituting the name of the variable you want to timestamp for the variable GENDER and the name of your calculated variable for SECTION_ONE_END. Be sure the datatype for this variable is Decimal Numbers and default value is 0 (zero).

The formula here (for cutting and pasting) is: ({Value:GENDER} != null && {Value:SECTION_ONE_END} ==
0) ? new Date().getTime() : {Value:SECTION_ONE_END};

7. Click OK to save the calculation.

Repeat steps 2-7 to create a second calculated variable to record the timestamp at the next milestone question.

Calculating the Difference Between Timestamps

You now have two timestamps: one showing when Illume received the answer to the last question in SECTION_ONE and another showing when Illume received the answer to the last question in SECTION_TWO. Follow these steps to create a third variable that calculates the difference between the two.

1. Choose Survey Add/Edit Survey Calculations… from the Survey Designer menu.
2. Click the Add… button to add a new calculated variable.
3. Type a descriptive name into the Unique Name field. In this example, the variable is called SECTION_TWO_TIME.
4. Set the data type to Decimal Numbers.
5. Use the calculation shown in the image below, substituting the name of your later timestamp variable for SECTION_TWO_END and your earlier timestamp variable for SECTION_ONE_END ({Value:SECTION_TWO_END} – {Value:SECTION_ONE_END}) / 60000

6. Click OK to save the calculation.

Details of the Formulas

The formula used to generate the timestamps above does the following:

1. The code that precedes the question mark tests to see whether the question GENDER was answered, and whether a value has been set for the calculated variable SECTION_ONE_END ({Value:GENDER} != null && {Value:SECTION_ONE_END} == 0)

2. If GENDER was answered, and SECTION_ONE_END is still equal to zero, then it’s time to set a real value for SECTION_ONE_END. Because the expression to the left of the question mark is true, the value of SECTION_ONE_END will be set to the expression between the question mark and the colon. new Date().getTime() Here, that value is new Date().getTime(), which returns the number of milliseconds between January 1, 1970 and now. Because Illume’s calculations provide full access to built-in JScript objects, you can create Date objects and call any of their methods. For information about what objects are available in JScript, see Microsoft’s JScript Language Reference. For information about the Date object in particular, see Microsoft’s documenation on the JScript Date Object.

3. If the expression before the question mark is false, which it will be before GENDER has been answered and after SECTION_ONE_TIME has been set, then SECTION_ONE_TIME is set to the value after the colon: {Value:SECTION_ONE_END} That is, it is re-set to it’s current value. This is important because calculated variables are re-calculated every time a participant moves from page to page. We do not want the timestamp to change once it has been set.

The formula used to calculate the difference between the two timestamps simply subtracts the earlier time from the later time and divides by 60000. Because the times are measured in milliseconds, the difference between the two times will be in milliseconds. To get the number of minutes, divide the number of milliseconds by 60000.