Recently in validations Category

One of the really cool aspects of XFA forms is the dependency tracking mechanism.  Dependency tracking is the feature where your field (and subform) calculations and validation re-fire whenever any of their dependents change.  Today I'll explain a bit about the mechanics of how dependency tracking is implemented as well as have a look at possible design issues related to this mechanism.

Simple Example

Suppose my purchase order form has order rows with fields: price, quantity and subtotal along with a total field.

The subtotal field has a JavaScript calculation: "price.rawValue * quantity.rawValue"

The total field has a FormCalc calculation: "sum(order[*].subtotal])"

Now whenever price or quantity fields change, the corresponding subtotal field will be recalculated automatically.
Whenever any of the subtotal field values change, the total calculation will re-fire.

Discovering Dependencies

When the form is first opened, all the calculations and validations are executed.  While executing a calculation or validation, the XFA engine keeps track of each object (field, subform, data node) that gets referenced during the script execution.  Each referenced object is added to its dependency list.  In our example, each subtotal field becomes dependent on the price and quantity fields from its row.  The total field becomes dependent on all the order subforms and on all the subtotal fields.  The mechanism is robust enough that even if you reference your objects indirectly, we still find the dependency.  e.g. the subtotal calculation could be written as:

parent.nodes.item(0).rawValue + parent.nodes.item(1).rawValue

Since the nodes we visited were still price and quantity, the dependencies are established just the same.

In some cases, the dependency list will grow as values change.  Consider this calculation:

if (A.rawValue < 100) {
    A.rawValue;
} else {
    B.rawValue;
}

Suppose the first time it executes, field A has a value of 20.  This means the code will not execute the else clause and will not access field B.  The initial dependency list will include only A.  However, if the A of changes to 200, the calculation will re-fire; the else clause will be evaluated and field B will be added to the dependency list.

If your calculation or validation calls a script object, dependencies will continue to be tracked during the execution of the script object method.

Dependent Properties

What constitutes a field change? What changes cause a calculation to re-fire?  I don't have a complete list.  But changing a field's visual properties (colours, captions, presence) do not cause a dependent to recalculate.  Changing the value or changing the items in a list box will cause a dependent calculation/validation to re-fire.

Turn off Automatic Calculations

If, for some reason, you're not happy with the default dependency tracking, then you can turn it off. There are two switches to control calculations and validations independently:

xfa.host.calculationsEnabled = false;
xfa.host.validationsEnabled = false;

Note that turning validationsEnabled off disables not only script validations, but also turns off mandatory field and picture clause validations.

Dependency Loops

It is possible to get into a situation where sequence of dependency changes goes into an infinite loop. In these cases, the XFA engine will allow the calculation or validation to re-fire a fixed number of times (10) and then will quietly stop re-firing the calculation or validation until the next time a dependent changes.  While the 10 times limit is ok in most cases, I have seen forms where this has been the root of performance issues.  If you have a dependency loop and your calculations are re-firing, you want to be aware of it and you need to fix it. 

Dependency loops are typically introduced when a calculation or validation modifies another field.

I should highlight that statement.  If your calculation or validation minds its own business and never modifies another field you shouldn't run into any loops.  There are lots of cases where a calculation or validation modifies another field and everything works fine -- but tread carefully.

Looping example

In order to track when my validations were changing, I wrote validation scripts write to a debug field:

field T1 validation:

debugTrace.rawValue += this.rawValue;
true;

field T2 validation:

debugTrace.rawValue += this.rawValue;
true;

Both T1 and T2 now have a dependency on field: debugTrace

The sequence of events:

1. Field T1 changes
2. T1 validation fires and modifies debugTrace
3. T2 validation fires and modifies debugTrace
4. T1 validation fires and modifies debugTrace
5. T2 validation fires and modifies debugTrace
6. ...

Here is a sample form to illustrate this example.

Note that if we remove the validation from T2, the circularity stops.  The validation on T1 modifies debugTrace, but after the validation completes, debugTrace does not change and T1's validation does not re-fire.

There is one question I get asked frequently enough that it warrants its own blog entry.

Someone tries a validation script that looks like this:

this.rawValue !== null;

But the validation never fails.  In spite of the fact that the field is null, this field always reports back that it is valid.

The reason is because of this rule: "empty/null fields never fail a script validation".  The reason behind the rule is that we want to allow designers to define validations without worrying about null as a special case.  Take an example:  the user needs to enter a value in a 'Age' field where the value must be greater than 18.  The way to express this is with this validation on the Age field:

this.rawValue > 18;

In an empty field this evaluates as "null > 18;" -- which evaluates to false.  Until the user enters a value, this validation script will always return false.

But allowing a script validation to fail on an empty field would be a lousy user experience.  As soon as they open a blank form they would get a pile of invalid fields -- even before beginning to enter data.  What should a form designer do?  They could change their script to explicitly allow null:

(this.rawValue === null) || (this.rawValue > 18);

But now there's the problem that we really do want to make sure they enter a value.  i.e. the experience we want is that the field starts out in a valid state.  Don't complain about script validations until they've actually entered a value in the field -- but also don't let them submit their data until the field has been populated with a value!

Of course, the answer is to make the field mandatory (the null test validation).  By making the Age field mandatory we know that the user will not be hassled as soon as they open the form; but they also won't be allowed to submit until they've provided a value.  And the form author doesn't have to handle a null value as a special case.

-----

Gone Again

I see Merriam-Webster has added "Staycation" to their dictionary.  Sounds good to me.  I will be away for another week at the family cottage.  I'll be be back to respond to comments on the 27th or 28th.

This post will be in the deep end for many of you.  But for those who design forms that extract position information from the xfa.layout object, you might well find some answers to longstanding problems -- especially if you make use of multiple content areas on your master pages.

This blog entry deals specifically with the functions:
xfa.layout.x(), xfa.layout.y(), xfa.layout.h(), xfa.layout.w().

Return Value

The description for layout.x() says: "Determines the x coordinate of a given form design object".  This is true.  Incomplete, but true.  In fact, the x() and y() methods return a coordinate relative to the parent of the object.  The offset of the root subform is relative to the content area.  To find an absolute page coordinate, you could (in theory) traverse up the hierarchy and add the x coordinate of each ancestor plus the x coordinate of the content area.  But the APIs don't make this easy.  First of all, it's not obvious which content area contains your object.  And if your ancestor subform originated on a different page, it will have a different offset parameter.  Finding an absolute page position is complicated.  But not impossible.

Parameters

The layout position methods all take the same set of parameters.
e.g. xfa.layout.x(<object>, <units>, <offset>);

object is self evident.  You pass in a reference to some container (field, subform, draw etc).

units is also easy. "in" will cause the function to return a numeric value representing inches.

offset is ... complicated.  The documentation for xfa.layout.x() says:

"An integer representing the number of pages to offset the x coordinate of the object, beginning with the first page the object occurs on. If left blank, the default value is 0."

This is partially true.  It is true only in the case where there is one content area per page.  The full truth is more complicated.  The documentation for xfa.layout.h() says:

"An integer representing the amount to offset the height value of a form design object, beginning with the first page the object occurs on. If left blank, the default value is 0."

Hmm.  Not true at all.  It's not an offset of the height value.  Someone forgot to copy/paste the part of the description that says "number of pages".  We'll work toward a more accurate description in the next section...

Multiple Content Areas

For forms with multiple content areas, here is the actual behaviour of the offset parameter:

The offset parameter indicates which relative content area the object appears in.  e.g. If a subform spans 2 pages and 5 content areas, you can call xfa.layout.h with offset values of 0, 1, 2, 3, 4 to return the the height of the object in each of the 5 content areas. 

This is great but, now the problem becomes: how do we find out which actual content area the object is in?  You can't assume that the object appears in each content area on a page.  A subform with an explicit overflow target might skip a content area on any given page.

Fortunately, there is a heuristic that allows us to infer the content area in which an object appears. 
When you make a call to: xfa.layout.pageContent(0, "", false); you get a list of all objects on a page -- including the content area objects.  If a field or subform appears in more than one content area on a page, that object will appear multiple times in the returned list.  The list appears in layout-order. Content area objects appear before the objects that have been placed in that content area.  To find out which content area an object appears in, look back up the list to find the most recent content area.

Finding Absolute Coordinates

Finding the x coordinate of an object involves adding the x coordinates of all the ancestors plus the x coordinate of the content area.  But, as mentioned above, the ancestor subforms may have originated in a different content area.  In order to add up the x coordinates, you have to find out the offset value of your parent object in this content area.  The script code to figure this all out is pretty complex.  I've put all the complexity under a script object that you can call:

// This function returns an array of extents for a given object
// There will be one entry in the array for each content area
// an object appears in.
// Each entry is an object with these properties:
//   Extent.page
//   Extent.contentArea  (SOM expression)
//   Extent.x            (absolute page position)
//   Extent.y
//   Extent.w
//   Extent.h  
// All measurements are inches.
function getExtents(vObject)

I have attached a sample form with multiple pages, multiple content areas, and which displays the results of querying layout positions.

Application

Once you're able to find page positions, how can you use that information? Placing transpromo content is the best example.  I have also developed another sample where I used a different kind of field highlighting.  For each field error, the form places a "highlighter" subform on the master page (the highlighter is an arrow).  When you open the form, click on the "highlight errors" button.

I have another purpose in mind, but that's the topic for a future blog post...

This post describes the last of the enhancements targeting form authoring usability in XFA 3.0/Acrobat 9.1.

The subform object has a new script method:

<container list> = subform.getInvalidObjects();

This function returns a list of all the descendant containers (field, exclGroup, subform) of a subform which are in an invalid state.  If the subform that this script method is called on is itself invalid, that subform will be included in the returned list. The list is only generated on demand by recursively traversing the subform. The list returned is in document order.

This function will allow form authors to selectively highlight errors on sections of their form.  For example, in the "exit" event of a subform, you could highlight the missing or invalid fields contained within that subform, and direct the form filler to complete that section before continuing with the rest of the form.

As mentioned in yesterday's post, missing mandatory fields are not considered invalid until they have been edited or there has been a call to execValidate().  To be consistent, the call to getInvalidObjects() respects the same rule.  The returned list will not include missing mandatory fields until the XFA processor considers them invalid. 

One of the places where the list of invalid fields is useful is during the submit process.  Form authors can use the preSubmit event to detect invalid fields, notify the user and cancel the submit action.  Note that submit performs a call to execValidate() -- but only after preSubmit has fired.  This means that a call to getInvalidObjects() in the preSubmit event might not include missing mandatory fields.  If you want to be sure those fields included, simply call execValidate() before the call to getInvalidObjects().

It will be some time before form authors can really take advantage of the XFA 3.0 enhancements -- mainly because it will be a while before Acrobat/Reader 9.1 is on enough desktops.  In the meantime, I expect authors will continue to make use of JavaScript frameworks to gain enough control over the user experience.  If you are new to the idea of JavaScript frameworks, have a look at the one that I've included in various samples.  The framework is described in a series of posts: Validation Patterns: Part 1, Part 2, Part 3 and here. The most recent version of the framework is embedded in the samples for this post. 

The framework was designed to:

• Imitate the "form authoring usability" enhancements described in the last few XFA 3.0 blog postings
• Leverage as much of the native XFA validation mechanism as possible for controlling validations
• Over time, be replaced by the new built-in XFA 3.0 capabilities

I anticipate one one more post on XFA 3.0 enhancements and then back to regular programming.

The field, exclGroup and subform elements have a new event: validationState.  The primary use of this event is to change the appearance of fields as they become valid or invalid.

Just for review -- there are three ways in which objects can be considered invalid:

1. A mandatory field/exclusion group that does not have a value
2. A field with a value that cannot be formatted with its validation picture format
3. A field/subform/exclusion group whose validation script returns "false"

Rules controlling the validationState Event

The validationState event fires under the following conditions:

• Immediately following the initialize event -- allowing the form author to control the initial appearance of the object
• When the field/subform/exclusion group transitions between a valid and invalid state (or vice versa).
• When the reason for the invalid state changes e.g. a field is in an invalid state because it is not filled in (missing mandatory).  The user then fills in a value that violates the validation picture clause.  In this case the field goes from one invalid state to a different invalid state and the event will fire.
• The form session begins tracking mandatory fields

The last point needs a bit more explanation.  When a field is marked as mandatory, it starts the form session in a valid state -- even though it does not have a value.  The reason is that we don't want users notified about a whole bunch of invalid empty fields as soon as they open a blank form.  There are 3 things that will put a missing mandatory field into an invalid state:

1. A populated mandatory field gets emptied
2. A call to execValidate() on the field or one of its ancestors
3. A submit action (which is really the same as an execValidate())

When any one of these three operations happen, the validationState event will fire for any unpopulated mandatory fields.

To be on the safe side, script associated with the validationState event should be robust enough to allow the event to fire multiple times -- even if the validation state hasn't actually changed.

The new errorText property

So the validationState event has fired... how does the script author know whether the field is now in a valid or invalid state?  Authors can check the new property on field/subform/exclusion group: errorText.  When a field is in a valid state, this property will return an empty string.  When in an invalid state, it will return the user-defined message for that error. 

Objects may have 3 different user-defined messages -- one for each kind of validation failure.  These messages are represented in the scripting model as:

field.validate.message.scriptTest.value
field.validate.message.formatTest.value
field.validate.message.nullTest.value

or via their shortcut properties:

field.validationMessage
field.formatMessage
field.mandatoryMessage

The errorText property will be populated with the message from the associated validation failure.  If the user has not defined a message, a default message is provided ("Validation failed.").

Turning off Acrobat/Reader Field Highlighting

If you are using the validationState event to change the appearance of fields, the automatic field highlighting of Acrobat/Reader can get in the way.  Fortunately, it can be turned off.  This script (placed on a form:ready event) will do the trick:

    if (xfa.host.name == "Acrobat")
       app.runtimeHighlight = false;

Example

This example will hide the border on valid fields and for invalid fields will colour the border red.  Note that the example places the script on a subform and uses event propagation to cause it to apply to all nested objects.

<subform>
  <event activity="validationState" listen="refAndDescendents">
    <script>
      if (HasValue($event.target.errorText)) then
        $event.target.edge.presence = "hidden";
      else
        $event.target.edge.presence = "visible";
        $event.target.borderColor = "255,0,0"
      endif
    </script>
  </event>
</subform>