Fix Layout Problems

This is the fourth of four exercises that were part of the LiveCycle best practices lab at MAX.

The purpose of this exercise is to fix layout problems by analyzing their PDF with XFADebugger.pdf.

In Designer, open: EX4 LayoutProblem.pdf (with preview data file: afiList.xml)
In preview, notice that :

• Content was pushed off the first page
• The second page is truncated

Open XFADebugger.pdf in Acrobat. Use the “Load XFA/PDF file” to open EX4 LayoutProblem.pdf.

• Explore the content in the debugger view. Expand the trees. Click on nodes.
• Click the “Find Warnings” button several times to see various warnings reported.
• Fix the warnings in Designer

Here is the file with the layout problems fixed: EX4 LayoutProblem (Solution).pdf

Script Performance Exercise

I enjoyed attending MAX last week. The best part was putting some faces to names of people who had previously been cyber-personalities.

We did a pre-conference lab on LiveCycle best practices. For the lab I prepared four exercises on various aspects of form design.  For those who weren’t able to attend, I’ll post those exercises as a series of blog entries.

Exercise #1: Measure and Improve JavaScript Performance

In this exercise we optimized a fairly simple script – a few lines in a loop. Depending on your machine, you should be able to improve the performance anywhere from a factor of three up to a factor of seven.

Open EX1 Performance Tests.pdf in Designer.

There are 50 subtotal values that need to be totaled. Each total has a test button that executes the Total script 500 times and measures the performance.

Note the 4 variations of the sum function:

• Test 1: The original (slow) version of the calculation
• Test 2: A copy of Test 1 that you will improve.
Look at the hints in the code comments in order to improve the script.
• Test 3: The solution
• Test4: Same calculation expressed in FormCalc

Understanding the solution:

• Undeclared variables are very expensive to reference
• Javascript variables are far less expensive to reference than XFA objects
• Evaluate as few dots as possible. e.g. c.d is faster than a.b.c.d
• Dots evaluated inside a resolveNode(s) expression are faster than dots evaluated in JavaScript objects. E.g. resolveNode("a.b.c") is faster than a.b.c

Why is FormCalc faster?

All references to XFA objects from JavaScript involve a round-trip from the script environment to the XFA processor and back. These round trips are expensive.

FormCalc runs native in the XFA processor and there are no round-tripping costs.

A Runtime Accessibility Checker

I continue to be interested in the challenges of developing forms that give a good experience for visually impaired users using screen readers.  Previously we have focussed on checking accessibility properties at design time. However, analysing the form design template doesn’t always fit the user workflow.  There are cases where accessibility properties are populated at runtime by script or by data binding. In this case, these properties are un-populated in the design version and consequently, the design-time accessibility checker will give warnings — yet at runtime the accessibility experience may be just fine.
In addition, we want to make sure that the accessibility experience adapts to changes in dynamic forms. We need to be able to test accessibility in various states — e.g. after subforms are added or removed.

The obvious answer is to perform accessibility checking at runtime — while the form is open in Reader or Acrobat.  Today we’ll look at a solution to that problem.  A mechanism for instrumenting your forms to allow you to analyse the state of accessibility information at runtime.

The Solution

For those of you who want to use the solution without necessarily understanding how it works, follow these steps:

• Drag this fragment anywhere onto your form.  It will add a hidden subform with no fields (just script)
• With the form open in Reader/Acrobat, paste the value “AccCheck” into any field on the form
• From the resulting dialog, select which accessibility checks to run
• Copy the results from the report dialog for future reference
• All elements with accessibility issues will be re-drawn with a 2pt red border.  If the element is a field or draw, it will also be shaded light red and the accessibility error text will be assigned to the tooltip.
• Close your form and go back to Designer to fix any identified issues

Try it out with this sample form. Note that this form dynamically populates some properties.  The Designer-based accessibility checker will report twice the number of errors.

Hopefully the embedded checker is innocuous enough that you could leave it in a production form.   But if not, you should be able to inject it during your testing phase and remove it when you’re ready to deploy your form.

The Deep End

For those who want to know how this works…

The challenge is to add something to your form without introducing an observer effect — i.e. analytics that can run without changes to your form UI and without impacting normal form processing.

The runtime accessibility checker is designed as a form fragment that you can drag onto any form.  The fragment is a single, hidden subform with no fields, no data binding — just script.
We want to trigger the analysis script without requiring the user add a “check accessibility” button to their form.  In this case, I added a propagating change event to the fragment subform.  This script checks the xfa.event.change event property.  If the change text is equal to “AccCheck”, we launch the accessibility checker.
The checker uses JavaScript dialogs to prompt for checks to perform and has a second dialog to display the results.
The JavaScript performing the checks is adapted from the Designer version of the accessibility checker.  The main difference is that it analyses the Form DOM (the runtime) instead of the Template DOM (the design).

Updated Form Report Tool

Has it been two years already?  Seems like just the other day I published a tool for summarizing form content.

I’m sure it’s been sitting on your desktop for regular use.  In my day job I’m, often asked to look at customer forms.  Or I sometimes need to go back and review one of my own form designs.  The first thing I always want to know is the "big picture" of what’s inside this form.  That’s what the summary tool is all about. You’d be surprised what you can learn from a report. I’ll hopefully point out a thing or two in follow-up blog posts.

Now that we have a macro capability in Designer (yes, yes, still in beta) it’s time to migrate this functionality from being an external tool to a Designer macro.

Here are the files you need: FormReport.pdf and FormReport.txt (rename to FormReport.js).  If you need a reminder about how to install them with Designer, then read this blog entry.

Debug merge and layout: Updated

Almost two years ago I published a blog post with a tool that I developed to help debug data merge and layout problems: http://blogs.adobe.com/formfeed/2009/05/debug_merge_and_layout.html

I continue to use this sample pdf on a regular basis.  It has a reserved location among my desktop icons for quick access.  But occasionally I’d run into PDF forms that cause it to hang.  So finally I took the time to find and fix the problem.  While I was at it, I took care of a couple other minor issues.  The updated form is available for download here.  I hope you find it useful.

Sample: Convert to Strict Scoping

In the past couple of months I’ve had the opportunity to help a couple of customers through migrating a form design so that it was not dependent on non-strict scoping.

First, to review what this means, you could read this blog post describing the problem.

There’s no single technique to convert a form script so that it conforms to strict scoping. But there are a couple of common patterns.  I’ll disclose these by way of an example.

I’ve designed a sample form that makes use of “expando properties” to add multi-step undo/redo functionality to individual form fields.  In this case, the form adds an expando script object to a text field.  For review purposes, the reason this is problematic is that allowing users to modify the JavaScript representations of XFA objects means that the script engine must preserve these JavaScript objects. On large forms, this means we cannot release JavaScript objects to garbage collection and memory usage climbs.

```var This = this;

This.ctl = new function() {
this.history = [This.rawValue],
this.current = 0,
this.startValue = null;

this.undo = function() {
if (this.current > 0) {
This.rawValue = this.history[this.current-1];
this.current--;
}
}
this.redo = function() {
if (this.current < (this.history.length - 1)) {
this.current++;
This.rawValue = this.history[this.current];
}
}
this.enter = function() {
this.startValue = This.rawValue;
}
this.exit = function() {
if (this.startValue != This.rawValue) {
this.current++;
this.history.length = this.current;
this.history[this.current] = This.rawValue;
}
}
};
```

With this code in place, the enter event can call:

```this.ctl.enter();
```

Similarly, assuming the text field is named “Test”, the click event on an undo button can call:

```Test.ctl.undo();
```

This is all pretty powerful, elegant stuff, and it’s tempting to adopt this form design pattern. But as I said — the side effects are a problem.

There are two possible ways to revise the form so that it works in strict scoping:

1. Move the expando properties to script objects
2. Re-code the logic so that it doesn’t use expando properties

Move the expando properties

The solution in this case consists of moving the logic from the field to a script object. A by-product of the move means we have to have a referencing mechanism from the script object back to the fields. (note that the strategy of moving expandos to script properties will work only in Reader version 8.1 9.0 and later)

The steps to modify this form to allow it to work under strict scoping.

1) Move the logic to a script object

In the original form, we added an instance of the control function to each field.  In our revised design, we’ll move the instances of the control function to a script object and maintain a mapping between each field and its corresponding control function.

I added a script object called “ctl” and moved the logic from the initization event there almost verbatim:

```var This;

var control = function() {
this.history = [This.rawValue],
this.current = 0,
this.startValue = null;

this.undo = function() {
. . . .
};
```

The difference is that in the initialization script I created an instance of the function (with the new operator) right away.  Whereas in the script object we simply declare the function and will create instances of it later.

I’ve added some bookkeeping functionality to the script object.  This “glue” code allows us to keep our original logic intact by providing a mapping from fields to the corresponding instance of the control object:

```// The fieldList object is a place to store instances
// of the control function for each field
var fieldList = {};

// We need to assign unique id's to each field.
// The unique ids will index into fieldList.

// Use nUID as a global counter to assign id's
var nUID = 0;

// register a field and create an id, and an instance of control
function register(fld) {

// We'll store the id in the field under "extras"
var UID = fld.extras.nodes.namedItem("UID");
if (UID === null) {
UID = xfa.form.createNode("text", "UID");
fld.extras.nodes.append(UID);
}
UID.value = "ID" + nUID.toString();
This = fld;

// create an instance of the control function
fieldList[UID.value] = new control();
nUID++;
}

// If a field is part of a subform that gets removed, remove it here also
function unregister(fld) {
delete fieldList[fld.extras.UID.value];
}

// Convenience methods for accessing the control functionality
// associated with each field
function getObj(field) {
return fieldList[field.extras.UID.value];
}
function enter(field) {
This = field;
getObj(field).enter();
}
function exit(field) {
This = field;
getObj(field).exit();
}
function undo(field) {
This = field;
getObj(field).undo();
}
function redo(field) {
This = field;
getObj(field).redo();
}
```

3) Change the code references

The syntax of code that called methods directly on the Test field now needs to change. The initialization script of the Test field now makes a call to:

```ctl.register(this);
```

The enter event changes to:

```ctl.enter(this);
```

The click event is now:

```ctl.undo(Test);
```

4) Cleanup

When the control function was hosted by the Test field, the instance was conveniently removed when the Test Field was removed.  But now this needs to be done explicitly. The code to remove the subform has a new call:

```ctl.unregister(Test);
_item.removeInstance(this.parent.index)
```

After following the steps above, the revised form works under strict scoping.  Check it out.

Recode the Logic

But now the question needs to be asked: was this the best way to convert the form?  The process I followed was geared to keeping the control logic intact.  My assumption was that this is where the customer has invested the most and they’d prefer to keep that code intact.  But the implementation of the control function could have been implemented differently.  The history is represented as a JavaScript array, but could be re-implemented as an array of elements under <extras>.  The final solution is more elegant — but the effort to get there is riskier.

I’ve attached a 3rd variation of the form that implements undo/redo using extras. I won’t go into details on how it works here.  But one of the interesting side-effects of using extras rather than JavaScript variables is that due to formstate functionality, the undo history is preserved when the form is closed and saved. i.e. you can re-open the form later and the undo history is intact.

JavaScript Lint check in Designer

Some time ago I distributed a sample form that used Douglas Crockford’s excellent JSLint script to check form JavaScript for common coding errors.  You can read about it here. The logical next step is to use macros to integrate this capability into Designer (macros introduced here).

Here is everything you need to install the macro.

One improvement I’ve made over the previous version is to automatically populate the list of global objects with all the subform and field names in the form.  Now it will flag far fewer global objects — and if it does flag a global, you really want to pay attention, since it is very likely an un-declared variable.

I really strongly recommend this tool to you. When you run this against your script you will very likely be surprised at some of the coding errors you’ve made.  I know I was.

PaperForms Barcode Performance

We have had some feedback around the performance of paper forms barcodes on large forms.  Seems that when customers use multiple barcodes to process large amounts of data, their form slows down.

The reason the form slows down is because the barcode recalculates every time a field changes.  The calculation is doing several things:

1. Generate minimal, unique names for each data item. When the names are included in the output, we want them to be as terse as possible, while still uniquely identifying the data.  To do this, each name needs to be compared to all other names.
2. Gathering data values to be included in the output
3. Formatting the result as delimited text

It’s the first item that takes the bulk of the time.  In order to find the minimal name, the script compares each data node name against all others and iteratively expands the names until they are unique.  The algorithm appears to have O(n2) performance — which means that it degrades quickly when the number of data elements grows large.

There are three techniques you can use to improve the performance:

1. Do the work in the prePrint event

Move the barcode calculation to the prePrint event. In the barcode properties, uncheck "Automatic Scripting" and move the script from the calculate event to the prePrint event. Now, instead of recomputing the barcode every time a field changes, we compute the barcode only once — just before it gets printed.

2. Use unique field names

When the script encounters duplicate field names, it does lots of extra work to resolve them.  So don’t ask the script to do so much work.  Use unique field names. For example, instead of:

item[0].quantity
item[0].price
item[1].quantity
item[1].price
item[2].quantity
item[2].price

try:

item[0].quantity0
item[0].price0
item[1].quantity1
item[1].price1
item[2].quantity2
item[2].price2

Not only will the script complete more quickly, but the names written to the barcode value will be shorter.  When they are not unique, they get prefixed with their subform name.  When they’re unique, they are left unqualified.

3. Do not include names — and modify the script

Since the bulk of the work that the script does is to come up with minimal unique names, let’s not write out the names.  Uncheck the "Include Field Names" option.  Unfortunately, the script goes through the effort to produce unique names even when names are not included in the output. You need to modify the script to prevent it from calculating names. Uncheck "Automatic Scripting" and add the lines in red below.

19 function encode(node)
20 {
21   var barcodeLabel = this.caption.value.text.value;
22   if (includeLabel == true && barcodeLabel.length > 0)
23   {
24     fieldNames.push(labelID);
25     fieldValues.push(barcodeLabel);
26   }
27
28   if(collection != null)
29   {
30     // Create an array of all child nodes in the form
31     var entireFormNodes = new Array();
if (includeFieldNames) {
32       collectChildNodes(xfa.datasets.data, entireFormNodes);
}
33
34     // Create an array of all nodes in the collection
35     var collectionNodes = new Array();
36     var nodes = collection.evaluate();
37
38     for(var i = 0; i < nodes.length; ++i)
39     {
40       collectChildNodes(nodes.item(i), collectionNodes);
41     }
42
43      // If the form has two or more fields sharing the …
44     // parents of these fields, as well as the subscript …
45     // their parents, will be used to differentiate …
46     // to take as little space in the barcode as possible, …
47     // data in the object names only when necessary …
if (includeFieldNames) {
48       resolveDuplicates(collectionNodes, entireFormNodes,…);
}

If you implement this option, odds are you won’t bother with the first two methods.  The performance of the script is now O(n) and should work fine in the calculate event with non-unique names. And … is it just me?  I get a kick out of watching the 2D barcode re-draw itself every time I modify a field.

Accessibility Bug Fix

Recently one of our customers discovered an accessibility bug that caused read order and tab order to diverge. When we investigated it, we discovered that the bug existed in the reader runtime, but that we could work-around the bug by changing the syntax that Designer generates. Fixing the problem in Designer is much preferable, since that doesn’t require upgrading Reader. Of course, waiting for a fix in Designer is not easy either. Fortunately, the macro capability offers us an alternative way to disseminate a fix.  First the usual caveat: macros are an experimental, unsupported product feature.

The Bug

The bug scenario is when a custom tab order exits a subform; the read order follows the next-in-geographic order, while the tab order follows the explicit custom order.  Described differently, imagine a form with fields named "One", "Two" and "Three" and a subform named "S’.  The geographic order is: S.One, Three, Two, but the custom tab order is S.One, Two, Three. In the Reader runtime, the tab order correctly follows the custom order, but the Reader order gets confused on exiting the subform.

If you’re only interested in correct tab order you will not notice the problem.  However if you are using a screen reader and following read order, you will follow an incorrect order.  (If you examine the resulting form with tools like inspect32 or AccExplorer32 you’ll be able to see the result without actually running a screen reader)

I have included a sample form that illustrates the problem, and a version of the form with the fix applied.

The Fix

Down below, I’ll discuss the technical details of the fix.  But first the macro (rename to FixReadOrder.js) When you install and run the macro, it will scan the entire form searching for all the places where a custom tab order exits a subform. Each of these places will be modified.

You will need to re-run the macro every time you make edits to the form that modify tab order.

The macro includes some logic so that it will stop functioning in the next version of Designer – given that we expect the bug to be fixed.

The Deep End

Custom tab order is controlled by the <traversal> and <traverse> elements.  The (stripped down) syntax of the original form looks like this:

1 <subform>
2  <subform name="S">
3    <field name="One">
4      <traversal>
5        <traverse ref="Two[0]"/>
6      </traversal>
7    </field>
8    <traversal>
9      <traverse operation="first" ref="One[0]"/>
10    </traversal>
11  </subform>
12  <field name="Three"/>
13  <field name="Two">
14    <traversal>
15      <traverse ref="Three[0]"/>
16    </traversal>
17  </field>
18  <traversal>
19    <traverse operation="first" ref="Subform1[0]"/>
20  </traversal>
21 </subform>

There are two variations of the traverse element used here:

1. operation="first": specifies the first child of a container (subform)
2. operation attribute is unspecified (the default) — which means operation="next". Specifies the next element.

The runtime problem occurs when the traverse from field "one" at line 5 does not correctly navigate to field "Two".

Here is the fixed version:

1 <subform>
2   <subform name="S">
3    <field name="One">
4       <traversal>
5         <traverse ref="Two[0]"/>
6       </traversal>
7     </field>
8     <traversal>
9       <traverse operation="first" ref="One[0]"/>
10       <traverse ref="Two[0]"/>
11     </traversal>
12   </subform>
13   <field name="Three"/>
14   <field name="Two">
15     <traversal>
16       <traverse ref="Three[0]"/>
17     </traversal>
18   </field>
19   <traversal>
20     <traverse operation="first" ref="Subform1[0]"/>
21   </traversal>
22 </subform>

The macro added a traverse element at line 10. Now we’ve explicitly told the subform which child to navigate to first and also where to navigate to when the children are finished.  In fact, we end up with two traverse elements pointing to field "Two".  Not surprisingly, the runtime correctly moves to "Two" in both tab order and read order.

I continue to spend time playing with the prototype macro capability found in Designer 9 (ES2).  I have a new sample that uses it, but of course before showing you, I need to reiterate that the macro stuff is still an unsupported trial feature.

I wanted to do something to further help people debug their form script.  The usual way this happens today is that you sit down and add console.println() commands to the various scripts that run.  There are a few drawbacks to this mode of debugging:

• It is tedious to add the debug/trace statements
• It is not always clear which scripts need to have trace added
• If you generate too much output, the console window fills up and stops showing content
• FormCalc scripts cannot access the console object
• Dumping large amounts of content to the console slows form execution considerably

To help matters along, I have written a macro that will modify scripts in your form and automatically add trace statements.  The injected code does the following:

• Adds a top-level script object (scTrace) for collecting trace statements
• scTrace stores trace statements in a JavaScript array — so that the size is not limited by the console
• Adds a pre-print event that allows the array of trace statements to be dumped to a file
• Adds a "full" event script to the root subform that can be used with execEvent() to emit strings from FormCalc scripts (this technique first introduced in this blog entry)

As an example, the before and after for a specific JavaScript looks like:

Before:

var vTarget = xfa.event.target;
if (vTarget.className === "field") {
vTarget.fillColor = "220,220,255";
}

After:

scTrace.traceContext("enter:", this);
var vTarget = xfa.event.target;
if (vTarget.className === "field") {
vTarget.fillColor = "220,220,255";
}

For FormCalc the result is a bit more verbose:

Before:

price * units

After:

;scTrace.trace
xfa.event.fullText = "calculate"
xfa.event.newText = \$.somExpression
\$form.#subform.execEvent("full")
price * units

Note that when you use this debugging technique you must not leave this script in your production forms.  It adds too much overhead.  When you want to trace/debug your form, follow these steps:

1. Save a copy of your form
2. Add trace to the copy
3. Run the copy in Acrobat (not Reader)
4. When you want a trace dump, print the form.  The pre-print event fires and prompts you to save the trace (when trace has been saved, cancel the print)
5. debug and migrate changes/fixes back to the original document

The actual macro is here.  You need to rename it to have a .js suffix and put it in a subdirectory under the Scripts directory in your Designer install (see detailed instructions in the original blog entry).

Here is a sample form where trace has been injected.

I’ve found this tracing technique to work well in cases where form performance may be impacted by lots of events and scripts.  Especially in cases where there are dependency loops (see the discussion under "Dependency Loops" in this blog entry).

Note that the macro does not add trace to the indexChange event.  I ran into a scenario where this combination produced some bad behavior, so I elected to exclude that trace.  Hopefully this problem gets sorted out in the next version of Designer.