Main

August 19, 2009

Base64 Encode a PDF attachment

Some time ago I experimented with PDF attachments -- trying to add them to my XML data.  Ultimately I wasn't happy with the outcome, and I was going to leave it.  But then I saw a customer scenario that called for this capability, and then one of my regular commenters (Bruce) brought up the topic as well.  So I thought I would share my experiment.  I'm still not happy with the result, but maybe others can think of improvements.

The end goal is to copy a PDF attachments into their XML data.  If you can do so, it opens up a couple of interesting possibilities:

  • Take any attachments a user has added to the PDF and include it in a form submission or in a web service request
  • Take image attachments and use them to populate image fields

When your attachment is in a binary format, the standard way to include it in an XML file is to encode it as base64.  The problem is that PDF attachments are returned in a hex-encoding (each byte written as a 2 digit hex value).  The problem becomes how to convert from hex encoding to base64 encoding.

At first glance, one way to encode would be to use the Acrobat SOAP object: SOAP.streamEncode().  However, as of Acrobat 9, this method stops encoding at the first null byte in the stream.

The good news is that it is possible to convert from one encoding to the other using JavaScript.  The bad news is that it is slow.  But hey, maybe it's just my lack of JavaScript expertise.  Maybe someone reading this blog will take the sample and make it run fast.

Acrobat methods for attachments

The acrobat document object has a nice set of methods for dealing with attachments: openDataObject, getDataObject, createDataObject, importDataObject, removeDataObject, getDataObjectContents, and setDataObjectContents.

The interesting method in our case is getDataObjectContents().  It returns a stream object with the contents of an attachment.  If your attachment happens to be textual, then you can use util.stringFromStream() to convert to a string.  But if your attachment is binary, the best you can do is use the stream read() method to return hex-encoded bytes.

Conversion to Base64

The attached sample form has an initialization script that displays a subform for each attachment.  There is a button that will take the corresponding attachment, convert it to base64 and assign it to the image field value.  The conversion reads through the hex encoded bytes three at a time and converts them into base64 encoding.

As mentioned, the conversion is slow.  The conversion to base64 involves a lot of expensive substring() and parseInt() operations.  The images in the sample form are around 5K, and you can see there is a slight delay in the time it takes to extract and convert the data.  When I tried a 130K image, it took 10 seconds.  I don't recommend doing this with megabyte-sized attachments.

The fastest way to get your attachments into your XML data would be to embed them in their hex encoding.  But of course, that will work only as long as the consumers of your data can handle that format.

August 13, 2009

Dependency Tracking

One of the really cool aspects of XFA forms is the dependency tracking mechanism.  Dependency tracking is the feature where your field (and subform) calculations and validation re-fire whenever any of their dependents change.  Today I'll explain a bit about the mechanics of how dependency tracking is implemented as well as have a look at possible design issues related to this mechanism.

Simple Example

Suppose my purchase order form has order rows with fields: price, quantity and subtotal along with a total field.

The subtotal field has a JavaScript calculation: "price.rawValue * quantity.rawValue"

The total field has a FormCalc calculation: "sum(order[*].subtotal])"

Now whenever price or quantity fields change, the corresponding subtotal field will be recalculated automatically.
Whenever any of the subtotal field values change, the total calculation will re-fire.

Discovering Dependencies

When the form is first opened, all the calculations and validations are executed.  While executing a calculation or validation, the XFA engine keeps track of each object (field, subform, data node) that gets referenced during the script execution.  Each referenced object is added to its dependency list.  In our example, each subtotal field becomes dependent on the price and quantity fields from its row.  The total field becomes dependent on all the order subforms and on all the subtotal fields.  The mechanism is robust enough that even if you reference your objects indirectly, we still find the dependency.  e.g. the subtotal calculation could be written as:

parent.nodes.item(0).rawValue + parent.nodes.item(1).rawValue

Since the nodes we visited were still price and quantity, the dependencies are established just the same.

In some cases, the dependency list will grow as values change.  Consider this calculation:

if (A.rawValue < 100) {
    A.rawValue;
} else {
    B.rawValue;
}

Suppose the first time it executes, field A has a value of 20.  This means the code will not execute the else clause and will not access field B.  The initial dependency list will include only A.  However, if the A of changes to 200, the calculation will re-fire; the else clause will be evaluated and field B will be added to the dependency list.

If your calculation or validation calls a script object, dependencies will continue to be tracked during the execution of the script object method.

Dependent Properties

What constitutes a field change? What changes cause a calculation to re-fire?  I don't have a complete list.  But changing a field's visual properties (colours, captions, presence) do not cause a dependent to recalculate.  Changing the value or changing the items in a list box will cause a dependent calculation/validation to re-fire.

Turn off Automatic Calculations

If, for some reason, you're not happy with the default dependency tracking, then you can turn it off. There are two switches to control calculations and validations independently:

xfa.host.calculationsEnabled = false;
xfa.host.validationsEnabled = false;

Note that turning validationsEnabled off disables not only script validations, but also turns off mandatory field and picture clause validations.

Dependency Loops

It is possible to get into a situation where sequence of dependency changes goes into an infinite loop. In these cases, the XFA engine will allow the calculation or validation to re-fire a fixed number of times (10) and then will quietly stop re-firing the calculation or validation until the next time a dependent changes.  While the 10 times limit is ok in most cases, I have seen forms where this has been the root of performance issues.  If you have a dependency loop and your calculations are re-firing, you want to be aware of it and you need to fix it. 

Dependency loops are typically introduced when a calculation or validation modifies another field.

I should highlight that statement.  If your calculation or validation minds its own business and never modifies another field you shouldn't run into any loops.  There are lots of cases where a calculation or validation modifies another field and everything works fine -- but tread carefully.

Looping example

In order to track when my validations were changing, I wrote validation scripts write to a debug field:

field T1 validation:

debugTrace.rawValue += this.rawValue;
true;

field T2 validation:

debugTrace.rawValue += this.rawValue;
true;

Both T1 and T2 now have a dependency on field: debugTrace

The sequence of events:

  1. Field T1 changes
  2. T1 validation fires and modifies debugTrace
  3. T2 validation fires and modifies debugTrace
  4. T1 validation fires and modifies debugTrace
  5. T2 validation fires and modifies debugTrace
  6. ...

Here is a sample form to illustrate this example.

Note that if we remove the validation from T2, the circularity stops.  The validation on T1 modifies debugTrace, but after the validation completes, debugTrace does not change and T1's validation does not re-fire.

June 18, 2009

Managing Tab Stops in Fields

When I was working on my previous blog entry, I struggled a bit with rendering JavaScript source code in fields.  The issue was how to accurately display text with the tab stops that are embedded in the code.  The goal was to have the JavaScript code appear with the same spacing you would see in the source editor.  For this to work, the tab spacing needed to be exactly 4 characters.  The solution involved:

  • Use a non-proportional font to display the text (courier new)
  • Figure out the width of 4 characters: a 10pt character in courier new has a width of 6 points.  4 characters is 24 points
  • Set the default tab for the field.  Since Designer doesn't expose tab properties for fields, the form uses an initialization script to set the default tab: this.para.tabDefault = "24pt";

Tab Leaders

Having accomplished that much, it seems worthwhile to spend some time showing some of the other cool things you can do with tab settings in fields.  In Acrobat/Reader 9 we introduced tab leaders.  To see how this feature works with static text, check out Stephanie Legault's tutorial.

That's fine for static text, but what about fields?  If you want to use tab leaders in your form fields, you need to understand the field properties that control tab behaviours.  One way to get there is to read the relevant portions in the XFA specification.  Tab settings are paragraph properties.  A field's paragraph properties are defined in the <para> element (Part 2/Template Specification/the para element).  Now, you could fill your head with all that book knowledge or you could code from examples

The first page of the sample shows some variations on tab settings.  Each sample field has an initialization script to set the tab properties.  In each case there is a corresponding field that displays the resulting <para> element syntax.

tabDefault

This property on the <para> element controls the default tab setting.  If you want a tab stop every half inch, this would look like:

<field>
    <para tabDefault="0.5in"/>
</field>

To set this via script:

form1.tabExample1.sample::initialize - (JavaScript, client)
this.para.tabDefault = "0.5in";

tab-stops

The general format of a tab leader is: "[alignment] [leader] location ".  You can string a bunch of these together to specify multiple tabs.  If you look at the sample form you will see some variations.  The alignment ( left (before) / right (after) / center / decimal ) determines how the text is aligned at the tab location.  The leader parameter determines how to fill the space leading up to the tab position -- space, dots, rule, characters.  The 3rd parameter is the actual tab position.   Hopefully it's all self-explanatory from the samples.

Tabs and Data Entry

So far, this discussion has been assuming that we're rendering data that has embedded tabs.  Do tab settings work with interactive fields?  Well, yes and no.  The tab settings work fine.  The problem is, how exactly do you enter a tab character into an interactive field?  The tab key moves you to the next field.  The workaround the example uses is to define a key sequence that will insert a tab character.  Pressing "shift" + "space" will insert a tab character.  The way this works is that the field has a change event to translate this sequence:

form1.#subform[0].tabExample1.sample::change - (JavaScript, client)
if (xfa.event.change === " " && xfa.event.shift) {
    xfa.event.change = "\t";
}

Tabs and Rich Text

Rich text introduces a  couple of wrinkles wrt tabs. 

The tab character is not handled by XHTML.  XHTML processing rules tell us to collapse all white space down to a space character.  To get past this, we define a special style attribute to specify a tab:

<span style='xfa-tab-count:1'/>

The attributes that exist on the <para> element are also available as style properties: tab-interval and tab-stop.  With a plain text field, the paragraph properties are constant for the whole field.  With rich text, the paragraph properties can be re-specified with each <p> element.

The second page of the sample form has an example where a table of values is formatted into a plain text and a rich text field.  To see the code, look at the form:ready event of each field.  The script for the plain text variation is much simpler, but the script for the rich text field does some more elaborate formatting.  But back up a moment.  Likely the idea of constructing rich text is new to some of you.  The technique is fairly straightforward -- build a string representing the XHTML content and then use the loadXML() method to apply the text to the field.  Using XHTML allows the second field to enhance the output:

  • The tab settings for the first line (paragraph) are different from the rest of the field (center aligned with a space leader)
  • The cents portion of the prices are displayed in a superscript
  • The tab leader is styled -- the baseline is raised so the dots appear in the middle of the line rather than the bottom.

The overall effect makes the second menu field a fair bit nicer.  Imagine what someone with artistic skills could do.

Aligned Dots

Did you notice?  The dots in the plain text menu line up with the dots used in the rich text menu.  By default, the dot leaders are aligned between different objects on the page. i.e. the dots are aligned on a grid defined at the page level.

The Deep End

Ambient Attributes

When the rich text is rendered, the display properties are determined by combining the field properties with the properties specified in the XHTML.  An example would help:

<field>
  <font typeface="Myriad Pro"/>
  <value>
    <exData contentType="text/html">
      <body>
        <p>Text 1<span style="font-family:Arial">Text 2</span></p>
      </body>
    </exData>
  </value>
</field>

In this example, the XHTML has not specified a font for the string "Text 1".  Whereas the string "Text 2" is explicitly styled as Arial.  This means that the font for "Text 1" is the ambient font -- the font inherited from the field definition -- in this case Myriad Pro.  Same applies to tab settings.  By default the XHTML will inherit the tab settings from the field.  If you want to deviate from the default, then specify it inside a <p> element in the rich text.  That explains why in my rich text menu example I explicitly styled the first paragraph, but allowed the remaining text to inherit the field paragraph properties.

Building Big Strings

XHTML strings can grow very large.  You need to be careful how you build the strings.  Using the " += " operator can lead to performance issues.  Appending to a large string is expensive.  It breaks down to these set of operations:

  1. allocate new storage for the larger string
  2. copy the old string to the new storage
  3. append the new string to the new storage

The larger the string and the more append operations, the more expensive this becomes.  A more efficient alternative is to build an array of small strings and at the end, use the join() method to generate one large string.  Here are two examples for comparison:;

var S = "hello world";
S += " more text";
S += " even more text";

In comparison:

var StringArray = [];
StringArray.push("hello world");
StringArray.push(" more text");
StringArray.push(" even more text");
var S = StringArray.join("");

Of course, in this specific example, the strings are so small that the performance difference would not be noticeable. But when the number of append operations grow, the array technique will become much faster.

June 2, 2009

Collected Form Development and Debugging Tips

In this post I have consolidated a bunch of tools and tips for developing and debugging XFA/PDF forms.

Use the console Object Effectively

Always show the console on errors

When your form encounters a JavaScript error, the error text shows up in the console.  But by default the console doesn't appear -- which means you could be getting errors without being aware of them.  To see errors as soon as they happen, configure Acrobat to pop up the console as soon as a message appears.  Under Edit/Preferences, set the options highlighted below...

consoleDialog

 

The console methods

console.println() is an indispensable tool for tracing what is happening in your form script logic.  It is much more effective than xfa.host.messageBox() or app.alert(), since it doesn't require dismissing a dialog with every message.

Did you know there are other methods on the console object?  clear(), hide(), show().  You might want to place a call to console.clear() when your form initializes.  When it's time to deploy your form, make sure you haven't left any extraneous messages in the console.

The Server Alternative

Since the console object is an acrobat-specific tool, you need to use something different to emit trace message for server-based XFA forms.  For the server, use xfa.log.message() to write your message to the server log file.

Trace Function Calls

As long as you're in the mode of dumping information to the console or log file, you may as well make it as easy as possible.  This blog entry describes a method for tracing the input parameters to JavaScript functions.

Have a look at the data DOM

It's often useful to see what the data currently looks like.  One way to get there is to add a big text field that displays the data dom.  Give it this one line calculation: xfa.datasets.saveXML("pretty");  Set the field as bind="none".  Now you will have an up-to-date snapshot of the state of the data dom.

Re-factor your form

Design for re-usability.  Develop common patterns that are shared between forms.  Create script frameworks inside script objects.  Put the framework script objects in your fragment library.

Use the Merge/Layout Debugger

If your problems are in merge or layout, then try this.  (Try it even if you're not having problems with merge/layout -- it's cool.)

Exception Handling

When an exception is thrown and displayed in the console (or in the server log file) the resulting message can be frustratingly cryptic. To get higher quality information (including a stack trace), catch errors and parse them into human-readable text.  There is sample code here to get you started.

String Manipulations

If you are manipulating lots of strings -- especially if you're parsing them, get to know the power of regular expression processing in JavaScript.  Have a look at this regular expression testing tool.

Be Version Aware

If you make extensive use of script objects or if you've dabbled in using dynamic properties on XFA objects then get educated about the effects of strict scoping and how it behaves in different versions of XFA and Reader.  Read lots more here and here.

Summarize your form

This post shows how to get a "big picture" summary of the form meta data and the content in the form.

Document your Script

Take the time to properly document your JavaScript methods.  Use JavaDoc conventions. Six months from now when you (or someone else) needs to modify the form, you'll be glad you took the time to explain your form logic.  To generate a report of all the script in the form, look here.

Reduce and Isolate your problem

If you are struggling with a particular problem, try to reproduce the problem in a new, simplified form.  The act of isolating will often lead to discovering any outside influences are affecting your form behaviour.  If you're still having the problem in your new simple form, then you have a simplified version of the problem that makes it easier to share with others when you ask for help.

Designer Script Editor

Turn on Line Numbers

In your designer script window, turn on line numbers by right clicking on the script window and selecting "Show line numbers" from the context menu.

Don't Leave Blank Lines at the Top of your Script

Blank lines at the beginning of a script get trimmed before the JavaScript interpreter processes the code.  If there are blank lines, then any line numbers reported by exceptions won't agree with the line numbers you see in Designer.

Shortcut Keys

<control>+f -- bring up the find dialog. Any highlighted text is automatically used as the find string.  <control>+h -- brings up the find/replace dialog.  F3 -- find next.  <shift>+F3 -- find previous.

Multiple monitors

Convince your boss you need two monitors.  Keep Designer in one monitor and place your script editor and Acrobat console in the second monitor.

"foo.bar is not a function"

Your script makes a call to foo.bar() and you get the mysterious message: "foo.bar is not a function".  But you look in your script object and it sure looks like it is defined.  The problem is that you have a syntax error somewhere in your foo script object.  This prevents the runtime from discovering the foo.bar() function.  When you see this message, run Designer's script syntax checker.

Declare your Variables

Don't code: for (i=0, i<myList.length; i++)
Instead, code: for (var i=0, i<myList.length; i++)
The JavaScript interpreters process explicitly declared variables more efficiently.

While you're at it, use variable names that are unlikely to collide with names of objects on your forms. Such as prefixing variables with a "v" or "n" or "s".  (vObject, nOffset, sStringValue).

Beware of Ambiguous Expressions

If you're writing code that is intended to be re-usable in multiple forms, beware of potential conflicts between property names and form objects.  For example, the expression subform.border.fill.color becomes ambiguous if this script in a context where the subform has a field child named "border".  To avoid ambiguity, use the expression: subform["#border"].fill.color.  The hash symbol specifies that what follows is a className instead of a container name.

Find the Root Subform

If you have re-usable script that needs to locate the root subform, you can use this expression:

xfa.form.resolveNode("#subform")

Or the slightly more terse:

xfa.form["#subform"]

Don't Modify Your Form in XML Source Mode

If you need to change a property that Designer doesn't expose in one of it's property editors, then change the property via script.  For example: Designer doesn't support making a positioned subform repeatable.  Rather than adding an <occur> element in XML source, set this property in script:

Subform1.occur.max = "2";
var vNewSubform = _Subform1.addInstance();

Batch Changes to your Form

For advanced users -- if you need to make global changes to the contents of your form, this blog entry shows how a form author can write JavaScript to modify their template in Designer.  Likely a bit easier than writing XSLT.

Use FormCalc Functions -- from JavaScript

FormCalc has some built-in functions that give access to some powerful functionality.  There is a way to bridge between JavaScript and FormCalc in order to use these functions.  Look here.

June 16, 2009 update

Use Strict JavaScript

Run lint against your JavaScript.  Choose options that make it as strict as possible.  Details here.

May 28, 2009

DIY Column Chart

If you think about it, the XFA grammar has all the graphics primitives you'd need to generate a basic column or bar chart.  It has lines, rectangles and text.  It has repeating elements (subforms).  Ok, so it doesn't have a shadow effect.  And 3D bars are off the table.  But theoretically, all that stands between us and a simple chart is a bit of JavaScript.

By searching the web you can find samples of packages that render charts in browsers using various JavaScript frameworks.  So why not do the same in an XFA form?

Have a look at this sample PDF (here's the data).  In particular, try modifying a value in the table and watch the bars update.  Try modifying a value to $200,000 so that the Y-axis needs to be re-scaled.  Cool.

To use one of these charts in your PDF form, open this sample in Designer and turn the chart subform into a fragment that you'll be able to bring into other forms.  When you open in Designer you'll see some instructions that get hidden at runtime.  The components within the chart can be styled and customized as required.  The aspects that need not be set at design time are: the positions of the dynamic elements (labels, tics, bar), the contents of the labels and the size of the bars.

Options

Once the chart is sized, positioned and styled as you like, then look at the calculate script on the chart  subform to set the rest of the parameters.  At the top of the script you will see variables for controlling the rest of the chart behaviour:

  • Bar colours
  • grid line control (short tics vs. lines that extend across the chart)
  • x label staggering (prevent overlapping labels)
  • define the gap size between bars
  • Set the maximum number of labels on the Y axis
  • Column stacking (on or off)
  • SOM expressions defining the data series

In all it was around 450 lines of generously commented JavaScript. 

Oh, and lest you think this was all original, I did borrow some Java code (converted to JavaScript) to help with the Y axis label generation.

Performance Considerations

The form makes use of dynamic subforms to draw the variable elements of the chart -- X and Y labels and bars. Once the subforms were created, the script resizes and positions them correctly.  A more efficient way to draw the chart would be to pre-create a fixed number of labels and bars at design time -- not wrapped in subforms.  Drawing the chart would simply use the pre-created objects and hide the ones not need.  The problem with this approach is that it is harder to author -- the author is responsible for making sure there are enough labels and bars.  My version of the form opts for easier design and a slightly less efficient runtime.  But either way, this approach of rendering graphics won't scale well to larger more complex implementations. 

May 8, 2009

Sort Subforms

You probably already knew it was possible to re-order (and hence sort) a group of subforms.  But in case you didn't -- or in case you needed a sample, here you go.

In the sample, the column headings are button fields.  Their click event sorts the rows by the values in that column.  If you click again it will sort in descending order.

I centralized the sort methods into a script object that ought to be re-usable in other contexts:

/**
* Sort a group of subforms.  Calling this the first time will sort
* ascending. Calling it again will toggle to descending.
* @param vButton the button field used to initiate the sort. 
*  We store the current sort order in the button rawValue property.
* @param vInstanceManager the instance manager of the subforms to
*  be sorted
* @param vSortFieldName the name of the field in the subforms used
*  to sort.  Null values always sort to the beginning.
*/
function sort(vButtonField, vInstanceManager, vSortFieldName)

/**
* Sort a group of subforms.  Calling this the first time will sort
* ascending. Calling it again will toggle to descending.
* @param vSubformInstance one instance of a subform in the set of
*  subforms to be sorted
* @param vFieldName the name of the field used to sort
* @param bAscending boolean: if true, sort in ascending order,
*  otherwise descending.
*/
function performSort(vSubformInstance, vFieldName, bAscending)

Note that sorting subforms will also cause their underlying data to also be re-ordered.

Notes

Rather than embarrass myself by implementing a rookie bubble sort, I used the JavaScript Array.sort() method.  Of course, this meant figuring out how to set up a sorting function and letting it have access to the sort parameters (sort field and sort order).  But figuring out how to store these under <extras> was easier than trying to impress you with a fancy heapsort algorithm.

I generated the sample data at this very clever website: http://www.generatedata.com/ 

Deep End

If you sort a very large set of subforms, you might run into performance issues.  If you need to improve the performance, try sorting the underlying data DOM using the remove/insert methods on the nodelist.  After re-ordering the data you would do an xfa.form.remerge() and the subforms should show up nicely sorted.

April 22, 2009

Performance of Object Expressions

In a previous post I described how object expressions in JavaScript worked and compared them to using calls to resolveNode() and using FormCalc.  There is a trade off between readability and performance.  From the previous post:

"SOM parser evaluation is faster than JavaScript object evaluation.  If you have an expression that must be executed many, many times, then it is better to wrap it in resolveNode() or use FormCalc.  However, for most forms with modest amounts of script, the difference in overall form performance is negligible. Most of the time ease-of-scripting and code readability are more important."

We were working on an internal prototype that made me want to quantify the difference in performance between object expressions and SOM expressions.  In order to measure performance I needed a form that would evaluate many, many expressions. 
The result is a sample form that simulates John Conway's famous Game of Life

The Game of Life Form

A few notes about how the form works:

  • Each cell is a check box field.  You can re-arrange the starting pattern by toggling the fields
  • The grid is initialized by creating nested row/column subforms
  • The row and column subforms are renamed so that the object expressions don't use indexes. 
    e.g. grid.row[3].column[4] becomes grid.row_3.column_4

Most applications that simulate this game access the cells in nested loops.  For this sample, the loops are flattened so that we have very large scripts with hard coded object expressions.  For example, the code to find neighbours of the cell at (3,4) is:

vNeighbourCount = 0;
vNeighbourCount += form1.grid.row_2.column_3.cell.rawValue;
vNeighbourCount += form1.grid.row_3.column_3.cell.rawValue;
vNeighbourCount += form1.grid.row_4.column_3.cell.rawValue;
vNeighbourCount += form1.grid.row_2.column_4.cell.rawValue;
vNeighbourCount += form1.grid.row_4.column_4.cell.rawValue;
vNeighbourCount += form1.grid.row_2.column_5.cell.rawValue;
vNeighbourCount += form1.grid.row_3.column_5.cell.rawValue;
vNeighbourCount += form1.grid.row_4.column_5.cell.rawValue;
if (form1.grid.row_3.column_4.cell.rawValue)
    vNewState = (vNeighbourCount == 2 || vNeighbourCount == 3) ?1:0;
else
    vNewState = (vNeighbourCount == 3) ? 1:0;
form1.grid.row_3.column_4.cell.extras.newState.value = vNewState;

After one pass where we stored the new state for each of the cells in the extras of the cell field, we assign the new state:

form1.grid.row_3.column_4.cell.rawValue =
          form1.grid.row_3.column_4.cell.extras.newState.value;

Repeat that code for each of the 2,500 cells in the grid and now the form has around 36,900 lines of JavaScript and 29,405 object references.  That should be good enough to test the performance of object expressions.

Then for comparison purposes, the form has another  script that does the same operation using calls to resolveNode():

vNeighbourCount=0;
vNeighbourCount +=
      this.resolveNode("form1.grid.row_2.column_3.cell").rawValue;
vNeighbourCount +=
      this.resolveNode("form1.grid.row_3.column_3.cell").rawValue;
vNeighbourCount += 
      this.resolveNode("form1.grid.row_4.column_3.cell").rawValue;
vNeighbourCount +=
      this.resolveNode("form1.grid.row_2.column_4.cell").rawValue;
vNeighbourCount +=
      this.resolveNode("form1.grid.row_4.column_4.cell").rawValue;
vNeighbourCount +=
      this.resolveNode("form1.grid.row_2.column_5.cell").rawValue;
vNeighbourCount +=
      this.resolveNode("form1.grid.row_3.column_5.cell").rawValue;
vNeighbourCount +=
      this.resolveNode("form1.grid.row_4.column_5.cell").rawValue;

if (this.resolveNode("form1.grid.row_3.column_4.cell").rawValue)
    vNewState = (vNeighbourCount == 2 || vNeighbourCount == 3) ?1:0;
else
    vNewState = (vNeighbourCount == 3) ? 1 : 0;
this.resolveNode
      ("form1.grid.row_3.column_4.cell.extras.newState").value = 
      vNewState;

And one more comparison where we do the same in FormCalc:

vNeighbourCount = 0
vNeighbourCount = vNeighbourCount + form1.grid.row_2.column_3.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_3.column_3.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_4.column_3.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_2.column_4.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_4.column_4.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_2.column_5.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_3.column_5.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_4.column_5.cell
if (form1.grid.row_3.column_4.cell) then
    vNewState = if(vNeighbourCount == 2 or vNeighbourCount == 3,1,0)
else
    vNewState = if(vNeighbourCount == 3, 1, 0)
endif
form1.grid.row_3.column_4.cell.extras.newState = vNewState;

Results

The good news is, that all three variations are *very* slow -- great for measuring performance :-)

The performance results when I ran this on my laptop:

JS with Object Expressions

JS with calls to resolveNode()

FormCalc

Mlliseconds

7154

4673

1735

Milliseconds
per Expression

0.243

0.159

0.059

Clearly there are differences.  I was surprised at how much faster the FormCalc version was.  I would like to understand that better some day.

But now before you run off and re-code your forms to use resolveNode() or to use FormCalc, we have to put the numbers into perspective. For the slowest variation (object expressions in JavaScript), we evaluated 29,405 expressions in just over 7 seconds.  That means each expression evaluated in roughly 0.24 milliseconds.  I'm going to go out on a limb here and assume that this is fast enough for most forms.  Sure, the FormCalc version does it in  0.06 milliseconds, but your end users will not notice the difference.  Unless, of course, your form is doing something frivolous such as simulating the Game of Life.  The test validated the original assumption: in most cases code readability and maintainability trump any performance issues.

Something Useful

So far this blog post has not uncovered anything particularly useful, and I feel obliged to leave you with something that you can use in your day-to-day form design.

In the previous post that dealt with object expressions, I described how the JavaScript engines deal with "naked field references".  There is a nuance that you should be aware of:  the JavaScript engines treat a variable with an explicit declaration differently from a variable without a declaration. i.e.

var foo = "hello world";
xfa.host.messageBox(foo);

is handled differently from

foo = "hello world";
xfa.host.messageBox(foo);

In the first case, the JavaScript processor will ask the XFA engine if it recognizes "foo".  The XFA engine says "no" and from that point on, the JavaScript processor will not ask again when it encounters more references to "foo".  In the second case where the "var" keyword is not used, the JavaScript processor asks the XFA engine for a definition of "foo" each time it encounters "foo" in the script.

Now consider what happens when you code:

for (var i=0; i<10000; i++)
  total += po.nodes.item(i).subtotal.rawValue;

vs.

for (i=0; i<10000; i++)
  total += po.nodes.item(i).subtotal.rawValue;

In the first case, the XFA processor evaluates "i" once.  In the second case, the XFA processor evaluates "i" 30,000 times.  Would you notice the difference?  It depends on two factors:

  1. How many iterations in the loop and how many references to the loop counter
  2. The cost of one lookup - how many objects are in scope when we do the evaluation.  When the XFA processor searches for "i", it does a physical scan through all objects that are within scope of the current context.

The Deep End

If you are writing script for a form that will run on the server, there is a bug you might want to be aware of.  As mentioned, once the JavaScript engine determines that an identifier is a variable or an XFA object, it will not ask again.  On the client this happens on a per-script basis.  However on the server this happens on a form-wide basis.  e.g. if an initialization script uses "foo" as a variable, then it will be assumed to be a variable in all other scripts on the form.  Another reason why my preference is to avoid using common names for variables.  For reliability -- and for readability I prefer to prefix my variables e.g. "vIndex".

February 6, 2009

Form Compatibility

In order for customers to successfully distribute forms, they require predictability.  Predictability is more important than bug-free software (if there were such a thing).  Predictability means the form you originally developed continues to work the same when opened in a new version of Adobe Reader.  "the same" means it has the same set of features and the same set of bugs as in the previous version.

Same features, same bugs.  If you want the new features and bug fixes you need to bring your form into Designer and target it for a newer release of Reader.  But even so, for complex forms you probably prefer to get the new features without necessarily getting the new bug fixes.  Bug fixes can change form appearance and/or script behaviour.  What users most often want is to use that cool new feature, but not to tweak their layout due to the fixes in form rendering.

To accommodate these needs, there are two dials that control the form: the target version and the original version.

Target version

field.setItems() was an enhancement to the XFA object model for Reader 9.  Suppose Adobe Reader 9 opens a form Designed for Reader 8.  What should it do if this Reader 8 form happens to have a script calling field.setItems() ?  It should do the same thing that Reader 8 does: throw an error. 

Each form is stamped with a target version.  The stamp is expressed as the XFA version.  Reader 8 uses XFA version 2.6.  Reader 9 uses XFA version 2.8.  Whenever Reader is asked to execute a script function, it first checks whether that script function is valid for that version of XFA.

This applies to more than just script functions.  It applies to XFA markup as well.   e.g. Hyphenation was added in XFA 2.8.  If Reader encounters a 2.6 form that has the markup commands to turn on hyphenation, the hyphenation will not be enabled.

The XFA version is defined by the XML namespace of the template: e.g.

<template xmlns="http://www.xfa.org/schema/xfa-template/2.8/">

The target version is controlled in the Designer UI under form properties/defaults.  Note that by default, Designer will set the target version one version back from the current shipping version of Adobe Reader.

Original Version

Form behaviour is determined by more than just new features, it is also determined by bug fixes. In some cases fixing a bug in Reader can actually break existing forms that inadvertently relied on the buggy behaviour. Therefore by default we preserve old behaviours when migrating a form forward.

We preserve the old behaviour by storing the original version of the form in a processing instruction in the template. When you modify your form in Designer to take it from Reader 8.1 to Reader 9.0, we embed the original version inside the form so that you end up with the version 9 features, but the version 8.1 behaviours.  The processing instruction looks like this:

<?originalXFAVersion http://www.xfa.org/schema/xfa-template/2.6/?>

When Reader 9 opens a form with an original version of 2.6, it makes sure that the behaviours are consistent with Reader 8.1.

This is all well and good, but there are times where you really, really need the latest bug fixes.  Unfortunately there's no UI to move the original version forward.  If you need the latest and greatest bug fixes, the go into the XML source tab, find the originalXFAVersion processing instruction and delete it.  Now your form will get all the behaviours and features of XFA 2.8 and Reader 9.

Sample

I have attached a sample form that demonstrates changing the target and original versions.  There are three interesting part to the form.

1. The form has both target version and original version set to 2.6 (Reader 8.1).  There is a button on the form that extracts these values from the template and populates a couple of fields with the result.

2. When you open the form in Designer, you will see a field displaying current date and time in ISO8601 format.  This field has a calculation that looks like this:

this.rawValue = "20090205T163000";
xfa.host.currentDateTime();

When you preview this form, the first line executes fine, but the second line encounters the currentDateTime() function that was added to XFA version 2.8.  As a result, the script fails on the second line, and the value of the field remains "20090205T163000".  If you open the script console in Acrobat (control+J) you'll see the error.

3. The bottom part of the form illustrates a bug fix that was made in Reader 9.  Hidden fields are not supposed to affect layout.  The hidden property has always behaved properly in flowed subforms, but we discovered a bug in positioned containers.  Prior to Reader 9 a hidden field would expand the extent of a growable, positioned subform.  In this form, the subform is taller than it should be.

Update The Target Version

Change the target version (in the form properties/defaults tab) from "Acrobat and Adobe Reader 8.1 or later" to "Acrobat and Adobe Reader 9.0 and later".  Now when you preview the form you will see:

1. The target version is now 2.8, but the original version remains 2.6.

2. The calculation now works without error. The field displays the current date and time.

3. The growable subform renders at the same height as it did in Reader 8.1

Update the Original Version

Go into XML source mode and remove the originalXFAVersion processing instruction.  Now when you preview the form you will see:

1. Both target and original versions are 2.8

2. The calculation continues to work

3. The growable subform has collapsed to an extent that does not include the hidden field

The Deep End

Discovering Processing Instructions at Runtime

I was able to get at the processing instructions by loading an E4X XML object with the results of xfa.template.saveXML().  I don't recommend doing this in a production form, since the result of calling saveXML() on the template can be a very, very large string -- especially if there are images embedded in the template. 

If you wanted just the target version (the xml namespace), there is an easy way to get it with the ns property: xfa.template.ns. 

FormTargetVersion

I simplified a couple of details in order to make this easier to explain.  The full story is a little more complicated.

When Designer opens a form that has markup that is newer than the target version, it will automatically update the namespace of the XFA template.  So for example: Designer opens a 2.6 template with a target version of Reader 8.1.  It discovers syntax for turning on letter spacing, so it will automatically update the template to 2.8.  But the target version remains Reader 8.1.  This is because the UI for target version is actually driven by another processing instruction:

<?templateDesigner FormTargetVersion 26?>

Under most circumstances, Designer will keep the FormTargetVersion and the XFA template version synchronized.  If the template version is newer, then you will undoubtedly find a bunch of messages in your warnings tab.  This is Designer's way of telling you that you're using features that do not work in your target version.  Until you clean those up, your form will not work correctly in your target version.

Choosing Selected Changes

Changing the original version is pretty high level switch.  You either get all the bug fixes for a new release, or you get none.  In reality, there are qualifiers that allow you to preserve selected behaviours.  Suppose that in my sample form I wanted the bug fixes that came with Reader 9/XFA 2.8 except for the fix for hidden fields in positioned subforms.  In that case I can specify the processing instruction as:

<?originalXFAVersion http://www.xfa.org/schema/xfa-template/2.8/ v2.6-hiddenPositioned:1?>

The qualifier: v2.6-hiddenPositioned:1 tells Reader 9 to preserve the XFA 2.6 behaviour for hidden fields in positioned subforms.

Scripting bug fixes

Most of the time, I am happy with the default behaviour where original version behaviour is enforced by default.  However there is one exception.  My previous post described how the behaviour of strict scoping changed from 8.1 to 9.0.  The difference is that with strict scoping on in 8.1 we release JavaScript variables declared in script objects.  In 9.0 we preserve these variables.  If you are targeting 9.0 and make extensive use of script objects, make sure that you set your original version to 9.0 as well.  

Changing Default Behaviours

There have been a couple of times where we have changed behaviour and made the new behaviour the default without protecting it with the original version mechanism.  The most infamous was when we added "direct rendering" for dynamic forms.  In this case we had developed a way to dramatically improve the performance of dynamic forms.  We had to choose between turning the new behaviour (and the performance boost) on for all forms or just for those forms designed for 8.1 and later.  We chose to make the new behaviour the default.  If this caused problems, the form author could "opt out" by inserting an originalXFAVersion processing instruction.

This is described in the knowledge base article: http://kb.adobe.com/selfservice/viewContent.do?externalId=kb402409

January 30, 2009

Populating list boxes

One of the Reader 9 enhancements was a new API call to populate list boxes: field.setItems(). The motivation for the new API is to provide better performance for populating lists.

Prior to Reader 9, the standard way to populate a list box is to call:

field.addItem(displayValue [, boundValue])

for each item in the list.  The new API looks like:

field.setItems(itemListString [, numColumns])

The first parameter is a comma-separated list of values, the second parameter is an integer telling the field how many columns are in the data (defaults to one).  The second parameter is designed for future extensibility if we choose to some day implement a multi-column list box.

Examples

A call to populate a listbox with currencies might look like:

Currency.setItems(
  "US Dollar,Canadian Dollar,Euro,United Kingdom Pounds");

Or if there were a bound value, it would look like:

Currency.setItems("US Dollar,USD,Canadian Dollar,CAD,Euro,EUR,United Kingdom Pounds,GBP", 2);

Prior to Reader 9, this second variation would have been coded as:

Currency.clearItems();
Currency.addItem("US Dollar", "USD");
Currency.addItem("Canadian Dollar", "CAD");
Currency.addItem("Euro", "EUR");
Currency.addItem("United Kingdom Pounds", "GBP");

Alternative

There is a 3rd method for populating listboxes: binding them to data.  Designer allows you to point your field at a location in your instance data where list box contents will be stored.  While this method has very good performance, it has the disadvantages that a) your data is not always in the correct format for binding, b) the listbox gets populated from data only during the initial data load.

Performance

If you are using listboxes only casually you probably will not notice the difference in performance between the two methods. But if you are using listboxes intensively, the new method is a life-saver.

I have attached a form where I compare the old performance to the new. On my laptop, I populate a listbox with 500 items in 125 milliseconds using addItem() calls, and in 16 milliseconds using setItems(). Neither of these numbers may seem significant, but we have customers with forms containing many list boxes with many, many entries where the difference in performance is critical.

Compatibility

If you are designing a form to use this new API, be sure and set your target version (in Form Properties/Default) to "Acrobat and Adobe Reader 9.0 or later".  Unless you do this, calls to setItems() will not work -- even though you might open the form in Reader 9.

December 16, 2008

Object Expressions in XFA

Form authors tend to resolve SOM (Script Object Model) expressions inside calls to resolveNode() more often than they need to. As a result, their JavaScript code is less readable than it could be. There are alternatives coding patterns I can recommend, but first some background.

Object expressions vs SOM expressions

If you wanted to get the value of a field in a purchase order form, there are a couple of ways to get it:

xfa.form.purchaseOrder.shipTo.address.city.rawValue;

or

xfa.resolveNode
           ("xfa.form.purchaseOrder.shipTo.address.city").rawValue;

The first form is an object expression.  In the second form, the argument passed to resolveNode() is a SOM expression. These are not evaluated the same way, but they almost always return the same result.

Dotted expressions are evaluated by the JavaScript interpreter. Every time the JavaScript interpreter reaches a dot in an expression, it calls into the XFA processor to evaluate the token following the dot. De-constructing the first example:

  1. The JavaScript engine encounters “xfa”. It resolves this easily since “xfa” has been pre-registered with the engine as a known global object.
  2. JS encounters “.form”. It asks the XFA processor for the “form” property of the xfa object. XFA returns the form model object.
  3. JS encounters “.purchaseOrder”. It asks the XFA processor for the “purchaseOrder” property of the form object. XFA returns the purchaseOrder subform object.
  4. We continue to evaluate dots until we reach the “city” object. Then when JS asks XFA for the “rawValue” property of the “city” object, XFA returns a scalar value.

De-constructing the second example:

  1. The JavaScript engine encounters “xfa”, and returns the pre-registered xfa global object..
  2. JS encounters “resolveNode(...)”.It asks the xfa object to evaluate the method resolveNode(). XFA resolves the SOM expression and returns the city object
  3. JS encounters “.rawValue” and asks XFA for the rawValue property of city.

You can see we take two very different paths, but have ended up in the same place.

The main difference is that the object expression yields more readable code, but is less efficient. Resolving SOM expressions internally is more efficient than the many more round-trips from JS to the XFA engine. However, in the vast majority of cases, the performance difference is negligible in the context of overall processing. My bias is toward using object expressions for the sake of code readability – unless we know we are in an area of performance critical code.

Note that FormCalc handles object expressions differently from JavaScript. The expression:

xfa.form.purchaseOrder.shipTo.address.city.rawValue is not evaluated dot-by-dot. The FormCalc engine will pass the entire expression to XFA for evaluation as a SOM expression.

Using "#" in SOM

We use the "#" character to disambiguate properties (font, border, caption etc) from containers (fields, subforms etc). Using "#" tells the SOM parser to treat the following token as a className rather than as a named object. Take this example:

<subform name="Country">
    <border/>
    <field name="border"/>
</subform>

The expression “Country.border” is ambiguous. It could refer to either the <border> property or the field named “border”. The rules are that by default the named object takes precedence. If you want to explicitly get the border property you must use “Country.#border”.

Since the "#" character is illegal in JavaScript, any expressions that use it are usually wrapped in a call to resolveNode(). E.g.:

Country.resolveNode("#border").presence = "visible";

As it turns out, there is a more terse syntax available. JavaScript properties can be accessed by two different syntaxes. e.g. A.B can also be expressed as: A["B"]. So the expression above could also be coded as:

Country["#border"].presence = "visible";

But simpler yet, if we are not worried about ambiguity, we can code this as:

Country.border.presence = "visible";

(As I write this, I realize that some of the script in my samples is not careful enough. Any general purpose re-usable code that references the subform.border property really must use the #border notation.)

As I said, in the case where you are not worried about ambiguity, you do not need to qualify with the “#” symbol. In the case with field properties we don't really need to worry about name conflicts. Unlike subforms, fields do not have children with user-defined names. (ok, that’s not true, but let’s pretend it is for now.) You should be able to code “field.border” without worrying about ambiguity.

Accessing field.property instead of field["#property"] is possible as long as the property does not have an unlimited number of occurrences. This means that we cannot refer to “field.event” – because fields can have any number of <event> elements.

We’ll work through this case with an example. I have seen code that changes the email address of a submit button that looks something like:

this.resolveNode("SubmitButton.#event[0].#submit").target =  
                                       "mailto:" + email.rawValue;

For starters, this could be expressed more tersely as:

SubmitButton["#event"].submit.target = "mailto:" + email.rawValue;

But do not do this.  This is fragile code. It assumes there is only one event, and addresses the first <event> element. To make this easier, Designer has started populating the name property on the <event> element:

<field name="SubmitButton"> 
  <event name="event__click" activity="click"> 
    <submit format="xml" textEncoding="UTF-8" 
                               target="mailto:foo@foo.com"/> 
  </event>
</field>

As a result, the script can now be reliably be coded as:

SubmitButton.event__click.submit.target = "mailto:" + email.rawValue;

Finally, to close the loop on what I said above: “fields do not have children with user-defined names.”. As you see, events (and a couple other properties) can have arbitrary names. But we shouldn’t have to worry about name collisions. There is no interface in designer for naming these properties. When Designer names them, it generates unique names. As long as nobody has modified their XML source and changed their event to look like <event name="border">, then field.border remains unambiguous.

Transparent Subforms

You will often see references to "#subform[0]". As in:

xfa[0].form[0].form1[0].#subform[0].Button1[0]

The reason you see “#subform” in this expression is because the subform is nameless.

This is the most explicit possible SOM expression that leads us to Button1.


However, this expression also gets us there:

xfa.form.form1.Button1

We don't need the [0] references because by default, object expression will always return the first occurrence.
The reason we can omit the #subform is because nameless subforms are treated as transparent.


In this example that means that all the children of the nameless subform may be referenced as if they were children of form1.

Performance

As noted above, SOM parser evaluation is faster than JavaScript object evaluation.  If you have an expression that must be executed many, many times, then it is better to wrap it in resolveNode() or use FormCalc.  However, for most forms with modest amounts of script, the difference in overall form performance is negligible. Most of the time ease-of-scripting and code readability are more important.

However, whether using SOM or not, reducing evaluations is good practise.
Beware of coding patterns that look like this:

xfa.form.form1.header.address.Button1.presence = "hidden";
xfa.form.form1.header.address.Button2.presence = "hidden";
xfa.form.form1.header.address.text1.presence = "hidden";
xfa.form.form1.header.address.text2.presence = "hidden";
xfa.form.form1.header.address.text3.presence = "hidden";

In cases like these, it is more efficient (and readable) to resolve a variable to the common parent node e.g.:

var addr = xfa.form.form1.header.address;
addr.Button1.presence = "hidden";
addr.Button2.presence = "hidden";
addr.text1.presence = "hidden";
addr.text2.presence = "hidden";
addr.text3.presence = "hidden";

When resolveNode() is Necessary

Some parts of SOM syntax are not supported by the object evaluation of our script engines. I can think of two examples:

  • SOM predicates (see previous blog entry)
  • The elipsis operator (..) that recurses down the hierarchy finding nodes. While it is handled by FormCalc, it is not legal in JavaScript.

Here is a sample script that shows how these can be combined using resolveNode() to make all hidden fields in a nested subform visible:

var vHiddenList = xfa.resolveNodes('xfa.form..address.#field.[presence == "hidden"]');
for (i=0; i<vHiddenList.length;i++)
  vHiddenList.item(i).presence = "visible";

The Deep End

Differences

When comparing object expressions to SOM evaluation I said: “These are not evaluated the same way, but they almost always return the same result.”

You might want to know under what circumstances they will return different results.  The difference between object expressions and SOM evaluation is that SOM evaluation will take into account the relative occurrence numbers (index) of objects, while object expressions cannot. In the example below we have fields bound to this data:

<order>
  <price>10.00</price>


  <quantity>2</price>


  <price>20.00</price>


  <quantity>3</quantity>


</order>


The form looks like this:

<subform name="order">
   <field name="price"/>

   <field name="quantity"/>

   <field name="subtotal">

      <calculate>

         <script contentType="application/x-javascript">

           order.price.rawValue * order.quantity.rawValue;

         </script>

      </calculate>

   </field>

   <field name="price"/>

   <field name="quantity"/>

   <field name="subtotal">

      <calculate>

         <script contentType="application/x-javascript">

           order.price.rawValue * order.quantity.rawValue

         </script>

      </calculate>

   </field>

</subform>


The calculations for the subtotal fields will both return the result “20.00”. This is because when we evaluate the expression: order.price and order.quantity from the context of the subtotal field, the JavaScript engine will always return the first occurrence of the price and quantity fields – for both occurrences of the subtotal field. However, we could change the calculation to:

subtotal.resolveNode("order.price").rawValue *
   subtotal.resolveNode("order.quantity").rawValue

In this case the second occurrence of the subtotal field would evaluate to “60.00”. This is because the occurrence number of the subtotal field causes the SOM evaluator to return the second occurrence of the price and quantity fields. If we were using FormCalc, then the expression order.price * order.quantity would return the expected result, since FormCalc uses SOM natively.  Fortunately, we don't encounter this pattern very often, so most form authors can assume that there is no difference between SOM evaluation and object expression.

Naked Field references

Based on what I’ve described above, you might wonder how we are able to resolve “order.price” in JavaScript. How does JS resolve a naked reference to “order”? We can’t register all field names with the JavaScript interpreter. If you were used to coding in Acroforms, you used expressions like

this.getField("order")

in order to find field objects.

The way the XFA processor works is that when JavaScript encounters an unrecognized object, it tries to resolve it as a property of “this”. In our example, “order” is really evaluated as “this.order”. Technically the subtotal field doesn’t have a property called “order”, but XFA does a check to see if there is an object called “order” that is in scope of subtotal.

While this is really great for keeping the code easy to write and readable, it has a side-effect that causes grief. When encountering an unknown token, the JavaScript engine first asks the XFA processor to resolve the token, and if not found, then checks JavaScript variables.  This order of evaluation causes issues for a calculation script that looks like:

var x = 42;
field.rawValue = x;

In this example. the field will get populated with the value of the x property (this.x).  This is one of the reasons why I normally prefix my variable names with a "v", in order to reduce the likelihood of a conflict with a property name.

Note that FormCalc allows variables to take precedence over properties.  In FormCalc the script above would behave as expected and populate the field with "42".  The reason for the evaluation order in JavaScript is due to a limitation with the engine when this feature was first implemented.  As far as we know, that limitation is now gone.  As some point we would like to change the order of evaluation for JavaScript -- but would do so in a safe way so that older forms continue to behave as they do today.

December 9, 2008

Adventures with JavaScript Objects

I have to start with an admission -- I'm not a trained JavaScript programmer.  This became evident to me as I reviewed the latest version of the exclusion group sample form.  I was not happy with the performance, so I thought I would stick in a timer and see if I could measure and improve performance.  Adding the timer was no problem.  I changed the code on the subform remove button.  Note the time calculations at the beginning and end:

form1.#subform[0].NPSA3.NPTable.Row1.Cell14::click - (JavaScript, client)

var vTime1 = (new Date()).getTime(); // gets the time in milliseconds

// Removing a subform in the middle of a slie can
// change the SOM expressions of subforms later in the list
// This means that we have to clear the error list
scValidate.clearErrList();

if (_Row1.count > _Row1.min)
    _Row1.removeInstance(Row1.index);
// rebuild the error list
xfa.form.execCalculate();
xfa.form.execValidate();

var vTime2 = (new Date()).getTime();
console.println(vTime2 - vTime1);

 

For my test I added 20 subform rows.  With a total of 21 rows there are 65 exclusion groups on the form.  Deleting a row causes all 65 groups to be re-evaluated.  (actually most of them get evaluated twice, but that is a bug behavior that I cannot control).  With the measurement in place I could see that deleting one row was taking 2753 milliseconds on my laptop.  As I deleted more, the deletions sped up -- since there were fewer groups to evaluate each time.  By the time I was down to two rows, the delete took 454 milliseconds.  All much too slow.

It did not take long for me to realize that my problem was the JavaScript objects I had defined to implement exclusion group behavior.  Here is where I expose my JavaScript naiveté.  I wanted to define an abstract group definition so that the implementation could be either based on a subform or on an array group.  When I set up the base class, I followed a C++ pattern.  I defined a base class with methods that the derived classes would override:

function exclusionGroup()
{
}
exclusionGroup.prototype.length = function() {return 0;}
exclusionGroup.prototype.item   = function(nItem) {return null;}
exclusionGroup.prototype.getMax = function() {return null;}
exclusionGroup.prototype.getMin = function() {return null;}
...

One of the derived classes:

// Implementation of exclusionGroup for a subform container
function subformGroup(vSubform)
{
    this.mSubform = vSubform;
}
subformGroup.prototype         = new exclusionGroup();

subformGroup.prototype.length  = function()        
    {return this.mSubform.nodes.length;};

subformGroup.prototype.getMax  = function()        
    {return this.mSubform.vMaxSelected.value;}

subformGroup.prototype.getMin  = function()        
    {return this.mSubform.vMinSelected.value;}

subformGroup.prototype.selectedContainer = function()
{
    if (this.mSubform.vOldSelectedContainer.value == "")
        return null;
    return this.mSubform.resolveNode(this.mSubform.vOldSelectedContainer.value);
}
...

What I failed to take into account is that JavaScript is completely interpreted.  e.g. When the getMax() method on the derived class gets called, it matters none that the method exists in a base class.  The interpreter simply checks the class instance to see if it has the getMax() method.  The base class was just extra processing overhead with no benefit.  The base class might have had some benefit if we had shared implementations of some methods -- but we didn't. So I removed the base class.  I created two classes: subformGroup and arrayGroup that both define the same member variables and functions.  There is no type checking.  The onus is on the programmer to make sure the property names, methods and parameters all match. 

The other revelation is that I did not need to extend the objects using prototype.  Using prototype impacts the class definition -- i.e. all instances of a class.  You can extend a single instance of a class without bothering with prototype.  That seemed to improve performance as well.

In the revised form, my object definitions looked like:

// Implementation of exclusionGroup for a subform container
function subformGroup(vSubform)
{
    this.mSubform          = vSubform;
    this.len               = vSubform.nodes.length;
    this.maximum           = vSubform.vMaxSelected.value;
    this.minimum           = vSubform.vMinSelected.value;
    this.selectedContainer = function()
    {
        if (this.mSubform.vOldSelectedContainer.value == "")
            return null;
        return this.mSubform.resolveNode
                    (this.mSubform.vOldSelectedContainer.value);
    } 
    ...
}

// Implementation of exclusionGroup for an array group
function arrayGroup(vHost)
{
    this.mHost             = vHost;
    this.len               = vHost.length.value; 
    this.maximum           = vHost.max.value;
    this.minimum           = vHost.min.value;
    this.selectedContainer = function()
    {
        if (this.mHost.vOldSelectedContainer.value == "")
            return null;
        return this.mHost.parent.resolveNode
                   (this.mHost.vOldSelectedContainer.value);
    }
    ...
}

Simplifying the object definitions had a big impact on performance.  Deleting the 21st subform went from 2753 milliseconds to 1236 milliseconds.  Well over a 2x improvement.  Still not fast enough for my liking, but good enough for one round of performance tuning.  Now maybe I'll go out and buy a JavaScript book...