Posts in Category "Performance Tuning"

Populating List Boxes

When I occasionally browse around some of the LiveCycle forums, I frequently see questions around how to populate a drop down list.  I have put together a sample form that illustrates several different options.

Data Source

There are two basic sources for updating a drop down list definition: data binding or script.  If the list contents are defined as part of your form data, and if they don’t change during your form session, then use data binding.  If the definitions are more fluid, then use script.

The preOpen Event

The most important take-away from this blog entry concerns what event to use for updating lists.  I have seen customer forms that update list contents using change, exit, calculate, validate, mouseover, and enter events.  However, the proper place to do it is in the preOpen event.  The preOpen event fires when the user activates the control for the choice list.  Think of it as the "just-in-time" option.  If you try to maintain your list box definition from other events then often your script will update your list box contents too frequently. 

The only reason for updating a choice list sooner than the preOpen event is if you need to assign a value to a list field.  e.g. if your drop down list has a display value: "ONE" and a bound value: "1", then assigning
field.rawValue = 1; will cause the field to display "ONE".  Obviously this works only if the field has up to date list contents.  If you need your list contents updated more frequently, you should still put the list populating logic in the preOpen event, and use execEvent("preOpen") to populate the list from other contexts where it’s needed.

The sample form has four choice lists that get populated from their preOpen event, using data found in form field values, JavaScript arrays and XML data.

Script Commands

There are two script methods for setting your list box contents:
field.addItem()
   and
field.setItems()

The API: field.addItem() works in all versions of Reader.  field.setItems() was introduced in Reader 9.0 and is much faster and more convenient.  The sample form has script that illustrates how to use each method.

Binding to Data

I constructed some sample data (dogs.xml) that looks like this:

<dogs>
  <category file="ugliest">
    <rank>1</rank><dog>Chinese Crested</dog>
    <rank>2</rank><dog>Pug</dog>
    <rank>3</rank><dog>Shih Tzu</dog>
    <rank>4</rank><dog>Standard Schnauzer</dog>
    <rank>5</rank><dog>Chinese Shar Pei</dog>
    <rank>6</rank><dog>Whippet</dog>
    <rank>7</rank><dog>Dandie Dinmont Terrier</dog>
    <rank>8</rank><dog>Japanese Chin</dog>
    <rank>9</rank><dog>French Bulldog</dog>
    <rank>10</rank><dog>Chihuahuas</dog>
  </category>
  <category file="dumbest">
   …
  </category>
  <category file="comicBook">
   …
  </category>
  <category file="smartest">
   …
  </category>
</dogs>

On my form I want two drop down lists: one with the names of the categories and a second that gets populated with the contents of the category.  Since this XML is part of my data, I bound the category using these expressions:

binding

The second list has a preOpen event that locates the category in the data, and then populates a second listbox from the category contents:

// Find the data group that corresponds to the category chosen
// in the Category field

var vDogs = xfa.datasets.data.dogs.category.all;
var vChoice = null;
var i;
for (i = 0; i < vDogs.length; i++) {
    if (vDogs.item(i).file.value === Category.rawValue) {
        vChoice = vDogs.item(i);
        break;
    }
}
if (vChoice !== null) {
    // vChoice.dog.all is the equivalent of the
    //
SOM expression: "category.dog[*]"
    var vDisplayValues = vChoice.dog.all;
    var vBindValues = vChoice.rank.all;
    for (i = 0; i < vDisplayValues.length; i++) {
        this.addItem(vDisplayValues.item(i).value,
                    
vBindValues.item(i).value);   
    }
}

Performance

Populating a list from data is very efficient.  But populating large lists or many lists using addItem() can be slow.

The performance gains of setItems() over addItem() is substantial.  If your form makes extensive use of choice lists or has choice lists with large contents, you will appreciate the improvements of setItems().  Of course, this option is available only in forms designed for Reader 9 or later.

Web Services

In some cases, the definition of the choice list may be held on a server.  In this scenario, the best strategy is to add a WSDL connection to your form that retrieves the list contents.  Have your list box bind its contents to the data retrieved via the SOAP call.

Away for a week

After I eat lots of turkey on the weekend (Canadian Thanksgiving), I’m spending next week trying to empty the job jar at home.

Base64 Encode a PDF attachment

This blog entry has been re-published with updated information.

Some time ago I experimented with PDF attachments — trying to add them to my XML data.  I wasn’t happy with the outcome at that time, and I was going to leave it.  But then I saw a customer scenario that called for this capability, and then one of my regular commenters (Bruce) brought up the topic as well.  So I’ve tried again and had a little more success this time. 

The end goal is to copy a PDF attachments into XML data.  If you can do so, it opens up a couple of interesting possibilities:

  • Take any attachments a user has added to the PDF and include it in a form submission or in a web service request
  • Take image attachments and use them to populate image fields

Acrobat methods for attachments

The acrobat document object has a property: dataObjects that returns an array of all the attachments in the current document.  Then are a set of methods for dealing with attachments: openDataObject, getDataObject, createDataObject, importDataObject, removeDataObject, getDataObjectContents, and setDataObjectContents.

The interesting method in our case is getDataObjectContents().  It returns a stream object with the contents of an attachment.  If your attachment happens to be textual, then you can use util.stringFromStream() to convert to a string value:

var inputStream = event.target.getDataObjectContents("MyNotes.txt");
Notes.rawValue = util.stringFromStream(inputStream);

The default encoding for binary attachments is a hex-encoding (each byte written as a 2 digit hex value).  However, when your attachment is in a binary format, the standard way to include it in an XML file is to encode it as base64.  To convert to a base64 encoding, use the Acrobat Net.streamEncode() method:

// Get a stream for the image attachment
var inputStream = event.target.getDataObjectContents("Smile.jpg");

// Get a new stream with the image encoded as base64
var vEncodedStream = Net.streamEncode(inputStream, "base64");

// Get a string from the stream
var sBase64 = util.stringFromStream(vEncodedStream);

// assign the base64 encoded value to an image field
sampleImage.rawValue = sBase64;

We know there are issues with Net.streamEncode() failing where content has null bytes.  However, when used in the context of encoding an attachment, it seems to work fine.

When I first looked at this problem I assumed that the Net.StreamEncode() method wasn’t working so I wrote a base64 encoding JavaScript.  It works fine — but it is slow!  On a 140K image, it takes 10 seconds to encode.  I’ve included this code in the sample just for interest sake.

Conversion to Base64

The attached sample form has an initialization script that displays a subform for each attachment.  There are two buttons that will take the corresponding attachment, convert it to base64 and assign it to the image field value.  One button uses the (slow) JavaScript encoding algorithm the other button uses Net.streamEncode() and works pretty quickly. 

The easiest and most reliable way to encode attachments in your XML data is to keep them in a hex encoding.  But of course, for this to work the consumer of your XML needs to be able to handle hex encoding as well.

Dependency Tracking

One of the really cool aspects of XFA forms is the dependency tracking mechanism.  Dependency tracking is the feature where your field (and subform) calculations and validation re-fire whenever any of their dependents change.  Today I’ll explain a bit about the mechanics of how dependency tracking is implemented as well as have a look at possible design issues related to this mechanism.

Simple Example

Suppose my purchase order form has order rows with fields: price, quantity and subtotal along with a total field.

The subtotal field has a JavaScript calculation: "price.rawValue * quantity.rawValue"

The total field has a FormCalc calculation: "sum(order[*].subtotal])"

Now whenever price or quantity fields change, the corresponding subtotal field will be recalculated automatically.
Whenever any of the subtotal field values change, the total calculation will re-fire.

Discovering Dependencies

When the form is first opened, all the calculations and validations are executed.  While executing a calculation or validation, the XFA engine keeps track of each object (field, subform, data node) that gets referenced during the script execution.  Each referenced object is added to its dependency list.  In our example, each subtotal field becomes dependent on the price and quantity fields from its row.  The total field becomes dependent on all the order subforms and on all the subtotal fields.  The mechanism is robust enough that even if you reference your objects indirectly, we still find the dependency.  e.g. the subtotal calculation could be written as:

parent.nodes.item(0).rawValue + parent.nodes.item(1).rawValue

Since the nodes we visited were still price and quantity, the dependencies are established just the same.

In some cases, the dependency list will grow as values change.  Consider this calculation:

if (A.rawValue < 100) {
    A.rawValue;
} else {
    B.rawValue;
}

Suppose the first time it executes, field A has a value of 20.  This means the code will not execute the else clause and will not access field B.  The initial dependency list will include only A.  However, if the A of changes to 200, the calculation will re-fire; the else clause will be evaluated and field B will be added to the dependency list.

If your calculation or validation calls a script object, dependencies will continue to be tracked during the execution of the script object method.

Dependent Properties

What constitutes a field change? What changes cause a calculation to re-fire?  I don’t have a complete list.  But changing a field’s visual properties (colours, captions, presence) do not cause a dependent to recalculate.  Changing the value or changing the items in a list box will cause a dependent calculation/validation to re-fire.

Turn off Automatic Calculations

If, for some reason, you’re not happy with the default dependency tracking, then you can turn it off. There are two switches to control calculations and validations independently:

xfa.host.calculationsEnabled = false;
xfa.host.validationsEnabled = false;

Note that turning validationsEnabled off disables not only script validations, but also turns off mandatory field and picture clause validations.

Dependency Loops

It is possible to get into a situation where sequence of dependency changes goes into an infinite loop. In these cases, the XFA engine will allow the calculation or validation to re-fire a fixed number of times (10) and then will quietly stop re-firing the calculation or validation until the next time a dependent changes.  While the 10 times limit is ok in most cases, I have seen forms where this has been the root of performance issues.  If you have a dependency loop and your calculations are re-firing, you want to be aware of it and you need to fix it. 

Dependency loops are typically introduced when a calculation or validation modifies another field.

I should highlight that statement.  If your calculation or validation minds its own business and never modifies another field you shouldn’t run into any loops.  There are lots of cases where a calculation or validation modifies another field and everything works fine — but tread carefully.

Looping example

In order to track when my validations were changing, I wrote validation scripts write to a debug field:

field T1 validation:

debugTrace.rawValue += this.rawValue;
true;

field T2 validation:

debugTrace.rawValue += this.rawValue;
true;

Both T1 and T2 now have a dependency on field: debugTrace

The sequence of events:

  1. Field T1 changes
  2. T1 validation fires and modifies debugTrace
  3. T2 validation fires and modifies debugTrace
  4. T1 validation fires and modifies debugTrace
  5. T2 validation fires and modifies debugTrace

Here is a sample form to illustrate this example.

Note that if we remove the validation from T2, the circularity stops.  The validation on T1 modifies debugTrace, but after the validation completes, debugTrace does not change and T1′s validation does not re-fire.

Managing Tab Stops in Fields

When I was working on my previous blog entry, I struggled a bit with rendering JavaScript source code in fields.  The issue was how to accurately display text with the tab stops that are embedded in the code.  The goal was to have the JavaScript code appear with the same spacing you would see in the source editor.  For this to work, the tab spacing needed to be exactly 4 characters.  The solution involved:

  • Use a non-proportional font to display the text (courier new)
  • Figure out the width of 4 characters: a 10pt character in courier new has a width of 6 points.  4 characters is 24 points
  • Set the default tab for the field.  Since Designer doesn’t expose tab properties for fields, the form uses an initialization script to set the default tab: this.para.tabDefault = "24pt";

Tab Leaders

Having accomplished that much, it seems worthwhile to spend some time showing some of the other cool things you can do with tab settings in fields.  In Acrobat/Reader 9 we introduced tab leaders.  To see how this feature works with static text, check out Stephanie Legault’s tutorial.

That’s fine for static text, but what about fields?  If you want to use tab leaders in your form fields, you need to understand the field properties that control tab behaviours.  One way to get there is to read the relevant portions in the XFA specification.  Tab settings are paragraph properties.  A field’s paragraph properties are defined in the <para> element (Part 2/Template Specification/the para element).  Now, you could fill your head with all that book knowledge or you could code from examples

The first page of the sample shows some variations on tab settings.  Each sample field has an initialization script to set the tab properties.  In each case there is a corresponding field that displays the resulting <para> element syntax.

tabDefault

This property on the <para> element controls the default tab setting.  If you want a tab stop every half inch, this would look like:

<field>
    <para tabDefault="0.5in"/>
</field>

To set this via script:

form1.tabExample1.sample::initialize – (JavaScript, client)
this.para.tabDefault = "0.5in";

tab-stops

The general format of a tab leader is: "[alignment] [leader] location ".  You can string a bunch of these together to specify multiple tabs.  If you look at the sample form you will see some variations.  The alignment ( left (before) / right (after) / center / decimal ) determines how the text is aligned at the tab location.  The leader parameter determines how to fill the space leading up to the tab position — space, dots, rule, characters.  The 3rd parameter is the actual tab position.   Hopefully it’s all self-explanatory from the samples.

Tabs and Data Entry

So far, this discussion has been assuming that we’re rendering data that has embedded tabs.  Do tab settings work with interactive fields?  Well, yes and no.  The tab settings work fine.  The problem is, how exactly do you enter a tab character into an interactive field?  The tab key moves you to the next field.  The workaround the example uses is to define a key sequence that will insert a tab character.  Pressing "shift" + "space" will insert a tab character.  The way this works is that the field has a change event to translate this sequence:

form1.#subform[0].tabExample1.sample::change – (JavaScript, client)
if (xfa.event.change === " " && xfa.event.shift) {
    xfa.event.change = "\t";
}

Tabs and Rich Text

Rich text introduces a  couple of wrinkles wrt tabs. 

The tab character is not handled by XHTML.  XHTML processing rules tell us to collapse all white space down to a space character.  To get past this, we define a special style attribute to specify a tab:

<span style=’xfa-tab-count:1′/>

The attributes that exist on the <para> element are also available as style properties: tab-interval and tab-stop.  With a plain text field, the paragraph properties are constant for the whole field.  With rich text, the paragraph properties can be re-specified with each <p> element.

The second page of the sample form has an example where a table of values is formatted into a plain text and a rich text field.  To see the code, look at the form:ready event of each field.  The script for the plain text variation is much simpler, but the script for the rich text field does some more elaborate formatting.  But back up a moment.  Likely the idea of constructing rich text is new to some of you.  The technique is fairly straightforward — build a string representing the XHTML content and then use the loadXML() method to apply the text to the field.  Using XHTML allows the second field to enhance the output:

  • The tab settings for the first line (paragraph) are different from the rest of the field (center aligned with a space leader)
  • The cents portion of the prices are displayed in a superscript
  • The tab leader is styled — the baseline is raised so the dots appear in the middle of the line rather than the bottom.

The overall effect makes the second menu field a fair bit nicer.  Imagine what someone with artistic skills could do.

Aligned Dots

Did you notice?  The dots in the plain text menu line up with the dots used in the rich text menu.  By default, the dot leaders are aligned between different objects on the page. i.e. the dots are aligned on a grid defined at the page level.

The Deep End

Ambient Attributes

When the rich text is rendered, the display properties are determined by combining the field properties with the properties specified in the XHTML.  An example would help:

<field>
  <font typeface="Myriad Pro"/>
  <value>
    <exData contentType="text/html">
      <body>
        <p>Text 1<span style="font-family:Arial">Text 2</span></p>
      </body>
    </exData>
  </value>
</field>

In this example, the XHTML has not specified a font for the string "Text 1".  Whereas the string "Text 2" is explicitly styled as Arial.  This means that the font for "Text 1" is the ambient font — the font inherited from the field definition — in this case Myriad Pro.  Same applies to
tab settings.  By default the XHTML will inherit the tab settings from the field.  If you want to deviate from the default, then specify it inside a <p> element in the rich text.  That explains why in my rich text menu example I explicitly styled the first paragraph, but allowed the remaining text to inherit the field paragraph properties.

Building Big Strings

XHTML strings can grow very large.  You need to be careful how you build the strings.  Using the " += " operator can lead to performance issues.  Appending to a large string is expensive.  It breaks down to these set of operations:

  1. allocate new storage for the larger string
  2. copy the old string to the new storage
  3. append the new string to the new storage

The larger the string and the more append operations, the more expensive this becomes.  A more efficient alternative is to build an array of small strings and at the end, use the join() method to generate one large string.  Here are two examples for comparison:;

var S = "hello world";
S += " more text";
S += " even more text";

In comparison:

var StringArray = [];
StringArray.push("hello world");
StringArray.push(" more text");
StringArray.push(" even more text");
var S = StringArray.join("");

Of course, in this specific example, the strings are so small that the performance difference would not be noticeable. But when the number of append operations grow, the array technique will become much faster.

Collected Form Development and Debugging Tips

In this post I have consolidated a bunch of tools and tips for developing and debugging XFA/PDF forms.

Use the console Object Effectively

Always show the console on errors

When your form encounters a JavaScript error, the error text shows up in the console.  But by default the console doesn’t appear — which means you could be getting errors without being aware of them.  To see errors as soon as they happen, configure Acrobat to pop up the console as soon as a message appears.  Under Edit/Preferences, set the options highlighted below…

consoleDialog

 

The console methods

console.println() is an indispensable tool for tracing what is happening in your form script logic.  It is much more effective than xfa.host.messageBox() or app.alert(), since it doesn’t require dismissing a dialog with every message.

Did you know there are other methods on the console object?  clear(), hide(), show().  You might want to place a call to console.clear() when your form initializes.  When it’s time to deploy your form, make sure you haven’t left any extraneous messages in the console.

The Server Alternative

Since the console object is an acrobat-specific tool, you need to use something different to emit trace message for server-based XFA forms.  For the server, use xfa.log.message() to write your message to the server log file.

Trace Function Calls

As long as you’re in the mode of dumping information to the console or log file, you may as well make it as easy as possible.  This blog entry describes a method for tracing the input parameters to JavaScript functions.

Have a look at the data DOM

It’s often useful to see what the data currently looks like.  One way to get there is to add a big text field that displays the data dom.  Give it this one line calculation: xfa.datasets.saveXML("pretty");  Set the field as bind="none".  Now you will have an up-to-date snapshot of the state of the data dom.

Re-factor your form

Design for re-usability.  Develop common patterns that are shared between forms.  Create script frameworks inside script objects.  Put the framework script objects in your fragment library.

Use the Merge/Layout Debugger

If your problems are in merge or layout, then try this.  (Try it even if you’re not having problems with merge/layout — it’s cool.)

Exception Handling

When an exception is thrown and displayed in the console (or in the server log file) the resulting message can be frustratingly cryptic. To get higher quality information (including a stack trace), catch errors and parse them into human-readable text.  There is sample code here to get you started.

String Manipulations

If you are manipulating lots of strings — especially if you’re parsing them, get to know the power of regular expression processing in JavaScript.  Have a look at this regular expression testing tool.

Be Version Aware

If you make extensive use of script objects or if you’ve dabbled in using dynamic properties on XFA objects then get educated about the effects of strict scoping and how it behaves in different versions of XFA and Reader.  Read lots more here and here.

Summarize your form

This post shows how to get a "big picture" summary of the form meta data and the content in the form.

Document your Script

Take the time to properly document your JavaScript methods.  Use JavaDoc conventions. Six months from now when you (or someone else) needs to modify the form, you’ll be glad you took the time to explain your form logic.  To generate a report of all the script in the form, look here.

Reduce and Isolate your problem

If you are struggling with a particular problem, try to reproduce the problem in a new, simplified form.  The act of isolating will often lead to discovering any outside influences are affecting your form behaviour.  If you’re still having the problem in your new simple form, then you have a simplified version of the problem that makes it easier to share with others when you ask for help.

Designer Script Editor

Turn on Line Numbers

In your designer script window, turn on line numbers by right clicking on the script window and selecting "Show line numbers" from the context menu.

Don’t Leave Blank Lines at the Top of your Script

Blank lines at the beginning of a script get trimmed before the JavaScript interpreter processes the code.  If there are blank lines, then any line numbers reported by exceptions won’t agree with the line numbers you see in Designer.

Shortcut Keys

<control>+f — bring up the find dialog. Any highlighted text is automatically used as the find string.  <control>+h — brings up the find/replace dialog.  F3 — find next.  <shift>+F3 — find previous.

Multiple monitors

Convince your boss you need two monitors.  Keep Designer in one monitor and place your script editor and Acrobat console in the second monitor.

"foo.bar is not a function"

Your script makes a call to foo.bar() and you get the mysterious message: "foo.bar is not a function".  But you look in your script object and it sure looks like it is defined.  The problem is that you have a syntax error somewhere in your foo script object.  This prevents the runtime from discovering the foo.bar() function.  When you see this message, run Designer’s script syntax checker.

Declare your Variables

Don’t code: for (i=0, i<myList.length; i++)
Instead, code: for (var i=0, i<myList.length; i++)
The JavaScript interpreters process explicitly declared variables more efficiently.

While you’re at it, use variable names that are unlikely to collide with names of objects on your forms. Such as prefixing variables with a "v" or "n" or "s".  (vObject, nOffset, sStringValue).

Beware of Ambiguous Expressions

If you’re writing code that is intended to be re-usable in multiple forms, beware of potential conflicts between property names and form objects.  For example, the expression subform.border.fill.color becomes ambiguous if this script in a context where the subform has a field child named &quot
;border".  To avoid ambiguity, use the expression: subform["#border"].fill.color.  The hash symbol specifies that what follows is a className instead of a container name.

Find the Root Subform

If you have re-usable script that needs to locate the root subform, you can use this expression:

xfa.form.resolveNode("#subform")

Or the slightly more terse:

xfa.form["#subform"]

Don’t Modify Your Form in XML Source Mode

If you need to change a property that Designer doesn’t expose in one of it’s property editors, then change the property via script.  For example: Designer doesn’t support making a positioned subform repeatable.  Rather than adding an <occur> element in XML source, set this property in script:

Subform1.occur.max = "2";
var vNewSubform = _Subform1.addInstance();

Batch Changes to your Form

For advanced users — if you need to make global changes to the contents of your form, this blog entry shows how a form author can write JavaScript to modify their template in Designer.  Likely a bit easier than writing XSLT.

Use FormCalc Functions — from JavaScript

FormCalc has some built-in functions that give access to some powerful functionality.  There is a way to bridge between JavaScript and FormCalc in order to use these functions.  Look here.

June 16, 2009 update

Use Strict JavaScript

Run lint against your JavaScript.  Choose options that make it as strict as possible.  Details here.

DIY Column Chart

If you think about it, the XFA grammar has all the graphics primitives you’d need to generate a basic column or bar chart.  It has lines, rectangles and text.  It has repeating elements (subforms).  Ok, so it doesn’t have a shadow effect.  And 3D bars are off the table.  But theoretically, all that stands between us and a simple chart is a bit of JavaScript.

By searching the web you can find samples of packages that render charts in browsers using various JavaScript frameworks.  So why not do the same in an XFA form?

Have a look at this sample PDF (here’s the data).  In particular, try modifying a value in the table and watch the bars update.  Try modifying a value to $200,000 so that the Y-axis needs to be re-scaled.  Cool.

To use one of these charts in your PDF form, open this sample in Designer and turn the chart subform into a fragment that you’ll be able to bring into other forms.  When you open in Designer you’ll see some instructions that get hidden at runtime.  The components within the chart can be styled and customized as required.  The aspects that need not be set at design time are: the positions of the dynamic elements (labels, tics, bar), the contents of the labels and the size of the bars.

Options

Once the chart is sized, positioned and styled as you like, then look at the calculate script on the chart  subform to set the rest of the parameters.  At the top of the script you will see variables for controlling the rest of the chart behaviour:

  • Bar colours
  • grid line control (short tics vs. lines that extend across the chart)
  • x label staggering (prevent overlapping labels)
  • define the gap size between bars
  • Set the maximum number of labels on the Y axis
  • Column stacking (on or off)
  • SOM expressions defining the data series

In all it was around 450 lines of generously commented JavaScript. 

Oh, and lest you think this was all original, I did borrow some Java code (converted to JavaScript) to help with the Y axis label generation.

Performance Considerations

The form makes use of dynamic subforms to draw the variable elements of the chart — X and Y labels and bars. Once the subforms were created, the script resizes and positions them correctly.  A more efficient way to draw the chart would be to pre-create a fixed number of labels and bars at design time — not wrapped in subforms.  Drawing the chart would simply use the pre-created objects and hide the ones not need.  The problem with this approach is that it is harder to author — the author is responsible for making sure there are enough labels and bars.  My version of the form opts for easier design and a slightly less efficient runtime.  But either way, this approach of rendering graphics won’t scale well to larger more complex implementations. 

Sort Subforms

You probably already knew it was possible to re-order (and hence sort) a group of subforms.  But in case you didn’t — or in case you needed a sample, here you go.

In the sample, the column headings are button fields.  Their click event sorts the rows by the values in that column.  If you click again it will sort in descending order.

I centralized the sort methods into a script object that ought to be re-usable in other contexts:

/**
* Sort a group of subforms.  Calling this the first time will sort
* ascending. Calling it again will toggle to descending.
* @param vButton the button field used to initiate the sort. 
*  We store the current sort order in the button rawValue property.
* @param vInstanceManager the instance manager of the subforms to
*  be sorted
* @param vSortFieldName the name of the field in the subforms used
*  to sort.  Null values always sort to the beginning.
*/
function sort(vButtonField, vInstanceManager, vSortFieldName)

/**
* Sort a group of subforms.  Calling this the first time will sort
* ascending. Calling it again will toggle to descending.
* @param vSubformInstance one instance of a subform in the set of
*  subforms to be sorted
* @param vFieldName the name of the field used to sort
* @param bAscending boolean: if true, sort in ascending order,
*  otherwise descending.
*/
function performSort(vSubformInstance, vFieldName, bAscending)

Note that sorting subforms will also cause their underlying data to also be re-ordered.

Notes

Rather than embarrass myself by implementing a rookie bubble sort, I used the JavaScript Array.sort() method.  Of course, this meant figuring out how to set up a sorting function and letting it have access to the sort parameters (sort field and sort order).  But figuring out how to store these under <extras> was easier than trying to impress you with a fancy heapsort algorithm.

I generated the sample data at this very clever website: http://www.generatedata.com/ 

Deep End

If you sort a very large set of subforms, you might run into performance issues.  If you need to improve the performance, try sorting the underlying data DOM using the remove/insert methods on the nodelist.  After re-ordering the data you would do an xfa.form.remerge() and the subforms should show up nicely sorted.

Performance of Object Expressions

In a previous post I described how object expressions in JavaScript worked and compared them to using calls to resolveNode() and using FormCalc.  There is a trade off between readability and performance.  From the previous post:

“SOM parser evaluation is faster than JavaScript object evaluation.  If you have an expression that must be executed many, many times, then it is better to wrap it in resolveNode() or use FormCalc.  However, for most forms with modest amounts of script, the difference in overall form performance is negligible. Most of the time ease-of-scripting and code readability are more important.”

We were working on an internal prototype that made me want to quantify the difference in performance between object expressions and SOM expressions.  In order to measure performance I needed a form that would evaluate many, many expressions.
The result is a sample form that simulates John Conway’s famous Game of Life.

The Game of Life Form

A few notes about how the form works:

  • Each cell is a check box field.  You can re-arrange the starting pattern by toggling the fields
  • The grid is initialized by creating nested row/column subforms
  • The row and column subforms are renamed so that the object expressions don’t use indexes.
    e.g. grid.row[3].column[4] becomes grid.row_3.column_4

Most applications that simulate this game access the cells in nested loops.  For this sample, the loops are flattened so that we have very large scripts with hard coded object expressions.  For example, the code to find neighbours of the cell at (3,4) is:

vNeighbourCount = 0;
vNeighbourCount += form1.grid.row_2.column_3.cell.rawValue;
vNeighbourCount += form1.grid.row_3.column_3.cell.rawValue;
vNeighbourCount += form1.grid.row_4.column_3.cell.rawValue;
vNeighbourCount += form1.grid.row_2.column_4.cell.rawValue;
vNeighbourCount += form1.grid.row_4.column_4.cell.rawValue;
vNeighbourCount += form1.grid.row_2.column_5.cell.rawValue;
vNeighbourCount += form1.grid.row_3.column_5.cell.rawValue;
vNeighbourCount += form1.grid.row_4.column_5.cell.rawValue;
if (form1.grid.row_3.column_4.cell.rawValue)
  vNewState = (vNeighbourCount == 2 || vNeighbourCount == 3) ?1:0;
else
  vNewState = (vNeighbourCount == 3) ? 1:0;
form1.grid.row_3.column_4.cell.extras.newState.value = vNewState;

After one pass where we stored the new state for each of the cells in the extras of the cell field, we assign the new state:

form1.grid.row_3.column_4.cell.rawValue =
form1.grid.row_3.column_4.cell.extras.newState.value;

Repeat that code for each of the 2,500 cells in the grid and now the form has around 36,900 lines of JavaScript and 29,405 object references.  That should be good enough to test the performance of object expressions.

Then for comparison purposes, the form has another  script that does the same operation using calls to resolveNode():

vNeighbourCount=0;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_2.column_3.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_3.column_3.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_4.column_3.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_2.column_4.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_4.column_4.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_2.column_5.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_3.column_5.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_4.column_5.cell”).rawValue;

if (this.resolveNode(“form1.grid.row_3.column_4.cell”).rawValue)
  vNewState = (vNeighbourCount == 2 || vNeighbourCount == 3) ?1:0;
else
  vNewState = (vNeighbourCount == 3) ? 1 : 0;
this.resolveNode
  (“form1.grid.row_3.column_4.cell.extras.newState”).value =
vNewState;

And one more comparison where we do the same in FormCalc:

vNeighbourCount = 0
vNeighbourCount = vNeighbourCount + form1.grid.row_2.column_3.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_3.column_3.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_4.column_3.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_2.column_4.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_4.column_4.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_2.column_5.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_3.column_5.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_4.column_5.cell
if (form1.grid.row_3.column_4.cell) then
  vNewState = if(vNeighbourCount == 2 or vNeighbourCount == 3,1,0)
else
  vNewState = if(vNeighbourCount == 3, 1, 0)
endif
form1.grid.row_3.column_4.cell.extras.newState = vNewState;

Results

The good news is, that all three variations are *very* slow — great for measuring performance :-)

The performance results when I ran this on my laptop:

JS with Object Expressions JS with calls to resolveNode() FormCalc
Mlliseconds 7154 4673 1735
Milliseconds
per Expression
0.243 0.159 0.059

Clearly there are differences.  I was surprised at how much faster the FormCalc version was.  I would like to understand that better some day.

But now before you run off and re-code your forms to use resolveNode() or to use FormCalc, we have to put the numbers into perspective. For the slowest variation (object expressions in JavaScript), we evaluated 29,405 expressions in just over 7 seconds.  That means each expression evaluated in roughly 0.24 milliseconds.  I’m going to go out on a limb here and assume that this is fast enough for most forms.  Sure, the FormCalc version does it in  0.06 milliseconds, but your end users will not notice the difference.  Unless, of course, your form is doing something frivolous such as simulating the Game of Life.  The test validated the original assumption: in most cases code readability and maintainability trump any performance issues.

Something Useful

So far this blog post has not uncovered anything particularly useful, and I feel obliged to leave you with something that you can use in your day-to-day form design.

In the previous post that dealt with object expressions, I described how the JavaScript engines deal with “naked field references”.  There is a nuance that you should be aware of:  the JavaScript engines treat a variable with an explicit declaration differently from a variable without a declaration. i.e.

var foo = “hello world”;
xfa.host.messageBox(foo);

is handled differently from

foo = “hello world”;
xfa.host.messageBox(foo);

In the first case, the JavaScript processor will ask the XFA engine if it recognizes “foo”.  The XFA engine says “no” and from that point on, the JavaScript processor will not ask again when it encounters more references to “foo”.  In the second case where the “var” keyword is not used, the JavaScript processor asks the XFA engine for a definition of “foo” each time it encounters “foo” in the script.

Now consider what happens when you code:

for (var i=0; i<10000; i++)
  total += po.nodes.item(i).subtotal.rawValue;

vs.

for (i=0; i<10000; i++)
total += po.nodes.item(i).subtotal.rawValue;

In the first case, the XFA processor evaluates “i” once.  In the second case, the XFA processor evaluates “i” 30,000 times.  Would you notice the difference?  It depends on two factors:

  1. How many iterations in the loop and how many references to the loop counter
  2. The cost of one lookup – how many objects are in scope when we do the evaluation.  When the XFA processor searches for “i”, it does a physical scan through all objects that are within scope of the current context.

The Deep End

If you are writing script for a form that will run on the server, there is a bug you might want to be aware of.  As mentioned, once the JavaScript engine determines that an identifier is a variable or an XFA object, it will not ask again.  On the client this happens on a per-script basis.  However on the server this happens on a form-wide basis.  e.g. if an initialization script uses “foo” as a variable, then it will be assumed to be a variable in all other scripts on the form.  Another reason why my preference is to avoid using common names for variables.  For reliability — and for readability I prefer to prefix my variables e.g. “vIndex”.

Form Compatibility

In order for customers to successfully distribute forms, they require predictability.  Predictability is more important than bug-free software (if there were such a thing).  Predictability means the form you originally developed continues to work the same when opened in a new version of Adobe Reader.  "the same" means it has the same set of features and the same set of bugs as in the previous version.

Same features, same bugs.  If you want the new features and bug fixes you need to bring your form into Designer and target it for a newer release of Reader.  But even so, for complex forms you probably prefer to get the new features without necessarily getting the new bug fixes.  Bug fixes can change form appearance and/or script behaviour.  What users most often want is to use that cool new feature, but not to tweak their layout due to the fixes in form rendering.

To accommodate these needs, there are two dials that control the form: the target version and the original version.

Target version

field.setItems() was an enhancement to the XFA object model for Reader 9.  Suppose Adobe Reader 9 opens a form Designed for Reader 8.  What should it do if this Reader 8 form happens to have a script calling field.setItems() ?  It should do the same thing that Reader 8 does: throw an error. 

Each form is stamped with a target version.  The stamp is expressed as the XFA version.  Reader 8 uses XFA version 2.6.  Reader 9 uses XFA version 2.8.  Whenever Reader is asked to execute a script function, it first checks whether that script function is valid for that version of XFA.

This applies to more than just script functions.  It applies to XFA markup as well.   e.g. Hyphenation was added in XFA 2.8.  If Reader encounters a 2.6 form that has the markup commands to turn on hyphenation, the hyphenation will not be enabled.

The XFA version is defined by the XML namespace of the template: e.g.

<template xmlns="http://www.xfa.org/schema/xfa-template/2.8/">

The target version is controlled in the Designer UI under form properties/defaults.  Note that by default, Designer will set the target version one version back from the current shipping version of Adobe Reader.

Original Version

Form behaviour is determined by more than just new features, it is also determined by bug fixes. In some cases fixing a bug in Reader can actually break existing forms that inadvertently relied on the buggy behaviour. Therefore by default we preserve old behaviours when migrating a form forward.

We preserve the old behaviour by storing the original version of the form in a processing instruction in the template. When you modify your form in Designer to take it from Reader 8.1 to Reader 9.0, we embed the original version inside the form so that you end up with the version 9 features, but the version 8.1 behaviours.  The processing instruction looks like this:

<?originalXFAVersion http://www.xfa.org/schema/xfa-template/2.6/?>

When Reader 9 opens a form with an original version of 2.6, it makes sure that the behaviours are consistent with Reader 8.1.

This is all well and good, but there are times where you really, really need the latest bug fixes.  Unfortunately there’s no UI to move the original version forward.  If you need the latest and greatest bug fixes, the go into the XML source tab, find the originalXFAVersion processing instruction and delete it.  Now your form will get all the behaviours and features of XFA 2.8 and Reader 9.

Sample

I have attached a sample form that demonstrates changing the target and original versions.  There are three interesting part to the form.

1. The form has both target version and original version set to 2.6 (Reader 8.1).  There is a button on the form that extracts these values from the template and populates a couple of fields with the result.

2. When you open the form in Designer, you will see a field displaying current date and time in ISO8601 format.  This field has a calculation that looks like this:

this.rawValue = "20090205T163000";
xfa.host.currentDateTime();

When you preview this form, the first line executes fine, but the second line encounters the currentDateTime() function that was added to XFA version 2.8.  As a result, the script fails on the second line, and the value of the field remains "20090205T163000".  If you open the script console in Acrobat (control+J) you’ll see the error.

3. The bottom part of the form illustrates a bug fix that was made in Reader 9.  Hidden fields are not supposed to affect layout.  The hidden property has always behaved properly in flowed subforms, but we discovered a bug in positioned containers.  Prior to Reader 9 a hidden field would expand the extent of a growable, positioned subform.  In this form, the subform is taller than it should be.

Update The Target Version

Change the target version (in the form properties/defaults tab) from "Acrobat and Adobe Reader 8.1 or later" to "Acrobat and Adobe Reader 9.0 and later".  Now when you preview the form you will see:

1. The target version is now 2.8, but the original version remains 2.6.

2. The calculation now works without error. The field displays the current date and time.

3. The growable subform renders at the same height as it did in Reader 8.1

Update the Original Version

Go into XML source mode and remove the originalXFAVersion processing instruction.  Now when you preview the form you will see:

1. Both target and original versions are 2.8

2. The calculation continues to work

3. The growable subform has collapsed to an extent that does not include the hidden field

The Deep End

Discovering Processing Instructions at Runtime

I was able to get at the processing instructions by loading an E4X XML object with the results of xfa.template.saveXML().  I don’t recommend doing this in a production form, since the result of calling saveXML() on the template can be a very, very large string — especially if there are images embedded in the template. 

If you wanted just the target version (the xml namespace), there is an easy way to get it with the ns property: xfa.template.ns. 

FormTargetVersion

I simplified a couple of details in order to make this easier to explain.  The full story is a little more complicated.

When Designer opens a form that has markup that is newer than the target version, it will automatically update the namespace of the XFA template.  So for example: Designer opens a 2.6 template with a target version of Reader 8.1.  It discovers syntax for turning on letter spacing, so it will automatically update the template to 2.8.  But the target version remains Reader 8.1.  This is because the UI for target version is actually driven by another processing instruction:

<?templateDesigner FormTargetVersion 26?>

Under most circumstances, Designer will keep the FormTargetVersion and the XFA template version synchronized.  If the template version is newer, then you will undoubtedly find a bunch of messages in your warnings tab.  This is Designer’s way of telling you that you’re using features that do not work in your target version.  Until you clean those up, your form will not work correctly in your target version.

Choosing Selected Changes

Changing the original version is pretty high level switch.  You either get all the bug fixes for a new release, or you get none.  In reality, there are qualifiers that allow you to preserve selected behaviours.  Suppose that in my sample form I wanted the bug fixes that came with Reader 9/XFA 2.8 except for the fix for hidden fields in positioned subforms.  In that case I can specify the processing instruction as:

<?originalXFAVersion http://www.xfa.org/schema/xfa-template/2.8/ v2.6-hiddenPositioned:1?>

The qualifier: v2.6-hiddenPositioned:1 tells Reader 9 to preserve the XFA 2.6 behaviour for hidden fields in positioned subforms.

Scripting bug fixes

Most of the time, I am happy with the default behaviour where original version behaviour is enforced by default.  However there is one exception.  My previous post described how the behaviour of strict scoping changed from 8.1 to 9.0.  The difference is that with strict scoping on in 8.1 we release JavaScript variables declared in script objects.  In 9.0 we preserve these variables.  If you are targeting 9.0 and make extensive use of script objects, make sure that you set your original version to 9.0 as well.  

Changing Default Behaviours

There have been a couple of times where we have changed behaviour and made the new behaviour the default without protecting it with the original version mechanism.  The most infamous was when we added "direct rendering" for dynamic forms.  In this case we had developed a way to dramatically improve the performance of dynamic forms.  We had to choose between turning the new behaviour (and the performance boost) on for all forms or just for those forms designed for 8.1 and later.  We chose to make the new behaviour the default.  If this caused problems, the form author could "opt out" by inserting an originalXFAVersion processing instruction.

This is described in the knowledge base article: http://kb.adobe.com/selfservice/viewContent.do?externalId=kb402409

Populating list boxes

One of the Reader 9 enhancements was a new API call to populate list boxes: field.setItems(). The motivation for the new API is to provide better performance for populating lists.

Prior to Reader 9, the standard way to populate a list box is to call:

field.addItem(displayValue [, boundValue])

for each item in the list.  The new API looks like:

field.setItems(itemListString [, numColumns])

The first parameter is a comma-separated list of values, the second parameter is an integer telling the field how many columns are in the data (defaults to one).  The second parameter is designed for future extensibility if we choose to some day implement a multi-column list box.

Examples

A call to populate a listbox with currencies might look like:

Currency.setItems(
  "US Dollar,Canadian Dollar,Euro,United Kingdom Pounds");

Or if there were a bound value, it would look like:

Currency.setItems("US Dollar,USD,Canadian Dollar,CAD,Euro,EUR,United Kingdom Pounds,GBP", 2);

Prior to Reader 9, this second variation would have been coded as:

Currency.clearItems();
Currency.addItem("US Dollar", "USD");
Currency.addItem("Canadian Dollar", "CAD");
Currency.addItem("Euro", "EUR");
Currency.addItem("United Kingdom Pounds", "GBP");

Alternative

There is a 3rd method for populating listboxes: binding them to data.  Designer allows you to point your field at a location in your instance data where list box contents will be stored.  While this method has very good performance, it has the disadvantages that a) your data is not always in the correct format for binding, b) the listbox gets populated from data only during the initial data load.

Performance

If you are using listboxes only casually you probably will not notice the difference in performance between the two methods. But if you are using listboxes intensively, the new method is a life-saver.

I have attached a form where I compare the old performance to the new. On my laptop, I populate a listbox with 500 items in 125 milliseconds using addItem() calls, and in 16 milliseconds using setItems(). Neither of these numbers may seem significant, but we have customers with forms containing many list boxes with many, many entries where the difference in performance is critical.

Compatibility

If you are designing a form to use this new API, be sure and set your target version (in Form Properties/Default) to "Acrobat and Adobe Reader 9.0 or later".  Unless you do this, calls to setItems() will not work — even though you might open the form in Reader 9.