Performance of Object Expressions

In a previous post I described how object expressions in JavaScript worked and compared them to using calls to resolveNode() and using FormCalc.  There is a trade off between readability and performance.  From the previous post:

“SOM parser evaluation is faster than JavaScript object evaluation.  If you have an expression that must be executed many, many times, then it is better to wrap it in resolveNode() or use FormCalc.  However, for most forms with modest amounts of script, the difference in overall form performance is negligible. Most of the time ease-of-scripting and code readability are more important.”

We were working on an internal prototype that made me want to quantify the difference in performance between object expressions and SOM expressions.  In order to measure performance I needed a form that would evaluate many, many expressions.
The result is a sample form that simulates John Conway’s famous Game of Life.

The Game of Life Form

A few notes about how the form works:

  • Each cell is a check box field.  You can re-arrange the starting pattern by toggling the fields
  • The grid is initialized by creating nested row/column subforms
  • The row and column subforms are renamed so that the object expressions don’t use indexes.
    e.g. grid.row[3].column[4] becomes grid.row_3.column_4

Most applications that simulate this game access the cells in nested loops.  For this sample, the loops are flattened so that we have very large scripts with hard coded object expressions.  For example, the code to find neighbours of the cell at (3,4) is:

vNeighbourCount = 0;
vNeighbourCount += form1.grid.row_2.column_3.cell.rawValue;
vNeighbourCount += form1.grid.row_3.column_3.cell.rawValue;
vNeighbourCount += form1.grid.row_4.column_3.cell.rawValue;
vNeighbourCount += form1.grid.row_2.column_4.cell.rawValue;
vNeighbourCount += form1.grid.row_4.column_4.cell.rawValue;
vNeighbourCount += form1.grid.row_2.column_5.cell.rawValue;
vNeighbourCount += form1.grid.row_3.column_5.cell.rawValue;
vNeighbourCount += form1.grid.row_4.column_5.cell.rawValue;
if (form1.grid.row_3.column_4.cell.rawValue)
  vNewState = (vNeighbourCount == 2 || vNeighbourCount == 3) ?1:0;
else
  vNewState = (vNeighbourCount == 3) ? 1:0;
form1.grid.row_3.column_4.cell.extras.newState.value = vNewState;

After one pass where we stored the new state for each of the cells in the extras of the cell field, we assign the new state:

form1.grid.row_3.column_4.cell.rawValue =
form1.grid.row_3.column_4.cell.extras.newState.value;

Repeat that code for each of the 2,500 cells in the grid and now the form has around 36,900 lines of JavaScript and 29,405 object references.  That should be good enough to test the performance of object expressions.

Then for comparison purposes, the form has another  script that does the same operation using calls to resolveNode():

vNeighbourCount=0;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_2.column_3.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_3.column_3.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_4.column_3.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_2.column_4.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_4.column_4.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_2.column_5.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_3.column_5.cell”).rawValue;
vNeighbourCount +=
  this.resolveNode(“form1.grid.row_4.column_5.cell”).rawValue;

if (this.resolveNode(“form1.grid.row_3.column_4.cell”).rawValue)
  vNewState = (vNeighbourCount == 2 || vNeighbourCount == 3) ?1:0;
else
  vNewState = (vNeighbourCount == 3) ? 1 : 0;
this.resolveNode
  (“form1.grid.row_3.column_4.cell.extras.newState”).value =
vNewState;

And one more comparison where we do the same in FormCalc:

vNeighbourCount = 0
vNeighbourCount = vNeighbourCount + form1.grid.row_2.column_3.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_3.column_3.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_4.column_3.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_2.column_4.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_4.column_4.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_2.column_5.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_3.column_5.cell
vNeighbourCount = vNeighbourCount + form1.grid.row_4.column_5.cell
if (form1.grid.row_3.column_4.cell) then
  vNewState = if(vNeighbourCount == 2 or vNeighbourCount == 3,1,0)
else
  vNewState = if(vNeighbourCount == 3, 1, 0)
endif
form1.grid.row_3.column_4.cell.extras.newState = vNewState;

Results

The good news is, that all three variations are *very* slow — great for measuring performance :-)

The performance results when I ran this on my laptop:

JS with Object Expressions JS with calls to resolveNode() FormCalc
Mlliseconds 7154 4673 1735
Milliseconds
per Expression
0.243 0.159 0.059

Clearly there are differences.  I was surprised at how much faster the FormCalc version was.  I would like to understand that better some day.

But now before you run off and re-code your forms to use resolveNode() or to use FormCalc, we have to put the numbers into perspective. For the slowest variation (object expressions in JavaScript), we evaluated 29,405 expressions in just over 7 seconds.  That means each expression evaluated in roughly 0.24 milliseconds.  I’m going to go out on a limb here and assume that this is fast enough for most forms.  Sure, the FormCalc version does it in  0.06 milliseconds, but your end users will not notice the difference.  Unless, of course, your form is doing something frivolous such as simulating the Game of Life.  The test validated the original assumption: in most cases code readability and maintainability trump any performance issues.

Something Useful

So far this blog post has not uncovered anything particularly useful, and I feel obliged to leave you with something that you can use in your day-to-day form design.

In the previous post that dealt with object expressions, I described how the JavaScript engines deal with “naked field references”.  There is a nuance that you should be aware of:  the JavaScript engines treat a variable with an explicit declaration differently from a variable without a declaration. i.e.

var foo = “hello world”;
xfa.host.messageBox(foo);

is handled differently from

foo = “hello world”;
xfa.host.messageBox(foo);

In the first case, the JavaScript processor will ask the XFA engine if it recognizes “foo”.  The XFA engine says “no” and from that point on, the JavaScript processor will not ask again when it encounters more references to “foo”.  In the second case where the “var” keyword is not used, the JavaScript processor asks the XFA engine for a definition of “foo” each time it encounters “foo” in the script.

Now consider what happens when you code:

for (var i=0; i<10000; i++)
  total += po.nodes.item(i).subtotal.rawValue;

vs.

for (i=0; i<10000; i++)
total += po.nodes.item(i).subtotal.rawValue;

In the first case, the XFA processor evaluates “i” once.  In the second case, the XFA processor evaluates “i” 30,000 times.  Would you notice the difference?  It depends on two factors:

  1. How many iterations in the loop and how many references to the loop counter
  2. The cost of one lookup – how many objects are in scope when we do the evaluation.  When the XFA processor searches for “i”, it does a physical scan through all objects that are within scope of the current context.

The Deep End

If you are writing script for a form that will run on the server, there is a bug you might want to be aware of.  As mentioned, once the JavaScript engine determines that an identifier is a variable or an XFA object, it will not ask again.  On the client this happens on a per-script basis.  However on the server this happens on a form-wide basis.  e.g. if an initialization script uses “foo” as a variable, then it will be assumed to be a variable in all other scripts on the form.  Another reason why my preference is to avoid using common names for variables.  For reliability — and for readability I prefer to prefix my variables e.g. “vIndex”.

6 Responses to Performance of Object Expressions

  1. Keith Gross says:

    After reading this another issue occurred to me. What impact would it have if we turned off strict scoping rules. So I grabbed the PDF and ran one round of tests with Strict Scoping and one without. The following were the averages.JS expressions with Strict Scoping – 12091JS resolveNode with Strict Scoping – 7969FormCalc with Strict Scoping – 2727JS expressions no Strict Scoping – 7079JS resolveNode no Strict Scoping – 5226FormCalc no Strict Scoping – 2748So strict scoping has no effect on FormCalc which is what I’d expect. On the Javascript based tests though removing strict scoping has a sizable impact making expressions more competitive with resolveNode. In fact removing strict scoping has a bigger impact then rewriting your code to use resolve node.This seems strange as I’ve been told more then once that I’d pay a performance penalty by turning it off but this seems to indicate exactly the opposite. If loose scoping is faster and IMHO easier to program for, why is Adobe pushing so hard for strict?

  2. Keith:That’s a great observation and a really good question.This really helps to illustrate differences between strict and non-strict scoping.I can explain…The performance differences are primarily around how we manage memory.When an XFA element is referenced in JavaScript, we create a JavaScript object corresponding to that element.With Strict scoping on, those JavaScript objects are released and are subsequently removed by the JavaScript garbage collector.With Strict scoping off, those JavaScript objects are never released.To test this hypothesis, follow these steps:1. start up your task manager in windows. Go to the “Processes” tab and sort on Image Name.2. Now open the non-strict version of gameoflife. You should see AcroRd32.exe at the top of the display.3. On my system the “Mem Usage” column for the non-strict version is around 124MB4. Close Reader. Either wait for the AcroRd32 process to go away or kill it.5. Open the strict version of gameoflife6. On my system the strict version has “Mem Usage” of around 92MB.It would appear the extra JavaScript objects in the non-strict scoping version use up around 30 MB of memory.(The memory usage is higher on open because there are initialization scripts that have already visited every object on the form).To test this more thoroughly, I modified the form and moved all the initialization scripts to a button.In this case, both forms open with the same memory footprint.When I click the initialize button for the non-strict version, the memory climbs by 30MB and stays there.When I click the initialize button for the strict version, the memory climbs by 30MB and then drops again.Then to really belabour the point, I added a button that shaded all the subforms and fields on the grid.i.e. for each object execute this script:vObject.border.fill.color.value = “192,192,192”;This will create JavaScript objects for each , and element on the form.In the non-strict version, the memory usage climbed by 45MB and stayed there.In the strict version it climbed and dropped again.So clearly, from a memory-usage point of view, strict is much better. And when memory usage continues to climb, it will begin to impact processing speed.This still doesn’t explain why the strict version was slower in this case — since our memory useage was not out-of-control.The theory is that the strict version is getting penalized by the garbage collection.(Just a theory because I have no way to prove it).But assuming that the theory is true, then scripts that do not access thousands of objects will not suffer the same garbage collection penalty.One way to test the theory would be to run the game of life on a much smaller grid.i.e. instead of accessing 5000 different objects 5 times each, try accessing 1000 objects 25 times each.In this case I’d expect the performance difference between strict and non-strict to be greatly reduced.Hope this helps.John

  3. Keith Gross says:

    I figured the difference was more the result of the persistence of Javascript objects helping the loose scoping version then garbage collection hurting the strict scoping version. I based this on the thought that once we’ve visited a particular cell a series of Javascript objects have been built that represent the path to the cell. Next time you access that cell or something along the same general path the engine doesn’t need to lookup up some of the objects a second time and build a script object. In addition, this assumes looking up an existing Javascript object attribute is faster then the equivalent resolveNode or Javascript Object expression.After thinking about this for a moment I decided that simply turning on strict scoping didn’t really provide a very good representation of how script scoping performs as the code wasn’t written with loose scoping in mind. The main issue would be in loose scoping you wouldn’t create the extra for each cell. You’d simply use a javascript attribute. So I changed the scripts and retested. Here are the results focusing only on Javascript object expressions in the 3 cases of strict scoping,loose scoping with extras for new value and loose scoping using attributes for the newValue.Original- On open 94MB used- Max during run 116MB- 5 seconds after run completes 98MB- 12.091 seconds runtimeOriginal with Loose scoping- On open 124MB used- Max during run 147MB- 5 seconds after run completes 129MB- 7.079 seconds runtimeLoose scoping rewritten- On open 96MB used- Max during run 126MB- 5 seconds after run completes 109MB- 6.501 seconds runtimeEliminating the use of the extras saved almost 20MB between the 2 loose scoping setups and made the loose scoping version much more competitive. Interestingly enough it also knocked another half second off the runtime which I hadn’t really been expecting.Overall in my tests loose scoping added roughly 10MB in the long term memory usage. Based on a very uneducated guess this 10MB represents ~10,000 Javascript objects which works out to about 1K per object which seems rather heavy. Of course perhaps more objects are created then I’m guessing and these objects might be full copies of the data from those nodes rather then simple proxies to those nodes.

  4. Keith:> I figured the difference was more the result of the persistence of Javascript objects helping> the loose scoping version then garbage collection hurting the strict scoping version.I suspect the non-strict version is also penalized because it has to create the JavaScript objects in each iteration — whereas the non-strict version re-uses them from a cache.> I based this on the thought that once we’ve visited a particular cell a series of Javascript> objects have been built that represent the path to the cell.I created a test for this case. The test has a subform called “Subform1” with an optional child subform called “border”.Initially there are no instances of “border”. Then when I ran this script:app.alert(Subform1.border.className);_border.addInstance();app.alert(Subform1.border.className);The first alert results in “border”, while the second alert results in “subform”.That indicates that the path is not remembered — it’s re-evaluated each time.> looking up an existing Javascript object attribute is faster then the equivalent resolveNode> or Javascript Object expression.Yes, this is true.If you wanted to design this form to run faster, there’s no question that you’d try to do more in native JavaScript and less with XFA objects and properties.> the code wasn’t written with loose scoping in mindWell, more to the point, it wasn’t written to get good performance :-)The fastest version of this form would probably define the gameoflife grid using JavaScript arrays in a Script object.After evaluating one iteration you would update just those fields whose value changed.This version would work fine (and fast) with both strict and non-strict scoping.> Overall in my tests loose scoping added roughly 10MB in the long term memory usage.> Based on a very uneducated guess this 10MB represents ~10,000 Javascript objects which works> out to about 1K per object which seems rather heavy. Of course perhaps more objects are> created then I’m guessing and these objects might be full copies of the data from those nodes> rather then simple proxies to those nodes.Everything that is represented as an XML element in the XFA template grammar gets an associated JavaScript object when accessed in script.In the original example, storing a property as an extra created two JavaScript objects from the and elements.(But you knew that already because you factored it in when you guessed 10,000 objects).I’m quite certain that the JavaScript objects do not cache XFA property values.Each time an XFA property is accessed, we go back to the XFA DOM — whether strict or non-strict scoping.(You can test this by modifying a field value via the data dom and then check that field.rawValue sees the change)Having said all that, I can’t account for the 1K/object size. I’ll ask around.There’s another ugly side to non-strict scoping — when you remove objects from the XFA DOM, the JavaScript objects do not get released.I changed the form to have the initialize script remove and re-create the cell subforms. In this case the memory footprint grew incrementally by 10MB each time.If your form allows users to add/remove subforms, this behaviour could cause memory to grow indefinitely over a long form session.My conclusions:a) Non-strict scoping causes the memory footprint to grow. You can mitigate by accessing fewer XFA properties, but those you do access will still persist.b) In order to gain performance on very large, very intense forms, you might want to store more properties in JavaScript — but that doesn’t mean you have to create them as properties on XFA objects.You ought to be able to arrange your code so that the JavaScript properties are stored in Script objects where they’ll work fine with strict scoping.c) For the vast majority of forms, the performance difference between using XFA properties and Javascript properties will not be noticeable to the end user.You need to also appreciate that not all users have your technical depth and won’t understand the limits of non-strict scoping.While it may be possible to build forms that perform well with non-strict scoping, it does require discipline. We had many support calls from customers who ran into severe memory-related performance issues that were fixed by moving to strict scoping.For these reasons, in XFA 3.0 we no longer support non-strict scoping. In reader 9.1 if the form specifies strict scoping we will run that form in 2.8-compatibility mode.With non-strict it’s just too easy for customers to create problematic forms.John

  5. Keith Gross says:

    We’ve known for a while that non-strict scoping was on the way out and have been cleaning up our framework to reduce our exposure but eliminating it entirely has been difficult.Item b in your conclusions catches my attention though. I know we tried what your describing and it didn’t work. But now I realize the answer was back in your post on Scope of JavaScript Objects. We had not yet made the transition to Reader 9.x and in version 8.1.x strict scoping released Script object variables along with everything else.You know I had a support call open on Script variables and strict scoping back in November because of this issue. If I gotten a better explanation of the implication of turning off non-strict scoping and the changes with respect to Script variable cleanup then I could have moved to version 9 sooner and avoided a lot of work.One final question though on your follow-up below. Not quite as important since based on the insight above I might actually be able to migrate to strict scoping. But in your experiment were you rebuild the table and the memory rises by 10MB each time. In non-strict scoping if I access a DOM node and a Javascript object gets built but I retain no reference to it and the XFA node gets removed in some fashion will the memory for the node and the object get collected?

  6. Keith:> One final question though on your follow-up below.> Not quite as important since based on the insight above I might actually be able to migrate to strict scoping.> But in your experiment were you rebuild the table and the memory rises by 10MB each time.> In non-strict scoping if I access a DOM node and a Javascript object gets built but I retain no reference to it> and the XFA node gets removed in some fashion will the memory for the node and the object get collected?In this case, I think the memory for the DOM object gets released, but the memory for the JavaScript object does not. The problem is that when we create a JavaScript object from the C++ code we have no way of knowing when the JavaScript reference count reaches zero. In non-strict mode, we kept JavaScript objects around permanently. The C++ code never releases it. In strict mode we mark the JavaScript objects to be released immediately after their script context goes out of scope. Of course, you can force them to stay in scope by having a script object variable reference to the object.I suspect your next question might be “why don’t we allow custom properties on JavaScript objects created for XFA objects in strict mode?”There are two answers:1) With strict scoping it is not obvious to most users that when they access the same node twice in two different scripts that they’ll likely get two different JavaScript objects — since the first reference had been garbage collected.They would not understand why a custom property created in their initialization script does not stick around for their validation script.2) Our JavaScript engine on the server does not support custom properties on JavaScript objects. One of our value propositions is that the same form works for both client and server.John