Recently in variables Category

It’s a little late in the season for spring cleaning, but better late than never. I have struggled with how best to deliver and (more importantly) update the samples delivered on this blog. Normally what I do is add a note to the blog entry where the sample was introduced and then hope that people notice (do RSS readers notify you when an entry gets updated?) But then there’s the problem where a framework has evolved over a series of blog entries. Which entry has the most recent version of the validation framework? Dunno. How do you know if you’re using the most recent version of the debugger or the lint checker?  What if I find a bug in the tool and want to send you an update? How do I best communicate that to you?

I am fairly certain that I’m not the first to struggle with these problems. Many of you will have similar issues in distributing forms to your end-users. Given that, I am working on a methodical way to track content. The result is a framework that you could adapt for your own form distribution.

Here’s my revised plan for distributing content:

• Distribute script objects as fragments (in addition to embedding them inside the samples)
• Embed version information in both the sample forms and fragments
• Post an inventory XML file with version information on the blog site
• Embed script in the sample forms so they can “call home” and check their version number(s)

Distribute as fragments

From now on when I distribute a sample I will also distribute any re-usable pieces (subforms or scripts) as downloadable fragments (XDP files).

The sample forms will have their fragment references embedded. This is to make it easier to open the sample in Designer -- without the need to re-connect the fragments.

Embed Version History

Once version information is embedded in a form or fragment, then we have the basis for checking whether there is a newer version available.  The question is how best to embed the version information.

My first thought was to embed the information in the form metadata. But there is not yet enough tooling and support around metadata to make this work well. In the long run, I think this would be the right solution. But in the short term we can’t easily get there from here.

The next best solution is to embed the version information in script object variables. Here is the script object variable for the fragment used in the sample that allows formcalc to be called from JavaScript:

var oVersionInfo = {
  identifier: "FormCalcFromJavaScript",
  assetType: "fragment",
  description: "JavaScript access to FormCalc built-in functions.", 
  currentVersion: 2,
};

Some interesting things to note:

• The same script object is used to describe both fragments and forms.
• The variable must be named “oVersionInfo” in order to be detected by the version checking mechanism.
• A form that uses multiple fragments will have several of these “oVersionInfo” variables embedded.

Inventory

Having version information in the form is the first step.  The second step is to have a central location to store the up to date version information. To this end, I've posted an "inventory" at: http://blogs.adobe.com/formfeed/Samples/Inventory.xml

The inventory will be used to compare the version information in a given form against the most recent available. The xml for the inventory looks very similar to the JavaScript object:

<inventory xmlns="http://blogs.adobe.com/formfeed/">
  <asset>
    <identifier>PhoneHome</identifier>
    <assetType>fragment</assetType>
    <description>Compare version info to the latest</description>
    <currentVersion>1</currentVersion>
    <filename>checkVersion.xdp</filename>
  </asset>

  <asset>
    <identifier>FormCalcFromJavaScript</identifier>
    <assetType>fragment</assetType>
    <description>Support for FormCalc functions</description>
    <currentVersion>2</currentVersion>
    <filename>scFormCalc.xdp</filename>
  </asset>

  <asset>
    <identifier>ExceptionHandler</identifier>
    <assetType>fragment</assetType>
    <description>Handle exception objects</description>
    <currentVersion>2</currentVersion>
    <filename>scMessage.xdp</filename>
  </asset>

  <asset>
    <identifier>FCfromJSSample</identifier>
    <assetType>form</assetType>
    <description>Demonstrates how to call FC from JS.</description>
    <currentVersion>2</currentVersion>
    <filename>FormCalcFromJS.pdf</filename>
  </asset>
  ...
</inventory>

Phone Home

Now to put it all together.  We need some script to check whether a form has the most recent versions of all its content.

The first step is to update the sample from the “FormCalc from JavaScript” entry.  I added support for FormCalc's Get() function.   In order to retrieve the inventory, the form performs a call:

fc.func.Get(“http://blogs.adobe.com/formfeed/Samples/Inventory.xml”);

(Acrobat/Reader will warn you about this request)

The result then loaded into an E4X XML object. Meanwhile we scan the rest of the form looking for script objects with an “oVersionInfo” variable. We compare the version information embedded in the form against the version information in the inventory and notify regarding any available updates.

The fragments to make this all possible are:

scFormCalc.xdp : The subform holding the FormCalc functionality.

scMessage.xdp : Updated exception handling methods

scVersion.xdp : Contains the logic for the "phone home" capability -- compare the form version information against the inventory.

scXML.xdp : Has a method for trimming an XML string so that E4X can load it.

Futures

Server-side logic

I chose to store the version checking logic in the sample form itself.  A better solution would be for that logic to reside on the server.  That way, the version checking logic could be updated without changing all the distributed forms.  But given my limitation of distributing via a blog site, I went with the client side logic.

Tools

I have a couple house-keeping tools I'm using that are not ready for prime time:

• A form to harvest version information from a fragment/form and update Inventory.xml
• A form to externally introspect a form for the latest version information (as opposed to embedding this logic in the form itself)

If there's interest, I will consider polishing these to the point where they can be shared.

Source Control

I've looked at the versioning problem primarily from a distribution point of view.  I didn't try to do anything from the perspective of source control.  There are things that could be done to make source code control easier:

• Store revision details -- e.g. Store comments, dates, author info for each change made to a form or fragment
• A tool for tracking differences between versions of forms. e.g. report on all added/removed/modified fields/subforms/scripts etc between versions.
• Integration of version information with form metadata

(No promises that I'll manage to get to these topics.)

Samples

In the short term my challenge is to update a bunch of the samples I've previously distributed.

For form calculations, validations and general scripting problems, our form designers can choose between using FormCalc and JavaScript. FormCalc was designed as an expression grammar targeting forms functionality -- and designed to be easy to use for a novice form author familiar with expressions in Excel.  JavaScript on the other hand, is a much more general purpose and powerful scripting language.  Over time, form authors have gravitated more toward JavaScript, mainly because of its familiarity.  However, there are functionality gaps -- specifically a set of built in functions that are available in FormCalc but not in JavaScript. 

Many of the built-in FormCalc functions can be easily mimicked in JavaScript.  It doesn't take a rocket scientist to write a JavaScript script object function to imitate the FormCalc sum() function. However, there are some functions that are not so easily mimicked.  The most notable of these are the Parse() and Format() functions available in FormCalc.  Parse() and Format() are the entry points into the very rich picture clause processing functionality.  When you consider the inherent locale handling, date/time functionality, different calendars etc. it's plain to see that you don't want to do this in JavaScript.

But now we have a problem.  Many users are committed to JavaScript because they built frameworks for controlling validations and other form behaviours.  I did the same in the series of blog posts on validations (the most recent version of the framework was in the sample from this post).  The problem is that you cannot directly call FormCalc from JavaScript.  So it would seem that you can't enjoy both the power and familiarity of JavaScript as well as the built-in functions of FormCalc.  Well, ... actually you can.

There are two design patterns I'd like to cover:

1. Validating fields with picture clauses
2. General mechanism for calling FormCalc functions

Validating Fields with Picture Formats

Picture clause validations are a simple, declarative mechanism for form authors to ensure that input data complies with a specific format.  If the field value can successfully be formatted with the validation picture clause, then the field is considered valid.  e.g. if your validation picture clause is "Date{YYYYMMDD}", then the field is considered valid only if its value can be formatted as a date.  If you were to express this validation as a script, you could write this FormCalc expression:

form1.#subform[0].DateTimeField1::validate - (FormCalc, client)
format("YYYYMMDD", $) <> ""

Now the question is how to tap into this functionality from JavaScript.  The short answer:

1. Define a *display* picture format (do not a validation picture format)
2. Write a JavaScript validation script that returns true when the field.rawValue is different from field.formattedValue

Most often when you use a validation picture clause you also use a display picture.  In fact, there's really no reason why these picture clauses need to be different.  Combine that knowledge with an understanding of how the field.formattedValue property works:  when a field value can be formatted using the display picture, field.formattedValue will return the result of the format operation.  If the format operation fails, field.formattedValue returns the same as field.rawValue.  So to find out if a field was formatted correctly, use this JavaScript validation:

form1.#subform[0].DateTimeField1::validate - (JavaScript, client)
this.rawValue != this.formattedValue;

Mechanism for Calling FormCalc Functions from JavaScript

The solution is to use the execCalculate() method to indirectly cross the bridge between JavaScript and FormCalc.  When you open the attached sample, you will find a subform called "fc" that holds a script object called "func".  "func" has a series of embedded JavaScript functions that mimic the FormCalc functions with the same name.  Each function populates form variables with the function name and input parameters.  It then calls execCalculate() on the fc subform and returns the resulting value:

FCfromJS.fc.#variables[0].func - (JavaScript, client)
function Format(vPicture, vValue)
{
    F.value = "Format";
    P1.value = vPicture;
    P2.value = vValue;
    this.execCalculate();
    return ResultString.value;
}

The subform calculation script looks like this:

 FCfromJS.fc::calculate - (FormCalc, client)
; execute the requested function based on the input
; request parameters
if (F == "WordNum") then
    ResultString = WordNum(P1)

elseif (F == "Parse") then
    ResultString = Parse(P1, P2)

elseif (F == "Format") then
    ResultString = Format(P1, P2)

elseif (F == "Uuid") then
    ResultString = Uuid(P1)

elseif (F == "UnitValue") then
    ResultString = UnitValue(P1, P2)

else
    ResultString = ""
endif

The field that wants to use the format() functionality has a simple calculate script:

FCfromJS.#subform[1].format.Result::calculate - (JavaScript, client)
fc.func.Format(PictureClause.rawValue, Value.rawValue);

This fc subform can easily be incorporated as a custom library object in Designer that can be dragged onto any form.  It should be pretty easy to follow the design pattern if you want to extend the sample and add other FormCalc functions.

One usage note -- if you want to call these functions from initialization scripts, then be sure to place the fc subform at the beginning of your template.  This is necessary because the fc subform has an initialization script that creates the necessary form variables.  By placing this subform at the top of the form hierarchy, we'll be certain that the fc initialization event fires before other initialization events.

August 12, 2009 Update

I have updated the sample form:

• Added access to the formcalc Get() function. 
• Added a version checking mechanism (see this post for details).
• The subform housing this functionality is now available as a downloadable fragment: scFormCalc.xdp
• The functionality is accessed using the subform "enter" event rather than the calculate.  The calculate event was introducing unwanted calculation dependencies.

If you are in the business of producing statements for your customers, you are likely interested in adding some targeted advertising. Just adding an advertisement to a form is not hard – the hard part is figuring out how to add the advertisement without causing more pages to be added to your printed output. i.e. how do we place ads in the available whitespace.

There are a couple of different design patterns where you can use JavaScript to discover (and use) available whitespace on dynamic XFA/PDF forms. I will cover the first pattern in today’s post.

To see what the end result looks like, open this sample form. On open we have determined that there is enough room for a very large ad, and we place a “flyer” that is 8 inches tall. Try adding more detail rows by pressing the “+” button. This causes the available white space to shrink and forces us to choose smaller advertisement subforms. After adding two rows, the ad shrinks to a 7 inch flyer. This continues until there is no room for any ads. But then after adding one or two more rows we spill over to a new page. Now once again there is room for a large ad on the second page.

Let’s take a closer look at how this form was designed.

Do nothing until layout:ready

Rendering a dynamic form goes through several processing stages:

1. Loading content
2. Merging data with template (create Form DOM)
3. Executing calculations and validations
4. Layout (pagination)
5. Render

There a couple of script events that tell us when some of these processing stages are complete. The form:ready event fires when step 3 is complete. The layout:ready event fires after step 4.

Discovering whitespace is dependent on having a completed layout. Once all our content has been placed on pages we can examine the resulting placements. Consequently, we put the logic for placing our ad in a layout:ready event. But there is a problem: changing the form after layout:ready causes … another layout and another layout:ready event. Unless we are careful, we will cause an infinite loop of constant [re]layout. To guard against this, the form creates a form variable: vEvalTranspromo to indicate that we are ready for evaluation. Once we have placed our ad, we set it to false. Any time we change the structure of the form (add/remove a subform) we set it to true. (Note that normally we would do this on the server with a print form and would not worry about changes to the structure after the initial layout.)

Structuring the form

This example requires placing your ad content in your form at the spot where a page break will occur. At this location we place an ad container subform which is a wrapper that holds a series of [optional] child subforms of varying sizes. These child subforms hold the ad content.  Initially there is no content in the ad container, so it has a height of zero and does not impact layout. Our whitespace calculation is based on taking the Y position of the ad container, and subtracting it from the height of the content area.

When we have discovered how much space is available, we create the largest possible child ad subform to fit.

Next Steps

This form looks at only one aspect of the transpromo problem: discovery/usage of whitespace. Another key component is to select ads based on context. If this were a credit card statement, we would select the ads based on the transaction and customer data. This level of logic requires some sort of integration with a rules engine. This could happen as a pre-processing step or as a web-service request during layout.

Hopefully this will have whetted your appetite for transpromo. The sample is not overly complex (less than 100 lines of JavaScript). There is another Design pattern that involves more script, but also allows greater flexibility in ad placement. But that will have to wait until January.

Until then, enjoy your holidays. 'Peace on earth, goodwill toward men'.

Continuing from the previous post, we are looking at the problem of validating fields without inundating the user with message boxes during their form session in Adobe Reader -- and without centralizing all the validation logic.

In the previous post, we established a design pattern for highlighting fields where a script validation fails. In this post we will deal with mandatory (required) fields.  Today, when you mark fields as mandatory, then any time your form is validated you will get an error message for each missing field.  We can do better.

The updated sample (with data) builds on the previous sample. The strategy is to

• author the form fields using the standard XFA “required field” settings
• At runtime disable the “required field” setting and store our own mandatory flag elsewhere in the field
• Use our setStatus() function during validation to monitor whether fields are populated or not.

Changes from the previous sample:

• Marked various fields as being mandatory
• Additional logic in scValidate.setStatus() to cause empty mandatory fields to show up as invalid
• A new script function: scValidate.setFieldMandatory()

Storing the mandatory state in a field involves a technique described in a previous post where we store a variable under a field.desc element.

Once we have logic in the form that stores our mandatory state, we let the XFA processor believe that no fields on the form are required. This way we don’t get any warnings from Acrobat/Reader, but we graphically modify the fields to highlight them for the user.

The only deviation from standard form design is that for mandatory fields you need to add a call to utility.setStatus() in the validation script – even if there is no script validation required.  i.e. if you mark a field as mandatory, you need to add this validation script:

expenseReport.details.empID::validate - (JavaScript, client)
scValidate.setStatus(this, true);

Conditionally Mandatory

On the sample form, when the payment type field choice is "Direct Deposit", there are three more fields that need to be filled in. If the payment type is any other choice, these fields do not need to be filled in.

Normally we toggle a field’s mandatory status by setting the property: field.mandatory. But now that our design pattern for mandatory fields has co-opted the XFA mandatory mechanism, we will use one of our new global methods: scValidate.setFieldMandatory(vField, vMandatoryState).

For this to work, the form defines validation scripts on the financialInstitution, branchNumber and accountNumber fields that look like this:

expenseReport.payment.deposit.financialInstitution::validate

var vMandatory = directDeposit.rawValue == 1;
scValidate.setFieldMandatory(this, vMandatory);
scValidate.setStatus(this, true);

Updated Exclusion Groups

If you are looking at the script code you will notice that I have included the exclusion group script objects that were defined in a previous post.  The form has placed the direct deposit subform in an exclusion group with the other payment types. I have modified those scripts so that they work with the validation framework. (You will notice I have also broken them out into separate script objects with a new naming scheme).

The exclusion group sample has been updated so that:

• When an exclusion subform has not yet reached its minimum number of entries, the subform gets highlighted instead of the individual fields in the subform.
• The call to scGroup.setMinAndMax() now includes an optional validation message parameter.  This is needed because designer does not yet expose a UI for assigning a validation message to a subform.

Note that this sample now introduces the concept of an invalid subform.  If the user has not filled in all the credit card fields, then we mark the subform as invalid -- similar to the case where the subform exclusion group did not reach its minimum number of entries.  The script to do this looks like:

form1.#subform[0].Payment.CC.CardType::validate - (JavaScript, client)
// Valid state is where either all fields are null or all fields are populated.
var vValid = (CardType.isNull  && CardNo.isNull  && Expiry.isNull) ||
             (!CardType.isNull && !CardNo.isNull && !Expiry.isNull);

// populate the subform validation message so that users get a meaningful message
if (this.parent.validationMessage.length == 0)
    this.parent.validationMessage = "All credit card fields must be filled in";

scValidate.setStatus(this.parent, vValid );

The validation button

The samples have also updated the logic for the button to validate the form.  Now when there are no errors, the button caption will be updated to say: "no errors" (and will be made read-only).  This works as long as you do not change the name of the button from : "checkValid".  You might prefer a variation on this where the button is hidden when there are no errors.

Backward compatibility

We would like to make it easier you to control validation message handling in a future version of Reader, but for the here and now, you are likely targeting Reader 7/8/9 and need an immediate solution.  You can take these script objects, copy them into new forms and use them in forms compatible as far back as Reader 7.

Next up: Another error reporting option

In the sample from my previous post I introduced an integer form variable without drawing attention to it.  In this post I will describe form variables in much more detail.

Creating Variables in Designer

In order to create a form variable from Designer, you choose file/form properties/variables.  In this screen shot there are two variables: "foo" and "bar". 

The resulting XFA markup looks like:

<template>
   <subform>
      <variables>
         <text name="foo">123</text>
         <text name="bar">abc</text>
      </variables>
   </subform>
</template>

There are some limitations to Designer support for variables:

• Creates only <text> variables
• Cannot create a variable with an initial value that is empty
• Places variables under the root subform only

Creating Variables in Script

In a previous post I described the code for creating a form variable in JavaScript.

var vNewVar = xfa.form.createNode("text", "vMyVar");
vNewVar = "abc";
ExclusionGroup.variables.nodes.append(vNewVar);

The first parameter to createNode() can be any of the XFA content types.  The set that you would likely find useful in a scripting context are: <boolean>, <decimal>, <float>, <integer>, <text>.  You can create other types such as <date>, but a <date> behaves like a <text> (a string) because there is no date data type in JavaScript.

Typed variables

The .value property on a variable is a variant.  It returns a typed value based on the content type.  It will return string for <text> , it will return number for <integer>, <decimal> and <float> and returns boolean for <boolean>

Typed variables have the advantage that script operations that use them will do the right thing.  e.g. if you define two <text> variables: v1.value="3"; v2.value="5", then the expression "v1.value + v2.value" will return "35".  If they are defined as <integer>, the result of the expression is "8".

Use any Subform

The <variables> element may appear under any subform.  It often makes sense to create the variable in the context where it is used e.g. in the case where it is used within a repeating subform or a fragment.

Hidden from the Designer

When you define script objects that use variables, it can be inconvenient to create the form variables from the Designer UI.  If you do, your script then has an external dependency, and other people modifying the form can muck with your variables.  It really only makes sense to create them from Designer if they truly have global scope.  Of course, defining them in Designer does have the advantage that they show up in object assist.

Form State

Anatole Matveief wrote a nice form state tutorial on Stefan Cameron's blog.  As of Acrobat 9, form variables are stored as part of the preserved form state i.e. after save/close/open, any form variables will be restored.  This can be a very valuable feature, but can also require some caution -- when creating a new variable, you should first check for its existence.  If not, you may end up with multiple copies of the variable.  The existence check looks like:

if (mySubform.variables.nodes.namedItem("vMyVar") == null)
{
    var vNewVariable = xfa.form.createNode("integer", "vMyVar");
    vNewVariable.value = 42;
    mySubform.variables.nodes.append(vNewVariable);
}

Dependency Tracking

Unfortunately, variables do no participate in dependency tracking.  If you define a calculation that uses a variable, the calculation will not re-fire automatically when the value of the variable changes.

What about Fields?

It is really convenient to be able to define variables in the context of a subform, but what about the case where your context is a field?  The good news is that there are two places where we can place content elements within a field: <desc> and <extras>.  For example, to create content under a field you can code:

if (myField.desc.nodes.namedItem("vFortyTwo") == null)
{
    var vNewVariable = xfa.form.createNode("integer", "vFortyTwo");
    vNewVariable.value = 42;
    myField.desc.nodes.append(vNewVariable);
}

The content can then be referenced in script as: "myField.desc.vFortyTwo.value". 

Sample

I have added a sample that has code to demonstrate creating and referencing form variables and content elements under a field <desc> element.  The script lives under the "Create Variables" button.