Archive for January, 2011

Server Side PDF Flattening*

This blog entry is based on an internal document I wrote some time ago.  I’ve updated it for the current release of LiveCycle ES2.5 and cleaned it up a bit so it is more accessible to an external audience.

*Please note that the title contains a purposeful anachronism. Although the term “Flattening” has entered the vernacular, most PDFs are not really transformed into a flat PDF. Except in limited special cases, the Form is re-rendered as a flat PDF.

 

There has been much confusion around the ability of LiveCycle ES/ES2/ES2.5 to create flat PDFs from various types of interactive PDF forms. This has been exacerbated by functions added to LiveCycle ES Assembler which includes the ability to remove XFA (NoXFA) and form (NoForm) capabilities from a PDF. Specifically, there have been many questions, rumors and theories as to how flat PDFs can be created with and without LiveCycle ES Output.

Despite the ability of various LC components to create flat documents in certain cases, the most comprehensive application for creating flat PDFs is LiveCycle ES Output. As you will see there are certain circumstances where Output is not required to create a flat PDF.  However; these are specific circumstances.  The decision to propose a non LC Output solution must be taken carefully with understanding of the limitations that will be imposed being understood by the development team and customer. This will avoid the trap of what worked in a simple proof is not working when the customer proceeds with the actual application.

This post lays out these limitations and provides some guidance as to when Output is required.

Common Definitions

The following terms are used throughout this document. You shouldn’t consider these to be tremendously technically accurate, but they will work for the purposes of this discussion. 

Interactive Form – A form that can be filled in and potentially signed with a definition of the form in XFA and the data stream in XML.

Non-interactive Form – This is an interactive form with fields locked. It still has all of the XFA and XML in it, but the fields are all locked to be read-only.

Flat PDF – A PDF with no XFA stream and no non-signature elements. Basically there are no fields in the document.  It is important to note that a “Flat” PDF may still have some layers, such as the comment layer.  In this case the flatness refers to the lack of data fields.

Static PDF – A PDF which contains an XFA stream and the form layout does not change. Static forms may be interactive (a user can still fill in fields). If a dynamic XDP is rendered with LiveCycle Forms with the Render At Client option set to “No” then the resulting PDF is no longer dynamic – it is now static and behaves like any other static PDF.

Dynamic PDF – Dynamic PDFs allow the layout of the form to be altered either through user interaction or through script. An example is a form that adds subforms based on a user input. If a static XDP is rendered with LiveCycle Forms with the Render At Client option set to “Yes” then the resulting PDF is no longer static – it is now dynamic and behaves like any other dynamic PDF.

Acroform – A non-XFA based PDF form, usually created directly in Adobe Acrobat (as opposed to using LiveCycle Designer).

Artwork or XFAF – A PDF form created with from a flat PDF document using the “Create an Interactive Form with Fixed Pages” option in designer. For the purpose of this document Artwork PDFs operate the same as static PDFs.

Different Methods

There are a variety of ways to create a non-interactive form out of an interactive form using Adobe LiveCycle ES/ES2/ES2.5. The use of these different methods depends on the customer process:

  • Assembler – Assembler can invoke a DDX operation that can have either a noXFA or noForms tag. The result of both of these operations is to turn an interactive PDF into a flat PDF
  • DocConverter – DocConverter includes a feature (Convert to PDF/A) that will convert PDFs to a PDF/A format. PDF/A documents are flat PDFs with additional archival restrictions
  • Output – LiveCycle Output includes a transformPDF feature that, amongst other things, can create flat PDFs.

What works and what doesn’t

The following matrix lists what works (Y) and what does not work (N) when you try to convert interactive PDF documents to flat PDFs.

whatWorks

Notes:

  • [2] With LiveCycle ES, if a PDF (XFA or Acroform based) is merged with data using LiveCycle Forms (or Form Data Integration), the resulting document is not fully rendered. If the resulting PDF is processed with Assembler and the NoXFA or NoForms tags are used then the resulting PDF will have no data. The reason the PDF is not re-rendered is to increase server performance. The solution is to use LiveCycle Output to render the flat PDF directly instead of using a combination of Forms and Assembler.  I understand that this was changed with LiveCycle ES2, so it should not be a problem if you are using a more recent version of the product.
  • Artwork based PDFs work in the same way as other static PDFs.
  • Some developers have attempted to use the toPS operation to convert an interactive PDF into a PostScript file with the intention of converting the PS file back to a flat PDF using LiveCycle Generator. This will not work as toPS only works with flat PDFs.

Explanation

Dynamic PDFs require regeneration of the layout because they contain dynamic XFA. Only LiveCycle Output ES and LiveCycle Forms ES (Forms can only generate interactive PDFs so is not applicable in this case) include the XFAForm.exe application.  XFAForm.exe is called behind the scenes through the use of the LiveCycle services and is used in the layout creation. So any scenario that requires rendering XFA, such as merging data or displaying dynamic XFA, will ultimately require Output to do the rendering. Static XFA is already rendered, so unless there is new data being merged, Output is not required.

AcroForms can be rendered by the Gibson libraries (within the limitations of the Assembler and PDF Generator implementations), and therefore do not require Output.

The key message is that, in general, LiveCycle Output ES is the product for documents of record (flat PDF) from XFA and forms.

Customer Experience Management for the Pragmatist

As you may be aware, the term Customer Experience Management (CEM) has become one of the latest buzzwords in the enterprise software industry.  Many large companies have formed CEM groups and a few have even created “C” level positions that are dedicated to improving the customer experience.  Adobe has embraced CEM in a big way (you can see the Experience Delivers blog  or Steven Webster’s blog for just a couple of examples), so I felt I needed to understand more about it.

In my search for answers I came across tons of white papers, research notes, videos and other content on CEM and its related topics (such as Rich Internet Applications (RIA), User Experience (UX), Experience Oriented Architecture (XOA), etc.).  There were web pages that call out “bad” design and praise “good” design.  Some sites had some best practices (although these were mostly based on specific examples).  I found a plethora of other phrases that seemed to float around the subject – customer oriented design, put the customer first, contextual design, etc.

I also spoke with some of Adobe’s CEM proponents.  Unfortunately these conversations tended to bend towards rather philosophical discussions.  One time someone actually said “the Medium is the Message”.    At this point my the part of my brain that performs cognitive reasoning ran down the hallway and hid in a quiet corner.  I can’t help it; McLuhan has that effect on me.

Nothing really hit home with me.  I might be able to recognize a CEM application if I saw one, but I still couldn’t describe to you why it was CEM.

It started to feel like when I played baseball as a kid.  Both my little league coach and my dad kept telling me to “keep your eye on the ball”.  Repeating the phrase over and over made it essentially meaningless.  One day someone (probably a frustrated first baseman) said something like – “look, just watch the ball as it leaves my hand.  Keep watching it as it goes through the air and then put your glove in front of it.”  All of a sudden it made sense.  I still suck at baseball, but at least I don’t get beaned as often.

One day, while searching for cheap vacation flights, things started to congeal.  Navigating through several sites that essentially did the same thing, I started to understand what was meant by customer experience.  More importantly I started to get why it was so important.

When you get right down to it, what does CEM mean in a practical sense?  Its simple really – build software that helps the user accomplish their goal.  At the very least it should not actively make the user mad.

Like so many simple statements, that has some rather complex consequences.  For one thing, as a designer/developer you really need to understand what the user is trying to do as well as how they feel confortable doing it.  This includes not only understanding the direct path to the goal, but how to assist them.  Also allow people to get back when they go down a different path.

If I look at my own recent experience with online ticket retailers, there were precious few who designed their applications in this way.  For the most part the sites were designed for people to book flights; but I, for one, wasn’t using it that way.  I was shopping for flights – changing dates/times, departure location (I’m Canadian, so it sometimes pays to drive to a US airport), airlines, etc.  One site made me re-enter all of the information each time I changed one thing.  Others would only give me one flight choice and would not suggest others.  Some would require me to know things that are industry specific (such as airport codes).  The sites I really appreciated, and therefor spent more time on, made my life easier.

Here are some general things that I observed:

  • Cut down on the clicking
    • If you’re going to require me to read something then just show it to me.  Don’t make me click on a button that brings it up.  If its that important than gimme it.  This really goes to presenting the right information at the right time.
  • Get on with it! (or where’s the “f@*ck off” button)
    • Of course the flip side is presenting too much information.  If I don’t need it right now, get rid of it.
  • Just do what I tell you to do
    • Software is a tool.  As developers we often think that the software itself is the most important thing.  We must keep in mind that the user is trying to get something done (like shop for a flight) and the software is just a means to an end.
  • I don’t care about your process
    • Remember that the user is not concerned with what is going on behind the scenes.  Your product codes, industry regulations, inter-departmental routing, etc. does not matter to the user.  While you do want to tell the user what is going on, they don’t need to know what’s going on in the sausage factory.
  • Customize when necessary, but but not necessarily customize
    • There is a trend to allow the user to customize every aspect an application.  In some circumstances this is a really nice thing (iGoogle is a prime example), but it has a dark side.  I’ve used too many applications that require lengthy configuration to provide a personalized experience.  I know they theory is that by customizing the interface the user takes ownership, but Its not always necessary or even wanted.  Sometimes I just want to do a quick search and get on with my life (google.com is a prime example).

As I said, it all boils down to designing software that helps the user do something.