Recently I have been seeing reports of stalled / failed processes, both short-lived and long-lived. It turns out that in some cases, the process developers were updating processes on their live systems without taking due precautions. The following post is to explain what is actually happening under the hood, and why you can’t simply “hot change” a process.
A simple process
Imagine you have a process called MyNotify with two steps in it: a first task (not start point) AssignUser, and a second task SendEmail. Pretty straight forward.
You have deployed this process, and process instances are merrily being kicked off. Great. Now you realize that you don’t want the AssignUser task; rather, you want a Workspace Start Point. Remove the AssignUser task, configure the Workspace Start Point to open the same form as you had in the now deletd AssignUser task. Save and deploy.
You may now start getting calls from your users saying their form submission never resulted in an email. Upon investigating, you will likely find errors saying that “Job ID could not be found for Job [...]“.
Oh no. What happened?
Tasks not found
Let’s wind back to where there were two steps. When a process instance is kicked off, what happens is that an entry in the database is made saying that process MyNotify has been kicked off by user “Alice” (for example), and it has been assigned to Alice, in the AssignUser task.
When Alice submits the form, LiveCycle goes to its database and checks the process – Alice just submitted the AssignUser step, so it checks MyNotify for an AssignUser step, and figures out what to do next from that.
If you have deleted the AssignUser task from MyNotify, LiveCycle will be unable to determine what to do next – and stall the process instance. Future instances should work fine (they are before the change point), anything that was at the SendEmail point in the process will be fine as well (they are after the change point). It is specifically process instances that are in a state that could be affected by the change that risk failure, and that will happen without possibility of roll-back or recuperation (bar performing some incantations in the database to recuperate the data, and then constructing some half-processes to mimick pre-change parts of the changed process, in the hope of pushing the data onwards — a lot of avoidable hassle, itself prone to failure).
It is understandable that you may think that if the process were modified, already-running processes would continue with the older version. This is not so, or at least, not in this scenario.
When kicking off a new process instance, LiveCycle does not automatically make a copy of the process to be followed should a new edit of the process come into existence. Maintaining a copy of all the configurations of a process for each instance created would cause extremely severe performance issues on the server, so a more de-coupled implementation serves best. The work of ensuring that a new copy of the process configuration is created is left up to the process developer.
Once a process is deployed, there are only two “safe” ways of updating the process.
The first is to create a new Application version, and when all the new changes are made, deploy the new version.
- All old instances will continue their course in the old version.
- Workspace processes will kick off in the new version
- Calls to the old version of the process will still call the old version
Once you have ascertained for sure that NO more processes from the old version are running, and that no applications rely on the old version, you can safely undeploy it from runtime, and delete it if you so wish.
When will references point to the new version automatically?
Having done this, all references to sub-processes inside the same application will be updated, as will all resources referenced relatively inside the application. For example, MyApplication/1.0/ProcessA can reference /FormA (in Assign User tasks for example) which means “the current application’s version’s FormA”. A sub-process on the process-workflow convas will be referenced relatively if the sub-process is contained inside the same application.
When do you need to update references manually?
Take special heed however — processes and resources that do not reside in the same Application do not see their references updated — they are absolutely referenced. If MyApplication/1.0/ProcessA is upgraded, it will not change the references to, for example, OtherApplication/1.0/OtherProcess. Similarly, if we upgrade to OtherApplication/2.0/OtherProcess, then MyApplication/1.0/ProcessA will still point to OtherApplication/1.0/OtherProcess.
Finally, note that if you create a new version of an application, all resources and processes are copied over with it with new version numbers too, and any textual references to other resources remain the same. So if MyNotify/1.0/ProcessA has a variable referencing MyNotify/1.0/FormA, then MyNotify/2.0/ProcessA will also reference MyNotify/1.0/FormA ; the same goes for resources uch as XDPs, XSLTs, and any other resources stored in an Application. The reason for this is that these references are stored as “strings” (pieces of text) and Workbench cannot, upon creating a new version, differentiate what snippets of text all through the Application are versions to update, what versions should not be updated, and what strings do not actually contain versions at all.
Down time (clean “brute force” update)
The other way is to plan downtime.
- Prevent people from kicking off new processes
- Let all processes complete
- … and only then deploy the new edit of the process
You could choose the second method to avoid having to update references to resources, but it would defeat the purpose of having this versioning at all and does, unavoidably, incur the overhead of a downtime.