Archive for August, 2008

Natural Language Mashups with Ubiquity

So you fire up your browser, you type "Book a flight to Chicago next Monday to Thursday, no red-eyes, the cheapest. Then, email my friends the itinerary and add it to my calendar". Your browser responds with:

This is the aim of the Ubiquity project at Mozilla, which aims to parse natural language queries to create on-the-fly mashups. In the words of the Mozilla team, it’s about "connecting the Web with language in an attempt to find new user interfaces that could make it possible for everyone to do common Web tasks more quickly and easily." This is a tremendously exciting idea.

In Web 2.0, the idea of a "mashup" is very static…though we can create services that are themselves compositions of micro-services available over the web, our final service is somewhat static, in the sense that our mashup is specific to the intent of the developer that created the mashup.

Projects like Yahoo! Pipes have tried to make it easier to create mashups more "on the fly", by creating a way of graphically orchestrating disparate web-service calls.  This is a very similar approach to the "enterprise orchestrations" that can be created with LiveCycle Process Management ES which allows us to graphically orchestrate pre-supplied and custom services into enterprise business processes.

However, as much as Yahoo Pipes may make it easier to create a mashup, the resulting service, despite being a composition of several disparate services, is still static, in the sense that it is only fit for its single intended purpose ("Fetch me the New York Times business stories from the RSS feed, and show me Flickr images alongside each story, with links to Forbes.com profiles for any companies or people mentioned in the story.")

What I love about the vision behind Ubiquity, is that it aspires to offer the most simple, easy and effective of user-experiences – a natural language query or imperative – while "behind the glass" have the most complex of dynamic orchestrations from a catalogue of known or discoverable web-services. Much like Tivo, or purchasing music while sat in your sofa with your wireless iPod, this has all the opportunities of seeming like the easiest transaction to offer, truly hiding the end-user from the smarts, the intelligence and the complexity required in silicon and code to make it happen.

Get this project working effectively, put speech to text on the natural language query, and this gets even more exciting…

Check out the project at http://labs.mozilla.com/2008/08/introducing-ubiquity/

Call for Representatives on the Cairngorm Committee

As Alistair just posted, some of the most active Cairngorm contributors within the Adobe Consulting team met for the day on Monday, to discuss our various innovations and thoughts and ideas on how we move the Cairngorm projects towards a Version 3 release.  One of the outcomes of this meeting, was to re-ignite the idea of a Cairngorm committee that draws from Adobe Consulting, customers, partners and community leaders who are actively using Cairngorm to the degree we are, and who can provide counsel in future roadmap, drawing up the final charter for the project, and helping to manage the day to day logistics of running an open-source project.

What we’re looking for is representatives from companies, system integrators, partners, or from individual contractors and consultants who are actively using Cairngorm – on a near daily basis – in their engagements, to join a team of Adobe Consultants.  I’m very excited, as much as anything else, about the opportunity for collaboration this will create between Adobe Consulting and community leaders!

If you think you’d have energy to contribute and value to add to the team, please drop me a personal email to ("Steven").toLowerCase().substr(0,1).concat( ("Webster").toLowerCase(), "@", (("Adobe Systems, Inc.").split(" ", 2))[0]).toLowerCase(), ".com").

If you do manage to parse that above line in a way that a spambot can’t, and drop me an email, then I’d like to hear from you about the number of projects you have deployed using Cairngorm, something about the scale and complexity of these projects, the typical environments (Java, .NET, LiveCycle Data Services, Blaze DS, Data Management Services, etc) your projects are deployed in, the challenges you have faced, the innovations you have made in your own projects around Cairngorm, the number of developers that you have worked with and mentored in using Cairngorm, etc.  I’d also love to hear your thoughts around how you would want to contribute to the project, and the directions you think we can take.

We anticipate a number of different roles for participation, from the committee itself, to commiters, partial committers, etc.

I very much look forward to hearing from you!

(PS.  Alistair and I pair-programmed on the string above.  We also wrote tests, but if I put the tests here then I’d just be putting my email address in the assertion! 

AC@MAX on "Architectural Best Practices for Flex and LiveCycle ES Applications"

Since MAX 2004 in New Orleans, I’ve presented to one extent or another around the theme of architectural best-practices for delivering enterprise applications upon Adobe technologies.  This year is no different, and I’m very much looking forward to presenting with one of my colleagues from our Chicago Office, Tunde Turner, on our collective experiences within Adobe Consulting in delivering Enterprise Rich Internet Applications that leverage Flex and LiveCycle ES.

In previous years, I’d like to think that we illuminated many ideas around software architecture as it would apply to Rich Internet Applications, through presenting and contributing around the Cairngorm Framework for Rich Internet Applications. What I aspire to achieve in Tunde and I’s talk, is to help Rich Internet Application developers take the complexity and scale of their applications to new levels, by understanding the class of recurring problems where Flex, LiveCycle Data Services and LiveCycle ES play strongly, and the patterns and practices that recur in architecting these solutions.

A theme I introduced at MAX last year, during both the Chicago and Barcelona keynotes, was the idea that combining Flex and LiveCycle allowed us to "create innovative experiences on both sides of the glass".  I’ve continued to use that metaphor in helping customers understand the value that Adobe technology can bring, and indeed this is baked into the way Adobe Consulting help customers conceive of innovative solutions, and it’s baked into our go-to-marketing message (you can hear more about that message in the video at the foot of this page linked here ). 

Tunde and I aim to re-introduce this concept of creating innovative solutions "on the glass" and "behind the glass", to show some stunning examples of this metaphor applied, and to use this as the highest-level of abstraction for discussion around microarchitectures for Enterprise Rich Internet Applications.

On the Glass

On the glass, we’ll definitely talk about architectural best-practices for Flex and AIR development; we’ll take a brief tour of Cairngorm, explore some of the emerging ideas around Cairngorm and how a Cairngorm application can logically extend it’s reach towards an enterprise back-end with LiveCycle Data Services and LiveCycle ES.

On the glass, we’ll also look at some little appreciated features of LiveCycle ES – the Workspace and FormGuide APIs that allow us to create Rich Internet Applications for review and approval patterns, and for data capture patterns of user-experience.

Behind the Glass – Data Oriented Architectures and Microarchitecture Pattern Catalogues

Where things get really interesting however, is when we start to reach behind the glass.  A key consideration which I urge development teams to consider, is whether an application is a service oriented architecture or a data oriented archtiecture.  I will explain this consideration in detail – as to me this is a compelling reason to understand the best-practices between Blaze DS and LiveCycle DS that are often, naively, and incorrectly, reduced to "do you require data push".  Tunde and I will then spend some time exploring the Service Oriented Architecture view of LiveCycle ES, and the manner in which we can invoke services within the LiveCycle ES container, and what that actually means.

For Rich Internet Application developers, we’ll present a service-oriented view of LiveCycle ES, and help you to understand the different services that are available that I’m confident you’ll have spent far too much time in Java in the past trying to implement simplified versions of yourself.  We’ll show the rapid-development model offered by LiveCycle ES, and then identify patterns of applications that consistently leverage the same solution components and services of LiveCycle ES.

Microarchitecture Patterns

I’m confident that the idea of "microarchitecture patterns" will be the most significant contribution from this presentation; and that it will offer the same "Eureka" moment that Cairngorm offered many RIA developers.

Think about this – in Cairngorm, we identified a recurring application problem ("a Rich Internet Application sitting upon a service oriented architecture") for which we identified a network/collaboration of design patterns that became the Cairngorm microarchitecture. 

As our applications reach even deeper into an enterprise, and we concern ourselves not just with the architecture, patterns and practices "on the glass", but the architecture, patterns and practices "behind the glass", then there are recurring patterns of application:

  • RIA that results in a document being processed through an organisation and generating some final paper output, eg Applying for a Credit Card and getting your welcome pack and confirmation letter in the post
  • RIA dashboard that tracks an approval process, eg Loan or Mortgage Approval
  • RIA dashboard for real-time high-volume data, eg Trading Platform, Instrumentation Dashboard
  • RIA that captures information that needs to be secured and archived, eg Clinical Trial Management, Filing of Crime Reports
  • RIA application that configures electronic documents that are then pushed through a high-volume printing process, eg Electronic Statements, eBanking

As we consider the different, yet recurring, suites of services that these application types might consume, as we consider the different ways in which we engage through custom RIA development, through Form Guides and Workspace component suites in Flex, leveraging AIR and PDF for offline data capture or as the tool for moving information through an organisation, we are in effect identfying a series of microarchitectures that are larger networks of patterns than Cairngorm, microarchitectures that span both sides of the glass.

In essence – if Cairngorm was one microarchitecture for RIA upon a generic Service Oriented Architecture, we have yet to expose you to our microarchitectures for RIA upon Data Oriented Architectures, and RIA upon service and data-oriented architectures that focus on document-centric, form-centric, workflow-centric applications where people engage in business processes, where the digital and paper worlds collide, and where a significant number of enterprise problems exist.

Tunde and I look forward to broadening your vocabulary of microarchitectures to consider these different classes of Enterprise RIA!

Beyond Microarchitecture Patterns to Solution Accelerators

There’s a natural next step here; there comes a point where if you’re continually applying a particular microarchitecture that spans both sides of the glass, that the recurrence isn’t jus technical recurrence, but recurrence in the business requirements, from customer to customer, from enagement to engagement.  This next, higher, level of abstraction is what we call a solution accelerator – and I’ll be talking about solution accelerators in a separate talk at MAX with another colleague from Ottawa, Danny Saikaly.

So my question before we finish (ahem, start) the slides….are there recurring patterns of Enterprise RIA that you are developing, that span both the client and the server-side, that require consistency of approach from engagement to engagement on both sides of the glass ?

Cairngorm 3 – Adobe Consulting Led but Community Driven

Cairngorm Mountains during a new dawn...

As I hope you’ve seen on Alistair’s blogpost, the team at Adobe Consulting are delighted to have the Cairngorm project promoted to a first-class citizen at opensource.adobe.com.  In this blog post, I’d like to provide a bit more detail of our motivations and intentions as we start thinking about Cairngorm 3.0, and invite your thoughts as to how you would like to see us govern the project.  Adobe Consulting has never been more committed to the Cairngorm project, and continues to deliver innovative solutions upon Cairngorm to customers the world over — the fact that we’ve been open-source since 2004 means that much of the frenzy and pace happened some years ago, and for those who have joined the Flex community long since that time, Cairngorm may have appeared to be in stasis rather than stability.  First and foremost, I hope that what follows alleviates any of these concerns.

Alistair’s announcement covers the detail of us moving to opensource.adobe.com, but in summary, this now means that we have:

  • The source code available in a subversion repository, with the ability to submit patches.
  • A JIRA bugbase through which you can log bugs and submit community-driven feature requests
  • Developer forums through which you can engage with Adobe Consulting and the broader Cairngorm community, whether your questions are around using the framework or contributing and participating in the frameworks future direction

Cairngorm Committee

In one of the seminal books on open-source, "The Cathedral and the Bazaar" by Eric S Raymond, he talks about the need for the "Keeper of the Project Vision", someone that ensures that the project remain true to some core ideals.  Adobe Consutling will continue to play that governance role, but will do so with more formal counsel and participation of a group of enterprise customers, partners and community leaders who are themselves immersed in delivering projects on a daily basis upon Cairngorm.

If you think you could participate here, I’d be very keen to hear from you, hear what you’re doing with Cairngorm, and share your thoughts on how we best collaborate. Many of you have already reached out in this regard, and we’re exciting about working more closely with the community and partners.

Cairngorm Charter

One of the things we’ll look to lock down as we go forward from here, is a simple charter by which we measure suggestions; this has been very implicit and shared understanding amongst the Adobe Consulting team, but we need to document and agree this with the community as we go forward.  But in essence, it’s as important to decide what Cairngorm is, as well as what it isn’t, both now and in the future.

By way of example; we only ever intended and will continue to intend that Cairngorm serve application developers working with Flex, and as a natural consequence, AIR.  We have no desire to convolute the framework, to introduce abstraction, or to discourage the use of idioms or techniques specific to Flex, in order that we might create applications that could be ported to other presentation-tier technologies.  We are in the business of delivering innovative solutions upon Flex and AIR, and Cairngorm supports that as best we possibly can.

We wish for Cairngorm to encourage best-practice leverage of underlying Flex features, and we are keen to ensure that we never add something to Cairngorm that is better suited to the underlying Flex SDK.  With both projects open-source, we can make these choices.  This extends also to FlexBuilder – we are desparately keen to improve tooling for RIA development, and in several cases we have "started" upon initiatives to better support Cairngorm in this regard.  However, we also collaborate deeply with the engineering team, to ensure we are not duplicating future effort.  We will continue to do so.

Something else we’ve cared deeply about – and something that has quite frankly imposed restrictions on some of the things we’d like to do – has been backwards compatibility.   Whether you appreciate this or not, there is a huge community of customers building projects upon Cairngorm, and we feel duty-bound to support that committment and investment by ensuring that whereever possible, new releases to the framework can be easily embraced without breaking existing code-bases.  As we look to support Flex 4, I’m sure we’ll continue to wrestle with some of the challenges this principle can surface — and as we start thinking about Cairngorm 3.0, we’ll look to partcipate with community in wrestling with many of these discussions.

It’s not our intention to make Cairngorm any heavier or more complex than it needs to be, or to introduce feature that is available in other products.  We have no desire to be an open-source alternative to commercial products.

Looking forward, I personally want to ensure that there’s extensibility to Cairngorm, in order that 3rd party extensions or additions can be released without forking of the Cairngorm core code-base.  As extensions gain in popularity or become "de facto", we can then consider rolling them into the main branch.  The team at Universal Mind have released extensions, which unfortunately have to fork the code-base.  Eric Feminalla has extensions in the community for developing with AIR, and the Adobe Consulting team wish to make LiveCycle ES easier and more available to developers with Flex/AIR who wish to embrace LiveCycle, but I believe that rather than bloating Cairngorm for those not leveraging LiveCycle, a cairngorm-livecycle.swc that extends Cairngorm for LiveCycle ES projects would be more appropriate.  A few years ago at MAX, I showed some very early work of using Cairngorm for FlashLite….the mobile ecosystem grows and grows, and I would imagine that future direction of Cairngorm would continue to ensure that architecture and approach be consistent across screens and devices.

What about other RIA Frameworks ?

So often, I see desire for discussion of "Cairngorm versus…." and to be clear, I don’t think this is overly valuable conversation to engage in.  The health of our ecosystem is diversity, and the evolution and proliferation of other frameworks to support similar development, is something we can only encourage.  Blog posts that "<framework> is evil", "<framework feature> is bad", "<framework> sucks" certainly catch a great deal of fleeting attention, but I don’t believe that investing our limited time and energy in these debates is the most effective way of advancing the state of our art.  In any ecosystem, the cross-pollenation and cross-fertilisation of ideas increases the overall fitness of the ecosystem – so I encourage innovation to occur on all levels, and for discussion and debate to focus inwardly rather than outwardly. 

Or in other words, to quote a mantra used often within our own team – "don’t bring me your problems; bring me your ideas for solutions.  Through those, I will grasp the underlying problem".

And so what of Cairngorm 3.0 ?

We’re not announcing Cairngorm 3.0 – but we’re very much thinking about it.  The Adobe Consulting team is coming together internally for a "Cairngorm summit" (do the etymologists amongst you see what we did there ?) in the next few weeks, where we’ll assemble some of the deepest thinkers around Cairngorm within Adobe and collaborate around the different approaches, ideas and innovations we’ve been seeing in our own projects.  As we collate that knowledge, we’ll be opening this up to share with the community, as we start to think about what Cairngorm 3.0 will look like for the innovative solutions we will all be delivering in the future.

Finally – and unrelated to Cairngorm – opensource.adobe.com offers a forum through which Adobe Consulting can spin-out other open-source initiatives around the repeating and recurring approaches we take to making customers successful upon Adobe technology. 

I look forward to continuing to collaborate and share with the community, our partners and our customers through opensource.adobe.com, and in the meantime, very much look forward to your thoughts on Cairngorm going forward.

Innovative Mashups: Meatware + Hardware + Software

There’s a tremendous article on the BBC showcasing an upcoming television program, that encapsulates so much of what fascinates me right now as mashups don’t just focus on bringing together different online data sources, but take real-world information, whether that be people or things, and bring that information into software applications.  What’s even more interesting, is how this in itself creates an "architecture of participation", a suite of data that can be visualised over time, and from which insights can be gleaned that themselves may lead to innovations.

"Britain from Above" will be first broadcast in the UK on Sunday 10th August, at 2100 on BBC One, and features some stunning visualisations of data captured and overlaid on Britain itself.  In this short video clip on BBC iPlayer (I’m not sure if this will be geo-locked to the UK or not) you can see some tremendous examples:

  • Watch the shipping channels through the straits of Dover; satellite imagery overlaid with all of the day’s shipping as a computer visualisation
  • Watch every flight in and out of the UK, flying through stricly controlled air corridors, and observe where and when the most "stacking" of flights occurs waiting to come into land

I think the examples that I find most intriguing however, are the GPS tracking of London taxi-drivers; the drivers leverage the main thoroughfares, but as congestion begins to peak, you can observe the myriad of rat-runs and short-cuts that emerge through the backstreets of London.  Many SatNav companies are now starting to track this data to offer different recommended routes from A to B according to time of day and historic data.  What’s fascinated me for some time however, is how cars themselves become packets of data on real highways, communicating their recent journey segments, weighted by the collective opinion of other cars who have also passed the same routes, so that the network of cars themselves communicate route congestion much like ants communicate as they pass each other in lines, or other redundant networks are able to intelligently record, replay and re-route to avoid congestion.

I first encountered this idea of smart networks a loooong time ago when writing my University dissertation on "The Enabling Technologies of the Trunkl Network" (a dissertation that discussed some technology called ADSL that might become popular  in the last-mile of the copper telephone network amongst other things) amongst a myriad of British Telecom research papers around "Intelligent Networks" and intelligent switching and routing of traffic for video and audio.

The final example in the above clip is of the way "London wakes up" by visualising the patterns of telephone calls that take place in the UK.

Increasingly as I meet with customers around strategies and visions for the future, there’s ever more desire to create architectures that can bring real-world information into online applications, whether that be for improved visualisation to support real-time decision making, or physical information that can be combined seamlessly with rich and interactive applications. 

When we talk about Rich Internet Applications, we consider not only visually-rich or interaction-rich, but the richness of data.  When we think about creating architectures of participation where the wisdom is not just gathered from the crowd, but from the accelerometers, the GPS transceivers, and the myriad of other sensors that attach hardware to meatware to software.  An example I often use that really embodies the idea of hardware, connected to humans, that leverages a software architecture through which a data and visualisation and interaction rich experience can be delivered, is the NikePlus collaboration with Apple and iPods.

In your experiences, are there other applications where "meatware + hardware + software" is a recipe for innovative visualisations, data platforms, or overall digital experiences ?

 

AC@MAX2008 on "Agile Testing of Enterprise RIA with Flex and LiveCycle ES"

Alex Uhlmann (Adobe Consulting, Edinburgh) and Herve Dupriez (Adobe Consulting, London) are teaming up to present on all things related to the unit-testing, test-driven development, test-coverage analysis and optimisation of the kind of "Experiences on both sides of the glass" that we create when we build innovative and complex RIA front-ends in Flex or AIR and sit them upon enterprise architectures implemented in Java and/or LiveCycle ES as a service oriented architecture. 

We’ve been murmuring about a new version of FlexUnit for a while now, but we’re close to rehousing the FlexUnit project as a more active community open-source project; the most major contribution from Adobe Consulting to the project will be a brand new visual test runner implemented in Flex by a number of our user-experience and technical team, that we’ve been using internally on our own projects within Adobe for some time now. 

To the presentation at MAX; starting on the client-side, Alex will be focussing his talk around unit-testing best-practices for Rich Internet Applications.  Alex has been working on some incredibly complex Rich Internet Applications of late, on code-bases that have teams of 20-30 developers in projects measured in years rather than months. As you can only imagine, the complexity of such application demands rigor and discipline around unit-testing, and Alex will be sharing his experience and best-practices.

One such area is having the tooling to help you identify the strength of your test battery; identifying the areas of your application where test-coverage is weak using test coverage tools. Both Alex and Joe Berkowitz had been immersed in parallel efforts creating test coverage tools, recently joining their efforts to contribute to the open-source FlexCover project.  Alex will, I’m sure, be discussing how to leverage FlexCover in your development workflow, including your continuous build workflow, to ensure that you’re testing in all the right places. 

Meanwhile, Herve will be focussing on how we achieve the same level of coverage on the other side of the wire, where ActionScript becomes Java.  For pure-play Java development, the techniques are well understood – LiveCycle ES however offers a rapid development approach to many classes of enteprise application; just because we alleviate the complexity of implementation, the number of things that could possibly break does not change, therefore nor should our diligence around unit and regression testing of our delivered software.

LiveCycle ES allows the encapsulation of business logic into discrete components called DSCs; these are essentially service classes, that are packaged with resource files and component descriptors that allow them to be assembled and debugged graphically in the LiveCycle Workbench ES environment. Just because business logic is being encapsulated in a DSC, doesn’t mean it should become untestable.

LiveCycle Workbench ES allows your custom DSCs to be seamlessly choreographed and orchestrated into complex business processes, leveraging a number of pre-built services alongside other invocation methods out to 3rd party services.  While these orchestrations can be easily exposed to a Flex or AIR application through endpoints that can be invoked through RemoteObject or WebService tags, we should also think about how we create unit and regression test suites for the code that is essentially generated by the orchestration in LiveCycle Designer ES.

As best-practices emerge in how to ensure we are designing-for-test and testing design with LiveCycle ES, Adobe Consulting are keen to share them with the community.

I’ve attended MAX since the day Ely Greenfield stepped up on stage in New Orleans and showed this forthcoming project called Royale (what’s now Flex) to an audience of designers and developers, most of whom hadn’t heard the phrase Rich Internet Application.  That was the same conference where we open-sourced Cairngorm, and it’s amazing to look back over what’s really only a relatively short period of time, to see the kind and complexity of the applications we are building today upon the Flash platform.  Cross-pollenating software engineering best-practices into our enterprise RIA development has never been more important!

I’m confident that in attending this presentation, you’ll have the confidence to attack the most complex of Rich Internet Applications safe in the knowledge that defensive testing strategies can be used across the Adobe platform; and I know that Alex and Herve will do a tremendous job leading this discussion.

Session Details:

Agile Testing of Enterprise RIAs with Flex and LiveCycle ES

Hear how Adobe Consulting delivers complex rich Internet applications built on Flex and LiveCycle ES, while still employing agile engineering methods to assure quality. Learn about testing with FlexUnit; using Code Coverage tools, designing for testing, and improving test coverage; and how to extend these capabilities to unit-test LiveCycle ES applications "behind the glass."
Speakers: Alex Uhlmann, Herve Dupriez
Audience: Architect, Application Developer
Skill: Intermediate
Products: LiveCycle ES, Flex, AIR
When:
Wednesday, November 19, 2:00 pm – 3:00 pm