The Current State of (Touch) Events

One of the main goals of the jQuery Mobile project is to allow developers to extend the reach of their application content to a wide variety of browsers on different devices. If you take a look at some of the web-enabled devices that are currently out on the market, you will see that there are many different means being employed to allow users to navigate and click/activate elements within a web page.

Older or low-end devices, with no touch screen support, usually have hardware buttons, scroll-wheels, nubs/joysticks, or track-balls. Devices that use buttons and scroll-wheels usually scroll the page, highlighting actionable (clickable) items along the way. When the user activates the highlighted element on screen, a click event is usually dispatched to trigger any actions associated with that element. Devices that use nubs/joysticks or track-balls typically display a cursor on screen, and usually dispatch mouse and click events just like the desktop. The main point to note here is that the browsers on these devices are using the standard desktop mouse events to trigger actions on a web page.

Newer or high-end devices, now rely on touch screens as the main means for scrolling and manipulating items within the display. Although there are many options for browsing the web on these devices, a growing number of  them are deploying WebKit based browsers as the default.

One of the common misconceptions I hear quite frequently is the assumption that because all these browsers are all based on WebKit that they all share the same features and work identically. The reallity is that WebKit is just a rendering engine with a set of APIs that allow developers to write browsers on top of it to communicate and drive the rendering of the page. It doesn’t know how to load a file, it doesn’t know what hardware/platform it is running on, what graphics library is being used to render objects to the screen, or even how to deal with OS level events.  All of these things are what browsers, built on top of WebKit, need to provide, and this is what is going to make things interesting and challenging for the next few years. All of these WebKit based browsers are either written entirely by the device vendor, or supplied with the OS, but modified by vendors to work better with their hardware and/or add/remove browser and Web Kit features.

All of these factors create a mobile environment where there are lots of WebKit based browsers, but the features they support, performance, and user experience all vary quite a bit.

When Safari for mobile hit the scene, via iOS, it introduced a set of new touch events:

  • touchstart
  • touchmove
  • touchend
  • touchcancel

These are the DOM-level events that Safari mobile dispatches in real-time as the user places one or more fingers (touches) on the screen and drags them around. The big problem is that most of the pages on the web assume the use of mouse and click events. To keep most web pages functional, mobile Safari dispatches synthesized mouse events after the user lifts his finger so the web page receives a series of mouse events in the following order:

  • mouseover
  • mousemove
  • mousedown
  • mouseup
  • click

At this point you may be asking “why didn’t the Safari folks just use mouse events instead of creating a whole new set of events?” I think the answer has to do with the fact that the iOS devices support multi-touch. On traditional computing platforms there was always a notion of a single mouse with a main (left) button and maybe center and right buttons. Although you could click and hold down these buttons at different times to generate multiple overlapping mousedown and mouseup events, they were still tied to a single source for the move/positioning information.  Also, folks have become accustomed to the fact that these buttons do specific actions. For example right mouse buttons are typically associated with bringing up a context menu, etc. With the new multi-touch events, not only can you have more than 3 touches, each touch generates its own set of touchstart, touchmove, and touchend events, and in some cases touchmoves could be coalesced into single events if more than one touch shares the same target. It suffices to say that the newer touch events are fundamentally different in behavior and perhaps the Safari folks did not want to break or modify the well established mouse usage and behavioral model.

There are a few interesting things to note about touch events on iOS:

  • Only one event for each mouse event type is dispatched.
  • Mouse events are dispatched approximately 300+ milliseconds after the user lifts his finger.
  • Mouse events are not dispatched if the touch results in the screen scrolling. Scroll events are also not dispatched until after the user lifts their finger.
  • Mouse events are not dispatched if the user initially touches the screen with more than one finger.
  • Touch events are not dispatched to textfields and textareas. Only mouse events are dispatched.

Ok, so getting back to the larger picture, vendors with touch-based devices and WebKit-based browsers have decided to adopt Safari’s touch events. The problem is now each vendor has to implement the event code to drive the touch events. It was explained to me by a device vendor that every hardware device and OS has its own unique implementation and API for dispatching events and that this leads to some interesting differences in browser behavior and event implementations. After playing with several iOS, Android and BlackBerry devices, I have seen first hand that this is indeed true. Some examples off the top of my head include:

  • BlackBerry dispatches interleaved touch and mouse events real-time while Android and iOS dispatch single mouse events after the user lifts their finger.
  • Some devices dispatch scroll events in a somewhat real-time manor, while others only dispatch a single event after the user lifts their finger.
  • Android devices require preventDefault on touchstart to prevent screen scrolling, while other devices require a preventDefault on touchmove, but this causes form elements to break because you can no longer click on them.
  • iOS dispatches a touchend event when the screen scrolls, but some platforms just stop dispatching touch events while the screen scrolls.

Some of these differences are bugs, or temporary problems due to current implementation, but the fact remains that the devices with these problems may exist and be used for a long time since vendors decide if and when these devices can be updated with fixes. Hopefully things will get better as standards emerge.

Another complicating factor is that some devices have both a touch-screen and a nub/joystick/track-ball. For jQuery Mobile, we need to support both touch and mouse events within all our components. We can’t just rely on mouse events because they don’t provide the real-time feedback/response that is necessary to make things feel snappy when the user is touching the screen. But supporting both is a big headache because it complicates event handling. For example, we need to set up a component to listen for both touch and mouse events, but then we need to disable mouse event handlers if touch events are used so that handlers/actions are only triggered once. We then need to re-enable the mouse handlers when the touch events are all done, but sometimes “done” is hard to figure out due to the fact that sometimes touch events just stop coming because the screen just scrolled.

Over the next few weeks we’ll be blogging about some of the ways we are dealing with these challenges while trying to reduce the event code complexity for jQuery Mobile components and implementing features like faux momentum scrolling. Stay tuned!

8 Responses to The Current State of (Touch) Events

  1. Alan says:

    Nice article! I’ve been struggling with dealing with these issues. Looking forward what jQuery Mobile brings to the table.

  2. Pingback: Recent news, steady progress « jd/adobe

  3. Arnout says:

    I have been monitoring the process of the jQuery Mobile project for a couple of months now. It’s great to see that it’s all coming together now, the adoption of HTML5 data-attributes is a smart way to create a rich experience without having to interact with allot JavaScript. It will be a great to create Native applications using phone gap and jQuery mobile.

    But I do wonder if deploying on a mobile website would be a wise thing to do considering the cache limits of mobile browsers. You will be including jQuery ( which still contains cross browser “patches” for IE, FireFox and Opera ) and add another layer on top of that to support the variety of Mobile browsers. Usually you create a mobile view to make a lighter version of your site but it seems this adds quite allot of weight to your site. Having that said, the project is still far from complete but I’m looking forward seeing a 1.0 release :).

    ps: With all the buzz about jQuery can we expect discontinuation of the Spry Framework (jQuery templates, jQuery Grid and your jQuery dataset ;))?

    • kblas says:

      @Arnout

      Hi Arnout,

      The jQuery core folks have already decided that there will be a single version of jquery.js for all platforms. Aside from the fact that it would be very difficult to manage multiple versions of core targeted at specific platforms/browsers, the fact is that some of the same “ cross-browser patches” are still necessary in the mobile space because the browsers on some of the devices are ports of their desktop brethren.

      Within the jQuery Mobile project itself, we’ve been using feature detection to determine if specific enhancements can be made/used, or if things should degrade to default browser handling and behaviors, so we’ve managed to avoid browser specific patching so far.

      The goal of the project is to extend the reach of your web page/content to as many devices as possible without having to create different pages for each device/platform you are targeting.

      That said, I have seen some folks creating libraries that mimic the basics of what they need from the jQuery API to try and reduce the size of things. These libraries are intended for use within specific platforms/browsers like Phone Gap’s web view, where they know what is/isn’t supported. So there are options for folks trying to reduce their download/cache size.

      Regarding Spry … which you already know is near and dear to my heart 🙂 … we’ve heard from our customers and seen the usage data, jQuery is the library of choice, so we are going to move ahead in the direction of supporting it more.

  4. Camilo Lopez says:

    nice one, can’t wait for CS6 with jQuery, jQuery Mobile integration.

  5. Alex says:

    Really excited to see if you guys can truly nail the faux momentum scrolling. Thanks for all the fantastic work you’ve been doing!

  6. simplynonna says:

    The hardest thing to do is simplify. Maybe we need to think like 2 year olds in order to make it heuristic?I liken it to the old days when chess machines knew all the winning moves but had to “dumb down” in order to not discourage the human from wanting to play an “unmatched” game. Thank u for this thought provoking piece!

  7. Have you looked at Sencha Touch? Much better IMHO.