Adidas Megalizer – Behind the scenes

In the following article, I am going to outline a project that we have been working on for Adidas to help them promote the release of their sneaker collection MEGA. We’ve basically created a system that would allow dancers to create their own beat or music with their shoes. Some sort of modern tap dancing :)

Aside from this film, we’ve also build a real interactive installation in several Shops in France to allow real people to test and play with the Megalizer.

Requirements

We had to meet the following requirements :

  • the system needs to be wireless, worn by the dancer and should not embarrass him in any way
  • the system needs to be placed on each shoe (no cable running on the dancer’s leg, or belt case)
  • the system should remain efficient at a 6-7 meter range from the speakers
  • a very low latency is critical (max 30ms)
  • the whole system should handle up to 6 shoes simultaneously
  • no fake: the sound should be genuinely generated by the dancers’ steps

Hardware and assembly

The concept is rather straightforward: I needed two force sensors for each shoe (one for the heel and one for the toe), and a wireless transmitter per shoe to capture the pressure applied on each of the sensors.

I used the Force Sensitive Resistor from Interlink [http://www.sparkfun.com/products/9376]. They have the right square size, are super-thin, and accurate enough to sense the pressure of a dancer’s foot on the ground.

As for the transmitter, I just had a size constraint: I wanted something really small, capable of handling two analogical inputs: one for the heel and one for the toe.

Only thin components such as the sensors can be placed under the internal sole of the shoe.

The first transmitter prototype I built was using a BlueTooth chip (class 3). Although the result was satisfactory at half-a-meter distance from the receiver, the latency increased with the distance. At 7 meters, the latency went as much as 1/8th second.

Then I gave XBee technology a try: these wireless modules have very narrow settings, a low power consumption, and I was confident I could reach a minimum latency at a decent distance.

Once I had solved the range / latency issue, one of the major challenges was to make the XBee module so small as to be wearable on a shoe. As we didn’t have the budget to industrialize the process, all of the assembly was made by hand ;)

Below the two final prototypes before and after building optimization :

On each shoe, the module included:

  • A 6-pin dock to connect the sensors and charge the battery
  • A status LED
  • A switch
  • A 110 mah liPo battery
  • A voltage regulator
  • A XBee chip
  • Some resistors

Finally, a receiver is connected to the computer to gather the signals from all the XBee chips.

The XBee chip (the emitter) can be programmed so that it emits on its own at a certain frequency both analogical inputs to another XBee chip (the receiver).

The receiver is a XBee chip plugged on a USB dongle. At the beginning, I had a single Xbee chip to handle the signal of the 6 emitters. I became quickly aware of  performance and latency issues, due to colliding packets. To solve this, I put three XBee chips as receivers. Each chip was in charge of handling two emitters and the collision/latency issue was gone.

The software

As for the software, two separate programs had to be developed:

  • a server processing the receivers’ inputs (Processing)
  • an AIR application that interprets the signals, chooses and play the sounds.

The communication between the processing server and the air application is achieved by a socket server hosted by the processing server.

The signal analysis was not an easy task. My intention was to assess when the heel or the toe was hit. The major difficulty is that when the dancer dances or even stands, there can be significant shifts of balance from one sensor to another, without an actual impact having really happened.

I created a rather simple algorithm, which you’ll find the basics below:

  • evaluate the average pressure applied on the sensor during the last second
  • evaluate the slope between the current average pressure and the average pressure applied ‘n’ milliseconds ago
  • if the slope matches a specific percentage, assume a positive impact
  • lock the sensor during ‘n’ milliseconds to avoid any duplicates

The application offered the following options:

  • loop a sound after one single impact
  • the ability to stop the sound of sensor X when sensor Y was triggered
  • mute the whole shoe when two sensors were hit simultaneously (for a short duration).

To design the frontend of the AIR application, I used the Minimal Components of Keith Peters.

Photos

Conclusion

I did face loads of technical and hardware problems during the production and tests. Generating live music requires very low latency, and creating a shoe-wearable wireless system was a real challenge.

The dancers that operated on the video clip did train a lot to succeed in synchronizing to play the actual song.

Aside from the technical standpoint, it has been for me a great and fulfilling experience to discover the dance environment.

Credits

Conception : SID LEE Paris

Film : YAK Films

Dancers &  Singer : Lamine, Laurent & Larry (Twins), Mounir, La Fouine.

Thanks to Keith Petters Minimal Components [http://www.minimalcomps.com/]

Thanks to my brothers Fabien & Fabrice for their help.

If you have any questions on programming XBees, on the processing server or anything else, share them with me in the comments: I am no electronics expert, but I did learn a few things on that project :)

Links

Sparkfun shopping list  : http://www.sparkfun.com/wish_lists/13870

Making-of full res photos pack : http://sidlee.didierbrun.com/adidas/megalizer_photos.zip

Emitter electronic scheme PDF  : http://sidlee.didierbrun.com/adidas/megalizer_electronic_scheme.pdf

Processing server sources : http://sidlee.didierbrun.com/adidas/megalizer_processing_sources.zip

 

Conqu – one of the coolest Android apps outhere

Fellow evangelist, Terry Ryan, showed me yesterday a tablet app for Android called Conqu. After I installed and played with it for about 5 minutes, I was like: “Wow! Mate, this is the coolest app for Android tablets ever!”.

The application is helping you to “Get the things done” – managing tasks. Here is a video and screenshots:

And by the way, this application is built using Flex 4.5 (“Hero”) and uses Adobe AIR for Android as a runtime. Now this is something pretty cool. Well done guys!!!

Because they use the Flash Plaform, it isn’t that hard to target other Flash supported devices such as: PlayBook, desktops, or iOS. In fact if you take a look at their roadmap, this is exactly what they are up to.

I really like the fact that they work on a feature that allows synchronization between devices over the cloud. This will facilitate team work and make this app an interesting and appealing choice for small/medium business.

Getting the App

You can install the application from here (for now there is only tablet version) and here you can check the roadmap and features.

Congratulations to the people behind this project. I’m looking forward to test their desktop/PlayBook/iPad versions and hope to meet them in person. Really curious to talk  about their experience building this app.

Asynchronous bitmap decoding in the Adobe Flash runtimes

I just wanted to show you one little improvement we added in Adobe AIR 2.6 (also coming in a future release of the Adobe Flash Player) to improve responsiveness of your applications. For many years when loading big images, your UI could be impacted during the image decoding. As the decoding could be intensive and done on the UI thread, it would generally lock your application and impact the overall responsiveness and make your animations not smooth. We now decode images in a separate thread.

Here is how you would specify this in ActionScript 3 :

// create a LoaderContext
var loaderContext:LoaderContext = new LoaderContext();
// specify async decoding
loaderContext.imageDecodingPolicy = ImageDecodingPolicy.ON_LOAD;
// create a Loader
var loader:Loader = new Loader();
// inform the Loader
loader.load( new URLRequest("http://dl.dropbox.com/u/7009356/IMG_1398.jpg"), loaderContext );

Note that this feature also work with the Loader.loadBytes() API. Find below a little demo of the value of this new feature that we are also bringing to the Flash Player. I am playing a little animation and loading a big image in the background. Note how the animation is locked for a few ms when not using the async decoding :

While we are working more generally on the concurrency topic, we are also working at the same time on other little improvements like this, to keep improving the responsiveness of your application with very little effort.

As always, thanks Chris for the beautiful asset ;)

First steps with 3D in Flash

Building 3D games and apps in general might look like a big step for developers, who did only 2D programming before. The aim of my 3D tutorial series is to show you, that it’s pretty straightforward to start thinking in 3 dimensions.


(Try CubeRotator app)

As a 3D programmer you usually combine following things together:

  • View/Scene
  • – e.g. 800×600 rectangle

  • Camera
  • 3D objects
  • – primitives (Cube, Plane, Sphere)
    – models (imported from file via Collada, 3DS, etc.)
    – 3D objects are placed in a 3D container

  • Materials
  • – color fill
    – textures/bitmaps
    – environment materials and other advanced or intelligent materials

  • Lights
  • – Directional
    – Point light
    – Omni light

  • Shadows
  • Physics
  • Animations
  • Shaders
  • – little programs that can change visual look of models by changing geometry (vertex shader) or pixels on a material (fragment shader), shaders are very widely used in 3D programming and you can use Adobe Pixel Bender 3D to create them

All currently available Flash 3D engines are slightly different in terms of API, but learning one over the another doesn’t take long – differences are mainly in the features and speed of development – some provide less features, but it’s easier to work with them, others are more powerful, but take more time do something. All in all – pick the one that suites you most:
Away3D
Alternativa 3D
Flare3D
Sophie3D
Yoghurt3D
Minko

(some of them have Molehill versions already available)

Let’s create a simple scene with a red box, watching it from top – visible as a square.

Alternativa 3D 7.6 code: (run example | source)

public function Alternativa1()
{
	camera = new Camera3D();
 	camera.view = new View(stage.stageWidth, stage.stageHeight);
	camera.z = -500; // zoom out
	addChild(camera.view);
 
	container = new Object3DContainer();
	container.addChild(camera);
 
	cube = new Box();
	cube.setMaterialToAllFaces(new FillMaterial(0xFF0000));
	container.addChild(cube);
 
	stage.addEventListener(Event.ENTER_FRAME, onEnterFrame);
}
 
protected function onEnterFrame(event:Event):void
{
	camera.render();
}

Away3D 4 Broomstick (Molehill): (run example | source)

public function Away1()
{
	view = new View3D();
	view.camera.z = -500; // zoom out
	addChild(view); // add to the sprite
 
	container = new ObjectContainer3D();
	view.scene.addChild(container); // add to scene
 
	cube = new Cube()
	cube.material = new ColorMaterial(0xFF0000)
	container.addChild(cube); // add to container
 
	stage.addEventListener(Event.ENTER_FRAME, onEnterFrame);
}
 
protected function onEnterFrame(event:Event):void
{
	view.render();
}

Flare3D code: (run example | source)

public function Flare1()
{
	scene = new Scene3D(this);
	scene.camera.setPosition( 0, 0, -50 ) // zoom out
 
	cube = new Cube();
	cube.setMaterial(new FlatColorMaterial("red",0xFF0000));
	scene.addChild(cube);
 
	// see that there is no need for calling render() in Flare3D
}

As you can see, it’s basically same approach, just with little differences. Of course more differences come with more code written, just pick one that is better for you.

Try adjusting camera position:

Open CubeRotator (simple app, that helps you understand how position and rotation work)

Away3D code:

view = new View3D();
view.camera.z = -500;
view.camera.y = -350;
view.camera.x = -350;
view.camera.rotationX = 45;
view.camera.rotationY = 0;
view.camera.rotationZ = -45;
addChild(view); // add to the sprite

*Note that all frameworks are slightly different with +/-
*For instance in Flare3D camera.rotationX will be -45, not 45. In Alternativa you will be setting a radian value.

We will do more stuff in the next tutorial. Loading models, rotations, camera hovering, lights, materials, and so on. I will keep these tutorials short, but focused on particular issues.

Examples and Sources

Alternativa1 | source
Away1 | source
Flare1 | source

CubeRotator | source

JavaScript Voronoi port

I have always been amazed by some of the digital art work that Mario Klingemann (aka @Quasimondo has created using with Voronois. After doing some searching, I found an early ActionScript 1 Voronoi implementation that Mario did, and I ported it to JavaScript.

I wanted to share the results:

 

Here is a graphic I created by playing around with the code:

 

The code is pretty much a straight up port, with some minor optimizations for JavaScript. I also replaced the rendering from the Flash display list API, to EaselJS. All of the credit really goes to Mario who wrote the original code.

You can view all of the code on my GitHub repository, and grab the original code from Mario’s website.

Lucid Design Group Uses the Flash Platform to Help Save the World One Dashboard at a Time

With global warming on the rise, Lucid Design Group, a privately held cleantech software company, wanted to create something that would educate and inspire people to change their daily habits to help reduce consumption in their homes and offices (like turning off lights and unplugging appliances). Lucid developed Building Dashboard, a data visualization and communication application that monitors the use of electricity, water, natural gas and heating, and encourages social networking around the topic of resource conservation. It’s available via the Web, kiosks and mobile devices using the Adobe Flash Platform and Adobe Creative Suite Design Premium. With Building Dashboard (a MAX 2010 award finalist) in place, Fortune 500 companies, universities and residential customers saw consistent energy reductions up to 20 percent.

Coined “the first social network for buildings,” Building Dashboard employed the Flash Platform to help develop and deliver widgets, apps, maps and flow lists to encourage users of all technical backgrounds to save resources and understand their resource consumption levels. Currently, nearly 100 U.S. colleges and universities are using Building Dashboard to support key sustainability initiatives to reduce carbon dioxide emissions across campus.

To streamline the developmental process behind Building Dashboard, Lucid used a variety of Adobe products including Flash Catalyst, Flash Player, Flex, Flash Professional, Photoshop and Flash Builder. The advantage of working with various Adobe products is seamless integration. For example, Adobe Flash Builder helped improve the Flex development process while Adobe Flash Catalyst and Adobe Flash Builder expedited design/developer workflow, ultimately reducing design and development time by one third. By adding Adobe Flash Player to the mix, Lucid developers could run their product across different platforms, devices and operating systems, while cutting testing time in half.

To learn more about how Lucid worked with Adobe technologies and to see how New York-based Hamilton College uses Building Dashboard, read their story here.

Planet of the AOPs at 360|Flex

This year at 360|Flex in Denver I’ll be co-presenting a session with Mike Labriola called “Planet of the AOPs“. What is the Planet of the AOPs? Let me give you a little hint…

public class Foo {
  private function getPrivateBar():String {
    return "private bar";
  }
}
var foo:Foo = new Foo();
foo['getPrivateBar']();

Yup! It’s gonna be fun! The session will be on Tuesday April 12th at 2:50pm. Hope to see you there!

Also, Mike and I are doing a 360|Flex speaker chat today at 11am (MDT).

Flash game Machinarium currently #3 in Mac App Store

Amanita Design from Czech Republic is designing adventure Flash games since ages. You might remember some of their titles like Samorost, Samorost 2, Questionaut and so on.

What’s even better is that you can now get their latest game Machinarium in Mac App Store for $9.99 by the end of the March.

It’s currently number 3 under Top Paid Games – right after Angry Birds and Angry Birds Rio.

Personally I am looking forward to more Flash games available in Mac App Store, and imagine “Molehill” ,)

Here are some tips and tricks for uploading your game to the Mac App Store by Gary Rosenzweig, who has his Gold Strike in Mac App Store since day 1.

Flex on the iPad

Here is a video showing a sample Flex application running on the iPad. This Mobile Trader application shows chart manipulation and drill-downs using touch events, real time market data updates (in lists and charts), as well as video-conferencing (with your financial advisor) and screen synchronization between clients (“simulations” and “what-ifs” collaboration use cases).

In addition to the iPad 2, I also show the same Flex application running on an iPod Touch and on an Android tablet (Samsung Galaxy Tab).

This application was built with the current engineering builds of Flex and AIR with iOS support.

Adobe Pass Makes TV Everywhere a Reality

Today we are excited to announce the Adobe Pass authentication service, which enables pay TV subscribers to easily access premium content across the web from virtually any Internet-connected device.  
 
“TV Everywhere” is the Industry term that refers to Pay TV subscribers being able to access the same content they subscribe to through their traditional TV provider, online allowing customers to access premium TV content in web browsers, smartphones, tablets or IPTV’s.  The biggest problem to date with TV Everywhere is having a system that is transparent and easy to use for customers, conforms to all of the different parties business rules, and mostly importantly – is secure.  Adobe Pass solves these problems for all parties- consumers, programmers and pay TV providers alike.
 
Adobe has worked hard over the past year to integrate Adobe Pass with major Pay TV providers and content programmers.  By providing a secure, hosted solution, Adobe ensures that both the providers and the programmers have minimal integration work to perform as they pull together their many different online viewing options.  
 
Adobe Pass  leverages the Flash Platform for a seamless high-quality experience along with Adobe Flash Access® for enhanced security.  Adobe Pass also utilizes HTML5 for devices where Adobe Flash technology is not yet available.  
 
Adobe Pass is now live on several sites. Any Comcast, Verizon, Dish or Cox customer can try out the Turner TV everywhere sites, just click on the orange key to unlock more content by entering your pay TV username and password (typically the same information you would use to view your bill) at: www.tnt.tv/tveverywhere. What’s more, as you move to another site like www.tbs.com/tveverywhere, the premium contents come alive without a 2nd login. Verizon FiOS customers can also check out www.mtv.com/tve and again, the authentication will persist.

Adobe Pass is a great opportunity for Adobe to promote the Flash Platform across the entire media and entertainment industry, and to generate consistent, recurring revenue by solving a key business problem.  Keep up with the latest information on Adobe Pass on our product page at www.adobe.com/go/adobepass.

- Todd Greenbaum, Sr. Product Manager, Adobe Pass