March 10, 2012
Mercedes makes a real-world Content-Aware Fill
Brilliant use of LEDs & cameras:
[Via Rob Cantor]
February 21, 2012
Scalado Remove promises handheld tourist-zapping
About five years ago we gave Photoshop the ability to stack multiple images together, then eliminate moving or unwanted details. Similar techniques have appeared in other tools, and now it appears you’ll be able to do all the capture & processing with just your phone. Here’s a quick preview:
The Verge has a bit more detail on the user experience. [Via John Dowdell]
November 30, 2011
“Photoshopped or Not? A Tool to Tell”
My longtime boss Kevin Connor left Adobe earlier this year to launch a startup, Fourandsix, aimed at “revealing the truth behind every photograph.” Now his co-founder (and Adobe collaborator) Hany Farid has published some interesting research:
Dr. Farid and Eric Kee, a Ph.D. student in computer science at Dartmouth, are proposing a software tool for measuring how much fashion and beauty photos have been altered, a 1-to-5 scale that distinguishes the infinitesimal from the fantastic. Their research is being published this week in a scholarly journal, The Proceedings of the National Academy of Sciences.
Check out the interactive presentation of before & after images. Details are on the NY Times.
November 07, 2011
Mobile facial recognition promises clever new apps
Check it out:
Petapixel writes,
The video at the top of this post is a Polar Rose demo of an app called “Recognizr”, which recognizes people’s faces and provides you with links to their social media accounts.
Imagine a world where every person on the street can be identified by simply pointing your phone at their face. Curious about a stranger? Point your camera at them to pull up their Facebook profile. People who had concerns over facial recognition in Facebook photos are going to have a fit about this one…
I remain eager to see what developers can do in terms of building photography & design apps. If you see anything cool, give a shout.
November 05, 2011
Demo: Microsoft’s 3D “Holodesk”
This project leverages a Kinect sensor to let you manipulate 3D virtual objects with your hands:
[Via]
November 03, 2011
Sneak peek: Adobe image recognition technology
Researcher Jon Brandt demos a potential new feature for searching through a large library of images by identifying images that contain the same people, backgrounds, landmarks, etc.:
October 25, 2011
Amazing tech for turning video into 3D
I will never get over how lucky I am to work with people like this:
In this video, Sylvain Paris will show you a sneak peak of a potential feature for editing videos, including the ability to create 3D fly-throughs of 2D videos and change focus and depth of field.
October 24, 2011
Research: Auto-selecting good stills from a video
A couple of years ago, Esquire shot a magazine cover using not a still camera but a high-res RED video camera. What was groundbreaking becomes commonplace, and as video capture resolution increases, so does the possibility of using stills as photos.
To make that easier, Adobe engineers & University of Washington researchers are collaborating on a method of automatically finding the best candid shots in a video clip. Check it out:
Very cool–though I continue to suspect there’s a market for auto-selecting the most ridiculous, unflattering images of one’s friends…
October 18, 2011
Eye-popping tech for inserting 3D objects into photos
“With a single image and a small amount of annotation,” writes researcher Kevin Karsch, “our method creates a physical model of the scene that is suitable for realistically rendering synthetic objects.” Fascinating:
Check out the project site for much more detailed info. [Via Zorana Gee]
October 17, 2011
Adobe demos amazing deblurring tech (new video)
Last week over a million people (!) watched a handheld recording of this demo. Here’s a far clearer version*:
And here’s a before/after image (click for higher resolution):
Now, here’s the thing: This is just a technology demo, not a pre-announced feature. It’s very exciting, but much hard work remains to be done. Check out details right from the researchers via the Photsohop.com team blog. [Update: Yes, it's real. See the researchers' update at the bottom of the post.]
* Downside of this version: Bachman Turner Overdrive. Upside: Rainn Wilson.
October 16, 2011
The Throwable Panoramic Ball Camera
The Throwable Panoramic Ball Camera captures a full spherical panorama when thrown into the air. At the peak of its flight, which is determined using an accelerometer, a full panoramic image is captured by 36 mobile phone camera modules.
[Via Jeff Tranberry]
September 21, 2011
Video: Bizarre face-substitution technology
God that’s creepy. I want to look away… but I cannot.
Creator Kyle McDonald credits Jason Saragih’s FaceTracker library, the ofxFaceTracker addon, and openFrameworks. [Via]
August 22, 2011
“Really Being John Malkovich”
Adobe researcher Eli Shechtman & collaborators have created this excellent bit of madness:
Given a photo of person A, we seek a photo of person B with similar pose and expression. Solving this problem enables a form of puppetry, in which one person appears to control the face of another.
Now, let’s see if we can pry a webcam version out of them… [Via]
July 28, 2011
Exhibit: Photo Tampering Throughout History
You can build a business manipulating photos; how about building one by detecting those manipulations?
My longtime boss Kevin Connor was instrumental in building Photoshop, Lightroom, and PS Elements into the successes they are today, and he taught me the ropes of product management. After 15 years he was ready to try starting his own company, so this spring he teamed up with Dr. Hany Farid (“the father of digital image forensics,” said NOVA). Together they’ve started forensics company Fourandsix (get the pun?), aimed at “revealing the truth behind every photograph.”
Now they’ve put up Photo Tampering Throughout History, an interesting collection of famous (and infamous) forgeries & manipulations from Abraham Lincoln’s day to the present. Numerous examples include before & after images plus brief histories of what happened.
I wish Kevin & Hany great success in this new endeavor, and I can’t wait to see the tools & services they introduce.
Related/previous:
June 30, 2011
Google adds “Search by Image”
Ah–I’d been wondering what that little camera icon in the Google Images search field meant. As the company explains,
You might have an old vacation photo, but forgot the name of that beautiful beach. Typing [guy on a rocky path on a cliff with an island behind him] isn’t exactly specific enough to help find your answer. So when words aren’t as descriptive as the image, you can now search using the image itself.
Or, “Any sufficiently advanced technology is indistinguishable from magic.” — Arthur C. Clarke
April 13, 2011
Photography: Shallow depth of field on iPhone
Stanford professor & occasional Photoshop team collaborator Marc Levoy has created SynthCam, an interesting tool for simulating large-aperture photo effects using a tiny-aperture cell phone camera:
For more examples, tutorials, etc., see Marc’s site. [Via]
January 07, 2011
Trimensional: 3-D Object Scanning for iPhone
Fascinating:
Here’s more info. [Via] As more and more devices can capture and display 3D data, I think it’ll become clearer why we’ve invested in giving Photoshop a 3D infrastructure.
Video: Show your bones
Crafty German folks + gaming hardware = Creepy good times.
“The cross section isn’t actually the user’s skeleton but a volume visualization of a medical data set,” notes PCWorld. Here’s more info on the Medical Augmented Reality project.
November 29, 2010
Sneak peeks: New Adobe digital imaging tech
At Adobe MAX last month, digital imaging researcher Sylvain Paris showed off some tech he & colleagues are cooking up in Adobe’s Boston office. Here he touches on color/tone matching between photos; more sophisticated auto-correction of color and tone (based on analyzing thousands of adjustments made by pro photographers); and image de-blurring:
Lots of other really interesting MAX sneaks are collected here.
November 23, 2010
Video: Body Dysmorphia, Xbox edition
I’ve been a fan of Robert Hodgin’s visual experiments for many years, and now he’s creating some intriguing work by hacking a Microsoft Kinect:
See also his Dueling Kinects demo. (And I’m probably alone in this, but these weird characters are giving me flashbacks of the bad guy in RoboCop 2.) [Via]
October 19, 2010
Bokode: Using bokeh to transmit info
What an interesting idea: researchers are using bokeh (lens blur), instead of sharpness, to transmit lots of data into cameras (think barcodes/QR codes on steroids). Check out the demo:
Check out the project page for more details. [Via]
October 18, 2010
“Diminished Reality” removes objects from video in real time
Interesting tech, to say the least:
More info (in German) is on the project site. [Via Tobias Hoellrich]
October 15, 2010
Todor talks plenoptic imaging
Did you know that the Photoshop team has a resident theoretical physicist? If you’d like to meet him, check out next Thursday’s Silicon Valley ACM SIGGRAPH talk:
Recently we and others have gained deeper understanding of the fundamentals of the plenoptic camera and Lippmann sensor. As a result, we have developed new rendering approaches to improve resolution, remove artifacts, and render in real time. By capturing multiple modalities simultaneously, our camera captures images that are focusable after the fact and which can be displayed in multi view stereo. The camera can also be configured to capture HDR, polarization, multispectral color and other modalities. With superresolution techniques we can even render results that approach full sensor resolution. During our presentation we will demonstrate interactive real time rendering of 3D views with after the fact focusing.
See previous video: Adobe demos refocusable images.
October 07, 2010
Video: Reshaping human bodies on the move
Built at last, built at last, thank God almighty, I’ll be built at last… According to Popular Science:
Developers at the Max Planck Institute for Informatics in Saarbrücken, Germany compiled 3D scans of 120 men and women of varying sizes, merging them into a single model that can be morphed to any shape and overlaid atop original footage.
The software, called MovieReshape, builds on existing programs that track an actor’s silhouette through a scene, mapping the body into a morphable model. Using the compiled 3D scans, the program can create realistic-looking and moving body parts to the programmer’s specifications.
Check out the project site for more info. [Via Jerry Harris]
September 25, 2010
Adobe demos refocusable images
At NVIDIA’s technology conference this week, Adobe researcher Todor Georgiev demonstrated GPU-accelerated processing of plenoptic images. As Engadget puts it, “Basically, a plenoptic lens is composed of a litany of tiny “sub-lenses,” which allow those precious photons you’re capturing to be recorded from multiple perspectives.” Plenoptic image capture could open the door to easier object matting/removal (as the scene can be segmented by depth), variable perspective after capture, and more.
This brief demo takes a little while to get going, but I still think it’s interesting enough to share.
July 21, 2010
Video: Local layering ideas
Jim McCann is a graphics researcher (you might remember his interesting work with gradient-domain painting), and I’m happy to say he’s joining the Adobe advanced technology staff. He has some ideas about dealing with the limitations of traditional graphical layering models (as seen in Photoshop, After Effects, Flash, etc.):
For more videos & papers on the subject, check out the project page. [Via Jerry Harris]
July 20, 2010
Video: Technicolor dreamglove controls a computer
When he’s not helping bring Puppet Warp to Photoshop, Jovan Popovic does interesting work at MIT in computer interfaces and… fashion?
More videos & info are on the project team’s site.
[Via]
July 16, 2010
Video: HDR relighting technology
Here’s some very cool imaging tech, though it’ll be interesting to see how many people will take the time to create multiple exposures, each with different controlled lighting:
If this is up your alley, check out a paper and video on the subject that some Adobe researchers put together a couple of years back.
April 07, 2010
Video: Automated reshaping of human bodies
Oh my; how long until we see the Ralph Lauren EmaciatorPro(™) Edition?
Here’s more info on the project. (And no, unlike Puppet Warp, it’s not a CS5 thing.) [Via Jerry Harris]
March 03, 2010
“Enhance!” Redux
Heh–here’s a nice little satire of phony image enhancement on TV (see previous montage):
Of course, image scientists continue to work on all sorts of new craziness, so it’s all just a matter of time… right?
February 22, 2010
Video: “A computational model of aesthetics”
People always like to joke about Photoshop eventually adding a big red “Make My Photo Good” button, automatically figuring out what looks good & what adjustments are needed. Of course, researchers are working on just that sort of thing:
As someone who aspires to be creative, I have mixed feelings. The idea of rating images according to precomputed standards of beauty makes me think of the Robin Williams character in Dead Poets Society excoriating a textbook that rated poetry along two axes:
Excrement! That’s what I think of Mr. J. Evans Pritchard! We’re not laying pipe! We’re talking about poetry. How can you describe poetry like American Bandstand? “I like Byron, I give him a 42 but I can’t dance to it!”
And yet, I find I’m intrigued by the idea, wanting to run the algorithm on my images–if only, maybe, to have fun flouting it. I also have to admit that I’d like to see the images taken by certain of my family members (not you, hon) run through such algorithms–if only to crop in on the good stuff.
[Via Jerry Harris]
January 04, 2010
Photos to sound & back again
- A technology called Photosounder can treat images as audio (demo). “Sounds, once turned into images,” they say, “can be powerfully modified to achieve effects and results that couldn’t be obtained in any other way, while images of all sorts reveal the infinite kinds of otherworldly sounds they contain.” [Via]
- In a related vein, scientists have turned dolphin calls into kaleidoscopic patterns. (Note the image gallery navigation controls on the right.) [Via]
October 20, 2009
Video: New from Adobe Labs, Content-Aware Fill in Photoshop
You like? :-) (Here’s some more background on the technology.) To see higher-res detail, I recommend hitting the full-screen icon or visiting the Facebook page that hosts the video.
As with all such sneak peeks, I have to be really clear in saying that this is just a demo, and as such it’s not a promise that a technology will ship inside a particular version of Photoshop. (As the late Mac columnist Don Crabb told me years ago, “There’s many a slip ‘twixt cup & lip.”) Still, it’s fun to show some of the stuff with which we’re experimenting.
October 07, 2009
PhotoSketch: Internet Image Montage
Oh, that’s rather cool, then:
I’ve seen various experiments at Adobe that fetch & automatically composite images, but the idea of basing searches on sketches is new to me. Details are in the researchers’ paper (PDF).
Almost completely unrelated, but in the spirit of cool image science, during last night’s sneak peeks at Adobe MAX, Dan Goldman showed a little taste of “PatchMatch” (“content-aware healing”) integrated into Photoshop. (As always, no promises, this is just a test, yadda yadda.)
July 10, 2009
Wide-angle image correction tech
Adobe researcher Aseem Agarwala, working with Maneesh Agrawala & Robert Carroll at Berkeley, has demonstrated techniques to enable “Content-Preserving Projections for Wide-Angle Images.” That may sound a little dry, but check out the demo video (10MB QT) to see how the work enables extremely wide-angle photography. [Via Dan Goldman]
Aseem contributed the depth-of-field extension feature to Photoshop CS4. For previous entries showing advanced imaging work, check out this blog’s Image Science category.
July 01, 2009
Super cool video stabilization technology
Adobe researchers Hailin Jin and Aseem Agarwala*, collaborating with U.Wisconsin prof. Michael Gleicher & Feng Liu, have unveiled their work on “Content-Preserving Warps for 3D Video Stabilization.” In other words, their tech can give your (and my) crappy hand-held footage the look of a Steadicam shot.
Check out the demonstration video, shot at & around Adobe’s Seattle office. (Hello, Fremont Lenin!) It compares the new technique to what’s available in iMovie ’09 and other commercial tools.
As with all research papers/demos, I should point out making technology ready for real-world use can require plenty of additional work & tuning. Still, these developments are encouraging. [Via]
[Previously: Healing Brush & Content-Aware Scaling on (really good) drugs.]
* If you’ve created a panorama using Photoshop, you’ve used Hailin’s (image alignment) and Aseem’s (image blending) work.
June 01, 2009
Image science radness o’ the day
“This is your Healing Brush.
“This is your Content-Aware Scaling.
“*This* is your Healing Brush & Content-Aware Scaling on (really good) drugs…”
Adobe researchers Eli Shechtman & Dan Goldman, working together with Prof. Adam Finkelstein from Princeton & PhD student Connelly Barnes, have introduced PatchMatch, “A Randomized Correspondence Algorithm for Structural Image Editing.”
No, I wouldn’t know a randomized correspondence algorithm for structural image editing if it bit me on the butt, either, but just check out the very cool video demo. More details are in the paper (one of the 17 papers featuring Adobe collaboration presented at SIGGRAPH this year).
So, what do you think? [Via]
April 10, 2009
Adobe papers light up SIGGRAPH
I was excited to hear that researchers at Adobe have submitted 22% of all papers accepted at SIGGRAPH this year. That’s a pretty incredible accomplishment*. In addition, Wojciech Matusik has been selected as this year’s recipient of the ACM SIGGRAPH Significant New Research Award. Congrats, guys!
The company has been making significant investments & attracting top talent in this area in recent years, and it’s great to see those efforts bearing fruit. It’ll be even better when we start harvesting more of this research as real-world features in Photoshop and other apps–and believe me, we’re working to do just that.
* By way of comparison, Microsoft had 6 papers accepted this year (vs. Adobe’s 17). Microsoft has 90,000 employees; Adobe has 7,000.
December 13, 2008
Adobe previews “Infinite Images” technology
Remember Shai Avidan, the co-creator of seam carving (Content-Aware Scaling) who joined Adobe last year? Just as he did at Adobe MAX last year, Shai took to the stage this year with an eye-catching demo. Collaborating with Prof. Bill Freeman and a team from MIT, Shai has been working on "Infinite Images," "a system for exploring large collections of photos in a virtual 3D space." The team writes:
Our system does not assume the photographs are of a single real 3D location, nor that they were taken at the same time. Instead, we organize the photos in themes, such as city streets or skylines, and let users navigate within each theme using intuitive 3D controls that include pan, zoom and rotate…
We present results on a collection of several millions of images downloaded from Flickr and broken into themes that consist of a few hundred thousands images each. A byproduct of our system is the ability to construct extremely long panoramas, as well as image taxi, a program that generates a virtual tour between a user supplied start and finish images.
To read up on some details, check out the PDF (shared via Acrobat.com):
You could also visit Shai’s site to read up on “Non-Parametric Acceleration, Super-Resolution, and Off-Center Matting,” not to mention “Part Selection with Sparse Eigenvectors”–but I’d recommend being a lot smarter than I am. ;-) (We just may have to name our next child “Eigenvector.”)
December 04, 2008
Promising video research from Adobe
"Dan Goldman is an old friend of mine from ILM," writes FX pro Stu Maschwitz. "He now works for Adobe’s top-secret G*d Dammit Put This In A Product Now division." Check out Dan’s Interactive Video Object Manipulation demo to see if you agree. (Now that Photoshop Extended can work with video, it’s fun to imagine the possibilities. No promises, of course.)
September 09, 2008
Colliding hadrons, sinking subways, & more
- Casas de sciencia:
- The Big Picture features some gorgeous images of the Large Hadron Collider, nearly ready to create a black hole and swallow the world as we know it. (The dorkiness of their little fire-fighting vehicle could rupture spacetime, too.)
- Seed shows eerily deserted labs at night.
- Image manipulation isn’t just for political candidates anymore: even the weather isn’t safe.
- No fakery needed: Environmental Graffiti hosts the 30 Most Incredible Abstract Satellite Images of Earth.
- The British Library’s Bodies of Knowledge online exhibition is loaded with great imagery, exploring how the body has been viewed through history. [Via]
- The NYT reports on a deep-sea home for subway cars. I wonder whether global warming will someday make the idea of deliberately flooding subways seem quaint.
June 29, 2008
Hot image science o’ the day
Pravin Bhat & friends at the University of Washington have put together a rather eye-popping video that demonstrates Using Photographs to Enhance Videos of a Static Scene. I think you’ll dig it. (The removal of the No Parking sign is especially impressive.) [Via Jeff Tranberry]
The work builds upon research by Adobe’s Aseem Agarwala (who was instrumental in bringing Auto-Blend to Photoshop CS3). Adobe Senior Principal Scientist (and UW prof.) David Salesin is helping facilitate more collaboration between Adobe teams & academia, recruiting full-time hires like Aseem & sponsoring visiting researchers like Hany Farid.
(Note: As always, please don’t take my mentioning of various tech demos as a hint about any specific feature showing up in a particular Adobe product. I just post things that I find interesting & inspiring.)
Previously:
June 08, 2008
Cool painting tech demo o’ the day
Photoshop engineer Jerry Harris is responsible for the application’s painting tools, and he’s always got an eye open for interesting developments in the field of computerized painting. This morning he passed along a cool demo video of James McCann and Nancy Pollard’s Real-time Gradient-domain Painting technology.
In a nutshell, according to the video, "A gradient brush allows me to paint with intensity differences. When I draw a stroke, I am specifying that one side is lighter than the other." Uh, okay… And the video is a little ho-hum until the middle. That’s when things get rather cool. Check out cloning/duplicating pixels along a path, plus the interesting approach to painting a band of color.
August 28, 2007
Imaging heavy hitters join Adobe
A number of rock stars from the world of image science have recently joined Adobe:
- That crazy-cool image resizing demo I mentioned last week continues to get all kinds of attention. I was therefore happy to learn that co-creator Shai Avidan joined the Adobe office in Newton, MA (just down the ‘pike from MIT) last Monday. Here’s a bit more info and Shai’s photo.
- Wojciech Matusik began work at Adobe in May. He’s done some really cool work in the emerging fields of multi-aperture photography, 3D TV, and much more. Like Shai, he works from the Newton office.
- Sylvain Paris is due to join Adobe in a couple of weeks. He’s worked on techniques for matching tones across photos ("Make my image pop like Ansel’s"), generating 3D data from 2D captures, and more. His paper on bilateral filtering was written with MIT colleagues Jiawen Chen (who interned this summer at Adobe) and Fredo Durand.
Adobe Senior Principal Scientist David Salesin, who manages this crew, notes that "If you count their SIGGRAPH papers as well, you’ll see that current Adobe employees had 11 of the 108 papers in the conference."
Now, let me inject a disclaimer: Just because a particular researcher has worked on a particular technology in his or her past life, it’s not possible to conclude that a specific feature will show up in a particular Adobe product. How’s that for non-commital? ;-) In any case, it’s just exciting that so many smart folks are joining the team (more brains to hijack!).
[Update: Cambridge, MA-based Xconomy provides additional context for this news.]
August 19, 2007
“Holy crap”-worthy imaging technology
Wow–now this I haven’t seen before: Israeli brainiacs Shai Avidan and Ariel Shamir have created a pretty darn interesting video that demonstrates their technique of "Seam Carving for Content-Aware Image Resizing." When scaling an image horizontally or vertically (e.g. making a panorama narrower), the technology looks for paths of pixels that can be removed while causing the least visual disruption. Just as interesting, if not more so, I think, is the way the technology can add pixels when increasing image dimensions. Seriously, just check out the video; I think you’ll be blown away. (More info is in a 20MB PDF, in which they cite work by Adobe’s Aseem Agarwala–the creator of Photoshop CS3′s Auto-Blend Layer code.) [Via Geoff Stearns]
I hope to share more good stuff from SIGGRAPH soon. While I was being stuffed with ham sandwiches by kindly Irish folks, a number of Adobe engineers were speaking at & exploring the show. Todor Georgiev, one of the key minds behind the Healing Brush, has been busily gluing together his own cutting edge optical systems. More on that soon.
March 05, 2007
Digital imaging goes to court
CNET reported recently on a court case that involved image authentication software as well as human experts, both seeking to distinguish unretouched photographs from those created or altered using digital tools. After disallowing the software, written by Hany Farid & his team at Dartmouth, the judge ultimately disallowed a human witness, ruling that neither one could adequately distinguish between real & synthetic images. The story includes some short excerpts from the judge’s rulings, offering some insight into the legal issues at play (e.g. "Protected speech"–manmade imagery–"does not become unprotected merely because it resembles the latter"–illegal pornography, etc.).
As I’ve mentioned previously, Adobe has been collaborating with Dr. Farid & his team for a few years, so we wanted to know his take on the ruling. He replied,
The news story didn’t quite get it right. Our program correctly classifies about 70% of photographic images while correctly classifying 99.5% of computer-generated images. That is, an error rate of 0.5%. We configured the classifier in this way so as to give the benefit of the doubt to the defendant. The prosecutor decided not to use our testimony because of other reasons, not because of a high error rate.
The defense argues that the lay person cannot tell the difference between photographic and CG images. Following this ruling by Gertner, we performed a study to see just how well human subjects are at distinguishing. They turn out to be surprisingly good. Here is a short abstract describing our results. [Observers correctly classified 83% of the photographic images and 82% of the CG images.]
Elsewhere in the world of "Fauxtography" and image authenticity:
- In the wake of last summer’s digital manipulation blow-up, Reuters has posted guidelines on what is–and is not–acceptable to do to an image in Photoshop. [Via]
- Calling it "’The Most Culturally Significant Feature’ of Canon’s new 1D MkIII," Micah Marty heralds "the embedding of inviolable GPS coordinates into ‘data-verifiable’ raw files."
- Sort of the Ur-Photoshop: This page depicts disappearing commissars and the like from Russia, documenting the Soviet government’s notorious practice or doctoring photos to remove those who’d fallen from favor. [Via]
- These practices know no borders, as apparently evidenced by a current Iranian controversy, complete with Flash demo. [Via Tom Hogarty]
- Of course, if you really want to fake people out, just take a half-naked photo of yourself, mail it to the newspaper, and tell them that it’s a Gucci ad. Seems to work like a charm. [Via]
[Update: PS--Not imaging but audio: Hart Shafer reports on Adobe Audition being used to confirm musical plagiarism.]
Terms of Use | Privacy Policy and Cookies (Updated)