A Good Day

Photoshop pushes a machine harder than most programs. It’s part of squeezing out as much performance from a machine as possible. Sometimes that means marginal hardware can show problems that look an awful lot like software bugs but aren’t. Or, even worse, cause a machine to mysteriously reboot (it’s happened). In other words, we take performance seriously, but that can sometimes tickle machine weaknesses.

So, as we’ve heard the occasional reports of mysterious pauses or bad slowdowns with CS2, we have really been paying attention. Trying to find a reproducible case can be very hard, though – and it’s hard to fix a performance issue if you can’t reproduce it (or at least have enough information to simulate it). And sometimes, through the course of dealing with a helpful user, it can become clear that there is something very weird or particular about the machine they have, and boy, at that point, do you really want to get your hands on that machine and take a good, deep look.

Well, through a series of events, we got lucky enough to find a user in the local area with a slowdown issue. So Seetha and I packed up his laptop in the car and headed out yesterday afternoon. When we got there, our user was so happy to have us.

We started with a demonstration of the problem. It took a while to happen. But when things started going “funny”, it was pretty clear that something was not normal. Ah, good, we thought. Not intermittent or one of those funny bugs that can simply disappear when two senior engineers walk in the room (you’d be surprised at how often *that* happens), but a repeatable problem that just happened to be tied to a specific machine.

I had us do the thing I always do when starting to try and figure out a bug – we pulled up Performance Monitor (on a Windows box – I pull up Activity Monitor on Macs, or install X Resource Monitor) and repeated the problem. Our user once again showed us the problem as Performance Monitor was watching CPU, disk activity, and free RAM. I have a set of colors I tend to use for these things, so that I can glance at Performance Monitor and know quickly what is going on. So we tried to reproduce the problem quickly. And the lines in Performance Monitor looked OK. So we sat back to think about what things might be going on – and after a short bit, I noticed that the blue line I use for free RAM was making a steady march downwards. With Photoshop idle. Ugh. And we watched the memory go to near-zero, at which point disk activity started going up, and we started poking around, and it now became clear why performance on this machine was getting so bad so quickly – a memory leak. A bad one. One that we had never seen before. So now the hunt was on for why…

We started up Task Manager and went to the Performance tab. Yup, available RAM was sitting at 3MB. On Windows XP, if your machine every has less than about 15MB of free RAM, you know you’re going to be in for trouble performance-wise. Ugh. So we quit Photoshop – which took a bit, since there was so much of Photoshop paged out due to the leak. I switch Task Manager to the Processes tab, added Virtual Memory Size, and started Photoshop, and didn’t touch it. Yup, there, plain as day. Munch, munch, munch went memory.

We did some first-step debugging – restarting without plug-ins, without scripting. No good. Still leaking memory. With all the easy answers gone, we hooked Seetha’s laptop up to our user’s network, installed the remote debugging stubs on the user’s machine, and started up a debug build. When the debug build started up, with all of it’s extra asserts and checks and paranoia built in, the smoking gun showed up right away. Aha. Bad font. And the light bulb went off in my head… The font preview menu. You see, I implemented the font preview menu to take absolutely zero extra time at startup. It generates all the previews on the fly, in the background, at idle time. Works really well, especially as most machines can read in a font and generate a preview just about as fast as they could read in a cache. We can usually generate all your previews in the first 20 seconds or so while the app is up, while some other programs that have WYSIWYG menus take more than twice as long to read in their cache.

Anyway, it matched the idle-time symptom of the memory leak we saw. And it struck me that if – **if** – a font failed in a certain way that caused our font rasterizer component to let the failure leak out (which isn’t supposed to happen) we could not only drop our memory on the floor but not remember that we tried that font. Bad news. So, we tried the workaround of turning off the preview menu, and of course, no memory leak. Now, both Seetha and I are eager to figure out which font it is, so we can bring the sucker back with us and dissect what went wrong. So, we bring up the debug build again, catch the problems, read the indicators, and get the font name. Cool. We turn on the preview menus and remove those fonts from the system, and start Photoshop and watch. Shoot. The memory leak is still happening. Seetha posits that it could be more than one font. So, we start a full-on search for the bad font (yup, the very same binary search-for-the-bad-font that we ask users to do when we suspect the problem is related to a bad font). Sure enough, with nothing but the minimal font set installed, Photoshop starts up fine – no leak. So we start moving fonts back, chunks at a time. No leak. We move more No leak. We’re back down to the one font we thought was bad, and moved it back. No leak. Huh?

At this point, we’ve actually completely solved the user’s problem with no workaround (all their fonts are back in place and the program is performing great). And we have enough information to simulate the bug back in the lab (which I’ve now done). So we pack up and head out, thanking our user for their time (I hope we conveyed how immensely we appreciated it – we really did). Our best guess is that some pre-caching or performance mechanism in the operating system somehow got a bad shadow copy of the font and that bad shadow copy was causing the issue, and that re-installing all the fonts (which is what we essentially ended up doing) force the bad shadow copy to be refreshed from the good copy. But that’s a guess.

Now, while I’d really like to know what actually went wrong, we ended the day with a happy user, information that might help some other users, and a cornered bug to be fixed for the next release. And both Seetha and I reflected on our way back to Adobe about all the bad software we deal with and programmers who wouldn’t even think of going to a user’s site to debug an issue (and certainly not 2 senior engineers like us).

A good day.


What I did on my summer vacation…

OK, so it wasn’t exactly summer. But I recently had my third sabbatical, which I combined with vacation to make for a whole 6 weeks away from work. I highly recommend it. It’s why there haven’t really been any entries here for a while. I really made a break from work – heck, I didn’t even fire up my VPN client for the entire 6 weeks. Or check e-mail. No, I’m serious – I think in this age of blackberries and always-on connections that taking a time out is a very good thing to do. I had worked so hard in helping Photoshop CS2 get out the door that the “Jane, Get me off this crazy thing!” feeling was overwhelming. So I stepped off the merry-go-round for a bit.

Am I implying that I didn’t have any computer contact? Of course not. I spent some time fighting against my media center PC. There was *finally* an ATI video card driver update that allows ‘PowerStrip’:http://entechtaiwan.net/util/ps.shtm to set up a proper 1080i output on the analog VGA output. I had previously used the ‘Omega’:http://www.omegadrivers.net/ modified driver, which is cool, but doesn’t come through Windows Update, and sometimes has a few additional bugs. Anyway, the 1080i output to my old Mitsubishi HDTV is now much more predictable across sleep/wake and reboots. And nailing the refresh rate properly got ‘PowerDVD’:http://www.cyberlink.com/multi/products/main_1_ENU.html into a state where it’s just doing the upsampling and not doing any temporal funkiness. DVDs now play back really beautifully. Why not use the Media Center DVD player? It sucks – it’s inefficient at doing the upsampling, doesn’t reliably do 5.1 output over the optical out from the sound card, and it does some decoding on the display card without allowing for a software override. And the ATI display card totally suck in this regard – it’s got the ‘chroma upsampling bug’:http://www.hometheaterhifi.com/volume_8_2/dvd-benchmark-special-report-chroma-bug-4-2001.html on the red channel something awful! (You may not want to follow the link – if you have a DVD player with the bug, you probably haven’t noticed. Read the article and you will. And it will irritate you. I now can’t help seeing the bug on players when it’s there.)

I also tried to do some capture off of the TiVo. Unfortunately, this is where the z540 having suboptimal components hurt. It’s got a Hauppage capture card in it. And Media Center doesn’t have a “record from VCR/line input” option anywhere, and other capture solutions either try and do on-the-fly encoding (which always ends up with a glitch somewhere) or try and use the native card encoding, which in the Hauppages’ case is the HCW2 codec – which, of course, doesn’t have a software decoder for it anywhere. Yuch. The best I’ve been able to do is to a full-frame capture with a post-process encoding using ‘Windows Media Encoder’:http://www.microsoft.com/windows/windowsmedia/9series/encoder/default.aspx. Of course, Windows Media Encoder itself has some problems – its de-interlacing isn’t very good. And recording full frames takes a seriously fast drive – the “spare” firewire drive I had around can only really do 12MB/s sustained, which didn’t quite cut it (I wouldn’t mind ordering one of the HP Personal Media Drives that will slip into a slot on the front of the machine, but in typical HP fashion, there is no easy way to find out what sizes of those drives are available or what the real specs on the drives are). I tried some of the other capture programs: ‘Open Video Capture’:http://www.008soft.com/, which can’t do a proper full-frame capture; ‘McFunSoft Video Capture’:http://www.mcfunsoft.com/mvs/, which, beside the absolutely awful company name, froze any time I tried to have a full-frame buffer capture of any length, and which has a horrendously bad user interface – that always forgot all the settings any time you quit it.

I know what I want here, and that’s what’s frustrating. And Media Center should have done it – yet another component of it which is inadequate. I really want a video re-encoder that knows how to use capture card-native format (or full-frame) and re-encode that through the ‘DivX’:http://www.divx.com/ encoder – my favorite, with the best de-interlacing and really nice quality. Windows Media Encoder only does Windows Media Player format, which is unfortunate. The quality is reasonable, so I guess that’s what I’ll have to live with for now. Yes, if this was a build-my-own machine, I would have tossed that Hauppage capture card and gotten something that could output something useful. That’s the problem with the HP z540 (and probably with the follow-ons, though I don’t have direct experience with them) – you really don’t feel like you can replace any components. At least not without cutting up the back of the machine. And given it’s cooling problems, I’m loath to do anything that can make them worse.

Anyway, that’s certainly not all I did with my time off! It wasn’t even a vey significant part of it – just the part that generates the most comments because, well, when things go right you tend not to notice them, and dealing with that machine gives you plenty to notice.

I started off my time off with a couple of weeks on Kauai. We overlapped with my brother Douglas for a couple of days. If you’re in the Poi’pu area, and you see a guy in a red wheelchair, say hi. Kauai is my favorite of the islands – the pace there is definitely the slowest, and when on vacation, that’s pretty much what I’m looking for. Susan likes to travel more, but this time I just needed a vacation.

After that, it was mostly catching up on the to-do list. I washed windows – which, on our house, is a major undertaking. There are so many windows in our house that when we were calling around for estimates to replace them a few years back, we got asked more than once if this was a bid for an apartment complex. It’s not that big a house, just a lot of windows. And, as is my way with that sort of thing, if I’m going to do it, I do it right – got all the screens down and pressure washed, cleaned and the coated all the windows with a polymer coating that should keep them clean longer. Even got the window above the entry way that’s real hard to get to.

I refinished a chair. Nothing spectacular, just an old chair with lots of nooks and crannies, which makes stripping a bear. But then, the days were mild with no wind, so I was able to do the work out on the patio. Me, my iPod, and harsh chemicals – what more could you want?

I also dragged my bicycle down, cleaned it up and rode it for the first time in, what, 7 years? Ugh. Gonna keep it down now and squeeze some morning rides in on the weekends. Nothing like a ride in crisp fall air to clear the head.

Tuned up my 5-burner barbeque. It’s an older Turbo model – black enamel, serious looking (none of that pansy stainless steel here! :-). Replaced the starter on it, which was interesting – the contrast between the original and the new one showed years of design tweaking. I’m ready for Thanksgiving now. For the past few years I’ve brined and smoked a couple of smaller turkeys on it (15 lbs. each). I’ve gotten really good at it. Susan and I have a bunch of friends and family over on Friday (so that our friends aren’t torn between our gathering and their own family things). The past couple of years, the smoked turkeys have always managed to disappear the fastest.

Oh, and I fixed up the yard lights. Or at least did the bulk of the fixing up. After lots of digging and crawling under the house (which I hate – it’s tight under there and I’m not exactly small) and lots of cursing at the unknown persons who originally ran the lights (and thought it was OK to simply twist wires together, wrap them in electrical tape and bury them), I’m most of the way to having this project done. I was originally hoping to actually have a couple of days at the end of my time off for just pure bumming around, but, well, even things like the time it takes to dig holes and run wires can be mis-estimated.

So, it was nice to escape the complexity of thinking about Photoshop for a while. Like sore muscles, my brain just needed some resting up. Nice, physical tasks, out in the air, without much complexity to them. It’s about as opposite of working on Photoshop as I can imagine – just the sort of break I needed to prepare for the next round of digging in and fixing things up…

Favorite Tools

Photoshop is, at the heart of it all, a tool. And while it contains the combined heart and skill of an experienced and dedicated team, it’s primary purpose is to let people get things done with images.

And while I have quite a collection of tools in my life – software and hardware – I always find it interesting to think about the tools I like the best, and what it is that that collection of tools has in common – and how can I make sure that Photoshop shares those attributes.

Now, the tool I use the most, by an incredibly large margin, is “Emacs”:http://www.gnu.org/software/emacs/emacs.html. Not surprising that a text editor is a developer’s most-used tool. But there are a few things about Emacs that I think set it far apart, especially from the “toy” editors that come as part of development environments. Now, I’ve been using it since, oh, I was in high school (that was a *really* long time ago), so I’ve got a symbiosis going on with in that probably biases my opinion against othe editors tremendously. But then, I’ve watched Photoshop users who are clearly the same way – knowing the shortcuts and keystrokes to drive it almost as an extension of their minds. Emacs is pretty much like that for me. Useable out of the box, but customizable using Lisp, it’s far more capable than other text editors, and I’ve added things and tweaked things so that it works just the way I want. It’s a replacement for Terminal on the Mac, it hosts Cygwin bash shell on Windows, it talks to our source control system, it’s completely cross platform, I have taught it how to steal the errors from the respective platform IDEs so that I can flip through errors more quickly, it highlights code the way I like, and I long ago set up an indentation style for coding that originally exactly matched – and then slightly improves upon – the original code indentation Mark Hamburg used (it’s a style I really like and find more readable than others). The customizability is something I’ve been advocating in Photoshop. Emacs does customizability right – guided options for those who don’t want to deal with immense complexity, and Lisp for those willing to roll up their sleeves. This is opposed to the customizability in Office, where it’s unguided and everywhere, leading to a quick mess in the UI if you’re not careful.

While I’m on development tools, I’ll mention “Perforce”:http://www.perforce.com, which is what we use for source control. So many people build overly complex source control systems, or systems that are missing the key concept of check-in atomicity. With a simple client/server system (no complex pretending to be a file system, or mounting remote directories or junk like that), and a branching system that is flexible enough to let us do what we want without being completely incomprehensible, it just lets us get our work done without getting in the way too much. I’ve used RCS, CVS, SourceSafe (oh, no!), and various others – and Perforce is my favorite so far. I know other people have ones they like better, but if it doesn’t provide that atomicity, I wouldn’t be interested.

My favorite tool around the house is my DeWalt 18v cordless drill. It’s just a brute. I like the three speed clutch – can really help with the Forstner bits – and the chuck is really nice. It’s great when a tool is robust enough to put up with the kinds of abuse and just not give up. I don’t like having any bad bugs in Photoshop, and the ones I hate the worst and the ones that end up on the top of my priority list are any that threaten user data. I think Photoshop should be as abusable as my drill.

While I’m on my woodworking tools, I really like the Bosch table saw I got myself last year. It’s got this great soft-start mechanism that doesn’t kick things off the table, a really beefy fence with a nice extension section. But I think the two best things are that it doesn’t take a screwdriver to pull out the throat plate, and there’s a little lever to lock the blade for changing (I had a blade grabber for my previous saw, and I was always misplacing it). It’s a slick, professional level tool (not a full size professional table, but just right for a pro-sumer like me) that feels really well made and thought out every time I use it.

The tool I spend the most time at home with, however, is my HD “TiVo”:http://www.tivo.com. Yes, I watch too much TV. But at least I’m really efficient at it ;-). The HD TiVo is a great example of where the software is more important than the hardware, but couldn’t do anything if the hardware wasn’t at least reasonably capable. Yeah, I know there are many uber-geeks with MythTV setups, etc., etc. But I really like the fire-and-forget nature of the TiVo. It can record two things at once, contains two OTA ATSC tuners (made it finally worth it to struggle getting an antenna on my roof – thanks Bob for the help!), and talks to my older Mitsubishi HDTV (with a transcoder in place). What is interesting to me about the TiVO was the setup. I got an HTPC (the HP DEC Z540 – very not recommended) at the same time, and it’s setup was a nightmare. TiVo, on the other hand, was much easier. And a good chunk of that was because of one thing – on the front, they burned some real estate for a small button and four indicator lights that tell you what the output format currently is – 480i, 480p, 720p, 1080i. That one thing right there made all the difference in the out of box experience (the HTPC, on the other hand, is a complete bear to get talking to my TV – usually takes three reboots with the right magic settings each time). And when the TiVo occasionally glitches – very, very rare – it’s easy to verify when it’s booted to the point where I *should* be seeing something on the TV. Combine all that with one of the best remotes out there (compare that to the HP’s awful remote), and software that hides a lot of complexity. Sometimes hiding too much for someone liek mw who always likes to have many knobs and buttons available, but for the most part seems like a really good compromise. I understand that DirecTV’s new sattelites only talk MPEG4, and I can understand why, but if they think that they can pry the TiVo out of my hands for their cheesy DVR box “upgrade”, they’re nuts (and I suspect DirecTV always made TiVO take out software features just so their own box doesn’t look so horrible when it finally limps out the door). Heck, if I had the Home Media option on the TiVo, I probably wouldn’t have gotten the HP DEC at all. I sure hope TiVo is allowed to produce an upgrade with MPEG4 capability (and hopefully *all* the software features enabled) – I’ll be first in line. Yeah, yeah, yeah, I can put together better boxes, hack Linux, yada yada (I already have enough programming projects around the house, thank you – more on those later) – the TiVo is a great, well executed *appliance*. A combination of hardware and software (and TiVo is clearly a software company – but they’ve done so well on the hardware for a software company that it’s hard to believe sometimes) that is there – reliably, simple, easily there.

I’ve got other favorite tools around: a Little Giant ladder – flexible but very sturdy and stores compactly, a DeWalt 18v reciprocating saw – terribly fun to cut things that way, Microsoft Trackball Explorer (5 of them, actually) – I like large-style trackballs and I find the thumb-based primary button doesn’t hurt my hand. Tools that are reliable, flexible, customizable, well-executed. And sometimes fun.

Isn’t that what a tool is supposed to be?

Finding (out about) Photoshop

As an engineer, I don’t necessarily get to use Photoshop every day. I think it’s important for me to try to experience the program as a user as much as possible, and to try and honestly compare it to some of the other software I use from a user perspective as well. To that end, I do try and set up time to put Photoshop to use.

Because that time is not as much as I’d like, I also try to keep tabs on the Photoshop community, to see what buzz there is. If there’s something that people are discovering, I’ll try and use that same part of the program in some way – while I probably know some of the engineering details about a large part of the program, that may not always translate into being able to help people out.

To that end, the site I’m probably most frequently visiting these days is Photoshop News – it’s a nice, concise summary of top issues, with links off to other sites that I probably wouldn’t find on my own. I stick the RSS feed off in my Firefox (what browser did you think I would use? I’m a cross platform kind of guy…) bookmarks toolbar, so I just have to occasionally pull down the menu to see what new stories have made it up there. I also add Rob Galbraith DPI as an RSS feed (what did we ever do without RSS?). And then, of course, the User-to-User forums, when I have time. Unfortunately, that seems less and less these days, though I’m trying to get back there more often now.

I also try and get to a Photoshop user conference whenever I can. I encourage others on the team to do so as well. Nothing is quite so instructive as watching someone demo on a big screen and catching when the ‘oohs’ and ‘ahs’ come out. Or seeing where the glitches in the various demos occur (and writing them down for fixing). But while I wish that being at a Photoshop conference could be a daily activity, it isn’t…

I’m sure I’ve missed some interesting places for Photoshop news. Let me know about them – I probably don’t have time to add too many to my daily routine, but who knows?


Saluting General Specific (Canon 20Da)

So, I find this camera fascinating. Not for it’s specific features, but more for what it represents and means. Here is a camera, based on a popular DSLR, that just by tweaking a few things could target the fairly specific needs of an not-very-large target market. Now, if you read the reviews of the camera on the general photography sites, you might not understand how neat this camera is. But if you know someone who does astrophotography as a hobby (like my father), you would know different.

The review here can be helpful in understanding part of why this camera is so interesting. With the capability of taking images nearly as good as sensors costing 4 times as much, it’s quite a bargin. But that’s only the beginning. Because, you see, with those other sensors, you have to have a tethered laptop with you. Not only that, the images are stored in a proprietary file format that has some, well, annoying software that you have to use. I actually tried to help my father with some processing one evening and was tearing my hair out by the end of it. (Though, if you’re lucky, you can get the files into FITS format, in which case the FITS Liberator is your friend). Now, if you are picky enough to know that you just don’t want to deal with Bayer filter geometrical artifacts, those more expensive solutions are probably still for you. But for most, it is a hobby, and like most hobbies, you never get to spend enough time at it, and anything like the 20Da that saves you that much time and lets you have more fun is worth it.

What is interesting to me is that a large company such as Canon would not only stumble into having a general camera such as the 20D that had some nice properties for this hobby, but they would actually listen to their customers and add the now incremental improvements to create the market specific 20Da – and then listen a second time and make the camera available worldwide after a Japan-only initial release.

I think that’s kinda cool.

And so it begins…

Welcome to my blog. In the great traditions of blogs (sorry, there’s no sarcasm tag), this first entry won’t be about much.

Living Photoshop is a purposefully vague title; it lets me fold many different Photoshop related subjects underneath it. It many ways I live Photoshop pretty much every day, and have been doing so for a very long time now. I’m also interested in the ways others live Photoshop every day, because it’s for those people I work so hard at helping to create this beast.

I am currently co-architect of Photoshop (along with Russell Williams) and Senior Lead, but it’s been a long and interesting path getting here. I started at Adobe in 1988, if you can believe that. Straight out of college. I started in the Display PostScript group. I went and helped out on the very first version of Adobe Type Manager (if you can remember that far back), did a printer with ATM in it (had to define an OEM interface for ATM, an interesting experience), did various ports of Display PostScript (including the first 64-bit port to the Dec Alpha – cool system, sigh…), put ATM in as a font server instance in X (wow, getting esoteric there).

I still remember the day I was walking through the halls back to my office and saw the BarneyScan box on a printer in the hall. I asked someone about it, and as I walked back to my office it really struck me what it could mean. As time passed (and it eventually became Photoshop, and then was rewritten in C++), I continuously suggested how I thought getting it onto Unix, where it could compliment Display PostScript, was a really good idea. Someone must have paid attention to the young punk, because I ended up working on that first port of Adobe Photoshop 2.5 to Sun and SGI. Followed that up with a version of Adobe Illustrator, then another of Adobe Photoshop (did some really cool stuff in that one, and we had it up on a 64 processor Sparc system one afternoon just for giggles – didn’t scale all the way up, but not much would). Well, the Unix applications weren’t really going anywhere, so I ended up moving to the core Photoshop team. Been there ever since. Done many features, big and small.

The best thing about working on Photoshop is that it’s never boring. We have a great customer focused team, and we all get excited to see what people do with our product, and we all take it pretty personally when things don’t work right. I’d like to think I’ve helped instill that attitude in those that haven’t been here as long.

So there ya go.