Legal
The views expressed in this blog are my own and do not necessarily reflect the views of Adobe Systems Incorporated.
Search
September 23, 2008
Reboot: New Habit, New Machine
Ok, time to get this thing restarted. I just need to make this a weekly habit. I've been better at being able to form good, new habits recently - a Google search turns up varying answers for how long it takes. I think it just takes conscious effort until it doesn't.
So, there's an interesting engineer-ism that I've seen be true more often than not: software will run best on whatever is on the engineer's desk. Well, here at work, for my Windows box, I've now been running Vista since it came out, and the 64-bit version of Vista for over a year. (I spend equal time on my Macs, so don't get any crazy ideas). The machine here at work has two quad-cores in it, which is great for compiling, but is kind of overkill for Photoshop. It's got 8GB RAM in it, which isn't overkill anymore.
Knowing that engineer-ism, I let that inform me when I recently bought a new machine for home. I bet I'm like a lot of you when it comes to such things - I like to look for the best value I can get, including being fast enough so that the lifetime of the machine is a good, long time.
The machine I got is a workstation-class (nothing high-end, mind you, but not consumer-level either) refurbished machine. It's got a quad-core processor in it - not the fastest, but not far from it - 4GB ram, 3 hard drives (I couldn't quite justify a NAS RAID setup at home... yet. The machine came with 1 drive, I just moved the biggest, most recently purchased drives over from the old machine using a couple of SATA-IDE adapters). I don't do any 3D work, and most of the time, I use the machine remotely from downstairs, so the onboard video was good enough. That's always easy enough to upgrade later if need be. I made sure that the motherboard supported 8GB of RAM, so I'm not hitting a wall there, either.
The machine came from a company which was proud of their "XP downgrade". Blech. I suspect it's just because that's what's on most of their engineers desks. Sorry, this geek likes Vista. I like the faster boot, the more aggressive caching (unused RAM is wasted RAM), the fixed video driver model, and rock-solid stability. Yes, everything got moved around from XP, and some of the moves still don't make sense to me. But things got moved around between NT and Win2k too, and I lived through that.
So, I looked through the pre-installed, uhm, software on the machine, didn't see anything worth caring about. I then went through my normal process for re-imaging a machine - bring up Add/Remove Programs on the old machine (yes, the renaming of that item in Vista still annoys me - it's harder to find in the middle of the Control Panel menu - which I still always set up to be a big flat menu off the Start button list), walk through the programs list, writing down those applications which I remember being useful. Then went and found updated drivers for the new machine (before re-installing the OS) , got them on a USB stick. And finally, did a full, fresh install of the 64-bit version of Vista Ultimate. Ah. Much better.
Is moving to an 64-bit version of Vista for everybody? My advice is the same as it's always been: check for driver availability and software compatibility before leaping. My personal perception is that driver availability for the 64-bit versions of Vista has changed dramatically over the summer, and I haven't had an issue with the recent versions of the any of the software I use. Your mileage may vary. My 3-year-old laptop is still running XP precisely because of those caveats.
So, there you have it. This engineer, on his main Windows machines, is running 64-bit editions of Vista. Take it for what you will.
September 12, 2007
Photoshop World - Heavy Lifting
Well, another Photoshop World come and gone. This year Adam Jerugim and I did a presentation on setting up a machine for doing very large file work. Thank you again to those who attended - we were up against some interesting stuff. I know I promised to have this entry up on Monday, but I've been struggling with getting PowerPoint to let me extract the information in a good way. I'll probably have to update this entry a couple more times as I figure out how to get what I want pulled out.
I wanted to call out a couple of things that are currently buried in the speaker notes, and I'm not sure if I got them across appropriately. First, setting Photoshop's memory percentage to 100% only makes sense if you've got more than 4GB of RAM in the machine, and again, only if you haven't run into trouble running the filters you need that way, and are on CS3. We've improved our ability to back off in the case that the machine we're on starts to page every version. However, it's still important to watch that free RAM (or, in the case of Vista, the amount still being used for the system file cache). It's important that you watch what's going on on your system when pushing things to their limit. If you're regularly seeing free memory (or the amount of free memory + system cache on vista) go below 20MB, it's time to back off that memory percentage setting and try again.
Someone had asked a question at the end about RAID types, and I wanted to repeat that here - for fastest performance, RAID 0 with 2-6 drives for the Photoshop primary scratch disk is the fastest way to go, but isn't a good place for storing files unless you're going to back them up very frequently (daily?) to a big server on the network. RAID 0+1 or RAID 5 would add in the reliability at some cost in performance or the need for additional drives. We still need to measure which of those is the best way to go. It'll be about throughput.
So, here's the current version of the presentation. The animations don't come across, and I still have to go through the speaker notes and clean them up and get a good export of that. Hopefully within the next day or so, and I'll update this post.
psworldperformancepresentation_export
[Update: I meant to get the version of the presentation with extended notes up before my travels, apologies for not getting that done. I'll get that up and catch up on the comments when I return in October. -Scott]
[Update: Sorry this took so long, but PowerPoint and I had to come to an, um... understanding. So below is the link to the annotated version of the presentation, with all slide builds manually exploded apart so that they are actually usefull. -Scott]
December 20, 2006
64 bits...when?
I've gotten a number of questions on the beta forums as to why Photoshop CS3 won't have a 64-bit version. It's definitely a when question, not an if, and there are a lot of factors involved. I though I might collect some of the information together here.
First, let's check all the 64-bit hype at the door. Being a 64-bit program means, most simply, that pointers in an application are 64 bits wide, not 32 bits. A 32 bit pointer means that an application can address 2 ^ 32 bytes of memory, or 4GB, at the most. The operating system an application runs on slices that application address space up, so that the application can actually only allocate a part of that address space for itself. Thus, on Windows XP, an application can use 2GB of address space, on Macintosh OS X 10.4, an application gets 3GB, on Windows XP 64-bit edition, a 32-bit application gets nearly 4GB of address space. Applications don't allocate RAM on most modern operating systems - that's a common misconception and a gross oversimplification your computer geek friends tell you because they don't want to explain virtual memory, TLBs and the like.
A 64-bit application doesn't have same that limit on its address space, since pointers are 64 bits - they can address a much larger amount of memory. That's pretty much it. 64-bit applications don't magically get faster access to memory, or any of the other key things that would help most applications perform better. Yes, the x64 instruction set has some more registers available, and that helps in limited circumstances, but the processing throughput of a memory bandwidth bound application is pretty much not going to benefit from being a 64-bit application. In fact, it gets worse in many cases because the internal data structures the application is dealing with get bigger (since many data structures contain pointers, and pointers in a 64-bit application are twice as big as in a 32-bit application). Memory bandwidth has not kept up with processor speeds, and has become a precious resource.
Let's contrast to the old switch from 16-bit to 32-bit computing. During that switch, 16-bit applications were already not really true 16-bit applications (in order to address 640k of memory, there was a really ugly hack that meant pointers weren't really pointers and weren't really 16 bits) - so the data structures an application was dealing with pretty much didn't change in size at all. Even if they did, memory bandwidth at that point in history was high enough relative to processor performance that what data structure size increases did happen were easily absorbed and didn't affect performance significantly. Not only that, but for many operations, moving to 32-bit computing meant a lot of fixed point math could be done in a lot fewer instructions, and a lot faster. For many applications, this was such a huge win -and one of the reasons why the switch to 32-bit computing happened fast relative to the 64-bit computing switch. These days, many 32-bit processors already have math instructions for doing 64-bit fixed point math to the degree most applications need it, and many things are done in floating point. Combine the performance penalty for the data structure size increase with the lack of any sort of other performance win, and the number of situations in which an application being 64-bit are a performance win is very small.
And, in the cases where it would be a win - dealing with a very large data set (Photoshop files over 1GB is a good guess as to about where things get interesting) - having the operating system use any unused RAM to cache scratch disk reads and writes can be nearly as effective (depending on how good the operating system is at this - and there are even a few add-on products which can make it even better) as having Photoshop itself being able to hold all of scratch within it's own address space. (Side note: a computer system is most efficient when it has nearly no free RAM - there really is no point in having RAM sit empty. An aggressive file cache is one good way of accomplishing this. Vista is supposed to be really good at this, but I don't have enough direct experience yet to know for sure.)
Ok, so in a very limited set of cases there might be a little bit of a win for Photoshop to be a 64-bit application. But...
A 64-bit application can't be run on a 32-bit chip or a 32-bit operating system. That means a 64-bit version of Photoshop would have to be a completely separate binary from the 32-bit version. A separate binary has a huge cost associated with it (in terms of features we can't do because we have to spend resources elsewhere). Quality assurance is a big part of that - you essentially have to dive in and test every corner of the app for each binary, across the matrix of supported operating systems - but there are also many ancillary pieces that add to that cost as well. And given that a Universal Binary application really is two separate binaries smashed together (and accounting for the kinds of issues that an application can have going from big endian (PPC) to little endian) we already had a lot of extra cost going through this cycle. Adding the cost of adding a 64-bit version to the mix of things that were already on the have-to-do list - especially in light of the very limited benefits - and doing a 64-bit version this cycle really became an unreasonable thing to think about it.
There's more. Since a 64-bit application can only run on a 64-bit chip with a 64-bit operating system, just how many users could run a 64-bit version in the first place? Macintosh OS X 10.4 isn't fully 64-bit - many of the libraries an application would need to be fully 64-bit aren't available. So, on the Macintosh side the answer is zero. Ok, so consider the Windows side: Windows XP 64-bit edition has been out a bit. How many Photoshop customers run it? A vanishingly small percentage - there's been a lack of compatible drivers, especially for the kinds of things that make up a Photoshop workflow, like scanner divers, monitor calibration bug drivers, and the like. Things are looking better for Vista - it comes with a wider spread of 64-bit drivers out of the box - but there are still 64-bit versions of some of the pieces of the puzzle missing there, and the expected Vista adoption rates mean that the number of Photoshop customers running the 64-bit version of Vista will remain very tiny over the next couple of years. Yes, there's a whole look to the future thing, but even considering that, the timing just isn't right yet.
As for the engineering part of it, well, I want to do it. I want the transition to 64-bit computing to happen sooner rather than later, I'd really like to not have to worry about address space limitations so much. Once, long ago, I ported Display PostScript to the Dec Alpha workstations (what a nice processor, sigh...), so I have a good idea of what the level of engineering work will be.
At some point, some of these things will change - certainly the number of systems capable of running a 64-bit version of Photoshop will - and at some point it will make sense to do a 64-bit version. That wasn't this time around. But like I said, it's a when, not an if.
December 14, 2006
Beta!
It's real. A big, public beta of Photoshop CS3. Now, if you own CS2, you can see what's been keeping us so busy lately.
Now, you can read all about features on other sites - I don't think this needs to be yet another me too list of features. And this isn't the place for reporting bugs or asking questions - use the Labs forums for that, I'll be there along with some of the other engineers, time allowing (hey, we have to finish this thing).
We are doing this mostly because we wanted to get the Macintosh Universal Binary of the product into your hands as soon as we could. It really wasn't possible to do that with an updated CS2, it really did take that much effort, and it really wasn't ready until recently. But then, it wouldn't be fair to do just a beta on the Mac side and not let the Windows users along for the ride.
This is a beta, but I think it's in pretty darned good shape. Part of why is that as a development team we broke free of the old waterfall methodology. If you're a developer, visit Waterfall 2006. I find it pretty funny, maybe that's just me - I've been living the waterfall nightmare for so long, the laughter comes from a dark place...
What it means is that we've kept the bug count low the entire cycle - especially the nastiest bugs. Now, your "favorite" long-time bugs and annoyances may or may not be fixed (yes, we do listen, I keep a list, and we get to as many as we can), and after all, this is still a beta, there will be dragons there, but for the most part for something still this far from release, it's in pretty good shape - and this wasn't a special, lots-a-pain bug-fix build, but pretty much just pulling out a daily. Yeah, I know, some of you developers will be going "but we've been doing things that way for a long time". Hey, around here, a lot of the groups fall right back into waterfall at the first sign of trouble. Nasty.
A couple of notes - due to the platform rules, the Macintosh version when running native on an Intel-based Macintosh will not load and run PPC-only plug-ins. You'll have to run under Rosetta to access those old scanner plug-ins. Or better yet, just use the scanner software in stand-alone mode. Also, brush-sized cursors aren't working when running Macintosh Intel native yet (they do work in Rosetta). With a little help from Apple, the replacement for the old framebuffer-fiddling methods are well underway, but they didn't make the beta.
The new UI can test Windows XP video drivers a little more, so you may want to make sure you have those updated, especially if you see some problems. I like the new palette panes, but they do take getting used to, so give them a chance. There are legacy workspaces in there if you really need the floating palettes back. Vista support is in there, too.
As for performance, you should see start up times that are much lower than for CS2. We've really worked hard on this - there's more to go, of course, this is just a beta. Oh, and as for comparing performance between platforms now that they use the same chip and we're a Universal Binary on the Macintosh, well, I just don't think such comparisons are valid - there are so many variables involved: number of processors, system memory bandwidth, disk I/O bandwidth, OS scheduling, API performance. So if you see sites claiming that one platform or the other performs better, take it with a big grain of salt - I do.
So, if you've got the time and inclination (and Photoshop CS2) go ahead and grab the beta from Adobe Labs tomorrow and see what we've been up to.
[Edit - 12/15/06 9:30amPST - The links still aren't live on the labs pages yet. It's being worked on. - Scott]
[Edit - 12/15/06 12:15PST - They're live now. Visit the forums at Adobe Labs for more help and info.]
September 5, 2006
Vegas, baby!
Ah, Photoshop World! A chance to see some of the wacky and wonderful ways people use Photoshop in their work. And, of course, to listen and watch for the kinds of problems people see in their everyday work.
I'll be there, and so will several other members of the Photoshop engineering team. Find me if you can, say hello, wish me a happy 40th birthday (it's only a few days early), tell me your one biggest annoyance (but keep it to one). And if you ask, I can show you some of the pictures I took while in the Galapagos in June.
March 8, 2006
Reap what you Measure
You reap what you measure. Measure twice cut once. Don't tweak without measuring.
So you want Photoshop to perform faster? You're gonna want to measure. But measure what?
There are many things on a system that can go wrong. You can run out of RAM, a disk can be going bad, you can be bottlenecking on that disk, and you could, in fact, simply be trying to do too much at once. Unless you watch most of this at once, you're not going to know what could be going wrong.
Fortunately, both the Macintosh and Windows systems have tools available - built-in or easily available - which measure most of what you need to know. On Windows, I use the built-in Performance Monitor. On the Macintosh, there's Activity Monitor, but I personally prefer a great little application called X Resource Graph, because I like the visual history of the strip chart format, and it has most of the indicators I like to use.
On the Windows side, I have a standard set of things I like to map on each machine. I use a consistent color scheme, so that I can go from machine to machine and just glance at Performance Monitor and know what I'm looking at - if you only have one machine, this may not be an issue for you. I choose blue for memory, red for processor (and green for a dual processor system), yellow for networking items, and various of the dark colors for disk items, with a dashed style. I map both Available KBytes and Available MBytes so I can play the dual-sensitivity trick - I adjust the scaling on Available MBytes so that the blue line is somewhere in the middle of the strip window, adjusting the scale on the graph so that the blue line ends up somewhere between 10 and 100 (on a machine with 1GB, you'll end up with it somewhere in the upper half using 0.1, on a machine with 2GB to 5GB, you'll end up with it in the bottom half using 0.01), then I adjust the scaling on Available KBytes to match, then set the scale to 0.001. What this does is give you a gross-level idea of free memory on the system, and when memory gets below 100MB free (the danger zone), you get a detailed view. For each physical disk, I like to add % Disk Time, I set the line style to dashed and a dark color, for no particular reason beyond that's the way I've always done it. And of course, mapping % Processor Time to a big bright red is to make it obvious. I then hit the General tab and turn off all the Display Elements. You may need the legend turned on until you've learned what all the colors are, but I'm used to this setup by now, and it lets me have a real uncluttered window with no wasted real estate. While I'm there, I set the update interval to something useful like 5 seconds. Yeah, it's not as much fun to watch that way, but you can usually get a better sense of tends over time this way.
Once I've done that, I go back into the Performance Monitor part and use View / Customize View to turn off everything. I then go into File / Options and set Console mode to User Mode, limited access, single window. I then use File / Save As to save out perfmon.msc, putting it into my Documents and Settings\<USERID>\Start Menu\Programs\Startup folder, so that it loads up when I log in. Yeah, you could save it somewhere else and open it when you need it, but I find that having it up all the time is a big advantage. I then close Performance Monitor, and select perfmon.msc from the Start Menu, All Programs / Startup (you'll see it there if you saved it there in the previous step). That gets you this nice, clean, compact monitor window that you won't mind always having up and that you can glance at (when it's not buried behind the Photoshop main window - it's a great window to go along with all the palettes on your new second monitor 
and see what's going on.
On the Macintosh side, things are a little easier. You can go with the standard Activity Monitor, in which case I'll leave it up to you to figure out where to watch, or you can install the excellent X Resource Graph, which will let you quickly map all the items you need, and again has a nice strip chart format that gets you that sense of history that can often help figure out what's going on. I know some of the folks around here like some of the other monitor programs, but when I glance over to see what's going on, I don't need to try and have me brain interpret fancy colored circles on anything. The key thing here again is turning on enough items in the preferences, and set up a color scheme you like. Again, I set this as one of my login items. I really like X Resource Graph's ability to stalactite/stalagmite some of the graphs, but I really wish I could have a strip graph of the memory items instead of the paging graph (well, in addition to, really). And as on Windows, I like to slow down the refresh time to once every five seconds, to get a reasonable amount of history into those strip charts. There are color schemes which you can download which aren't so colorful, and won't be so intrusive on your image editing, and I usually have mine set up a little more subdued than this. And again, it's a great thing to tuck over on that second monitor.
On both systems, the question becomes, just what are you looking for?
Well, let's start with memory. Photoshop uses a lot of memory. And it has it's own virtual memory systems because it has to deal with data sets far larger than the address space allowed to 32-bit applications (and we can't just do a 64-bit version and it's still not clear if it's worth it), plus additional reasons I'll cover some other time. What it means is Photoshop is going to use the memory percentage of free RAM you've told it, and try and live within that amount. What's key here is that while living within that amount, you really, really don't want the amount of system free memory to get below 20MB. What happens if you get somewhere below that amount of free RAM (the exact amount shifts around between the operating systems, and there are other variables - 20MB seems to be a reasonable safe zone on both platforms) is that the operating systems start trying to get free RAM back by paging out some things. Well, given that Photoshop at that point is probably occupying most of that RAM that the operating system wants to reclaim, it's highly likely that parts of Photoshop memory will get paged out. Well, when Photoshop then goes to try and operate on the part of the image that was in that memory that got paged out, it has to wait for it to get paged in. Even worse, if the system paging file and Photoshop's scratch file are on the same physical drive (meaning you only have one set of drive heads), what will often happen is that Photoshop wants to read a new tile into memory from it's scratch files, but the memory it's trying to read that into has been paged out - so a little gets paged in (operating systems have annoyingly small page sizes), Photoshop continues trying to read in it's tile, a little gets read in, then more of the place it's trying to read it into needs to get paged back in, then a little more gets read, then... Well, you get the idea. Now, when both Photoshop scratch and the paging file are on the same physical disk, each of those little flips between reading the scratch file and paging in the memory to read it into forces the drive head to go to that part of the disk. Photoshop will now be running about the slowest it could run on your machine. 
So, let me repeat myself. You really, really don't want free RAM to go below 20MB. Complicating matters are the filters that like to allocate a single chunk of memory to hold the part that's going to be operated on. You'll know these filters because memory will get used up in a big chunk. Photoshop will try and be good about giving up tile memory to trade in, but it's probably not getting it all back (I'll cover that some other time, too). If you use those filters often (or if you use a few other memory hungry applications at the same time), then your overall session performance may be better if you back off on the Photoshop memory percentage. Again, the key is that free memory not go below 20MB (I think I heard that somewhere else before... ).
The next thing to watch for is disk. There's disk activity, of course, related to paging activity which will also show up in the paging graphs. And then there is disk activity related to reading files, and related to reading scratch. The latter is more interesting, especially as it relates to CPU usage. Kick off a global filter on a big file, and depending on what the filter is, you'll see a bunch of CPU activity, then a bunch of disk activity. There should be very little paging activity during this period (you did watch that free memory amount, right?). Shortening the period of time the CPUs are waiting for the disk would be the purpose of setting up a RAID array for scratch, if you're so inclined. You'll notice that on a heavily fragmented disk, the disk read amounts won't form a smooth graph. To avoid scratch file fragmentation, setting up the Photoshop scratch on it's own disk is a good thing to do. Concentrating on scratch over actual file open/save reads and writes is preferable because you are usually exercising the disk for scratch usage many times more than for file reading and writing. It's often instructive to watch how what you are doing in Photoshop affects the CPU and disk activity.
So, who else is hogging all that memory anyway? That's where Windows Task Manager or Activity Monitor can come in handy once in a while. Simply bring them up (and switch to the process tab in Task Manager) and look for the other applications which are using big chunks of memory. Sometimes, you'll find surprises there, like e-mail programs taking up 80MB, or a web browser eating 40MB, or an MP3 player taking 60MB (yes, these amounts seem big to me - but I don't know what's going on inside of them, they could be being efficient, but I doubt it). Now, if you run Photoshop occasionally, and you want to keep those other programs running, no problem - just turn back Photoshop's memory percentage. With that level of use, tuning Photoshop's memory percentage to the last megabyte isn't going to buy you enough over that short a term to justify the likelihood that you'll get below that magic 20MB free number. In the busy machine case like this, less is often more. If a machine is used to run Photoshop, pretty much Photoshop, from morning to night, then watching and tuning the memory percentage is definitely going to save you some time over the long run. And, even better, if there are unavoidable reasons why your machine acts slow sometimes (like a disk is already busy even before you start saving a file) you can catch it and plan for that inevitable coffee break.
Note that there are side issues, such as the current pause on Macs with more than 4GB of RAM in them (a stale leftover of the Unix days is kicking in and spoiling the fun - the disk cache that Photoshop writes the scratch file through in large memory situations is getting flushed occasionally, and the operating system ends up forcing Photoshop to wait until that's done), but this should get you started with the gross-level tuning of your Photoshop environment that is fairly easy and, best of all, cheap.
-Scott
February 10, 2006
Modifiers
Modifiers. I have a love-hate relationship with modifiers. On the one hand, they allow instant access to a lot of functionality in a way that's easy to manipulate with your non-dominant hand. On the other hand, they have, in the past, hidden useful functionality in ways that are almost impossible to discover. This was exasperated with the introduction of Tablet PCs, and their complete lack of useful non-dominant hand input buttons or other devices (which is a very annoying shortcoming of Tablet PCs - c'mon, I may not be able to control a second mouse or whatever with my left hand, but for gosh sakes I can push a darned button!).
In some ways, they are a necessary evil - we've already essentially used the entire keyboard for shortcuts, the menu system is already complicated enough without being overloaded with more command variants, and while we've been on a general kick to make sure all previously hidden modifier only behavior is exposed in the UI somewhere, if you're really in a groove you don't want to have to track your mouse all the way over to the options bar just to add a chunk onto a selection. Pointer locality is an important part of usability (even if we do sometimes fall short - a subject for later). Heck, I've always meant to either find or implement some sort of floating window modifier substitute widget in my copious free time.
We're in such desperate straights for additional modifier keys that we treat the space bar as one (which has caused more than one funky issue in the past, with keyboard and locale switching). I've joked in the past that we need Emacs style prefix keys, but I really was only joking, as nobody would get those. It doesn't help that when you're trying to squeak in a feature at the 11th hour, and it's already too late to modify the documentation, that attaching behavior to a modifier doesn't make a liar out of the screen shots. But it also doesn't make for a discoverable feature.
We have tried to fix this in general by exposing what was modifier behavior only in the user interface - particularly in the options bar. We've also tried to rectify this a little bit for the Macintosh folks by adding the tool hint text to the Info palette in Photoshop CS2. Windows users will recognize that this is the same text that was in the status bar previously (there were other reasons for getting rid of the status bar, plus I'm a stickler for trying to maintain platform parity). Unfortunately, the Info palette is underneath the Navigator palette by default. So, bring that Info palette to the front (the tool hint text is enabled by default - check out the Info palette options from the palette flyout menu for more stuff) and watch the tool hint text while you use different tools.
Yes, of course that doesn't explain everything. Like it misses explaining what hitting a modifier before the first mouse click with one of the lasso tools does (it is different than what happens with the modifiers after the first click with one of the lasso tools). In that case, the cursor should give some hint of what will happen.
It also doesn't help when you're not trying to do something with a tool. Like with the Option/Alt key in most dialogs (hint: watch the button labels). Or in Photoshop 6, 7 and CS, holding the shift key while modifying a linked type layer will operate on all of the type layers (in Photoshop CS2, you can just multi-select the type layers).
Now, most if this is documented in the manual I'm sure you've read (nudge, nudge, wink, wink). But there's always the undocumented ones, too, like holding the shift key on Windows when dragging the Photoshop CS2 application window (I'm going to make you try it instead of describing it - and no, it's not perfect, but it really helps in multi-monitor situations).
Remember, nothing goes boom when you press a modifier. And there are times when us engineers really want to put in useful functionality when it's too late to modify the UI. So don't forget to give your local friendly modifiers a try.
-Scott
January 16, 2006
Shovelware
So, by this point we hopefully have all heard about spyware, adware, malware. And we all know that this is nasty stuff, and that we should avoid having it on our systems - they tend to slow things down, eat resources, get in the way, destabilize things, what have you. Bad stuff.
Today, I want to cover shovelware. Shovelware is that software which is of such low quality or is otherwise so nearly useless to you (or has an atrocious usefulness to resources ratio) that just about the only way the companies can get it in front of you is to pay hardware manufacturers to pre-install it on new machines, or otherwise bundle it with other software you do want. If you don’t get a choice to install it or not, it’s probably shovelware. Shovelware takes up resources, but usually isn’t as malevolent as spyware - it’s not trying to record keystrokes or the web pages you visit. But it does cost you. And often it ends up in front of you because someone somewhere along the supply chain didn’t have the right motivations - they weren’t thinking about how to help you, the customer, but how to extract the biggest revenue dollars. Maybe some engineer stopped being excited about going to work and it just became a job.
Who knows?
I’ve hit a bunch of examples of shovelware lately. It bugs me, because I like a neat, clean system. A clean system performs better, feels better. And logs in a whole bunch faster.
The first resource offense most shovelware commits is the task tray icon on Windows. Hey, some software really has a legitimate reason for starting up when I log in and having an icon there. Palm Hot Sync, a virus scanner, Microsoft AntiSpyware - these all have a legitimate reason for starting up when I log in, and might as well have an easy access icon. But QuickTime? Java? C’mon, these things can be lazy started. And don’t insult me by giving me an option to hide the darned icon. That’s not what I want - I want you to not eat resources just because I logged in. Some of these things are even more pointless. HP’s printer drivers now install an additional little tray icon utility, and I don’t see the value add - everything it gives you access to is available elsewhere as far as I can figure out. But can I stop it from loading at startup? Is some critical service wrapped up in that resource theft? Do I really need some WinCinema manager in the taskbar tray making login take an extra 10 seconds? And just why is it so hard to keep Microsoft Messenger from starting up?
Of course, there’s also the Start menu pollution and disk space munching that happens. I needed to burn a CD from an .iso image on my Media Center the other day, and the program I would normally use (CDRWin) doesn’t like that drive. So, I’d heard about Nero, decided to give that a try. First, the download was 100MB - crazy. Clearly, there’s other junk in there. When I installed the trial, it installed all sorts of shovelware I didn’t want (now, I don’t know if what they installed was good stuff or not - it was just useless for me, and that’s the key). I just wanted the burning component. But because I couldn’t just get and buy the burning component, I gave up, uninstalled Nero, and searched a little harder, finding CD Burner XP Pro. Not only was it free, it worked and didn’t come with any shovelware. Nice.
Heck, I think shovelware is one of the things that hurt RealPlayer so much. Once they started bundling all the shovelware with the player, I gave up on it. Who needs all that garbage - some of which verges on adware - when there are ways of getting media content without it?
Then there’s the shovelware that comes with a new system. Both Dell and HP are bad about this (I’m sure others are as well, I just don’t have direct experience with them). It’s why many of us have a policy of wiping out and installing from a real Windows XP install CD right after getting a new machine, and why I think most of those “restore” disks are practically useless. My policy is to never buy a machine that doesn’t come with a real Windows install CD - a policy I violated with my Media Center PC, and of course ended up regretting when I restored it to try and stabilize it only to find that all the shovelware HP was installing meant the system didn’t even start out too terribly stable. Yuck. Wouldn’t it be nice if a system just coming out of the box was actually clean, and not the mucked up, dirty, registry scrambled mess that most manufacturers think pass for acceptable these days?
Of course, there’s also the case of software that starts out useful and devolves. I like my MusicMatch radio, but over the past two years, the darned thing has gone from eating 10% of my machine CPU to just about half. I don’t know what you do to waste that sort of time, but that’s getting near the limit of tolerability - that is software that is on the verge of becoming shovelware for me (though I really like having music playing while I work, so I can tolerate quite a bit). MP3 players tend to skirt this edge more than other programs it seems, grabbing memory and keystrokes and otherwise doing things that get in the way.
Now, I’m sure that any and all this stuff may be useful to someone, somewhere. But it isn’t to me, and I don’t want to have to go digging around afterward shutting it off. I know conventional wisdom says that modern machines have enough power and memory for all this stuff, but there is just so much of this junk that it’s death by a thousand paper cuts. Every percentage of CPU actually counts - it could make the difference between a clean and glitchy video capture. And every chunk of RAM that is forcefully kept alive counts - when that blue line (free RAM) in my Performance Monitor hits 10MB, things are going to come to a crawl (I always have Performance Monitor running on Windows and X Resource Graph running on Mac, though Activity Monitor is OK in a pinch).
I suppose the worst part is that it’s just so darned hard to keep a machine clean. Yeah, I go into Add/Remove Programs regularly and clean up old junk - but sometimes things are bundled in ways that you can’t get rid of the bad without getting rid of the good. And even if you could, you still have to have a good registry cleaner because so many of the uninstallers are half-hearted efforts. And you shouldn’t really have to pull open the Program Files directory to go cleaning up disk files left behind, but often do. It used to be that you’d have to completely re-install Windows 98 every so often because the system itself became unstable. Now, while Windows XP itself is stable, it’s getting to the point where you have to do a re-install just to really get rid of all the vestiges of all that shovelware.
Sigh.
-Scott
January 10, 2006
The Sad State of Hardware
I (vaguely) remember when I was a young, naive programmer, and I assumed that compilers could essentially do no wrong. Gosh, if something was broken or didn’t work right, it must be my fault. Of course, that assumption didn’t last long. I remember when I first realized that my program was compiling differently based on if and where comments were - must have been some bug in the compiler parser, who knows? Once that line was crossed, though, I began to get a sense of what problems might be mine, what actually might be the compiler, when I should pull up the generated code to look for compiler faults - and figuring out how to work around them. And I’d get a sense of which compilers were the worst offenders. Heck, there was one compiler which seemed to have the development philosophy of shoving in every optimization they could think of with a major version release, then back them out in patches as they got code generation bug reports.
Hardware was different. Because hardware could always break, you’d always be on the lookout for it to go funny. From loose cables, to unseated cards in slots, to overheating problems. You’d expect something to eventually go bad because it lived in the physical world and eventually everything dies. But you didn’t expect design flaws - at least not flakey ones that affected specific software packages. If something was busted, it was pretty consistently busted. And there wasn’t nearly the variety of hardware that you could plug together - not to mention that tolerances of dodgy parts was looser.
Things have changed, and not for the better. It was always the case that chips might have errata - but they’d usually be covered up in the microcode or in the BIOS before things shipped. Devices would have issues, but the driver would be tweaked before the device was shipped to compensate. In general, you could buy a box, and have reasonable confidence it was put together reasonably and would do what you wanted. Sure, there were the manufacturers who had the (very) annoying habit of putting in cheesed-up and dumbed down OEM versions of certain boards, and it would be hard to get driver updates for them. You just learned to avoid them.
Then there was that first interesting case of a design flaw on the motherboard. An undersized capacitor specification on a reference motherboard design would allow the system memory bus to go undervolt if the bus was saturated for more than a certain length of time. Guess which program was best at managing to do that? I think for a lot of the users affected by that problem it was the first time they had encountered real hardware issues - and much like my early experience with compilers, they were hard pressed to believe that it was a hardware issue. Heck, isn’t it always the fault of the software? There had always been bad RAM issues that Photoshop seemed to be the only piece of software that could tickle it, but this was the first really widespread hardware issue. And worse, the only fix was a physical motherboard swap.
Now, of course, it’s even worse. With the internet being an accepted method of delivering drivers and BIOS updates and whatnot, I think most PC manufacturers have gotten lazy. Things no longer necessarily go out the door in a working state. Heck, the Media Center PC I have at home couldn’t burn DVDs for the first 10 months I owned it, until a BIOS update and a driver firmware update (and a re-image).
Don’t get me wrong - I think the internet updates are great for letting slightly older hardware adapt to new things without having to go replacing it all. Much better to just flash your WiFi card to add better security than it is to replace it.
But I think we’ve hit a point where things are just abusive for the average consumer. We now see RAM issues so frequently that it’s part of the FAQ list. Video drivers seem to be perpetually broken in some way or another. I simply don’t feel comfortable recommending machines from any name brand PC makers anymore because they all seem to have serious weak points in their product lines. Now, one could argue that it’s the price preassures that have been at least partly to blame for this - if people didn’t try and save every last dollar from their purchases, that the manufacturers could use more reliable and better tested parts.
I just nailed down an issue today having to do with Hyperthreading and memory allocation. Yes, a BIOS update solved it (as did turning off hyperthreading). But should the average Photoshop user really have to know how to figure out the motherboard manufacturer, find the BIOS update and apply it? Or turn off Hyperthreading? And why wasn’t the BIOS update pushed out over Windows Update for such a serious issue? To the user, it just looked like Photoshop was freezing up in the middle of painting or other operations.
And, of course, it hits Photoshop more than other applications. It’s the nature of the beast - the data sets we’re dealing with are very large for an end-user application, and we move it around a lot faster than other applications. It’ll expose marginal hardware more often than the best system diagnostic software.
Some would argue that we should make Photoshop tolerant of bad hardware. On this I have to disagree. We’re talking about penalizing all users because some people bought dicey systems or cheap RAM. For those who get bad hardware, it sucks - but the right place to take that up with is the hardware manufacturer (yes, I know such things generally fall on deaf ears). But until they get some kickback, they’re going to continue to put out flakey stuff, crammed with shovelware, that’ll manage to run your MP3 player or browser, but gives up the ghost when trying to capture video or touch up your pictures. Or, let’s push the PC magazines to put together some real stress tests, and rate the hardware vendors on long-term stability - knowing what machine is fastest at Quake 4 is useless if it reboots after 9 hours of heavy use because of a thermal issue. I’d say start buying Macs, but things don’t seem to be too much better there, either. I think the hardware is (generally) in better shape, but I think the OS could use a bit more bug fixing.
Until then, just realize that marginal hardware can affect software, especially those programs which try and get the best performance out of your machine. Users shouldn’t have to become hardware diagnosticians just to remove red eye from their kids’ pictures, but that’s where we’re at.
Sucks but true.
November 18, 2005
A Good Day
Photoshop pushes a machine harder than most programs. It’s part of squeezing out as much performance from a machine as possible. Sometimes that means marginal hardware can show problems that look an awful lot like software bugs but aren’t. Or, even worse, cause a machine to mysteriously reboot (it’s happened). In other words, we take performance seriously, but that can sometimes tickle machine weaknesses.
So, as we’ve heard the occasional reports of mysterious pauses or bad slowdowns with CS2, we have really been paying attention. Trying to find a reproducible case can be very hard, though - and it’s hard to fix a performance issue if you can’t reproduce it (or at least have enough information to simulate it). And sometimes, through the course of dealing with a helpful user, it can become clear that there is something very weird or particular about the machine they have, and boy, at that point, do you really want to get your hands on that machine and take a good, deep look.
Well, through a series of events, we got lucky enough to find a user in the local area with a slowdown issue. So Seetha and I packed up his laptop in the car and headed out yesterday afternoon. When we got there, our user was so happy to have us.
We started with a demonstration of the problem. It took a while to happen. But when things started going “funny”, it was pretty clear that something was not normal. Ah, good, we thought. Not intermittent or one of those funny bugs that can simply disappear when two senior engineers walk in the room (you’d be surprised at how often that happens), but a repeatable problem that just happened to be tied to a specific machine.
I had us do the thing I always do when starting to try and figure out a bug - we pulled up Performance Monitor (on a Windows box - I pull up Activity Monitor on Macs, or install X Resource Monitor) and repeated the problem. Our user once again showed us the problem as Performance Monitor was watching CPU, disk activity, and free RAM. I have a set of colors I tend to use for these things, so that I can glance at Performance Monitor and know quickly what is going on. So we tried to reproduce the problem quickly. And the lines in Performance Monitor looked OK. So we sat back to think about what things might be going on - and after a short bit, I noticed that the blue line I use for free RAM was making a steady march downwards. With Photoshop idle. Ugh. And we watched the memory go to near-zero, at which point disk activity started going up, and we started poking around, and it now became clear why performance on this machine was getting so bad so quickly - a memory leak. A bad one. One that we had never seen before. So now the hunt was on for why…
We started up Task Manager and went to the Performance tab. Yup, available RAM was sitting at 3MB. On Windows XP, if your machine every has less than about 15MB of free RAM, you know you’re going to be in for trouble performance-wise. Ugh. So we quit Photoshop - which took a bit, since there was so much of Photoshop paged out due to the leak. I switch Task Manager to the Processes tab, added Virtual Memory Size, and started Photoshop, and didn’t touch it. Yup, there, plain as day. Munch, munch, munch went memory.
We did some first-step debugging - restarting without plug-ins, without scripting. No good. Still leaking memory. With all the easy answers gone, we hooked Seetha’s laptop up to our user’s network, installed the remote debugging stubs on the user’s machine, and started up a debug build. When the debug build started up, with all of it’s extra asserts and checks and paranoia built in, the smoking gun showed up right away. Aha. Bad font. And the light bulb went off in my head… The font preview menu. You see, I implemented the font preview menu to take absolutely zero extra time at startup. It generates all the previews on the fly, in the background, at idle time. Works really well, especially as most machines can read in a font and generate a preview just about as fast as they could read in a cache. We can usually generate all your previews in the first 20 seconds or so while the app is up, while some other programs that have WYSIWYG menus take more than twice as long to read in their cache.
Anyway, it matched the idle-time symptom of the memory leak we saw. And it struck me that if - if - a font failed in a certain way that caused our font rasterizer component to let the failure leak out (which isn’t supposed to happen) we could not only drop our memory on the floor but not remember that we tried that font. Bad news. So, we tried the workaround of turning off the preview menu, and of course, no memory leak. Now, both Seetha and I are eager to figure out which font it is, so we can bring the sucker back with us and dissect what went wrong. So, we bring up the debug build again, catch the problems, read the indicators, and get the font name. Cool. We turn on the preview menus and remove those fonts from the system, and start Photoshop and watch. Shoot. The memory leak is still happening. Seetha posits that it could be more than one font. So, we start a full-on search for the bad font (yup, the very same binary search-for-the-bad-font that we ask users to do when we suspect the problem is related to a bad font). Sure enough, with nothing but the minimal font set installed, Photoshop starts up fine - no leak. So we start moving fonts back, chunks at a time. No leak. We move more No leak. We’re back down to the one font we thought was bad, and moved it back. No leak. Huh?
At this point, we’ve actually completely solved the user’s problem with no workaround (all their fonts are back in place and the program is performing great). And we have enough information to simulate the bug back in the lab (which I’ve now done). So we pack up and head out, thanking our user for their time (I hope we conveyed how immensely we appreciated it - we really did). Our best guess is that some pre-caching or performance mechanism in the operating system somehow got a bad shadow copy of the font and that bad shadow copy was causing the issue, and that re-installing all the fonts (which is what we essentially ended up doing) force the bad shadow copy to be refreshed from the good copy. But that’s a guess.
Now, while I’d really like to know what actually went wrong, we ended the day with a happy user, information that might help some other users, and a cornered bug to be fixed for the next release. And both Seetha and I reflected on our way back to Adobe about all the bad software we deal with and programmers who wouldn’t even think of going to a user’s site to debug an issue (and certainly not 2 senior engineers like us).
A good day.
-Scott
August 26, 2005
Finding (out about) Photoshop
As an engineer, I don't necessarily get to use Photoshop every day. I think it's important for me to try to experience the program as a user as much as possible, and to try and honestly compare it to some of the other software I use from a user perspective as well. To that end, I do try and set up time to put Photoshop to use.
Because that time is not as much as I'd like, I also try to keep tabs on the Photoshop community, to see what buzz there is. If there's something that people are discovering, I'll try and use that same part of the program in some way - while I probably know some of the engineering details about a large part of the program, that may not always translate into being able to help people out.
To that end, the site I'm probably most frequently visiting these days is Photoshop News - it's a nice, concise summary of top issues, with links off to other sites that I probably wouldn't find on my own. I stick the RSS feed off in my Firefox (what browser did you think I would use? I'm a cross platform kind of guy...) bookmarks toolbar, so I just have to occasionally pull down the menu to see what new stories have made it up there. I also add Rob Galbraith DPI as an RSS feed (what did we ever do without RSS?). And then, of course, the User-to-User forums, when I have time. Unfortunately, that seems less and less these days, though I'm trying to get back there more often now.
I also try and get to a Photoshop user conference whenever I can. I encourage others on the team to do so as well. Nothing is quite so instructive as watching someone demo on a big screen and catching when the 'oohs' and 'ahs' come out. Or seeing where the glitches in the various demos occur (and writing them down for fixing). But while I wish that being at a Photoshop conference could be a daily activity, it isn't...
I'm sure I've missed some interesting places for Photoshop news. Let me know about them - I probably don't have time to add too many to my daily routine, but who knows?
-Scott
August 19, 2005
Saluting General Specific (Canon 20Da)
So, I find this camera fascinating. Not for it's specific features, but more for what it represents and means. Here is a camera, based on a popular DSLR, that just by tweaking a few things could target the fairly specific needs of an not-very-large target market. Now, if you read the reviews of the camera on the general photography sites, you might not understand how neat this camera is. But if you know someone who does astrophotography as a hobby (like my father), you would know different.
The review here can be helpful in understanding part of why this camera is so interesting. With the capability of taking images nearly as good as sensors costing 4 times as much, it's quite a bargin. But that's only the beginning. Because, you see, with those other sensors, you have to have a tethered laptop with you. Not only that, the images are stored in a proprietary file format that has some, well, annoying software that you have to use. I actually tried to help my father with some processing one evening and was tearing my hair out by the end of it. (Though, if you're lucky, you can get the files into FITS format, in which case the FITS Liberator is your friend). Now, if you are picky enough to know that you just don't want to deal with Bayer filter geometrical artifacts, those more expensive solutions are probably still for you. But for most, it is a hobby, and like most hobbies, you never get to spend enough time at it, and anything like the 20Da that saves you that much time and lets you have more fun is worth it.
What is interesting to me is that a large company such as Canon would not only stumble into having a general camera such as the 20D that had some nice properties for this hobby, but they would actually listen to their customers and add the now incremental improvements to create the market specific 20Da - and then listen a second time and make the camera available worldwide after a Japan-only initial release.
I think that's kinda cool.