I went into this endeavor thinking that I would clearly delineate between different Quadro level cards and along the way understand what each one offered in the way of performance.  I was methodical in setting up my system and in trying to create real-world tests that would emphasize what the GPU brought to the table.

While I did learn a lot (which I will pass on to you in due course), the experience and tests in some cases were not always as clear-cut as I had expected.

In my role at Adobe, I sometimes have to present an ‘executive summary’ that gives the key points in a pithy and concise manner.  For those of you that have no time to waste and lets be honest that’s most of us, here are my key findings:

• The idea of a balanced system for Adobe CS5 and above with regards to 64-bit native applications is critically important to having the very best performance.
• While you still get the Mercury Playback Engine without having a GPU card installed on your system, I was genuinely surprised by how much of a difference it made on many of the tests I ran.
• What the GPU provides in terms of a boost is dependent on the kinds of codecs you’re using.  This is similar to the CPU which makes me point you back to my first point again (!)
• The difference in cards is a mixed bag, but there are clear differences in performance as you scale up.
• Determining the right card for you depends on two primary things: the rest of your system’s capabilities (point 1 again) and the kind of codec(s) you’re using.
• Finally, the fractional resolution playback system in Premiere Pro CS5 and later is a marvelous feature that allows lesser systems to still have acceptable or even good performance with big, heavy productions.  Combined with the GPU, this allows the most powerful and flexible editing/playback engine out there today.  Some people call playing back at half-res as ‘cheating.’  On the contrary, I find that I don’t even know when I’m doing that and have to check periodically.  It’s a SUPER feature that I think some people don’t understand.

The system:

I am using a 2-year-old HP Z800 with 16GB of RAM and an internal 4-drive SATA RAID array.  The drives were getting a consistent 424MB/s read and write using the AJA utility.   I have two physical CPU’s each with six cores and hyper-threading turned on to present a 24-core system.  The chips are XEON X5660 and have a clock speed of 2.8Ghz.  The system contains an AJA Xena/Kona 3 and I may test the GPU capabilities with AJA and others in the future, but for this test, all sequences were Adobe native.

In my opinion, I would say the following about the general sense of the system:

Prepping the box:

To make sure the Z800 was using the latest drivers, I visited the HP website and upgraded the BIOS as well as all Windows 7 files that were necessary.  As a person who is routinely playing with a variety of software packages, I uninstalled as many as were reasonable and also ran my favorite utility – Advanced System Care Free on the HP as well.  This great little utility helps allow for optimization, especially for people that are not uber geeks.

Adobe Premiere Pro and Adobe Media Encoder each had a minor update right before the tests commenced, so I used 5.51 for both.

For NVIDIA drivers, I used at the latest (at the time of this article, 275.89)

On the slightly geekier side, I went through and tried to remove as many processes as I could to provide the most performance out of the box without over-clocking or other semi-questionable activities.

The Cards to be tested:

I wanted to keep this pretty real, but also exclusively Quadro (I’ll explain more on that in a bit).  In the category of ‘real’ I excluded the Quadro 6000 simply because I don’t think there are many people who are prepared to shell out that kind of money for GPU acceleration within Premiere Pro.  Hint – you need to be a RED user to even really have a good reason to in my opinion.

I wanted to use the popular Quadro FX4800 (from the last generation of products) to provide a baseline reference and I wanted to see how much performance I could get out of a Quadro 2000.  Finally, the two that make the most sense for Premiere Pro users – The Quadro 4000 and Quadro 5000.

Quadro 2000 gets booted at the last moment:

Unfortunately, I had to pull out the Quadro 2000 during my tests because I discovered a bug that I need to verify.  Rather than wait for NVIDIA to send me another Quadro 2000 and have Adobe engineers verify if I found something, I pulled the card from the tests.  I had problems with the RED tests but not the other sequences.  I hope to isolate what the problem is and deliver a Quadro 2000 test as an amendment to this article in the future.  The Quadro 2000 is perhaps one of the most intriguing cards there as it’s about $500 and fits the price of cost conscious video editors.

Ok, why no love for the GeForce?

Why just Quadros?  Why not the very popular GeForce cards?  Granted, I think these cards are great and will run equal with a Quadro card for a fraction of the cost.  But as I’ve mentioned previously, the Quadro cards are engineered to perform for long times without failure and if it gets hot, it gracefully degrades performance.  It’s over-engineered and manufactured by NVIDIA whereas the GeForce cards can be made by any of a dozen different companies.  Consequently, while a GeForce may look and perform as good or even better than a Quadro in the short term, I know that all of my Premiere engineers have repeatedly praised Quadro cards.  While NVIDIA may not like the analogy, its like a Toyota and a Mercedes: They both go from point A to B, but the quality and longevity of the Mercedes shows itself again and again.

The real truth about the tests:

Sadly, there is no single, quantitative test that I could run that would make my results read out as ‘Card A did X. Card B did X+Y’.  If there were, this would be a blessedly briefer article.  In the end I used a couple of real-world tests to help come out with some useful information.

While this test was done on a PC, I think it fair to say that the gist of it translates well to Apple computers as well.  The Quadro 4000 Mac is a good card overall and provides a true GPU enabled workflow for Premiere Pro users on the Mac platform.

PPBM5 test: “Premiere Pro Benchmark CS5”

First, I ran the PPBM5.5 tests for each card on the same system.  PPBM5 aims to try and give a numerical performance for 4 key metrics.  Surprisingly, the scores of non-GPU metrics changed.  Here’s the table of the scores for each – each number represents seconds and the total time represents the sum of the four tests:

Each of the numbers represents the time it took to do the task and the Total Time reflects the overall performance.  The site discusses the testing methodology and it also is worth a read. If you’re interested in seeing how your system matches up, go to the site and look at the large list of systems that have submitted their scores.  That itself is an interesting look at how different systems perform.  More thoughts on the PPBM5 test in my conclusions below.

Real-world GPU test (aka ‘my GPU torture test’)

The second series of tests were my own and based on building sequences of different material and testing frame rates (dropped frames) at given resolutions. I used an internal testing mechanism to measure a few things such as CPU utilization.  The most important thing I was looking at though is the number of dropped frames.  Even dropped frames don’t necessarily tell the whole story because of different codecs.  I found that multiple codecs task different aspects of the system so I picked three different ones to use for these tests.

AVCHD – Highly temporal in nature makes the CPU decode a chore.  In other words, it’s a LongGOP format. However, it is fairly lightweight in disk throughput and so not necessarily taxing on the disk drives.  AVCHD is also very similar to Canon’s popular DSLR H.264 codec, so that was an added bonus.

DVCProHD – This is not a LongGOP format but a bit heavier in terms of disk space.  At about 13MB/s it is not overly difficult for modern drive RAIDs, but still considerably bigger than AVCHD.  Its frame size is actually 1280×1020…

RED – This is a worst-case scenario for a GPU test.  Huge frame sizes and large data rates make this a chore for both the CPU, RAM and the drive array.  4K clips on any system are hard.  Throwing on effects is like turning the knife and pouring salt on a wound at the same time.  Ouch.

In creating sequences of these codecs, I wanted to focus on GPU performance and how that translated in dropped frames.  So, I stacked on as many effects as I could to push the GPU in the system.  For each of the three codecs, I created two sequences: One with a single clip with as many effects as possible; Second, I took four clips as PIPs and did the same.  Each type of sequence had the exact same effects applied in each case.

The results:

Some notes about the tables:

While I strove to make everything as absolutely accurate as possible, the fact is that I had to estimate on some of the values.  For example, the CPU utilization is changing all of the time.  I looked at both my internal CPU meter and the task manager performance panel and chose what I thought the average was over the duration of the sequence.  Scrubbing quality is also a subjective quality.  ‘Great’ meant that I could scrub faster than real-time and it would keep up.  ‘Fair’ meant that it was real-time or close to it.  ‘Poor’ describes intermittent frames showing in the program view and ‘Beyond Poor’ was…well.. you get the idea…

Lastly, I ran these tests several times and averaged the numbers to give the most accurate view.  In some cases, a timeline would drop no frames one time and then a few the next.  I tried to make a quality assessment based on multiple test runs.

Quadro FX4800:

Quadro FX4800 Performance Table

Quadro 4000:

Quadro 4000 Performance Table

Quadro 5000:

Quadro 5000 Performance Table

Some conclusions:

First, lets tackle the PPBM5 test.  Even with the MPE scores coming in nearly identical between all three cards, the net result changes in between cards.  Since, literally nothing changed in the system except the card itself, this lends itself to the idea that the total system score is the thing you really want to be after.  As they discuss on their site, there is no perfect test and the graphics card can affect the overall performance.

For my tests, I had a couple of surprises, one of which is how good Premiere Pro is at decoding RED files.  I would have expected anything short of the Quadro 5000 to drop some frames when dealing with four 4K files with effects on them at half resolution.  The fact that the FX4800 dropped 10 frames total (about 1 frame every 2 seconds) is remarkable.

This brings up another point.  We have to remember that dropping a couple of frames in a given duration is no big deal in most cases and it’s doubtful that we even see it (despite people attesting to the contrary).  Obviously, we always want to strive to have zero dropped frames all of the time, but given that a lot of online video is NOT 24 or 30fps and no one complains should be an indication that there are acceptable levels of frame rates that don’t hurt our editing or viewing experience.  Additionally, throwing 20 effects simultaneously on timelines isn’t something that you’re going to do even once in a while.  While it is intellectually interesting to look at this, the big thing we should come away with is that a good GPU can really improve the day to day workflow and efficiency of an editor.

Back to RED – Turning off the GPU in all cases has a clear result in performance.  The system dropped frames up to 95% of all frames at half-res without GPU turned on.  That’s just downright amazing and provides the RED user a clear answer on whether to get a good GPU card.  Answer – get one!

AVCHD really proved to be more challenging than I would have imagined.  It shows that CPU can still be challenged.  The GPU really makes a difference here too.  You see in the case all three cases, that the GPU improved playback performance by a factor of 3.  The question here for me is why is Premiere Pro not capitalizing on more CPU power here?  It’s a question I don’t have the answer to at present, but I’m taking it to the Premiere Pro team with the hopes of getting one.  I think the big takeaway for me on AVCHD is that I will work in half resolution all of the time and get a MUCH better editing experience (scrubbing, playback, etc).  Temporal codecs like AVCHD and H.264 are getting better, but can still be challenging on a lower-end system.

DVCProHD clearly was the least taxing of the codecs with regards to GPU enabled systems.  I think it’s because it is an intraframe codec meaning every frame is a full frame(picture).  While the disk usage is higher than AVCHD, the drive system in the PC was not going to break a sweat with 90MB/s.  Yes, the FX4800 dropped a fair number of frames (65) but again, that’s 20 effects and if you bump down to half resolution, all your problems go away.

Scrubbing with GPU is also enhanced but not universally.  There are anomalies, where I thought the FX4800 had the best scrubbing of AVCHD material overall.  I don’t have a good answer as to why but I can say that the subjective difference over a number of days is probably minimal.

My final conclusion:

GPU’s are good.  CUDA enabled NVIDIA GPU’s are awesome when editing with Premiere Pro CS5 or later.  Personally, I came away surprised at how much better the system performed with the NVIDIA GPU.  The performance between the cards scaled more or less linearly and in general they confirmed what I had suspected: The Quadro 4000 is a GREAT card for the money and provides real improvement for less than $1000.  The same could be said for the Quadro FX4800 if you find them used.  If editing is your business and you’re using Adobe Premiere Pro, then investing the money for a Quadro 5000 card is a wise investment.

Of course, the one caveat to the above is that you need to have a balanced system in order to make everything perform well.  Not having enough RAM is the single biggest problem I encounter.  It would be interesting to add another 8GB of RAM to the test machine and run the test again just to see what would happen.  Similarly, If you’re using RED and have two drives striped together, you’re asking for a problem.  CPU + RAM + HDD + GPU = awesome Adobe Premiere Pro editing platform.

Balance your Production Premium Mac or PC!

Thanks for reading and I hope this article generates some conversation and comments which I welcome.  Talk amongst yourselves and I’ll reply to any good comments or questions.

Dennis – Adobe guy

So to begin, it makes sense to start with defining what MPE is.  It is NOT(!) just about hardware GPU acceleration.

The Mercury Playback Engine is three discrete components:

• 64-bit native application – as opposed to 32-bit like most applications
• 64-bit memory addressing – use more RAM
• GPU hardware acceleration for effects – ‘go faster juice’ for your system

Todd Kopriva recently did a run down on MPE, CUDA and what it means to Premiere Pro. You should give this page a peak and then come on back.  By the way, Todd is a great resource and his blog is a great page to bookmark.

Now, lets get specific on the Mac and some of the questions I’ve gotten over the last several months…

Q: Can I use Premiere Pro on a Mac?

YES. Shockingly (at least to me), there is still a portion of users that are not aware that Premiere Pro is available today on a Mac.  It is worth noting that Premiere, which started 20 years ago, originally started on a Mac.  It’s been written from the ground up three times including the current 64-bit CS5 version. With CS3 (our first re-write), we returned to the Mac in response to customer demand and delivered the first Mac version of Production Premium, which is the primary product that contains Premiere Pro today.

Q: Is Premiere Pro 64-bit ‘native’ or ‘optimized’, because I hear there is a difference?

Premiere Pro CS5 on a Mac is a 64-bit native application and all of the benefits that this implies.  Premiere Pro and After Effects in the CS4 timeframe were 64-bit ‘optimized’ meaning they could address more than 4GB of total memory, but did not do much beyond that.   As a side note, Adobe Media Encoder is also 64-bit native.

Q: Do I need an NVIDIA graphics card (GPU) to be able to use Premiere Pro CS5 on a Mac? All of the new Macs (both towers and laptops) use ATI graphics cards and I’m on a budget!

Let me be very clear about this answer – ABSOLUTELY NOT! This is a big one.  As much as I absolutely love NVIDIA technology and what it can do for our users, it is not necessary for running Premiere Pro or any other CS5 application.  Remember: the MPE is a combination of three technologies or features, but you don’t have to have all three in order to use it. If you get a Mac Pro tower and it has an ATI card, you’re still getting two-thirds of the MPE technology and it will still run rings around FCP7 or any other software NLE in terms of how many video streams and effects you can run in real-time.  Why?  It’s the 64-bit goodness and memory addressing that makes up Premiere Pro, After Effects, Photoshop and Adobe Media Encoder.

I’ve been accused of being an Adobe marketing weasel because Mac users and particularly FCP users think that Premiere Pro can’t do it’s magic without a great GPU card.  In particular, many FCP users feel that a GPU is essential in order to edit and playback challenging temporal codecs such as those found on Canon DSLR and AVCHD cameras.  Nothing could be further from the truth!

Q: Okay, if Premiere Pro on a Mac doesn’t need a GPU to actually edit AVCHD and DSLR footage smoothly, then what DOES the GPU add to my editing system?

Great question, I’m glad you asked.

The NVIDIA graphics card provides processing of effects that you apply to your clips on the timeline. Once you understand and process this, you’ll come to this conclusion: Having an NVIDIA GPU gives you the ability to balance the load between different computing components.  If effects like color correction, dissolves, etc. can be moved over to the GPU, then that means the CPUs have more headroom and ability to focus on what they’re good at – decoding video and playing it.

Q: If I have the other two aspects of MPE, what does that really give me?

The 64-bit native nature of the application allows Adobe to utilize all of your CPU cores (and virtual  ones) much more effectively than with a non-64-bit application.  How many cores do you have?  Wouldn’t you like to be able to use all of them?

The ability to address far more than 4GB of total memory with an application also means that large projects or projects that contain After Effects Projects, Photoshop documents and the like will continue to perform well.  To my knowledge, current Macs (early 2011) can address up to 32GB of total memory.  As you can see from this picture, Adobe applications recognize others that are open and can more effectively share or release to another application.  In the case of After Effects, you can also reserve a number of CPUs to remain outside of its domain again allowing other applications to work well (not just Adobe). So, if you have 16GB of memory, you can effectively use it all and recognize performance gains as well.

Many popular applications are 32-bit only and can only address up to 4GB of memory.  As we’ve quickly moved to HD, 2K, DSLR, RED and even DPX files, the ability for a professional 32-bit application to deal with this kind of media en masse has become an increasing challenge.  Take it from us, we went through it!

Q: I need to work with other FCP editors and so I need to use ProRes.  That means I can’t use Premiere Pro for those kinds of projects can I?

You can use Premiere Pro CS5 and cut ProRes all day long.

There are probably two scenarios that I can envision.  First, you start with FCP and want to move it to Premiere Pro.  Second, you start in Premiere Pro and want to hand off to FCP.

The first scenario is really easy.  If the other editor has cut ProRes with FCP, he only needs to export his timelines as XML and give you the media.  As long as your Mac (or PC) includes the ProRes decoders in QT, you don’t need anything else to edit ProRes on a Mac.  While Premiere Pro CS5 doesn’t explicitly give you ProRes presets, it is very easy to create them.

The second scenario is almost as easy.  However, you will need to be on a Mac and have FCP installed.  If you know you’re going to hand off to a FCP editor and need to use ProRes – then use Premiere Pro and Adobe Media Encoder (AME) to create another preset to transcode that footage to ProRes.  One thing that a lot of Mac users don’t know about AME is that you can create a watch folder that will automatically convert anything you drop there to your chosen codec.  This is a 64-bit, multi-threaded, background process that allows you to work on other things even as it’s churning through the media.  From there, do your edit and hand your FCP buddy an XML and pristine ProRes when you’re ready.

Q: Apple is using ATI and a similar technology to CUDA called OpenCL. Will Adobe embrace OpenCL in the future?

When CS5 was under development and I showed the Technology Sneak Peek, OpenCL wasn’t even ratified or finished.  CS5 wouldn’t have had GPU acceleration at all had it been based on OpenCL.  Even today (March, 2011), we don’t have a lot of real OpenCL applications. Adobe is continuing to evaluate OpenCL for future development but today Adobe’s GPU acceleration technology is based on CUDA from NVIDIA.

One final note for Mac users.  There is a great group of passionate Premiere Pro users that have developed a PC-based benchmarking system.  They have been open to developing a Mac-based one, but don’t have the knowledge and Macs to get it done.  Want to find out how your Mac really stacks up?  Volunteer by going to Premiere Pro Benchmark CS5 site.

UPDATE 3/14/11: Dave Helmly has published a short video that shows Premiere Pro running on the latest MacBook Pro (8.3) with an ATI/AMD card. In this video you see the remarkable performance without running a CUDA GPU.  When watching the video consider that he’s running the OS, Premiere Pro and screen capture software all on a fairly measly 8GB of RAM – pretty amazing to me.