Flash and GPU acceleration: NVIDIA and Adobe comment
With the Flash Player becoming available on an increasingly broad profile of devices from car stereo systems to smartphones, Adobe’s role has become ever greater. No longer do they just need to have their player run on multiple operating systems, but now hardware platforms x86, ARM, MIPS etc, are added to the mix of desktop and embedded operating systems.
While arguments continue over whether Flash is “bloated”, or a “CPU hog”, the fact is, it is a rich media runtime, and any kind of rich media is bound to be CPU-intensive whether it uses Flash or not. Fact is though, that there is already a large amount of rich multimedia content created and readily available in Flash, so bringing it to as many platforms as possible is an obvious choice.
Compared to desktop computers though, embedded devices usually have much more constrained resources in terms of computer power and battery life. As such for these devices, it is important to take full benefit of the hardware available, which in the case of Flash means utilizing the GPU which is now present in an many higher end mobile and other embedded devices.
NVIDIA’s Tegra is one of the hardware platforms on which GPU acceleration for Flash content is possible. In light of this we had a few questions from Adobe and NVIDIA, and they obliged us by answering them. Neil Trevett, Vice President of Mobile Content at NVIDIA answered our questions on their behalf, and Anantharaman P.N, Director Engineering at Adobe Systems on the behalf of Adobe.
Q) How will this work going ahead? As Adobe releases Flash Player 10.2 or whatever their next version will be, and as you release newer versions of your platform.
Nvidia: NVIDIA has worked side-by-side with Adobe to offload Flash content processing to graphics processors for smooth performance and extended battery life. We’re looking forward to continuing our work with Adobe to provide consumers with rich Internet experiences across any device, anywhere with future versions of Flash Player. Initial functionality is in Flash Player 10.1 and will continue to improve with future versions of Flash.
Q) Flash Player 10.1 was supposed to bring hardware acceleration for vector graphics as well as video, but the hardware acceleration was turned off in betas.
a) Is it hardware accelerated now?
Nvidia: Both video and graphics are accelerated on NVIDIA Tegra. Tegra will be the one of the first platforms with complete Flash Player acceleration.
b) How much of Flash is accelerated?
Nvidia: On Tegra, 100% is accelerated – both graphics and video.
Q) Flash Player 10.1 feature notes specifically state that “A GPU-based vector renderer replaces the software renderer on smartphones and other mobile devices.”
a) Why is this not available for PCs? What is the roadblock here?
Nvidia: Tegra is the initial platform on which complete GPU offload has been developed.
Adobe: We are letting mobile take the lead given the performance and power characteristics of devices and then bring the technology to the desktop. The goal for us is a gradual rollout as most sites were not built to leverage a GPU. This allows us to help sites optimize for hardware drawing using smartphones first. Finally, we already offer some GPU support today on the Mac via Core Animation and in full screen mode on Win and Mac.
b) What about Tegra-based devices running Linux? Will they be able to the GPU based vector renderer?
Nvidia: Yes. All Tegra-supported platforms will use GPU acceleration for graphics and video.
Q) What about Flash Lite 4? Is it receiving as much optimization as Flash Player?
Adobe: Flash Lite 4 is targeted at a different market and set of devices than Flash Player 10.1.
Flash Lite 4 is a highly optimized runtime for low powered, mass market mobile phones and consumer electronic devices (excluding higher powered, IP enabled digital home devices). Flash Lite 4 is supports SWF 10 and ActionScript 3. Flash Lite has been implemented by OEMs on over 1.3 billion devices as a plug-in to the mobile browser and/or as a standalone player for
applications.
There continues to be demand from many OEMs for Flash Lite to provide a more complete web browsing experience on lower powered mass market devices, while higher power devices will support Flash Player 10.1 (and AIR in the future). We will continue to evaluate the market conditions to determine the best path for our OEM and developer partners.
Q) What are the benefits to running Flash content on the Tegra platform?
Nvidia: NVIDIA, the world leader in graphics, knows how to accelerate graphics like no one else on the planet. It’s how we do Flash that makes Tegra superior. NVIDIA Tegra is the only mobile processor accelerating graphics and video for Flash using the GPU (graphics processor). Competitors use the CPU – sometimes for video and always for graphics, which is slow and power hungry, which leads to performance problems and short battery life. NVIDIA engineers worked side-by-side with Adobe engineers to offload Flash processing to the GPU. Tegra accelerates Flash like no one else, for unbeatable multimedia performance.
Q) Linux suffers from the lack of a standard hardware video acceleration interface. You have created the open source VDPAU. However it is not supported on Intel and ATI hardware.
a) Is anything being done to make VDPAU the standard?
Nvidia: We designed VDPAU with the hope that it would be adopted by other hardware vendors, and become a standard. The VDPAU API allows a vendor backend to be selected at run time. So far, S3 has provided a VDPAU implementation. We still hope that Intel and ATI will eventually provide VDPAU implementations.
b) If a new standard were adopted other than VDPAU, will Nvidia support it?
Nvidia: NVIDIA has a long history of embracing and supporting standards since a wider choice of languages improve the number and scope of applications that can exploit our products. NVIDIA supports standards that allow us to innovate at a rapid rate (i.e., CUDA), as well as industry standards (ie. OpenCL, DirectCompute). As a result, only NVIDIA gives developers the widest breadth of programming options. Typically with open standards, NVIDIA would support developers and would create a developer’s forum as well as a Jump Start guide and make it available online for developers.
Q) How does using VDPAU differ from using CUDA to accelerate the same video decoding functions?
Nvidia: VDPAU uses the dedicated video hardware within the GPU to perform the video decode, while using CUDA to accelerate video decode executes the decode algorithms on the general processing cores of the GPU.
a) Is it possible to use CUDA for this purpose?
Nvidia: Yes, it is possible to accelerate video decoding using CUDA. The special-purpose video decode hardware in the GPU will generally be faster, because it is tailored for the specific video decode operations needed by the codecs currently supported (MPEG2, VC-1, H.264, MPEG4-Part2, DivX). In general, special-purpose hardware will be faster for the special operations it was designed to perform.
General-purpose processors (e.g., the SMs which execute CUDA kernels) will be not quite as fast as special-purpose hardware, but they have the advantage of flexibility to perform operations beyond what the special-purpose hardware was designed to perform.
b) If so, could it be possible to allow for a software-based solution which accelerated new codecs such as VP8 using CUDA?
Nvidia: Yes.