Introduction
CS5 is one of the most eagerly anticipated releases of Flash Professional in years and the inclusion of the iPhone Packager finally makes it possible for Flash developers to write exciting content for iPhone using the same tools they use day in, day out.
Mobile development requires special design and coding considerations, and the iPhone is no exception. Although the Packager compiles all ActionScript byte code into native iPhone application code, an understanding of the Flash rendering pipeline on iPhone will help you maximize performance.
This article will lead you through code examples detailing how to get the best results from your iPhone when performing frame-based animation.
Requirements
To make the most of this article, you’ll need the following software and files:
- Flash Professional CS5
- Sample files: iphone_render_performance.zip
Prerequisite knowledge
Prior experience of Flash and ActionScript 3 is advised. In order to run the included sample files you will need to know how to compile and deploy Flash content on iPhone.
The iPhone Hardware
Given the iPhone’s limited hardware specification compared to a desktop PC does this mean you have to heavily optimise existing Flash content or significantly change your development strategy for new projects? Not necessarily. The engineers at Adobe have done an amazing job of ensuring that the majority of existing Flash content will compile and run on iPhone without any modification. However, while Apple has carefully managed fragmentation across their devices, there are considerable hardware differences between the various revisions that you should consider.
The following list highlights the CPU differences across models:
- iPhone 8GB – 400 MHz
- iPod touch 1st Gen – 400 MHz
- iPhone 3G 8GB – 412 MHz
- iPod touch 2nd Gen – 533 MHz
- iPhone 3GS – 600 MHz
- iPod touch 3rd Gen – 600 MHz
As you can see there is a 200 MHz difference between the first generation and third generation devices. Apple has also upgraded the GPU along the way, enabling recent revisions to push many more pixels per frame and to cache more bitmap data.
When developing your Flash content for iPhone you should consider the following when coming to a decision regarding how best to architect or how heavily you’ll need to optimize:
- The baseline iPhone you want to target.
- The graphical complexity of your application.
In all but the simplest cases, targeting the sub 533 MHz devices will likely require you to use ActionScript 3 for animation and some optimization tricks to realise your vision. The faster devices on the other hand will give you a little more freedom to utilise Flash’s timeline for animation instead of ActionScript. Of course, if you’re desperate to squeeze as much as possible from even the higher-end devices then you’ll probably have to rely more heavily on ActionScript.
It’s possible to determine the device your code is running on by examining the os
property of the Capabilities
class.
CPU v GPU Rendering
With the exception of Pixel Bender, apps created using the iPhone packager will contain the exact same software renderer that is present in the Flash desktop player. Software rendering is taken care of by the iPhone’s CPU but Adobe has provided an optional rendering path that allows developers to take advantage of the GPU.
GPU acceleration allows hardware transformation and compositing of software rendered images that have been cached in the GPU’s texture memory. However to make full use of the GPU accelerated path you’ll need to write your content to specifically take advantage of the GPU, which will be explained later in this tutorial. You will also need to explicitly select the GPU rendering mode from the iPhone Settings panel as shown in figure 1.
As a general rule, use GPU rendering when writing applications such as games that perform a lot of rotation and translation of display objects. Otherwise, use CPU rendering.
Vectors versus Bitmaps
Vector shapes must be calculated by the CPU at run-time and can impact the performance of your app. Bitmap graphics on the other hand provide higher performance and should be considered ahead of vector artwork.
When utilising the GPU, Flash attempts to keep bitmaps on the GPU allowing them to be quickly composited. Images that don’t change across frames are ideal for GPU caching.
Bitmap Caching
You can increase performance by caching a display object internally on the GPU by setting its cacheAsBitmap
property to true
. This is useful for display objects that are translated along their x or y axes. After the initial render, the iPhone will use the internal cached version that has been stored on the GPU, giving a significant performance increase. When using GPU rendering, Bitmap
display objects are always cached on the GPU. There is no need to explicitly set the cacheAsBitmap
property for them.
Adobe has also provided the cacheAsBitmapMatrix
property, which uses a Matrix
object to render the bitmap data that is to be cached on the GPU. The GPU uses the resulting bitmap when performing translations, scaling, rotation or alpha changes, eliminating the need for the CPU to re-render the display object when its x
, y
, xScale
, yScale
, rotateX
, rotateY
or alpha
properties change. As with cacheAsBitmap
, cacheAsBitmapMatrix
can lead to impressive performance gains.
Unfortunately neither cacheAsBitmap
nor cacheAsBitmapMatrix
are of any benefit when working with timeline based animation within a movie clip.
Timeline Animation and its Performance Bottleneck
Content that changes across frames must be re-rendered and depending on the rendering path, sent back to the GPU. Pixel traffic from the CPU to the GPU is expensive and should be minimized. Both rendering across frames and GPU traffic highlight a performance bottleneck that is present when performing animation via the timeline.
Consider a 24 frame animation. As the playhead moves across frames, each frame must be rendered. Therefore complex or concurrent animations can stress the CPU and impact performance.
In such situations you might be tempted to opt for the GPU rendering path but this would be a mistake. Not only would the CPU still be used to perform the software rendering of each frame, but there would also be an additional performance hit as each frame’s content is sent to the GPU.
Remember, traffic to the GPU is expensive and should be avoided during moments where a high and consistent frame rate is required. It makes little sense to transfer to the GPU bitmap data that is going to become redundant on the next frame.
If all 24 frames could be drawn upfront and sent to the GPU before the animation had begun then playback of that animation would be significantly smoother as each time the playhead changed position there would be reduced load on the CPU and no traffic going to the GPU.
However no such feature exists in Flash for one very good reason: ActionScript can be laced into each frame of a timeline, making it impossible to predict ahead of time how each frame will actually look when the playhead reaches it.
No matter what rendering path you opt for, timeline animation will be expensive. However it should at least be apparent that most existing Flash content that you wish to quickly port to iPhone should use CPU rendering.
This doesn’t mean you should avoid using the timeline, simply that you should be aware of the issue and the implications it may have for your application.
A conventional timeline animation
Let’s look at an example by opening animation_timeline.fla
and publishing it for iPhone.
As you can see by examining the FLA there really isn’t much to it. The stage contains multiple instances of a movie clip named Wheel
. Moving inside the clip reveals that its timeline spans 24 frames with each frame containing a different bitmap. Together the series of bitmaps represents an animation of a wheel turning 360 degrees.
When published, the SWF will show several wheel instances rotating. With the frame rate set at 16fps the SWF runs without a hitch on even a modest desktop. Publish and deploy it on a first generation iPod touch however and the result isn’t what most Flash developers would expect.
Whereas the desktop version runs at a rock solid 16fps, the very same content on the iPod touch utilising CPU rendering struggles to reach 12fps.
Although bitmaps have been used for every frame of animation within the wheel’s timeline, which does give a slight benefit, the device is being hammered thanks to the timeline performance bottleneck outlined previously. Performance is being further degraded due to the number of instances.
Republishing with the GPU rendering path set doesn’t help matters. This time the app only manages approximately 7fps on a first generation iPod touch.
As the playhead moves to the next frame within each instance, that instance’s frame must be sent to the GPU before being composited. This process is done for every single instance even though the graphical content of each instance’s frame is identical to all the other instances on the stage.
Thankfully, where performance is critical, bitmap animation on the timeline can be replaced with an ActionScript alternative. Let’s now look at how to do that.
Bitmap Animation using ActionScript
Since all 24 frames of the spinning wheel animation are known upfront, it would help performance significantly if those frames were pre-cached on the iPhone’s GPU.
Whereas a movie clip only offers caching for its current frame, you can use ActionScript 3 to manually cache all the bitmaps required for an animation before playback begins.
Manual bitmap caching
Any instances of the BitmapData
class are automatically cached on the GPU making it possible to significantly improve performance by storing the entire spinning wheel animation within a series of BitmapData
objects. This is known as manual bitmap caching. There are several ways to do this but perhaps the easiest is to grab each animation frame from a bitmap sheet.
Bitmap sheets
A bitmap sheet (also known as a sprite sheet) is a single bitmap image that holds all the individual frames of an animation. The sheet is separated into a grid with each frame of the animation occupying identically sized slots. This can be seen in figure 3 where a bitmap sheet has been constructed that holds the frames of the spinning wheel animation in an 8×3 grid. Each grid slot is 84 pixels wide and 83 pixels high.
You can find the bitmap sheet from figure 3 by loading animation_bitmaps.fla
and looking for wheels.png
in the library. Right-clicking on wheels.png
and selecting properties
will reveal that the bitmap has been exported for ActionScript and that it has been given a class name of Wheels
. This makes it possible to create an instance of the bitmap sheet in ActionScript and is shown below:
var bitmapSheet :Wheels = new Wheels();
The next step is to copy each frame from the bitmap sheet into a BitmapData
object of its own. Each of these individual bitmaps will be automatically cached on the iPhone’s GPU. The code snippet below outlines this:
// The grid dimensions. var gridW :int = 8; var gridH :int = 3; // Each frame’s dimensions. var frameW :int = 84; var frameH :int = 83; // The actual bitmap sheet. var bitmapSheet :Wheels = new Wheels(); // Store each frame’s bitmap data here. var bitmaps :Vector.<BitmapData> = new <BitmapData>[]; // Chop the bitmap sheet into individual frames. for( var y :int = 0; y < gridH; y++ ) { for( var x :int = 0; x < gridW; x++ ) { // Create a BitmapData object for this frame. var bd :BitmapData = new BitmapData( frameW, frameH, true ); // The rectangular area to be copied from the bitmap sheet. var srcRect :Rectangle = new Rectangle( ( x * frameW ), ( y * frameH ), frameW, frameH ); // Copy the frame’s pixels from the bitmap sheet. bd.copyPixels( bitmapSheet, srcRect, new Point( 0, 0 ) ); // Store the bitmap data in the array for use later. bitmaps.push( bd ); } } // Manually remove bitmap data that is no longer required. bitmapSheet.dispose(); bitmapSheet = null;
Let’s examine the code above in a little more detail.
The bitmap sheet is instantiated and an empty vector array that will store the bitmaps for each of the animation’s frames is initialised. A nested for-loop is then used to traverse the bitmap sheet, copying each of the frames from the sheet into their own BitmapData
objects. Each of these BitmapData
objects is added to the vector array for use later. Think of the vector array as a graphics library.
The final step is to manually dispose of the bitmap sheet’s bitmap data and release it for garbage collection. The bitmap sheet is no longer required since the frames of animation are now stored individually within the vector array and are in a format that we can more easily use.
Rendering
Now that we have a vector array of bitmap data it’s a fairly trivial task to render that content on screen. Simply create a Bitmap
object; point its bitmapData
property to one of the bitmap data objects stored in the vector array; and finally add the Bitmap
object to the display list.
var wheel :Bitmap = new Bitmap(); wheel.bitmapData = bitmaps[ 0 ]; addChild( wheel );
Take a look at the code example above. It creates a bitmap and points it to the first frame of animation stored in the bitmaps
vector array. It doesn’t take much more code to create a screen full of wheels:
var wheels :Vector.<Bitmap> = new <Bitmap>[]; var startX :int = -8; var startY :int = -9; var numWheelsX :int = 4; var numWheelsY :int = 6; for( var y :int = 0; y < numWheelsY; y++ ) { for( var x :int = 0; x < numWheelsX; x++ ) { var wheel :Bitmap = new Bitmap( bitmaps[ 0 ] ); wheel.x = startX + ( wheel.width * x ); wheel.y = startY + ( wheel.height * y ); wheels.push( wheel ); addChild( wheel ); } }
All Bitmap
objects are referencing the same bitmap data. This approach saves CPU resources and makes applications run faster, and it’s this approach that we’ll use to implement animation.
Before proceeding you might want to examine the ActionScript that’s sitting on the timeline of animation_bitmaps.fla
. Take a look at the createWheelBitmaps()
and addWheels()
functions, both of which should look familiar. The first function creates the library of bitmaps from the bitmap sheet, while the second adds a series of wheels to the display list.
Animating
The final step is to actually perform some animation. If you remember, we can change a Bitmap
object’s bitmapData
property to point to any of the bitmaps held in the bitmaps
vector array. Creating an animation should therefore be a simple case of changing that bitmapData
reference on each frame update.
This can be achieved by using a variable to store the current animation frame that is to be rendered then incrementing that variable’s value every time the screen is redrawn. The variable itself is used to access the correct bitmap data stored in the bitmaps
vector array. We’ll use an ENTER_FRAME
event to keep track of the current animation frame and to update the bitmap data reference used by each wheel:
var frame :int = 0; var maxFrames :int = bitmaps.length; addEventListener( Event.ENTER_FRAME, update ); function update( e :Event ) :void { // Loops the animation back to the first frame. if( ++frame == maxFrames ) { frame = 0; } // Update each wheel’s bitmap data reference. for( var i :int = 0; i < wheels.length; i++ ) { wheels[ i ].bitmapData = bitmaps[ frame ]; } }
And that is more or less it. Publish animation_bitmaps.fla
and install it on an iPhone or iPod touch. The performance increase over the timeline version should be significant, especially on earlier models. The device should now be happily churning out multiple wheel animations at a super smooth 16fps.
Where to Go From Here
At this point you now have the knowledge necessary to create your own frame-based animations using bitmap sheets and ActionScript 3. Spend some time understanding the example FLAs and consider how you might apply these techniques to your own iPhone apps. If performance is a priority then what you’ve learned from this tutorial should put you on the right track.
To research in more depth you may want to check the following resources:
- Packaging applications for the iPhone
- Optimizing Performance for the Flash Platform
- How to add a Provisioning Profile to your device through iTunes
- Create a Provisioning Profile in the iPhone Provisioning Portal
- How to create an App ID in the iPhone Provisioning Portal
Be sure to visit both the Flash Developer Center and the Mobile and Devices Developer Center, where you will find many helpful articles and sample projects over the coming months.
Acknowledgements
A huge thank you to Jay Armstrong and Jeff Swartz from Adobe for their valuable feedback.
Great tutorial and some surprises here too. Thanks for taking the time – there’s precious little out there at the moment on iPhone optimisation of Flash created apps.
I am currently trying to create a game that requires the player to aim a jet of particles and I’m having some real trouble getting the performance.
It’s pretty slow and not very playable. I am using the caurina tweener but I’ve tried EnterFrame animation techniques I have tried all sorts of different combinations of bitmap, vectors, CPU, GPU, Auto, cacheAsBitmap, cacheAsBitmapMatrix etc etc and I seem to have hit a brick wall.
I’m sure I am missing something and I can’t work out how to apply your methods to a stream of particles.
Any pointers?
Thanks again!
This tutorial is the practical most one i ever saw.
Great tips, opens up the door for trying new stuff.
Thanks a lot
BTW: Do you think that something in “angry birds league” could have been made on flash??
>>Do you think that something in “angry birds league” could have been made on flash??
Really depends on your target hardware. I’m not entirely convinced that the Packager for iPhone can create Flash content that performs well enough to do an Angry Birds clone. AIR for Android however is much more capable and we’re already seeing some encouraging content appearing on the Android Market. But I still think it would be big ask.
Flash is always going to struggle to compete with native apps when performance is critical, and that may help you decide which technology to use when writing your mobile app.
In other words, what technology you end up using really depends on the type of app you want to create. However, I personally think Flash is suited to many many types of games and apps, and I’m looking forward to seeing what appears over the coming months.
Hopefully some Flash mobile games guru out there will prove me wrong and create an amazing Angry Birds clone. Fingers crossed 🙂
Amazing! This has bumped my framerate from a measly 7/8fps up to a tasty 31!!
I love this method. I’m trying to do the same thing but with a sequence of images (not one image that is dissected to small cuts). It’s 60 frames that are called Class100-Class160. How could I create a loop to attach each Class to the frames and to the array? Thanks!!
Hi,
we just posted a blog entry about the Optimizations we made in Hungry Choo Choo: http://www.mushroom-games.com/
Tomer Reiss.
http://www.MushRoom-games.com
Hi,
we just posted a blog entry about the Optimizations we made in Hungry Choo Choo: http://blog.kwazi.co.il/?p=16
Tomer Reiss.
http://www.MushRoom-games.com
Hi Tomer,
Congratulations on your Hungry Choo Choo game. Looks really great! Thanks for linking to the post too – some really great suggestions in there.
Thanks Christopher.
Hello,
I’ve created a mobile app with flash that incorporates a pixel bender filter. Works great on my HTC, tried it on my wife’s iPhone 3 and it sucks…. I’m not sure if the pixel bender filter is working or not it is so slow. So, my question is, can you incorporate pixel bender into apps that will be packaged for the iphone?
🙂
Hi Josh. Pixel Bender isn’t currently supported on AIR for iOS. Fingers crossed that it’ll make an appearance in a future release.