Well Friday night was something special. The greatest minds in the Glasgow Flash community met up to discuss all things Flash related and to have a good ol’ iPhone bashing session. But wait a minute, it turns out that even Flash devs are divided by all this Flash v iPhone malarkey. Incredulously some even seem to think that Steve jobs is right to ban Flash from the iPhone! What the frak! The night nearly ended-up in fisty-cuffs I tells ya. Oh and a special guest from Finland turned up too, but let me introduce everyone first.

So who are all these brilliant individuals? Well the group was kinda split into two camps – actual Flash developers (myself, Lindsey, Jamie and Dave), and special hardcore mobile Flash Player developers (Alex, Cameron, and Dave again). You mean to say you had actual C engineers from Adobe at Glasgow Flash Meetup? Well not exactly. You see these clever dudes used to work for a company who built their own Flash player for mobile even before Macromedia released Flash Lite 1. Pretty cool huh?

To be honest, all the individuals I’ve mentioned are so good at what they do it makes me sick, but I’m gonna try and not hold it against them, and actually sing their praises a little. Let me start with Lindsey and Jamie. You see they’re both super special because unlike most people they’re pretty amazing coders and artists. What you don’t believe me? Well take a look at Lindsey’s gallery then, and this photo of an amazing painting Jamie did of a cheeseburger sitting on an Atari ST. If you take a look at the comic strips in this post you’ll realise that Jamie is a bit of a loose cannon but please don’t hold it against him.

Now onto the C engineers. Well Alex still works for the same company that produced the Flash Player and he’s also currently working on an XBox game in his spare time. I honestly think Alex must be the best C coder in the world. In fact I barely understand a word he says, mostly because I think he actually talks in C rather than English. Cameron too is a C guru who loves registering great domain names (his latest is a cracker) and dreams of stealing money from cash machines.

Finally there’s the enigma known as Dave. I could write several pages about Dave (in fact I did just that once) but this quick paragraph should suffice. Dave is an Apple fan boy and can code in almost any language. For several years he worked as a C engineer with Alex and Cameron writing a Flash player. His career took a turn for the worse when he started sniffing glue. Before Dave knew it, his life was out of control and he found himself reduced to coding in ActionScript. Even today he works full-time as an ActionScript 3 developer, but secretly dreams of the day when Steve Jobs successfully purges the planet of Flash.

It was a good night. Jamie and Lindsey had a big argument about Flash on mobile. I tried to stay impartial, as did Dave. Later on Jamie went a little crazy and started talking about killing animals or something!? The whole Flash on mobile/iPhone debate was very interesting. Jamie seemed to think that running Flash on mobile was simply impossible. Others argued that HTML5 and JavaScript on mobile wouldn’t be any faster than Flash. Personally I passionately believe that there is a place for Flash on mobile so it was good to see Lindsey fighting in my corner.

The night was made extra special by the appearance of a guest of Jamie’s who’s currently working for Nokia. I seem to remember loud music, lasers and pyrotechnics going off as he entered the room. But then again that might just be my mind playing tricks – I do like my Nokia phones. It was kinda like being in the same room as a rock star so it took me a while to pluck up the courage to talk to him. Glad I did though coz the dude had some pretty interesting things to say. For liking Nokia so much he installed a secret app on my phone. It turns any photographs you take into comic strips complete with speech bubbles. Pretty sophisticated stuff don’t you think? Sure did come in handy for this post.

Just thought I’d share a video of an iPhone app that I considered developing using Flash Professional CS5′s iPhone Packager. Of course I won’t be able to publish it to the App Store due to changes in Apple’s developer agreement but I still thought it would be interested to throw together a very rough prototype and see how it performed.

Well as you can see, even on my trusty old 1st gen iPod touch, I managed to achieve a more than acceptable frame rate using Flash. You can swipe horizontally to swap between a set of vehicles, or scroll vertically to change the skin of the current vehicle.

Okay, it’s hardly rocket science but there have been a lot of rumours going around of late that it’s almost impossible to get a decent frame rate on iPhone using Flash. That’s simply not true, but you do have to remember that you’re developing for a mobile device, which nearly always requires you to make special design and coding considerations.

Good news folks! If you use Flash and live in an alternate reality where Apple and Adobe get along just fine then you’ll probably find this tutorial quite useful. It details how to get the best performance when writing apps with Flash CS5 Professional’s Packager for iPhone.

I suppose for everybody else there’s nothing stopping you using Flash to quickly prototype your apps before committing to Objective C. What’s written in my tutorial will also benefit those wishing to target the Android platform and shows how to take advantage of a device’s GPU to increase animation performance.

Sure glad I only spent 5 minutes putting the tutorial together. It would have been quite irritating if I’d spent a few weeks on it only to find out Steve Jobs had changed the developers license agreement rendering all my hard work useless.

So anyway, here it is:

• Packager for iPhone: Animation Render Performance

Introduction

CS5 is one of the most eagerly anticipated releases of Flash Professional in years and the inclusion of the iPhone Packager finally makes it possible for Flash developers to write exciting content for iPhone using the same tools they use day in, day out.

Mobile development requires special design and coding considerations, and the iPhone is no exception. Although the Packager compiles all ActionScript byte code into native iPhone application code, an understanding of the Flash rendering pipeline on iPhone will help you maximize performance.

This article will lead you through code examples detailing how to get the best results from your iPhone when performing frame-based animation.

Requirements

To make the most of this article, you’ll need the following software and files:

• Flash Professional CS5
• Sample files: iphone_render_performance.zip

Prerequisite knowledge

Prior experience of Flash and ActionScript 3 is advised. In order to run the included sample files you will need to know how to compile and deploy Flash content on iPhone.

The iPhone Hardware

Given the iPhone’s limited hardware specification compared to a desktop PC does this mean you have to heavily optimise existing Flash content or significantly change your development strategy for new projects? Not necessarily. The engineers at Adobe have done an amazing job of ensuring that the majority of existing Flash content will compile and run on iPhone without any modification. However, while Apple has carefully managed fragmentation across their devices, there are considerable hardware differences between the various revisions that you should consider.

The following list highlights the CPU differences across models:

• iPhone 8GB – 400 MHz
• iPod touch 1st Gen – 400 MHz
• iPhone 3G 8GB – 412 MHz
• iPod touch 2nd Gen – 533 MHz
• iPhone 3GS – 600 MHz
• iPod touch 3rd Gen – 600 MHz

As you can see there is a 200 MHz difference between the first generation and third generation devices. Apple has also upgraded the GPU along the way, enabling recent revisions to push many more pixels per frame and to cache more bitmap data.

When developing your Flash content for iPhone you should consider the following when coming to a decision regarding how best to architect or how heavily you’ll need to optimize:

• The baseline iPhone you want to target.
• The graphical complexity of your application.

In all but the simplest cases, targeting the sub 533 MHz devices will likely require you to use ActionScript 3 for animation and some optimization tricks to realise your vision. The faster devices on the other hand will give you a little more freedom to utilise Flash’s timeline for animation instead of ActionScript. Of course, if you’re desperate to squeeze as much as possible from even the higher-end devices then you’ll probably have to rely more heavily on ActionScript.

It’s possible to determine the device your code is running on by examining the os property of the Capabilities class.

CPU v GPU Rendering

With the exception of Pixel Bender, apps created using the iPhone packager will contain the exact same software renderer that is present in the Flash desktop player. Software rendering is taken care of by the iPhone’s CPU but Adobe has provided an optional rendering path that allows developers to take advantage of the GPU.

Figure 1 selecting the rendering mode.

GPU acceleration allows hardware transformation and compositing of software rendered images that have been cached in the GPU’s texture memory. However to make full use of the GPU accelerated path you’ll need to write your content to specifically take advantage of the GPU, which will be explained later in this tutorial. You will also need to explicitly select the GPU rendering mode from the iPhone Settings panel as shown in figure 1.

As a general rule, use GPU rendering when writing applications such as games that perform a lot of rotation and translation of display objects. Otherwise, use CPU rendering.

Vectors versus Bitmaps

Vector shapes must be calculated by the CPU at run-time and can impact the performance of your app. Bitmap graphics on the other hand provide higher performance and should be considered ahead of vector artwork.

When utilising the GPU, Flash attempts to keep bitmaps on the GPU allowing them to be quickly composited. Images that don’t change across frames are ideal for GPU caching.

Bitmap Caching

You can increase performance by caching a display object internally on the GPU by setting its cacheAsBitmap property to true. This is useful for display objects that are translated along their x or y axes. After the initial render, the iPhone will use the internal cached version that has been stored on the GPU, giving a significant performance increase. When using GPU rendering, Bitmap display objects are always cached on the GPU. There is no need to explicitly set the cacheAsBitmap property for them.

Adobe has also provided the cacheAsBitmapMatrix property, which uses a Matrix object to render the bitmap data that is to be cached on the GPU. The GPU uses the resulting bitmap when performing translations, scaling, rotation or alpha changes, eliminating the need for the CPU to re-render the display object when its x, y, xScale, yScale, rotateX, rotateY or alpha properties change. As with cacheAsBitmap, cacheAsBitmapMatrix can lead to impressive performance gains.

Unfortunately neither cacheAsBitmap nor cacheAsBitmapMatrix are of any benefit when working with timeline based animation within a movie clip.

Timeline Animation and its Performance Bottleneck

Content that changes across frames must be re-rendered and depending on the rendering path, sent back to the GPU. Pixel traffic from the CPU to the GPU is expensive and should be minimized. Both rendering across frames and GPU traffic highlight a performance bottleneck that is present when performing animation via the timeline.

Consider a 24 frame animation. As the playhead moves across frames, each frame must be rendered. Therefore complex or concurrent animations can stress the CPU and impact performance.

In such situations you might be tempted to opt for the GPU rendering path but this would be a mistake. Not only would the CPU still be used to perform the software rendering of each frame, but there would also be an additional performance hit as each frame’s content is sent to the GPU.

Remember, traffic to the GPU is expensive and should be avoided during moments where a high and consistent frame rate is required. It makes little sense to transfer to the GPU bitmap data that is going to become redundant on the next frame.

If all 24 frames could be drawn upfront and sent to the GPU before the animation had begun then playback of that animation would be significantly smoother as each time the playhead changed position there would be reduced load on the CPU and no traffic going to the GPU.

However no such feature exists in Flash for one very good reason: ActionScript can be laced into each frame of a timeline, making it impossible to predict ahead of time how each frame will actually look when the playhead reaches it.

No matter what rendering path you opt for, timeline animation will be expensive. However it should at least be apparent that most existing Flash content that you wish to quickly port to iPhone should use CPU rendering.

This doesn’t mean you should avoid using the timeline, simply that you should be aware of the issue and the implications it may have for your application.

A conventional timeline animation

Let’s look at an example by opening animation_timeline.fla and publishing it for iPhone.

Figure 2 animation_timeline.fla running on iPod touch.

As you can see by examining the FLA there really isn’t much to it. The stage contains multiple instances of a movie clip named Wheel. Moving inside the clip reveals that its timeline spans 24 frames with each frame containing a different bitmap. Together the series of bitmaps represents an animation of a wheel turning 360 degrees.

When published, the SWF will show several wheel instances rotating. With the frame rate set at 16fps the SWF runs without a hitch on even a modest desktop. Publish and deploy it on a first generation iPod touch however and the result isn’t what most Flash developers would expect.

Whereas the desktop version runs at a rock solid 16fps, the very same content on the iPod touch utilising CPU rendering struggles to reach 12fps.

Although bitmaps have been used for every frame of animation within the wheel’s timeline, which does give a slight benefit, the device is being hammered thanks to the timeline performance bottleneck outlined previously. Performance is being further degraded due to the number of instances.

Republishing with the GPU rendering path set doesn’t help matters. This time the app only manages approximately 7fps on a first generation iPod touch.

As the playhead moves to the next frame within each instance, that instance’s frame must be sent to the GPU before being composited. This process is done for every single instance even though the graphical content of each instance’s frame is identical to all the other instances on the stage.

Thankfully, where performance is critical, bitmap animation on the timeline can be replaced with an ActionScript alternative. Let’s now look at how to do that.

Bitmap Animation using ActionScript

Since all 24 frames of the spinning wheel animation are known upfront, it would help performance significantly if those frames were pre-cached on the iPhone’s GPU.

Whereas a movie clip only offers caching for its current frame, you can use ActionScript 3 to manually cache all the bitmaps required for an animation before playback begins.

Manual bitmap caching

Any instances of the BitmapData class are automatically cached on the GPU making it possible to significantly improve performance by storing the entire spinning wheel animation within a series of BitmapData objects. This is known as manual bitmap caching. There are several ways to do this but perhaps the easiest is to grab each animation frame from a bitmap sheet.

Bitmap sheets

A bitmap sheet (also known as a sprite sheet) is a single bitmap image that holds all the individual frames of an animation. The sheet is separated into a grid with each frame of the animation occupying identically sized slots. This can be seen in figure 3 where a bitmap sheet has been constructed that holds the frames of the spinning wheel animation in an 8×3 grid. Each grid slot is 84 pixels wide and 83 pixels high.

Figure 3 A bitmap sheet containing all 24 frames of the wheel animation.

You can find the bitmap sheet from figure 3 by loading animation_bitmaps.fla and looking for wheels.png in the library. Right-clicking on wheels.png and selecting properties will reveal that the bitmap has been exported for ActionScript and that it has been given a class name of Wheels. This makes it possible to create an instance of the bitmap sheet in ActionScript and is shown below:

var bitmapSheet :Wheels = new Wheels();

The next step is to copy each frame from the bitmap sheet into a BitmapData object of its own. Each of these individual bitmaps will be automatically cached on the iPhone’s GPU. The code snippet below outlines this:

// The grid dimensions.
var gridW :int = 8;
var gridH :int = 3;

// Each frame’s dimensions.
var frameW :int = 84;
var frameH :int = 83;

// The actual bitmap sheet.
var bitmapSheet :Wheels = new Wheels();

// Store each frame’s bitmap data here.
var bitmaps :Vector.<BitmapData> = new <BitmapData>[];

// Chop the bitmap sheet into individual frames.
for( var y :int = 0; y < gridH; y++ )
{
    for( var x :int = 0; x < gridW; x++ )
    {
        // Create a BitmapData object for this frame.
        var bd :BitmapData = new BitmapData( frameW, frameH, true );

        // The rectangular area to be copied from the bitmap sheet.
        var srcRect :Rectangle = new Rectangle(
            ( x * frameW ), ( y * frameH ), frameW, frameH );

        // Copy the frame’s pixels from the bitmap sheet.
        bd.copyPixels( bitmapSheet, srcRect, new Point( 0, 0 ) );

        // Store the bitmap data in the array for use later.
        bitmaps.push( bd );
    }
}

// Manually remove bitmap data that is no longer required.
bitmapSheet.dispose();
bitmapSheet = null;

Let’s examine the code above in a little more detail.

The bitmap sheet is instantiated and an empty vector array that will store the bitmaps for each of the animation’s frames is initialised. A nested for-loop is then used to traverse the bitmap sheet, copying each of the frames from the sheet into their own BitmapData objects. Each of these BitmapData objects is added to the vector array for use later. Think of the vector array as a graphics library.

The final step is to manually dispose of the bitmap sheet’s bitmap data and release it for garbage collection. The bitmap sheet is no longer required since the frames of animation are now stored individually within the vector array and are in a format that we can more easily use.

Rendering

Now that we have a vector array of bitmap data it’s a fairly trivial task to render that content on screen. Simply create a Bitmap object; point its bitmapData property to one of the bitmap data objects stored in the vector array; and finally add the Bitmap object to the display list.

var wheel :Bitmap = new Bitmap();
wheel.bitmapData = bitmaps[ 0 ];
addChild( wheel );

Take a look at the code example above. It creates a bitmap and points it to the first frame of animation stored in the bitmaps vector array. It doesn’t take much more code to create a screen full of wheels:

var wheels :Vector.<Bitmap> = new <Bitmap>[];
var startX :int = -8;
var startY :int = -9;
var numWheelsX :int = 4;
var numWheelsY :int = 6;

for( var y :int = 0; y < numWheelsY; y++ )
{
    for( var x :int = 0; x < numWheelsX; x++ )
    {
        var wheel :Bitmap = new Bitmap( bitmaps[ 0 ] );
        wheel.x = startX + ( wheel.width * x );
        wheel.y = startY + ( wheel.height * y );

        wheels.push( wheel );

        addChild( wheel );
    }
}

All Bitmap objects are referencing the same bitmap data. This approach saves CPU resources and makes applications run faster, and it’s this approach that we’ll use to implement animation.

Before proceeding you might want to examine the ActionScript that’s sitting on the timeline of animation_bitmaps.fla. Take a look at the createWheelBitmaps() and addWheels() functions, both of which should look familiar. The first function creates the library of bitmaps from the bitmap sheet, while the second adds a series of wheels to the display list.

Animating

The final step is to actually perform some animation. If you remember, we can change a Bitmap object’s bitmapData property to point to any of the bitmaps held in the bitmaps vector array. Creating an animation should therefore be a simple case of changing that bitmapData reference on each frame update.

This can be achieved by using a variable to store the current animation frame that is to be rendered then incrementing that variable’s value every time the screen is redrawn. The variable itself is used to access the correct bitmap data stored in the bitmaps vector array. We’ll use an ENTER_FRAME event to keep track of the current animation frame and to update the bitmap data reference used by each wheel:

var frame :int = 0;
var maxFrames :int = bitmaps.length;
addEventListener( Event.ENTER_FRAME, update );

function update( e :Event ) :void
{
    // Loops the animation back to the first frame.
    if( ++frame == maxFrames )
    {
        frame = 0;
    }

    // Update each wheel’s bitmap data reference.
    for( var i :int = 0; i < wheels.length; i++ )
    {
        wheels[ i ].bitmapData = bitmaps[ frame ];
    }
}

And that is more or less it. Publish animation_bitmaps.fla and install it on an iPhone or iPod touch. The performance increase over the timeline version should be significant, especially on earlier models. The device should now be happily churning out multiple wheel animations at a super smooth 16fps.

Where to Go From Here

At this point you now have the knowledge necessary to create your own frame-based animations using bitmap sheets and ActionScript 3. Spend some time understanding the example FLAs and consider how you might apply these techniques to your own iPhone apps. If performance is a priority then what you’ve learned from this tutorial should put you on the right track.

To research in more depth you may want to check the following resources:

• Packaging applications for the iPhone
• Optimizing Performance for the Flash Platform
• How to add a Provisioning Profile to your device through iTunes
• Create a Provisioning Profile in the iPhone Provisioning Portal
• How to create an App ID in the iPhone Provisioning Portal

Be sure to visit both the Flash Developer Center and the Mobile and Devices Developer Center, where you will find many helpful articles and sample projects over the coming months.

Acknowledgements

A huge thank you to Jay Armstrong and Jeff Swartz from Adobe for their valuable feedback.

Several months back I toyed with the idea of porting my audio cassette music player to iPhone using Flash Professional CS5′s iPhone Packager, and as you can see from the video below I’ve done just that. Although Apple are blocking content written in Flash from the App Store I still thought it would be a worth while exercise and would probably be fairly straight forward to do.

Well it turned out to be anything but straight forward. The wisest Jedi of them all once said “You must unlearn what you have learned” and I swear the more that quote floats around my head the more convinced I am that Master Yoda had actually tried porting projects to Apple’s shiny little wonder.

I naively thought that my Flash Lite experience would ensure I was up and running in no time at all. However writing for iPhone brings about a whole new set of interesting problems for the developer to solve. I’ve spent the last two days taking apart my original audio cassette demo and reconstructing it so as best to take advantage of the iPhone’s hardware.

Its GPU loves texture maps, so converting as much of your content as possible to bitmaps and caching them on the GPU seems to get some pretty good results. As is the case with all mobile development, finding the best approach may require you to make compromises and small sacrifices in order to get the best performance.

I think the end result was worth the effort and I now understand considerably more about development for iPhone.

So Apple has just changed its developer agreement for those wishing to access the iPhone Developer Program and it could spell the end for Adobe’s iPhone Packager before it has even been released. Here’s the worrying snippet:

3.3.1 Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

With Apple and Adobe not exactly being the best of friends at the moment it is very possible that this is another blatant attempt by Apple to make Flash irrelevant. Unfortunately these new terms and conditions would also have serious implications for the excellent Unity development platform too.

Fingers crossed it’s just a big misunderstanding.

Pop over to NoRestForTheWicked and you’ll find a set of nifty ActionScript 3 iPhone components by Nick Jonas. He’s decided to start small but what’s there looks great and should come in handy for anyone wishing to prototype potential iPhone UIs in Flash or for those simply wanting to get a head start with their Flash CS5 Professional iPhone projects.

Currently the SWC comes in at a tidy 72 Kb and the components themselves can be added to your project using only a few lines of code. In fact, from his site you can preview the components and even click on a link next to each to generate the code required to set it up; it really couldn’t be simpler. You won’t be able to use any of the iPhone’s native UI components from within Flash so Nick’s components will no doubt be a lifesaver for many.

Most of you will have seen the ‘What is a browser?’ video where Google take to the streets of New York to see how many passers-by actually know what a web browser is. Even though they all seemed to surf the web regularly, only 8% of them knew what a web browser actually was. Well the video sparked an interesting conversation in the office between myself and a friend.

One question in particular came out from the conversation: Do people actually know what Flash is?

It would be really interesting to see a similar video asking this question to random folk. I suspect the results would be much worse than the miserable 8% from the Google video. Many of my friends and relatives know that I’m a Flash developer, but whenever I try explaining to them what Flash is their eyes glaze over, with some even seeming to slip into a semi-vegetative state.

So is this really a problem? Well it shouldn’t be. Your average user should just be able to enjoy browsing the web without caring about what technologies are being used to render the content. Unfortunately with the increasing penetration of the iPhone and the imminent release of the iPad looming, Adobe could be in trouble.

Most iPhone users, and I’d imagine this will be the case for those with iPads too, will probably get by with a stunted web experience. iPhone users have after all been making do with a Flash-less web for some time now and I suspect that most of them simply shrug their shoulders and move on when they stumble upon a Flash-based site.

If people don’t know what Flash is or even care then how are phone and tablet manufacturers going to convince them to purchase their Flash-enabled devices over Apple’s products? Other than geeks, who’s going to buy an Android phone because it says on the box that it supports Flash? Claiming that Flash-enabled devices will allow users to browser ‘the complete web’ isn’t going to do much good either, since Joe Public already mistakenly believe that the iPhone and iPad can already do this.

For me it’s difficult to see round this problem. Perhaps the trick is not to mention Flash at all, instead focusing marketing on popular Flash sites that will only be available to these Flash 10.1 enabled devices. Or maybe Adobe are just hoping that the handsets they’re targeting will eventually grab a large enough market share giving Flash the mobile penetration it needs to be relevant.

My main concern however is the time this will take. The longer iPhone and iPad dominate in their spaces, the more websites will drop Flash in order to cater for the ever increasing number of users who have Apple devices.

I’m extremely excited by the prospect of seeing Flash on mobile but there is an ever decreasing window of opportunity to get Flash 10.1 out there. Let’s hope that Adobe and its many partners manage to get Flash on enough devices before a tipping point is reached and websites stop using it.

Yup, it’s hard to believe but Adobe AIR is now two years old and it’s incredible just how far it has come in that short space of time. Given the growing debate over Flash’s future within the browser it’s often easy to forget that the Flash platform allows developers to create rich Internet applications that can be deployed to the desktop across multiple operating systems.

AIR now powers a variety of applications used by enterprises and everyday consumers in applications for social networking, gaming, eLearning, subscription-based services, video, productivity tools, collaboration services, and so much more. And thanks to its support for iPhone, which is due to be delivered as part of the Flash Professional CS5 release, 2010 will be a very exciting year indeed for AIR.

If you are in any doubt at all regarding the power of Adobe AIR, take a look at the video above, which demonstrates a collaboration between Adobe and Wired magazine to produce a new digital magazine concept running on an Android-based tablet. It’s quite stunning.

Yeah, okay, so this post has nothing to do with Flash whatsoever. But hey I’ve got plenty of love for other technologies – even the ones that don’t support Flash.

So if you happen to have an iPhone along with the other zillion people out there then I thoroughly recommend you check out WeeWorld’s WeeMee Avatar Creator app that we released last week. It has climbed to the #4 spot in the Social Networking category of both the UK and US App Stores and it’s pretty ace.

Download it and give it a go at: http://itunes.com/apps/WeeMeeAvatarCreator