Sidharth’s Posterous

An Adobe Flash developer on why the iPad can%u2019t use Flash

February 20th, 2010

Daniel Eran Dilger

Morgan Adams, an interactive content developer who knows a lot about building Flash, wrote in with an interesting perspective on Flash and the iPad. The remainder of this piece is his comments on the subject.

Inside Apple%u2019s iPad: Adobe Flash
.I%u2019m biased. I%u2019m a full-time Flash developer and I%u2019d love to get paid to make Flash sites for iPad. I want that to make sense%u2014but it doesn%u2019t. Flash on the iPad will not (and should not) happen%u2014and the main reason, as I see it, is one that never gets talked about:

Current Flash sites could never be made work well on any touchscreen device, and this cannot be solved by Apple, Adobe, or magical new hardware.

That%u2019s not because of slow mobile performance, battery drain or crashes. It%u2019s because of the hover or mouseover problem.

Many (if not most) current Flash games, menus, and even video players require a visible mouse pointer. They are coded to rely on the difference between hovering over something (mouseover) vs. actually clicking. This distinction is not rare. It%u2019s pervasive, fundamental to interactive design, and vital to the basic use of Flash content. New Flash content designed just for touchscreens can be done, but people want existing Flash sites to work. All of them%u2014not just some here and there%u2014and in a usable manner. That%u2019s impossible no matter what.

All that Apple and Adobe could ever do is make current Flash content visible. It would be seen, but very often would not work. Users would hate that broken promise much more than they hate gaps in pages, missing banner ads, and the need to download a game once from the App Store instead of re-downloading it every time they visit a Flash game page.

Mouseover examples:

* Video players where the controls appear on mouseover and hide otherwise. (This seems to be the norm, in fact. Whereas a click on the same video does something different: usually Pause. Try Hulu for instance.)

* Games where you steer with the mouse without clicking (extremely common).

* Menus that popup up subpage links when you mouse over a main button, vs. going directly to a main category page when you click.

* Buttons that have important explanations/summaries on mouseover, which you need to understand before deciding what to click.

* Functions that use mouseover to preview and click to commit; such as choosing hair colors for an avatar: you mouse over the colors until your character looks the way you like, and then you click to commit.

* Maps and diagrams that don%u2019t use click at all, but pop up info as you mouse around.

* Numerous other custom mouseover functions that %u201Cjust work%u201D with a mouse and need no explanation.

None of these things can work right with a finger (or traditional stylus) because on a touchscreen, pointing at something without clicking isn%u2019t a mouseover: it%u2019s just holding your finger vaguely in the air. The device doesn%u2019t even know it%u2019s happening.

In addition, some Flash sites rely on right-clicks (such as for security settings), and many rely on a physical keyboard. Especially games, which are the main kind of content people want from Flash. (I%u2019d say video, except video can easily be done without Flash, and sites are increasingly doing so. Much of the video missing from your favorite Flash site is probably easily found on YouTube anyway.) Games often use realtime key control, requiring a distinction between a single press and a long hold, and including the need for chording. For instance: holding right arrow continuously to walk, while simultaneously hitting the space bar to fire, and either hitting up-arrow once to jump or holding up-arrow longer to jump higher. A touchscreen keyboard can%u2019t handle these kinds of rapid, precise combinations well. And the keyboard would block the game view, too. Games on a touchscreen need controls suitable for a touchscreen (and/or tilt).

The only potential %u201Csolutions%u201D to the mouseover problem are terrible ones:

A) The best case: every Flash app on every site is re-thought by its designers and re-coded by its programmers (if they%u2019re even still available), just for touchscreens. They wouldn%u2019t use mouseovers any more%u2014or else they%u2019d have dual versions of all Flash content, so that mouse users could still benefit from the mouseovers they are used to. That%u2019s a ton of work across the Web, for thousands of parties, and just isn%u2019t going to happen. Plus, with many sites, mouseovers are so fundamental that the very concept of the site would be altered, creating a whole different experience that would annoy and confuse the site%u2019s existing users. (And would this be any easier than simply re-designing without Flash at all? Not always.)

B) Gestures, finger gymnastics or extra physical buttons are created that simulate mouseover%u2014which is absurd since mouseovers, by their nature, are meant to be simpler than a click/tap, not more complex. And meant to be natural, not something new to learn. Not a whole set of habits that violates our desktop habits. And any additional complexity is unworkable when it comes to games: you need to react quickly and simply, not remember when to hold the Simulate Mouseover button, or use three fingers, or whatever. The game itself is enough to deal with. Anything on top of that takes away fun.

C) Make clicking itself%u2014the fundamental, constantly-used action%u2014MORE complex. Such as requiring a double-tap or two-finger tap before anything is registered. (Two taps is how Mobile Safari does JavaScript popup menus: the first tap pops it up, the second selects.) But many Flash apps and games already use double-click (or rapid-fire clicking) for other things. Extra taps only make sense for certain limited situations (like menu popups). And it%u2019s not just clicking: you have to allow for movement: dragging vs. a moving mouseover. And even if a system could be created that was quick and simple enough to do all this in the middle of a game, how would the user know which parts of a web page played by these special rules? One part of a page (the Flash elements) would do fundamental things like scrolling or link-clicking differently from the rest of the page! (Not to mention the rest of your touch-based apps.)

D) Have a visible mouse pointer near your finger, and not interact with things directly. Use Apple track-pad style tap-and-drag gestures, as seen in some VNC clients. This kind of indirect control violates the very principle of direct touch manipulation. This is making the touchscreen be something %u201Clike a laptop but worse%u201D and has little reason to exist. And again, you%u2019d have to keep remembering whether you were in direct touch mode or %u201Cdrag the arrow%u201D mode, and which parts of the page behaved in which way.

E) Require extra force for a %u201Creal%u201D tap. So you%u2019d have to learn habits for a light tap vs. a hard tap. This extra complexity is non-intuitive, cramp-inducing, and easy for the user to get wrong (even with click feedback, as in RIM%u2019s failed BlackBerry SurePress experiment). This complicates the whole device just for the sake of one browser plugin, and makes it more expensive to build.

So it%u2019s not just that Apple has refused to support Flash. It cannot, logically, be done. A finger is not a mouse, and Flash sites are designed to require a mouse pointer (and keyboard) in fundamental ways. Someday that may change, and every Flash site could be redesigned with touch-friendly Flash. But that doesn%u2019t make Flash sites work now.

Even if slow performance, battery drain and crashes weren%u2019t problems with Flash (and they truly are), nothing can give users of any touchscreen, from any company, an acceptable experience with today%u2019s Flash sites. The thing so many complainers want is simply an impossibility.

By the way, imagine my embarrassment as a Flash developer when my own animated site wouldn%u2019t work on the newfangled iPhone! So I sat down and made new animations using WebKit%u2019s CSS animation abilities. Now desktop users still see Flash at adamsi.com, but iPhone users see animations too. It can be done.

Morgan Adams, adamsimmersive
interactive design and games

Support RoughlyDrafted!

Daniel Eran Dilger is the author of %u201CSnow Leopard Server (Developer Reference),%u201D a new book from Wiley available now from Amazon as a paperback or digital Kindle download.

Links:
Amazon | Facebook | Twitter

Bookmark:

Del.icio.us Slashdot Digg
RSS feed for comments on this post
TrackBack URI for this post

%u2190 Ten Myths of Apple%u2019s iPad: 2. It needs Flash, segment 2

27 comments

1 uberVU - social comments { 02.20.10 at 4:43 pm }

Social comments and analytics for this post%u2026

This post was mentioned on Twitter by DanielEran: New: An Adobe Flash developer on why the iPad can%u2019t use Flash %u2013 http://tinyurl.com/yfzqcyc...

2 jomi { 02.20.10 at 5:03 pm }

What about the most obvious solution: A single tap means a %u201Cclick%u201D while holding the finger down for a moment means %u201Chover%u201D.

The same principle is already used in many ways in Mac OS X (e.g. on the current MBP, when pressing F4, Dashboard pops up and disappears when pressing F4 a second time. However, if you press and hold F4 for a short time, Dashboard pops up, stays on screen while F4 is pressed and disappears when F4 is released%u2026)

%u201C* Functions that use mouseover to preview and click to commit; such as choosing hair colors for an avatar: you mouse over the colors until your character looks the way you like, and then you click to commit.%u201D

Well, guess what? The iPhone OS has used the principle explained above in exactly such a situation: In the keyboard. Just tap and hold the letter a%u2026

So, although partly true, some arguments in this post don%u2019t sound very knowledgeable%u2026

3 bartfat { 02.20.10 at 5:16 pm }

Actually, tapping and holding a website is already used in Mobile Safari. It scrolls the page. So that wouldn%u2019t work, because you can%u2019t assign two different functions to the same control. So in reality, Flash sites will have to be rethought from the ground up to be built using touchscreen principles. If you%u2019re rebuilding the whole site anyway, why not just use CSS animations instead of relying on a proprietary plugin?

Great article Dan, got me thinking on why Flash isn%u2019t practical on a touchscreen.

4 nat { 02.20.10 at 5:53 pm }

Actually, tapping and holding a website is already used in Mobile Safari. It scrolls the page.

And if we want to get even more technical, scrolling the page is done by vertical swiping.

Tapping-and-holding in MobileSafari is used for magnifying/selecting text and/or choosing other contextual options, e.g. tapping-and-holding a link to copy it or open it in another browser window.

So, when interacting with Flash on an iPhoneOS device like the iPad, the user would be expecting to select text or pictures and instead would get some weird onscreen cursor.

Morgan%u2019s email further demonstrates that even if Apple allowed Flash on the iPhone or iPad, most of Flash%u2019s unique applications that can%u2019t yet be replicated with things like JavaScript simply would not make sense on a multitouch device.

Fortunately, those unique Flash applications are 1) in the minority, 2) may be largely duplicable with HTML5 canvas, 3) ported to iPad using Adobe%u2019s Packager for iPhone, or 4) rewritten from the ground up as native apps. The most common uses of Flash on the web%u2014video, ads, and relatively simple menus%u2014can be done in JavaScript and H.264 video.

Here are two examples of what Packager for iPhone can do:

Simple Metronome (if you don%u2019t want iTunes to launch, copy/paste this link into the address bar and hit Return):
http://itunes.apple.com/us/app/simple-metronome/id350401073?mt=8

Wired-Adobe%u2019s digital version of Wired (the text of the article even mentions the iPad by name, twice):
http://blogs.adobe.com/digitalpublishing/2010/02/adobe_wired.html

Thanks for posting this, Dan.

5 lmasanti { 02.20.10 at 6:03 pm }

Flash could become the %u201Cnew IE6,%u201D as there are lots of sites that uses it and will break if Flash disappears, people do not move away from Flash.

Maybe, in no so time , we%u2019ll forget of Flash as we already forgot of diskettes%u2026 But it took the guts from Apple to build a diskette-less iMac first!

6 davesmall { 02.20.10 at 6:20 pm }

So how are they handling these issues on the Nexus One and other touch screen iPhone knock-offs that are supporting Flash?

7 t0m { 02.20.10 at 6:40 pm }

Holding a finger down currently (over a picture ) brings up the save option. Moving your finger around once it%u2019s held down cues the copy blue box to work. Apple%u2019s not going to let Flash muddy the UI waters on this one.

8 ianf { 02.20.10 at 6:40 pm }

Morgan Adams%u2019 objections are so insurmountably valid, that one has to ask why hasn%u2019t Adobe, present owners of Flash, seen it coming and/or done something about it. After all, it%u2019s not like they were entirely unaware of %u2013from their point of view%u2013 limitations of touchscreen interfaces prior to launch of the iPhone (2007). If transition [to a potential touchscreen-aware Flash] handled right, the triggering functions of present MouseOver and similar could probably be assigned to other events and common gestures. ActionScript would have to be extended to recognize multitouch patterns and the Flash as a whole %u201Clearn%u201D to accept input from e.g. device tilting, shaking, compass-alignment, and other tactile entry methods. Adobe KNEW the premises, yet they didn%u2019t lift a finger[sic!], and kept schtum. Why? What did they hope(d) to accomplish by making Apple shoulder the blame for Flash%u2019s absence on iPhone when they already knew the score?

9 ianf { 02.20.10 at 6:53 pm }

@davesmall : badly. And then mainly for presentational, chiefly ad use, where the sole permitted interaction is of the MouseDown = SingleFingerTap type.

10 David Dennis { 02.20.10 at 7:24 pm }

Jomi, the problem with this is that the hover is truly a casual gesture. You move your mouse over an object and something changes, usually immediately. So in web pages that use this, when you hover over a link it changes color, indicating that it can be clicked. To require a press for any kind of hover action clearly does not make sense.

You will note that iPhone simply does not support the a:hover construct in CSS.

It%u2019s true that you could tap and hold something. But generally the amount of time it takes to register a tap and hold is very long compared to the amount of time you would want to wait for a %u201Chovered%u201D element to pop up. Apple has done this in lieu of right clicks for a long time, and it drove me nuts when I still used a one-button mouse. Much easier to Option-Click on the Mac than to hold a key down that long.

D

11 carlo.98 { 02.20.10 at 7:42 pm }

This article reminded me of this: http://counternotions.com/2008/06/17/flash-iphone/

12 cy_starkman { 02.20.10 at 8:57 pm }

Thanks, I had not thought of these deeper UI issues, just the hoggy nature of it. Oh that and I block all flash on desktop anyway, why pay for data to be shown ads.

13 WebManWalking { 02.20.10 at 9:34 pm }

Morgan Adams, do you, by any chance, live in the Adams Morgan neighborhood of Washington, DC?

Just curious.

14 markhayden { 02.21.10 at 12:28 am }

I think everyone has some good insights but realistically if Apple wanted to support flash they could easily integrate a touch pad that slides up on the screen similar to the keyboard. It would not be optimal but it would be sufficient to give users the ability to view flash content. I think its a very bold and daring move to exclude flash since a majority of the web utilizes it in some way, shape or form. Its clear that things are moving from flash but in my opinion viewers are not quite ready to cut it out completely. Long story short I think it will kill the iPad. You can%u2019t have %u201Cthe best%u201D web experience without flash as of now.

15 === popurls.com === popular today { 02.21.10 at 12:48 am }

=== popurls.com === popular today%u2026

yeah! this story has entered the popular today section on popurls.com%u2026

16 beanie { 02.21.10 at 1:05 am }

Flash 10.1 (next version) will have multi-touch, gesture, accelerometer support and run on mobile phones. Seems like almost every major phone manufacture has agreed to support it except Apple. Seems like a killer app to me.

Most active Flash sites force you to install the latest runtime. When Flash 10.1 is released, watch how active Flash sites will probably be updated to support new features such as screen rotation, accelerometer, and mult-touch.

So what Hulu%u2019s current video player uses hover. Obviously, they will update their video player to support Flash on mobiles when 10.1 comes out. As will most other active Flash video sites.

Anyway, the iPhone/iPad user should decide to use Flash or not instead of Apple deciding for everyone. Apple just wants total control of the content. So if the user do not like Flash then do not use it. Apple lets a bunch of crap in the App Store and lets users decide to use it or not.

17 ianf { 02.21.10 at 1:38 am }

OK, you%u2019re the man, beanie. Let%u2019s say Apple lets Flash loose on the iPhone, %u2018Touch and iPad (and potential future devices). Day 1 we get to read how millions of end-users%u2019 fav. games, written but not updated for the new version, open but do not run properly. All that works are splash screens and animated banner ads. Day 2 is more of the same. Day 3 the search for the guilty party begins. Day 4 various self-styled pundits all over the blogosphere explain how Apple is to be blamed for not having equipped the iPhone with even the tiniest of mouses. So that existing stock of trashy shoot-em-ups could run on users%u2019 new pricy iPhones. All 40 minutes of it on full battery, but, hey! it%u2019s their CHOICE, innit?

18 nothingGrinder { 02.21.10 at 2:40 am }

1. Building an overlay controller for your game solves all of the problems you discuss.

2. Use of proper OOP practices should allow you to port your Flash projects to new screens without the need to modify the existing code.

3. Anything older than AS2 should be reprogrammed anyway. Event AS2 should be reported or reprogrammed for AS3 and Flash Player 10. If you are still trying to make money on games created years ago then you have lost sight of Adobe%u2019s vision for Flash.

There are so many excuses for being a lazy programmer. We are Flash developers and we should be trying to prove to the world that Flash is worth the struggle of its learning curve. Adobe has gone to great lengths to bring Flash back to the top of the web game over the years. They fought through all the %u201CFlash is crap%u201D hype in the 90s and they will continue their fight against all the %u201CFlash is dead%u201D rumors happening today.

Flash is the future of the internet. Adobe%u2019s partnerships with Google, Microsoft, Nvidia, Qualcomm, and everyone in the Open Screen Project is proof. The only one left out of the entire deal is Apple.

Open hardware is the way of the future. Google has proven this by giving away free Nexus One phones to thousands of people all over the world. Apple needs to realize that Flash is now the leading web technology and as Flash developers, we need to knuckle down and start porting our old projects for touch devices.

19 Berend Schotanus { 02.21.10 at 2:57 am }

Great post! I didn%u2019t think of it. It makes so perfectly sense.

Yet the problem is not just a Flash problem, it is there in HTML as well (be it to a lesser extent). Fundamentally it means that content must be designed with regard to interface. We can distinguish:
- %u201CClassical%u201D GUI desktop metaphor with keyboard and mouse.
- Portable touch device with motion sensor, GPS, compass, %u2026
- TV remote with DVD-type interface where iTunes LP/extra seems to be targeted at.

and of course we also still have the non-interactive interfaces like print and cinema.

20 Dorian { 02.21.10 at 6:52 am }

Good point! I agree that the experience of Flash content in the iPad browser would probably be very unsatisfying with most of the existing content. So even as a Flash/Flex developer I think it is not too bad to be forced to switch to HTML5 for interactive apps on websites. It makes a lot of sense on the long run.
The real problem in my eyes is that as of now there are no professional tools for interactive developers to create the same stuff in HTML5/Javascript that they used to make in Flash. I wrote down my thoughts about this here:
http://dasflash.com/2010/01/we-dont-need-flash-on-the-ipad-we-need-better-tools-to-build-html5-sites/

21 FreeRange { 02.21.10 at 7:01 am }

@Beanie %u2013 Yes beanie, we can read adobe%u2019s vaporware statements as well. The reality is that even with the new version of flash (10.1) there will be a ridiculous amount of overhead that is going to drastically impede performance and suck the life out of batteries. What needs to happen is for flash to die a quick death while better, more efficient open standards take hold. What Apple needs to do is not give the consumer %u201Cchoice%u201D but to hold their ground to force this pig out of the market. If people will stop building sites with this bloated crap, we%u2019d all be better off.

22 screaser { 02.21.10 at 7:12 am }

Excellent points.

Might be a tiny hole in the logic behind %u201Cno tablet, ever%u201D %u2014 there are several companies working on proximity sensing tablets (including Apple, according to one patent filing).

It%u2019s conceivable that a proximity sensing tablet could let you %u201Cmouseover%u201D by moving around within say an inch of the screen, with intelligent software to detect the difference between mouseover-type activity and coming in to tap.

23 mhodges { 02.21.10 at 7:31 am }

Is this a good argument? Basically, it says: flash should not be allowed on iPad/iPhone because some sites%u2013those sites not properly designed for multi-touch, won%u2019t work. But what%u2019s a better experience: to prevent access to ALL flash sites? Or allow flash so we can at least access the sites that DO work?

It seems like flash image galleries, like Autoviewer (which I use on tokyorealtime.com), could work on iPhone/iPad without any of the problems he talking about which affect games. Or am I wrong? There are other kinds of flash content, like animations and banner ads, which are not interactive at all. Who knows what kind of dynamic content the marketplace would create specifically for the iPhone if Flash were allowed.

I think the real reason is very simply: allow Flash on the iPhone and apps could be created in flash and sold through some 3rd-party, cutting Apple out of the vertically integrated monopoly they create with these devices as platforms for official App store software%u2026.

24 if { 02.21.10 at 7:53 am }

I agree the interface to flash is a problem. Hover isn%u2019t the only major problem and is likely to be easier to fix that the other one (press to hover, release to click, like the iPhone/iPad keyboard).

The bigger problem I see for mobile devices esp smartphones and Flash is the small screen size results in small touch targets, and that a finger held to a screen obscures what is underneath %u2014 unlike a mouse pointer %u2014 and this becomes much more the smaller the screen size. Last major issue I see is processing power. Phones and tablets are becoming smarter and faster, but so are computers, and the gap will remain. So, any Flash app or game built for a computer that uses all that power will deliver an uneven experience, or flat not work, on a mobile device. Flash is nice, but the UI should have %u201Cflashblock-style%u201D click to activate, should suspend if a window/tab is not foreground, and needs to have some clever zoom controls to make touch point interaction easier. There%u2019s a lot to do to make Flash usable.

Last thought %u2013 have a look at the HTC Hero and Nokia n900 is you want evidence for the difficulties of putting Flash on mobile devices. I have tested both. My thoughts here are based on that testing!!!

25 Donald { 02.21.10 at 8:28 am }

Mouse-over-activated pop-ups are becoming too common as annoying occurrences in content sections of websites (i.e., sections apart from overt advertisements, graphic hyperlinks, and games). In the past, we could have a fairly reasonable expectation that pop-ups or menus only appeared when the mouse was moved over a graphic or word that was clearly identified as being a menu or hyperlink. (You moved your mouse to point at something that was either pre-identified as a hotspot or became highlighted in some way when hovered above.) Not anymore. Nowadays you can hardly move your mouse anywhere without something popping up! I would love to see that change.

26 stefn { 02.21.10 at 9:02 am }

The HTML5 train is leaving the station; I don%u2019t want Apple to stand there waving goodbye. Leadership is the ability to get folks to places they would not go otherwise simply out of fear of change. Apple has often led (GUI, USB, iTunes, Multitouch) the way out of dead-ended technologies. Flash is one more example.

27 tfk { 02.21.10 at 9:02 am }

Regardless of the platform used, Flash. HTML5, isn%u2019t there a problem if there%u2019s isn%u2019t an agreed upon convention of what touch signals translate to what action across Apple%u2019s, Android%u2019s, Nokia%u2019s, Samsung%u2019s, etc. operating systems developers will have a major problem. They%u2019ll have to develop different version of heir apps for every different OS. It will be difficult unless the companies can agree on a common convention which will involve these companies giving away or cross licensing a lot of touch IP. Then you have the the problem of transitioning to the next generation %u201CMajority Report%u201D input mode.

You must log in to post a comment.

Excellent work

Long live Music Piracy & Bit Torrent

Google Buzz is a service that integrates with many Google products, so it's quite difficult to completely disable it. For example, to use Google Buzz, you need to have a searchable Google Profile.

While you can click on "turn off Buzz" in Gmail, you don't disable Buzz. The setting only hides the Buzz section and you can display it again by clicking "turn on Buzz".

Fortunately, Google added a new option that lets you disable Buzz, but you need to delete your Google Profile. Go to the profile editing page, scroll to the bottom of the page and select "Delete profile and disable Google Buzz completely". Google claims that "this will disable Google Buzz integration in Gmail and delete your Google profile and Buzz posts. It will also disconnect any connected sites and unfollow you from anyone you are following."

Even if you delete the profile, you'll still be able to create another profile later and you won't lose your Google account. After clicking the red link, Google shows a new page that informs you that "you are about to delete your public profile, including any Buzz posts you have made and your connected sites settings". There's also a strange option enabled by default: "unfollow me from anyone I am following in Buzz, Google Reader, and other Google products".

It seems strange to see that the mass unfollowing is optional, considering that you are about to disable Google Buzz completely. The option also shows that there's a connection between Buzz followers and Reader followers, but I don't see why you need to unfollow from everyone in Google Reader to disable Google Buzz.

After you click on "Yes, delete my profile and posts", you'll notice that you didn't actually disable Google Buzz completely. You've deleted all your posts, removed your connections and the people you were following, but the Buzz section from Gmail is still there and you still have the same followers. Now you can click "turn off Buzz" and you'll disable Google Buzz.

It would be great to be able to remove Google Buzz from the "edit services" page, without having to delete the profile and remove Google Reader followers, while still seeing the Buzz section in Gmail.

Labels: Google Buzz

Alex of Google Operating System tells you how to disable buzz and delete your google profile

So they are back on the growth path