Sep 062010
 

Following up on my previous article, I wanted to test a few more things. For the sake of generating clear numbers, I’m using yesterday’s 2-hour game SolarCraft as my testbed.

Throughout these tests I’m spawning 2000 collectors with cacheAsBitmap enabled, and running the test for 30 seconds and providing the average FPS (which will therefore include, but hopefully minimize, startup lag throwing the numbers off a bit). As I mentioned in my previous article, I’m trying to get real-world numbers (using believable in-game item counts) with other things going on (such as graphics and game logic loops).

Vector vs. Array

Vectors are new in Flash 10 and are, put simply, a typed array. Many claims have been made on the huge speed increases when using Vectors (I’ve heard 250%+); let’s test it out!

A 2000-length vector iterating twice per frame generates me an average of 26.5FPS.

Converting the code to untyped Arrays provides me an average of 26.0FPS.

That difference can easily be chalked up to various other things and is pretty much indistinguishable. It looks like, for the purposes of this game, Vector iteration isn’t much faster at all.

However, I still love vectors for being typed. Still worth using, no harm in it and your code will be easier to work with!

Loop Iterations

“Hey wait,” I hear you saying. “Why do you loop through the 2000-length vector twice per frame?

Good question! It was because I made sloppy code and thought it would be a good idea to simply loop through everything again rather than fix it (that’s how you make 2 hour games!).

For clarity, one loop is simply cycling through all game objects (in this case, 2000 collectors) and running their “update” function. This function calculates where the collector wants to go, runs a few randomization jitters, and does some simple coordinate translations to produce a vector. Then they “walk.” Nothing exciting, no big maths.

The other loop is just checking for dead collectors and culling them from the Vector (not that anything is dying in these tests).

As a baseline, we are currently getting 26.5 FPS.

So I’m curious.. let’s comment out the useless dead-check loops and see what kind of performance boost we get. Looks like 27.2 FPS. A slight increase, maybe even one worth sticking with; a single frame per second optimization in a few places might add up.

Let’s remove the walk-logic loop and see how that fares. Looks like 29 FPS. This one was to be expected; removing all the art movement and gamelogic from the game should very well speed things up. To be fair, the “current FPS” was locked at 30 for a long time, and we only got 29 FPS because of my 30-second-average including bootup delays.

Well, let’s play with this dead-check loop again. What happens if I make it run 10 times, thus iterating the game through 22,000 vector entries per frame? We get a result of 22 FPS. That’s a significant hit for a loop that doesn’t even do anything.

I think it’s safe to conclude that iteration through long lists – regardless of content or actions – is a big performance hit. But putting in a useless duplicate iteration 10x over? Not exactly a “big problem” in game design, I think. You’re better off ignoring this issue and optimizing what happens in the loop itself.

Playing with Target FPS

If you tell Flash to run 30 frames per second, it might run at 25 FPS thanks to your inefficient code. I’ve heard, however, that if you tell Flash to run 60 times per second with the same code – that 25 FPS might drop to 10FPS. Trying to over-achieve can actually hurt you!

I’m really interested to see if this holds true in my game.

I’m currently running at about 27FPS, on average, with a target framerate of 30.

Changing the target framerate to 60…  Has little effect! I’m averaging only 1 less: 26FPS.

Let’s crank the target framerate up to 120… Hmm. My monitor refresh is 60hz, I wonder if Flash can sense that and scale down my request? I’m showing a target of 60 still, and still hitting 26FPS. Oh well.

Just for kicks let’s drop the framerate down to 10 an see what happens: Wew, a solid 10 FPS. At least I know there wasn’t some weird overhead preventing me from hitting 30FPS before; it looks like my code really is a bit too slow (or I have too many dudes on the screen).

  7 Responses to “More Performance Tests”

Comments (6) Pingbacks (1)
  1. This is pretty interesting, but you *need* to post some code. There’s a whole bunch of ways to iterate over a Vector and even more ways to cull dead objects from them. Just plainly replacing Array’s with Vectors isn’t that exciting of a performance test.

    The “higher-framerate-gives-poorer-performance” claim is likely from this blog post: http://www.craftymind.com/2008/04/18/updated-elastic-racetrack-for-flash-9-and-avm2/

    But I’ve never seen this happen, all I’ve seen is that a higher framerate gets you more resources. An app that maxes out at 15fps when the limit is set to 30, may well do 25 if you up the limit to 60. However, this has gotten alot better with newer browser and flashplayer versions. On a related note: as of version 10.1 the max fps is locked at your screen’s refresh rate, so even if you set it to 120 it won’t go over that.

    That’s my rant for today ;)

  2. Hey grapefrukt,

    I’ll post the code soon* and let anyone take a look. The dead-culling loop was simply a for loop, but since nothing ever died in the test no code was fired at all.

    The game logic loop was a “for each” loop.

  3. Vectors are faster for ints and Numbers, for other objects Arrays are faster.
    In a for loop you can make something like 200.000 iterations of an Array access or write operation and you would get 30 fps easily so I don’t think your tests are really testing the Vector or Array speed because it’s insignificant compared to the time it takes to call the game’s logic. The Vector operation must be using like 0.01% of the time used in the loop.
    What usually consumes the most is the render logic. MUCH more than the other operations. So much more that it usually makes the rest of the operations pretty insignificant.
    To test performance you can use Grant Skinners performance harness at
    http://www.gskinner.com/blog/archives/2009/04/as3_performance.html

    Hope it helps.

  4. Thanks for the info, Santiago! I know a lot of these performance tests only show massive differences when you start getting up into huge arrays and vectors – I wanted to see if they made any appreciable difference in an actual/realistic game environment.

    So what I’m learning from these tests is:
    - First fix your rendering (bitmaps/blitting/etc.)
    - Second fix your game logic

    And once, and only once you’ve fixed those two, THEN start getting into arrays vs. vectors, loop types, and other relatively “minor” tweaks.

    When I ask for advice on speeding things up in my games, the only responses I get are things like “try vectors” or “for loops are slower than while loops” or “comments make things go slow”. While some of these hold true for other languages, and some of these do in fact increase speed, the relative increase as compared to rendering/game logic is so small that it’s safe to ignore.

  5. Something to bear in mind when performance testing is that framerate is an inverse metric, and therefore doesn’t tell a really accurate story.

    It’s much more meaningful to add timers at the start and end of your process loop (or if due to event usage, that’s not possible, then once per frame) and calculate milliseconds.

    E.g. The two versions produced 26.0 and 26.5 fps. In millis, this is 38.4 milliseconds and 37.7 milliseconds, a savings of .7 milliseconds.

    This is useful because numbers like that can be extrapolated into more meaningful terms.

    The other important point here is that in Flash, drawing and traversing the display graph are the bottlenecks, period. You’ll start noticing significant differences between arrays and vectors at 100,000′s of iterations, which is a number which WILL be unmatched by the number of game objects you have, which means you probably have a serious architecture issue.

    In other words: Don’t optimize prematurely. Do profiling and optimize where things are bottlenecking.

  6. “or I have too many dudes on the screen”

    Another thing to bear in mind when you do performance tests is to close all other applications, including gay porn.

Sorry, the comment form is closed at this time.