I like to think that I’m a responsible developer. I make good constructors and deconstructors to prevent memory leaks, I have fairly secure coding habits, I OOP the hell out of everything, and I do everything by-the-book.
Imagine my frustration, then, when all of my games (particularly my latest, Steambirds) have gigantic memory leaks. Like, 500+MB consumption in 5-30 minutes of gameplay.
At first I thought it was EventListeners keeping persistence on my trashed objects. I went through my code several times making sure I nabbed them all and set them to Weak.
Then I thought it was the weak listeners failing. My tests were showing orphaned objects still chugging away despite garbage collection making an observed pass. I’ve banged my head against several walls over the last year trying to figure this out, and in the end always giving up. “Restart the application if it gets slow!” was my only recourse. Flash doesn’t even let you force garbage collection on objects, so I couldn’t hack my way out of it.
Today I found a very interesting blog post. I always knew how the garbage collector worked; it traverses your tree of object references, and identifies any “islands”; that is to say, any objects that aren’t referenced by any variables in your code. At the end of it’s run, all the islands are trashed. What I didn’t know is that IF YOUR ISLANDS ARE BIG ENOUGH, FLASH GIVES UP AND ASSUMES IT’S NOT TRASH.
I guess by complexly-nesting my code, it resulted in something that was too much for the goddamn garbage man to handle. My games essentially think any object I ever create is for HOLDING ON TO FOREVER.
If you want some more technical data, I highly recommend you check out Tom Mason’s blog post on the topic. It was eye opening, to say the least. But now I have to go rewrite a shit-ton of code.
Lost Garden
Yeah, Flash’s undocumented features have cost me a lot of time over the years. Fixing memory issues in a project that is otherwise done is no fun. Good luck with that!
So doing a quick pass of on-death = null codes, I halved my memory footprint. I was able to drop that by a further 80% by commenting out all my “.cacheAsBitmap = true” statements.
My standardized test of “opening a shitton of missions then closing them again rapidly” often resulted in 500+mb of usage in just a minute or two. I now have trouble getting it over 50mb with any regularity. Peaks at about 170mb if I time some big events between garbage collections.
Hi there,
Being one of the guys that had a look at “who are these awesome people that developed Steambirds” (one of those %1.5), I found you from credits page. I’m really surprised to see such post since I had a major GC problem with my latest game on your blog. Thanks a lot. Hope to meet you some day in real life.
Cheers.
I’ve got into the habit of using the weakref dictionary hack to track when objects are disposed during debugging (via a trace that fires every 5 seconds or so). If I find an object stays around for too long I just add code that kills/nulls out parts of it until it gets garbage collected properly. Not exactly ideal… but then it is flash!
> I OOP the hell out of everything
You might want to go read some pretentious functional programming blogs. It’ll be educational.
@a,
are you suggesting that the memory leak problems are cured with functional techniques?
is more and more appealing every day. I’m writing an overhead RPG tile-based game (sorta like pokemon/zelda) in right now. It’s nothing but a pleasure. AS has pissed me off to no end.
woops, I guess they cut out tags from posts :3
CANVAS***!! arg
Wow, thats just too funny dude. Well done
Lou
http://www.anonymizer.us.tc
The tone of your article is much appreciated. My own adventures in garbage collection with AS3 usually result in a similar affect!
I love how you blame your bugs on Flash. It’s not Flash’s fault you’re leaving cycles in a graph garbage collector. This is pretty 101 stuff.
The ECMA specification, which I suspect you haven’t even read, is actually pretty clear about this: if there’s a cycle of any size, it’s not required to garbage collect.
Stop writing sloppy code then getting angry when Flash can only fix so many of your problems transparently. All you actually haev to do is break the cycle and wait a frame.
This isn’t Flash’s fault at all. This is you mis-using the language.
@Andy: Just came across your posting via Reddit. Played the game via Kongregate. Rated it 5 stars (most games don’t get above 4 from me).
I have heard a lot of Flash game devs (and others like most people in the world as flash videos skip and stutter) bash on Flash and its memory and performance issues. I wonder if it is a lack of understanding Flash and not the program/language as John says? I find it hard to believe that Flash could become the de facto standard of online web gaming and still have such issues unaddressed. But, maybe that is because Adobe nee Macromedia has a monopoly? The irony is that Microsoft’s Silverlight could come to the rescue and cause Adobe to put more dev resources into Flash’s performance if the problem lies with Flash. Let’s hope and keep up the good work. :)
@John Haugeland
Actually John, leaving cycles in an object graph is perfectly acceptable in almost every OO language and is handled well in all modern garbage collectors. There are numerous engineering/business cases that would benefit from allowing cyclic object graphs and it is fairly pathetic that Flash is showing this behavior in handling them. This behavior represents a serious limitation of the technology whether the documentation acknowledges it or not.
I was working on an AS3 based multi-user/multi-cam Flex project. We had brought in some Indian contractors to help save some time and bust out grunt work of the project. During one of the planning phases one of the Indian developers said they had tried using AS3 for this style of application at another company a few months earlier but couldn’t get streams to work correctly in AS3, it would only work correctly in AS2.
Thinking they were completely retarded I asked them to send me over both their AS2 and AS3 example. Everything looked fine the code was solid (as solid as a quick prototype could possibly be). They were creating their netConnections and netStreams as expected and everything would work fine in the AS2 versions just as they said. However in AS3 sometimes you would get 4 streams running then they would all crash on the 5th… sometimes you could get 8 and they’d crash on the 9th. It wasn’t consistent, it wasn’t apparent, and worst of all it was a huge time vampire for me personally as I tried tons of approaches.
Finally after about 5 days of fighting with the AS3 version in Flex I was sitting at my desk with one of the Indian developers and I decided to tweak the functionality of the test app. I was going to store the streams in an associative array and drop various streams in a non-linear fashion to see if this affected how/when the apps streaming functionality would crash. All of a sudden I’m running 200 concurrent streams and I can’t get the damn thing to break. Turns out it was the GC the whole time. Since the netStream objects were created inside the scope of a function they would get cleaned out by GC when Flash decided it was time for more memory, however they would run flawlessly for hours unless Flash decided it needed this memory. In AS2 as soon as you exited the scope of their parent function the streams would die as expected as their reference would be wiped from memory as far as the application was concerned.
Anyway, it was a stupid problem, I was an idiot for not thinking about it sooner… and it taught me a valuable lesson about working in Flash… don’t trust the GC or code execution flow.
whoops messed that up a bit in AS2 the streams would run fine since Flash recognized the netStream objects as being in use with a reference to the video displays on the stage.
@Derek
The Flash GC underwent some big changes in the jump from AS2 to AS3. Also Flash didn’t become king of the hill because it was great, it just didn’t have a lot of competition on the web outside of Java applets and huge Director/Shockwave files.
@John
Firstly, it’s not that hard to detect cycles and GC them. That capability has been in the Flashplayer GC since version 9: http://www.adobe.com/devnet/flashplayer/articles/garbage_collection.html
Secondly, you are not addressing the claim that ‘islands’ of sufficient size are not garbage collected. Just flat-out stating that the problem must be cycles is not very constructive. I can only conclude you are indeed what you link to: fullof.bs.
I always want to defend Flash when the problem is usually obviously the programmer expecting more from a runtime than the runtime provides.
But it always struck me how GC was supposed to prevent leaks and yet all it really has done is move the problem to a higher level of abstraction.
Reading Chad’s post on returning objects created in a local scope to an object outside of that scope is awesome. Not only has that happened to me in AS3 several times, it has happened to me in C++ several times. Just the same, unless something tries to reuse that memory it can work for a long time. Sometime later the code changes which causes the memory usage patterns to change and BAM you get a bug.
This isn’t a Flash problem. This is a programming problem.
I suspect this is a programming problem and not a flash probblem. Are you using a dispose method on new york objects to clear properties within each object that needs to pee garbage collected??
I’ll admit I’m not the best programmer in the world. I’m completely self-taught and have only been playing with AS3 for a year now. Things that are “basic 101″ might have skipped by me because I didn’t attend any 101 classes.
I rely heavily on Adobe’s documentation; But I can’t find any documentation from Adobe that states the GC stops at certain island sizes; or indeed what those sizes are. In fact, Adobe says:
.. I mean, that says to me that they’ve thought of this scenario and they’ve already coded in a solution for it. The process should just take a bit longer (more frames) than normal.
Can anyone find the docs where it says “if islandsize > XXXmb then mark as mainland”?
Andy,
It doesn’t depend on object size. It depends on whether the object is referenced by anything or not.
If it is referenced it will not be collected. So it sounds like the problem you have is that the objects not being collected are referenced by something.
Simple as that. All garbage collectors work like this. The differences are just in implementation (how does it figure out whether something is still referenced or not).
It’s not the fault of the garbage collector you’re creating circular references that cannot be collected.
@James,
I’m aware of how garbage collecting works and have removed/checked for all circular references. If you check the link at the bottom of the article you’ll see that island size does affect the collection process, and that’s what I experienced here.
By reducing my island sizes without fundamentally changing the code (I didn’t recode any referencing links or anything, I just split islands in half a few times), the problem resolved itself — indicating the assumption is correct.
Ignore idiots who claim it to be a programming problem and then invoke C++, of all things, to demonstrate it. Flash is known to be a horrible piece of shit and layers of hacks upon hacks. You should assume anything that could be fucked up was, because it most probably was.
Take a look at http://events.ccc.de/congress/2009/Fahrplan/events/3494.en.html to learn more about the wonders of the Flash “VM”. It’s long, but definitely worth watching.
Hey I’m sure many of you use some sort of model to handle removing events etc. But here is a something we have come up with that seems to help out. Its called Event Controller. It basically just adds features to the as3 event model like event clustering and logging of events. Its a small foot print but is great for finding objects sticking around also with the logging features. Cheers!
Finally I know where it stems from that FireFox eats up so much memory when running for too long, and why Youtube gets periodically stuck every few seconds then. It’s the Flash GC taking up all the processor time.
As for your is-it-a-Flash-or-a-programming-argument… well, I really don’t know a thing about Flash in particular, but I know quite a few things about programming in general. And I wonder what a garbage collector could be actually good for, if it doesn’t identify cyclic references and collects them. Freeing up acyclic graphs of objects is as simple as maintaining an internal reference counter with each object, and freeing it up as the counter reaches zero. This is less expensive and much more deterministic to do than scanning the memory for forsaken objects. The Perl scripting language does it this way, and therefore they clearly state in their docs that you have to manually break up cyclic references. The Java VM, in contrast, garbage collects anything which is not reachable. That is what a real garbage collector does. So much for “All garbage collectors work like this”.
If a garbage collector sometimes collects and sometimes doesn’t, the only difference being the size of the structures, it is clearly broken.
I believe the thinking behind it is “the garbage collector can operate over several frames, but if it’s mapping efforts in a certain area of memory take too long, it times out and declares it all valid.” That’s the big problem I have. There’s no way of knowing at what size this is, so the GC works fine up until a certain magical point then suddenly stops.
It’s not clearly documented anywhere. :(
Andy-
apologies for the miscategorization of this, but I couldn’t find a place to post a general message.
Just wanted to tell you that I’d become hooked on SteamBirds earlier today whilst at work, and am now standing in a horrifyingly long line at the Kaiser (local HMO) pharmacy with at least 40 minutes to kill. Imagine my dismay when I fired up the iTunes App Store and didn’t find a version of SteamBirds I could buy. I was ready to drop .99, 1.99, even 2.99 for the game to make my prison term here pass more quickly. But alas, it isn’t to be.
Totally understand if you’ve got ethical reasons against closed platforms, and definitely wouldn’t fault you for that, but just wanted to let you know that you’ve got at least 3 people at my office ready to send you enough in royalties to at least buy you a cheap lunch ;) I know we can’t be the only ones…
*chant* dev kit… dev kit… dev kit… (Satan says, with Enzyte man grin, “Come on Andy! Sell out!”)
thanks for a cool game, sir!
j5
Flash is the special waste of the internet!
VERY EASY fix: if ANY bitmap goes OFF the stage, do this: bitmapInstance.bitmapdata.dispose()
This is not by the books, true, but a REAL flashguy knows it. You will be impressed.
There’s tons of Flash games without memory leaks, right?
All you do is publicly exposing your lack of knowledge.
Sorry I don’t mean to be rude, but the amount of Flash bashing out of pure ignorance is ridiculous these days. Please do some research, ask for help in a forum before publishing misinformation with a catchy title.
@May: I’m open to changing my mind! Please provide me with some counter-evidence. All I’ve seen is my two games’ memory handling improve after reducing island size.
These effects could very well have been brought up by some other bug that I inadvertantly fixed, and I don’t currently think I’m wrong. All evidence points to this being true.
Again, if you can explain it some other way, I’m open to trying things and getting to the bottom of this. I publicly admit I have a lack of knowledge, and I can’t seem to find the answers. All I know is what works.
Isolate your issues, then post code to a forum for others to verify, then publish results not beliefs.
So far all you present is vague claims.
OK really guys. Do you even know how GC works? It goes from the root object and looks for all objects referenced, and then all objects they reference, until every object has been identified. Other objects on the heap are then cleared up and removed. Even if some of those objects reference each other, you can’t trace their references back to the root object. If it can’t be accessed there, it can’t be accessed ever again. There’s no reason to not collect objects outside of the reach of the root object. Even if they reference each other. Flash is dumb.
I really think you should star out your swear words. There are young people who read this blog too.
@Drew: Hm, children read this blog? Allright, children are hearby banned. It’s now against my ToS to allow a child to view my blog. :P
Adobe at some point in the near future has to change or explain this because it’s clearly the biggest caveat people have with Flash aside from the fact that it can be ‘intrusive’. Especially the Apple fanboys. The apparent use of 100+ % of the CPU and the memory leaks are unacceptable, and it will kill whatever’s left of Flash’s reputation if Adobe doesn’t do anything about this very soon. They can’t rely on people making cool stuff that makes Flash relevant anymore.. Flash is a fantastic product in many respects and AS3 a great language, so time to prove Steve Jobs wrong Adobe, instead of proving him right like you’re doing atm.
Garbage collection… I hate you!!!.
You have to be a lucky guy when you are doing a flash website, that’s not fair Adobe!!!.
Fuck the children. Swearing is essential to working with as3, helps keep you sane during times of extreme frusteration
@Andy Moore : Good to see you here with the GC’s stuff. I know that not many guys out there really understanding the so-called GC which leads to many unpleasant relies.
I’m working on take the most out of the GC’s working model to improve performance and memory and find this bug, too. The GC crashes and telling me that the code takes too long to run. It’s ridiculus, there is only a loop through a dictionary’s items, and there are not really many, only 5 for 6 ones.
I love optimization stuffs, and have some ideas to share, too. If you like we can talk though my gmail : thienhaflash@gmail.com
great post, that’s what I was looking for, I thank the author of this article with all my heart.