Andy Moore » Protonaut

Protonaut is Up for Bidding

Andy Moore — Sun, 02 May 2010 16:34:49 +0000

Protonaut has been suffering a slow death. Ever since the first beta, it hasn’t generated a single sale and is suffering from a severe lack of traffic. No traffic means no new levels, which means there’s no reason for traffic to keep coming back… One of those old Chicken-and-the-Egg problems.

Of course, since I last really worked on Protonaut, Greg Wohlwend has been off making fantastic trailers and new games, and I’ve been working on SteamBirds. We haven’t had a lot of time or desire to work on something that isn’t being successful.

Greg and I decided that Protonaut wasn’t really working; it’s missing a key nugget of user-retention-fun, and the effort required to fix it is exceeding our desire to continue work. We pretty much decided to write off the game and put it to rest, but I decided to give it one last push.

In the latest update, I added some character-building signposts and rewrote a bunch of the tutorial levels. Fixed replays and made the controls keyboard-and-mouse, which is a whole lot easier to master than the old keyboard-only layout. And then I did the crazy: I posted it up for bidding on FlashGameLicense.

Bidding is probably a week or three away from closing, but it’s looking good so far! The game is already exceeding average sponsorship rates, despite the fact that the game has been out for a while (albeit with no traffic) and has a few external links in it. Exclusive sponsorships probably won’t work without a fancy non-standard deal with a sponsor, and yet we still have a lot of interest!

If the sponsorship gives the game the boost of traffic it needs to jumpstart the engines, I’ll be able to continue developing it. But if it fizzles out over a month or two, I won’t put in much more effort than bugfixes. Crossing my fingers!

Bayesian Ratings: Your Salvation for User-Generated Content

Andy Moore — Tue, 16 Feb 2010 01:34:05 +0000

I’ve been meaning to write this article for about 6 months now, and it’s about time I got it off my chest. I’ll preface this by saying I don’t “know the lingo” of statistics and might get a few phrasings incorrect, but the article should otherwise stand on it’s own.

The Problem

Protonaut is a game that relies (in a very hard way) on user-generated content, and I wasn’t sure how to approach the presentation of it. As Community Manager for Fantastic Contraption, I’ve watched the community struggle with content management. Common questions or feature requests would revolve around a few key issues:

How do I find the latest good levels to play?
How do I find the best all-time content?
How do I know which of my levels were the most popular.. in the last week?
What are some promising newly-created levels?
What new levels are complete crap?
What authors produce the best levels?
How can we deal with people who always down-vote content other than their own?
And conversely, how do we deal with people who only up-vote their own content?

Even with me diving into Google Analytics for the whole domain (or using some third-party applications), most of these questions are impossible to answer; even those that came close were very difficult to deal with in an in-game setting.

The Common Solution

Most games implement the most basic ranking systems: offering a choice of (usually) 0 to 5 stars, and allowing you to sort the resultant list of median scores. This, unfortunately, has dozens of problems. The only real value in such systems is finding the “best score of all time,” and even then it does it poorly (not accounting for chronic upvoters or downvoters). Not to say that it isn’t easy to implement; indeed, it could hardly get simpler, and any lazy programmer such as myself will argue that “good enough is perfect.” But is it?

Each ranking system has several components; the voting system, the ranking system, the sorting system, and an optional adjustment system. There are various pros and cons to consider in the foundation — the ranking portion — of your system: Should you simply have a “like” button, as in your Facebook feed? Should you have upvote/downvote system? Should you have a scale of 3 stars? 5 stars? 9 chainsaws? 10 stars?

Each choice you make in the construction of your content-feedback system ads to a layer of complexity and results in a whack of cascading consequences, that will likely change your subsequent decisions. What looks simple on the surface can ruin your community-friendly ecosystem, and what looks dauntingly complex is shied-away from by most developers.

Thankfully, many people have done the thinking for us, and this will be one more article for the mix.

Understanding the Consequences

There are dozens of research papers out there on some of the different voting styles. I’m too lazy to make it convenient for you by linking some here; it’s been 6 months since I’ve read up on them, but feel free to Google them for yourself. Some of the key takeaways from actual research have been:

Given a neutral choice in addition to a positive/negative choice, neutral will be chosen way more often than the others. Since neutral is a non-vote that hardly effects the statistics, it’s just giving someone an “out” from making a decision.
Given a large-scale choice (such as 1-10 stars), people will either vote to the extremes (1 or 10) to game the system, or vote 7. “7″ votes are so statistically huge that any 10-star system skews towards it; it becomes the new “neutral” and is the “safe vote” that makes everyone feel happy. There’s even a running joke that game magazines only ever rank video games “7/10″.
Given a “like” button with no alternative, or only positive-scale responses, it’s hard to generate true statistics (there’s no way to differentiate between “all-right” and “this content is so horrible nobody should ever see it”).
After your user base reaches a large enough size, more and more people will take to “strategic voting”; voting the maximum on things you like and minimum on things you dislike.
Giving an incentive to voting will drastically increase your response rate, but also predictably produce spam votes, or votes “just for the reward” without any thought. It will also amplify any of the above listed effects.

I strongly recommend you take a look at your planned voting style and do a full bout of research on it. Check out Google Scholar and find some actual results, instead of going from what your heart says. Or better yet, build a system however you want and observe the results, and change them to suit your needs.

How Bayesian Sets Up The Stage

A Bayesian Rating system tackles most of our problems head-on. Let’s step through how it works.

Let’s say we’re dealing with user-generated levels in a video game. A brand new level shouldn’t be rated at 0; users will never find your new content, as it would be buried under all your crap content. Likewise, it’s unfair to make new levels with the maximum possible score, as people will spam bad levels just to remain at the top of the list. Most often, people will set the Bayesian rating to be somewhere in the middle – say, 50%. This way, every level starts off with a “fair chance”.

As more and more people vote on your level, the score will start to normalize towards the actual statistical median instead of the artifically-engineered 50% rating. This way, if a single user-made level gets 10 up-votes in a row, it’s ranking will appear as such:

50% with 0 votes
60% with 1 vote
80% with 4 votes
90% with 8 votes
100% with 10 votes

The magic behind the Bayesian system is this weighting (or dampening) system. It’s influenced by what is referred to as a “magic number”; this figure tells you how many votes need to be cast before the Bayesian weighting is no longer in effect, and the score truly is a median of all the other scores.

The weighting system actually takes into account the variety of scores, as well. If you only have two levels, one gets a 75% score and the other gets a 30% score, one will show up as 0 and the other as 10 stars. There is always a minimum and a maximum represented, and everything in between is taken as a scale of those two figures. This solves a great deal of weighting and balancing problems with large-range systems, and compensates for situations where nobody has ever voted 100% or 0% (or where everybody votes for a narrow range of percentages).

Essentially, what the weighting mechanic is doing is “making up” votes for things that don’t have enough votes to get real meaningful data to start with.

How We Can Take It To The Next Level

It’s not just that simple though! We can crank this baby up to the next level so you never need to tune it yourself ever again.

For example; who says 50% is the optimal “starting value”? Isn’t it more fair to make the entry-level rating the median of all current scores? Why not! You can calculate your new starting value on-the-fly by looking at all your current content. This means that every user-contributed piece of content will have it’s relative scores affected by every single vote cast.

And the magic doesn’t stop there; why hard-code your magic-number to something like “10 votes” when you can make it a variable as well? Make the magic number equivalent to the median number of votes across all your content. This again will make the scores of everything change with a single vote cast, but it helps corral fringe cases and unexpected userbase expansion (or contraction) as well. Imagine you run an online store where almost everything gets a 60% vote. Then all of a sudden, one particular item you have for sale attracts a huge bag of traffic and everyone votes 100% on this new product (without casting votes on all your older stuff).

As each vote is cast, your older products would normally be scaled farther and farther down the “top” lists. However, they are also sinking deeper and deeper into the “magic number” and the score is being balanced more towards your 0-vote figure. This figure in turn is going upwards, as the new product is heavily increasing your median score!

The system is amazingly resilient and self-balancing. If a lot of levels are made and nobody votes, the system compensates. If a single item gets a lot of votes, the system compensates. If you have a thousand votes on every item, the system compensates.

Protonaut uses all of these tweaks for it’s level ranking and system generation. To keep my database from being fried, I recalculate my Bayesian constants once per day (instead of on every vote being cast), so as the clock turns over at midnight you will often see levels rise or drop in ranking to reflect the day’s votes.

Furthermore, Protonaut looks at your average vote cast and uses that as a weight against how much your vote should be “worth” in the first place. This makes everyone’s vote count, but chronic upvoters (or downvoters!) will be technically punished for their preference.

This also helps stem runaway-voting on levels; a popular level floats to the top of the list, everyone plays it, and everyone votes it upward because it’s good — well if you only ever vote up, the system knows this and scales your votes back a bit. Players that explore the deep content of the game (as opposed to just browsing the top ten list) get a heftier vote. If that wasn’t fancy enough, the average-user-vote value is bayesian-ranked itself!

And BAM — there we have it! A benefit for voting, that self-regulates and balances as well. It’s a whole lot harder to game this system, and it rewards the true fans of the game.

Furthermore, I optionally artificially reduce each Bayesian-generated score by how many level IDs precede it, thus making it an excellent adjustable time-based system (a truly epic level could stay at the top of the list, but most levels will disappear in the natural “churn” of things).

It really is a beautiful system. And easy to implement, too. Taken from the link below, this is the basic bayesian formula:

Use this equation:

br = ( (avg_num_votes * avg_rating) + (this_num_votes * this_rating) ) / (avg_num_votes + this_num_votes)

Legend:

avg_num_votes: The average number of votes of all items that have num_votes>0

avg_rating: The average rating of each item (again, of those that have num_votes>0)

this_num_votes: number of votes for this item

this_rating: the rating of this item

Some links for further reading

Have any questions about Bayesian ratings? Used them yourself? Success story? Horror story? Post it to the comments! I’d love to hear about it.

Protonaut gets Goombas

Andy Moore — Wed, 28 Oct 2009 01:16:00 +0000

Protonaut has gotten universal praise on one thing, and one thing only: Shooting.

There isn’t much to shoot in the game, and the feature is one almost stripped several times throughout development – but it’s hard to scrap the one feature that makes people squeal with delight.

With some recent upgrades to the sound files, shooting is just getting more and more delicious – firing that cannon truly is an enjoyable experience. But I needed something a bit more to shoot at than the occasional Nitrogen atom.

Enter stage right: Goombas. Can you tell Greg has been busy for the last few weeks? :) My art isn’t exactly befitting of the game :P

Goombas just walk left and right, and have pretty decent detection for obstacles and pits – and try to avoid them. I’m planning on make a jumper and a flyer of some sort eventually, but I’ll see how this plays out in the next few builds. They kill you on touch and don’t yet have any sort of projectile capacity.

Enemies might not end up in the final game. This production causes some fairly obvious problems, most obviously: the theme of the game is completely off with them. I’ll see how things work out in terms of gameplay and see if I need to rework the whole thing.

It’s too bad the Level Editor hasn’t gotten more attention. I really feel like the level editor is a standalone work of art, and making things is incredibly easy and seamless – with tons of tools at your disposal. It’s almost a direct correlary to my programming work not getting any notice, and the more artistic stuff getting the attention – nobody cares about the backend or the tools! It’s like I made the best hammer in the world but I haven’t found a carpenter to use it yet.

But such is the path of a programmer, I don’t really have the ability to complain about that one. :)

Signposts: New feature on Probation

Andy Moore — Sat, 10 Oct 2009 20:06:00 +0000

Protonaut seriously needs a tutorial overhaul, and I’ve been thinking about how to best accomplish this. I originally envisioned putting “signposts” in the game, with static text on them – then the tutorial would be one big level with a lot of help along the way.

Before I had a chance to put in the signposts, I did a few tests – some people couldn’t beat my tutorial level! It was long, too difficult, and not very rewarding. An utter failure!

The signposts never made it in (instead opting for an “introduction” paragraph), and I broke the levels up into 10 easier (and more fulfilling) chunks. After launch, the complaints I get were – retrospectively – obvious: You now only get tutorial text at level launch, before it becomes relevant or obvious as to what it means. You cannot recall the tutorial text except by dying or restarting. A lot of folks took the tutorial text box as a “standard dialogue” and simply closed it.

One thing that really irks me is when players don’t read, and close the box that says “PRESS Z TO JUMP – THIS IS CONFIGURABLE IN OPTIONS” then complain that they can’t figure out how to jump or that Z is a stupid choice. Of course, I would do the same thing if it wasn’t my game, so yes: You, Sir Kettle, are black.

So, here is my proposed solution: an in-game signpost. You can test it here, though the text isn’t modifiable at this point (I need Greg to whip me up an interface to do so). I’m really interested to know what you think! Chime in with any thoughts or ideas. Here are some features:

The intro-box will remain for two reasons: Need the Click-on-level-load to capture keyboard focus, and it gives level designers a place to put a little backstory up if they so desire it.
The signpost text appears on touch only – thereby not “giving away” level elements or cluttering the screen.
The signpost trigger box is sizeable any way the level designer wishes. As the text appears from the center of the box, this allows for some creative control on where text appears.
The text fades out at a variable length, dependent on the 200WPM average that people read at.
Linebreaks work and the resultant text box auto-resizes to fit.
The signposts keep track of how often they have been touched, which can be recalled with the %COUNT% variable. This can eventually tie into a badge or victory condition (in the distant future) – allowing for laps counters or other more creative ventures.
The signposts respect your keyboard control configuration with the %LEFT%, %JUMP%, %RESET%, etc. variables.
The signpost trigger box is not yet art-ed up, art is not final :)

Again, try it out here, and let me know what you think!

Protonaut Beta Launch: A Retrospective

Andy Moore — Wed, 07 Oct 2009 03:16:00 +0000

It’s been nearly a month since launch, and it’s time for a retrospective on Protonaut!

First up, I should explain away my absence for the last 30-odd days. I’d like to say I’m busy attending GDC and other business stuffs (which is partially true), but the actual reason has been crushing disappointment.

I really shouldn’t be disappointed. My brain is telling me that everything is allright, and I know things will work out in the end – as I believe I do have a viable product here. The problem is, my heart doesn’t handle disappointment well, and it’s screwing up how my brain is responding. Just today I’ve managed to bundle up all the emotion and kick it out the door, and I’m finally ready to move forward.

Let’s rewind a bit and explore what went wrong.

When I launched Protonaut, I knew I had a serious content problem. It’s the classic chicken-and-the-egg scenario; I have a game that heavily relies on user-made-content to generate traffic, and contained very little user-made-content. All of my fancy level promotion and ranking tools are useless until I have mass traffic. I decided to launch an “open beta” of sorts, and get some traffic flowing.

Colin had a tremendous amount of luck with JayIsGames in the past, for his game Fantastic Contraption. The traffic was of the perfect demographic and seemed to fit with what I wanted my userbase to be, so I got a review up on there – then halted all media interaction immediately. I didn’t want this to get out; it is still a beta, after all. The big release is yet to come!

And that’s the traffic graph that pretty much overlays over the week I was at GDC. What’s funny is I actually garnered more traffic than Fantastic Contraption did in it’s first week, and I got more traffic from JayIsGames than FC did after the review went online. My entire problem lies in retention.

That there is a delicious pie chart of where people left my game for good, their IP address never to grace my website again. 45% of my visitors were lost completely before even loading a level. I then proceeded to lose around 20% of my traffic at each stop in the tutorial, to the point where less than 1% of total traffic finished the tutorial in it’s entirety.

What’s interesting, though, is that the average user spent over 10 minutes on the site, and loaded (or retried) levels on average 6 times each. A bit more digging shows me that quite a few people abandoned the tutorial partway through to get them to the user content. That’s a lesson learned right there. Another lesson learned is that most of those people were restarting tutorial 1 over and over again – a sign that it’s too hard, even in it’s simplicity.

Sorry this one is unlabelled, but it is how many times each level was played – and essentially goes in order around the wheel – Tutorial 1, tutorial 2, tutorial 3, tutorial 4, etc…

Half the people that played #1 never got to #2. It could mean tutorial 1 is too hard, but it could also mean that the player has already decided “nope! this game isn’t for me.” A more engaging first-user-experience could correct this.

I’m a real slut when it comes to charts, graphs, and analytics. The average user only viewed/tried 6 levels, but sent me 15 hits to Analytics for clicking in various places on the screen. But the real blow to my pride was this graph right here – the one that everyone gets excited about, TOTAL CONVERSIONS:

Yes, that’s right. I’m seeing a THREE PERCENT CONVERSION RATE! *balloons fall from ceiling, cue dance music*

Well, I did on that one day, anyway. If you average out the conversion rate since launch, it’s at a still-respectable 0.4%.

Thing is, my traffic has been so low, that the numbers are heavily skewed. On top of that, I made 5 test purchases during development, and this graph is nothing but smoke and mirrors; I actually have made a grand total of ZERO sales of Protonaut after launch (except the one I paid my girlfriend to make).

Anyway, I’m putting my emotions aside now and gearing up to kick some ass. I’m going to rewrite the entire tutorial system and make the whole experience more engaging for a new user; I really want to ramp up my retention figures.

At least I’m getting “SSSL Unblocking” traffic from google!

Protonaut: LAUNCH!

Andy Moore — Sat, 12 Sep 2009 04:42:00 +0000

That’s right, after months of hard work Protonaut is finally LIVE!

We’ve decided to launch the game in a sort-of open-beta format so we can continue improving it and adding features as we get more and more feedback. We have a lot of big updates and features planned in the future and we fully expect Protonaut to become an even better product as time goes on.

I have to give a big heaping helping thankyou to our Alphanauts, the earliest testers of our game and invaluable sources of feedback. Also big thanks to Greg Wohlwend, without whom I probably would have cancelled the project a month in.

I’d love to go on a big rambly launch wrap-up post, but this is just the beginning! (and it’s also beerfest weekend here in Victoria, BC).

Don’t melt my servers while I’m away at GDC! :D

Protonaut: Launch Day?

Andy Moore — Thu, 10 Sep 2009 20:46:00 +0000

Protonaut is on-schedule for release later today, likely sometime around midnight PST. Still waiting on some music/sound effects, and we have some shuffling of levels to do in the Trials, but otherwise the game is gold.

There will be one final build (before launch), so if you want to do us a favour and do some playtesting.. now’s the time to do it!

Click here to play Protonaut.

One Week to Go

Andy Moore — Tue, 01 Sep 2009 07:01:00 +0000

Though technically launch date is somewhere around September 10th, I’ve got so much going on there’s only around 6 days of actual work left for me to finish Protonaut. That means I don’t have a lot of time to post here :)

Just launched Build 44 on the site which contains a very new introduction made by Greg. It’s pretty wicked.

Other than that – polish, polish, polish! Now that gameplay is finalized it’s just bug fixing and pretty-making. I’m about 95% finalized on the set of tutorial levels (now numbering ten), and I’m.. well, maybe only 10% done the actual game levels. I’ll get there soon enough. I’d better.

Working out Replays

Andy Moore — Wed, 26 Aug 2009 17:22:00 +0000

My current project, Protonaut, somewhat recently got a “Replay” feature. You can save your victories and share them with your friends! But how to execute the replay feature is really tricky, especially when you only have finite space and bandwidth to dish them out with.

Protonaut uses a level editor and a collection of Box2D parts – somewhat similar to Fantastic Contraption. Replays in FC are called designs, and all it is is a snapshot of the starting conditions. Since there is now way to interact with FC mid-design, you can safely let the simulation “play out” on your computer, and it will turn out the same. It’s as if the fate of your contraption has been pre-determined.

I’m not able to do this obviously. You control the player character throughout, and you can change your mind halfway through playing a level – what with having a brain and free will and all.

After looking at a few options (taking a snapshot of all the objects in the level and tweening them for example), I decided to go with recording keypresses. While a replay is running, it’s as if one of those automatic-piano-playing-rollers is over your keyboard. The game otherwise thinks it’s you controlling it yourself!

Building the data structure was simple at first. I just wanted to get it done, and I wasn’t thinking about storage requirements. Here’s an example clip of the replay file:

[...] ,,1,1,0,0,0,0,0,0,,2,2,0,0,0,-1,10,0,,3,3,0,0,0,23,0,0 [...]

I used comma deliminators and the data is broken down as:

,,Array Index, Tick#, jumpstate, firestate, (other keys)…

Keystates could be At Rest (0), Just Released (-1), or Held Down (length).

There was several problems with this setup, mostly structural, but all my fault. It was possible to hold down, say, the fire button for millions of ticks – throwing huge numbers into the mix and making it a very big and clunky bit of code. To help smooth things over, I did ZIP the file – which shrunk things down nicely.

Yesterday I decided to clean things up. I realized that “just released” and “held length” could be programatically determined; If the last frame was > 0 and this frame == 0, then it means the key was down last tick.. but not this tick. -1 for release. So it was no longer necessary to store the actual keystate, but just an up/down 0/1 representation.

That means that the keystate could be summarized in a single 6-bit number, that has 64 possible combinations. From there it was easy to find a spot on the ASCII Character table that had 64 human-readable characters in a row, and just mapped the key value to that.

Then I stripped out the commas and dropped recording of null frames (where you aren’t pressing any keys). Then I dropped the array indexes, because they were completely unused. Now the replay data looks like this:

…191J192Q200p201’202’203’204’205W300L…

Quite a bit shorter! All it is is “Tick#” followed by a character that represents the keypresses.

After running this through the ZIP algorythm, total space on the database was reduced to 1/20th of it’s original size (using a speedrun on Tutorial #1 as a basepoint)!! I’m so very happy with this… I’m sure my database will agree with me. And my bandwidth bill.

Special thanks go out to Aubrey for helping me out with BitShifting and Ryan for writing me out a psuedocode Binary-to-String function.

The Pressure to Release on-Time

Andy Moore — Sat, 22 Aug 2009 06:41:00 +0000

I am starting to wonder if I can get Protonaut out the door on time.

Greg and I set a goal to have the game in releasable form by September 14th – the date I’ll be flying down to GDC Austin. I figured this was a good a date as any – might as well have your game done before going to a game conference. As of this posting, there is just about 3 weeks left. It seemed far enough away, at the time – and our progress on the game appears to be maintaining a steady pace. But then real life steps in.

I could start with the small complaints – tonight’s programming timeslot was instead wasted fighting the first-ever porn-spammer on the Fantastic Contraption forums. I am still very proud of how the community held together and stayed sane for over a year before this happened, and I’m pretty sure this will blow over as well – but due to some technical difficulties my entire night was consumed.

I have bigger fish to fry than a single night, though. Some old chums I used to work with have roped me into covering some shifts at their new office next week. Sure, it’s part time; I can work most (if not all) of it from home; and the workload is likely light. But it’ll harm my train of thought fairly hardcore, and code will progress at a halting pace at best.

Then there is the last-minute ticket purchase for PAX. I’ve never been to the Penny Arcade Expo, but I hear it is a blast – and that’s going to consume a hearty 4-day weekend, plus some recovery time.

Then there is the Great Canadian Beer Festival, for which I am not only attending both days – but am also volunteering for, doing the volunteer orientation, and doing the afterparty. The recovery time for this is going to be even longer.

Let’s not forget that Colin Northway, old chum of mine and writer of Fantastic Contraption, is in town and we will likely hang out at least a tiny bit.

That isn’t to mention my regular routines (Aikido, getting exercise on my bike, chores, harvest season in the garden, etc.) that pretty much slims my remaining working time down to just about 7 days worth of solid effort. It’s suddenly gotten dark in here, what with all the foreboding-ness. I’m officially daunted.

It’s not all doom-and-gloom, though. We’ve hit Build 35 in Protonaut now, which sports the first incarnation of music and sound effects, not to mention a few key tweaks and the usual barrage of new menu interfaces – which has really got me excited. For the first time, Protonaut feels like an unfinished game instead of a collection of parts that might amount to something someday. Roger Levy is now on board with Greg and I to do our music, and he produced a great 8-bit Russian folk-tune to fit the Red-theme of the game today. It reminds me of Monkey Island music, it’s really sweet.

For more external influences, Greg posted to his blog about the success of his latest game, Fig. 8, and how the bidding war went down. It’s a very interesting timeline and I think it does a good job at representing the value behind proper marketing, and putting your game in the correct (bidding) spotlight. Congrats to Greg & the others at Intuition. It’s very motivating reading about other’s success.

I suppose I just have to keep my chin high and my nose down to the grindstone. I guess I’ll be coding inverted.

Thanks for listening. :)