Lets get the fun back... with metrics?

I'm a professional software engineer.  Software Systems actually, I have never written code for a living.  Anyway, in the software world, metrics are a controversial topic.  For those who aren't familiar with metrics, they are ways to measure your work.  For example, to measure productivity you measure lines of code written per hour.  To measure test effectiveness you measure number of bugs found in a series of tests divided by the number of bugs that are found later.

Who remembers this?

It's sort of anti-metric... (and if you're mad at a certain CEO, freeze it at around 2:37)

But do metrics work?  And if you're working on a game, can you turn fun into a metric?  My experience is that metrics only work when you can measure what you really want.

Well sometimes we say that BP is getting less fun, so I started thinking - what can we measure to figure out "fun"?

I came up with three things:  difficulty, time, and engagement.

Difficulty: This is actually easiest to measure... just how much repair time per target (or per points).  I've been talking about this lately in my raid articles - when I saw that this raid cycle was giving me about twice as much repair time per point as the previous cycle, I concluded that this raid was about twice as difficult.  Players generally want this to be as low as possible, and Kixeye increases their revenue (ignoring participation) as this increases, but really there's probably a best value for this somewhere in the middle.

Time: This is also easy to measure... how much time does it take to beat a target?  Longer times result in players feeling like this is a grind, but targets that are too quick result in players not playing and getting bored (as well), since they don't get much time in actual gameplay.

Engagement:  This is the hardest of the three metrics to describe and thus hardest to quantify.  I'm thinking about engagement as ... how interesting are the targets to hit?  Are they boring or fun?  Time can enter into that, but the measure I thought of for this metric is time-independent.  To measure engagement, you can determine the ratio of damage taken on auto (when beating a target) to the damage taken when driving skillfully.  This metric should generally work out as higher is better, since a target that can be beaten on auto is not that interesting and takes no skill to fight.  As an alternative, clicks (user input) per time might be a decent engagement measure.  Sitting and waiting for a cavatator field to blink off isn't too engaging.

So I'm sure some of you are thinking - this is silly, these numbers are going to be all over the map for different players.  But since we are looking for a good user experience across the whole community, Kixeye really can measure the average and see how the player base is doing overall.  They could split it by level or coin spend.  The Engagement metric is a little tricky to measure from live data, but probably could be measured in the test environment.  

Averaging makes it even more powerful... one on the theories batted around is that the changes in the Reaver targets are based on the fact that players could use Ghostcrawlers and prep out targets for free points.  But if the number of players doing that wasn't too high, then the averaged metrics wouldn't be affected too much... and we can all just move along with the raid.

But just to play around with this, lets think about some of what is going on with our current BP targets through this lens.  Although I don't have access to player data, I can make some generalizations.

Rise of the Warlords: 

The latest event featured targets that were pretty much vulnerable to subs only and alternating cavitator fields to force some careful driving.  Much of the damage in these targets was driven by death explosions, forcing some strategy in deciding how many subs to bring around.  For players who kept up and had at least 2 Hellwraiths to work with, the way I would assess the metrics is:

Damage: Somewhat high but not totally unreasonable
Time: A little long for A sets, the careful driving forced by the cav fields caused the targets to take a while to complete
Engagement: Very good (using ships on auto wasn't a good strategy) but maybe also a bit too high, as the problem solving combined with the time required just limited the useful play time.

The latest changes we saw in the VXP targets, unfortunately, seem to be moving these metrics in the wrong directions... Damage went way up (obviously) and Engagement went down, basically because there was no smart way to drive against those targets.

Civil War cycle:

The last raid cycle featured Mega Hulls surrounded by a mix of turrets and ships. 

Damage: By the end of the cycle, I think damage on this raid was right around the 'happy medium'.
Time: A little long for A sets, and this was driven by massive maps.  It drove people to hit S sets (except on the last event of the cycle where damage got turned up to unreasonable, and so many people took more time to hit As.)
Engagement: Here is where this style shined - the targets were autoable, but good driving and strategy did allow a lot of damage mitigation.

Base Hitting:

For those who don't do it - you're missing out.  Base hitting is really fun, and I think it's getting even more fun with the crop of conqueror hulls and the options available.

Damage: It was getting out of control for a while there, but 30 minute - 1 hour repair times on Conqueror hulls have really turned this problem around.  Defense turrets and guard fleets are doing some damage, so base hits aren't free if you are hitting a player of comparable 'power'.  
Time:  5 minutes max... with many hits being done in less than 3 minutes.  Higher ship speeds is helping.  A base hit is usually quick & satisfying.  Preps that would drive up the time are a lot less common these days with the rise of sub defense and the nerfing/obsolescence of sub-based prep weapons (launcher / deluge). 
Engagement: You can't auto decent bases, and the strategy of base hits has really grown with the variety of damage types, deflection defense, and remote targeting.  Mixing up fleets and thinking about which ships to bring forward is the norm among good base hitters.  Click-the-back-of-the-base isn't happening too often, although some Fury owners are having success with this right now.

Furies aside, I think base hitting has been right in the sweet spot for a while now.

Weekly Forsaken Mission
We just had a shift from the 107s, where the avoiding the Angry Arc missile was the significant strategy driver, to the 109s, where most damage is incoming from missiles that are difficult to shoot down and Coldsnap rockets that are impossible to shoot down. 

Damage: Players were getting it down with the 107s, but the shift to 109s has ramped this up a bit.  With just a single tank taking the brunt of the damage, it is much more difficult to knock out the mission in a single evening without coining a repair. 
Time: Pretty good here - 3 109s and a few more points will get a round done instead of 5 107s.  The complaints about targets being gutted is really about time... if you are hunting around for a target to hit, it simply takes longer to get the mission done.
Engagement: Pretty good here, some thought should be given to approach angles and pathing through the targets to avoid some unnecessary damage.  Engagement in any target like this will go down over time as it becomes more and more of a rote pattern.

This post isn't going to help you play the game any better, but hopefully this helps us (and Kixeye) think a bit more about the fun (or lack of it) in this game, and WHY some targets are more fun than others.  I still owe you pirates a good look at the new FM targets and the latest crop of prizes, but I thought this article was a little more important to put out there.