Anson the Gnome

So, game metrics...

There are a few kinds of game metrics I can think of, when the subject comes up:

• Operational metrics
• Game state snapshots
• Event logs

Operational metrics: live data about the condition of the server. We provide a little bit of this, and we could provide more (and occasionally we add to the set of server metrics we provide). However, it doesn't seem like an area ripe for innovation.

Game state snapshots: this is the sort of data we provide with the character XML feeds - what your character looks like right now. If we save enough historical data (or generate and record reports on the data at regular intervals), then we can actually do a lot of analysis on trends over time. If we only save reports, and purge snapshots periodically, some information is lost, but we gain most of the benefit at a fraction of the data storage cost. Better, any report you know ahead of time you want to do can easily be reported in each snapshot.

One downside of how we're currently doing snapshots (storing the generated XML) is that it's very costly to actually do analysis of the whole data set (Essentially, it relies on parsing all of the XML each time, and traversing the in-memory document, rather than relying on the database server to keep the data collected by type and constantly organized. (With SQL Server 2005 we could conceivably work around some of these limitations by using the XML data type, which tells the database to do the parsing and organization... I have not fully explored this option yet.)

Event logs: you can infer events from changes in the state of snapshots (so far, even most of our internal metrics are snapshot-based, so we've done a lot of work in this regard), but you can also rebuild state based on events (again, it can be a real pain). In either case, though, you can't get one from the other quickly - if we were generating character XML based on replaying all the events relevant to a character, we could not keep the web data as up-to-date with the database view as we do.

One thing you can't get from event inference (starting with snapshots, and figuring out what happened between two snapshots) is the precise ordering of events... which in turn lets you reconstruct overall game play at a later date. You can answer one-off, unexpected questions as they come up, which can be a boon to bug hunting and customer feedback. You can also do neat things like RSS feeds in real time (where 'real time' means "predictable, very short lags between the event and its presence in the feed").

It really comes down to that - what is the best use of your time? If you know you're going to be asking a question regularly, record the answer to the question, not information that lets you extrapolate the answer. Imagine calculating PvP ratings by wandering through each PvP match, and calculating who the winner is based on who died how often... clearly, the answer is to at least record "who won" each time. But if you're displaying the PvP rating in-game, then it's clearly calculated in-game... so just record the rating as data on the player and don't ask your reporting team to calculate it later.

One interesting thing I've gotten to see, being at NCsoft, is games that try to focus solely on just event logs or snapshots, and ignore or pish-posh the other. So far, it seems pretty clear that either one on its own fails to provide some of the information (or at least, easy access to information) the other approach could easily provide.

I'm splintering the discussion going on at T=Machine (and to a lesser extent, Jeff Freeman's blog) because the discussion is already fragmenting, and I prefer typing long pieces of text in LiveJournal to comments in WordPress. Shannon has also reposted one of his responses in a previous post of mine, which also covers some of what I'm going to talk about here.

Before working on Dungeon Runners, I worked on the NCsoft billing and web tools team, and I actually did some work on the back end process for City of Heroes in-game petitions (the CoH game server would put a petition in the database, our process would poll the database and use XML-RPC to submit petitions to Right Now CRM software, which the GMs and CS people used to view petitions). I can say first-hand that at least some of the internal fields that Cryptic had to expose were invaluable, because it meant that the petition processor could provide a lot of data to Right Now beyond just the text of the petition.

On Dungeon Runners, we seem to have the same experience - some of the data are important for customer service tools, but most are not. If they're important to customer service, we try to (or plan to) expose them. We also provide a network interface and protocol documentation for requesting changes to the DB (to preserve cache consistency). That doesn't mean a mandate to expose all internal data is reasonable, but there is definitely value to exposing some of the data - and in a way that the developers writing the tools understand, which means accessible via SQL.

We're also working on providing more information for data mining and web display, but the trick here is that this data doesn't have to be perfectly up-to-date, so we can update it in the background without delaying interactions with the live game. Again, the value of this is that data miners, designers, and so on can make sense of the data without my constant involvement. We don't lose any performance by doing this work lazily and keeping it separate from the authoritative blobs, and our total dev time is reduced because we can use so much existing database infrastructure. We also win by having only one storage engine for all the data.

I touch on it above and in my other blog posts, but another advantage bears further explanation: we can easily replicate the relational data to another database that web servers talk to, and suddenly we've got a wealth of live data out on the web. Well, so far, it's not a wealth of data, but we're constantly improving it, and it's easy to improve because the steps involved are a) store the data in the database, b) write ~10 lines of PHP (mandated, sadly) to pull it out of the database, and c) drink a beer.

My feeling is this: there is enough benefit to having game data in relational format that it's worth using as a starting point. On the other hand, there are enough hurdles involved in reflecting game state in a relational database that you have to be careful. I don't fully agree with Cryptic's approach, but that's mostly because I think they go a little too far - not because I think they're going in the wrong direction.

Maybe one of these days we'll see an MMO that has a 1000-machine Oracle cluster with a 3:1 machine:DBA ratio, and they'll prove us all wrong on how to use databases in games. Until then, the debate rages ;-)

One of the interesting - or at least, very painful - parts of writing games is developing the user interface. Most modern PC games have a complete, from-scratch user interface system, at least fake one pretty well. Dungeon Runners, unfortunately, has neither... which presented several challenges in the upcoming (and in-progress) update to the social window.



There are a number of UI elements in that screen shot that simply don't exist in the game as it is currently available to players - check boxes, for instance. Column headers are another good example. Previously, one of my compatriots had his work cut out for him simply making some windows (the pop-up item windows you get from clicking on a linked item) draggable. On the plus side, developing a decent UI gave me the opportunity to actually work on the UI code - not just adding elements to existing UI, but actually improving the library so that the better UI elements dumped into the social window are available for all future UI development.

To me, designing UI widgets that are general enough to be used in multiple situations, but specific enough that making it useful in each case doesn't require a ton of boilerplate code, is an interesting problem. For checkboxes, it's not that difficult: track 'on/off' status, and support a variety of textures for both situations (and some special situations just in case, like when the mouse hovers over an element, or the element has keyboard focus, etc.).

Column headers are a bit more interesting - you have to know what columns there are (preferably automatically, by virtue of putting them there in the design tool), you have to know what the most recent sort direction for each column is, and what the active sort criteria is. After that, it's a fairly simple matter to use that information to inform a StrictWeakOrdering functor with which to sort. Oh, and you have to update the UI based on sort status, but when you have the information, that's pretty easy. :-)

An interesting wrinkle was pointed out a week or so ago, and I haven't gotten around to mentioning it here: there is actually less uncertainty about a Dungeon Runners player's performance than a chess player's performance. The big reason is that there is still some small inherent advantage afforded to higher level characters (due to the greater variety of skills available at higher levels, and the longer durations or more powerful effects of some skills), and there is a definite advantage to being higher on the power curve for your level (which is mostly accomplished by having higher-grade equipment).

I'm still sorting out how to handle this, but it's an interesting problem - there's ranking, and then there's overall power level. Do we just set up power bands? Do we make it possible for a very skilled, but underpowered, character to take on a more powerful (but less skilled) opponent (or team)? I'm not sure, but fortunately... guilds have to be implemented first. So I have some time to think about this. :-)