Anson the Gnome

Folks have started talking about the Data Access API for Pirates of the Burning Sea. Of course, this is an area of interest for me, and I have expressed some minor opinions on it. Joe Ludwig was kind enough to respond on that same page, and Darius K weighed in at his official work blog. As usual¹, I'm going to talk in more detail here rather than somewhere else because it's easier.

Working on the XML for Dungeon Runners, we considered something very similar to what Flying Lab is doing, for very similar reasons - the ability to monitor individual applications, to revoke misuses of the data, and so on. Now that they and we are doing it differently, I don't mean to criticize them on the subject either... just discuss it, with the understanding that they prefer their solution and we prefer ours. :-)

One reason we forewent API keys and developer identification was the simple principle about simplicity: KISS. More moving parts means more maintenance and more points of failure (and given our budget, single points of failure). I suspected at the time - and this has largely turned out to be true - that if something needs to be designed or coded or produced for the Dungeon Runners XML system, I'm going to be the one to do it... along with everything else I do.

Another reason was that we weren't already providing a way to access this data in-game (e.g., using an /inspect command or something like that to view another character). We didn't want to create a situation where most of the players were waiting to get the data (either another player, or us). The raw XML is readable, so the players who write applications using the data can't themselves be gatekeepers deciding who can see what. Perhaps that would actually be desirable for Burning Sea's PVP, I'm not sure.

Of course, "low barrier to entry" is always a concern for a game like DR - one that probably isn't as big a deal for Burning Sea, probably - and we want to encourage as many people as possible to look at and play with the XML.

Finally, we feel pretty confident that we can accomplish the sames goals as Joe mentions on an IP basis - find out what external systems are asking for XML, how much, etc. See what sites are asking for XML, how often, etc., and block IPs if anyone gets rowdy. Not a perfect solution, but a pretty good one, considering the other tradeoffs we had to deal with.

And yeah, we're struggling with the questions of privacy and PVP data exposure too. We haven't exposed many of the things we'd like to, simply because we haven't hit on the right way to do it yet. When we have those answers, you'll see another surge in data exposure from us, without a doubt. :-P

---

1. It came up when discussing Cryptic's Cryptic DB, and it still holds true: LiveJournal provides much better word processing facilities - even when I'm using raw HTML and not their WYSIWYG rich text editor - than most blog software's facility for leaving comments. My posts also don't go through a spam/offensiveness moderation stage, and I can include links (I never know whether links are being accepted on most other blogs until after I've written the comment).

Also, the comment facility here is actually better (allowing links and images, providing a sizable text input window, threaded discussions, comment preview, etc.) for continuing discussions. Finally, thanks to OpenID, people can more directly connect their comments here with their blogs elsewhere.

I'm splintering the discussion going on at T=Machine (and to a lesser extent, Jeff Freeman's blog) because the discussion is already fragmenting, and I prefer typing long pieces of text in LiveJournal to comments in WordPress. Shannon has also reposted one of his responses in a previous post of mine, which also covers some of what I'm going to talk about here.

Before working on Dungeon Runners, I worked on the NCsoft billing and web tools team, and I actually did some work on the back end process for City of Heroes in-game petitions (the CoH game server would put a petition in the database, our process would poll the database and use XML-RPC to submit petitions to Right Now CRM software, which the GMs and CS people used to view petitions). I can say first-hand that at least some of the internal fields that Cryptic had to expose were invaluable, because it meant that the petition processor could provide a lot of data to Right Now beyond just the text of the petition.

On Dungeon Runners, we seem to have the same experience - some of the data are important for customer service tools, but most are not. If they're important to customer service, we try to (or plan to) expose them. We also provide a network interface and protocol documentation for requesting changes to the DB (to preserve cache consistency). That doesn't mean a mandate to expose all internal data is reasonable, but there is definitely value to exposing some of the data - and in a way that the developers writing the tools understand, which means accessible via SQL.

We're also working on providing more information for data mining and web display, but the trick here is that this data doesn't have to be perfectly up-to-date, so we can update it in the background without delaying interactions with the live game. Again, the value of this is that data miners, designers, and so on can make sense of the data without my constant involvement. We don't lose any performance by doing this work lazily and keeping it separate from the authoritative blobs, and our total dev time is reduced because we can use so much existing database infrastructure. We also win by having only one storage engine for all the data.

I touch on it above and in my other blog posts, but another advantage bears further explanation: we can easily replicate the relational data to another database that web servers talk to, and suddenly we've got a wealth of live data out on the web. Well, so far, it's not a wealth of data, but we're constantly improving it, and it's easy to improve because the steps involved are a) store the data in the database, b) write ~10 lines of PHP (mandated, sadly) to pull it out of the database, and c) drink a beer.

My feeling is this: there is enough benefit to having game data in relational format that it's worth using as a starting point. On the other hand, there are enough hurdles involved in reflecting game state in a relational database that you have to be careful. I don't fully agree with Cryptic's approach, but that's mostly because I think they go a little too far - not because I think they're going in the wrong direction.

Maybe one of these days we'll see an MMO that has a 1000-machine Oracle cluster with a 3:1 machine:DBA ratio, and they'll prove us all wrong on how to use databases in games. Until then, the debate rages ;-)