A friend pointed me to an interesting ACM Queue article about Darkstar, Sun's open source Java-based MMOG server technology. It's a few months old, but hey, I just saw it!
One immediate reaction is to pooh-pooh Java for games, or high-concurrency servers; but the truth is, both have been done in Java before (the Star Wars Galaxies server tech was reportedly a mix of Java and C++, with most of the game logic - I think - being in Java). And anyway, as evidenced in the big debate from late '07 about scripting languages, we've been debating whether a pure compiled C++ (or C, in the case of Cryptic) or a lower-performing, easier-to-develop hybrid of C++ and Python/Lua/whatever for quite some time - with different developers taking different stances.
So "Java is too slow! Nobody uses Java for games!" is a tempting, but not entirely accurate response.1
Another reaction is one of indignation: I think linking MMOGs to stand-alone, single player games (or regular multi-player games like Unreal Tournament) ignores the strong historical connection from MUDs to MMOGs. Meridian 59, Ultima Online, EverQuest, the early games that helped define this genre are all MUD-inspired, featuring people who started as MUD players and wizards doing everything on the server. I don't particularly care for being referred to as "people who grew up on a single (PC) machine, running a single thread," for whom being asked "to master the intricacies of concurrent programming or distributed systems takes them away from their concentration on the game or world experience itself."
How Dr. Waldo describes us is, still, not a very compelling argument against Darkstar. Take a look at how some game developers talk about their players... the games still stand on their own merit, good or bad. One of the biggest problems I see, actually, is that a Java server and a C++ client (i.e., even a moderately demanding 3D client) will require two separate code bases. There will be a lot of duplicated code for game logic, network protocol, and so on, with a natural inclination to make the two look very similar (or possibly be generated from some master source, a significant engineering project on its own). I have no idea how SWG approached this problem, but they at least had the benefit of a hybrid C++/Java system where presumably the shared code stayed in C++.
I'm also very negatively inclined toward the data access layer. Everything is atomic? Collision detection and aggro calculation are just two of the things that come to mind that could lock the data for every player in a vicinity while they're accessed, and make the game stutter. Mix in the argument against geographic decomposition, and things get even worse - much, much faster than your ability to buy additional hardware.
I am pretty skeptical of Darkstar. At the same time, I expect the trend of which it's a part - making MMOG development more accessible to more people - to continue. Through a combination of middleware, new languages, and more web-centric development, MMOG clients and servers will continue to cover more of the spectrum between "EVE Online" and "Facebook."
Bleh. I should go to sleep. :-)
1. Part of me still resists: in a lot of ways I don't see Java as being particularly better than C++, especially as I've been exploring Boost further. In some ways, Java seems like the worst of two worlds: very little of the dynamic flexibility of proper scripting languages like Python or Lua or Ruby, and very little of the performance from C++ (never mind C). There are other perspectives, of course, like the fact that the learning curve on C++ is much steeper than any of the other languages; or that Java supports a view of good software engineering that the others - including C++, with its operator overloading and template specialization and so on - simply can't support the same way.
I'm not sure if it's up to doing a graphically challenging game client, who knows...
Let me put it another way: Java is, to some degree or another, slower than C++. So if other benefits came into play, like scalability or rapid development or a good server starting point (some of the things Darkstar promises), it can be worthwhile to use Java instead.
I happen to not agree that Java itself delivers much on those promises, in order to make working in Java overall a better experience than working in C++. However, something like Hadoop is well worth the price of admission; I don't think Darkstar is.
1. A group of grad students implemented a real-time multiplayer FPS with fully dynamic shadows (as a core part of gaeplay - hiding in the shadows of moving objects, for instance) in java ... 3+ years ago. Get over it. Java has performance problems (Sun is relatively slow to add compiler optimizations each time the CPU architectures add new assembler primitives) but in the vast majority of game cases its not an issue. Many many other things are going to be much much bigger performance issues during development.
2. Java would like to be to C++ much of what C++ was to C: ideally you write solutions to the same problems, but it takes you less time to do so because it appropriates the various language design lessons learnt in the years since C++ started. If you see it categorized as a scripting language its usally because the speaker doesnt really appreciate what a good scripting language is (java is far far too verbose and compiler-based for that)
Of course, in practice, Sun's stewardship of Java has been lacking. For a language that goes out of its way to take on-board the lessons-learnt from using C++ on big projects, and to be rather self-righteous about it all, to forget to include closures? Ouch.
And other shockers that haven't been fixed in over a decade of popular use - for instance, it STILL doesn't have any ability to control memory-layout at the language level, *despite* the fact that the platform level *does* provide direct access to native memory (so various people have hacked it together in user code, and it ... works ... ish. Ugh).
IMHO, though, one thing to try is making a large project with Java, preferably with a large number of other coders, and marvel at how much less painful it is than C++. Not because your best people get better (they're probably bitching about the artificial constraints of the language, and ommissions of core things like closures), but because it is exceptionally effective at letting them still write the majority of what they want to while also letting them all work together and with the intermediate people without it going to hell in a handbag. Stealing lessons learnt from C++ programmer/programmer and team/team interactions really did work well in a bunch of places.
But it'll probably just open your eyes to "wow. This *could have been* so much more. If only they'd kept X, and Y, and Z from C++" :).
Java is already in game development, and has had a very reasonable place there for some time (as do a lot of languages that aren't C++, as does C++). I'm not trying to start a flame war about programming languages - I'm working on further comments to that affect, but they'll have to wait until after hours - and I'll ask you to respect that effort and not start one yourself.
I've been creating MMO servers and server frameworks for the past 12+ years, all of which were distributed and multithreaded (that's right - POSIX threads have been around for a long time). The idea that a random game programmer can just move from writing a single-threaded single-player packaged PC game into the realm of the multiplayer server is complete nonsense. That doesn't happen. Most of the server programmers that I've worked with or known have a long background in distributed processing, network programming, and the like, before entering the games industry. So that's the end of that extremely weak argument.
My understanding of the Darkstar architecture is incomplete, but my belief is that it will not scale well to thousands of concurrent players (not to mention tens of thousands of NPCs). At some point, the communication and locking overhead will exceed the useful computation being done - this isn't a language issue, it is an architectural one. Even MMOs written in C/C++ don't turn every incoming client message into a "task" to be handled by an arbitrary worker thread - that is, to be kind, suboptimal. There is no silver bullet.
(Sorry for the anonymous posting, but I'm not going to create an account just to throw my two cents in. Matt, you can yell at me later at the meetup. -JB)
On the other hand, as he pointed out in the source article, it's a research project and they're actively soliciting feedback on it. I don't see anything wrong with exploring the space, especially while trying to build a feedback loop so that the Darkstar guys will come to learn what we already know. And, you know, we don't know everything either. :-)
As for anonymity... identification is enough, I'm not particularly enamored with LiveJournal's almost-requirement to authenticate to be honest. I may be switching to WordPress at some point, at which point that'll go away. See you tonight!
The question for everyone is, if folks were saying "C#" instead of "Java", would the argument still hold?
C# also has a lot of pluses for tools development. GUI design is much easier than any of the C++ options, and by default look at home on a Windows workstation. Between C# and Java for tools... I think it comes down to personal taste, frankly. I've used and/or written a number of tools written in Perl, C#, C++, and Java, and it really boils down to "does the tool do the job."
For servers... well, I prefer Linux servers for a variety of reasons, and I don't ever want to have to say "Mono is a core part of our infrastructure." Arguments can be made for Windows servers, and depending on the team and operations staff, it can ultimately make more sense; but right off the bat saying "we're going with Windows servers, because I think C# is the premier language for modern server development"? No.
Re: 'interesting.' A lot of programming is just programming, with different constraints and domain knowledge for particular fields. :-)
First-- the thought that Java is too slow may be true, but if so then so are C++ and C, as Java is now consistently out-performing those languages on various benchmarks. Java got the reputation for being slow when it first came out, was interpreted, and hadn't had much optimization in the VM. A lot of work has been done in the last decade on the performance. I will point out that C++ got much the same reputation when it first came out (serious work could only be done in C, C++ was wayyyy too slow), a reputation that stuck even after the compilers for C++ got lots better.
That said, I will say that I agree with your worry about having different code bases. There are ways of using non-Java code from Java, but they are pretty painful. However, by using Java (by which I mean things that generate bytecodes) we can move things around from server to server, which is part of what allows us to scale on-the-fly from one machine to multiple machines without the game programmer being involved. Which we think is a real advantage for large-scale games.
But there is a lot about what we are doing that has yet to be proved. That's why we are a research project. For those who are interested in how it all turns out, or who want to help shape the direction of the research, I'd invite you to join the Darkstar community ( http://www.projectdarkstar.com ) and help. Lord knows, we can use it...
Jim Waldo
I would say that really, the biggest problem with Darkstar is the data access layer and lack of geographic decomposition. Thinking about a generic modern MMOG, how do you calculate which monsters and/or fellow players were hit by an Area of Effect spell? From the description, it seems that a range calculation between the source of the AoE, and every player or monster active in the game, would have to be done... with some kind of transaction keeping the data in a consistent state for the entirety of the calculation. Is there something I'm missing
Regardless, thanks for stopping by! Hopefully sometime soon I'll have moved this blog somewhere that doesn't require registration or make you feel like an ass for not registering.
But seriously, back to some real issues. Darkstar doesn't preclude the use of geographic decomposition-- in fact, in the best of all worlds, games would use geographic decomposition as well. What it does is protect you from those cases where you get the geographic decomposition wrong. When lots more players than you thought (or your server can handle) end up in the same geographic space, Darkstar can spread them across multiple servers. What we are doing now is trying to find ways of doing this in such a way that those players who are interacting are kept on the same server; those who aren't go elsewhere (with the players that they are interacting with). Given that the alternative is either slowing everyone down or crashing, we think that we have a better solution.
Jim Waldo