noSQL - Is it a valid option for web based game?

Question

Out of an opportunity and boredom, a friend and I decided to make a web based game. This is the first 'game' I will be making, since usually I program web apps in django.

I've opted to use the same framework for the game, but I am not totally sure about the data storage, since I cannot foresee future requirements and DB related issues. Lately, the noSQL movement is gaining force and I would like to explore that area.

Therefore I have the following questions:

• Would a noSQL datastorage (document-based, such as MongoDB) would be suitable for a web-based game?
• What are the issues that might arise using a conventional RDBMS, such as MySQL?
• How to insurance data consistency amongst users with MongoDB, during gameplay or at timed events, such as cron jobs?

Game overview: Typical adventure game, with the player fighting monsters and gaining items and such.

• Some data is static across all players: items, weapons, potion, maps, etc.
• Some data tied to the player: inventory, metrics (stats), progress, etc.
• A player might need to know the stats and state of another player
• Scheduled tasks will be run at certain time interval

Answer 1

Would a noSQL database be suitable for a web-based game?

Absolutely! Generally speaking, non-relational databases (such as MongoDB) are much better for games, as they are more flexible in how they model data, while being more performant than relational databases (such as SQL) - making them a "win-win" choice.

What are the issues that might arise using a conventional RDBMS?

There are two major drawbacks to using relational databases:

• Lack of flexibility in the way you model data
• Performance

Lack of flexibility in the way you model data is a major drawback, because using a relational database forces you to model data to fit the needs of the database, rather than the database accommodating the needs of the model. For example, consider you're creating the model for how you'll store items in a game using a relational database and you decide on the following model:

weapon_type_id | weapon_type
1 | sharp

weapon_id | weapon| weapon_type_id | damage
1 | Short Sword | 1 | 5

potion_type_id | potion_type
1 | HP
2 | Mana

potion_id | potion| potion_type_id | modifier
1 | Healing Elixer | 1| 5
2 | Mana Drainer | 2| -5

Now consider how you'd need to modify the model to meet do the following:

• Add a new item type
• Create a weapon with multiple weapon types
• Create a potion item that can also be used as a weapon

Certainly all these actions are doable, but you won't be the happiest person while you're doing them, and you'll probably wonder, "is there a better solution out there for what I want to do?," when you could design a much more flexible model in a non-relational database.

Note: If you are unfamiliar with how document-based databases store information, please check out this link before continuing. The model presented below designed to use similar logic to what is presented in the aforementioned article.

db.itemTypes

name: Weapon
types: Sharp

name: Potion
types: HP, Mana

db.items

name: Short Sword
weapon
    type: Sharp (db.itemTypes reference)
    damage: 5

name: Healing Elixer
potion
    type:  HP (db.itemTypes reference)
    modifier: 5

name: Mana Drainer
potion
    type:  Mana (db.itemTypes reference)
    modifier: -5  

name:  Healing Potion Sword
potion
    type: HP (db.itemTypes reference)
    modifier: 10
weapon
    type: Sharp (db.itemTypes reference)
    damage: 15

There are a lot of good articles on the performance of NoSQL vs SQL databases, but here is one which provides application examples so you can perform your own tests: MongoDB vs. SQL Server 2008 Performance Showdown

How do I insure data consistency amongst users with MongoDB?

MongoDB supports read/write atomic operations on single data entities (documents), so query results from MongoDB should always be accurate with what is stored within the database.

Edit: Provided NoSQL example of item database

Answer 2

Couple of words do defend SQL databases.

1 - If you have SQL database you can work with your data not only by primary key. Most of queries in MMO goes by PK but when you need to find all users with level > 30 what will you do in NoSQL world?

2 - If you have SQL language you may create "hotfixes" to repair broken data. For example: "update player_items set flag=0 where item_type_id = 100". How can you fix this in NoSQL? I think you will have to make a script which will parse all your data...

3 - SQL databases are not so slow as you can think. My collegues create the web based MMO game, wich make 5000 transaction per second in MySQL. Is it not enough for you? If not, you may use sharding to scale your database.

4 - SQL has a foreign key constraints and such good stuff, which will help you to find bugs in your code.

NoSQL is not a silver bullet. It solves only couple of problems and brings many new problems. So my advice - think twice before choose NoSQL.

Answer 3

The answer is yes, it's a valid option but no, it probably isn't a great idea.

There are two major problems with SQL that NoSQL tries to address, when it comes to games.

The first is latency, or lag. In one sense, SQL databases are blazingly fast, processing thousands of transactions or more in tenths of a second. In another sense, they're incredibly slow, because processing just a few transactions can take that same amount of time. For most database uses, this doesn't matter, but games have a real-time component so it's not just about how many transactions you can do per second, but about the average time it takes to respond to a database transaction.

This latency is a major problem for real-time games. Many developers that need large databases (usually for MMOs) build out a caching layer that sits between the database and the game servers in order to avoid these stalls, and then flush the cache periodically. The problem? You just threw out the main reasons to use a database - atomicity, consistency, isolation, and durability.

But that problem doesn't apply to web games. You've already got a high-latency connection between the player and the game, so you might as well get the throughput SQL provides, as well as the benefits of mature, well-known software.

The second problem, however, does apply to you, and that's the object-relational impedance mismatch. Games deal with objects, databases deal with rows. If you're lucky an object can be made into a row just by listing its fields, but most of the time objects are hierarchical, and you've got to flatten them out in some way to get them into rows.

Document-based databases, like CouchDB and MongoDB, solve this problem by promoting the document to the top-level object, rather than row ("document" in this case is a synonym for "object"). But by doing this, they lose many of the benefits of SQL databases. The throughput is much lower, and the locking algorithms become much more complicated.

The good news is, there are already a lot of object-relational mapping (ORM) tools available for whatever language you're using. They let you translate between objects in memory and rows in the database fairly transparently. These tools are generally not appropriate for large-scale real-time games, because they can present the worst performance aspects of SQL and NoSQL databases. But it's fine for a web game - at worst, when you run into performance problems, you can go back to doing transactions the old-fashioned way. The latency problem doesn't hurt you, and the increased throughput helps you.

Finally, to belabor a point I've made elsewhere: This isn't about static game data. I'm not talking about your item descriptions or your weapon damage or sprites or anything here. I mean the real, mutable game data, like what's in a player's inventory or what their stats are. All you need for the static data is a data store. Maybe it turns out that the easiest thing to do is use the same database as the data store for that because you have a great ORM; maybe you'll just need the filesystem. It's two separate problems, and one answer needn't (and probably won't) fit both cases.

Answer 4

• Would a noSQL datastorage (document-based, such as MongoDB) would be suitable for a web-based game?

I'd advise to give a look at couchdb

Its interface is javascript-based, so it will blend pretty well with HTML5/javascript based games.

It is also able to work 'off-line' (making a local copy of the db) and then synchronize itself with the server later on (you might use this for instanced dungeons, for example)

• What are the issues that might arise using a conventional RDBMS, such as MySQL?

The main issue would be that no-sql databases handle pretty differently from relational databases. Making the mental switch can be difficult.

For example, take a look at how "queries" are done in couchdb. Everything is done in javascript.

• How to insurance data consistency amongst users with MongoDB, during gameplay or at timed events, such as cron jobs?

The process of 'synchronizing' databases is called replication in couchdb's lingo. You will also see the term consistency a lot. There are several built-in services that deal with replication & consistency. See for example the incremental replication process.

Answer 5

Depending on what you have in your game you might end up using both, and connecting them through the models in your game. For example player could and should be in RDB (login info) but player inventory/variable storage could go to noSQL , so your player module could login() using RDB and fetchInventory() using noSQL by the primary key.

Maps for example better to be saved in NoSQL (coordinates => array of different objects for example) i do not imagine saving what an area contains in RDB (except a serialized string which you need to fully unserialize to read some part of ? more CPU and RAM wasted !) well you could still save areas in RDB for example area "city X" contains the following cordinates/block ([3,4],[8,7] ...)

you could and probably should use both, and take a look into volatile storage like memcache which you might need or some temporary data (would be faster than using hard-disk for sure ! and saves you a lot of CPU but sure not RAM)

so in the end >

it depends on what you want to have in your game ! cheerz