15

I've got a small handful of competitive word games in progress, and while the preference is for (mostly asynchronous) play against other human opponents, I'd like to provide players the option of playing against an AI. I have my dictionary and I can easily give the AI full dictionary knowledge while it's playing, but my concern is that having the AI regularly playing words they're not familiar with will be a frustrating experience for players: 'I would have won that game if it'd just used words I know!' — even if the AI's overall skill level is turned down.

I'd rather create a weaker AI through a combination of (un)tuned play parameters and a weaker vocabulary — but I'm not sure how to limit that vocabulary to 'common' words. I've looked at several word frequency lists (for instance, the list of all words that appear in the Project Gutenberg books, sorted by number of occurences) but they all have a number of false negatives: words that everyone knows that simply don't show up with any real frequency (for instance, CHEETAH shows up less frequently in the PG texts than VOCATIVE or SUTTEE). I've tried using search results to get estimates of a word's popularity, but they also tend to be prone to spurious mis-estimates, and of course it's hard to get search results for an entire dictionary without running afoul of the terms of service on the search engines.

Does anyone have suggestions on other good means of determining a rough frequency of word usage, or other ways of limiting word game AI that will feel natural to players?

Steven Stadnicki
  • 4,026
  • 18
  • 26
  • 6
    you could let it learn! Every time the player uses a word it doesn't "know", it could have a 6/10 chance of learning it, otherwise it learns a new random word! (Thus it usually will be learning words the player knows, yet slowly bringing out words the player might not (education!!))

    Of course, this would mean you'd need a "AI dictionary" for every player.

     – Joel Jul 18 '12 at 06:50

3 Answers3

7

I think your solution would still be to go with some word frequency list. The Project Gutenberg might not be the one for you as it contains mainly older texts where the copyright has expired, giving strange frequencies for modern english speakers. I guess there was a time when "vocative" was actually a more commonly used word than "cheetah"...

So I think the solution is to find the right frequency list. Searching around a little had me stumbling on the 5000 most used words in all Simpsons episodes, that might be a bit more up to date. You might want to cross reference this list against a real dictionary though to sort out words like "krabappel" for instance :)

Hope it helps, I'm excited to hear what solution you go with!

Mikael Högström
  • 1,705
  • 13
  • 21
  • 1
    My strong impression is that the issues with 'words everyone knows but no one uses' are still going to persist with any dictionary, but it may well be the best I can do - and of course, there's a good chance that players won't even notice the easy words that the AI doesn't know, just the hard words that it does. – Steven Stadnicki Jul 19 '12 at 15:28
  • +1 You may be right but why is that? Generally it seems to me that people should learn the words they hear often. If you are correct it would be interesting to know the reason... :) My hypothesus would be that it is because of a bias in the used corpus, like in the case of the Gutenberg project so I'm sticking to the solution that you just need the right frequency list. – Mikael Högström Jul 30 '12 at 18:52
5

Crowd source it. Use all the multi-player games your players play to build up a word frequency table. I think any analytics service could help collect and organize this data. You could even weight the entries by how "good" the player using the word is.

P.T.
  • 625
  • 5
  • 9
  • I like this, but it still feels like it's solving the 'update' problem rather than the 'startup' problem (assuming, of course, that I want to launch the multiplayer and solo games side-by-side, which is a different question entirely). – Steven Stadnicki Jul 19 '12 at 15:25
3

Start with basic vocabulary according to your AI level. For example let below is your AI level list.

  • basic : 1000 words vocabulary
  • average : 2000 words vocabulary
  • hard : 5000 words vocabulary

You may choose this words from following web sites which you can see with google search.

These sites will give your starting words but I think google n-grams may be better choice. Anyway do not worry to much about your starting words, with updating/adding mechanism explained below your AI dictionary should straighten itself according to your players.

since your players learns while they play this game your AI should learn too. Allow it to learn most used words by their opponents.

For example 10 persons played against your AI basic, they used 100 unknown words, that is words which are not in your AI dictionary. Choose most used words and add them to your basic AI dictionary, also to your average and hard dictionary. You should also choose to update some words which are not used in your dictionary with words people use. This way your starting dictionary will be more suitable for your players knowledge. You may start with fewer words/bunch of different words but your AI will learn while they play against humans.

You may also consider starting your AI with basic level and upgrading it to other levels only when it learnt enough words.

Atilla Ozgur
  • 133
  • 5
  • This gives a decent way of updating the vocabulary, but it still leaves the startup question - where do those initial vocabularies come from? – Steven Stadnicki Jul 19 '12 at 02:49