[HN Gopher] Learning Machine Learning for Game Development
___________________________________________________________________
Learning Machine Learning for Game Development
Author : atg_abhishek
Score : 185 points
Date : 2021-03-20 13:49 UTC (9 hours ago)
(HTM) web link (ai.googleblog.com)
(TXT) w3m dump (ai.googleblog.com)
| correct_horse wrote:
| I'm looking forward to the next installment "Machine Learning
| Machine Learning for game development," the blog article written
| instructing general AI on how to generate better ML models for
| video games.
| CJefferson wrote:
| I'm in this research area, it's facinating, but also (like many
| things in AI) easy to get wrong.
|
| The trickiest bit is deciding exactly what you want your AI to
| optimise for -- If you want a game where "anyone can win", you
| end up with coin toss. If you want a game where "skill is
| everything", you can accidentally end up with "who can click
| fastest".
|
| On a project I recently worked on (and I'm writing up), we ended
| up on our 6th or 7th "optimisation function" before we found
| something where the "AI optimal" seemed actually enjoyable to
| play.
|
| Of course, if you already have a fairly fixed game, it can be
| easier to optimise constants, as you already know your target --
| but people have also been doing that for a long time previously.
| Quarrelsome wrote:
| I feel like some of these questions are a bit backwards. Isn't
| it better to give the model an arena to play within and try to
| make all of the options _viable_? I.E. Balance is maintained
| when the average of the weapons the gladiators choose to take
| into the arena are diverse and have less bearing on the win
| outcome?
|
| This means that rather focusing on what is merely broken one is
| focusing on both ends, what is broken and what is useless,
| attempting to push both towards an average of usability.
|
| Personally it astonishes me that groups like WOTC (who produce
| MTG) don't appear to be using such models given the quantity of
| mistakes they seem to continue to make in game design.
| vladTheInhaler wrote:
| Attempting to optimize the card generation process, or deck
| building, or just play itself seem like _unbelievably_
| difficult problems. Some effort at least has been made in the
| first part (see RoboRosewater) but the other two seem almost
| intractable, given the combinatorial explosion of
| possibilities in deckbuilding, the incredibly nonlinear
| interactions between cards (Splinter Twin is a 1 /10 card,
| Pestermite is maybe 3/10, Splinter Twin + Pestermite
| literally wins the game on the spot).
| meheleventyone wrote:
| For something like a deck building game, if all decks are
| viable (competitive) then there may as well be no deck
| building component. In fact a game like that is probably a
| pretty muddy brown color in terms of card variety. That would
| adversely affect the business model of selling new cards as
| well.
|
| Desirable qualities in a game in terms of how it is balanced
| are way more complex than just every option being viable and
| leading to an even win rate against a similarly skilled
| opponent.
| ericjang wrote:
| Interesting! Is there some kind of meta-objective for "AI
| optimal" that could have replaced the 6 or 7 iterations you did
| with human R&D? For instance, if you had real human playtesters
| interacting with the prototypes, is there some signal you could
| extract to measure that it's "good"?
| CJefferson wrote:
| The problem is AIs are very good at optimising what you asked
| them for, rather than what you meant to ask for, and figuring
| out what you want is super hard :)
|
| As a simple example:
|
| * Start by optimising "players can always do something on
| their turn" -- but that just ends up with everyone always
| having exactly one thing they can do (no choice).
|
| * So then say "give players more things to do each turn" --
| but then they end up being able to do everything every turn
| (the game gives them too much 'money' (still not really a
| choice)
|
| * OK, so we want to force players to make a choice -- so we
| say "No, give players as much choice as possible, but make
| sure if they choose an option it blocks off others (in
| practice, make as many sets of maximal tasks as possible)" --
| but then the AI will make sure every turn every player can do
| (for example) exactly 3 out of 6 things (any 3), and make
| sure no matter how well or badly they play they still always
| get to choose 3 from 6, so the game doesn't really progress,
| or vary.
|
| So, what we want is choice, but also variability, and
| progress, and players to feel like they are effecting the
| game, but also don't let one player run away too early, but
| also don't make it just "feel random who wins", etc.
| YeGoblynQueenne wrote:
| >> The trickiest bit is deciding exactly what you want your AI
| to optimise for -- If you want a game where "anyone can win",
| you end up with coin toss. If you want a game where "skill is
| everything", you can accidentally end up with "who can click
| fastest".
|
| In Magic: the Gathering this balancing act is achieved by a set
| of constantly changing "environments" (sets of cards) that are
| updated every few months, when a new set of cards is published.
| Sets "rotate" in and out of the various environments and
| there's a few where it's legal to play with all cards ever
| printed (excluding some that are banned or restricted for being
| OP- Over Powered).
|
| When a new set rotates (its cards become legal) in a given
| environment, there will be a reshuffle of the balance of power
| between already-established decks, that now acquire new cards
| or lose cards they needed, and some new strategies become
| available resulting in new decks being designed. Eventually the
| dust settles and the "tier one" decks (the ones that win the
| most) are found. Then tournament play in particular becomes a
| rock-paper-scissors affair, until the next update.
|
| In some of the environments were all cards from all sets are
| legal (again, minus the ones in the banned or restricted lists)
| games indeed often come down to a coin toss: the player who
| goes first wins (in one turn, through some OP combo,
| typically).
|
| I suspect that a perfectly balanced game is impossible to have
| and would probably be boring even if it was possible to have
| it. An element of creativity with constantly updated design
| parts, like in M:tG is probably the best one can do.
| vvanders wrote:
| Perfectly balanced games are plentiful, take rock, paper,
| scissors as an example.
|
| What makes those games interesting is the concept of Yomi[1]
| and learning to read an opponent. There's a decent amount of
| literature in the fighting game space and a bunch of other
| genres overlap as well.
|
| I remain pretty skeptical of ML being a primary tool here
| since a lot of Yomi is psychological and not necessarily an
| optimization problem.
|
| [1] https://www.gamasutra.com/blogs/CalebCompton/20190211/336
| 365...
| philipov wrote:
| "Nerf rock" - Scissors
| adkadskhj wrote:
| Personally, i'd like to see AI used for "simple" things, not
| often tied to the core gameplay loop itself. Ie conceptually if
| you could tie the right gameplay components to an RNG function
| then the user could get new/unexpected experiences for a longer
| time. The problem is RNG often makes crap outputs.
|
| I'd be curious to see ML take this problem. I imagine the
| examples are limitless, but one that comes to mind might be
| procedural worlds. Rather than tuning procedural worlds around
| seed values, what would it look like if we try to get an ML to
| craft the worlds? Procedurally generated worlds often have that
| feeling. Things don't make sense, etc. I imagine ML could make
| some far more diverse patterns that are both interesting and
| fun.
|
| I'd like to see this pattern of "informed RNG" in a lot of
| things. Skyrim with NPC patterns to feel a bit more authentic.
| Etc.
|
| I'm less interested in the classic AI dreams inside games, as
| it seems far out and less gamey. But our current games with
| informed RNG? That sounds.. neat.
| Jasper_ wrote:
| ML is a system that takes a large data set, and an error
| function, and finds a generated output that minimizes the
| loss. What data set and error function are you proposing for
| "RNG"?
| matsemann wrote:
| That's a wrong simplification of ML. Take RL for instance.
|
| And the parent already explained a concept: generating
| maps. It could still have an RNG as the base (noise
| function over something), but then use ML to place elements
| based on existing human-made maps.
| malka wrote:
| You feed it rng created world that were curated by humans.
| jonbaer wrote:
| I think you are right. There are SO many small mini games
| inside of nearly all RTS games which AI has yet to actually
| conquer, I will give another simple example, given a random
| map (and resources) identify chokepoints. As a human this is
| a huge clue on how to go about a game strategy (policy) but I
| have yet to find an AI do this (or optimize that part). From
| what I gather most time is spent on actually optimizing
| random build orders and modifying them along the way.
| fartcannon wrote:
| If anyone is having problems loading the page, noscript/disabling
| javascript will prevent it from displaying a white-only screen.
| DethNinja wrote:
| What if instead of spending time to train the ML model they just
| made a dummy client with it's own simple probabilistic state
| machine or behaviour tree to balance the game?
|
| How much time and resources would be spent for above mentioned
| approach compared to ML approach?
|
| I think due to statistical nature of ML, it is seen as kind of an
| hammer to every problem that might be solved statistically(which
| there are lots of them) but it might not be the most effective
| use of engineering resources.
| Dzugaru wrote:
| Anecdotally: I've never seen a "simple state machine" or
| "simple behaviour tree" in game AI. We've recently started
| using deep reinforcement learning for our games and its almost
| like a miracle how simple, effective and scalable the system
| is. There are some mentioned problems like designing rewards
| for player enjoyment, but its definitely got a massive
| reduction in engineering effort.
| Jasper_ wrote:
| > Anecdotally: I've never seen a "simple state machine" or
| "simple behaviour tree" in game AI.
|
| That's effectively what game AIs are, today. Users want an AI
| they can model and simulate in their head, and isn't too
| brutal of a challenge. Today's machine learning cannot
| provide a model like that.
| summerlight wrote:
| The whole point of this research is to early study on
| unknown aspects of game's design and its consequence on
| user behaviors and game balance. AI should be able to find
| and exhibit game play unexpected to the designer, which is
| not easily achievable with a tight control given to game
| designers. It's more of systematic state space exploration,
| not making AI fun to play with.
| wealthyyy wrote:
| Statistics nah?
| vogg wrote:
| People always pick these strategy games to use AI on. I would
| love to see someone do more of a first person shooter, or a
| Mordau type game. I'm sure it would crush real players but would
| be interesting to see two AI's compete.
| sabujp wrote:
| was hoping this game would be in the play store
| thurn wrote:
| This is super cool, but like the other deepmind projects it's a
| bit frustrating. I've got nothing against "we did something hard
| with AI and it worked well", but would greatly prefer it to be
| followed with "... and we're making this available as something
| you can use too!"
| b3kart wrote:
| It's not a DeepMind project. The authors are on the Stadia
| team.
| naringas wrote:
| the next civilization game _HAS TO_ use a deep learning based AI.
| it. simply. must.
| [deleted]
| atum47 wrote:
| I got a introduction to ML back in college (going through basic
| Algorithms) and I went straight to use them in game dev. I think
| it's a fascinating area.
| root_axis wrote:
| I'd love to see this type of work applied to 4X grand strategy
| games where the AI has historically been pretty terrible and
| forces the developers to ramp up difficulty by giving the AI
| direct material advantages for a skilled player to overcome.
| sillysaurusx wrote:
| One really cool aspect of this:
|
| _For the actual game state representation that the model would
| receive as input, we found that passing an "image" encoding to
| the CNN resulted in the best performance, beating all benchmark
| procedural agents and other types of networks (e.g. fully
| connected)._
|
| It sounds like they literally decompose the game screen into
| inputs. E.g. in FPS games, your health is often displayed in the
| exact same spot; it passes a screenshot of that into the network.
|
| Or maybe not. Maybe they decode the game state into an "image" in
| the sense that health is represented as a single pixel that
| ranges from RGB 0,0,0 to RGB 255,255,255. That would make more
| sense, but it's also slightly less exciting. Theoretically the
| model should be able to infer what health means simply by having
| enough experience, even if it's a more complex representation
| like Arabic numerals rather than a handcrafted input.
|
| Anyone know if there are any other details, like model weights
| (ha ha, not likely), an architecture diagram, a paper, or some
| code snippet from some prototype that the researchers used for
| inspiration? The "image" representation is really quite
| interesting to me, since I hadn't thought about feeding data to
| networks that way. Theoretically a GAN could learn a thing or two
| about the world from having this sort of "image" input too.
| kevinstubbs wrote:
| Cool to see this. I and surely many others in game development
| independently thought of doing this 5+ years ago. Fascinating to
| think that no matter how much software the world writes, we will
| keep finding entirely new ways to add value by writing even that
| much more code.
| WORLD_ENDS_SOON wrote:
| This sort of research is maybe less flashy than say using machine
| learning to automatically generate game assets from photos, but I
| think this sort of computer-aided game design is possibly the
| biggest way machine learning will transform video games. As games
| are becoming bigger and more complicated, the problem of tuning
| various gameplay parameters explodes exponentially. And, this
| sort of tuning can have a huge effect on player retention and
| overall game quality.
|
| In this research the machine learning is being used to balance
| the game across different asymmetric strategies (different decks
| in the card game), but you could imagine using similar techniques
| for balancing and tuning content for single player games as well.
| Once you have a reasonable model of the player's behavior, you
| can do all sorts of automatic tuning like balancing the
| difficulty of jumps in a platformer, tuning enemy positions in an
| FPS, etc.
| Jabbles wrote:
| Is this from the part of Stadia that got cancelled? It sounds
| genuinely innovative and seems astonishing that they would back
| out of game development.
|
| Or is this what they meant when they said they would be
| "concentrating on Stadia as a platform"?
| sfjailbird wrote:
| The most obvious use of ML in this context would be to predict
| player actions in multiplayer games. Most such games extrapolate
| the actions of remote players in order to give the illusion that
| there is no lag between updates. I bet for many situations, a
| good ML implementation could make this work almost flawlessly. No
| idea if anyone is doing this yet.
|
| Also, the headline is wrong: It's 'Leveraging Machine Learning
| for Game Development'.
| Synaesthesia wrote:
| So how will it be able to predict if I will turn left or right
| next?
| sfjailbird wrote:
| With a training set of all previous sessions played, I bet
| this would often be easy to predict.
| viktorcode wrote:
| The approach as described has one major downside. It is based on
| presumption that human players will employ the same strategies as
| a neural network which learned to play the game. It _may_ be the
| case, but in reality many imbalances in the game model remain
| undiscovered or unused by the real players, for various reasons.
|
| The good example of ML for playtesting is what King is doing with
| their Candy Crash Saga. They have trained neural network on real
| world usage data, from millions of players. That makes it behave
| like a real player too, not pathetically weak, and not inhumanely
| strong. This, if applicable for your game, is a better way to
| leverage ML.
|
| There are other examples of neural networks finding highly
| unorthodox strategies in various games, when they learning to
| play it. It is nothing like human behaviour.
| kjgkjhfkjf wrote:
| Trust Google to produce the most over-engineered online card game
| ever.
| jayd16 wrote:
| Still seems more approachable than Artifact.
| Yenrabbit wrote:
| Letting these ML agents loose on your game is also a good way to
| discover bugs in the implementation that a human player might
| never run into. When I first messed about with RL[1] I quickly
| discovered this, with my 'AI' learning to abuse invalid moves to
| get a re-roll of the dice or to stay put when it should have been
| forced to move. [1] -
| https://datasciencecastnet.home.blog/2020/01/24/swoggle-part...
| denysvitali wrote:
| You probably mistyped the title. The original title is:
| LEVERAGING Machine Learning for Game Development.
___________________________________________________________________
(page generated 2021-03-20 23:00 UTC)