[Seminar] AlphaGo in Chemistry – Solving Real-World Problems by Means of Game AI Methods

On Tuesday 22nd May 2019 the Game AI Group will host a seminar by Mike Preuss from Universiteit Leiden

Title: AlphaGo in Chemistry – Solving Real-World Problems by Means of Game AI Methods
Speaker: Mike Preuss, Universiteit Leiden
Time: 4pm to 5pm, May 22, 2019
Room: 3.01, Bancroft Road Teaching Rooms, Mile End campus

All welcome (especially students), no pre-booking required.
The seminar will be followed by drinks in The Hub.

Monte Carlo Tree Search (MCTS) and Deep Neural Networks (DNN) in combination have pushed the limits of what artificial intelligence (AI) can do in areas where humans have been perceived as dominant over machines, as for the game Go. However, so far this has been limited to domains with simple and known rules, such as board games, where perfect world knowledge and cheap environment simulators are available. Unfortunately, this is usually not the case for real-world problems, usually we cannot obtain the rules themselves but only acquire knowledge from interactions with the world or simulation.

Chemical retrosynthesis (you know the product, but not how to get there) is one of such real-life domains with highly non-trivial, partially unknown rules. We show that this problem can very effectively be tackled with MCTS constrained by DNNs that learn from essentially the complete history of organic chemistry, as described in a recent Nature paper [1]. Our algorithm produces plans which human chemists could not tell apart from established plans taken from the literature. But it does not have to end here. We attempt to generalize the approach in order to make it applicable to other research areas as well.

Mike Preuss is Assistant Professor at LIACS, the computer science institute of Universiteit Leiden in the Netherlands. Previously, he was with ERCIS (the information systems institute of WWU Muenster, Germany), and before with the Chair of Algorithm Engineering at TU Dortmund, Germany, where he received his PhD in 2013. His main research interests rest on the field of evolutionary algorithms for real-valued problems, namely on multimodal and multiobjective optimization, and on computational intelligence and machine learning methods for computer games, especially in procedural content generation (PGC) and realtime strategy games (RTS).

[1] Segler, M. H., Preuss, M., & Waller, M. P. (2018). Planning chemical syntheses with deep neural networks and symbolic AI. Nature, 555(7698), 604. https://www.nature.com/articles/nature25978

[Seminar] Evolutionary MCTS for Game AI and Beyond

On Tuesday 30th April 2019 the Game AI Group will host a seminar by Hendrik Baier from CWI Amsterdam

Title: Evolutionary MCTS for Game AI and Beyond 
Speaker:Hendrik Baier, CWI Amsterdam
Time: 11 am to 12 pm, Apr 30, 2019
Room: 3.02, Bancroft Road Teaching Rooms
All welcome (especially students), no pre-booking required.  After the talk we’re going for lunch with the speaker at Verdi’s.  Let me know if you’d like to join!
In this talk, I’m going to present a research direction that tries to combine the strengths of Monte Carlo Tree Search with those of Evolutionary Algorithms. Summarizing work presented at CIG and AIIDE last year, I’m first going to explain how Evolutionary MCTS (EMCTS) works, apply it to online planning in turn-based tactics/strategy games, and show how it can be extended to include a simple opponent model. Afterwards, I’m going to briefly go beyond the goal of “strongest possible AI” by showing first results on dynamic difficulty adaptation with EMCTS; and beyond the field of game AI altogether by discussing how I am applying RHEA, MCTS, and EMCTS in the context of managing flexibility on future energy markets!
I am currently living in the beautiful city of Amsterdam, working as researcher in the Intelligent and Autonomous Systems group at CWI. Before, I worked as research associate for artificial intelligence and data analytics at Digital Creativity Labs in York, England, and as research fellow in artificial intelligence in the Advanced Concepts Team of the European Space Agency in Noordwijk, Netherlands.  In 2015, I finished my Ph.D. in the Games and AI group at the Department of Knowledge Engineering, Maastricht University. My topic was planning and search — the search for an optimal strategy in a given problem, or just for a good next action to take. I used games as model problems. Application areas are both adversarial and collaborative scenarios with several agents, as well as optimization problems with a single agent.

[Seminar] Compressing Space, Warping Perception And Projecting Flowers Onto Cups Of Tea

OnWednesday 3rd April 2019 the Game AI Group will host a seminar by Michael Cook from Queen Mary University of London

Title: Compressing Space, Warping Perception And Projecting Flowers Onto Cups Of Tea
Speaker: Michael Cook, Queen Mary University of London
Time: 4pm to 5pm, Apr 3, 2019
Room: 1.04, Scape Building, Mile End Campus (building 64 on QMUL’s campus map.)
All welcome (especially students), no pre-booking required.


I’ll summarise some recent directions in my research on automated game design and procedural content generation, including two submissions to COG 2019.
* I’ve been looking at ways to compress state space graphs using the notion of reversible game actions, and why this might have exciting implications for game analysis in the future;
* Some new analytical techniques for procedural generators, allowing us to understand the behaviour of inputs to a generative system, and thus build tools for people which are easier to use.
* A new framework for studying AI that can play games for the purposes of curation, criticism and recommendation. I want to build AI that can appreciate games in ways other than just winning, and we’re hoping to organise a competition-like event around it to – we need your help!
Lastly, I’ll report on my visit to teamLab Borderless in Tokyo, a really inspirational experience that has made me rethink how we build digital experiences, and what the scope of our research can be.
Michael Cook is an AI researcher and game designer. He holds a Royal Academy of Engineering Research Fellowship at Queen Mary University of London, and is a Visiting Researcher at the Max Planck Institute for Software Systems in Kaiserslautern, Germany. His research interests include automated game design, computational creativity, procedural content generation and models of game design. He is best known as the designer of ANGELINA, a game-designing AI, and as the founder of PROCJAM, the procedural generation jam. In his spare time he designs games and writes about AI and game design. He once managed to collect every strawberry in Celeste.

You can find him on Twitter @mtrc, or find his papers, writing and more at possibilityspace.org

[Seminar] TextWorld – A framework for training reinforcement learning agents on text-based games

On Thursday 28th February 2019 the Game AI Group will host a seminar by Marc-Alexandre Côté from Microsoft Research Montreal

Title: TextWorld – A framework for training reinforcement learning agents on text-based games
Speaker: Marc-Alexandre Côté, Microsoft Research Montreal
Time: 11am to 12pm, Mar 28, 2019
Room: BR 4.02, Bancroft Road Teaching Rooms, QMUL Mile End Campus
All welcome (especially students), no pre-booking required.

Text-based games are complex, interactive simulations in which text describes the game state and players make progress by entering text commands. They are fertile ground for language-focused machine learning research. In addition to language understanding, successful play requires skills like long-term memory and planning, exploration (trial and error), and common sense.
The talk will introduce TextWorld, a sandbox learning environment for the training and evaluation of RL agents on text-based games. Its generative mechanisms give precise control over the difficulty, scope, and language of constructed games, and can be used to study generalization and transfer learning. This talk will also give an overview of the recent attempts to solve text-based games either using reinforcement learning or more handcrafted approaches.

Marc-Alexandre Côté is a researcher at Microsoft Research Montreal. His research focuses on machine learning, more specifically on generative models, reinforcement learning and language understanding. He is currently leading the TextWorld project, a learning environment for RL agent in the context of text-based games.

[Seminar] Building AI Dialogue Tools for Non-Technical Authors

On Wednesday 27th March 2019 the Game AI Group will host a seminar by Emily Short from Spirit AI.

Title: Building AI Dialogue Tools for Non-Technical Authors
Speaker: Emily Short, Spirit AI.
Time: 4pm to 5pm, Mar 27, 2019
Room: BR 3.02, Bancroft Road Teaching Rooms, QMUL Mile End Campus
All welcome (especially students), no pre-booking required.
This talk introduces Character Engine, a system for writing natural language or menu-based interactions with non-player characters, designed for a range of use cases from mobile gaming to voice-driven conversations with robots. We discuss the architecture of the system, the use of ML systems for understanding player intent, social models used to assist in contextual understanding of player input, and the application of more traditional game AI methods such as utility scoring and generative grammars to determine character responses. We also look at design features intended to support authors who might not themselves be experts in AI or NLP.


Emily Short is Chief Product Officer and an executive director at Spirit AI, where she works with natural language processing and generation, character moods and social behavior, and conversation modelling. She was previously the creative director of the Versu project, building interactive iPad stories around AI character agents. She has an extensive background in games writing and narrative design as well as tooling for writing interactive fiction. Her blog can be found at http://emshort.blog.

[Seminar] Solving 10×10 Hex

On Wednesday 20th February 2019 the Game AI Group will host a seminar by Ryan Hayward from the University of Alberta.

Title: Solving 10×10 Hex
Speaker: Ryan Hayward, University of Alberta
Time: 4pm to 5pm, Feb 20, 2019
Room: BR 3.02, Bancroft Road Teaching Rooms, QMUL Mile End Campus
Followed by drinks in the Informatics Hub.
All welcome (especially students), no pre-booking required.
In late winter 1949 John Nash bumped into David Gale in Princeton told him about a game — now called Hex — for which (unlike with chess or Go) he could prove that the first player has a winning strategy.  After hearing Nash’s description of the game, Gale built a board that he left in the math department’s common room, where it became popular. Eventually it caught the attention of Claude Shannon, who built 2 Hex-playing machines, and Martin Gardner, who wrote about it in his Mathematical Games column in Scientific American.

Nash’s proof is existential, and gives little information about how to find explicit strategies. You might try this problem for yourself: on nxn boards up to 5×5, finding a first player winning strategy is easy.  6×6 is more challenging, and the problem gets harder as n grows.  (Finding particular winning strategies for arbitrary positions is P-space-complete. Finding arbitrary strategies for the empty-board position might be easier.)

In this talk I will summarize the ideas that went into finding an (almost completely) explicit strategy for the first-player on the 10×10 board, and then say a few words about what it would take to solve 11×11.

This is joint work with Broderick Arneson, Phil Henderson, Aja Huang and Jakub Pawlewicz.
Ryan Hayward received his B.Sc. and M.Sc.  in mathematics from Queen’s University (Kingston) in 1981 and 1982 and his Ph.D.in computer science from McGill University in 1987. His doctoral thesis, Two Classes of Perfect Graphs, was supervised by Vasek Chvatal.  From 1986 through 1989 he was assistant professor in the Department of Computer Science at Rutgers University, after which he held an Alexander von Humboldt fellowship at the Institute for Discrete Mathematics in Bonn for 1989-90. From 1990 through 1992 he was assistant professor in the Department of Computing Science at Queen’s University. From 1992 he was assistant and then associate professor in the Department of Mathematics and Computer Science at the University of Lethbridge, until in 1999 joining the Department of Computing Science at the University of Alberta, where he was promoted to professor in 2004.

He has supervised 13 graduate and 29 undergraduate students,some of whom later became university professors. His current research interests include algorithms for two-player games. His group (including at times Yngvi Bjornsson, Michael Johanson, Broderick Arneson, Philip Henderson, Jakub Pawlewicz, and Aja Huang — later lead programmer of AlphaGo) has built the world’s strongest computer Hex player, and has solved two 1-move 10×10 Hex openings and all smaller-board openings.

With Bjarne Toft, he wrote “Hex, the full story”, published by Taylor-Francis in 2019.

Ryan lives in Edmonton where he commutes year-round by recumbent bike.

[Seminar] Generalisation of Simulation-Based Search for Autonomous Gameplaying

On Wednesday 6th February 2019 the Game AI Group will host a seminar by Alexander Dockhorn from Otto-von-Guericke University Magdeburg.

Title: Generalisation of Simulation-Based Search for Autonomous Gameplaying
Speaker: Alexander Dockhorn
Time: 4pm to 5pm, Feb 6, 2019
Room: BR 3.02, Bancroft Road Teaching Rooms, QMUL Mile End Campus
All welcome (especially students), no pre-booking required.

In my talk, I am going to present how various supervised learning mechanisms can lead to an approximation of an unknown game’s forward model, allowing agents to apply simulation-based search algorithms to general game learning tasks. The presentation will include an analysis of game model characteristics and the resulting development of the forward model approximation framework. Various forward model learning systems will be explored to highlight capabilities and limitations of the discussed framework.


I am PhD student at the Otto-von-Guericke University. My current research interests are computational intelligence in games and intelligent data analysis. My focus is on partial information games as well as learning rules, game models and playing strategies from previous experiences.

[Seminar] Agents with internal models

On Wednesday 6th February 2019 the Game AI Group will host a seminar by Theophane Weber from DeepMind.

Title: Agents with internal models
Speaker: Theophane Weber
Time: 4pm to 5pm, Feb 6, 2019
Room: BR 3.02, Bancroft Road Teaching Rooms, QMUL Mile End Campus
Followed by drinks in the Informatics Hub.
All welcome (especially students), no pre-booking required.
I will present recent work that studies agents endowed with an internal model of the world. This will include agents that learn world models by predicting the future, and learn to interpret those predictions in order to act better without suffering from model inaccuracies; agents with neural analogues of search algorithms such as Monte Carlo Tree Search; agents that learn temporally abstract models of the world in order to compute representations of their belief about the state of the world, agents that use their models to evaluate counterfactual scenarios and learn from those synthetic experiences, and agents with only implicit models of the world that still exhibit planning-like behavior.


I am a staff research scientist at DeepMind. My research interests span deep reinforcement learning, model-based RL and planning, probabilistic modeling and modeling of uncertainty. Prior to DeepMind, I worked at Lyric Labs, a skunkworks team of Analog Devices, working on applications of machine learning to the physical world. I hold an M.S. and Ph.D from MIT in Operations Research and an M.S. from Ecole Centrale Paris in Applied Mathematics.

Theophane’s Google Scholar profile.

[Seminar] Advancing Video Game AI With Intrinsically Motivated Reinforcement Learning

On Friday 7th December the Game AI Group will host a seminar by Christian Guckelsberger from QMUL.

Title: Advancing Video Game AI With Intrinsically Motivated Reinforcement Learning
Speaker:Christian Guckelsberger
Time: 2pm-3pm (GMT), Dec 6, 2018
Room: BR 3.02, Bancroft Road Teaching Rooms, QMUL Mile End Campus
Modern video games come with increasingly large and complex worlds to satisfy players’ demands for a rich and long-lasting playing experience. This development brings new challenges: designing robust believable characters that players can engage with in an open-ended way, and also with respect to evaluating content, especially when procedurally generated. In this talk, I will motivate the use of intrinsically motivated reinforcement learning to address the challenges of next-generation video games, a technique which currently gains strong momentum in the search for artificial general intelligence. I will give a comprehensive, interdisciplinary introduction to the concept of intrinsic motivation. I will motivate the development of computational models of intrinsic motivation, point out the opportunities they hold for game AI, and discuss the new challenges such models come with. My research on coupled empowerment maximisation for more believable non-player characters will illustrate the potential of such models, and motivate their combination with reinforcement learning. The use of intrinsically motivated reinforcement learning for video game AI is still in its infancy, and I will consequently finish with a set of open questions and interesting research projects.

[Seminar] The Blurring of Video Games and Gambling: Daily Fantasy, Esports, Live Streaming, and Loot Boxes

On Tuesday 27th November the Game AI Group will host a seminar by Mark R Johnson from the University of Alberta.

Title: The Blurring of Video Games and Gambling: Daily Fantasy, Esports, Live Streaming, and Loot Boxes
Speaker: Dr Mark R Johnson
Time: 1pm-2pm (GMT), Nov 27, 2018
Room: BR 3.02, Bancroft Road Teaching Rooms, QMUL
This talk will examine the growing importance of gambling in modern video game design, the growing importance of video game design to contemporary gambling forms, and the overall wider interweaving of the two within the past five to ten years and their implications for the future of digital play. Firstly, the rapid rise and newfound near-ubiquity of Esports gambling, both “within” games (e.g. skin betting) and “outside” through third-parties (e.g. sports betting), poses new questions about the increasing extent to which Esports are coming to resemble traditional physical sports, and how consumption of Esports competition is shifting as corporate interests and “professionalisation” increase. Secondly, the paper will examine the expanding ongoing controversies surrounding “loot boxes”, the question of whether or not these are gambling, and how anti-“gambling” discourses are being mobilised in opposition, despite the primary issue players report being a question of paying to win. Thirdly, I will consider daily fantasy sports platforms, their commonalities with “sports management” video games, and their status as ambiguous gambling-gaming artefacts which subvert the clear boundaries between the two. Fourthly, the talk will examine online poker broadcasts on the live-streaming platform Twitch.tv, viewed by millions, and also how numerous live streamers are “gamblifying” methods by which their viewers can give them monetary support, using a suite of psychological techniques to encourage donations. The talk will conclude by emphasising the importance of these four phenomena for the future relationship between games and money, and how we might understand the growing role of gambling, and gambling-style systems, in many different kinds of digital play.
Mark R Johnson is a Killam Memorial Postdoctoral Fellow at the University of Alberta in Canada. His work focuses on the intersections between play and money, such as professionalised video game competition (E-sports), the live broadcast and spectating of video games on personalised online “channels”, and the blurring of video games and gambling in numerous contexts.

Outside academia, he is also an independent game developer, a regular games writer, blogger and podcaster, and a former professional poker player.