courses:hcai:projects [IIS Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Mini-projects =====

Work in **groups of 2-4** and choose **one project**.  
In each project, you will develop three increasingly advanced agents using OpenSpiel, compare their behaviour, and present your results.

==== Intro to OpenSpiel ====

**Notebook:** [[https://colab.research.google.com/drive/1NReifoJfvAniWKIudB7DfampTLyKH0tQ?usp=sharing|HCAI_OpenSpiel_Introduction.ipynb]] (Work in Google Colab or download the .ipynb file to local environment)

==== About projects ====

=== Common requirements ===

Whichever project you choose, you will:
  * implement and retain all three agents;
  * compare them under consistent conditions;
  * include repeated runs where randomness is involved;
  * perform at least one controlled modification or ablation;
  * analyse at least one failure case;
  * use plots and a detailed trajectory to explain behaviour;
  * clearly document your modifications;
  * demonstrate the project and explain your decisions during the final presentation.

The advanced agent **does not need** to win every comparison. A correct implementation and a thoughtful analysis of its strengths and limitations are equally important.

=== Initial code package ===

  * Python: {{http://krzysztof.kutt.pl/didactics/hcai/HCAI_OpenSpiel_Python_v1.zip|HCAI_OpenSpiel_Python_v1.zip}}
  * C++: {{http://krzysztof.kutt.pl/didactics/hcai/HCAI_OpenSpiel_CPP_v0.1.zip|HCAI_OpenSpiel_CPP_v0.1.zip}} (NOTE: converted from Python version to use native OpenSpiel API with the MS Copilot Chat; not fully verified yet)

==== List of projects ====

=== Connect Four ===

**Topics:** Agents · Adversarial Search · Supervised Learning

Develop agents for the classic two-player Connect Four game:
  * a one-step agent with a hand-crafted evaluation function;
  * a minimax agent with alpha-beta pruning;
  * an alpha-beta agent with a learned evaluation function.

You will investigate whether better decisions come from **deeper search, better evaluation, or their combination**.

**Choose this project if you are interested in:** competitive games, minimax, heuristic design, and supervised learning.

=== Catch ===

**Topics:** Agents · Model-Based Search · Reinforcement Learning

Develop an agent that moves horizontally to catch a descending object:
  * a reactive or predictive agent;
  * a search agent that plans with the environment model and handles chance outcomes;
  * a Q-learning agent trained through interaction.

You will compare **explicit planning with learning from experience** in a stochastic environment.

**Choose this project if you are interested in:** dynamic environments, chance nodes, planning, and reinforcement learning.

=== Cliff Walking ===

**Topics:** Agents · Classical Search · Reinforcement Learning

Develop an agent that must reach a goal without falling from a cliff:
  * a greedy agent using immediate successor states;
  * an A* agent that plans a safe route;
  * a Q-learning agent trained on the same registered OpenSpiel game.

You will examine how different methods balance **route length, accumulated cost, and safety**.

**Choose this project if you are interested in:** pathfinding, A*, heuristic design, reward design, and reinforcement learning.