Implementation of the game "Skull" from Issue #1559

[Popret]

Here's a first crack at implementing #1559 , with a couple of lingering questions:

• Most importantly, I'd like some help implementing MakeObserver(). I'm rather new to large-codebase C++, being more used to plain C. Right now I am creating the Observers manually in the constructor, but if I do the same via MakeObserver I get heap corruption errors. Actually I get those when an implementation MakeObserver() exists, I think regardless of if I call it in the SkullGame constructor. I've tried, and my attempt is commented out right now, but any guidance here would be much appreciated.

• I'd also love feedback on the structure of Returns(). Especially on how best to add intermediate rewards without incentivizing some suboptimal behaviour. I'm going to do some literature review in general, to try to understand in what situations it matters. I was considering implementing a kReturnsType similar to how goofspiel currently does it, but I wanted to get a better understanding for for the reasons behind any approach before implementing any specifics.

• A note on the Tests, they could be cleaned up somewhat but before committing to any approach I had some lingering questions regarding policies. I am using preferred action policy and custom policies in skull_test.cc because depending on the game parameters skull can be a rather large game (e.g. many players, and perfect recall) that doesn't lend itself to a full tabular policy.

  1. Is a use case like this a reasonable use of policies, or for situations like this are bots more appropriate? (I chose to use policies over bots because it seemed to me policies are more core to OpenSpiel?) I know bots were added after the main paper, maybe some more documentation could be of use disambiguating bots vs policies?
  2. If this is a reasonable way to use policies could it make sense to maybe expand PreferredActionPolicy to also have optional Fallback Policies like partial_tabular, rather than failing by default?
     • How PreferredActionPolicy works:

     • open_spiel/open_spiel/policy.cc

       Lines 533 to 542 in 6c8edc8

       	ActionsAndProbs PreferredActionPolicy::GetStatePolicy(const State& state,
       	Player player) const {
       	std::vector<Action> legal_actions = state.LegalActions(player);
       	for (Action action : preference_order_) {
       	if (absl::c_find(legal_actions, action) != legal_actions.end()) {
       	return GetDeterministicPolicy(legal_actions, action);
       	}
       	}
       	SpielFatalError("No preferred action found in the legal actions!");
       	}

     • Compare to PartialTabularPolicy

     • open_spiel/open_spiel/policy.cc

       Lines 255 to 263 in 6c8edc8

       	ActionsAndProbs PartialTabularPolicy::GetStatePolicy(const State& state,
       	Player player) const {
       	auto iter = policy_table_.find(state.InformationStateString(player));
       	if (iter == policy_table_.end()) {
       	return fallback_policy_->GetStatePolicy(state);
       	} else {
       	return iter->second;
       	}
       	}

     • I could submit a PR if this is wanted.

Overall, I tried to be relatively thoughtful with my implementation, and tried to keep it simple. If there's any places stylistically that could be improved as well I'd welcome any feedback.
The next steps from my end would be to properly implement MakeObserver(), adapt this to use the new ActionStructs instead of this flat representation, and adapt the policies / bots into their own file after hearing back what path you recommend.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[Popret]

Hey, just saw the workflow was run and failed because of a goofy typo, sorry about that... Just pushed again.