BitRL Example 3 Using Gymnasium environments (Part 2)

In this example we will see how to interact with Gymnasium environments and specifically how to create an interact with CartPole environment.

As already mentioned in BitRL Example 1 Using Gymnasium environments (Part 1), Gymnasium-based environments are interacted over a REST-like API maintained here: bitrl-envs-api. bitrl itself implements classes that hide this interaction from the client code. In general, environment classes in bitrl, have to implement the bitrl::envs::EnvBase API.

In this example we will use the bitrl::envs::gymnasium::CartPole
class. This is a template class, see the example below, that itself inherits from bitrl::envs::gymnasium::GymnasiumEnvBase class.

Below is the driver code.

#include "bitrl/bitrl_consts.h"
#include "bitrl/bitrl_types.h"
#include "bitrl/envs/gymnasium/classic_control/cart_pole_env.h"
#include "bitrl/network/rest_rl_env_client.h"
 
#ifdef BITRL_DEBUG
#include <cassert>
#endif
 
#include <iostream>
#include <random>
#include <string>
#include <unordered_map>
 
namespace example
{
using namespace bitrl;
using namespace bitrl::envs::gymnasium;
using bitrl::network::RESTRLEnvClient;
 
void test_cart_pole(RESTRLEnvClient &server)
{
std::cout << "Is environment registered: " << server.is_env_registered(CartPole::name)
<< std::endl;
 
    // create the environment
    CartPole env(server);
 
    std::cout << "Name: " << env.name << std::endl;
    std::cout << "Number of actions: " << env.n_actions() << std::endl;
 
    // make the environment
    std::unordered_map<std::string, std::any> options;
    std::unordered_map<std::string, std::any> reset_ops;
    reset_ops.insert({"seed", static_cast<uint_t>(42)});
    env.make("v1", options, reset_ops);
 
    auto time_step = env.reset();
    std::cout << "Time step: " << time_step << std::endl;
 
    // step in the environment
    time_step = env.step(0);
    std::cout << "Time step after action: " << time_step << std::endl;
 
    env.close();
}
 
} // namespace example
 
int main()
{
 
    using namespace example;
 
    const std::string SERVER_URL = "http://0.0.0.0:8001/api";
 
    RESTRLEnvClient server(SERVER_URL, true);
 
    std::cout << "Testing CartPole..." << std::endl;
    example::test_cart_pole(server);
    std::cout << "====================" << std::endl;
 
    return 0;
}

In order to run the example you will need an instance of the bitrl-envs-api server running on your machine listening at port 8001. Note the actual example also shows how to use bitrl::envs::gymnasium::Pendulum, bitrl::envs::gymnasium::Acrobot and bitrl::envs::gymnasium::MountainCar environments.