Skip to content

API to access OpenAI Gym from other languages via HTTP

License

Notifications You must be signed in to change notification settings

fkm3/gym-http-api

 
 

Repository files navigation

gym-http-api

This project provides a local REST API to the gym open-source library, allowing development in languages other than python.

A python client is included, to demonstrate how to interact with the server.

Additional languages:

  • C++ (within this repository, author: Oleg Klimov)
  • R (within this repository, author: Paul Hendricks)
  • lua (within this repository, work-in-progress)
  • Scala (in progress, author: Flavio Truzzi)
  • ... Contributions of clients in other languages are welcomed!

Installation

To download the code and install the requirements, you can run the following shell commands:

git clone https://github.com/openai/gym-http-api
cd gym-http-api
pip install -r requirements.txt

Getting started

This code is intended to be run locally by a single user. The server runs in python. You can implement your own HTTP clients using any language; a demo client written in python is provided to demonstrate the idea.

To start the server from the command line, run this:

python gym_http_server.py

In a separate terminal, you can then try running the example python agent and see what happens:

python example_agent.py  

The example lua agent behaves very similarly:

cd binding-lua
lua example_agent.lua

You can also write code like this to create your own client, and test it out by creating a new environment. For example, in python:

remote_base = 'http://127.0.0.1:5000'
client = Client(remote_base)

env_id = 'CartPole-v0'
instance_id = client.env_create(env_id)
client.env_step(instance_id, 0)

Testing

This repository contains integration tests, using the python client implementation to send requests to the local server. They can be run using the nose2 framework. From a shell (such as bash) you can run nose2 directly:

cd gym-http-api
nose2

API specification

  • POST /v1/envs/

    • Create an instance of the specified environment
    • param: env_id -- gym environment ID string, such as 'CartPole-v0'
    • returns: instance_id -- a short identifier (such as '3c657dbc') for the created environment instance. The instance_id is used in future API calls to identify the environment to be manipulated
  • GET /v1/envs/

    • List all environments running on the server
    • returns: envs -- dict mapping instance_id to env_id (e.g. {'3c657dbc': 'CartPole-v0'}) for every env on the server
  • POST /v1/envs/<instance_id>/reset/

    • Reset the state of the environment and return an initial observation.
    • param: instance_id -- a short identifier (such as '3c657dbc') for the environment instance
    • returns: observation -- the initial observation of the space
  • POST /v1/envs/<instance_id>/step/

    • Step though an environment using an action.
    • param: instance_id -- a short identifier (such as '3c657dbc') for the environment instance
    • param: action -- an action to take in the environment
    • returns: observation -- agent's observation of the current environment
    • returns: reward -- amount of reward returned after previous action
    • returns: done -- whether the episode has ended
    • returns: info -- a dict containing auxiliary diagnostic information
  • GET /v1/envs/<instance_id>/action_space/

    • Get information (name and dimensions/bounds) of the env's action_space
    • param: instance_id -- a short identifier (such as '3c657dbc') for the environment instance
    • returns: info -- a dict containing 'name' (such as 'Discrete'), and additional dimensional info (such as 'n') which varies from space to space
  • GET /v1/envs/<instance_id>/observation_space/

    • Get information (name and dimensions/bounds) of the env's observation_space
    • param: instance_id -- a short identifier (such as '3c657dbc') for the environment instance
    • returns: info -- a dict containing 'name' (such as 'Discrete'), and additional dimensional info (such as 'n') which varies from space to space
  • POST /v1/envs/<instance_id>/monitor/start/

    • Start monitoring
    • param: instance_id -- a short identifier (such as '3c657dbc') for the environment instance
    • param: force (default=False) -- Clear out existing training data from this directory (by deleting every file prefixed with "openaigym.")
    • param: resume (default=False) -- Retain the training data already in this directory, which will be merged with our new data
    • (NOTE: the video_callable parameter from the native env.monitor.start function is NOT implemented)
  • POST /v1/envs/<instance_id>/monitor/close/

    • Flush all monitor data to disk
    • param: instance_id -- a short identifier (such as '3c657dbc') for the environment instance
  • POST /v1/upload/

    • Flush all monitor data to disk
    • param: training_dir -- A directory containing the results of a training run.
    • param: api_key -- Your OpenAI API key
    • param: algorithm_id (default=None) -- An arbitrary string indicating the paricular version of the algorithm (including choices of parameters) you are running.
  • POST /v1/shutdown/

    • Request a server shutdown
    • Currently used by the integration tests to repeatedly create and destroy fresh copies of the server running in a separate thread

TODOs

python TODOs

  • Make the directory structure better conform to standard python package structures (http://www.kennethreitz.org/essays/repository-structure-and-python, http://peterdowns.com/posts/first-time-with-pypi.html)
  • Implement 'sample' (and test it)
  • Implement 'contains' (and test it)
  • Docker integration
  • Handle ResetNeeded while monitor is active
  • Handle APIConnectionError: Unexpected error communicating with OpenAI Gym
  • Check: was anything improved by adding session / socket reuse?
  • Reports of "broken pipe" errors: reproduce and investigate
  • Measure latency/performance. How slow is HTTP, Flask? What is the use case for this implementation, versus a potential future faster ZeroMQ implementation?
  • Make remote environments have the same interface as non-remote environments
  • Get Travis CI working
  • Test all possible environments in integration tests
  • Handle the error thrown if the directory isn't cleared for the monitor
  • Document the fact that two-monitors-open will cause a problem; be clear that this is meant to be one-client

lua client wishlist:

  • Error handling, similar to what the python client currently has
  • Implement the ability to set "render=True"

Contributors

  • Catherine Olsson
  • Jie Tang
  • Greg Brockman
  • Paul Hendricks
  • Flavio Truzzi
  • Oleg Klimov
  • Jess Smith
  • Kory Mathewson
  • Leonardo Araujo dos Santos
  • Paul Anton
  • Ruben Fiszel
  • Niven Achenjang
  • David Savage

About

API to access OpenAI Gym from other languages via HTTP

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 40.6%
  • C++ 14.6%
  • Lua 11.6%
  • Rust 11.1%
  • TypeScript 11.0%
  • MATLAB 7.8%
  • Other 3.3%