Read the Docs build information Build id: 282633 Project: openai-education-spinningup Version: latest Commit: 0cba2886047b7de82c2cad4321df5875db644d61 Date: 2020-01-30T16:38:43.458638Z State: finished Success: True [rtd-command-info] start-time: 2020-01-30T16:38:46.647695Z, end-time: 2020-01-30T16:38:46.947692Z, duration: 0, exit-code: 0 git remote set-url origin git@github.com:openai/spinningup.git [rtd-command-info] start-time: 2020-01-30T16:38:47.044947Z, end-time: 2020-01-30T16:38:47.968877Z, duration: 0, exit-code: 0 git fetch origin --force --tags --prune --prune-tags --depth 50 Warning: Permanently added 'github.com,192.30.255.112' (RSA) to the list of known hosts. From github.com:openai/spinningup 4880fc6..0cba288 master -> origin/master [rtd-command-info] start-time: 2020-01-30T16:38:48.498085Z, end-time: 2020-01-30T16:38:48.806481Z, duration: 0, exit-code: 0 git checkout --force origin/master Previous HEAD position was 4880fc6 More mock imports HEAD is now at 0cba288 Even more mock imports [rtd-command-info] start-time: 2020-01-30T16:38:48.909230Z, end-time: 2020-01-30T16:38:49.220352Z, duration: 0, exit-code: 0 git clean -d -f -f [rtd-command-info] start-time: 2020-01-30T16:38:50.830318Z, end-time: 2020-01-30T16:38:56.389542Z, duration: 5, exit-code: 0 python3.6 -mvirtualenv --no-site-packages Using base prefix '/home/docs/.pyenv/versions/3.6.8' New python executable in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/bin/python3.6 Not overwriting existing python script /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/bin/python (you must use /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/bin/python3.6) Installing setuptools, pip, wheel... done. [rtd-command-info] start-time: 2020-01-30T16:38:56.853844Z, end-time: 2020-01-30T16:38:58.148024Z, duration: 1, exit-code: 0 python -m pip install --upgrade --cache-dir /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip pip Requirement already up-to-date: pip in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (20.0.2) [rtd-command-info] start-time: 2020-01-30T16:38:58.248618Z, end-time: 2020-01-30T16:39:03.414762Z, duration: 5, exit-code: 0 python -m pip install --upgrade --cache-dir /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip Pygments==2.3.1 setuptools==41.0.1 docutils==0.14 mock==1.0.1 pillow==5.4.1 alabaster>=0.7,<0.8,!=0.7.5 commonmark==0.8.1 recommonmark==0.5.0 sphinx<2 sphinx-rtd-theme<0.5 readthedocs-sphinx-ext<1.1 Requirement already up-to-date: Pygments==2.3.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (2.3.1) Collecting setuptools==41.0.1 Using cached setuptools-41.0.1-py2.py3-none-any.whl (575 kB) Requirement already up-to-date: docutils==0.14 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (0.14) Requirement already up-to-date: mock==1.0.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (1.0.1) Requirement already up-to-date: pillow==5.4.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (5.4.1) Requirement already up-to-date: alabaster!=0.7.5,<0.8,>=0.7 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (0.7.12) Requirement already up-to-date: commonmark==0.8.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (0.8.1) Requirement already up-to-date: recommonmark==0.5.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (0.5.0) Collecting sphinx<2 Using cached Sphinx-1.8.5-py2.py3-none-any.whl (3.1 MB) Collecting sphinx-rtd-theme<0.5 Using cached sphinx_rtd_theme-0.4.3-py2.py3-none-any.whl (6.4 MB) Requirement already up-to-date: readthedocs-sphinx-ext<1.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (1.0.1) Requirement already satisfied, skipping upgrade: future in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from commonmark==0.8.1) (0.18.2) Requirement already satisfied, skipping upgrade: snowballstemmer>=1.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx<2) (2.0.0) Requirement already satisfied, skipping upgrade: six>=1.5 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx<2) (1.14.0) Requirement already satisfied, skipping upgrade: Jinja2>=2.3 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx<2) (2.11.0) Requirement already satisfied, skipping upgrade: imagesize in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx<2) (1.2.0) Requirement already satisfied, skipping upgrade: babel!=2.0,>=1.3 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx<2) (2.8.0) Requirement already satisfied, skipping upgrade: requests>=2.0.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx<2) (2.22.0) Requirement already satisfied, skipping upgrade: sphinxcontrib-websupport in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx<2) (1.1.2) Requirement already satisfied, skipping upgrade: packaging in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx<2) (20.1) Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from Jinja2>=2.3->sphinx<2) (1.1.1) Requirement already satisfied, skipping upgrade: pytz>=2015.7 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from babel!=2.0,>=1.3->sphinx<2) (2019.3) Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx<2) (2019.11.28) Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx<2) (2.8) Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx<2) (1.25.8) Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx<2) (3.0.4) Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from packaging->sphinx<2) (2.4.6) Installing collected packages: setuptools, sphinx, sphinx-rtd-theme Attempting uninstall: setuptools Found existing installation: setuptools 45.1.0 Uninstalling setuptools-45.1.0: Successfully uninstalled setuptools-45.1.0 Attempting uninstall: sphinx Found existing installation: Sphinx 1.5.6 Uninstalling Sphinx-1.5.6: Successfully uninstalled Sphinx-1.5.6 Attempting uninstall: sphinx-rtd-theme Found existing installation: sphinx-rtd-theme 0.4.1 Uninstalling sphinx-rtd-theme-0.4.1: Successfully uninstalled sphinx-rtd-theme-0.4.1 Successfully installed setuptools-41.0.1 sphinx-1.8.5 sphinx-rtd-theme-0.4.3 [rtd-command-info] start-time: 2020-01-30T16:39:03.511064Z, end-time: 2020-01-30T16:39:06.769214Z, duration: 3, exit-code: 0 python -m pip install --exists-action=w --cache-dir /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip -r docs/docs_requirements.txt Requirement already satisfied: cloudpickle~=1.2.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 1)) (1.2.2) Requirement already satisfied: gym~=0.15.3 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 2)) (0.15.4) Requirement already satisfied: ipython in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 3)) (7.11.1) Requirement already satisfied: joblib in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 4)) (0.14.1) Requirement already satisfied: matplotlib in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 5)) (3.1.2) Requirement already satisfied: numpy in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 6)) (1.18.1) Requirement already satisfied: pandas in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 7)) (1.0.0) Requirement already satisfied: pytest in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 8)) (5.3.5) Requirement already satisfied: psutil in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 9)) (5.6.7) Requirement already satisfied: scipy in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 10)) (1.4.1) Requirement already satisfied: seaborn==0.8.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 11)) (0.8.1) Collecting sphinx==1.5.6 Using cached Sphinx-1.5.6-py2.py3-none-any.whl (1.6 MB) Requirement already satisfied: sphinx-autobuild==0.7.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 13)) (0.7.1) Collecting sphinx-rtd-theme==0.4.1 Using cached sphinx_rtd_theme-0.4.1-py2.py3-none-any.whl (5.4 MB) Requirement already satisfied: tensorflow<2.0,>=1.8.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 15)) (1.15.2) Requirement already satisfied: tqdm in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from -r docs/docs_requirements.txt (line 16)) (4.42.0) Requirement already satisfied: opencv-python in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from gym~=0.15.3->-r docs/docs_requirements.txt (line 2)) (4.1.2.30) Requirement already satisfied: pyglet<=1.3.2,>=1.2.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from gym~=0.15.3->-r docs/docs_requirements.txt (line 2)) (1.3.2) Requirement already satisfied: six in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from gym~=0.15.3->-r docs/docs_requirements.txt (line 2)) (1.14.0) Requirement already satisfied: pexpect; sys_platform != "win32" in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from ipython->-r docs/docs_requirements.txt (line 3)) (4.8.0) Requirement already satisfied: pygments in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from ipython->-r docs/docs_requirements.txt (line 3)) (2.3.1) Requirement already satisfied: backcall in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from ipython->-r docs/docs_requirements.txt (line 3)) (0.1.0) Requirement already satisfied: decorator in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from ipython->-r docs/docs_requirements.txt (line 3)) (4.4.1) Requirement already satisfied: setuptools>=18.5 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from ipython->-r docs/docs_requirements.txt (line 3)) (41.0.1) Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from ipython->-r docs/docs_requirements.txt (line 3)) (3.0.3) Requirement already satisfied: pickleshare in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from ipython->-r docs/docs_requirements.txt (line 3)) (0.7.5) Requirement already satisfied: traitlets>=4.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from ipython->-r docs/docs_requirements.txt (line 3)) (4.3.3) Requirement already satisfied: jedi>=0.10 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from ipython->-r docs/docs_requirements.txt (line 3)) (0.16.0) Requirement already satisfied: python-dateutil>=2.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from matplotlib->-r docs/docs_requirements.txt (line 5)) (2.8.1) Requirement already satisfied: kiwisolver>=1.0.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from matplotlib->-r docs/docs_requirements.txt (line 5)) (1.1.0) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from matplotlib->-r docs/docs_requirements.txt (line 5)) (2.4.6) Requirement already satisfied: cycler>=0.10 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from matplotlib->-r docs/docs_requirements.txt (line 5)) (0.10.0) Requirement already satisfied: pytz>=2017.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pandas->-r docs/docs_requirements.txt (line 7)) (2019.3) Requirement already satisfied: wcwidth in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pytest->-r docs/docs_requirements.txt (line 8)) (0.1.8) Requirement already satisfied: py>=1.5.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pytest->-r docs/docs_requirements.txt (line 8)) (1.8.1) Requirement already satisfied: more-itertools>=4.0.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pytest->-r docs/docs_requirements.txt (line 8)) (8.2.0) Requirement already satisfied: importlib-metadata>=0.12; python_version < "3.8" in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pytest->-r docs/docs_requirements.txt (line 8)) (1.5.0) Requirement already satisfied: attrs>=17.4.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pytest->-r docs/docs_requirements.txt (line 8)) (19.3.0) Requirement already satisfied: packaging in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pytest->-r docs/docs_requirements.txt (line 8)) (20.1) Requirement already satisfied: pluggy<1.0,>=0.12 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pytest->-r docs/docs_requirements.txt (line 8)) (0.13.1) Requirement already satisfied: requests>=2.0.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2.22.0) Requirement already satisfied: snowballstemmer>=1.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2.0.0) Requirement already satisfied: docutils>=0.11 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (0.14) Requirement already satisfied: Jinja2>=2.3 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2.11.0) Requirement already satisfied: alabaster<0.8,>=0.7 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (0.7.12) Requirement already satisfied: imagesize in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (1.2.0) Requirement already satisfied: babel!=2.0,>=1.3 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2.8.0) Requirement already satisfied: port-for==0.3.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx-autobuild==0.7.1->-r docs/docs_requirements.txt (line 13)) (0.3.1) Requirement already satisfied: PyYAML>=3.10 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx-autobuild==0.7.1->-r docs/docs_requirements.txt (line 13)) (5.3) Requirement already satisfied: argh>=0.24.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx-autobuild==0.7.1->-r docs/docs_requirements.txt (line 13)) (0.26.2) Requirement already satisfied: watchdog>=0.7.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx-autobuild==0.7.1->-r docs/docs_requirements.txt (line 13)) (0.10.0) Requirement already satisfied: pathtools>=0.1.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx-autobuild==0.7.1->-r docs/docs_requirements.txt (line 13)) (0.1.2) Requirement already satisfied: tornado>=3.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx-autobuild==0.7.1->-r docs/docs_requirements.txt (line 13)) (6.0.3) Requirement already satisfied: livereload>=2.3.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx-autobuild==0.7.1->-r docs/docs_requirements.txt (line 13)) (2.6.1) Requirement already satisfied: astor>=0.6.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (0.8.1) Requirement already satisfied: termcolor>=1.1.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (1.1.0) Requirement already satisfied: keras-applications>=1.0.8 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (1.0.8) Requirement already satisfied: tensorflow-estimator==1.15.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (1.15.1) Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (1.15.0) Requirement already satisfied: wrapt>=1.11.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (1.11.2) Requirement already satisfied: grpcio>=1.8.6 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (1.26.0) Requirement already satisfied: keras-preprocessing>=1.0.5 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (1.1.0) Requirement already satisfied: opt-einsum>=2.3.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (3.1.0) Requirement already satisfied: protobuf>=3.6.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (3.11.2) Requirement already satisfied: google-pasta>=0.1.6 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (0.1.8) Requirement already satisfied: gast==0.2.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (0.2.2) Requirement already satisfied: wheel>=0.26; python_version >= "3" in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (0.34.1) Requirement already satisfied: absl-py>=0.7.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (0.9.0) Requirement already satisfied: future in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pyglet<=1.3.2,>=1.2.0->gym~=0.15.3->-r docs/docs_requirements.txt (line 2)) (0.18.2) Requirement already satisfied: ptyprocess>=0.5 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pexpect; sys_platform != "win32"->ipython->-r docs/docs_requirements.txt (line 3)) (0.6.0) Requirement already satisfied: ipython-genutils in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from traitlets>=4.2->ipython->-r docs/docs_requirements.txt (line 3)) (0.2.0) Requirement already satisfied: parso>=0.5.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from jedi>=0.10->ipython->-r docs/docs_requirements.txt (line 3)) (0.6.0) Requirement already satisfied: zipp>=0.5 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from importlib-metadata>=0.12; python_version < "3.8"->pytest->-r docs/docs_requirements.txt (line 8)) (2.1.0) Requirement already satisfied: certifi>=2017.4.17 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2019.11.28) Requirement already satisfied: idna<2.9,>=2.5 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2.8) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (3.0.4) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (1.25.8) Requirement already satisfied: MarkupSafe>=0.23 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from Jinja2>=2.3->sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (1.1.1) Requirement already satisfied: h5py in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from keras-applications>=1.0.8->tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (2.10.0) Requirement already satisfied: werkzeug>=0.11.15 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (0.16.1) Requirement already satisfied: markdown>=2.6.8 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (3.1.1) Installing collected packages: sphinx, sphinx-rtd-theme Attempting uninstall: sphinx Found existing installation: Sphinx 1.8.5 Uninstalling Sphinx-1.8.5: Successfully uninstalled Sphinx-1.8.5 Attempting uninstall: sphinx-rtd-theme Found existing installation: sphinx-rtd-theme 0.4.3 Uninstalling sphinx-rtd-theme-0.4.3: Successfully uninstalled sphinx-rtd-theme-0.4.3 Successfully installed sphinx-1.5.6 sphinx-rtd-theme-0.4.1 [rtd-command-info] start-time: 2020-01-30T16:39:07.659589Z, end-time: 2020-01-30T16:39:07.967442Z, duration: 0, exit-code: 0 cat docs/conf.py #!/usr/bin/env python3 # -*- coding: utf-8 -*- # # Spinning Up documentation build configuration file, created by # sphinx-quickstart on Wed Aug 15 04:21:07 2018. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # import os import sys # Make sure spinup is accessible without going through setup.py dirname = os.path.dirname sys.path.insert(0, dirname(dirname(__file__))) # Mock mpi4py to get around having to install it on RTD server (which fails) # Also to mock PyTorch, because it is too large for the RTD server to download from unittest.mock import MagicMock class Mock(MagicMock): @classmethod def __getattr__(cls, name): return MagicMock() MOCK_MODULES = ['mpi4py', 'torch', 'torch.optim', 'torch.nn', 'torch.distributions', 'torch.distributions.normal', 'torch.distributions.categorical', 'torch.nn.functional', ] sys.modules.update((mod_name, Mock()) for mod_name in MOCK_MODULES) # Finish imports import spinup from recommonmark.parser import CommonMarkParser source_parsers = { '.md': CommonMarkParser, } # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. # # needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = ['sphinx.ext.imgmath', 'sphinx.ext.viewcode', 'sphinx.ext.autodoc', 'sphinx.ext.napoleon'] #'sphinx.ext.mathjax', ?? # imgmath settings imgmath_image_format = 'svg' imgmath_font_size = 14 # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: # source_suffix = ['.rst', '.md'] # source_suffix = '.rst' # The master toctree document. master_doc = 'index' # General information about the project. project = 'Spinning Up' copyright = '2018, OpenAI' author = 'Joshua Achiam' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = '' # The full version, including alpha/beta/rc tags. release = '' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. language = None # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This patterns also effect to html_static_path and html_extra_path exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'default' #'sphinx' # If true, `todo` and `todoList` produce output, else they produce nothing. todo_include_todos = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # # html_theme = 'alabaster' html_theme = "sphinx_rtd_theme" # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. # # html_theme_options = {} # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] html_logo = 'images/spinning-up-logo2.png' html_theme_options = { 'logo_only': True } #html_favicon = 'openai-favicon2_32x32.ico' html_favicon = 'openai_icon.ico' # -- Options for HTMLHelp output ------------------------------------------ # Output file base name for HTML help builder. htmlhelp_basename = 'SpinningUpdoc' # -- Options for LaTeX output --------------------------------------------- imgmath_latex_preamble = r''' \usepackage{algorithm} \usepackage{algorithmic} \usepackage{amsmath} \usepackage{cancel} \usepackage[verbose=true,letterpaper]{geometry} \geometry{ textheight=12in, textwidth=6.5in, top=1in, headheight=12pt, headsep=25pt, footskip=30pt } \newcommand{\E}{{\mathrm E}} \newcommand{\underE}[2]{\underset{\begin{subarray}{c}#1 \end{subarray}}{\E}\left[ #2 \right]} \newcommand{\Epi}[1]{\underset{\begin{subarray}{c}\tau \sim \pi \end{subarray}}{\E}\left[ #1 \right]} ''' latex_elements = { # The paper size ('letterpaper' or 'a4paper'). # # 'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). # # 'pointsize': '10pt', # Additional stuff for the LaTeX preamble. # 'preamble': r''' \usepackage{algorithm} \usepackage{algorithmic} \usepackage{amsmath} \usepackage{cancel} \newcommand{\E}{{\mathrm E}} \newcommand{\underE}[2]{\underset{\begin{subarray}{c}#1 \end{subarray}}{\E}\left[ #2 \right]} \newcommand{\Epi}[1]{\underset{\begin{subarray}{c}\tau \sim \pi \end{subarray}}{\E}\left[ #1 \right]} ''', # Latex figure (float) alignment # # 'figure_align': 'htbp', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'SpinningUp.tex', 'Spinning Up Documentation', 'Joshua Achiam', 'manual'), ] # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'spinningup', 'Spinning Up Documentation', [author], 1) ] # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ (master_doc, 'SpinningUp', 'Spinning Up Documentation', author, 'SpinningUp', 'One line description of project.', 'Miscellaneous'), ] def setup(app): app.add_stylesheet('css/modify.css') ########################################################################### # auto-created readthedocs.org specific configuration # ########################################################################### # # The following code was added during an automated build on readthedocs.org # It is auto created and injected for every build. The result is based on the # conf.py.tmpl file found in the readthedocs.org codebase: # https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/doc_builder/templates/doc_builder/conf.py.tmpl # import importlib import sys import os.path from six import string_types from sphinx import version_info # Get suffix for proper linking to GitHub # This is deprecated in Sphinx 1.3+, # as each page can have its own suffix if globals().get('source_suffix', False): if isinstance(source_suffix, string_types): SUFFIX = source_suffix elif isinstance(source_suffix, (list, tuple)): # Sphinx >= 1.3 supports list/tuple to define multiple suffixes SUFFIX = source_suffix[0] elif isinstance(source_suffix, dict): # Sphinx >= 1.8 supports a mapping dictionary for multiple suffixes SUFFIX = list(source_suffix.keys())[0] # make a ``list()`` for py2/py3 compatibility else: # default to .rst SUFFIX = '.rst' else: SUFFIX = '.rst' # Add RTD Static Path. Add to the end because it overwrites previous files. if not 'html_static_path' in globals(): html_static_path = [] if os.path.exists('_static'): html_static_path.append('_static') # Add RTD Theme only if they aren't overriding it already using_rtd_theme = ( ( 'html_theme' in globals() and html_theme in ['default'] and # Allow people to bail with a hack of having an html_style 'html_style' not in globals() ) or 'html_theme' not in globals() ) if using_rtd_theme: theme = importlib.import_module('sphinx_rtd_theme') html_theme = 'sphinx_rtd_theme' html_style = None html_theme_options = {} if 'html_theme_path' in globals(): html_theme_path.append(theme.get_html_theme_path()) else: html_theme_path = [theme.get_html_theme_path()] if globals().get('websupport2_base_url', False): websupport2_base_url = 'https://readthedocs.com/websupport' websupport2_static_url = 'https://media.readthedocs.com/' #Add project information to the template context. context = { 'using_theme': using_rtd_theme, 'html_theme': html_theme, 'current_version': "latest", 'version_slug': "latest", 'MEDIA_URL': "https://media.readthedocs.com/media/", 'STATIC_URL': "https://media.readthedocs.com/", 'PRODUCTION_DOMAIN': "readthedocs.com", 'versions': [ ("latest", "/en/latest/"), ], 'downloads': [ ("pdf", "//spinningup.openai.com/_/downloads/en/latest/pdf/"), ("html", "//spinningup.openai.com/_/downloads/en/latest/htmlzip/"), ("epub", "//spinningup.openai.com/_/downloads/en/latest/epub/"), ], 'subprojects': [ ], 'slug': 'openai-education-spinningup', 'name': u'spinningup', 'rtd_language': u'en', 'programming_language': u'words', 'canonical_url': 'https://spinningup.openai.com/en/latest/', 'analytics_code': 'UA-129132782-1', 'single_version': False, 'conf_py_path': '/docs/', 'api_host': 'https://readthedocs.com', 'proxied_api_host': 'https://readthedocs.com', 'github_user': 'openai', 'github_repo': 'spinningup', 'github_version': 'master', 'display_github': True, 'bitbucket_user': 'None', 'bitbucket_repo': 'None', 'bitbucket_version': 'master', 'display_bitbucket': False, 'gitlab_user': 'None', 'gitlab_repo': 'None', 'gitlab_version': 'master', 'display_gitlab': False, 'READTHEDOCS': True, 'using_theme': (html_theme == "default"), 'new_theme': (html_theme == "sphinx_rtd_theme"), 'source_suffix': SUFFIX, 'ad_free': False, 'user_analytics_code': 'UA-129132782-1', 'global_analytics_code': 'UA-17997319-2', 'commit': '0cba2886', } if 'html_context' in globals(): html_context.update(context) else: html_context = context # Add custom RTD extension if 'extensions' in globals(): # Insert at the beginning because it can interfere # with other extensions. # See https://github.com/rtfd/readthedocs.org/pull/4054 extensions.insert(0, "readthedocs_ext.readthedocs") else: extensions = ["readthedocs_ext.readthedocs"] # Add External version warning banner to the external version documentation if 'branch' == 'external': extensions.insert(1, "readthedocs_ext.external_version_warning") project_language = 'en' # User's Sphinx configurations language_user = globals().get('language', None) latex_engine_user = globals().get('latex_engine', None) latex_elements_user = globals().get('latex_elements', None) # Remove this once xindy gets installed in Docker image and XINDYOPS # env variable is supported # https://github.com/rtfd/readthedocs-docker-images/pull/98 latex_use_xindy = False chinese = any([ language_user in ('zh_CN', 'zh_TW'), project_language in ('zh_CN', 'zh_TW'), ]) japanese = any([ language_user == 'ja', project_language == 'ja', ]) if chinese: latex_engine = latex_engine_user or 'xelatex' latex_elements_rtd = { 'preamble': '\\usepackage[UTF8]{ctex}\n', } latex_elements = latex_elements_user or latex_elements_rtd elif japanese: latex_engine = latex_engine_user or 'platex' [rtd-command-info] start-time: 2020-01-30T16:39:08.074653Z, end-time: 2020-01-30T16:40:38.550973Z, duration: 90, exit-code: 0 python sphinx-build -T -E -b readthedocs -d _build/doctrees-readthedocs -D language=en . _build/html Running Sphinx v1.5.6 making output directory... loading translations [en]... done building [mo]: targets for 0 po files that are out of date building [readthedocs]: targets for 37 source files that are out of date updating environment: 37 added, 0 changed, 0 removed reading sources... [ 2%] algorithms/ddpg reading sources... [ 5%] algorithms/ppo reading sources... [ 8%] algorithms/sac reading sources... [ 10%] algorithms/td3 reading sources... [ 13%] algorithms/trpo reading sources... [ 16%] algorithms/vpg reading sources... [ 18%] etc/acknowledgements reading sources... [ 21%] etc/author reading sources... [ 24%] index reading sources... [ 27%] spinningup/bench reading sources... [ 29%] spinningup/bench_ddpg reading sources... [ 32%] spinningup/bench_ppo reading sources... [ 35%] spinningup/bench_sac reading sources... [ 37%] spinningup/bench_td3 reading sources... [ 40%] spinningup/bench_vpg reading sources... [ 43%] spinningup/exercise2_1_soln reading sources... [ 45%] spinningup/exercise2_2_soln reading sources... [ 48%] spinningup/exercises reading sources... [ 51%] spinningup/extra_pg_proof1 reading sources... [ 54%] spinningup/extra_pg_proof2 reading sources... [ 56%] spinningup/extra_tf_pg_implementation reading sources... [ 59%] spinningup/keypapers reading sources... [ 62%] spinningup/rl_intro reading sources... [ 64%] spinningup/rl_intro2 reading sources... [ 67%] spinningup/rl_intro3 reading sources... [ 70%] spinningup/rl_intro4 reading sources... [ 72%] spinningup/spinningup reading sources... [ 75%] user/algorithms reading sources... [ 78%] user/installation reading sources... [ 81%] user/introduction reading sources... [ 83%] user/plotting reading sources... [ 86%] user/running reading sources... [ 89%] user/saving_and_loading reading sources... [ 91%] utils/logger reading sources... [ 94%] utils/mpi reading sources... [ 97%] utils/plotter reading sources... [100%] utils/run_utils /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercises.rst:3: WARNING: Duplicate explicit target name: "solution available here.". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:343: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:419: WARNING: Line block ends without a blank line. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:341: ERROR: Duplicate target name, cannot be used as a unique reference: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:55: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:63: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:163: WARNING: Duplicate explicit target name: "cmdoption--ac_kwargs". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:306: WARNING: Inline strong start-string without end-string. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:148: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:173: WARNING: Duplicate explicit target name: "cmdoption-arg-default". looking for now-outdated files... none found pickling environment... done checking consistency... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ddpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ppo.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_sac.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_td3.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_vpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_1_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_2_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof1.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof2.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:: WARNING: document isn't included in any toctree done preparing documents... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro4.rst:: WARNING: document isn't included in any toctree done writing output... [ 2%] algorithms/ddpg writing output... [ 5%] algorithms/ppo writing output... [ 8%] algorithms/sac writing output... [ 10%] algorithms/td3 writing output... [ 13%] algorithms/trpo writing output... [ 16%] algorithms/vpg writing output... [ 18%] etc/acknowledgements writing output... [ 21%] etc/author writing output... [ 24%] index writing output... [ 27%] spinningup/bench writing output... [ 29%] spinningup/bench_ddpg writing output... [ 32%] spinningup/bench_ppo writing output... [ 35%] spinningup/bench_sac writing output... [ 37%] spinningup/bench_td3 writing output... [ 40%] spinningup/bench_vpg writing output... [ 43%] spinningup/exercise2_1_soln writing output... [ 45%] spinningup/exercise2_2_soln writing output... [ 48%] spinningup/exercises writing output... [ 51%] spinningup/extra_pg_proof1 writing output... [ 54%] spinningup/extra_pg_proof2 writing output... [ 56%] spinningup/extra_tf_pg_implementation writing output... [ 59%] spinningup/keypapers writing output... [ 62%] spinningup/rl_intro writing output... [ 64%] spinningup/rl_intro2 writing output... [ 67%] spinningup/rl_intro3 writing output... [ 70%] spinningup/rl_intro4 writing output... [ 72%] spinningup/spinningup writing output... [ 75%] user/algorithms writing output... [ 78%] user/installation writing output... [ 81%] user/introduction writing output... [ 83%] user/plotting writing output... [ 86%] user/running writing output... [ 89%] user/saving_and_loading writing output... [ 91%] utils/logger writing output... [ 94%] utils/mpi writing output... [ 97%] utils/plotter writing output... [100%] utils/run_utils generating indices... genindex py-modindex highlighting module code... [ 6%] spinup.algos.pytorch.ddpg.ddpg highlighting module code... [ 12%] spinup.algos.tf1.ddpg.ddpg highlighting module code... [ 18%] spinup.algos.pytorch.ppo.ppo highlighting module code... [ 25%] spinup.algos.tf1.ppo.ppo highlighting module code... [ 31%] spinup.algos.pytorch.sac.sac highlighting module code... [ 37%] spinup.algos.tf1.sac.sac highlighting module code... [ 43%] spinup.algos.pytorch.td3.td3 highlighting module code... [ 50%] spinup.algos.tf1.td3.td3 highlighting module code... [ 56%] spinup.algos.tf1.trpo.trpo highlighting module code... [ 62%] spinup.algos.pytorch.vpg.vpg highlighting module code... [ 68%] spinup.algos.tf1.vpg.vpg highlighting module code... [ 75%] spinup.utils.logx highlighting module code... [ 81%] spinup.utils.mpi_tools highlighting module code... [ 87%] spinup.utils.mpi_pytorch highlighting module code... [ 93%] spinup.utils.mpi_tf highlighting module code... [100%] spinup.utils.run_utils writing additional pages... search copying images... [ 2%] images/spinning-up-in-rl.png copying images... [ 4%] spinningup/../images/plots/pyt/pytorch_halfcheetah_performance.svg copying images... [ 7%] spinningup/../images/plots/tf1/tensorflow_halfcheetah_performance.svg copying images... [ 9%] spinningup/../images/plots/pyt/pytorch_hopper_performance.svg copying images... [ 12%] spinningup/../images/plots/tf1/tensorflow_hopper_performance.svg copying images... [ 14%] spinningup/../images/plots/pyt/pytorch_walker2d_performance.svg copying images... [ 17%] spinningup/../images/plots/tf1/tensorflow_walker2d_performance.svg copying images... [ 19%] spinningup/../images/plots/pyt/pytorch_swimmer_performance.svg copying images... [ 21%] spinningup/../images/plots/tf1/tensorflow_swimmer_performance.svg copying images... [ 24%] spinningup/../images/plots/pyt/pytorch_ant_performance.svg copying images... [ 26%] spinningup/../images/plots/tf1/tensorflow_ant_performance.svg copying images... [ 29%] spinningup/../images/plots/ddpg/ddpg_halfcheetah_performance.svg copying images... [ 31%] spinningup/../images/plots/ddpg/ddpg_hopper_performance.svg copying images... [ 34%] spinningup/../images/plots/ddpg/ddpg_walker2d_performance.svg copying images... [ 36%] spinningup/../images/plots/ddpg/ddpg_swimmer_performance.svg copying images... [ 39%] spinningup/../images/plots/ddpg/ddpg_ant_performance.svg copying images... [ 41%] spinningup/../images/plots/ppo/ppo_halfcheetah_performance.svg copying images... [ 43%] spinningup/../images/plots/ppo/ppo_hopper_performance.svg copying images... [ 46%] spinningup/../images/plots/ppo/ppo_walker2d_performance.svg copying images... [ 48%] spinningup/../images/plots/ppo/ppo_swimmer_performance.svg copying images... [ 51%] spinningup/../images/plots/ppo/ppo_ant_performance.svg copying images... [ 53%] spinningup/../images/plots/sac/sac_halfcheetah_performance.svg copying images... [ 56%] spinningup/../images/plots/sac/sac_hopper_performance.svg copying images... [ 58%] spinningup/../images/plots/sac/sac_walker2d_performance.svg copying images... [ 60%] spinningup/../images/plots/sac/sac_swimmer_performance.svg copying images... [ 63%] spinningup/../images/plots/sac/sac_ant_performance.svg copying images... [ 65%] spinningup/../images/plots/td3/td3_halfcheetah_performance.svg copying images... [ 68%] spinningup/../images/plots/td3/td3_hopper_performance.svg copying images... [ 70%] spinningup/../images/plots/td3/td3_walker2d_performance.svg copying images... [ 73%] spinningup/../images/plots/td3/td3_swimmer_performance.svg copying images... [ 75%] spinningup/../images/plots/td3/td3_ant_performance.svg copying images... [ 78%] spinningup/../images/plots/vpg/vpg_halfcheetah_performance.svg copying images... [ 80%] spinningup/../images/plots/vpg/vpg_hopper_performance.svg copying images... [ 82%] spinningup/../images/plots/vpg/vpg_walker2d_performance.svg copying images... [ 85%] spinningup/../images/plots/vpg/vpg_swimmer_performance.svg copying images... [ 87%] spinningup/../images/plots/vpg/vpg_ant_performance.svg copying images... [ 90%] spinningup/../images/ex2-1_trpo_hopper.png copying images... [ 92%] spinningup/../images/ex2-2_ddpg_bug.svg copying images... [ 95%] spinningup/../images/ex2-2_ddpg_bug_pytorch.png copying images... [ 97%] spinningup/../images/rl_diagram_transparent_bg.png copying images... [100%] spinningup/../images/rl_algorithms_9_15.svg copying static files... done WARNING: favicon file 'openai_icon.ico' does not exist copying extra files... done dumping search index in English (code: en) ... done dumping object inventory... done build succeeded, 25 warnings. [rtd-command-info] start-time: 2020-01-30T16:40:38.732632Z, end-time: 2020-01-30T16:41:53.409800Z, duration: 74, exit-code: 0 python sphinx-build -T -b readthedocssinglehtmllocalmedia -d _build/doctrees-readthedocssinglehtmllocalmedia -D language=en . _build/localmedia Running Sphinx v1.5.6 making output directory... loading translations [en]... done loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [readthedocssinglehtmllocalmedia]: all documents updating environment: 37 added, 0 changed, 0 removed reading sources... [ 2%] algorithms/ddpg reading sources... [ 5%] algorithms/ppo reading sources... [ 8%] algorithms/sac reading sources... [ 10%] algorithms/td3 reading sources... [ 13%] algorithms/trpo reading sources... [ 16%] algorithms/vpg reading sources... [ 18%] etc/acknowledgements reading sources... [ 21%] etc/author reading sources... [ 24%] index reading sources... [ 27%] spinningup/bench reading sources... [ 29%] spinningup/bench_ddpg reading sources... [ 32%] spinningup/bench_ppo reading sources... [ 35%] spinningup/bench_sac reading sources... [ 37%] spinningup/bench_td3 reading sources... [ 40%] spinningup/bench_vpg reading sources... [ 43%] spinningup/exercise2_1_soln reading sources... [ 45%] spinningup/exercise2_2_soln reading sources... [ 48%] spinningup/exercises reading sources... [ 51%] spinningup/extra_pg_proof1 reading sources... [ 54%] spinningup/extra_pg_proof2 reading sources... [ 56%] spinningup/extra_tf_pg_implementation reading sources... [ 59%] spinningup/keypapers reading sources... [ 62%] spinningup/rl_intro reading sources... [ 64%] spinningup/rl_intro2 reading sources... [ 67%] spinningup/rl_intro3 reading sources... [ 70%] spinningup/rl_intro4 reading sources... [ 72%] spinningup/spinningup reading sources... [ 75%] user/algorithms reading sources... [ 78%] user/installation reading sources... [ 81%] user/introduction reading sources... [ 83%] user/plotting reading sources... [ 86%] user/running reading sources... [ 89%] user/saving_and_loading reading sources... [ 91%] utils/logger reading sources... [ 94%] utils/mpi reading sources... [ 97%] utils/plotter reading sources... [100%] utils/run_utils /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercises.rst:3: WARNING: Duplicate explicit target name: "solution available here.". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:343: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:419: WARNING: Line block ends without a blank line. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:341: ERROR: Duplicate target name, cannot be used as a unique reference: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:55: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:63: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:163: WARNING: Duplicate explicit target name: "cmdoption--ac_kwargs". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:306: WARNING: Inline strong start-string without end-string. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:148: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:173: WARNING: Duplicate explicit target name: "cmdoption-arg-default". looking for now-outdated files... none found pickling environment... done checking consistency... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ddpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ppo.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_sac.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_td3.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_vpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_1_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_2_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof1.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof2.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro4.rst:: WARNING: document isn't included in any toctree done preparing documents... done assembling single document... user/introduction user/installation user/algorithms user/running user/saving_and_loading user/plotting spinningup/rl_intro spinningup/rl_intro2 spinningup/rl_intro3 spinningup/spinningup spinningup/keypapers spinningup/exercises spinningup/bench algorithms/vpg algorithms/trpo algorithms/ppo algorithms/ddpg algorithms/td3 algorithms/sac utils/logger utils/plotter utils/mpi utils/run_utils etc/acknowledgements etc/author writing... done writing additional files... copying images... [ 7%] images/spinning-up-in-rl.png copying images... [ 15%] spinningup/../images/rl_diagram_transparent_bg.png copying images... [ 23%] spinningup/../images/rl_algorithms_9_15.svg copying images... [ 30%] spinningup/../images/plots/pyt/pytorch_halfcheetah_performance.svg copying images... [ 38%] spinningup/../images/plots/tf1/tensorflow_halfcheetah_performance.svg copying images... [ 46%] spinningup/../images/plots/pyt/pytorch_hopper_performance.svg copying images... [ 53%] spinningup/../images/plots/tf1/tensorflow_hopper_performance.svg copying images... [ 61%] spinningup/../images/plots/pyt/pytorch_walker2d_performance.svg copying images... [ 69%] spinningup/../images/plots/tf1/tensorflow_walker2d_performance.svg copying images... [ 76%] spinningup/../images/plots/pyt/pytorch_swimmer_performance.svg copying images... [ 84%] spinningup/../images/plots/tf1/tensorflow_swimmer_performance.svg copying images... [ 92%] spinningup/../images/plots/pyt/pytorch_ant_performance.svg copying images... [100%] spinningup/../images/plots/tf1/tensorflow_ant_performance.svg copying static files... WARNING: favicon file 'openai_icon.ico' does not exist done copying extra files... done dumping object inventory... done build succeeded, 25 warnings. [rtd-command-info] start-time: 2020-01-30T16:41:53.587879Z, end-time: 2020-01-30T16:42:00.913317Z, duration: 7, exit-code: 0 python sphinx-build -b latex -D language=en -d _build/doctrees . _build/latex Running Sphinx v1.5.6 making output directory... loading translations [en]... done loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [latex]: all documents updating environment: 37 added, 0 changed, 0 removed reading sources... [ 2%] algorithms/ddpg reading sources... [ 5%] algorithms/ppo reading sources... [ 8%] algorithms/sac reading sources... [ 10%] algorithms/td3 reading sources... [ 13%] algorithms/trpo reading sources... [ 16%] algorithms/vpg reading sources... [ 18%] etc/acknowledgements reading sources... [ 21%] etc/author reading sources... [ 24%] index reading sources... [ 27%] spinningup/bench reading sources... [ 29%] spinningup/bench_ddpg reading sources... [ 32%] spinningup/bench_ppo reading sources... [ 35%] spinningup/bench_sac reading sources... [ 37%] spinningup/bench_td3 reading sources... [ 40%] spinningup/bench_vpg reading sources... [ 43%] spinningup/exercise2_1_soln reading sources... [ 45%] spinningup/exercise2_2_soln reading sources... [ 48%] spinningup/exercises reading sources... [ 51%] spinningup/extra_pg_proof1 reading sources... [ 54%] spinningup/extra_pg_proof2 reading sources... [ 56%] spinningup/extra_tf_pg_implementation reading sources... [ 59%] spinningup/keypapers reading sources... [ 62%] spinningup/rl_intro reading sources... [ 64%] spinningup/rl_intro2 reading sources... [ 67%] spinningup/rl_intro3 reading sources... [ 70%] spinningup/rl_intro4 reading sources... [ 72%] spinningup/spinningup reading sources... [ 75%] user/algorithms reading sources... [ 78%] user/installation reading sources... [ 81%] user/introduction reading sources... [ 83%] user/plotting reading sources... [ 86%] user/running reading sources... [ 89%] user/saving_and_loading reading sources... [ 91%] utils/logger reading sources... [ 94%] utils/mpi reading sources... [ 97%] utils/plotter reading sources... [100%] utils/run_utils /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercises.rst:3: WARNING: Duplicate explicit target name: "solution available here.". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:3: WARNING: Duplicate explicit target name: "on github". looking for now-outdated files... none found pickling environment... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:343: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:419: WARNING: Line block ends without a blank line. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:341: ERROR: Duplicate target name, cannot be used as a unique reference: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:55: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:63: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:163: WARNING: Duplicate explicit target name: "cmdoption--ac_kwargs". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:306: WARNING: Inline strong start-string without end-string. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:148: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:173: WARNING: Duplicate explicit target name: "cmdoption-arg-default". done checking consistency... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ddpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ppo.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_sac.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_td3.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_vpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_1_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_2_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof1.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof2.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro4.rst:: WARNING: document isn't included in any toctree done processing SpinningUp.tex... index user/introduction user/installation user/algorithms user/running user/saving_and_loading user/plotting spinningup/rl_intro spinningup/rl_intro2 spinningup/rl_intro3 spinningup/spinningup spinningup/keypapers spinningup/exercises spinningup/bench algorithms/vpg algorithms/trpo algorithms/ppo algorithms/ddpg algorithms/td3 algorithms/sac utils/logger utils/plotter utils/mpi utils/run_utils etc/acknowledgements etc/author resolving references... writing... done copying images... images/spinning-up-in-rl.png spinningup/../images/rl_diagram_transparent_bg.png spinningup/../images/rl_algorithms_9_15.svg spinningup/../images/plots/pyt/pytorch_halfcheetah_performance.svg spinningup/../images/plots/tf1/tensorflow_halfcheetah_performance.svg spinningup/../images/plots/pyt/pytorch_hopper_performance.svg spinningup/../images/plots/tf1/tensorflow_hopper_performance.svg spinningup/../images/plots/pyt/pytorch_walker2d_performance.svg spinningup/../images/plots/tf1/tensorflow_walker2d_performance.svg spinningup/../images/plots/pyt/pytorch_swimmer_performance.svg spinningup/../images/plots/tf1/tensorflow_swimmer_performance.svg spinningup/../images/plots/pyt/pytorch_ant_performance.svg spinningup/../images/plots/tf1/tensorflow_ant_performance.svg copying TeX support files... done build succeeded, 24 warnings. [rtd-command-info] start-time: 2020-01-30T16:42:01.340481Z, end-time: 2020-01-30T16:42:02.957742Z, duration: 1, exit-code: 0 pdflatex -interaction=nonstopmode /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/_build/latex/SpinningUp.tex This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex) restricted \write18 enabled. entering extended mode (/home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/c heckouts/latest/docs/_build/latex/SpinningUp.tex LaTeX2e <2017-04-15> Babel <3.18> and hyphenation patterns for 84 language(s) loaded. (./sphinxmanual.cls Document Class: sphinxmanual 2017/03/26 v1.5.4 Document class (Sphinx manual) (/usr/share/texlive/texmf-dist/tex/latex/base/report.cls Document Class: report 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))) (/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty (/usr/share/texlive/texmf-dist/tex/latex/base/utf8.def (/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.dfu) (/usr/share/texlive/texmf-dist/tex/latex/base/ot1enc.dfu) (/usr/share/texlive/texmf-dist/tex/latex/base/omsenc.dfu))) (/usr/share/texlive/texmf-dist/tex/latex/cmap/cmap.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty (/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.def)<>) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty For additional information on amsmath, use the `?' option. (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty)) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty)) (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amssymb.sty (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amsfonts.sty)) (/usr/share/texlive/texmf-dist/tex/generic/babel/babel.sty (/usr/share/texlive/texmf-dist/tex/generic/babel/switch.def) (/usr/share/texlive/texmf-dist/tex/generic/babel-english/english.ldf (/usr/share/texlive/texmf-dist/tex/generic/babel/babel.def (/usr/share/texlive/texmf-dist/tex/generic/babel/txtbabel.def)))) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/times.sty) (/usr/share/texlive/texmf-dist/tex/latex/fncychap/fncychap.sty) (/usr/share/texlive/texmf-dist/tex/latex/tools/longtable.sty) (./sphinx.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty) (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty) (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/graphics.cfg) (/usr/share/texlive/texmf-dist/tex/latex/graphics-def/pdftex.def))) (/usr/share/texlive/texmf-dist/tex/latex/fancyhdr/fancyhdr.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/textcomp.sty (/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.def (/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.dfu))) (/usr/share/texlive/texmf-dist/tex/latex/titlesec/titlesec.sty) (/usr/share/texlive/texmf-dist/tex/latex/tabulary/tabulary.sty (/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty)) (/usr/share/texlive/texmf-dist/tex/latex/base/makeidx.sty) (/usr/share/texlive/texmf-dist/tex/latex/framed/framed.sty) (/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg)) (/usr/share/texlive/texmf-dist/tex/latex/fancyvrb/fancyvrb.sty Style option: `fancyvrb' v2.7a, with DG/SPQR fixes, and firstline=lastline fix <2008/02/07> (tvz)) (/usr/share/texlive/texmf-dist/tex/latex/threeparttable/threeparttable.sty) (./footnotehyper-sphinx.sty (/usr/share/texlive/texmf-dist/tex/latex/mdwtools/footnote.sty)) (/usr/share/texlive/texmf-dist/tex/latex/float/float.sty) (/usr/share/texlive/texmf-dist/tex/latex/wrapfig/wrapfig.sty) (/usr/share/texlive/texmf-dist/tex/latex/parskip/parskip.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/alltt.sty) (/usr/share/texlive/texmf-dist/tex/latex/upquote/upquote.sty) (/usr/share/texlive/texmf-dist/tex/latex/capt-of/capt-of.sty) (./needspace.sty) (./sphinxhighlight.sty) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/kvoptions.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ltxcmds.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/kvsetkeys.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/infwarerr.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/etexcmds.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifluatex.sty)))) (/usr/share/texlive/texmf-dist/tex/generic/pdftex/pdfcolor.tex) ** (sphinx) defining (legacy) text style macros without \sphinx prefix ** if clashes with packages, set latex_keep_old_macro_names=False in conf.py ) (/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifpdf.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifvtex.sty) (/usr/share/texlive/texmf-dist/tex/generic/ifxetex/ifxetex.sty)) (/usr/share/texlive/texmf-dist/tex/latex/multirow/multirow.sty) (/usr/share/texlive/texmf-dist/tex/latex/eqparbox/eqparbox.sty (/usr/share/texlive/texmf-dist/tex/latex/environ/environ.sty (/usr/share/texlive/texmf-dist/tex/latex/trimspaces/trimspaces.sty))) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/hyperref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-hyperref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-generic.sty)) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/auxhook.sty) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def) (/usr/share/texlive/texmf-dist/tex/latex/latexconfig/hyperref.cfg) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/puenc.def) (/usr/share/texlive/texmf-dist/tex/latex/url/url.sty)) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/hpdftex.def (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/rerunfilecheck.sty)) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/hypcap.sty) (/usr/share/texlive/texmf-dist/tex/latex/algorithms/algorithm.sty (/usr/share/texlive/texmf-dist/tex/latex/base/ifthen.sty)) (/usr/share/texlive/texmf-dist/tex/latex/algorithms/algorithmic.sty) (/usr/share/texlive/texmf-dist/tex/latex/cancel/cancel.sty) Writing index file SpinningUp.idx No file SpinningUp.aux. (/usr/share/texlive/texmf-dist/tex/latex/base/ts1cmr.fd) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1ptm.fd) (/usr/share/texlive/texmf-dist/tex/context/base/mkii/supp-pdf.mkii [Loading MPS to PDF converter (version 2006.09.02).] ) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/epstopdf-base.sty (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/grfext.sty) (/usr/share/texlive/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg)) *geometry* driver: auto-detecting *geometry* detected driver: pdftex (/usr/share/texlive/texmf-dist/tex/latex/hyperref/nameref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/gettitlestring.sty)) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1phv.fd)<><><><> (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsa.fd) (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsb.fd) [1{/var/lib/texmf/fo nts/map/pdftex/updmap/pdftex.map}] [2] [1] [2] (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1pcr.fd) [1 <./spinning-up-in- rl.png>] [2] Chapter 1. (/usr/share/texlive/texmf-dist/tex/latex/psnfss/ts1ptm.fd) LaTeX Warning: Hyper reference `user/introduction:introduction' on page 3 undef ined on input line 80. LaTeX Warning: Hyper reference `user/introduction:what-this-is' on page 3 undef ined on input line 83. LaTeX Warning: Hyper reference `user/introduction:why-we-built-this' on page 3 undefined on input line 86. LaTeX Warning: Hyper reference `user/introduction:how-this-serves-our-mission' on page 3 undefined on input line 89. LaTeX Warning: Hyper reference `user/introduction:code-design-philosophy' on pa ge 3 undefined on input line 92. LaTeX Warning: Hyper reference `user/introduction:long-term-support-and-support -history' on page 3 undefined on input line 95. [3] [4] [5] [6] Chapter 2. LaTeX Warning: Hyper reference `user/installation:installation' on page 7 undef ined on input line 233. LaTeX Warning: Hyper reference `user/installation:installing-python' on page 7 undefined on input line 236. LaTeX Warning: Hyper reference `user/installation:installing-openmpi' on page 7 undefined on input line 239. LaTeX Warning: Hyper reference `user/installation:ubuntu' on page 7 undefined o n input line 242. LaTeX Warning: Hyper reference `user/installation:mac-os-x' on page 7 undefined on input line 245. LaTeX Warning: Hyper reference `user/installation:installing-spinning-up' on pa ge 7 undefined on input line 250. LaTeX Warning: Hyper reference `user/installation:check-your-install' on page 7 undefined on input line 253. LaTeX Warning: Hyper reference `user/installation:installing-mujoco-optional' o n page 7 undefined on input line 256. [7] [8] [9] [10] Chapter 3. LaTeX Warning: Hyper reference `user/algorithms:algorithms' on page 11 undefine d on input line 387. LaTeX Warning: Hyper reference `user/algorithms:what-s-included' on page 11 und efined on input line 390. LaTeX Warning: Hyper reference `user/algorithms:why-these-algorithms' on page 1 1 undefined on input line 393. LaTeX Warning: Hyper reference `user/algorithms:the-on-policy-algorithms' on pa ge 11 undefined on input line 396. LaTeX Warning: Hyper reference `user/algorithms:the-off-policy-algorithms' on p age 11 undefined on input line 399. LaTeX Warning: Hyper reference `user/algorithms:code-format' on page 11 undefin ed on input line 404. LaTeX Warning: Hyper reference `user/algorithms:the-algorithm-function-pytorch- version' on page 11 undefined on input line 407. LaTeX Warning: Hyper reference `user/algorithms:the-algorithm-function-tensorfl ow-version' on page 11 undefined on input line 410. LaTeX Warning: Hyper reference `user/algorithms:the-core-file' on page 11 undef ined on input line 413. [11] [12] [13] [14] Chapter 4. LaTeX Warning: Hyper reference `user/running:running-experiments' on page 15 un defined on input line 607. LaTeX Warning: Hyper reference `user/running:launching-from-the-command-line' o n page 15 undefined on input line 610. LaTeX Warning: Hyper reference `user/running:choosing-pytorch-or-tensorflow-fro m-the-command-line' on page 15 undefined on input line 613. LaTeX Warning: Hyper reference `user/running:setting-hyperparameters-from-the-c ommand-line' on page 15 undefined on input line 616. LaTeX Warning: Hyper reference `user/running:launching-multiple-experiments-at- once' on page 15 undefined on input line 619. LaTeX Warning: Hyper reference `user/running:special-flags' on page 15 undefine d on input line 622. LaTeX Warning: Hyper reference `user/running:environment-flag' on page 15 undef ined on input line 625. LaTeX Warning: Hyper reference `user/running:shortcut-flags' on page 15 undefin ed on input line 628. LaTeX Warning: Hyper reference `user/running:config-flags' on page 15 undefined on input line 631. LaTeX Warning: Hyper reference `user/running:where-results-are-saved' on page 1 5 undefined on input line 636. LaTeX Warning: Hyper reference `user/running:how-is-suffix-determined' on page 15 undefined on input line 639. LaTeX Warning: Hyper reference `user/running:extra' on page 15 undefined on inp ut line 644. LaTeX Warning: Hyper reference `user/running:launching-from-scripts' on page 15 undefined on input line 649. LaTeX Warning: Hyper reference `user/running:using-experimentgrid' on page 15 u ndefined on input line 652. [15] [16] [17] [18] [19] (/usr/share/texlive/texmf-dist/tex/latex/psnfss/ts1pcr.fd) [20] [21] [22] Chapter 5. LaTeX Warning: Hyper reference `user/saving_and_loading:experiment-outputs' on page 23 undefined on input line 1001. LaTeX Warning: Hyper reference `user/saving_and_loading:algorithm-outputs' on p age 23 undefined on input line 1004. LaTeX Warning: Hyper reference `user/saving_and_loading:pytorch-save-directory- info' on page 23 undefined on input line 1007. LaTeX Warning: Hyper reference `user/saving_and_loading:tensorflow-save-directo ry-info' on page 23 undefined on input line 1010. LaTeX Warning: Hyper reference `user/saving_and_loading:save-directory-location ' on page 23 undefined on input line 1015. LaTeX Warning: Hyper reference `user/saving_and_loading:loading-and-running-tra ined-policies' on page 23 undefined on input line 1018. LaTeX Warning: Hyper reference `user/saving_and_loading:if-environment-saves-su ccessfully' on page 23 undefined on input line 1021. LaTeX Warning: Hyper reference `user/saving_and_loading:environment-not-found-e rror' on page 23 undefined on input line 1024. LaTeX Warning: Hyper reference `user/saving_and_loading:using-trained-value-fun ctions' on page 23 undefined on input line 1027. [23] LaTeX Warning: Hyper reference `user/saving_and_loading:details-for-pytorch-sav es-below' on page 24 undefined on input line 1073. LaTeX Warning: Hyper reference `user/saving_and_loading:details-for-tensorflow- saves-below' on page 24 undefined on input line 1082. [24] [25] [26] Underfull \hbox (badness 10000) in paragraph at lines 1293--1293 [][][]\T1/pcr/m/n/10 It[][]looks[][]like[][]the[][]environment[][]wasn\TS1/pcr/ m/n/10 '\T1/pcr/m/n/10 t[][]saved,[][][]and[][]we[][]can\TS1/pcr/m/n/10 '\T1/pc r/m/n/10 t[][]run[][]the[][]agent[][]in[][]it. Underfull \hbox (badness 10000) in paragraph at lines 1295--1295 [][][]\T1/pcr/m/n/10 Check[][]out[][]the[][]readthedocs[][]page[][]on[][]Experi ment[][]Outputs[][]for[][]how[][]to[][]handle[][]this[] [27] [28] Chapter 6. [29] [30] Chapter 7. LaTeX Warning: Hyper reference `spinningup/rl_intro:part-1-key-concepts-in-rl' on page 31 undefined on input line 1426. LaTeX Warning: Hyper reference `spinningup/rl_intro:what-can-rl-do' on page 31 undefined on input line 1429. LaTeX Warning: Hyper reference `spinningup/rl_intro:key-concepts-and-terminolog y' on page 31 undefined on input line 1432. LaTeX Warning: Hyper reference `spinningup/rl_intro:optional-formalism' on page 31 undefined on input line 1435. [31] [32 <./rl_diagram_transparent_bg.png>] [33] [34] [35] [36] [37] [38] [39] [40] Chapter 8. LaTeX Warning: Hyper reference `spinningup/rl_intro2:part-2-kinds-of-rl-algorit hms' on page 41 undefined on input line 1839. LaTeX Warning: Hyper reference `spinningup/rl_intro2:a-taxonomy-of-rl-algorithm s' on page 41 undefined on input line 1842. LaTeX Warning: Hyper reference `spinningup/rl_intro2:links-to-algorithms-in-tax onomy' on page 41 undefined on input line 1845. ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.1860 ...ncludegraphics{{rl_algorithms_9_15}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.1860 ...ncludegraphics{{rl_algorithms_9_15}.svg} LaTeX Warning: Hyper reference `spinningup/rl_intro2:citations-below' on page 4 1 undefined on input line 1861. [41] [42] [43] [44] Chapter 9. LaTeX Warning: Hyper reference `spinningup/rl_intro3:part-3-intro-to-policy-opt imization' on page 45 undefined on input line 2002. LaTeX Warning: Hyper reference `spinningup/rl_intro3:deriving-the-simplest-poli cy-gradient' on page 45 undefined on input line 2005. LaTeX Warning: Hyper reference `spinningup/rl_intro3:implementing-the-simplest- policy-gradient' on page 45 undefined on input line 2008. LaTeX Warning: Hyper reference `spinningup/rl_intro3:expected-grad-log-prob-lem ma' on page 45 undefined on input line 2011. LaTeX Warning: Hyper reference `spinningup/rl_intro3:don-t-let-the-past-distrac t-you' on page 45 undefined on input line 2014. LaTeX Warning: Hyper reference `spinningup/rl_intro3:implementing-reward-to-go- policy-gradient' on page 45 undefined on input line 2017. LaTeX Warning: Hyper reference `spinningup/rl_intro3:baselines-in-policy-gradie nts' on page 45 undefined on input line 2020. LaTeX Warning: Hyper reference `spinningup/rl_intro3:other-forms-of-the-policy- gradient' on page 45 undefined on input line 2023. LaTeX Warning: Hyper reference `spinningup/rl_intro3:recap' on page 45 undefine d on input line 2026. [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] Chapter 10. LaTeX Warning: Hyper reference `spinningup/spinningup:spinning-up-as-a-deep-rl- researcher' on page 55 undefined on input line 2448. LaTeX Warning: Hyper reference `spinningup/spinningup:the-right-background' on page 55 undefined on input line 2451. LaTeX Warning: Hyper reference `spinningup/spinningup:learn-by-doing' on page 5 5 undefined on input line 2454. LaTeX Warning: Hyper reference `spinningup/spinningup:developing-a-research-pro ject' on page 55 undefined on input line 2457. LaTeX Warning: Hyper reference `spinningup/spinningup:doing-rigorous-research-i n-rl' on page 55 undefined on input line 2460. LaTeX Warning: Hyper reference `spinningup/spinningup:closing-thoughts' on page 55 undefined on input line 2463. LaTeX Warning: Hyper reference `spinningup/spinningup:ps-other-resources' on pa ge 55 undefined on input line 2466. LaTeX Warning: Hyper reference `spinningup/spinningup:references' on page 55 un defined on input line 2469. [55] [56] [57] [58] [59] [60] Chapter 11. LaTeX Warning: Hyper reference `spinningup/keypapers:key-papers-in-deep-rl' on page 61 undefined on input line 2578. LaTeX Warning: Hyper reference `spinningup/keypapers:model-free-rl' on page 61 undefined on input line 2581. LaTeX Warning: Hyper reference `spinningup/keypapers:exploration' on page 61 un defined on input line 2584. LaTeX Warning: Hyper reference `spinningup/keypapers:transfer-and-multitask-rl' on page 61 undefined on input line 2587. LaTeX Warning: Hyper reference `spinningup/keypapers:hierarchy' on page 61 unde fined on input line 2590. LaTeX Warning: Hyper reference `spinningup/keypapers:memory' on page 61 undefin ed on input line 2593. LaTeX Warning: Hyper reference `spinningup/keypapers:model-based-rl' on page 61 undefined on input line 2596. LaTeX Warning: Hyper reference `spinningup/keypapers:meta-rl' on page 61 undefi ned on input line 2599. LaTeX Warning: Hyper reference `spinningup/keypapers:scaling-rl' on page 61 und efined on input line 2602. LaTeX Warning: Hyper reference `spinningup/keypapers:rl-in-the-real-world' on p age 61 undefined on input line 2605. LaTeX Warning: Hyper reference `spinningup/keypapers:safety' on page 61 undefin ed on input line 2608. LaTeX Warning: Hyper reference `spinningup/keypapers:imitation-learning-and-inv erse-reinforcement-learning' on page 61 undefined on input line 2611. LaTeX Warning: Hyper reference `spinningup/keypapers:reproducibility-analysis-a nd-critique' on page 61 undefined on input line 2614. LaTeX Warning: Hyper reference `spinningup/keypapers:bonus-classic-papers-in-rl -theory-or-review' on page 61 undefined on input line 2617. [61] [62] Overfull \vbox (103.35579pt too high) has occurred while \output is active [63] [64] Chapter 12. LaTeX Warning: Hyper reference `spinningup/exercises:exercises' on page 65 unde fined on input line 2706. LaTeX Warning: Hyper reference `spinningup/exercises:problem-set-1-basics-of-im plementation' on page 65 undefined on input line 2709. LaTeX Warning: Hyper reference `spinningup/exercises:problem-set-2-algorithm-fa ilure-modes' on page 65 undefined on input line 2712. LaTeX Warning: Hyper reference `spinningup/exercises:challenges' on page 65 und efined on input line 2715. Underfull \hbox (badness 10000) in paragraph at lines 2740--2741 []\T1/ptm/m/n/10 PyTorch ver-sion: \T1/pcr/m/n/10 spinup/exercises/pytorch/prob lem_set_1_solutions/ Underfull \hbox (badness 10000) in paragraph at lines 2743--2744 []\T1/ptm/m/n/10 Tensorflow ver-sion: \T1/pcr/m/n/10 spinup/exercises/tf1/probl em_set_1_solutions/ [65] Underfull \hbox (badness 10000) in paragraph at lines 2771--2772 []\T1/ptm/m/n/10 PyTorch ver-sion: \T1/pcr/m/n/10 spinup/exercises/pytorch/prob lem_set_1_solutions/ Underfull \hbox (badness 10000) in paragraph at lines 2774--2775 []\T1/ptm/m/n/10 Tensorflow ver-sion: \T1/pcr/m/n/10 spinup/exercises/tf1/probl em_set_1_solutions/ [66] [67] [68] Chapter 13. LaTeX Warning: Hyper reference `spinningup/bench:benchmarks-for-spinning-up-imp lementations' on page 69 undefined on input line 2910. LaTeX Warning: Hyper reference `spinningup/bench:performance-in-each-environmen t' on page 69 undefined on input line 2913. LaTeX Warning: Hyper reference `spinningup/bench:halfcheetah-pytorch-versions' on page 69 undefined on input line 2916. LaTeX Warning: Hyper reference `spinningup/bench:halfcheetah-tensorflow-version s' on page 69 undefined on input line 2919. LaTeX Warning: Hyper reference `spinningup/bench:hopper-pytorch-versions' on pa ge 69 undefined on input line 2922. LaTeX Warning: Hyper reference `spinningup/bench:hopper-tensorflow-versions' on page 69 undefined on input line 2925. LaTeX Warning: Hyper reference `spinningup/bench:walker2d-pytorch-versions' on page 69 undefined on input line 2928. LaTeX Warning: Hyper reference `spinningup/bench:walker2d-tensorflow-versions' on page 69 undefined on input line 2931. LaTeX Warning: Hyper reference `spinningup/bench:swimmer-pytorch-versions' on p age 69 undefined on input line 2934. LaTeX Warning: Hyper reference `spinningup/bench:swimmer-tensorflow-versions' o n page 69 undefined on input line 2937. LaTeX Warning: Hyper reference `spinningup/bench:ant-pytorch-versions' on page 69 undefined on input line 2940. LaTeX Warning: Hyper reference `spinningup/bench:ant-tensorflow-versions' on pa ge 69 undefined on input line 2943. LaTeX Warning: Hyper reference `spinningup/bench:experiment-details' on page 69 undefined on input line 2948. LaTeX Warning: Hyper reference `spinningup/bench:pytorch-vs-tensorflow' on page 69 undefined on input line 2951. ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2969 ...s{{pytorch_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2969 ...s{{pytorch_halfcheetah_performance}.svg} [69] ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2978 ...tensorflow_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2978 ...tensorflow_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2987 ...aphics{{pytorch_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2987 ...aphics{{pytorch_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2996 ...ics{{tensorflow_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2996 ...ics{{tensorflow_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3005 ...hics{{pytorch_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3005 ...hics{{pytorch_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3014 ...s{{tensorflow_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3014 ...s{{tensorflow_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3023 ...phics{{pytorch_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3023 ...phics{{pytorch_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3032 ...cs{{tensorflow_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3032 ...cs{{tensorflow_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3041 ...egraphics{{pytorch_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3041 ...egraphics{{pytorch_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3050 ...aphics{{tensorflow_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3050 ...aphics{{tensorflow_ant_performance}.svg} [70] [71] [72] Chapter 14. LaTeX Warning: Hyper reference `algorithms/vpg:vanilla-policy-gradient' on page 73 undefined on input line 3104. LaTeX Warning: Hyper reference `algorithms/vpg:background' on page 73 undefined on input line 3107. LaTeX Warning: Hyper reference `algorithms/vpg:quick-facts' on page 73 undefine d on input line 3110. LaTeX Warning: Hyper reference `algorithms/vpg:key-equations' on page 73 undefi ned on input line 3113. LaTeX Warning: Hyper reference `algorithms/vpg:exploration-vs-exploitation' on page 73 undefined on input line 3116. LaTeX Warning: Hyper reference `algorithms/vpg:pseudocode' on page 73 undefined on input line 3119. LaTeX Warning: Hyper reference `algorithms/vpg:documentation' on page 73 undefi ned on input line 3124. LaTeX Warning: Hyper reference `algorithms/vpg:documentation-pytorch-version' o n page 73 undefined on input line 3127. LaTeX Warning: Hyper reference `algorithms/vpg:saved-model-contents-pytorch-ver sion' on page 73 undefined on input line 3130. LaTeX Warning: Hyper reference `algorithms/vpg:documentation-tensorflow-version ' on page 73 undefined on input line 3133. LaTeX Warning: Hyper reference `algorithms/vpg:saved-model-contents-tensorflow- version' on page 73 undefined on input line 3136. LaTeX Warning: Hyper reference `algorithms/vpg:references' on page 73 undefined on input line 3141. LaTeX Warning: Hyper reference `algorithms/vpg:relevant-papers' on page 73 unde fined on input line 3144. LaTeX Warning: Hyper reference `algorithms/vpg:why-these-papers' on page 73 und efined on input line 3147. LaTeX Warning: Hyper reference `algorithms/vpg:other-public-implementations' on page 73 undefined on input line 3150. [73] [74] Underfull \hbox (badness 10000) in paragraph at lines 3243--3243 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 5862) in paragraph at lines 3243--3243 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , [75] [76] Underfull \hbox (badness 10000) in paragraph at lines 3436--3436 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3436--3436 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_l r=0.0003\T1/ptm/m/n/10 , \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , [77] [78] [79] [80] Chapter 15. LaTeX Warning: Hyper reference `algorithms/trpo:trust-region-policy-optimizatio n' on page 81 undefined on input line 3640. LaTeX Warning: Hyper reference `algorithms/trpo:background' on page 81 undefine d on input line 3643. LaTeX Warning: Hyper reference `algorithms/trpo:quick-facts' on page 81 undefin ed on input line 3646. LaTeX Warning: Hyper reference `algorithms/trpo:key-equations' on page 81 undef ined on input line 3649. LaTeX Warning: Hyper reference `algorithms/trpo:exploration-vs-exploitation' on page 81 undefined on input line 3652. LaTeX Warning: Hyper reference `algorithms/trpo:pseudocode' on page 81 undefine d on input line 3655. LaTeX Warning: Hyper reference `algorithms/trpo:documentation' on page 81 undef ined on input line 3660. LaTeX Warning: Hyper reference `algorithms/trpo:saved-model-contents' on page 8 1 undefined on input line 3663. LaTeX Warning: Hyper reference `algorithms/trpo:references' on page 81 undefine d on input line 3668. LaTeX Warning: Hyper reference `algorithms/trpo:relevant-papers' on page 81 und efined on input line 3671. LaTeX Warning: Hyper reference `algorithms/trpo:why-these-papers' on page 81 un defined on input line 3674. LaTeX Warning: Hyper reference `algorithms/trpo:other-public-implementations' o n page 81 undefined on input line 3677. [81] [82] [83] Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 delt a=0.01\T1/ptm/m/n/10 , \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 \T1/ptm/m/it/10 train_v_iters=80\T1/ptm/m/n/10 , \T1/ptm/m/it/10 damp-ing_coeff =0.1\T1/ptm/m/n/10 , \T1/ptm/m/it/10 cg_iters=10\T1/ptm/m/n/10 , \T1/ptm/m/it/1 0 back-track_iters=10\T1/ptm/m/n/10 , \T1/ptm/m/it/10 back- [84] [85] Overfull \vbox (100.02797pt too high) has occurred while \output is active [86] [87] [88] Chapter 16. LaTeX Warning: Hyper reference `algorithms/ppo:proximal-policy-optimization' on page 89 undefined on input line 4096. LaTeX Warning: Hyper reference `algorithms/ppo:background' on page 89 undefined on input line 4099. LaTeX Warning: Hyper reference `algorithms/ppo:quick-facts' on page 89 undefine d on input line 4102. LaTeX Warning: Hyper reference `algorithms/ppo:key-equations' on page 89 undefi ned on input line 4105. LaTeX Warning: Hyper reference `algorithms/ppo:exploration-vs-exploitation' on page 89 undefined on input line 4108. LaTeX Warning: Hyper reference `algorithms/ppo:pseudocode' on page 89 undefined on input line 4111. LaTeX Warning: Hyper reference `algorithms/ppo:documentation' on page 89 undefi ned on input line 4116. LaTeX Warning: Hyper reference `algorithms/ppo:documentation-pytorch-version' o n page 89 undefined on input line 4119. LaTeX Warning: Hyper reference `algorithms/ppo:saved-model-contents-pytorch-ver sion' on page 89 undefined on input line 4122. LaTeX Warning: Hyper reference `algorithms/ppo:documentation-tensorflow-version ' on page 89 undefined on input line 4125. LaTeX Warning: Hyper reference `algorithms/ppo:saved-model-contents-tensorflow- version' on page 89 undefined on input line 4128. LaTeX Warning: Hyper reference `algorithms/ppo:references' on page 89 undefined on input line 4133. LaTeX Warning: Hyper reference `algorithms/ppo:relevant-papers' on page 89 unde fined on input line 4136. LaTeX Warning: Hyper reference `algorithms/ppo:why-these-papers' on page 89 und efined on input line 4139. LaTeX Warning: Hyper reference `algorithms/ppo:other-public-implementations' on page 89 undefined on input line 4142. [89] [90] [91] Underfull \hbox (badness 10000) in paragraph at lines 4289--4289 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 5862) in paragraph at lines 4289--4289 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , [92] [93] Underfull \hbox (badness 10000) in paragraph at lines 4500--4500 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 5274) in paragraph at lines 4500--4500 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 clip _ratio=0.2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_lr=0.0003\T1/ptm/m/n/10 , Underfull \hbox (badness 6461) in paragraph at lines 4500--4500 \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 train_pi_iters=80\T 1/ptm/m/n/10 , \T1/ptm/m/it/10 train_v_iters=80\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lam=0.97\T1/ptm/m/n/10 , \T1/ptm/m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , [94] [95] [96] [97] [98] Chapter 17. LaTeX Warning: Hyper reference `algorithms/ddpg:deep-deterministic-policy-gradi ent' on page 99 undefined on input line 4725. LaTeX Warning: Hyper reference `algorithms/ddpg:background' on page 99 undefine d on input line 4728. LaTeX Warning: Hyper reference `algorithms/ddpg:quick-facts' on page 99 undefin ed on input line 4731. LaTeX Warning: Hyper reference `algorithms/ddpg:key-equations' on page 99 undef ined on input line 4734. LaTeX Warning: Hyper reference `algorithms/ddpg:the-q-learning-side-of-ddpg' on page 99 undefined on input line 4737. LaTeX Warning: Hyper reference `algorithms/ddpg:the-policy-learning-side-of-ddp g' on page 99 undefined on input line 4740. LaTeX Warning: Hyper reference `algorithms/ddpg:exploration-vs-exploitation' on page 99 undefined on input line 4745. LaTeX Warning: Hyper reference `algorithms/ddpg:pseudocode' on page 99 undefine d on input line 4748. LaTeX Warning: Hyper reference `algorithms/ddpg:documentation' on page 99 undef ined on input line 4753. LaTeX Warning: Hyper reference `algorithms/ddpg:documentation-pytorch-version' on page 99 undefined on input line 4756. LaTeX Warning: Hyper reference `algorithms/ddpg:saved-model-contents-pytorch-ve rsion' on page 99 undefined on input line 4759. LaTeX Warning: Hyper reference `algorithms/ddpg:documentation-tensorflow-versio n' on page 99 undefined on input line 4762. LaTeX Warning: Hyper reference `algorithms/ddpg:saved-model-contents-tensorflow -version' on page 99 undefined on input line 4765. LaTeX Warning: Hyper reference `algorithms/ddpg:references' on page 99 undefine d on input line 4770. LaTeX Warning: Hyper reference `algorithms/ddpg:relevant-papers' on page 99 und efined on input line 4773. LaTeX Warning: Hyper reference `algorithms/ddpg:why-these-papers' on page 99 un defined on input line 4776. LaTeX Warning: Hyper reference `algorithms/ddpg:other-public-implementations' o n page 99 undefined on input line 4779. [99] [100] [101] [102] Underfull \hbox (badness 10000) in paragraph at lines 4948--4948 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 4948--4948 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- [103] [104] Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_lr=0.001\T1/ptm /m/n/10 , \T1/ptm/m/it/10 q_lr=0.001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 batch_size =100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=10000\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_ever y=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 act_noise=0.1\T1/ptm/m/n/10 , \T1/ptm/m/it /10 num_test_episodes=10\T1/ptm/m/n/10 , [105] [106] [107] [108] Chapter 18. LaTeX Warning: Hyper reference `algorithms/td3:twin-delayed-ddpg' on page 109 u ndefined on input line 5339. LaTeX Warning: Hyper reference `algorithms/td3:background' on page 109 undefine d on input line 5342. LaTeX Warning: Hyper reference `algorithms/td3:quick-facts' on page 109 undefin ed on input line 5345. LaTeX Warning: Hyper reference `algorithms/td3:key-equations' on page 109 undef ined on input line 5348. LaTeX Warning: Hyper reference `algorithms/td3:exploration-vs-exploitation' on page 109 undefined on input line 5351. LaTeX Warning: Hyper reference `algorithms/td3:pseudocode' on page 109 undefine d on input line 5354. LaTeX Warning: Hyper reference `algorithms/td3:documentation' on page 109 undef ined on input line 5359. LaTeX Warning: Hyper reference `algorithms/td3:documentation-pytorch-version' o n page 109 undefined on input line 5362. LaTeX Warning: Hyper reference `algorithms/td3:saved-model-contents-pytorch-ver sion' on page 109 undefined on input line 5365. LaTeX Warning: Hyper reference `algorithms/td3:documentation-tensorflow-version ' on page 109 undefined on input line 5368. LaTeX Warning: Hyper reference `algorithms/td3:saved-model-contents-tensorflow- version' on page 109 undefined on input line 5371. LaTeX Warning: Hyper reference `algorithms/td3:references' on page 109 undefine d on input line 5376. LaTeX Warning: Hyper reference `algorithms/td3:relevant-papers' on page 109 und efined on input line 5379. LaTeX Warning: Hyper reference `algorithms/td3:other-public-implementations' on page 109 undefined on input line 5382. [109] [110] [111] [112] Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 batch_size=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=1000 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/ it/10 up-date_every=50\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 act_noise=0.1\T1/ptm/m/n/10 , \T1/ptm/m/it/10 tar-get_noise=0.2 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 noise_clip=0.5\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pol-icy_delay=2\T1/ptm/m/n/10 , [113] [114] Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 \T1/ptm/m/it/10 pol-icy_delay=2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_test_episod es=10\T1/ptm/m/n/10 , \T1/ptm/m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 log-ger_kwargs={}\T1/ptm/m/n/10 , [115] [116] [117] [118] Chapter 19. LaTeX Warning: Hyper reference `algorithms/sac:soft-actor-critic' on page 119 u ndefined on input line 5950. LaTeX Warning: Hyper reference `algorithms/sac:background' on page 119 undefine d on input line 5953. LaTeX Warning: Hyper reference `algorithms/sac:quick-facts' on page 119 undefin ed on input line 5956. LaTeX Warning: Hyper reference `algorithms/sac:key-equations' on page 119 undef ined on input line 5959. LaTeX Warning: Hyper reference `algorithms/sac:entropy-regularized-reinforcemen t-learning' on page 119 undefined on input line 5962. LaTeX Warning: Hyper reference `algorithms/sac:id1' on page 119 undefined on in put line 5965. LaTeX Warning: Hyper reference `algorithms/sac:exploration-vs-exploitation' on page 119 undefined on input line 5970. LaTeX Warning: Hyper reference `algorithms/sac:pseudocode' on page 119 undefine d on input line 5973. LaTeX Warning: Hyper reference `algorithms/sac:documentation' on page 119 undef ined on input line 5978. LaTeX Warning: Hyper reference `algorithms/sac:documentation-pytorch-version' o n page 119 undefined on input line 5981. LaTeX Warning: Hyper reference `algorithms/sac:saved-model-contents-pytorch-ver sion' on page 119 undefined on input line 5984. LaTeX Warning: Hyper reference `algorithms/sac:documentation-tensorflow-version ' on page 119 undefined on input line 5987. LaTeX Warning: Hyper reference `algorithms/sac:saved-model-contents-tensorflow- version' on page 119 undefined on input line 5990. LaTeX Warning: Hyper reference `algorithms/sac:references' on page 119 undefine d on input line 5995. LaTeX Warning: Hyper reference `algorithms/sac:relevant-papers' on page 119 und efined on input line 5998. LaTeX Warning: Hyper reference `algorithms/sac:other-public-implementations' on page 119 undefined on input line 6001. [119] [120] [121] [122] [123] Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1 /ptm/m/n/10 , \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lr=0 .001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 al-pha=0.2\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 batch_size=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=1000 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/ it/10 up-date_every=50\T1/ptm/m/n/10 , [124] [125] [126] Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lr=0.001\T1/ptm/m/ n/10 , \T1/ptm/m/it/10 al-pha=0.2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 batch_size=10 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=10000\T1/ptm/m/n/10 , \T1/ptm/m/i t/10 up- Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_every=5 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_test_episodes=10\T1/ptm/m/n/10 , \T1/ptm/ m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , [127] [128] [129] [130] Chapter 20. LaTeX Warning: Hyper reference `utils/logger:logger' on page 131 undefined on i nput line 6687. LaTeX Warning: Hyper reference `utils/logger:using-a-logger' on page 131 undefi ned on input line 6690. LaTeX Warning: Hyper reference `utils/logger:examples' on page 131 undefined on input line 6693. LaTeX Warning: Hyper reference `utils/logger:logging-and-pytorch' on page 131 u ndefined on input line 6696. LaTeX Warning: Hyper reference `utils/logger:logging-and-mpi' on page 131 undef ined on input line 6699. LaTeX Warning: Hyper reference `utils/logger:logger-classes' on page 131 undefi ned on input line 6704. LaTeX Warning: Hyper reference `utils/logger:loading-saved-models-pytorch-only' on page 131 undefined on input line 6707. LaTeX Warning: Hyper reference `utils/logger:loading-saved-graphs-tensorflow-on ly' on page 131 undefined on input line 6710. [131] [132] [133] [134] LaTeX Warning: Hyper reference `utils/logger:spinup.utils.logx.Logger' on page 135 undefined on input line 7035. [135] [136] [137] [138] Chapter 21. [139] [140] Chapter 22. LaTeX Warning: Hyper reference `utils/mpi:mpi-tools' on page 141 undefined on i nput line 7195. LaTeX Warning: Hyper reference `utils/mpi:module-spinup.utils.mpi_tools' on pag e 141 undefined on input line 7198. LaTeX Warning: Hyper reference `utils/mpi:mpi-pytorch-utilities' on page 141 un defined on input line 7201. LaTeX Warning: Hyper reference `utils/mpi:mpi-tensorflow-utilities' on page 141 undefined on input line 7204. [141] [142] [143] [144] Chapter 23. LaTeX Warning: Hyper reference `utils/run_utils:run-utils' on page 145 undefine d on input line 7382. LaTeX Warning: Hyper reference `utils/run_utils:experimentgrid' on page 145 und efined on input line 7385. LaTeX Warning: Hyper reference `utils/run_utils:calling-experiments' on page 14 5 undefined on input line 7388. [145] Underfull \hbox (badness 10000) in paragraph at lines 7540--7540 []\T1/ptm/m/it/10 exp_name\T1/ptm/m/n/10 , \T1/ptm/m/it/10 thunk\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_cpu=1\T1/ptm/m/n/1 0 , [146] [147] [148] Chapter 24. [149] [150] Chapter 25. [151] [152] Chapter 26. [153] [154] LaTeX Warning: Reference `utils/mpi:module-spinup.utils.mpi_pytorch' on page 15 5 undefined on input line 7673. LaTeX Warning: Reference `utils/mpi:module-spinup.utils.mpi_tf' on page 155 und efined on input line 7674. LaTeX Warning: Reference `utils/mpi:module-spinup.utils.mpi_tools' on page 155 undefined on input line 7675. [155] No file SpinningUp.ind. (./SpinningUp.aux) Package rerunfilecheck Warning: File `SpinningUp.out' has changed. (rerunfilecheck) Rerun to get outlines right (rerunfilecheck) or use package `bookmark'. LaTeX Warning: There were undefined references. LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right. ) (see the transcript file for additional information){/usr/share/texlive/texmf-d ist/fonts/enc/dvips/base/8r.enc}< /usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr5.pfb> Output written on SpinningUp.pdf (159 pages, 1179834 bytes). Transcript written on SpinningUp.log. [rtd-command-info] start-time: 2020-01-30T16:42:03.047486Z, end-time: 2020-01-30T16:42:03.401564Z, duration: 0, exit-code: 0 makeindex -s python.ist SpinningUp.idx This is makeindex, version 2.15 [TeX Live 2017] (kpathsea + Thai support). Scanning style file ./python.ist.......done (7 attributes redefined, 0 ignored). Scanning input file SpinningUp.idx....done (88 entries accepted, 0 rejected). Sorting entries....done (589 comparisons). Generating output file SpinningUp.ind....done (154 lines written, 0 warnings). Output written in SpinningUp.ind. Transcript written in SpinningUp.ilg. [rtd-command-info] start-time: 2020-01-30T16:42:03.464151Z, end-time: 2020-01-30T16:42:04.968952Z, duration: 1, exit-code: 0 pdflatex -interaction=nonstopmode /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/_build/latex/SpinningUp.tex This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex) restricted \write18 enabled. entering extended mode (/home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/c heckouts/latest/docs/_build/latex/SpinningUp.tex LaTeX2e <2017-04-15> Babel <3.18> and hyphenation patterns for 84 language(s) loaded. (./sphinxmanual.cls Document Class: sphinxmanual 2017/03/26 v1.5.4 Document class (Sphinx manual) (/usr/share/texlive/texmf-dist/tex/latex/base/report.cls Document Class: report 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))) (/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty (/usr/share/texlive/texmf-dist/tex/latex/base/utf8.def (/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.dfu) (/usr/share/texlive/texmf-dist/tex/latex/base/ot1enc.dfu) (/usr/share/texlive/texmf-dist/tex/latex/base/omsenc.dfu))) (/usr/share/texlive/texmf-dist/tex/latex/cmap/cmap.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty (/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.def)<>) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty For additional information on amsmath, use the `?' option. (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty)) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty)) (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amssymb.sty (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amsfonts.sty)) (/usr/share/texlive/texmf-dist/tex/generic/babel/babel.sty (/usr/share/texlive/texmf-dist/tex/generic/babel/switch.def) (/usr/share/texlive/texmf-dist/tex/generic/babel-english/english.ldf (/usr/share/texlive/texmf-dist/tex/generic/babel/babel.def (/usr/share/texlive/texmf-dist/tex/generic/babel/txtbabel.def)))) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/times.sty) (/usr/share/texlive/texmf-dist/tex/latex/fncychap/fncychap.sty) (/usr/share/texlive/texmf-dist/tex/latex/tools/longtable.sty) (./sphinx.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty) (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty) (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/graphics.cfg) (/usr/share/texlive/texmf-dist/tex/latex/graphics-def/pdftex.def))) (/usr/share/texlive/texmf-dist/tex/latex/fancyhdr/fancyhdr.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/textcomp.sty (/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.def (/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.dfu))) (/usr/share/texlive/texmf-dist/tex/latex/titlesec/titlesec.sty) (/usr/share/texlive/texmf-dist/tex/latex/tabulary/tabulary.sty (/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty)) (/usr/share/texlive/texmf-dist/tex/latex/base/makeidx.sty) (/usr/share/texlive/texmf-dist/tex/latex/framed/framed.sty) (/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg)) (/usr/share/texlive/texmf-dist/tex/latex/fancyvrb/fancyvrb.sty Style option: `fancyvrb' v2.7a, with DG/SPQR fixes, and firstline=lastline fix <2008/02/07> (tvz)) (/usr/share/texlive/texmf-dist/tex/latex/threeparttable/threeparttable.sty) (./footnotehyper-sphinx.sty (/usr/share/texlive/texmf-dist/tex/latex/mdwtools/footnote.sty)) (/usr/share/texlive/texmf-dist/tex/latex/float/float.sty) (/usr/share/texlive/texmf-dist/tex/latex/wrapfig/wrapfig.sty) (/usr/share/texlive/texmf-dist/tex/latex/parskip/parskip.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/alltt.sty) (/usr/share/texlive/texmf-dist/tex/latex/upquote/upquote.sty) (/usr/share/texlive/texmf-dist/tex/latex/capt-of/capt-of.sty) (./needspace.sty) (./sphinxhighlight.sty) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/kvoptions.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ltxcmds.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/kvsetkeys.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/infwarerr.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/etexcmds.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifluatex.sty)))) (/usr/share/texlive/texmf-dist/tex/generic/pdftex/pdfcolor.tex) ** (sphinx) defining (legacy) text style macros without \sphinx prefix ** if clashes with packages, set latex_keep_old_macro_names=False in conf.py ) (/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifpdf.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifvtex.sty) (/usr/share/texlive/texmf-dist/tex/generic/ifxetex/ifxetex.sty)) (/usr/share/texlive/texmf-dist/tex/latex/multirow/multirow.sty) (/usr/share/texlive/texmf-dist/tex/latex/eqparbox/eqparbox.sty (/usr/share/texlive/texmf-dist/tex/latex/environ/environ.sty (/usr/share/texlive/texmf-dist/tex/latex/trimspaces/trimspaces.sty))) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/hyperref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-hyperref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-generic.sty)) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/auxhook.sty) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def) (/usr/share/texlive/texmf-dist/tex/latex/latexconfig/hyperref.cfg) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/puenc.def) (/usr/share/texlive/texmf-dist/tex/latex/url/url.sty)) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/hpdftex.def (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/rerunfilecheck.sty)) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/hypcap.sty) (/usr/share/texlive/texmf-dist/tex/latex/algorithms/algorithm.sty (/usr/share/texlive/texmf-dist/tex/latex/base/ifthen.sty)) (/usr/share/texlive/texmf-dist/tex/latex/algorithms/algorithmic.sty) (/usr/share/texlive/texmf-dist/tex/latex/cancel/cancel.sty) Writing index file SpinningUp.idx (./SpinningUp.aux LaTeX Warning: Label `alg1' multiply defined. LaTeX Warning: Label `alg1' multiply defined. LaTeX Warning: Label `alg1' multiply defined. LaTeX Warning: Label `alg1' multiply defined. LaTeX Warning: Label `alg1' multiply defined. ) (/usr/share/texlive/texmf-dist/tex/latex/base/ts1cmr.fd) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1ptm.fd) (/usr/share/texlive/texmf-dist/tex/context/base/mkii/supp-pdf.mkii [Loading MPS to PDF converter (version 2006.09.02).] ) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/epstopdf-base.sty (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/grfext.sty) (/usr/share/texlive/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg)) *geometry* driver: auto-detecting *geometry* detected driver: pdftex (/usr/share/texlive/texmf-dist/tex/latex/hyperref/nameref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/gettitlestring.sty)) (./SpinningUp.out) (./SpinningUp.out) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1phv.fd)<><><><> (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsa.fd) (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsb.fd) [1{/var/lib/texmf/fo nts/map/pdftex/updmap/pdftex.map}] [2] (./SpinningUp.toc [1] [2]) [3] [4] (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1pcr.fd) [1 <./spinning-up-in- rl.png>] [2] Chapter 1. (/usr/share/texlive/texmf-dist/tex/latex/psnfss/ts1ptm.fd) [3] [4] [5] [6] Chapter 2. [7] [8] [9] [10] Chapter 3. [11] [12] [13] [14] Chapter 4. [15] [16] [17] [18] [19] (/usr/share/texlive/texmf-dist/tex/latex/psnfss/ts1pcr.fd) [20] [21] [22] Chapter 5. [23] [24] [25] [26] Underfull \hbox (badness 10000) in paragraph at lines 1293--1293 [][][]\T1/pcr/m/n/10 It[][]looks[][]like[][]the[][]environment[][]wasn\TS1/pcr/ m/n/10 '\T1/pcr/m/n/10 t[][]saved,[][][]and[][]we[][]can\TS1/pcr/m/n/10 '\T1/pc r/m/n/10 t[][]run[][]the[][]agent[][]in[][]it. Underfull \hbox (badness 10000) in paragraph at lines 1295--1295 [][][]\T1/pcr/m/n/10 Check[][]out[][]the[][]readthedocs[][]page[][]on[][]Experi ment[][]Outputs[][]for[][]how[][]to[][]handle[][]this[] [27] [28] Chapter 6. [29] [30] Chapter 7. [31] [32 <./rl_diagram_transparent_bg.png>] [33] [34] [35] [36] [37] [38] [39] [40] Chapter 8. ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.1860 ...ncludegraphics{{rl_algorithms_9_15}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.1860 ...ncludegraphics{{rl_algorithms_9_15}.svg} [41] [42] [43] [44] Chapter 9. [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] Chapter 10. [55] [56] [57] [58] [59] [60] Chapter 11. [61] [62] Overfull \vbox (103.35579pt too high) has occurred while \output is active [63] [64] Chapter 12. Underfull \hbox (badness 10000) in paragraph at lines 2740--2741 []\T1/ptm/m/n/10 PyTorch ver-sion: \T1/pcr/m/n/10 spinup/exercises/pytorch/prob lem_set_1_solutions/ Underfull \hbox (badness 10000) in paragraph at lines 2743--2744 []\T1/ptm/m/n/10 Tensorflow ver-sion: \T1/pcr/m/n/10 spinup/exercises/tf1/probl em_set_1_solutions/ [65] Underfull \hbox (badness 10000) in paragraph at lines 2771--2772 []\T1/ptm/m/n/10 PyTorch ver-sion: \T1/pcr/m/n/10 spinup/exercises/pytorch/prob lem_set_1_solutions/ Underfull \hbox (badness 10000) in paragraph at lines 2774--2775 []\T1/ptm/m/n/10 Tensorflow ver-sion: \T1/pcr/m/n/10 spinup/exercises/tf1/probl em_set_1_solutions/ [66] [67] [68] Chapter 13. ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2969 ...s{{pytorch_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2969 ...s{{pytorch_halfcheetah_performance}.svg} [69] ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2978 ...tensorflow_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2978 ...tensorflow_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2987 ...aphics{{pytorch_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2987 ...aphics{{pytorch_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2996 ...ics{{tensorflow_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2996 ...ics{{tensorflow_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3005 ...hics{{pytorch_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3005 ...hics{{pytorch_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3014 ...s{{tensorflow_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3014 ...s{{tensorflow_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3023 ...phics{{pytorch_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3023 ...phics{{pytorch_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3032 ...cs{{tensorflow_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3032 ...cs{{tensorflow_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3041 ...egraphics{{pytorch_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3041 ...egraphics{{pytorch_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3050 ...aphics{{tensorflow_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3050 ...aphics{{tensorflow_ant_performance}.svg} [70] [71] [72] Chapter 14. [73] [74] Underfull \hbox (badness 10000) in paragraph at lines 3243--3243 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 5862) in paragraph at lines 3243--3243 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , [75] [76] Underfull \hbox (badness 10000) in paragraph at lines 3436--3436 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3436--3436 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_l r=0.0003\T1/ptm/m/n/10 , \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , [77] [78] [79] [80] Chapter 15. [81] [82] [83] Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 delt a=0.01\T1/ptm/m/n/10 , \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 \T1/ptm/m/it/10 train_v_iters=80\T1/ptm/m/n/10 , \T1/ptm/m/it/10 damp-ing_coeff =0.1\T1/ptm/m/n/10 , \T1/ptm/m/it/10 cg_iters=10\T1/ptm/m/n/10 , \T1/ptm/m/it/1 0 back-track_iters=10\T1/ptm/m/n/10 , \T1/ptm/m/it/10 back- [84] [85] Overfull \vbox (100.02797pt too high) has occurred while \output is active [86] [87] [88] Chapter 16. [89] [90] [91] Underfull \hbox (badness 10000) in paragraph at lines 4289--4289 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 5862) in paragraph at lines 4289--4289 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , [92] [93] Underfull \hbox (badness 10000) in paragraph at lines 4500--4500 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 5274) in paragraph at lines 4500--4500 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 clip _ratio=0.2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_lr=0.0003\T1/ptm/m/n/10 , Underfull \hbox (badness 6461) in paragraph at lines 4500--4500 \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 train_pi_iters=80\T 1/ptm/m/n/10 , \T1/ptm/m/it/10 train_v_iters=80\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lam=0.97\T1/ptm/m/n/10 , \T1/ptm/m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , [94] [95] [96] [97] [98] Chapter 17. [99] [100] [101] [102] Underfull \hbox (badness 10000) in paragraph at lines 4948--4948 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 4948--4948 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- [103] [104] Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_lr=0.001\T1/ptm /m/n/10 , \T1/ptm/m/it/10 q_lr=0.001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 batch_size =100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=10000\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_ever y=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 act_noise=0.1\T1/ptm/m/n/10 , \T1/ptm/m/it /10 num_test_episodes=10\T1/ptm/m/n/10 , [105] [106] [107] [108] Chapter 18. [109] [110] [111] [112] Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 batch_size=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=1000 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/ it/10 up-date_every=50\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 act_noise=0.1\T1/ptm/m/n/10 , \T1/ptm/m/it/10 tar-get_noise=0.2 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 noise_clip=0.5\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pol-icy_delay=2\T1/ptm/m/n/10 , [113] [114] Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 \T1/ptm/m/it/10 pol-icy_delay=2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_test_episod es=10\T1/ptm/m/n/10 , \T1/ptm/m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 log-ger_kwargs={}\T1/ptm/m/n/10 , [115] [116] [117] [118] Chapter 19. [119] [120] [121] [122] [123] Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1 /ptm/m/n/10 , \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lr=0 .001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 al-pha=0.2\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 batch_size=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=1000 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/ it/10 up-date_every=50\T1/ptm/m/n/10 , [124] [125] [126] Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lr=0.001\T1/ptm/m/ n/10 , \T1/ptm/m/it/10 al-pha=0.2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 batch_size=10 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=10000\T1/ptm/m/n/10 , \T1/ptm/m/i t/10 up- Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_every=5 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_test_episodes=10\T1/ptm/m/n/10 , \T1/ptm/ m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , [127] [128] [129] [130] Chapter 20. [131] [132] [133] [134] [135] [136] [137] [138] Chapter 21. [139] [140] Chapter 22. [141] [142] [143] [144] Chapter 23. [145] Underfull \hbox (badness 10000) in paragraph at lines 7540--7540 []\T1/ptm/m/it/10 exp_name\T1/ptm/m/n/10 , \T1/ptm/m/it/10 thunk\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_cpu=1\T1/ptm/m/n/1 0 , [146] [147] [148] Chapter 24. [149] [150] Chapter 25. [151] [152] Chapter 26. [153] [154] [155] (./SpinningUp.ind [156] Underfull \hbox (badness 7522) in paragraph at lines 47--48 []\T1/ptm/m/n/10 add() (spinup.utils.run_utils.ExperimentGrid method), Overfull \hbox (5.61969pt too wide) in paragraph at lines 48--49 []\T1/ptm/m/n/10 apply_gradients() (spinup.utils.mpi_tf.MpiAdamOptimizer Overfull \hbox (17.83952pt too wide) in paragraph at lines 74--75 []\T1/ptm/m/n/10 compute_gradients() (spinup.utils.mpi_tf.MpiAdamOptimizer [157] Underfull \hbox (badness 10000) in paragraph at lines 105--106 []\T1/ptm/m/n/10 mpi_statistics_scalar() (in mod-ule Underfull \hbox (badness 10000) in paragraph at lines 122--123 []\T1/ptm/m/n/10 run() (spinup.utils.run_utils.ExperimentGrid method), Underfull \hbox (badness 10000) in paragraph at lines 131--132 []\T1/ptm/m/n/10 setup_pytorch_for_mpi() (in mod-ule Underfull \hbox (badness 10000) in paragraph at lines 132--133 []\T1/ptm/m/n/10 setup_pytorch_saver() (spinup.utils.logx.Logger Underfull \hbox (badness 10000) in paragraph at lines 149--150 []\T1/ptm/m/n/10 variant_name() (spinup.utils.run_utils.ExperimentGrid Underfull \hbox (badness 10000) in paragraph at lines 150--151 []\T1/ptm/m/n/10 variants() (spinup.utils.run_utils.ExperimentGrid [158]) (./SpinningUp.aux) Package rerunfilecheck Warning: File `SpinningUp.out' has changed. (rerunfilecheck) Rerun to get outlines right (rerunfilecheck) or use package `bookmark'. LaTeX Warning: There were multiply-defined labels. ) (see the transcript file for additional information){/usr/share/texlive/texmf-d ist/fonts/enc/dvips/base/8r.enc}< /usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr5.pfb> Output written on SpinningUp.pdf (164 pages, 1209705 bytes). Transcript written on SpinningUp.log. [rtd-command-info] start-time: 2020-01-30T16:42:05.052179Z, end-time: 2020-01-30T16:42:05.362588Z, duration: 0, exit-code: 0 mv -f /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/_build/latex/SpinningUp.pdf /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/artifacts/latest/sphinx_pdf/openai-education-spinningup.pdf [rtd-command-info] start-time: 2020-01-30T16:42:05.426342Z, end-time: 2020-01-30T16:43:32.219044Z, duration: 86, exit-code: 0 python sphinx-build -T -b epub -d _build/doctrees-epub -D language=en . _build/epub Running Sphinx v1.5.6 making output directory... loading translations [en]... done loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [epub]: targets for 37 source files that are out of date updating environment: 37 added, 0 changed, 0 removed reading sources... [ 2%] algorithms/ddpg reading sources... [ 5%] algorithms/ppo reading sources... [ 8%] algorithms/sac reading sources... [ 10%] algorithms/td3 reading sources... [ 13%] algorithms/trpo reading sources... [ 16%] algorithms/vpg reading sources... [ 18%] etc/acknowledgements reading sources... [ 21%] etc/author reading sources... [ 24%] index reading sources... [ 27%] spinningup/bench reading sources... [ 29%] spinningup/bench_ddpg reading sources... [ 32%] spinningup/bench_ppo reading sources... [ 35%] spinningup/bench_sac reading sources... [ 37%] spinningup/bench_td3 reading sources... [ 40%] spinningup/bench_vpg reading sources... [ 43%] spinningup/exercise2_1_soln reading sources... [ 45%] spinningup/exercise2_2_soln reading sources... [ 48%] spinningup/exercises reading sources... [ 51%] spinningup/extra_pg_proof1 reading sources... [ 54%] spinningup/extra_pg_proof2 reading sources... [ 56%] spinningup/extra_tf_pg_implementation reading sources... [ 59%] spinningup/keypapers reading sources... [ 62%] spinningup/rl_intro reading sources... [ 64%] spinningup/rl_intro2 reading sources... [ 67%] spinningup/rl_intro3 reading sources... [ 70%] spinningup/rl_intro4 reading sources... [ 72%] spinningup/spinningup reading sources... [ 75%] user/algorithms reading sources... [ 78%] user/installation reading sources... [ 81%] user/introduction reading sources... [ 83%] user/plotting reading sources... [ 86%] user/running reading sources... [ 89%] user/saving_and_loading reading sources... [ 91%] utils/logger reading sources... [ 94%] utils/mpi reading sources... [ 97%] utils/plotter reading sources... [100%] utils/run_utils /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercises.rst:3: WARNING: Duplicate explicit target name: "solution available here.". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:343: WARNING: Duplicate explicit target name: "here". looking for now-outdated files... none found pickling environment... done checking consistency... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:419: WARNING: Line block ends without a blank line. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:341: ERROR: Duplicate target name, cannot be used as a unique reference: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:55: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:63: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:163: WARNING: Duplicate explicit target name: "cmdoption--ac_kwargs". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:306: WARNING: Inline strong start-string without end-string. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:148: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:173: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ddpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ppo.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_sac.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_td3.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_vpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_1_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_2_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof1.rst:: WARNING: document isn't included in any toctree done preparing documents... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof2.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro4.rst:: WARNING: document isn't included in any toctree done writing output... [ 2%] algorithms/ddpg writing output... [ 5%] algorithms/ppo writing output... [ 8%] algorithms/sac writing output... [ 10%] algorithms/td3 writing output... [ 13%] algorithms/trpo writing output... [ 16%] algorithms/vpg writing output... [ 18%] etc/acknowledgements writing output... [ 21%] etc/author writing output... [ 24%] index writing output... [ 27%] spinningup/bench writing output... [ 29%] spinningup/bench_ddpg writing output... [ 32%] spinningup/bench_ppo writing output... [ 35%] spinningup/bench_sac writing output... [ 37%] spinningup/bench_td3 writing output... [ 40%] spinningup/bench_vpg writing output... [ 43%] spinningup/exercise2_1_soln writing output... [ 45%] spinningup/exercise2_2_soln writing output... [ 48%] spinningup/exercises writing output... [ 51%] spinningup/extra_pg_proof1 writing output... [ 54%] spinningup/extra_pg_proof2 writing output... [ 56%] spinningup/extra_tf_pg_implementation writing output... [ 59%] spinningup/keypapers writing output... [ 62%] spinningup/rl_intro writing output... [ 64%] spinningup/rl_intro2 writing output... [ 67%] spinningup/rl_intro3 writing output... [ 70%] spinningup/rl_intro4 writing output... [ 72%] spinningup/spinningup writing output... [ 75%] user/algorithms writing output... [ 78%] user/installation writing output... [ 81%] user/introduction writing output... [ 83%] user/plotting writing output... [ 86%] user/running writing output... [ 89%] user/saving_and_loading writing output... [ 91%] utils/logger writing output... [ 94%] utils/mpi writing output... [ 97%] utils/plotter writing output... [100%] utils/run_utils generating indices... genindex py-modindex writing additional pages... copying images... [ 2%] images/spinning-up-in-rl.png copying images... [ 4%] spinningup/../images/plots/pyt/pytorch_halfcheetah_performance.svg copying images... [ 7%] spinningup/../images/plots/tf1/tensorflow_halfcheetah_performance.svg copying images... [ 9%] spinningup/../images/plots/pyt/pytorch_hopper_performance.svg copying images... [ 12%] spinningup/../images/plots/tf1/tensorflow_hopper_performance.svg copying images... [ 14%] spinningup/../images/plots/pyt/pytorch_walker2d_performance.svg copying images... [ 17%] spinningup/../images/plots/tf1/tensorflow_walker2d_performance.svg copying images... [ 19%] spinningup/../images/plots/pyt/pytorch_swimmer_performance.svg copying images... [ 21%] spinningup/../images/plots/tf1/tensorflow_swimmer_performance.svg copying images... [ 24%] spinningup/../images/plots/pyt/pytorch_ant_performance.svg copying images... [ 26%] spinningup/../images/plots/tf1/tensorflow_ant_performance.svg copying images... [ 29%] spinningup/../images/plots/ddpg/ddpg_halfcheetah_performance.svg copying images... [ 31%] spinningup/../images/plots/ddpg/ddpg_hopper_performance.svg copying images... [ 34%] spinningup/../images/plots/ddpg/ddpg_walker2d_performance.svg copying images... [ 36%] spinningup/../images/plots/ddpg/ddpg_swimmer_performance.svg copying images... [ 39%] spinningup/../images/plots/ddpg/ddpg_ant_performance.svg copying images... [ 41%] spinningup/../images/plots/ppo/ppo_halfcheetah_performance.svg copying images... [ 43%] spinningup/../images/plots/ppo/ppo_hopper_performance.svg copying images... [ 46%] spinningup/../images/plots/ppo/ppo_walker2d_performance.svg copying images... [ 48%] spinningup/../images/plots/ppo/ppo_swimmer_performance.svg copying images... [ 51%] spinningup/../images/plots/ppo/ppo_ant_performance.svg copying images... [ 53%] spinningup/../images/plots/sac/sac_halfcheetah_performance.svg copying images... [ 56%] spinningup/../images/plots/sac/sac_hopper_performance.svg copying images... [ 58%] spinningup/../images/plots/sac/sac_walker2d_performance.svg copying images... [ 60%] spinningup/../images/plots/sac/sac_swimmer_performance.svg copying images... [ 63%] spinningup/../images/plots/sac/sac_ant_performance.svg copying images... [ 65%] spinningup/../images/plots/td3/td3_halfcheetah_performance.svg copying images... [ 68%] spinningup/../images/plots/td3/td3_hopper_performance.svg copying images... [ 70%] spinningup/../images/plots/td3/td3_walker2d_performance.svg copying images... [ 73%] spinningup/../images/plots/td3/td3_swimmer_performance.svg copying images... [ 75%] spinningup/../images/plots/td3/td3_ant_performance.svg copying images... [ 78%] spinningup/../images/plots/vpg/vpg_halfcheetah_performance.svg copying images... [ 80%] spinningup/../images/plots/vpg/vpg_hopper_performance.svg copying images... [ 82%] spinningup/../images/plots/vpg/vpg_walker2d_performance.svg copying images... [ 85%] spinningup/../images/plots/vpg/vpg_swimmer_performance.svg copying images... [ 87%] spinningup/../images/plots/vpg/vpg_ant_performance.svg copying images... [ 90%] spinningup/../images/ex2-1_trpo_hopper.png copying images... [ 92%] spinningup/../images/ex2-2_ddpg_bug.svg copying images... [ 95%] spinningup/../images/ex2-2_ddpg_bug_pytorch.png copying images... [ 97%] spinningup/../images/rl_diagram_transparent_bg.png copying images... [100%] spinningup/../images/rl_algorithms_9_15.svg copying static files... WARNING: favicon file 'openai_icon.ico' does not exist done copying extra files... done writing mimetype file... writing META-INF/container.xml file... writing content.opf file... WARNING: unknown mimetype for _static/openai-favicon2_32x32.ico, ignoring WARNING: unknown mimetype for _static/openai_icon.ico, ignoring writing nav.xhtml file... writing toc.ncx file... writing SpinningUp.epub file... build succeeded, 27 warnings. [rtd-command-info] start-time: 2020-01-30T16:43:32.352646Z, end-time: 2020-01-30T16:43:32.660787Z, duration: 0, exit-code: 0 mv -f /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/_build/epub/SpinningUp.epub /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/artifacts/latest/sphinx_epub/openai-education-spinningup.epub