Read the Docs build information Build id: 283464 Project: openai-education-spinningup Version: latest Commit: ed725b36023e37d9bfd5e3d70f381750ee305784 Date: 2020-02-02T18:09:46.532903Z State: finished Success: True [rtd-command-info] start-time: 2020-02-02T18:09:48.458657Z, end-time: 2020-02-02T18:09:50.520395Z, duration: 2, exit-code: 0 git clone --no-single-branch --depth 50 git@github.com:openai/spinningup.git . Cloning into '.'... Warning: Permanently added 'github.com,192.30.255.112' (RSA) to the list of known hosts. [rtd-command-info] start-time: 2020-02-02T18:09:50.776918Z, end-time: 2020-02-02T18:09:51.237857Z, duration: 0, exit-code: 0 git checkout --force origin/master Note: checking out 'origin/master'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b HEAD is now at ed725b3 Merge pull request #207 from sagnik-chatterjee/dev [rtd-command-info] start-time: 2020-02-02T18:09:51.298276Z, end-time: 2020-02-02T18:09:51.513491Z, duration: 0, exit-code: 0 git clean -d -f -f [rtd-command-info] start-time: 2020-02-02T18:09:52.652345Z, end-time: 2020-02-02T18:09:58.135717Z, duration: 5, exit-code: 0 python3.6 -mvirtualenv --no-site-packages Using base prefix '/home/docs/.pyenv/versions/3.6.8' New python executable in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/bin/python3.6 Also creating executable in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/bin/python Installing setuptools, pip, wheel... done. [rtd-command-info] start-time: 2020-02-02T18:09:58.202679Z, end-time: 2020-02-02T18:09:59.228287Z, duration: 1, exit-code: 0 python -m pip install --upgrade --cache-dir /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip pip Requirement already up-to-date: pip in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (20.0.2) [rtd-command-info] start-time: 2020-02-02T18:09:59.288155Z, end-time: 2020-02-02T18:10:12.836482Z, duration: 13, exit-code: 0 python -m pip install --upgrade --cache-dir /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip Pygments==2.3.1 setuptools==41.0.1 docutils==0.14 mock==1.0.1 pillow==5.4.1 alabaster>=0.7,<0.8,!=0.7.5 commonmark==0.8.1 recommonmark==0.5.0 sphinx<2 sphinx-rtd-theme<0.5 readthedocs-sphinx-ext<1.1 Collecting Pygments==2.3.1 Downloading Pygments-2.3.1-py2.py3-none-any.whl (849 kB) Collecting setuptools==41.0.1 Downloading setuptools-41.0.1-py2.py3-none-any.whl (575 kB) Collecting docutils==0.14 Downloading docutils-0.14-py3-none-any.whl (543 kB) Collecting mock==1.0.1 Downloading mock-1.0.1.tar.gz (818 kB) Collecting pillow==5.4.1 Downloading Pillow-5.4.1-cp36-cp36m-manylinux1_x86_64.whl (2.0 MB) Collecting alabaster!=0.7.5,<0.8,>=0.7 Downloading alabaster-0.7.12-py2.py3-none-any.whl (14 kB) Collecting commonmark==0.8.1 Downloading commonmark-0.8.1-py2.py3-none-any.whl (47 kB) Collecting recommonmark==0.5.0 Downloading recommonmark-0.5.0-py2.py3-none-any.whl (9.8 kB) Collecting sphinx<2 Downloading Sphinx-1.8.5-py2.py3-none-any.whl (3.1 MB) Collecting sphinx-rtd-theme<0.5 Downloading sphinx_rtd_theme-0.4.3-py2.py3-none-any.whl (6.4 MB) Collecting readthedocs-sphinx-ext<1.1 Downloading readthedocs_sphinx_ext-1.0.1-py2.py3-none-any.whl (14 kB) Collecting future Downloading future-0.18.2.tar.gz (829 kB) Collecting six>=1.5 Downloading six-1.14.0-py2.py3-none-any.whl (10 kB) Collecting requests>=2.0.0 Downloading requests-2.22.0-py2.py3-none-any.whl (57 kB) Collecting snowballstemmer>=1.1 Downloading snowballstemmer-2.0.0-py2.py3-none-any.whl (97 kB) Collecting imagesize Downloading imagesize-1.2.0-py2.py3-none-any.whl (4.8 kB) Collecting packaging Downloading packaging-20.1-py2.py3-none-any.whl (36 kB) Collecting babel!=2.0,>=1.3 Downloading Babel-2.8.0-py2.py3-none-any.whl (8.6 MB) Collecting Jinja2>=2.3 Downloading Jinja2-2.11.1-py2.py3-none-any.whl (126 kB) Collecting sphinxcontrib-websupport Downloading sphinxcontrib_websupport-1.1.2-py2.py3-none-any.whl (39 kB) Collecting certifi>=2017.4.17 Downloading certifi-2019.11.28-py2.py3-none-any.whl (156 kB) Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 Downloading urllib3-1.25.8-py2.py3-none-any.whl (125 kB) Collecting idna<2.9,>=2.5 Downloading idna-2.8-py2.py3-none-any.whl (58 kB) Collecting chardet<3.1.0,>=3.0.2 Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB) Collecting pyparsing>=2.0.2 Downloading pyparsing-2.4.6-py2.py3-none-any.whl (67 kB) Collecting pytz>=2015.7 Downloading pytz-2019.3-py2.py3-none-any.whl (509 kB) Collecting MarkupSafe>=0.23 Downloading MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl (27 kB) Building wheels for collected packages: mock, future Building wheel for mock (setup.py): started Building wheel for mock (setup.py): finished with status 'done' Created wheel for mock: filename=mock-1.0.1-py3-none-any.whl size=23754 sha256=971f4c4ddc8c7d6882f4582a9306b7b8b95b003f2a4f935aefbacee3540b0094 Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/15/fd/39/f238f155700d6d5e6231e82ae3ccf3aa7490a1b17d2073251b Building wheel for future (setup.py): started Building wheel for future (setup.py): finished with status 'done' Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491058 sha256=34667e17f89d2b739173ca29b950992a0c0ae51e06b4c15e942fa916a5e5b5ea Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/6e/9c/ed/4499c9865ac1002697793e0ae05ba6be33553d098f3347fb94 Successfully built mock future Installing collected packages: Pygments, setuptools, docutils, mock, pillow, alabaster, future, commonmark, six, certifi, urllib3, idna, chardet, requests, snowballstemmer, imagesize, pyparsing, packaging, pytz, babel, MarkupSafe, Jinja2, sphinxcontrib-websupport, sphinx, recommonmark, sphinx-rtd-theme, readthedocs-sphinx-ext Attempting uninstall: setuptools Found existing installation: setuptools 45.1.0 Uninstalling setuptools-45.1.0: Successfully uninstalled setuptools-45.1.0 Successfully installed Jinja2-2.11.1 MarkupSafe-1.1.1 Pygments-2.3.1 alabaster-0.7.12 babel-2.8.0 certifi-2019.11.28 chardet-3.0.4 commonmark-0.8.1 docutils-0.14 future-0.18.2 idna-2.8 imagesize-1.2.0 mock-1.0.1 packaging-20.1 pillow-5.4.1 pyparsing-2.4.6 pytz-2019.3 readthedocs-sphinx-ext-1.0.1 recommonmark-0.5.0 requests-2.22.0 setuptools-41.0.1 six-1.14.0 snowballstemmer-2.0.0 sphinx-1.8.5 sphinx-rtd-theme-0.4.3 sphinxcontrib-websupport-1.1.2 urllib3-1.25.8 [rtd-command-info] start-time: 2020-02-02T18:10:12.900173Z, end-time: 2020-02-02T18:11:27.565613Z, duration: 74, exit-code: 0 python -m pip install --exists-action=w --cache-dir /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip -r docs/docs_requirements.txt Collecting cloudpickle~=1.2.1 Downloading cloudpickle-1.2.2-py2.py3-none-any.whl (25 kB) Collecting gym~=0.15.3 Downloading gym-0.15.4.tar.gz (1.6 MB) Collecting ipython Downloading ipython-7.12.0-py3-none-any.whl (777 kB) Collecting joblib Downloading joblib-0.14.1-py2.py3-none-any.whl (294 kB) Collecting matplotlib Downloading matplotlib-3.1.2-cp36-cp36m-manylinux1_x86_64.whl (13.1 MB) Collecting numpy Downloading numpy-1.18.1-cp36-cp36m-manylinux1_x86_64.whl (20.1 MB) Collecting pandas Downloading pandas-1.0.0-cp36-cp36m-manylinux1_x86_64.whl (10.1 MB) Collecting pytest Downloading pytest-5.3.5-py3-none-any.whl (235 kB) Collecting psutil Downloading psutil-5.6.7.tar.gz (448 kB) Collecting scipy Downloading scipy-1.4.1-cp36-cp36m-manylinux1_x86_64.whl (26.1 MB) Collecting seaborn==0.8.1 Downloading seaborn-0.8.1.tar.gz (178 kB) Collecting sphinx==1.5.6 Downloading Sphinx-1.5.6-py2.py3-none-any.whl (1.6 MB) Collecting sphinx-autobuild==0.7.1 Downloading sphinx-autobuild-0.7.1.tar.gz (14 kB) Collecting sphinx-rtd-theme==0.4.1 Downloading sphinx_rtd_theme-0.4.1-py2.py3-none-any.whl (5.4 MB) Collecting tensorflow<2.0,>=1.8.0 Downloading tensorflow-1.15.2-cp36-cp36m-manylinux2010_x86_64.whl (110.5 MB) Collecting tqdm Downloading tqdm-4.42.0-py2.py3-none-any.whl (59 kB) Requirement already satisfied: six in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from gym~=0.15.3->-r docs/docs_requirements.txt (line 2)) (1.14.0) Collecting pyglet<=1.3.2,>=1.2.0 Downloading pyglet-1.3.2-py2.py3-none-any.whl (1.0 MB) Collecting opencv-python Downloading opencv_python-4.1.2.30-cp36-cp36m-manylinux1_x86_64.whl (28.3 MB) Requirement already satisfied: pygments in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from ipython->-r docs/docs_requirements.txt (line 3)) (2.3.1) Collecting traitlets>=4.2 Downloading traitlets-4.3.3-py2.py3-none-any.whl (75 kB) Collecting jedi>=0.10 Downloading jedi-0.16.0-py2.py3-none-any.whl (1.1 MB) Collecting pexpect; sys_platform != "win32" Downloading pexpect-4.8.0-py2.py3-none-any.whl (59 kB) Requirement already satisfied: setuptools>=18.5 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from ipython->-r docs/docs_requirements.txt (line 3)) (41.0.1) Collecting decorator Downloading decorator-4.4.1-py2.py3-none-any.whl (9.2 kB) Collecting backcall Downloading backcall-0.1.0.tar.gz (9.7 kB) Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 Downloading prompt_toolkit-3.0.3-py3-none-any.whl (348 kB) Collecting pickleshare Downloading pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB) Collecting kiwisolver>=1.0.1 Downloading kiwisolver-1.1.0-cp36-cp36m-manylinux1_x86_64.whl (90 kB) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from matplotlib->-r docs/docs_requirements.txt (line 5)) (2.4.6) Collecting python-dateutil>=2.1 Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB) Collecting cycler>=0.10 Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB) Requirement already satisfied: pytz>=2017.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pandas->-r docs/docs_requirements.txt (line 7)) (2019.3) Collecting py>=1.5.0 Downloading py-1.8.1-py2.py3-none-any.whl (83 kB) Collecting wcwidth Downloading wcwidth-0.1.8-py2.py3-none-any.whl (17 kB) Collecting importlib-metadata>=0.12; python_version < "3.8" Downloading importlib_metadata-1.5.0-py2.py3-none-any.whl (30 kB) Collecting pluggy<1.0,>=0.12 Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB) Requirement already satisfied: packaging in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pytest->-r docs/docs_requirements.txt (line 8)) (20.1) Collecting attrs>=17.4.0 Downloading attrs-19.3.0-py2.py3-none-any.whl (39 kB) Collecting more-itertools>=4.0.0 Downloading more_itertools-8.2.0-py3-none-any.whl (43 kB) Requirement already satisfied: Jinja2>=2.3 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2.11.1) Requirement already satisfied: alabaster<0.8,>=0.7 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (0.7.12) Requirement already satisfied: imagesize in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (1.2.0) Requirement already satisfied: snowballstemmer>=1.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2.0.0) Requirement already satisfied: docutils>=0.11 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (0.14) Requirement already satisfied: requests>=2.0.0 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2.22.0) Requirement already satisfied: babel!=2.0,>=1.3 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2.8.0) Collecting watchdog>=0.7.1 Downloading watchdog-0.10.1.tar.gz (94 kB) Collecting argh>=0.24.1 Downloading argh-0.26.2-py2.py3-none-any.whl (30 kB) Collecting pathtools>=0.1.2 Downloading pathtools-0.1.2.tar.gz (11 kB) Collecting PyYAML>=3.10 Downloading PyYAML-5.3.tar.gz (268 kB) Collecting tornado>=3.2 Downloading tornado-6.0.3.tar.gz (482 kB) Collecting port_for==0.3.1 Downloading port-for-0.3.1.tar.gz (18 kB) Collecting livereload>=2.3.0 Downloading livereload-2.6.1-py2.py3-none-any.whl (23 kB) Collecting tensorboard<1.16.0,>=1.15.0 Downloading tensorboard-1.15.0-py3-none-any.whl (3.8 MB) Collecting astor>=0.6.0 Downloading astor-0.8.1-py2.py3-none-any.whl (27 kB) Collecting termcolor>=1.1.0 Downloading termcolor-1.1.0.tar.gz (3.9 kB) Requirement already satisfied: wheel>=0.26; python_version >= "3" in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from tensorflow<2.0,>=1.8.0->-r docs/docs_requirements.txt (line 15)) (0.34.2) Collecting gast==0.2.2 Downloading gast-0.2.2.tar.gz (10 kB) Collecting wrapt>=1.11.1 Downloading wrapt-1.11.2.tar.gz (27 kB) Collecting keras-applications>=1.0.8 Downloading Keras_Applications-1.0.8-py3-none-any.whl (50 kB) Collecting keras-preprocessing>=1.0.5 Downloading Keras_Preprocessing-1.1.0-py2.py3-none-any.whl (41 kB) Collecting protobuf>=3.6.1 Downloading protobuf-3.11.2-cp36-cp36m-manylinux1_x86_64.whl (1.3 MB) Collecting google-pasta>=0.1.6 Downloading google_pasta-0.1.8-py3-none-any.whl (57 kB) Collecting grpcio>=1.8.6 Downloading grpcio-1.26.0-cp36-cp36m-manylinux2010_x86_64.whl (2.4 MB) Collecting tensorflow-estimator==1.15.1 Downloading tensorflow_estimator-1.15.1-py2.py3-none-any.whl (503 kB) Collecting absl-py>=0.7.0 Downloading absl-py-0.9.0.tar.gz (104 kB) Collecting opt-einsum>=2.3.2 Downloading opt_einsum-3.1.0.tar.gz (69 kB) Requirement already satisfied: future in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from pyglet<=1.3.2,>=1.2.0->gym~=0.15.3->-r docs/docs_requirements.txt (line 2)) (0.18.2) Collecting ipython-genutils Downloading ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB) Collecting parso>=0.5.2 Downloading parso-0.6.0-py2.py3-none-any.whl (97 kB) Collecting ptyprocess>=0.5 Downloading ptyprocess-0.6.0-py2.py3-none-any.whl (39 kB) Collecting zipp>=0.5 Downloading zipp-2.1.0-py3-none-any.whl (4.6 kB) Requirement already satisfied: MarkupSafe>=0.23 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from Jinja2>=2.3->sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (1.1.1) Requirement already satisfied: idna<2.9,>=2.5 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2.8) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (3.0.4) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (1.25.8) Requirement already satisfied: certifi>=2017.4.17 in /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/envs/latest/lib/python3.6/site-packages (from requests>=2.0.0->sphinx==1.5.6->-r docs/docs_requirements.txt (line 12)) (2019.11.28) Collecting werkzeug>=0.11.15 Downloading Werkzeug-0.16.1-py2.py3-none-any.whl (327 kB) Collecting markdown>=2.6.8 Downloading Markdown-3.1.1-py2.py3-none-any.whl (87 kB) Collecting h5py Downloading h5py-2.10.0-cp36-cp36m-manylinux1_x86_64.whl (2.9 MB) Building wheels for collected packages: gym, psutil, seaborn, sphinx-autobuild, backcall, watchdog, pathtools, PyYAML, tornado, port-for, termcolor, gast, wrapt, absl-py, opt-einsum Building wheel for gym (setup.py): started Building wheel for gym (setup.py): finished with status 'done' Created wheel for gym: filename=gym-0.15.4-py3-none-any.whl size=1648482 sha256=cdd4099b97867d3961a440399cd6d0fcbcde9212270939345b935035757223ce Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/c6/c8/c5/470ac235d9c7808c13aa840227111d6367d0dd139570cc6f63 Building wheel for psutil (setup.py): started Building wheel for psutil (setup.py): finished with status 'done' Created wheel for psutil: filename=psutil-5.6.7-cp36-cp36m-linux_x86_64.whl size=274516 sha256=f3b32099d380c08125130df3ca2ebd4a933a45ef6cb221158861a738e327f59d Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/b2/b3/a3/551db06cebc9141c437c6b58aa751d3670471494e9f6d1428d Building wheel for seaborn (setup.py): started Building wheel for seaborn (setup.py): finished with status 'done' Created wheel for seaborn: filename=seaborn-0.8.1-py3-none-any.whl size=184860 sha256=b3bb70aa3fe396145c4e00c5dfc5dd554d46af8588ae94419aa60409609e3eb4 Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/b4/8a/f8/954cba570daa622117cc602e6e2cd97b15233d213546348f6e Building wheel for sphinx-autobuild (setup.py): started Building wheel for sphinx-autobuild (setup.py): finished with status 'done' Created wheel for sphinx-autobuild: filename=sphinx_autobuild-0.7.1-py3-none-any.whl size=10075 sha256=08159cd1643b11312dc90751f6ab58c68fe6da78faea98370d444a7fcf0eae14 Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/5a/0c/a2/2cdb7792799e01fcd7c7adbe3e6efd5637d8eb9f5ccd4be712 Building wheel for backcall (setup.py): started Building wheel for backcall (setup.py): finished with status 'done' Created wheel for backcall: filename=backcall-0.1.0-py3-none-any.whl size=10413 sha256=6de244b48ab2d49c924e87272a6de262406eb06672f9f50b0e4098a64b259538 Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/b4/cb/f1/d142b3bb45d488612cf3943d8a1db090eb95e6687045ba61d1 Building wheel for watchdog (setup.py): started Building wheel for watchdog (setup.py): finished with status 'done' Created wheel for watchdog: filename=watchdog-0.10.1-py3-none-any.whl size=73502 sha256=7b57c3beef30a0965492d8a260ded2895bce34b888a5c65b379b97e01dac1524 Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/d1/e2/03/88fe4ca76024cfd5b4fc04a07c1fd72cdc9d18a9217fd74dcd Building wheel for pathtools (setup.py): started Building wheel for pathtools (setup.py): finished with status 'done' Created wheel for pathtools: filename=pathtools-0.1.2-py3-none-any.whl size=8784 sha256=4196f4d5ddcd555fc8f68cd119c32d51dfa5f5fb593b8108cdde98cee9690e72 Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/42/ea/90/e37d463fb3b03848bf715080595de62545266f53dd546b2497 Building wheel for PyYAML (setup.py): started Building wheel for PyYAML (setup.py): finished with status 'done' Created wheel for PyYAML: filename=PyYAML-5.3-cp36-cp36m-linux_x86_64.whl size=44229 sha256=3d5a96a88994d0a5ad044dac64f8dd47be490445780cd2cba495c37d22e6cdf2 Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/b1/86/0d/10e6c39d3a2b85ba807d7657ee80f08cc16c03f2aa2adf8e46 Building wheel for tornado (setup.py): started Building wheel for tornado (setup.py): finished with status 'done' Created wheel for tornado: filename=tornado-6.0.3-cp36-cp36m-linux_x86_64.whl size=423198 sha256=6aed2f2271cf20cab49080582fac932b755ba388ca4b1e0ac9bbaf26a9d20120 Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/b2/92/5c/3bfc125cdf46ab0487727e574f513f9568e9b7974b15237abf Building wheel for port-for (setup.py): started Building wheel for port-for (setup.py): finished with status 'done' Created wheel for port-for: filename=port_for-0.3.1-py3-none-any.whl size=19605 sha256=4f447756ad9e35ddb53d2a51651d61c970fe78e6f5735b27dd066d48ef80a057 Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/fb/03/4c/3eecaa0ca324237111374932c36eed973bf19662e65f4490c9 Building wheel for termcolor (setup.py): started Building wheel for termcolor (setup.py): finished with status 'done' Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4830 sha256=e114239751e93736826eb1c6a504c081aa354ace5dd3711556411e16ddefd57b Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/93/2a/eb/e58dbcbc963549ee4f065ff80a59f274cc7210b6eab962acdc Building wheel for gast (setup.py): started Building wheel for gast (setup.py): finished with status 'done' Created wheel for gast: filename=gast-0.2.2-py3-none-any.whl size=7539 sha256=c6fff5969dc83e89d2bf6b8e8e98e7cf638521ae03f56d97d9b13b5d587da50e Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/19/a7/b9/0740c7a3a7d1d348f04823339274b90de25fbcd217b2ee1fbe Building wheel for wrapt (setup.py): started Building wheel for wrapt (setup.py): finished with status 'done' Created wheel for wrapt: filename=wrapt-1.11.2-cp36-cp36m-linux_x86_64.whl size=69780 sha256=a723fb8f979cc072949d15ffaa4728ea7f24c2164c4f0ec80698a583854745c0 Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/0d/85/48/15d7bfab92a2d0e87372224c1f628fc57db7447a663a58e86c Building wheel for absl-py (setup.py): started Building wheel for absl-py (setup.py): finished with status 'done' Created wheel for absl-py: filename=absl_py-0.9.0-py3-none-any.whl size=121931 sha256=d1fc2c6d3359e036edd4a0a98be4ae67d7f4d8220d446f54d53dc5db417b30bf Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/c3/af/84/3962a6af7b4ab336e951b7877dcfb758cf94548bb1771e0679 Building wheel for opt-einsum (setup.py): started Building wheel for opt-einsum (setup.py): finished with status 'done' Created wheel for opt-einsum: filename=opt_einsum-3.1.0-py3-none-any.whl size=61681 sha256=3df6438feb01d56bf589987634594ca2fd9fb76dd0f3440965a604e78cd1fe0d Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/.cache/pip/wheels/1f/6f/14/e0f4f6bceec756bbab45c392c4d5429a684b14d849464ebfbf Successfully built gym psutil seaborn sphinx-autobuild backcall watchdog pathtools PyYAML tornado port-for termcolor gast wrapt absl-py opt-einsum Installing collected packages: cloudpickle, numpy, scipy, pyglet, opencv-python, gym, ipython-genutils, decorator, traitlets, parso, jedi, ptyprocess, pexpect, backcall, wcwidth, prompt-toolkit, pickleshare, ipython, joblib, kiwisolver, python-dateutil, cycler, matplotlib, pandas, py, zipp, importlib-metadata, pluggy, attrs, more-itertools, pytest, psutil, seaborn, sphinx, pathtools, watchdog, argh, PyYAML, tornado, port-for, livereload, sphinx-autobuild, sphinx-rtd-theme, protobuf, absl-py, werkzeug, markdown, grpcio, tensorboard, astor, termcolor, gast, wrapt, h5py, keras-applications, keras-preprocessing, google-pasta, tensorflow-estimator, opt-einsum, tensorflow, tqdm Attempting uninstall: sphinx Found existing installation: Sphinx 1.8.5 Uninstalling Sphinx-1.8.5: Successfully uninstalled Sphinx-1.8.5 Attempting uninstall: sphinx-rtd-theme Found existing installation: sphinx-rtd-theme 0.4.3 Uninstalling sphinx-rtd-theme-0.4.3: Successfully uninstalled sphinx-rtd-theme-0.4.3 Successfully installed PyYAML-5.3 absl-py-0.9.0 argh-0.26.2 astor-0.8.1 attrs-19.3.0 backcall-0.1.0 cloudpickle-1.2.2 cycler-0.10.0 decorator-4.4.1 gast-0.2.2 google-pasta-0.1.8 grpcio-1.26.0 gym-0.15.4 h5py-2.10.0 importlib-metadata-1.5.0 ipython-7.12.0 ipython-genutils-0.2.0 jedi-0.16.0 joblib-0.14.1 keras-applications-1.0.8 keras-preprocessing-1.1.0 kiwisolver-1.1.0 livereload-2.6.1 markdown-3.1.1 matplotlib-3.1.2 more-itertools-8.2.0 numpy-1.18.1 opencv-python-4.1.2.30 opt-einsum-3.1.0 pandas-1.0.0 parso-0.6.0 pathtools-0.1.2 pexpect-4.8.0 pickleshare-0.7.5 pluggy-0.13.1 port-for-0.3.1 prompt-toolkit-3.0.3 protobuf-3.11.2 psutil-5.6.7 ptyprocess-0.6.0 py-1.8.1 pyglet-1.3.2 pytest-5.3.5 python-dateutil-2.8.1 scipy-1.4.1 seaborn-0.8.1 sphinx-1.5.6 sphinx-autobuild-0.7.1 sphinx-rtd-theme-0.4.1 tensorboard-1.15.0 tensorflow-1.15.2 tensorflow-estimator-1.15.1 termcolor-1.1.0 tornado-6.0.3 tqdm-4.42.0 traitlets-4.3.3 watchdog-0.10.1 wcwidth-0.1.8 werkzeug-0.16.1 wrapt-1.11.2 zipp-2.1.0 [rtd-command-info] start-time: 2020-02-02T18:11:28.445350Z, end-time: 2020-02-02T18:11:28.667700Z, duration: 0, exit-code: 0 cat docs/conf.py #!/usr/bin/env python3 # -*- coding: utf-8 -*- # # Spinning Up documentation build configuration file, created by # sphinx-quickstart on Wed Aug 15 04:21:07 2018. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # import os import sys # Make sure spinup is accessible without going through setup.py dirname = os.path.dirname sys.path.insert(0, dirname(dirname(__file__))) # Mock mpi4py to get around having to install it on RTD server (which fails) # Also to mock PyTorch, because it is too large for the RTD server to download from unittest.mock import MagicMock class Mock(MagicMock): @classmethod def __getattr__(cls, name): return MagicMock() MOCK_MODULES = ['mpi4py', 'torch', 'torch.optim', 'torch.nn', 'torch.distributions', 'torch.distributions.normal', 'torch.distributions.categorical', 'torch.nn.functional', ] sys.modules.update((mod_name, Mock()) for mod_name in MOCK_MODULES) # Finish imports import spinup from recommonmark.parser import CommonMarkParser source_parsers = { '.md': CommonMarkParser, } # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. # # needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = ['sphinx.ext.imgmath', 'sphinx.ext.viewcode', 'sphinx.ext.autodoc', 'sphinx.ext.napoleon'] #'sphinx.ext.mathjax', ?? # imgmath settings imgmath_image_format = 'svg' imgmath_font_size = 14 # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix(es) of source filenames. # You can specify multiple suffix as a list of string: # source_suffix = ['.rst', '.md'] # source_suffix = '.rst' # The master toctree document. master_doc = 'index' # General information about the project. project = 'Spinning Up' copyright = '2018, OpenAI' author = 'Joshua Achiam' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = '' # The full version, including alpha/beta/rc tags. release = '' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. language = None # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This patterns also effect to html_static_path and html_extra_path exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'default' #'sphinx' # If true, `todo` and `todoList` produce output, else they produce nothing. todo_include_todos = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # # html_theme = 'alabaster' html_theme = "sphinx_rtd_theme" # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. # # html_theme_options = {} # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] html_logo = 'images/spinning-up-logo2.png' html_theme_options = { 'logo_only': True } #html_favicon = 'openai-favicon2_32x32.ico' html_favicon = 'openai_icon.ico' # -- Options for HTMLHelp output ------------------------------------------ # Output file base name for HTML help builder. htmlhelp_basename = 'SpinningUpdoc' # -- Options for LaTeX output --------------------------------------------- imgmath_latex_preamble = r''' \usepackage{algorithm} \usepackage{algorithmic} \usepackage{amsmath} \usepackage{cancel} \usepackage[verbose=true,letterpaper]{geometry} \geometry{ textheight=12in, textwidth=6.5in, top=1in, headheight=12pt, headsep=25pt, footskip=30pt } \newcommand{\E}{{\mathrm E}} \newcommand{\underE}[2]{\underset{\begin{subarray}{c}#1 \end{subarray}}{\E}\left[ #2 \right]} \newcommand{\Epi}[1]{\underset{\begin{subarray}{c}\tau \sim \pi \end{subarray}}{\E}\left[ #1 \right]} ''' latex_elements = { # The paper size ('letterpaper' or 'a4paper'). # # 'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). # # 'pointsize': '10pt', # Additional stuff for the LaTeX preamble. # 'preamble': r''' \usepackage{algorithm} \usepackage{algorithmic} \usepackage{amsmath} \usepackage{cancel} \newcommand{\E}{{\mathrm E}} \newcommand{\underE}[2]{\underset{\begin{subarray}{c}#1 \end{subarray}}{\E}\left[ #2 \right]} \newcommand{\Epi}[1]{\underset{\begin{subarray}{c}\tau \sim \pi \end{subarray}}{\E}\left[ #1 \right]} ''', # Latex figure (float) alignment # # 'figure_align': 'htbp', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'SpinningUp.tex', 'Spinning Up Documentation', 'Joshua Achiam', 'manual'), ] # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ (master_doc, 'spinningup', 'Spinning Up Documentation', [author], 1) ] # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ (master_doc, 'SpinningUp', 'Spinning Up Documentation', author, 'SpinningUp', 'One line description of project.', 'Miscellaneous'), ] def setup(app): app.add_stylesheet('css/modify.css') ########################################################################### # auto-created readthedocs.org specific configuration # ########################################################################### # # The following code was added during an automated build on readthedocs.org # It is auto created and injected for every build. The result is based on the # conf.py.tmpl file found in the readthedocs.org codebase: # https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/doc_builder/templates/doc_builder/conf.py.tmpl # import importlib import sys import os.path from six import string_types from sphinx import version_info # Get suffix for proper linking to GitHub # This is deprecated in Sphinx 1.3+, # as each page can have its own suffix if globals().get('source_suffix', False): if isinstance(source_suffix, string_types): SUFFIX = source_suffix elif isinstance(source_suffix, (list, tuple)): # Sphinx >= 1.3 supports list/tuple to define multiple suffixes SUFFIX = source_suffix[0] elif isinstance(source_suffix, dict): # Sphinx >= 1.8 supports a mapping dictionary for multiple suffixes SUFFIX = list(source_suffix.keys())[0] # make a ``list()`` for py2/py3 compatibility else: # default to .rst SUFFIX = '.rst' else: SUFFIX = '.rst' # Add RTD Static Path. Add to the end because it overwrites previous files. if not 'html_static_path' in globals(): html_static_path = [] if os.path.exists('_static'): html_static_path.append('_static') # Add RTD Theme only if they aren't overriding it already using_rtd_theme = ( ( 'html_theme' in globals() and html_theme in ['default'] and # Allow people to bail with a hack of having an html_style 'html_style' not in globals() ) or 'html_theme' not in globals() ) if using_rtd_theme: theme = importlib.import_module('sphinx_rtd_theme') html_theme = 'sphinx_rtd_theme' html_style = None html_theme_options = {} if 'html_theme_path' in globals(): html_theme_path.append(theme.get_html_theme_path()) else: html_theme_path = [theme.get_html_theme_path()] if globals().get('websupport2_base_url', False): websupport2_base_url = 'https://readthedocs.com/websupport' websupport2_static_url = 'https://media.readthedocs.com/' #Add project information to the template context. context = { 'using_theme': using_rtd_theme, 'html_theme': html_theme, 'current_version': "latest", 'version_slug': "latest", 'MEDIA_URL': "https://media.readthedocs.com/media/", 'STATIC_URL': "https://media.readthedocs.com/", 'PRODUCTION_DOMAIN': "readthedocs.com", 'versions': [ ("latest", "/en/latest/"), ], 'downloads': [ ("pdf", "//spinningup.openai.com/_/downloads/en/latest/pdf/"), ("html", "//spinningup.openai.com/_/downloads/en/latest/htmlzip/"), ("epub", "//spinningup.openai.com/_/downloads/en/latest/epub/"), ], 'subprojects': [ ], 'slug': 'openai-education-spinningup', 'name': u'spinningup', 'rtd_language': u'en', 'programming_language': u'words', 'canonical_url': 'https://spinningup.openai.com/en/latest/', 'analytics_code': 'UA-129132782-1', 'single_version': False, 'conf_py_path': '/docs/', 'api_host': 'https://readthedocs.com', 'proxied_api_host': 'https://readthedocs.com', 'github_user': 'openai', 'github_repo': 'spinningup', 'github_version': 'master', 'display_github': True, 'bitbucket_user': 'None', 'bitbucket_repo': 'None', 'bitbucket_version': 'master', 'display_bitbucket': False, 'gitlab_user': 'None', 'gitlab_repo': 'None', 'gitlab_version': 'master', 'display_gitlab': False, 'READTHEDOCS': True, 'using_theme': (html_theme == "default"), 'new_theme': (html_theme == "sphinx_rtd_theme"), 'source_suffix': SUFFIX, 'ad_free': False, 'user_analytics_code': 'UA-129132782-1', 'global_analytics_code': 'UA-17997319-2', 'commit': 'ed725b36', } if 'html_context' in globals(): html_context.update(context) else: html_context = context # Add custom RTD extension if 'extensions' in globals(): # Insert at the beginning because it can interfere # with other extensions. # See https://github.com/rtfd/readthedocs.org/pull/4054 extensions.insert(0, "readthedocs_ext.readthedocs") else: extensions = ["readthedocs_ext.readthedocs"] # Add External version warning banner to the external version documentation if 'branch' == 'external': extensions.insert(1, "readthedocs_ext.external_version_warning") project_language = 'en' # User's Sphinx configurations language_user = globals().get('language', None) latex_engine_user = globals().get('latex_engine', None) latex_elements_user = globals().get('latex_elements', None) # Remove this once xindy gets installed in Docker image and XINDYOPS # env variable is supported # https://github.com/rtfd/readthedocs-docker-images/pull/98 latex_use_xindy = False chinese = any([ language_user in ('zh_CN', 'zh_TW'), project_language in ('zh_CN', 'zh_TW'), ]) japanese = any([ language_user == 'ja', project_language == 'ja', ]) if chinese: latex_engine = latex_engine_user or 'xelatex' latex_elements_rtd = { 'preamble': '\\usepackage[UTF8]{ctex}\n', } latex_elements = latex_elements_user or latex_elements_rtd elif japanese: latex_engine = latex_engine_user or 'platex' [rtd-command-info] start-time: 2020-02-02T18:11:28.738689Z, end-time: 2020-02-02T18:13:10.813796Z, duration: 102, exit-code: 0 python sphinx-build -T -E -b readthedocs -d _build/doctrees-readthedocs -D language=en . _build/html Running Sphinx v1.5.6 making output directory... loading translations [en]... done building [mo]: targets for 0 po files that are out of date building [readthedocs]: targets for 37 source files that are out of date updating environment: 37 added, 0 changed, 0 removed reading sources... [ 2%] algorithms/ddpg reading sources... [ 5%] algorithms/ppo reading sources... [ 8%] algorithms/sac reading sources... [ 10%] algorithms/td3 reading sources... [ 13%] algorithms/trpo reading sources... [ 16%] algorithms/vpg reading sources... [ 18%] etc/acknowledgements reading sources... [ 21%] etc/author reading sources... [ 24%] index reading sources... [ 27%] spinningup/bench reading sources... [ 29%] spinningup/bench_ddpg reading sources... [ 32%] spinningup/bench_ppo reading sources... [ 35%] spinningup/bench_sac reading sources... [ 37%] spinningup/bench_td3 reading sources... [ 40%] spinningup/bench_vpg reading sources... [ 43%] spinningup/exercise2_1_soln reading sources... [ 45%] spinningup/exercise2_2_soln reading sources... [ 48%] spinningup/exercises reading sources... [ 51%] spinningup/extra_pg_proof1 reading sources... [ 54%] spinningup/extra_pg_proof2 reading sources... [ 56%] spinningup/extra_tf_pg_implementation reading sources... [ 59%] spinningup/keypapers reading sources... [ 62%] spinningup/rl_intro reading sources... [ 64%] spinningup/rl_intro2 reading sources... [ 67%] spinningup/rl_intro3 reading sources... [ 70%] spinningup/rl_intro4 reading sources... [ 72%] spinningup/spinningup reading sources... [ 75%] user/algorithms reading sources... [ 78%] user/installation reading sources... [ 81%] user/introduction reading sources... [ 83%] user/plotting reading sources... [ 86%] user/running reading sources... [ 89%] user/saving_and_loading reading sources... [ 91%] utils/logger reading sources... [ 94%] utils/mpi reading sources... [ 97%] utils/plotter reading sources... [100%] utils/run_utils /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercises.rst:3: WARNING: Duplicate explicit target name: "solution available here.". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:343: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "on github". looking for now-outdated files... none found pickling environment... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:419: WARNING: Line block ends without a blank line. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:341: ERROR: Duplicate target name, cannot be used as a unique reference: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:55: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:63: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:163: WARNING: Duplicate explicit target name: "cmdoption--ac_kwargs". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:306: WARNING: Inline strong start-string without end-string. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:148: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:173: WARNING: Duplicate explicit target name: "cmdoption-arg-default". done checking consistency... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ddpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ppo.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_sac.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_td3.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_vpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_1_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_2_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof1.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof2.rst:: WARNING: document isn't included in any toctree done /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro4.rst:: WARNING: document isn't included in any toctree preparing documents... done writing output... [ 2%] algorithms/ddpg writing output... [ 5%] algorithms/ppo writing output... [ 8%] algorithms/sac writing output... [ 10%] algorithms/td3 writing output... [ 13%] algorithms/trpo writing output... [ 16%] algorithms/vpg writing output... [ 18%] etc/acknowledgements writing output... [ 21%] etc/author writing output... [ 24%] index writing output... [ 27%] spinningup/bench writing output... [ 29%] spinningup/bench_ddpg writing output... [ 32%] spinningup/bench_ppo writing output... [ 35%] spinningup/bench_sac writing output... [ 37%] spinningup/bench_td3 writing output... [ 40%] spinningup/bench_vpg writing output... [ 43%] spinningup/exercise2_1_soln writing output... [ 45%] spinningup/exercise2_2_soln writing output... [ 48%] spinningup/exercises writing output... [ 51%] spinningup/extra_pg_proof1 writing output... [ 54%] spinningup/extra_pg_proof2 writing output... [ 56%] spinningup/extra_tf_pg_implementation writing output... [ 59%] spinningup/keypapers writing output... [ 62%] spinningup/rl_intro writing output... [ 64%] spinningup/rl_intro2 writing output... [ 67%] spinningup/rl_intro3 writing output... [ 70%] spinningup/rl_intro4 writing output... [ 72%] spinningup/spinningup writing output... [ 75%] user/algorithms writing output... [ 78%] user/installation writing output... [ 81%] user/introduction writing output... [ 83%] user/plotting writing output... [ 86%] user/running writing output... [ 89%] user/saving_and_loading writing output... [ 91%] utils/logger writing output... [ 94%] utils/mpi writing output... [ 97%] utils/plotter writing output... [100%] utils/run_utils generating indices... genindex py-modindex highlighting module code... [ 6%] spinup.algos.pytorch.ddpg.ddpg highlighting module code... [ 12%] spinup.algos.tf1.ddpg.ddpg highlighting module code... [ 18%] spinup.algos.pytorch.ppo.ppo highlighting module code... [ 25%] spinup.algos.tf1.ppo.ppo highlighting module code... [ 31%] spinup.algos.pytorch.sac.sac highlighting module code... [ 37%] spinup.algos.tf1.sac.sac highlighting module code... [ 43%] spinup.algos.pytorch.td3.td3 highlighting module code... [ 50%] spinup.algos.tf1.td3.td3 highlighting module code... [ 56%] spinup.algos.tf1.trpo.trpo highlighting module code... [ 62%] spinup.algos.pytorch.vpg.vpg highlighting module code... [ 68%] spinup.algos.tf1.vpg.vpg highlighting module code... [ 75%] spinup.utils.logx highlighting module code... [ 81%] spinup.utils.mpi_tools highlighting module code... [ 87%] spinup.utils.mpi_pytorch highlighting module code... [ 93%] spinup.utils.mpi_tf highlighting module code... [100%] spinup.utils.run_utils writing additional pages... search copying images... [ 2%] images/spinning-up-in-rl.png copying images... [ 4%] spinningup/../images/plots/pyt/pytorch_halfcheetah_performance.svg copying images... [ 7%] spinningup/../images/plots/tf1/tensorflow_halfcheetah_performance.svg copying images... [ 9%] spinningup/../images/plots/pyt/pytorch_hopper_performance.svg copying images... [ 12%] spinningup/../images/plots/tf1/tensorflow_hopper_performance.svg copying images... [ 14%] spinningup/../images/plots/pyt/pytorch_walker2d_performance.svg copying images... [ 17%] spinningup/../images/plots/tf1/tensorflow_walker2d_performance.svg copying images... [ 19%] spinningup/../images/plots/pyt/pytorch_swimmer_performance.svg copying images... [ 21%] spinningup/../images/plots/tf1/tensorflow_swimmer_performance.svg copying images... [ 24%] spinningup/../images/plots/pyt/pytorch_ant_performance.svg copying images... [ 26%] spinningup/../images/plots/tf1/tensorflow_ant_performance.svg copying images... [ 29%] spinningup/../images/plots/ddpg/ddpg_halfcheetah_performance.svg copying images... [ 31%] spinningup/../images/plots/ddpg/ddpg_hopper_performance.svg copying images... [ 34%] spinningup/../images/plots/ddpg/ddpg_walker2d_performance.svg copying images... [ 36%] spinningup/../images/plots/ddpg/ddpg_swimmer_performance.svg copying images... [ 39%] spinningup/../images/plots/ddpg/ddpg_ant_performance.svg copying images... [ 41%] spinningup/../images/plots/ppo/ppo_halfcheetah_performance.svg copying images... [ 43%] spinningup/../images/plots/ppo/ppo_hopper_performance.svg copying images... [ 46%] spinningup/../images/plots/ppo/ppo_walker2d_performance.svg copying images... [ 48%] spinningup/../images/plots/ppo/ppo_swimmer_performance.svg copying images... [ 51%] spinningup/../images/plots/ppo/ppo_ant_performance.svg copying images... [ 53%] spinningup/../images/plots/sac/sac_halfcheetah_performance.svg copying images... [ 56%] spinningup/../images/plots/sac/sac_hopper_performance.svg copying images... [ 58%] spinningup/../images/plots/sac/sac_walker2d_performance.svg copying images... [ 60%] spinningup/../images/plots/sac/sac_swimmer_performance.svg copying images... [ 63%] spinningup/../images/plots/sac/sac_ant_performance.svg copying images... [ 65%] spinningup/../images/plots/td3/td3_halfcheetah_performance.svg copying images... [ 68%] spinningup/../images/plots/td3/td3_hopper_performance.svg copying images... [ 70%] spinningup/../images/plots/td3/td3_walker2d_performance.svg copying images... [ 73%] spinningup/../images/plots/td3/td3_swimmer_performance.svg copying images... [ 75%] spinningup/../images/plots/td3/td3_ant_performance.svg copying images... [ 78%] spinningup/../images/plots/vpg/vpg_halfcheetah_performance.svg copying images... [ 80%] spinningup/../images/plots/vpg/vpg_hopper_performance.svg copying images... [ 82%] spinningup/../images/plots/vpg/vpg_walker2d_performance.svg copying images... [ 85%] spinningup/../images/plots/vpg/vpg_swimmer_performance.svg copying images... [ 87%] spinningup/../images/plots/vpg/vpg_ant_performance.svg copying images... [ 90%] spinningup/../images/ex2-1_trpo_hopper.png copying images... [ 92%] spinningup/../images/ex2-2_ddpg_bug.svg copying images... [ 95%] spinningup/../images/ex2-2_ddpg_bug_pytorch.png copying images... [ 97%] spinningup/../images/rl_diagram_transparent_bg.png copying images... [100%] spinningup/../images/rl_algorithms_9_15.svg copying static files... WARNING: favicon file 'openai_icon.ico' does not exist done copying extra files... done dumping search index in English (code: en) ... done dumping object inventory... done build succeeded, 25 warnings. [rtd-command-info] start-time: 2020-02-02T18:13:11.010467Z, end-time: 2020-02-02T18:14:28.522098Z, duration: 77, exit-code: 0 python sphinx-build -T -b readthedocssinglehtmllocalmedia -d _build/doctrees-readthedocssinglehtmllocalmedia -D language=en . _build/localmedia Running Sphinx v1.5.6 making output directory... loading translations [en]... done loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [readthedocssinglehtmllocalmedia]: all documents updating environment: 37 added, 0 changed, 0 removed reading sources... [ 2%] algorithms/ddpg reading sources... [ 5%] algorithms/ppo reading sources... [ 8%] algorithms/sac reading sources... [ 10%] algorithms/td3 reading sources... [ 13%] algorithms/trpo reading sources... [ 16%] algorithms/vpg reading sources... [ 18%] etc/acknowledgements reading sources... [ 21%] etc/author reading sources... [ 24%] index reading sources... [ 27%] spinningup/bench reading sources... [ 29%] spinningup/bench_ddpg reading sources... [ 32%] spinningup/bench_ppo reading sources... [ 35%] spinningup/bench_sac reading sources... [ 37%] spinningup/bench_td3 reading sources... [ 40%] spinningup/bench_vpg reading sources... [ 43%] spinningup/exercise2_1_soln reading sources... [ 45%] spinningup/exercise2_2_soln reading sources... [ 48%] spinningup/exercises reading sources... [ 51%] spinningup/extra_pg_proof1 reading sources... [ 54%] spinningup/extra_pg_proof2 reading sources... [ 56%] spinningup/extra_tf_pg_implementation reading sources... [ 59%] spinningup/keypapers reading sources... [ 62%] spinningup/rl_intro reading sources... [ 64%] spinningup/rl_intro2 reading sources... [ 67%] spinningup/rl_intro3 reading sources... [ 70%] spinningup/rl_intro4 reading sources... [ 72%] spinningup/spinningup reading sources... [ 75%] user/algorithms reading sources... [ 78%] user/installation reading sources... [ 81%] user/introduction reading sources... [ 83%] user/plotting reading sources... [ 86%] user/running reading sources... [ 89%] user/saving_and_loading reading sources... [ 91%] utils/logger reading sources... [ 94%] utils/mpi reading sources... [ 97%] utils/plotter reading sources... [100%] utils/run_utils /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercises.rst:3: WARNING: Duplicate explicit target name: "solution available here.". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:343: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:419: WARNING: Line block ends without a blank line. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:341: ERROR: Duplicate target name, cannot be used as a unique reference: "here". looking for now-outdated files... none found pickling environment... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:55: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:63: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:163: WARNING: Duplicate explicit target name: "cmdoption--ac_kwargs". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:306: WARNING: Inline strong start-string without end-string. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:148: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:173: WARNING: Duplicate explicit target name: "cmdoption-arg-default". done checking consistency... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ddpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ppo.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_sac.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_td3.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_vpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_1_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_2_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof1.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof2.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:: WARNING: document isn't included in any toctree done preparing documents... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro4.rst:: WARNING: document isn't included in any toctree done assembling single document... user/introduction user/installation user/algorithms user/running user/saving_and_loading user/plotting spinningup/rl_intro spinningup/rl_intro2 spinningup/rl_intro3 spinningup/spinningup spinningup/keypapers spinningup/exercises spinningup/bench algorithms/vpg algorithms/trpo algorithms/ppo algorithms/ddpg algorithms/td3 algorithms/sac utils/logger utils/plotter utils/mpi utils/run_utils etc/acknowledgements etc/author writing... done writing additional files... copying images... [ 7%] images/spinning-up-in-rl.png copying images... [ 15%] spinningup/../images/rl_diagram_transparent_bg.png copying images... [ 23%] spinningup/../images/rl_algorithms_9_15.svg copying images... [ 30%] spinningup/../images/plots/pyt/pytorch_halfcheetah_performance.svg copying images... [ 38%] spinningup/../images/plots/tf1/tensorflow_halfcheetah_performance.svg copying images... [ 46%] spinningup/../images/plots/pyt/pytorch_hopper_performance.svg copying images... [ 53%] spinningup/../images/plots/tf1/tensorflow_hopper_performance.svg copying images... [ 61%] spinningup/../images/plots/pyt/pytorch_walker2d_performance.svg copying images... [ 69%] spinningup/../images/plots/tf1/tensorflow_walker2d_performance.svg copying images... [ 76%] spinningup/../images/plots/pyt/pytorch_swimmer_performance.svg copying images... [ 84%] spinningup/../images/plots/tf1/tensorflow_swimmer_performance.svg copying images... [ 92%] spinningup/../images/plots/pyt/pytorch_ant_performance.svg copying images... [100%] spinningup/../images/plots/tf1/tensorflow_ant_performance.svg copying static files... done copying extra files... WARNING: favicon file 'openai_icon.ico' does not exist done dumping object inventory... done build succeeded, 25 warnings. [rtd-command-info] start-time: 2020-02-02T18:14:28.682571Z, end-time: 2020-02-02T18:14:36.100959Z, duration: 7, exit-code: 0 python sphinx-build -b latex -D language=en -d _build/doctrees . _build/latex Running Sphinx v1.5.6 making output directory... loading translations [en]... done loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [latex]: all documents updating environment: 37 added, 0 changed, 0 removed reading sources... [ 2%] algorithms/ddpg reading sources... [ 5%] algorithms/ppo reading sources... [ 8%] algorithms/sac reading sources... [ 10%] algorithms/td3 reading sources... [ 13%] algorithms/trpo reading sources... [ 16%] algorithms/vpg reading sources... [ 18%] etc/acknowledgements reading sources... [ 21%] etc/author reading sources... [ 24%] index reading sources... [ 27%] spinningup/bench reading sources... [ 29%] spinningup/bench_ddpg reading sources... [ 32%] spinningup/bench_ppo reading sources... [ 35%] spinningup/bench_sac reading sources... [ 37%] spinningup/bench_td3 reading sources... [ 40%] spinningup/bench_vpg reading sources... [ 43%] spinningup/exercise2_1_soln reading sources... [ 45%] spinningup/exercise2_2_soln reading sources... [ 48%] spinningup/exercises reading sources... [ 51%] spinningup/extra_pg_proof1 reading sources... [ 54%] spinningup/extra_pg_proof2 reading sources... [ 56%] spinningup/extra_tf_pg_implementation reading sources... [ 59%] spinningup/keypapers reading sources... [ 62%] spinningup/rl_intro reading sources... [ 64%] spinningup/rl_intro2 reading sources... [ 67%] spinningup/rl_intro3 reading sources... [ 70%] spinningup/rl_intro4 reading sources... [ 72%] spinningup/spinningup reading sources... [ 75%] user/algorithms reading sources... [ 78%] user/installation reading sources... [ 81%] user/introduction reading sources... [ 83%] user/plotting reading sources... [ 86%] user/running reading sources... [ 89%] user/saving_and_loading reading sources... [ 91%] utils/logger reading sources... [ 94%] utils/mpi reading sources... [ 97%] utils/plotter reading sources... [100%] utils/run_utils /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercises.rst:3: WARNING: Duplicate explicit target name: "solution available here.". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:343: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:419: WARNING: Line block ends without a blank line. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:341: ERROR: Duplicate target name, cannot be used as a unique reference: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:55: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:63: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:163: WARNING: Duplicate explicit target name: "cmdoption--ac_kwargs". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:306: WARNING: Inline strong start-string without end-string. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:148: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:173: WARNING: Duplicate explicit target name: "cmdoption-arg-default". looking for now-outdated files... none found pickling environment... done checking consistency... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ddpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ppo.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_sac.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_td3.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_vpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_1_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_2_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof1.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof2.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:: WARNING: document isn't included in any toctree done /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro4.rst:: WARNING: document isn't included in any toctree processing SpinningUp.tex... index user/introduction user/installation user/algorithms user/running user/saving_and_loading user/plotting spinningup/rl_intro spinningup/rl_intro2 spinningup/rl_intro3 spinningup/spinningup spinningup/keypapers spinningup/exercises spinningup/bench algorithms/vpg algorithms/trpo algorithms/ppo algorithms/ddpg algorithms/td3 algorithms/sac utils/logger utils/plotter utils/mpi utils/run_utils etc/acknowledgements etc/author resolving references... writing... done copying images... images/spinning-up-in-rl.png spinningup/../images/rl_diagram_transparent_bg.png spinningup/../images/rl_algorithms_9_15.svg spinningup/../images/plots/pyt/pytorch_halfcheetah_performance.svg spinningup/../images/plots/tf1/tensorflow_halfcheetah_performance.svg spinningup/../images/plots/pyt/pytorch_hopper_performance.svg spinningup/../images/plots/tf1/tensorflow_hopper_performance.svg spinningup/../images/plots/pyt/pytorch_walker2d_performance.svg spinningup/../images/plots/tf1/tensorflow_walker2d_performance.svg spinningup/../images/plots/pyt/pytorch_swimmer_performance.svg spinningup/../images/plots/tf1/tensorflow_swimmer_performance.svg spinningup/../images/plots/pyt/pytorch_ant_performance.svg spinningup/../images/plots/tf1/tensorflow_ant_performance.svg copying TeX support files... done build succeeded, 24 warnings. [rtd-command-info] start-time: 2020-02-02T18:14:36.406886Z, end-time: 2020-02-02T18:14:37.995028Z, duration: 1, exit-code: 0 pdflatex -interaction=nonstopmode /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/_build/latex/SpinningUp.tex This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex) restricted \write18 enabled. entering extended mode (/home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/c heckouts/latest/docs/_build/latex/SpinningUp.tex LaTeX2e <2017-04-15> Babel <3.18> and hyphenation patterns for 84 language(s) loaded. (./sphinxmanual.cls Document Class: sphinxmanual 2017/03/26 v1.5.4 Document class (Sphinx manual) (/usr/share/texlive/texmf-dist/tex/latex/base/report.cls Document Class: report 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))) (/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty (/usr/share/texlive/texmf-dist/tex/latex/base/utf8.def (/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.dfu) (/usr/share/texlive/texmf-dist/tex/latex/base/ot1enc.dfu) (/usr/share/texlive/texmf-dist/tex/latex/base/omsenc.dfu))) (/usr/share/texlive/texmf-dist/tex/latex/cmap/cmap.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty (/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.def)<>) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty For additional information on amsmath, use the `?' option. (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty)) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty)) (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amssymb.sty (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amsfonts.sty)) (/usr/share/texlive/texmf-dist/tex/generic/babel/babel.sty (/usr/share/texlive/texmf-dist/tex/generic/babel/switch.def) (/usr/share/texlive/texmf-dist/tex/generic/babel-english/english.ldf (/usr/share/texlive/texmf-dist/tex/generic/babel/babel.def (/usr/share/texlive/texmf-dist/tex/generic/babel/txtbabel.def)))) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/times.sty) (/usr/share/texlive/texmf-dist/tex/latex/fncychap/fncychap.sty) (/usr/share/texlive/texmf-dist/tex/latex/tools/longtable.sty) (./sphinx.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty) (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty) (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/graphics.cfg) (/usr/share/texlive/texmf-dist/tex/latex/graphics-def/pdftex.def))) (/usr/share/texlive/texmf-dist/tex/latex/fancyhdr/fancyhdr.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/textcomp.sty (/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.def (/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.dfu))) (/usr/share/texlive/texmf-dist/tex/latex/titlesec/titlesec.sty) (/usr/share/texlive/texmf-dist/tex/latex/tabulary/tabulary.sty (/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty)) (/usr/share/texlive/texmf-dist/tex/latex/base/makeidx.sty) (/usr/share/texlive/texmf-dist/tex/latex/framed/framed.sty) (/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg)) (/usr/share/texlive/texmf-dist/tex/latex/fancyvrb/fancyvrb.sty Style option: `fancyvrb' v2.7a, with DG/SPQR fixes, and firstline=lastline fix <2008/02/07> (tvz)) (/usr/share/texlive/texmf-dist/tex/latex/threeparttable/threeparttable.sty) (./footnotehyper-sphinx.sty (/usr/share/texlive/texmf-dist/tex/latex/mdwtools/footnote.sty)) (/usr/share/texlive/texmf-dist/tex/latex/float/float.sty) (/usr/share/texlive/texmf-dist/tex/latex/wrapfig/wrapfig.sty) (/usr/share/texlive/texmf-dist/tex/latex/parskip/parskip.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/alltt.sty) (/usr/share/texlive/texmf-dist/tex/latex/upquote/upquote.sty) (/usr/share/texlive/texmf-dist/tex/latex/capt-of/capt-of.sty) (./needspace.sty) (./sphinxhighlight.sty) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/kvoptions.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ltxcmds.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/kvsetkeys.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/infwarerr.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/etexcmds.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifluatex.sty)))) (/usr/share/texlive/texmf-dist/tex/generic/pdftex/pdfcolor.tex) ** (sphinx) defining (legacy) text style macros without \sphinx prefix ** if clashes with packages, set latex_keep_old_macro_names=False in conf.py ) (/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifpdf.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifvtex.sty) (/usr/share/texlive/texmf-dist/tex/generic/ifxetex/ifxetex.sty)) (/usr/share/texlive/texmf-dist/tex/latex/multirow/multirow.sty) (/usr/share/texlive/texmf-dist/tex/latex/eqparbox/eqparbox.sty (/usr/share/texlive/texmf-dist/tex/latex/environ/environ.sty (/usr/share/texlive/texmf-dist/tex/latex/trimspaces/trimspaces.sty))) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/hyperref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-hyperref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-generic.sty)) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/auxhook.sty) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def) (/usr/share/texlive/texmf-dist/tex/latex/latexconfig/hyperref.cfg) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/puenc.def) (/usr/share/texlive/texmf-dist/tex/latex/url/url.sty)) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/hpdftex.def (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/rerunfilecheck.sty)) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/hypcap.sty) (/usr/share/texlive/texmf-dist/tex/latex/algorithms/algorithm.sty (/usr/share/texlive/texmf-dist/tex/latex/base/ifthen.sty)) (/usr/share/texlive/texmf-dist/tex/latex/algorithms/algorithmic.sty) (/usr/share/texlive/texmf-dist/tex/latex/cancel/cancel.sty) Writing index file SpinningUp.idx No file SpinningUp.aux. (/usr/share/texlive/texmf-dist/tex/latex/base/ts1cmr.fd) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1ptm.fd) (/usr/share/texlive/texmf-dist/tex/context/base/mkii/supp-pdf.mkii [Loading MPS to PDF converter (version 2006.09.02).] ) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/epstopdf-base.sty (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/grfext.sty) (/usr/share/texlive/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg)) *geometry* driver: auto-detecting *geometry* detected driver: pdftex (/usr/share/texlive/texmf-dist/tex/latex/hyperref/nameref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/gettitlestring.sty)) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1phv.fd)<><><><> (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsa.fd) (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsb.fd) [1{/var/lib/texmf/fo nts/map/pdftex/updmap/pdftex.map}] [2] [1] [2] (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1pcr.fd) [1 <./spinning-up-in- rl.png>] [2] Chapter 1. (/usr/share/texlive/texmf-dist/tex/latex/psnfss/ts1ptm.fd) LaTeX Warning: Hyper reference `user/introduction:introduction' on page 3 undef ined on input line 80. LaTeX Warning: Hyper reference `user/introduction:what-this-is' on page 3 undef ined on input line 83. LaTeX Warning: Hyper reference `user/introduction:why-we-built-this' on page 3 undefined on input line 86. LaTeX Warning: Hyper reference `user/introduction:how-this-serves-our-mission' on page 3 undefined on input line 89. LaTeX Warning: Hyper reference `user/introduction:code-design-philosophy' on pa ge 3 undefined on input line 92. LaTeX Warning: Hyper reference `user/introduction:long-term-support-and-support -history' on page 3 undefined on input line 95. [3] [4] [5] [6] Chapter 2. LaTeX Warning: Hyper reference `user/installation:installation' on page 7 undef ined on input line 233. LaTeX Warning: Hyper reference `user/installation:installing-python' on page 7 undefined on input line 236. LaTeX Warning: Hyper reference `user/installation:installing-openmpi' on page 7 undefined on input line 239. LaTeX Warning: Hyper reference `user/installation:ubuntu' on page 7 undefined o n input line 242. LaTeX Warning: Hyper reference `user/installation:mac-os-x' on page 7 undefined on input line 245. LaTeX Warning: Hyper reference `user/installation:installing-spinning-up' on pa ge 7 undefined on input line 250. LaTeX Warning: Hyper reference `user/installation:check-your-install' on page 7 undefined on input line 253. LaTeX Warning: Hyper reference `user/installation:installing-mujoco-optional' o n page 7 undefined on input line 256. [7] [8] [9] [10] Chapter 3. LaTeX Warning: Hyper reference `user/algorithms:algorithms' on page 11 undefine d on input line 387. LaTeX Warning: Hyper reference `user/algorithms:what-s-included' on page 11 und efined on input line 390. LaTeX Warning: Hyper reference `user/algorithms:why-these-algorithms' on page 1 1 undefined on input line 393. LaTeX Warning: Hyper reference `user/algorithms:the-on-policy-algorithms' on pa ge 11 undefined on input line 396. LaTeX Warning: Hyper reference `user/algorithms:the-off-policy-algorithms' on p age 11 undefined on input line 399. LaTeX Warning: Hyper reference `user/algorithms:code-format' on page 11 undefin ed on input line 404. LaTeX Warning: Hyper reference `user/algorithms:the-algorithm-function-pytorch- version' on page 11 undefined on input line 407. LaTeX Warning: Hyper reference `user/algorithms:the-algorithm-function-tensorfl ow-version' on page 11 undefined on input line 410. LaTeX Warning: Hyper reference `user/algorithms:the-core-file' on page 11 undef ined on input line 413. [11] [12] [13] [14] Chapter 4. LaTeX Warning: Hyper reference `user/running:running-experiments' on page 15 un defined on input line 607. LaTeX Warning: Hyper reference `user/running:launching-from-the-command-line' o n page 15 undefined on input line 610. LaTeX Warning: Hyper reference `user/running:choosing-pytorch-or-tensorflow-fro m-the-command-line' on page 15 undefined on input line 613. LaTeX Warning: Hyper reference `user/running:setting-hyperparameters-from-the-c ommand-line' on page 15 undefined on input line 616. LaTeX Warning: Hyper reference `user/running:launching-multiple-experiments-at- once' on page 15 undefined on input line 619. LaTeX Warning: Hyper reference `user/running:special-flags' on page 15 undefine d on input line 622. LaTeX Warning: Hyper reference `user/running:environment-flag' on page 15 undef ined on input line 625. LaTeX Warning: Hyper reference `user/running:shortcut-flags' on page 15 undefin ed on input line 628. LaTeX Warning: Hyper reference `user/running:config-flags' on page 15 undefined on input line 631. LaTeX Warning: Hyper reference `user/running:where-results-are-saved' on page 1 5 undefined on input line 636. LaTeX Warning: Hyper reference `user/running:how-is-suffix-determined' on page 15 undefined on input line 639. LaTeX Warning: Hyper reference `user/running:extra' on page 15 undefined on inp ut line 644. LaTeX Warning: Hyper reference `user/running:launching-from-scripts' on page 15 undefined on input line 649. LaTeX Warning: Hyper reference `user/running:using-experimentgrid' on page 15 u ndefined on input line 652. [15] [16] [17] [18] [19] (/usr/share/texlive/texmf-dist/tex/latex/psnfss/ts1pcr.fd) [20] [21] [22] Chapter 5. LaTeX Warning: Hyper reference `user/saving_and_loading:experiment-outputs' on page 23 undefined on input line 1001. LaTeX Warning: Hyper reference `user/saving_and_loading:algorithm-outputs' on p age 23 undefined on input line 1004. LaTeX Warning: Hyper reference `user/saving_and_loading:pytorch-save-directory- info' on page 23 undefined on input line 1007. LaTeX Warning: Hyper reference `user/saving_and_loading:tensorflow-save-directo ry-info' on page 23 undefined on input line 1010. LaTeX Warning: Hyper reference `user/saving_and_loading:save-directory-location ' on page 23 undefined on input line 1015. LaTeX Warning: Hyper reference `user/saving_and_loading:loading-and-running-tra ined-policies' on page 23 undefined on input line 1018. LaTeX Warning: Hyper reference `user/saving_and_loading:if-environment-saves-su ccessfully' on page 23 undefined on input line 1021. LaTeX Warning: Hyper reference `user/saving_and_loading:environment-not-found-e rror' on page 23 undefined on input line 1024. LaTeX Warning: Hyper reference `user/saving_and_loading:using-trained-value-fun ctions' on page 23 undefined on input line 1027. [23] LaTeX Warning: Hyper reference `user/saving_and_loading:details-for-pytorch-sav es-below' on page 24 undefined on input line 1073. LaTeX Warning: Hyper reference `user/saving_and_loading:details-for-tensorflow- saves-below' on page 24 undefined on input line 1082. [24] [25] [26] Underfull \hbox (badness 10000) in paragraph at lines 1293--1293 [][][]\T1/pcr/m/n/10 It[][]looks[][]like[][]the[][]environment[][]wasn\TS1/pcr/ m/n/10 '\T1/pcr/m/n/10 t[][]saved,[][][]and[][]we[][]can\TS1/pcr/m/n/10 '\T1/pc r/m/n/10 t[][]run[][]the[][]agent[][]in[][]it. Underfull \hbox (badness 10000) in paragraph at lines 1295--1295 [][][]\T1/pcr/m/n/10 Check[][]out[][]the[][]readthedocs[][]page[][]on[][]Experi ment[][]Outputs[][]for[][]how[][]to[][]handle[][]this[] [27] [28] Chapter 6. [29] [30] Chapter 7. LaTeX Warning: Hyper reference `spinningup/rl_intro:part-1-key-concepts-in-rl' on page 31 undefined on input line 1426. LaTeX Warning: Hyper reference `spinningup/rl_intro:what-can-rl-do' on page 31 undefined on input line 1429. LaTeX Warning: Hyper reference `spinningup/rl_intro:key-concepts-and-terminolog y' on page 31 undefined on input line 1432. LaTeX Warning: Hyper reference `spinningup/rl_intro:optional-formalism' on page 31 undefined on input line 1435. [31] [32 <./rl_diagram_transparent_bg.png>] [33] [34] [35] [36] [37] [38] [39] [40] Chapter 8. LaTeX Warning: Hyper reference `spinningup/rl_intro2:part-2-kinds-of-rl-algorit hms' on page 41 undefined on input line 1839. LaTeX Warning: Hyper reference `spinningup/rl_intro2:a-taxonomy-of-rl-algorithm s' on page 41 undefined on input line 1842. LaTeX Warning: Hyper reference `spinningup/rl_intro2:links-to-algorithms-in-tax onomy' on page 41 undefined on input line 1845. ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.1860 ...ncludegraphics{{rl_algorithms_9_15}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.1860 ...ncludegraphics{{rl_algorithms_9_15}.svg} LaTeX Warning: Hyper reference `spinningup/rl_intro2:citations-below' on page 4 1 undefined on input line 1861. [41] [42] [43] [44] Chapter 9. LaTeX Warning: Hyper reference `spinningup/rl_intro3:part-3-intro-to-policy-opt imization' on page 45 undefined on input line 2002. LaTeX Warning: Hyper reference `spinningup/rl_intro3:deriving-the-simplest-poli cy-gradient' on page 45 undefined on input line 2005. LaTeX Warning: Hyper reference `spinningup/rl_intro3:implementing-the-simplest- policy-gradient' on page 45 undefined on input line 2008. LaTeX Warning: Hyper reference `spinningup/rl_intro3:expected-grad-log-prob-lem ma' on page 45 undefined on input line 2011. LaTeX Warning: Hyper reference `spinningup/rl_intro3:don-t-let-the-past-distrac t-you' on page 45 undefined on input line 2014. LaTeX Warning: Hyper reference `spinningup/rl_intro3:implementing-reward-to-go- policy-gradient' on page 45 undefined on input line 2017. LaTeX Warning: Hyper reference `spinningup/rl_intro3:baselines-in-policy-gradie nts' on page 45 undefined on input line 2020. LaTeX Warning: Hyper reference `spinningup/rl_intro3:other-forms-of-the-policy- gradient' on page 45 undefined on input line 2023. LaTeX Warning: Hyper reference `spinningup/rl_intro3:recap' on page 45 undefine d on input line 2026. [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] Chapter 10. LaTeX Warning: Hyper reference `spinningup/spinningup:spinning-up-as-a-deep-rl- researcher' on page 55 undefined on input line 2448. LaTeX Warning: Hyper reference `spinningup/spinningup:the-right-background' on page 55 undefined on input line 2451. LaTeX Warning: Hyper reference `spinningup/spinningup:learn-by-doing' on page 5 5 undefined on input line 2454. LaTeX Warning: Hyper reference `spinningup/spinningup:developing-a-research-pro ject' on page 55 undefined on input line 2457. LaTeX Warning: Hyper reference `spinningup/spinningup:doing-rigorous-research-i n-rl' on page 55 undefined on input line 2460. LaTeX Warning: Hyper reference `spinningup/spinningup:closing-thoughts' on page 55 undefined on input line 2463. LaTeX Warning: Hyper reference `spinningup/spinningup:ps-other-resources' on pa ge 55 undefined on input line 2466. LaTeX Warning: Hyper reference `spinningup/spinningup:references' on page 55 un defined on input line 2469. [55] [56] [57] [58] [59] [60] Chapter 11. LaTeX Warning: Hyper reference `spinningup/keypapers:key-papers-in-deep-rl' on page 61 undefined on input line 2578. LaTeX Warning: Hyper reference `spinningup/keypapers:model-free-rl' on page 61 undefined on input line 2581. LaTeX Warning: Hyper reference `spinningup/keypapers:exploration' on page 61 un defined on input line 2584. LaTeX Warning: Hyper reference `spinningup/keypapers:transfer-and-multitask-rl' on page 61 undefined on input line 2587. LaTeX Warning: Hyper reference `spinningup/keypapers:hierarchy' on page 61 unde fined on input line 2590. LaTeX Warning: Hyper reference `spinningup/keypapers:memory' on page 61 undefin ed on input line 2593. LaTeX Warning: Hyper reference `spinningup/keypapers:model-based-rl' on page 61 undefined on input line 2596. LaTeX Warning: Hyper reference `spinningup/keypapers:meta-rl' on page 61 undefi ned on input line 2599. LaTeX Warning: Hyper reference `spinningup/keypapers:scaling-rl' on page 61 und efined on input line 2602. LaTeX Warning: Hyper reference `spinningup/keypapers:rl-in-the-real-world' on p age 61 undefined on input line 2605. LaTeX Warning: Hyper reference `spinningup/keypapers:safety' on page 61 undefin ed on input line 2608. LaTeX Warning: Hyper reference `spinningup/keypapers:imitation-learning-and-inv erse-reinforcement-learning' on page 61 undefined on input line 2611. LaTeX Warning: Hyper reference `spinningup/keypapers:reproducibility-analysis-a nd-critique' on page 61 undefined on input line 2614. LaTeX Warning: Hyper reference `spinningup/keypapers:bonus-classic-papers-in-rl -theory-or-review' on page 61 undefined on input line 2617. [61] [62] Overfull \vbox (103.35579pt too high) has occurred while \output is active [63] [64] Chapter 12. LaTeX Warning: Hyper reference `spinningup/exercises:exercises' on page 65 unde fined on input line 2706. LaTeX Warning: Hyper reference `spinningup/exercises:problem-set-1-basics-of-im plementation' on page 65 undefined on input line 2709. LaTeX Warning: Hyper reference `spinningup/exercises:problem-set-2-algorithm-fa ilure-modes' on page 65 undefined on input line 2712. LaTeX Warning: Hyper reference `spinningup/exercises:challenges' on page 65 und efined on input line 2715. Underfull \hbox (badness 10000) in paragraph at lines 2740--2741 []\T1/ptm/m/n/10 PyTorch ver-sion: \T1/pcr/m/n/10 spinup/exercises/pytorch/prob lem_set_1_solutions/ Underfull \hbox (badness 10000) in paragraph at lines 2743--2744 []\T1/ptm/m/n/10 Tensorflow ver-sion: \T1/pcr/m/n/10 spinup/exercises/tf1/probl em_set_1_solutions/ [65] Underfull \hbox (badness 10000) in paragraph at lines 2771--2772 []\T1/ptm/m/n/10 PyTorch ver-sion: \T1/pcr/m/n/10 spinup/exercises/pytorch/prob lem_set_1_solutions/ Underfull \hbox (badness 10000) in paragraph at lines 2774--2775 []\T1/ptm/m/n/10 Tensorflow ver-sion: \T1/pcr/m/n/10 spinup/exercises/tf1/probl em_set_1_solutions/ [66] [67] [68] Chapter 13. LaTeX Warning: Hyper reference `spinningup/bench:benchmarks-for-spinning-up-imp lementations' on page 69 undefined on input line 2910. LaTeX Warning: Hyper reference `spinningup/bench:performance-in-each-environmen t' on page 69 undefined on input line 2913. LaTeX Warning: Hyper reference `spinningup/bench:halfcheetah-pytorch-versions' on page 69 undefined on input line 2916. LaTeX Warning: Hyper reference `spinningup/bench:halfcheetah-tensorflow-version s' on page 69 undefined on input line 2919. LaTeX Warning: Hyper reference `spinningup/bench:hopper-pytorch-versions' on pa ge 69 undefined on input line 2922. LaTeX Warning: Hyper reference `spinningup/bench:hopper-tensorflow-versions' on page 69 undefined on input line 2925. LaTeX Warning: Hyper reference `spinningup/bench:walker2d-pytorch-versions' on page 69 undefined on input line 2928. LaTeX Warning: Hyper reference `spinningup/bench:walker2d-tensorflow-versions' on page 69 undefined on input line 2931. LaTeX Warning: Hyper reference `spinningup/bench:swimmer-pytorch-versions' on p age 69 undefined on input line 2934. LaTeX Warning: Hyper reference `spinningup/bench:swimmer-tensorflow-versions' o n page 69 undefined on input line 2937. LaTeX Warning: Hyper reference `spinningup/bench:ant-pytorch-versions' on page 69 undefined on input line 2940. LaTeX Warning: Hyper reference `spinningup/bench:ant-tensorflow-versions' on pa ge 69 undefined on input line 2943. LaTeX Warning: Hyper reference `spinningup/bench:experiment-details' on page 69 undefined on input line 2948. LaTeX Warning: Hyper reference `spinningup/bench:pytorch-vs-tensorflow' on page 69 undefined on input line 2951. ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2969 ...s{{pytorch_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2969 ...s{{pytorch_halfcheetah_performance}.svg} [69] ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2978 ...tensorflow_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2978 ...tensorflow_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2987 ...aphics{{pytorch_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2987 ...aphics{{pytorch_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2996 ...ics{{tensorflow_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2996 ...ics{{tensorflow_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3005 ...hics{{pytorch_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3005 ...hics{{pytorch_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3014 ...s{{tensorflow_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3014 ...s{{tensorflow_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3023 ...phics{{pytorch_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3023 ...phics{{pytorch_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3032 ...cs{{tensorflow_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3032 ...cs{{tensorflow_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3041 ...egraphics{{pytorch_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3041 ...egraphics{{pytorch_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3050 ...aphics{{tensorflow_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3050 ...aphics{{tensorflow_ant_performance}.svg} [70] [71] [72] Chapter 14. LaTeX Warning: Hyper reference `algorithms/vpg:vanilla-policy-gradient' on page 73 undefined on input line 3104. LaTeX Warning: Hyper reference `algorithms/vpg:background' on page 73 undefined on input line 3107. LaTeX Warning: Hyper reference `algorithms/vpg:quick-facts' on page 73 undefine d on input line 3110. LaTeX Warning: Hyper reference `algorithms/vpg:key-equations' on page 73 undefi ned on input line 3113. LaTeX Warning: Hyper reference `algorithms/vpg:exploration-vs-exploitation' on page 73 undefined on input line 3116. LaTeX Warning: Hyper reference `algorithms/vpg:pseudocode' on page 73 undefined on input line 3119. LaTeX Warning: Hyper reference `algorithms/vpg:documentation' on page 73 undefi ned on input line 3124. LaTeX Warning: Hyper reference `algorithms/vpg:documentation-pytorch-version' o n page 73 undefined on input line 3127. LaTeX Warning: Hyper reference `algorithms/vpg:saved-model-contents-pytorch-ver sion' on page 73 undefined on input line 3130. LaTeX Warning: Hyper reference `algorithms/vpg:documentation-tensorflow-version ' on page 73 undefined on input line 3133. LaTeX Warning: Hyper reference `algorithms/vpg:saved-model-contents-tensorflow- version' on page 73 undefined on input line 3136. LaTeX Warning: Hyper reference `algorithms/vpg:references' on page 73 undefined on input line 3141. LaTeX Warning: Hyper reference `algorithms/vpg:relevant-papers' on page 73 unde fined on input line 3144. LaTeX Warning: Hyper reference `algorithms/vpg:why-these-papers' on page 73 und efined on input line 3147. LaTeX Warning: Hyper reference `algorithms/vpg:other-public-implementations' on page 73 undefined on input line 3150. [73] [74] Underfull \hbox (badness 10000) in paragraph at lines 3243--3243 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 5862) in paragraph at lines 3243--3243 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , [75] [76] Underfull \hbox (badness 10000) in paragraph at lines 3436--3436 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3436--3436 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_l r=0.0003\T1/ptm/m/n/10 , \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , [77] [78] [79] [80] Chapter 15. LaTeX Warning: Hyper reference `algorithms/trpo:trust-region-policy-optimizatio n' on page 81 undefined on input line 3640. LaTeX Warning: Hyper reference `algorithms/trpo:background' on page 81 undefine d on input line 3643. LaTeX Warning: Hyper reference `algorithms/trpo:quick-facts' on page 81 undefin ed on input line 3646. LaTeX Warning: Hyper reference `algorithms/trpo:key-equations' on page 81 undef ined on input line 3649. LaTeX Warning: Hyper reference `algorithms/trpo:exploration-vs-exploitation' on page 81 undefined on input line 3652. LaTeX Warning: Hyper reference `algorithms/trpo:pseudocode' on page 81 undefine d on input line 3655. LaTeX Warning: Hyper reference `algorithms/trpo:documentation' on page 81 undef ined on input line 3660. LaTeX Warning: Hyper reference `algorithms/trpo:saved-model-contents' on page 8 1 undefined on input line 3663. LaTeX Warning: Hyper reference `algorithms/trpo:references' on page 81 undefine d on input line 3668. LaTeX Warning: Hyper reference `algorithms/trpo:relevant-papers' on page 81 und efined on input line 3671. LaTeX Warning: Hyper reference `algorithms/trpo:why-these-papers' on page 81 un defined on input line 3674. LaTeX Warning: Hyper reference `algorithms/trpo:other-public-implementations' o n page 81 undefined on input line 3677. [81] [82] [83] Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 delt a=0.01\T1/ptm/m/n/10 , \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 \T1/ptm/m/it/10 train_v_iters=80\T1/ptm/m/n/10 , \T1/ptm/m/it/10 damp-ing_coeff =0.1\T1/ptm/m/n/10 , \T1/ptm/m/it/10 cg_iters=10\T1/ptm/m/n/10 , \T1/ptm/m/it/1 0 back-track_iters=10\T1/ptm/m/n/10 , \T1/ptm/m/it/10 back- [84] [85] Overfull \vbox (100.02797pt too high) has occurred while \output is active [86] [87] [88] Chapter 16. LaTeX Warning: Hyper reference `algorithms/ppo:proximal-policy-optimization' on page 89 undefined on input line 4096. LaTeX Warning: Hyper reference `algorithms/ppo:background' on page 89 undefined on input line 4099. LaTeX Warning: Hyper reference `algorithms/ppo:quick-facts' on page 89 undefine d on input line 4102. LaTeX Warning: Hyper reference `algorithms/ppo:key-equations' on page 89 undefi ned on input line 4105. LaTeX Warning: Hyper reference `algorithms/ppo:exploration-vs-exploitation' on page 89 undefined on input line 4108. LaTeX Warning: Hyper reference `algorithms/ppo:pseudocode' on page 89 undefined on input line 4111. LaTeX Warning: Hyper reference `algorithms/ppo:documentation' on page 89 undefi ned on input line 4116. LaTeX Warning: Hyper reference `algorithms/ppo:documentation-pytorch-version' o n page 89 undefined on input line 4119. LaTeX Warning: Hyper reference `algorithms/ppo:saved-model-contents-pytorch-ver sion' on page 89 undefined on input line 4122. LaTeX Warning: Hyper reference `algorithms/ppo:documentation-tensorflow-version ' on page 89 undefined on input line 4125. LaTeX Warning: Hyper reference `algorithms/ppo:saved-model-contents-tensorflow- version' on page 89 undefined on input line 4128. LaTeX Warning: Hyper reference `algorithms/ppo:references' on page 89 undefined on input line 4133. LaTeX Warning: Hyper reference `algorithms/ppo:relevant-papers' on page 89 unde fined on input line 4136. LaTeX Warning: Hyper reference `algorithms/ppo:why-these-papers' on page 89 und efined on input line 4139. LaTeX Warning: Hyper reference `algorithms/ppo:other-public-implementations' on page 89 undefined on input line 4142. [89] [90] [91] Underfull \hbox (badness 10000) in paragraph at lines 4289--4289 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 5862) in paragraph at lines 4289--4289 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , [92] [93] Underfull \hbox (badness 10000) in paragraph at lines 4500--4500 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 5274) in paragraph at lines 4500--4500 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 clip _ratio=0.2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_lr=0.0003\T1/ptm/m/n/10 , Underfull \hbox (badness 6461) in paragraph at lines 4500--4500 \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 train_pi_iters=80\T 1/ptm/m/n/10 , \T1/ptm/m/it/10 train_v_iters=80\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lam=0.97\T1/ptm/m/n/10 , \T1/ptm/m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , [94] [95] [96] [97] [98] Chapter 17. LaTeX Warning: Hyper reference `algorithms/ddpg:deep-deterministic-policy-gradi ent' on page 99 undefined on input line 4725. LaTeX Warning: Hyper reference `algorithms/ddpg:background' on page 99 undefine d on input line 4728. LaTeX Warning: Hyper reference `algorithms/ddpg:quick-facts' on page 99 undefin ed on input line 4731. LaTeX Warning: Hyper reference `algorithms/ddpg:key-equations' on page 99 undef ined on input line 4734. LaTeX Warning: Hyper reference `algorithms/ddpg:the-q-learning-side-of-ddpg' on page 99 undefined on input line 4737. LaTeX Warning: Hyper reference `algorithms/ddpg:the-policy-learning-side-of-ddp g' on page 99 undefined on input line 4740. LaTeX Warning: Hyper reference `algorithms/ddpg:exploration-vs-exploitation' on page 99 undefined on input line 4745. LaTeX Warning: Hyper reference `algorithms/ddpg:pseudocode' on page 99 undefine d on input line 4748. LaTeX Warning: Hyper reference `algorithms/ddpg:documentation' on page 99 undef ined on input line 4753. LaTeX Warning: Hyper reference `algorithms/ddpg:documentation-pytorch-version' on page 99 undefined on input line 4756. LaTeX Warning: Hyper reference `algorithms/ddpg:saved-model-contents-pytorch-ve rsion' on page 99 undefined on input line 4759. LaTeX Warning: Hyper reference `algorithms/ddpg:documentation-tensorflow-versio n' on page 99 undefined on input line 4762. LaTeX Warning: Hyper reference `algorithms/ddpg:saved-model-contents-tensorflow -version' on page 99 undefined on input line 4765. LaTeX Warning: Hyper reference `algorithms/ddpg:references' on page 99 undefine d on input line 4770. LaTeX Warning: Hyper reference `algorithms/ddpg:relevant-papers' on page 99 und efined on input line 4773. LaTeX Warning: Hyper reference `algorithms/ddpg:why-these-papers' on page 99 un defined on input line 4776. LaTeX Warning: Hyper reference `algorithms/ddpg:other-public-implementations' o n page 99 undefined on input line 4779. [99] [100] [101] [102] Underfull \hbox (badness 10000) in paragraph at lines 4948--4948 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 4948--4948 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- [103] [104] Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_lr=0.001\T1/ptm /m/n/10 , \T1/ptm/m/it/10 q_lr=0.001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 batch_size =100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=10000\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_ever y=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 act_noise=0.1\T1/ptm/m/n/10 , \T1/ptm/m/it /10 num_test_episodes=10\T1/ptm/m/n/10 , [105] [106] [107] [108] Chapter 18. LaTeX Warning: Hyper reference `algorithms/td3:twin-delayed-ddpg' on page 109 u ndefined on input line 5339. LaTeX Warning: Hyper reference `algorithms/td3:background' on page 109 undefine d on input line 5342. LaTeX Warning: Hyper reference `algorithms/td3:quick-facts' on page 109 undefin ed on input line 5345. LaTeX Warning: Hyper reference `algorithms/td3:key-equations' on page 109 undef ined on input line 5348. LaTeX Warning: Hyper reference `algorithms/td3:exploration-vs-exploitation' on page 109 undefined on input line 5351. LaTeX Warning: Hyper reference `algorithms/td3:pseudocode' on page 109 undefine d on input line 5354. LaTeX Warning: Hyper reference `algorithms/td3:documentation' on page 109 undef ined on input line 5359. LaTeX Warning: Hyper reference `algorithms/td3:documentation-pytorch-version' o n page 109 undefined on input line 5362. LaTeX Warning: Hyper reference `algorithms/td3:saved-model-contents-pytorch-ver sion' on page 109 undefined on input line 5365. LaTeX Warning: Hyper reference `algorithms/td3:documentation-tensorflow-version ' on page 109 undefined on input line 5368. LaTeX Warning: Hyper reference `algorithms/td3:saved-model-contents-tensorflow- version' on page 109 undefined on input line 5371. LaTeX Warning: Hyper reference `algorithms/td3:references' on page 109 undefine d on input line 5376. LaTeX Warning: Hyper reference `algorithms/td3:relevant-papers' on page 109 und efined on input line 5379. LaTeX Warning: Hyper reference `algorithms/td3:other-public-implementations' on page 109 undefined on input line 5382. [109] [110] [111] [112] Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 batch_size=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=1000 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/ it/10 up-date_every=50\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 act_noise=0.1\T1/ptm/m/n/10 , \T1/ptm/m/it/10 tar-get_noise=0.2 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 noise_clip=0.5\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pol-icy_delay=2\T1/ptm/m/n/10 , [113] [114] Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 \T1/ptm/m/it/10 pol-icy_delay=2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_test_episod es=10\T1/ptm/m/n/10 , \T1/ptm/m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 log-ger_kwargs={}\T1/ptm/m/n/10 , [115] [116] [117] [118] Chapter 19. LaTeX Warning: Hyper reference `algorithms/sac:soft-actor-critic' on page 119 u ndefined on input line 5950. LaTeX Warning: Hyper reference `algorithms/sac:background' on page 119 undefine d on input line 5953. LaTeX Warning: Hyper reference `algorithms/sac:quick-facts' on page 119 undefin ed on input line 5956. LaTeX Warning: Hyper reference `algorithms/sac:key-equations' on page 119 undef ined on input line 5959. LaTeX Warning: Hyper reference `algorithms/sac:entropy-regularized-reinforcemen t-learning' on page 119 undefined on input line 5962. LaTeX Warning: Hyper reference `algorithms/sac:id1' on page 119 undefined on in put line 5965. LaTeX Warning: Hyper reference `algorithms/sac:exploration-vs-exploitation' on page 119 undefined on input line 5970. LaTeX Warning: Hyper reference `algorithms/sac:pseudocode' on page 119 undefine d on input line 5973. LaTeX Warning: Hyper reference `algorithms/sac:documentation' on page 119 undef ined on input line 5978. LaTeX Warning: Hyper reference `algorithms/sac:documentation-pytorch-version' o n page 119 undefined on input line 5981. LaTeX Warning: Hyper reference `algorithms/sac:saved-model-contents-pytorch-ver sion' on page 119 undefined on input line 5984. LaTeX Warning: Hyper reference `algorithms/sac:documentation-tensorflow-version ' on page 119 undefined on input line 5987. LaTeX Warning: Hyper reference `algorithms/sac:saved-model-contents-tensorflow- version' on page 119 undefined on input line 5990. LaTeX Warning: Hyper reference `algorithms/sac:references' on page 119 undefine d on input line 5995. LaTeX Warning: Hyper reference `algorithms/sac:relevant-papers' on page 119 und efined on input line 5998. LaTeX Warning: Hyper reference `algorithms/sac:other-public-implementations' on page 119 undefined on input line 6001. [119] [120] [121] [122] [123] Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1 /ptm/m/n/10 , \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lr=0 .001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 al-pha=0.2\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 batch_size=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=1000 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/ it/10 up-date_every=50\T1/ptm/m/n/10 , [124] [125] [126] Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lr=0.001\T1/ptm/m/ n/10 , \T1/ptm/m/it/10 al-pha=0.2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 batch_size=10 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=10000\T1/ptm/m/n/10 , \T1/ptm/m/i t/10 up- Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_every=5 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_test_episodes=10\T1/ptm/m/n/10 , \T1/ptm/ m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , [127] [128] [129] [130] Chapter 20. LaTeX Warning: Hyper reference `utils/logger:logger' on page 131 undefined on i nput line 6687. LaTeX Warning: Hyper reference `utils/logger:using-a-logger' on page 131 undefi ned on input line 6690. LaTeX Warning: Hyper reference `utils/logger:examples' on page 131 undefined on input line 6693. LaTeX Warning: Hyper reference `utils/logger:logging-and-pytorch' on page 131 u ndefined on input line 6696. LaTeX Warning: Hyper reference `utils/logger:logging-and-mpi' on page 131 undef ined on input line 6699. LaTeX Warning: Hyper reference `utils/logger:logger-classes' on page 131 undefi ned on input line 6704. LaTeX Warning: Hyper reference `utils/logger:loading-saved-models-pytorch-only' on page 131 undefined on input line 6707. LaTeX Warning: Hyper reference `utils/logger:loading-saved-graphs-tensorflow-on ly' on page 131 undefined on input line 6710. [131] [132] [133] [134] LaTeX Warning: Hyper reference `utils/logger:spinup.utils.logx.Logger' on page 135 undefined on input line 7035. [135] [136] [137] [138] Chapter 21. [139] [140] Chapter 22. LaTeX Warning: Hyper reference `utils/mpi:mpi-tools' on page 141 undefined on i nput line 7195. LaTeX Warning: Hyper reference `utils/mpi:module-spinup.utils.mpi_tools' on pag e 141 undefined on input line 7198. LaTeX Warning: Hyper reference `utils/mpi:mpi-pytorch-utilities' on page 141 un defined on input line 7201. LaTeX Warning: Hyper reference `utils/mpi:mpi-tensorflow-utilities' on page 141 undefined on input line 7204. [141] [142] [143] [144] Chapter 23. LaTeX Warning: Hyper reference `utils/run_utils:run-utils' on page 145 undefine d on input line 7382. LaTeX Warning: Hyper reference `utils/run_utils:experimentgrid' on page 145 und efined on input line 7385. LaTeX Warning: Hyper reference `utils/run_utils:calling-experiments' on page 14 5 undefined on input line 7388. [145] Underfull \hbox (badness 10000) in paragraph at lines 7540--7540 []\T1/ptm/m/it/10 exp_name\T1/ptm/m/n/10 , \T1/ptm/m/it/10 thunk\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_cpu=1\T1/ptm/m/n/1 0 , [146] [147] [148] Chapter 24. [149] [150] Chapter 25. [151] [152] Chapter 26. [153] [154] LaTeX Warning: Reference `utils/mpi:module-spinup.utils.mpi_pytorch' on page 15 5 undefined on input line 7673. LaTeX Warning: Reference `utils/mpi:module-spinup.utils.mpi_tf' on page 155 und efined on input line 7674. LaTeX Warning: Reference `utils/mpi:module-spinup.utils.mpi_tools' on page 155 undefined on input line 7675. [155] No file SpinningUp.ind. (./SpinningUp.aux) Package rerunfilecheck Warning: File `SpinningUp.out' has changed. (rerunfilecheck) Rerun to get outlines right (rerunfilecheck) or use package `bookmark'. LaTeX Warning: There were undefined references. LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right. ) (see the transcript file for additional information){/usr/share/texlive/texmf-d ist/fonts/enc/dvips/base/8r.enc}< /usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr5.pfb> Output written on SpinningUp.pdf (159 pages, 1179803 bytes). Transcript written on SpinningUp.log. [rtd-command-info] start-time: 2020-02-02T18:14:38.105742Z, end-time: 2020-02-02T18:14:38.370386Z, duration: 0, exit-code: 0 makeindex -s python.ist SpinningUp.idx This is makeindex, version 2.15 [TeX Live 2017] (kpathsea + Thai support). Scanning style file ./python.ist.......done (7 attributes redefined, 0 ignored). Scanning input file SpinningUp.idx....done (88 entries accepted, 0 rejected). Sorting entries....done (589 comparisons). Generating output file SpinningUp.ind....done (154 lines written, 0 warnings). Output written in SpinningUp.ind. Transcript written in SpinningUp.ilg. [rtd-command-info] start-time: 2020-02-02T18:14:38.439573Z, end-time: 2020-02-02T18:14:39.868871Z, duration: 1, exit-code: 0 pdflatex -interaction=nonstopmode /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/_build/latex/SpinningUp.tex This is pdfTeX, Version 3.14159265-2.6-1.40.18 (TeX Live 2017/Debian) (preloaded format=pdflatex) restricted \write18 enabled. entering extended mode (/home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/c heckouts/latest/docs/_build/latex/SpinningUp.tex LaTeX2e <2017-04-15> Babel <3.18> and hyphenation patterns for 84 language(s) loaded. (./sphinxmanual.cls Document Class: sphinxmanual 2017/03/26 v1.5.4 Document class (Sphinx manual) (/usr/share/texlive/texmf-dist/tex/latex/base/report.cls Document Class: report 2014/09/29 v1.4h Standard LaTeX document class (/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))) (/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty (/usr/share/texlive/texmf-dist/tex/latex/base/utf8.def (/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.dfu) (/usr/share/texlive/texmf-dist/tex/latex/base/ot1enc.dfu) (/usr/share/texlive/texmf-dist/tex/latex/base/omsenc.dfu))) (/usr/share/texlive/texmf-dist/tex/latex/cmap/cmap.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty (/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.def)<>) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty For additional information on amsmath, use the `?' option. (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty)) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty)) (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amssymb.sty (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amsfonts.sty)) (/usr/share/texlive/texmf-dist/tex/generic/babel/babel.sty (/usr/share/texlive/texmf-dist/tex/generic/babel/switch.def) (/usr/share/texlive/texmf-dist/tex/generic/babel-english/english.ldf (/usr/share/texlive/texmf-dist/tex/generic/babel/babel.def (/usr/share/texlive/texmf-dist/tex/generic/babel/txtbabel.def)))) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/times.sty) (/usr/share/texlive/texmf-dist/tex/latex/fncychap/fncychap.sty) (/usr/share/texlive/texmf-dist/tex/latex/tools/longtable.sty) (./sphinx.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty) (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty) (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/graphics.cfg) (/usr/share/texlive/texmf-dist/tex/latex/graphics-def/pdftex.def))) (/usr/share/texlive/texmf-dist/tex/latex/fancyhdr/fancyhdr.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/textcomp.sty (/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.def (/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.dfu))) (/usr/share/texlive/texmf-dist/tex/latex/titlesec/titlesec.sty) (/usr/share/texlive/texmf-dist/tex/latex/tabulary/tabulary.sty (/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty)) (/usr/share/texlive/texmf-dist/tex/latex/base/makeidx.sty) (/usr/share/texlive/texmf-dist/tex/latex/framed/framed.sty) (/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg)) (/usr/share/texlive/texmf-dist/tex/latex/fancyvrb/fancyvrb.sty Style option: `fancyvrb' v2.7a, with DG/SPQR fixes, and firstline=lastline fix <2008/02/07> (tvz)) (/usr/share/texlive/texmf-dist/tex/latex/threeparttable/threeparttable.sty) (./footnotehyper-sphinx.sty (/usr/share/texlive/texmf-dist/tex/latex/mdwtools/footnote.sty)) (/usr/share/texlive/texmf-dist/tex/latex/float/float.sty) (/usr/share/texlive/texmf-dist/tex/latex/wrapfig/wrapfig.sty) (/usr/share/texlive/texmf-dist/tex/latex/parskip/parskip.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/alltt.sty) (/usr/share/texlive/texmf-dist/tex/latex/upquote/upquote.sty) (/usr/share/texlive/texmf-dist/tex/latex/capt-of/capt-of.sty) (./needspace.sty) (./sphinxhighlight.sty) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/kvoptions.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ltxcmds.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/kvsetkeys.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/infwarerr.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/etexcmds.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifluatex.sty)))) (/usr/share/texlive/texmf-dist/tex/generic/pdftex/pdfcolor.tex) ** (sphinx) defining (legacy) text style macros without \sphinx prefix ** if clashes with packages, set latex_keep_old_macro_names=False in conf.py ) (/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifpdf.sty) (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifvtex.sty) (/usr/share/texlive/texmf-dist/tex/generic/ifxetex/ifxetex.sty)) (/usr/share/texlive/texmf-dist/tex/latex/multirow/multirow.sty) (/usr/share/texlive/texmf-dist/tex/latex/eqparbox/eqparbox.sty (/usr/share/texlive/texmf-dist/tex/latex/environ/environ.sty (/usr/share/texlive/texmf-dist/tex/latex/trimspaces/trimspaces.sty))) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/hyperref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-hyperref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-generic.sty)) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/auxhook.sty) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def) (/usr/share/texlive/texmf-dist/tex/latex/latexconfig/hyperref.cfg) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/puenc.def) (/usr/share/texlive/texmf-dist/tex/latex/url/url.sty)) (/usr/share/texlive/texmf-dist/tex/latex/hyperref/hpdftex.def (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/rerunfilecheck.sty)) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/hypcap.sty) (/usr/share/texlive/texmf-dist/tex/latex/algorithms/algorithm.sty (/usr/share/texlive/texmf-dist/tex/latex/base/ifthen.sty)) (/usr/share/texlive/texmf-dist/tex/latex/algorithms/algorithmic.sty) (/usr/share/texlive/texmf-dist/tex/latex/cancel/cancel.sty) Writing index file SpinningUp.idx (./SpinningUp.aux LaTeX Warning: Label `alg1' multiply defined. LaTeX Warning: Label `alg1' multiply defined. LaTeX Warning: Label `alg1' multiply defined. LaTeX Warning: Label `alg1' multiply defined. LaTeX Warning: Label `alg1' multiply defined. ) (/usr/share/texlive/texmf-dist/tex/latex/base/ts1cmr.fd) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1ptm.fd) (/usr/share/texlive/texmf-dist/tex/context/base/mkii/supp-pdf.mkii [Loading MPS to PDF converter (version 2006.09.02).] ) (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/epstopdf-base.sty (/usr/share/texlive/texmf-dist/tex/latex/oberdiek/grfext.sty) (/usr/share/texlive/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg)) *geometry* driver: auto-detecting *geometry* detected driver: pdftex (/usr/share/texlive/texmf-dist/tex/latex/hyperref/nameref.sty (/usr/share/texlive/texmf-dist/tex/generic/oberdiek/gettitlestring.sty)) (./SpinningUp.out) (./SpinningUp.out) (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1phv.fd)<><><><> (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsa.fd) (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsb.fd) [1{/var/lib/texmf/fo nts/map/pdftex/updmap/pdftex.map}] [2] (./SpinningUp.toc [1] [2]) [3] [4] (/usr/share/texlive/texmf-dist/tex/latex/psnfss/t1pcr.fd) [1 <./spinning-up-in- rl.png>] [2] Chapter 1. (/usr/share/texlive/texmf-dist/tex/latex/psnfss/ts1ptm.fd) [3] [4] [5] [6] Chapter 2. [7] [8] [9] [10] Chapter 3. [11] [12] [13] [14] Chapter 4. [15] [16] [17] [18] [19] (/usr/share/texlive/texmf-dist/tex/latex/psnfss/ts1pcr.fd) [20] [21] [22] Chapter 5. [23] [24] [25] [26] Underfull \hbox (badness 10000) in paragraph at lines 1293--1293 [][][]\T1/pcr/m/n/10 It[][]looks[][]like[][]the[][]environment[][]wasn\TS1/pcr/ m/n/10 '\T1/pcr/m/n/10 t[][]saved,[][][]and[][]we[][]can\TS1/pcr/m/n/10 '\T1/pc r/m/n/10 t[][]run[][]the[][]agent[][]in[][]it. Underfull \hbox (badness 10000) in paragraph at lines 1295--1295 [][][]\T1/pcr/m/n/10 Check[][]out[][]the[][]readthedocs[][]page[][]on[][]Experi ment[][]Outputs[][]for[][]how[][]to[][]handle[][]this[] [27] [28] Chapter 6. [29] [30] Chapter 7. [31] [32 <./rl_diagram_transparent_bg.png>] [33] [34] [35] [36] [37] [38] [39] [40] Chapter 8. ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.1860 ...ncludegraphics{{rl_algorithms_9_15}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.1860 ...ncludegraphics{{rl_algorithms_9_15}.svg} [41] [42] [43] [44] Chapter 9. [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] Chapter 10. [55] [56] [57] [58] [59] [60] Chapter 11. [61] [62] Overfull \vbox (103.35579pt too high) has occurred while \output is active [63] [64] Chapter 12. Underfull \hbox (badness 10000) in paragraph at lines 2740--2741 []\T1/ptm/m/n/10 PyTorch ver-sion: \T1/pcr/m/n/10 spinup/exercises/pytorch/prob lem_set_1_solutions/ Underfull \hbox (badness 10000) in paragraph at lines 2743--2744 []\T1/ptm/m/n/10 Tensorflow ver-sion: \T1/pcr/m/n/10 spinup/exercises/tf1/probl em_set_1_solutions/ [65] Underfull \hbox (badness 10000) in paragraph at lines 2771--2772 []\T1/ptm/m/n/10 PyTorch ver-sion: \T1/pcr/m/n/10 spinup/exercises/pytorch/prob lem_set_1_solutions/ Underfull \hbox (badness 10000) in paragraph at lines 2774--2775 []\T1/ptm/m/n/10 Tensorflow ver-sion: \T1/pcr/m/n/10 spinup/exercises/tf1/probl em_set_1_solutions/ [66] [67] [68] Chapter 13. ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2969 ...s{{pytorch_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2969 ...s{{pytorch_halfcheetah_performance}.svg} [69] ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2978 ...tensorflow_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2978 ...tensorflow_halfcheetah_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2987 ...aphics{{pytorch_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2987 ...aphics{{pytorch_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2996 ...ics{{tensorflow_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.2996 ...ics{{tensorflow_hopper_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3005 ...hics{{pytorch_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3005 ...hics{{pytorch_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3014 ...s{{tensorflow_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3014 ...s{{tensorflow_walker2d_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3023 ...phics{{pytorch_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3023 ...phics{{pytorch_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3032 ...cs{{tensorflow_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3032 ...cs{{tensorflow_swimmer_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3041 ...egraphics{{pytorch_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3041 ...egraphics{{pytorch_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3050 ...aphics{{tensorflow_ant_performance}.svg} ! LaTeX Error: Unknown graphics extension: .svg. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.3050 ...aphics{{tensorflow_ant_performance}.svg} [70] [71] [72] Chapter 14. [73] [74] Underfull \hbox (badness 10000) in paragraph at lines 3243--3243 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 5862) in paragraph at lines 3243--3243 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , [75] [76] Underfull \hbox (badness 10000) in paragraph at lines 3436--3436 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3436--3436 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_l r=0.0003\T1/ptm/m/n/10 , \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , [77] [78] [79] [80] Chapter 15. [81] [82] [83] Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 delt a=0.01\T1/ptm/m/n/10 , \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 3818--3818 \T1/ptm/m/it/10 train_v_iters=80\T1/ptm/m/n/10 , \T1/ptm/m/it/10 damp-ing_coeff =0.1\T1/ptm/m/n/10 , \T1/ptm/m/it/10 cg_iters=10\T1/ptm/m/n/10 , \T1/ptm/m/it/1 0 back-track_iters=10\T1/ptm/m/n/10 , \T1/ptm/m/it/10 back- [84] [85] Overfull \vbox (100.02797pt too high) has occurred while \output is active [86] [87] [88] Chapter 16. [89] [90] [91] Underfull \hbox (badness 10000) in paragraph at lines 4289--4289 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 5862) in paragraph at lines 4289--4289 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , [92] [93] Underfull \hbox (badness 10000) in paragraph at lines 4500--4500 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 5274) in paragraph at lines 4500--4500 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=50\ T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1/ptm/m/n/10 , \T1/ptm/m/it/10 clip _ratio=0.2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_lr=0.0003\T1/ptm/m/n/10 , Underfull \hbox (badness 6461) in paragraph at lines 4500--4500 \T1/ptm/m/it/10 vf_lr=0.001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 train_pi_iters=80\T 1/ptm/m/n/10 , \T1/ptm/m/it/10 train_v_iters=80\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lam=0.97\T1/ptm/m/n/10 , \T1/ptm/m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , [94] [95] [96] [97] [98] Chapter 17. [99] [100] [101] [102] Underfull \hbox (badness 10000) in paragraph at lines 4948--4948 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 4948--4948 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- [103] [104] Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pi_lr=0.001\T1/ptm /m/n/10 , \T1/ptm/m/it/10 q_lr=0.001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 batch_size =100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=10000\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5107--5107 \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_ever y=50\T1/ptm/m/n/10 , \T1/ptm/m/it/10 act_noise=0.1\T1/ptm/m/n/10 , \T1/ptm/m/it /10 num_test_episodes=10\T1/ptm/m/n/10 , [105] [106] [107] [108] Chapter 18. [109] [110] [111] [112] Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 batch_size=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=1000 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/ it/10 up-date_every=50\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5523--5523 \T1/ptm/m/it/10 act_noise=0.1\T1/ptm/m/n/10 , \T1/ptm/m/it/10 tar-get_noise=0.2 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 noise_clip=0.5\T1/ptm/m/n/10 , \T1/ptm/m/it/10 pol-icy_delay=2\T1/ptm/m/n/10 , [113] [114] Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 5707--5707 \T1/ptm/m/it/10 pol-icy_delay=2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_test_episod es=10\T1/ptm/m/n/10 , \T1/ptm/m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 log-ger_kwargs={}\T1/ptm/m/n/10 , [115] [116] [117] [118] Chapter 19. [119] [120] [121] [122] [123] Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/ 10 , \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epoch s=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 re- Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 gamma=0.99\T1 /ptm/m/n/10 , \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lr=0 .001\T1/ptm/m/n/10 , \T1/ptm/m/it/10 al-pha=0.2\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6232--6232 \T1/ptm/m/it/10 batch_size=100\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=1000 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/ it/10 up-date_every=50\T1/ptm/m/n/10 , [124] [125] [126] Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 []\T1/ptm/m/it/10 env_fn\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac-tor_critic=\T1/ptm/m/n/10 , \T1/ptm/m/it/10 ac_kwargs={}\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 steps_per_epoch=4000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 epochs=100 \T1/ptm/m/n/10 , \T1/ptm/m/it/10 re-play_size=1000000\T1/ptm/m/n/10 , \T1/ptm/m /it/10 gamma=0.99\T1/ptm/m/n/10 , Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 polyak=0.995\T1/ptm/m/n/10 , \T1/ptm/m/it/10 lr=0.001\T1/ptm/m/ n/10 , \T1/ptm/m/it/10 al-pha=0.2\T1/ptm/m/n/10 , \T1/ptm/m/it/10 batch_size=10 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 start_steps=10000\T1/ptm/m/n/10 , \T1/ptm/m/i t/10 up- Underfull \hbox (badness 10000) in paragraph at lines 6426--6426 \T1/ptm/m/it/10 date_after=1000\T1/ptm/m/n/10 , \T1/ptm/m/it/10 up-date_every=5 0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_test_episodes=10\T1/ptm/m/n/10 , \T1/ptm/ m/it/10 max_ep_len=1000\T1/ptm/m/n/10 , [127] [128] [129] [130] Chapter 20. [131] [132] [133] [134] [135] [136] [137] [138] Chapter 21. [139] [140] Chapter 22. [141] [142] [143] [144] Chapter 23. [145] Underfull \hbox (badness 10000) in paragraph at lines 7540--7540 []\T1/ptm/m/it/10 exp_name\T1/ptm/m/n/10 , \T1/ptm/m/it/10 thunk\T1/ptm/m/n/10 , \T1/ptm/m/it/10 seed=0\T1/ptm/m/n/10 , \T1/ptm/m/it/10 num_cpu=1\T1/ptm/m/n/1 0 , [146] [147] [148] Chapter 24. [149] [150] Chapter 25. [151] [152] Chapter 26. [153] [154] [155] (./SpinningUp.ind [156] Underfull \hbox (badness 7522) in paragraph at lines 47--48 []\T1/ptm/m/n/10 add() (spinup.utils.run_utils.ExperimentGrid method), Overfull \hbox (5.61969pt too wide) in paragraph at lines 48--49 []\T1/ptm/m/n/10 apply_gradients() (spinup.utils.mpi_tf.MpiAdamOptimizer Overfull \hbox (17.83952pt too wide) in paragraph at lines 74--75 []\T1/ptm/m/n/10 compute_gradients() (spinup.utils.mpi_tf.MpiAdamOptimizer [157] Underfull \hbox (badness 10000) in paragraph at lines 105--106 []\T1/ptm/m/n/10 mpi_statistics_scalar() (in mod-ule Underfull \hbox (badness 10000) in paragraph at lines 122--123 []\T1/ptm/m/n/10 run() (spinup.utils.run_utils.ExperimentGrid method), Underfull \hbox (badness 10000) in paragraph at lines 131--132 []\T1/ptm/m/n/10 setup_pytorch_for_mpi() (in mod-ule Underfull \hbox (badness 10000) in paragraph at lines 132--133 []\T1/ptm/m/n/10 setup_pytorch_saver() (spinup.utils.logx.Logger Underfull \hbox (badness 10000) in paragraph at lines 149--150 []\T1/ptm/m/n/10 variant_name() (spinup.utils.run_utils.ExperimentGrid Underfull \hbox (badness 10000) in paragraph at lines 150--151 []\T1/ptm/m/n/10 variants() (spinup.utils.run_utils.ExperimentGrid [158]) (./SpinningUp.aux) Package rerunfilecheck Warning: File `SpinningUp.out' has changed. (rerunfilecheck) Rerun to get outlines right (rerunfilecheck) or use package `bookmark'. LaTeX Warning: There were multiply-defined labels. ) (see the transcript file for additional information){/usr/share/texlive/texmf-d ist/fonts/enc/dvips/base/8r.enc}< /usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr5.pfb> Output written on SpinningUp.pdf (164 pages, 1209709 bytes). Transcript written on SpinningUp.log. [rtd-command-info] start-time: 2020-02-02T18:14:39.938160Z, end-time: 2020-02-02T18:14:40.159441Z, duration: 0, exit-code: 0 mv -f /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/_build/latex/SpinningUp.pdf /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/artifacts/latest/sphinx_pdf/openai-education-spinningup.pdf [rtd-command-info] start-time: 2020-02-02T18:14:40.221376Z, end-time: 2020-02-02T18:16:10.305827Z, duration: 90, exit-code: 0 python sphinx-build -T -b epub -d _build/doctrees-epub -D language=en . _build/epub Running Sphinx v1.5.6 making output directory... loading translations [en]... done loading pickled environment... not yet created building [mo]: targets for 0 po files that are out of date building [epub]: targets for 37 source files that are out of date updating environment: 37 added, 0 changed, 0 removed reading sources... [ 2%] algorithms/ddpg reading sources... [ 5%] algorithms/ppo reading sources... [ 8%] algorithms/sac reading sources... [ 10%] algorithms/td3 reading sources... [ 13%] algorithms/trpo reading sources... [ 16%] algorithms/vpg reading sources... [ 18%] etc/acknowledgements reading sources... [ 21%] etc/author reading sources... [ 24%] index reading sources... [ 27%] spinningup/bench reading sources... [ 29%] spinningup/bench_ddpg reading sources... [ 32%] spinningup/bench_ppo reading sources... [ 35%] spinningup/bench_sac reading sources... [ 37%] spinningup/bench_td3 reading sources... [ 40%] spinningup/bench_vpg reading sources... [ 43%] spinningup/exercise2_1_soln reading sources... [ 45%] spinningup/exercise2_2_soln reading sources... [ 48%] spinningup/exercises reading sources... [ 51%] spinningup/extra_pg_proof1 reading sources... [ 54%] spinningup/extra_pg_proof2 reading sources... [ 56%] spinningup/extra_tf_pg_implementation reading sources... [ 59%] spinningup/keypapers reading sources... [ 62%] spinningup/rl_intro reading sources... [ 64%] spinningup/rl_intro2 reading sources... [ 67%] spinningup/rl_intro3 reading sources... [ 70%] spinningup/rl_intro4 reading sources... [ 72%] spinningup/spinningup reading sources... [ 75%] user/algorithms reading sources... [ 78%] user/installation reading sources... [ 81%] user/introduction reading sources... [ 83%] user/plotting reading sources... [ 86%] user/running reading sources... [ 89%] user/saving_and_loading reading sources... [ 91%] utils/logger reading sources... [ 94%] utils/mpi reading sources... [ 97%] utils/plotter reading sources... [100%] utils/run_utils /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercises.rst:3: WARNING: Duplicate explicit target name: "solution available here.". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:343: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "on github". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:3: WARNING: Duplicate explicit target name: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:419: WARNING: Line block ends without a blank line. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro3.rst:341: ERROR: Duplicate target name, cannot be used as a unique reference: "here". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:55: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/plotting.rst:63: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:163: WARNING: Duplicate explicit target name: "cmdoption--ac_kwargs". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/running.rst:306: WARNING: Inline strong start-string without end-string. /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:148: WARNING: Duplicate explicit target name: "cmdoption-arg-default". /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/user/saving_and_loading.rst:173: WARNING: Duplicate explicit target name: "cmdoption-arg-default". looking for now-outdated files... none found pickling environment... done checking consistency... /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ddpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_ppo.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_sac.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_td3.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/bench_vpg.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_1_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/exercise2_2_soln.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof1.rst:: WARNING: document isn't included in any toctree done /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_pg_proof2.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/extra_tf_pg_implementation.rst:: WARNING: document isn't included in any toctree /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/spinningup/rl_intro4.rst:: WARNING: document isn't included in any toctree preparing documents... done writing output... [ 2%] algorithms/ddpg writing output... [ 5%] algorithms/ppo writing output... [ 8%] algorithms/sac writing output... [ 10%] algorithms/td3 writing output... [ 13%] algorithms/trpo writing output... [ 16%] algorithms/vpg writing output... [ 18%] etc/acknowledgements writing output... [ 21%] etc/author writing output... [ 24%] index writing output... [ 27%] spinningup/bench writing output... [ 29%] spinningup/bench_ddpg writing output... [ 32%] spinningup/bench_ppo writing output... [ 35%] spinningup/bench_sac writing output... [ 37%] spinningup/bench_td3 writing output... [ 40%] spinningup/bench_vpg writing output... [ 43%] spinningup/exercise2_1_soln writing output... [ 45%] spinningup/exercise2_2_soln writing output... [ 48%] spinningup/exercises writing output... [ 51%] spinningup/extra_pg_proof1 writing output... [ 54%] spinningup/extra_pg_proof2 writing output... [ 56%] spinningup/extra_tf_pg_implementation writing output... [ 59%] spinningup/keypapers writing output... [ 62%] spinningup/rl_intro writing output... [ 64%] spinningup/rl_intro2 writing output... [ 67%] spinningup/rl_intro3 writing output... [ 70%] spinningup/rl_intro4 writing output... [ 72%] spinningup/spinningup writing output... [ 75%] user/algorithms writing output... [ 78%] user/installation writing output... [ 81%] user/introduction writing output... [ 83%] user/plotting writing output... [ 86%] user/running writing output... [ 89%] user/saving_and_loading writing output... [ 91%] utils/logger writing output... [ 94%] utils/mpi writing output... [ 97%] utils/plotter writing output... [100%] utils/run_utils generating indices... genindex py-modindex writing additional pages... copying images... [ 2%] images/spinning-up-in-rl.png copying images... [ 4%] spinningup/../images/plots/pyt/pytorch_halfcheetah_performance.svg copying images... [ 7%] spinningup/../images/plots/tf1/tensorflow_halfcheetah_performance.svg copying images... [ 9%] spinningup/../images/plots/pyt/pytorch_hopper_performance.svg copying images... [ 12%] spinningup/../images/plots/tf1/tensorflow_hopper_performance.svg copying images... [ 14%] spinningup/../images/plots/pyt/pytorch_walker2d_performance.svg copying images... [ 17%] spinningup/../images/plots/tf1/tensorflow_walker2d_performance.svg copying images... [ 19%] spinningup/../images/plots/pyt/pytorch_swimmer_performance.svg copying images... [ 21%] spinningup/../images/plots/tf1/tensorflow_swimmer_performance.svg copying images... [ 24%] spinningup/../images/plots/pyt/pytorch_ant_performance.svg copying images... [ 26%] spinningup/../images/plots/tf1/tensorflow_ant_performance.svg copying images... [ 29%] spinningup/../images/plots/ddpg/ddpg_halfcheetah_performance.svg copying images... [ 31%] spinningup/../images/plots/ddpg/ddpg_hopper_performance.svg copying images... [ 34%] spinningup/../images/plots/ddpg/ddpg_walker2d_performance.svg copying images... [ 36%] spinningup/../images/plots/ddpg/ddpg_swimmer_performance.svg copying images... [ 39%] spinningup/../images/plots/ddpg/ddpg_ant_performance.svg copying images... [ 41%] spinningup/../images/plots/ppo/ppo_halfcheetah_performance.svg copying images... [ 43%] spinningup/../images/plots/ppo/ppo_hopper_performance.svg copying images... [ 46%] spinningup/../images/plots/ppo/ppo_walker2d_performance.svg copying images... [ 48%] spinningup/../images/plots/ppo/ppo_swimmer_performance.svg copying images... [ 51%] spinningup/../images/plots/ppo/ppo_ant_performance.svg copying images... [ 53%] spinningup/../images/plots/sac/sac_halfcheetah_performance.svg copying images... [ 56%] spinningup/../images/plots/sac/sac_hopper_performance.svg copying images... [ 58%] spinningup/../images/plots/sac/sac_walker2d_performance.svg copying images... [ 60%] spinningup/../images/plots/sac/sac_swimmer_performance.svg copying images... [ 63%] spinningup/../images/plots/sac/sac_ant_performance.svg copying images... [ 65%] spinningup/../images/plots/td3/td3_halfcheetah_performance.svg copying images... [ 68%] spinningup/../images/plots/td3/td3_hopper_performance.svg copying images... [ 70%] spinningup/../images/plots/td3/td3_walker2d_performance.svg copying images... [ 73%] spinningup/../images/plots/td3/td3_swimmer_performance.svg copying images... [ 75%] spinningup/../images/plots/td3/td3_ant_performance.svg copying images... [ 78%] spinningup/../images/plots/vpg/vpg_halfcheetah_performance.svg copying images... [ 80%] spinningup/../images/plots/vpg/vpg_hopper_performance.svg copying images... [ 82%] spinningup/../images/plots/vpg/vpg_walker2d_performance.svg copying images... [ 85%] spinningup/../images/plots/vpg/vpg_swimmer_performance.svg copying images... [ 87%] spinningup/../images/plots/vpg/vpg_ant_performance.svg copying images... [ 90%] spinningup/../images/ex2-1_trpo_hopper.png copying images... [ 92%] spinningup/../images/ex2-2_ddpg_bug.svg copying images... [ 95%] spinningup/../images/ex2-2_ddpg_bug_pytorch.png copying images... [ 97%] spinningup/../images/rl_diagram_transparent_bg.png copying images... [100%] spinningup/../images/rl_algorithms_9_15.svg copying static files... WARNING: favicon file 'openai_icon.ico' does not exist done copying extra files... done writing mimetype file... writing META-INF/container.xml file... writing content.opf file... WARNING: unknown mimetype for _static/openai-favicon2_32x32.ico, ignoring WARNING: unknown mimetype for _static/openai_icon.ico, ignoring writing nav.xhtml file... writing toc.ncx file... writing SpinningUp.epub file... build succeeded, 27 warnings. [rtd-command-info] start-time: 2020-02-02T18:16:10.399456Z, end-time: 2020-02-02T18:16:10.615698Z, duration: 0, exit-code: 0 mv -f /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/checkouts/latest/docs/_build/epub/SpinningUp.epub /home/docs/checkouts/readthedocs.org/user_builds/openai-education-spinningup/artifacts/latest/sphinx_epub/openai-education-spinningup.epub