:orphan:

.. _examples:

Examples
========

This section provides examples of how to build different learning-based MPC agents via
:mod:`mpcrl`.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. thumbnail-parent-div-close

.. raw:: html

    </div>

Gradient-based off-policy learning agents
-----------------------------------------

The following example showcases how to use gradient-based Reinforcement Learning
techniques (in particular, Q-learning) to train a Model Predictive Controller (MPC)
scheme for a simple task in an off-policy way.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example tries to reproduce the results from the linear MPC numerical experiment in gros_datadriven_2020, but in an off-polic setting. We are given an RL environment whose cost function is">

.. only:: html

  .. image:: /auto_examples/gradient-based-offpolicy/images/thumb/sphx_glr_q_learning_offpolicy_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_gradient-based-offpolicy_q_learning_offpolicy.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Off-policy Q-learning</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>

Gradient-based on-policy learning agents
----------------------------------------

The following examples showcase how to use gradient-based Reinforcement Learning
techniques (in particular, Q-learning and Deterministic Policy Gradient) to train a
Model Predictive Controller (MPC) scheme for a simple task in an on-policy fashion.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example tries to reproduce the results from the linear MPC numerical experiment in gros_datadriven_2020. We are given an RL environment whose cost function is">

.. only:: html

  .. image:: /auto_examples/gradient-based-onpolicy/images/thumb/sphx_glr_q_learning_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_gradient-based-onpolicy_q_learning.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">On-policy Q-learning</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example takes inspiration from the linear MPC numerical experiment in gros_datadriven_2020, but applies DPG to it instead of Q-learning. We are given an RL environment whose cost function is">

.. only:: html

  .. image:: /auto_examples/gradient-based-onpolicy/images/thumb/sphx_glr_dpg_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_gradient-based-onpolicy_dpg.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">On-policy Deterministic Policy Gradient</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>

Gradient-free learning agents
-----------------------------

The example below demonstrates how to use gradient-free methodologies to find the
optimal parametrization of an MPC controller.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example, instead of using RL to tune the parametrization \theta of an MPC controller, we will use Bayesian optimization (BO). The task can be summarized as">

.. only:: html

  .. image:: /auto_examples/gradient-free/images/thumb/sphx_glr_bayesopt_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_gradient-free_bayesopt.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Bayesian Optimization for MPC Data-driven Tuning</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>

Other examples
--------------

Here we include other examples that are not (strictly) related to MPC and RL.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example demonstrates how to use the mpcrl.util.geometry.ConvexPolytopeUniformSampler to sample uniformly from the interior and surface of a convex polytope in a N-dimensional space.">

.. only:: html

  .. image:: /auto_examples/others/images/thumb/sphx_glr_sampling_from_polytopes_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_others_sampling_from_polytopes.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Sampling from a convex polytopes</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example demonstrates the use of Input Constrained Control Barrier Functions (ICCBFs) in the context of the Adaptive Cruise Control (ACC) problem, taken from [1]_. The ACC problem is a classic control problem where a vehicle is tasked with maintaining a desired velocity while keeping a safe distance from the vehicle in front. This safety constraint is enforced using a ICCBF, which is a class of CBFs that can handle the case where the control input is constrained in its norm.">

.. only:: html

  .. image:: /auto_examples/others/images/thumb/sphx_glr_acc_with_iccbf_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_others_acc_with_iccbf.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Adapative Cruise Control with Input Constrained Control Barrier Function</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:
   :includehidden:


   /auto_examples/gradient-based-offpolicy/index.rst
   /auto_examples/gradient-based-onpolicy/index.rst
   /auto_examples/gradient-free/index.rst
   /auto_examples/others/index.rst


.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-gallery

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download all examples in Python source code: auto_examples_python.zip </auto_examples/auto_examples_python.zip>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download all examples in Jupyter notebooks: auto_examples_jupyter.zip </auto_examples/auto_examples_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_