..
      Copyright 2015 Mirantis Inc. All Rights Reserved.

      Licensed under the Apache License, Version 2.0 (the "License"); you may
      not use this file except in compliance with the License. You may obtain
      a copy of the License at

          http://www.apache.org/licenses/LICENSE-2.0

      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
      License for the specific language governing permissions and limitations
      under the License.

.. _tutorial_step_1_setting_up_env_and_running_benchmark_from_samples:

Step 1. Setting up the environment and running a task from samples
==================================================================

.. contents::
   :local:

In this demo basic operations in Rally are performed, such as adding
OpenStack cloud deployment, running task against it and generating report.

It's assumed that you have gone through :ref:`install` and
have an already existing OpenStack deployment with Keystone available at
*<KEYSTONE_AUTH_URL>*.


Installing rally-openstack package
----------------------------------
First, you have to provider Rally with `rally-openstack
<https://github.com/openstack/rally-openstack>`_ package, to be done
with ``pip install rally-openstack`` command.


Registering an OpenStack environment in Rally
---------------------------------------------

After successful installation, you have to provide Rally with an OpenStack
cloud that should be tested. Such cloud is described by an *environment spec*
- a mapping where each key is a name of a platform plugin and each value is a
configuration of that platform. The plugin that knows how to talk to an
already deployed OpenStack cloud is `existing@openstack
<../../plugins/plugin_reference.html#existing-platform>`_.

In case you already have an `OpenRC file
<https://docs.openstack.org/keystone/latest/admin/openrc.html>`_, there is no
need to write the spec at all. The *env create* command with ``--from-sysenv``
argument iterates over all available platform plugins and builds the spec
based on the environment variables that are exported by *OpenRC*:

.. code-block:: console

   $ . openrc admin admin
   $ rally env create --name=existing --from-sysenv
   Your system environment includes specifications of 1 platform(s).
   Discovery information:
        - existing@openstack : Available.
   Using environment: 4ea58e84-7d05-43da-b7e3-550b666f6b5e
   +---------------------+------------------------------------------------+
   | uuid                | 4ea58e84-7d05-43da-b7e3-550b666f6b5e           |
   | name                | existing                                       |
   | status              | READY                                          |
   | created_at          | 2025-01-18T00:11:38.059983                     |
   | updated_at          | 2025-01-18T00:11:38.071983                     |
   | description         |                                                |
   | extras              | {}                                             |
   | platform: openstack | {                                              |
   |                     |   "admin": {                                   |
   |                     |     "username": "admin",                       |
   |                     |     "password": "myadminpass",                 |
   |                     |     "user_domain_name": "Default",             |
   |                     |     "project_domain_name": "Default",          |
   |                     |     "tenant_name": "admin",                    |
   |                     |     "auth_url": "http://example.net:5000/v3/", |
   |                     |     "endpoint_type": "public",                 |
   |                     |     "region_name": "RegionOne",                |
   |                     |     "https_cacert": "",                        |
   |                     |     "https_cert": "",                          |
   |                     |     "https_key": "",                           |
   |                     |     "https_insecure": false,                   |
   |                     |     "profiler_hmac_key": null,                 |
   |                     |     "profiler_conn_str": null,                 |
   |                     |     "domain_name": null                        |
   |                     |   },                                           |
   |                     |   "users": [],                                 |
   |                     |   "api_info": {                                |
   |                     |     "keystone": {                              |
   |                     |       "version": 3,                            |
   |                     |       "service_type": "identityv3"             |
   |                     |     }                                          |
   |                     |   }                                            |
   |                     | }                                              |
   +---------------------+------------------------------------------------+

Alternatively, you can write the spec yourself and store it in a JSON or YAML
file (let's call it *existing.json*). See `more samples
<https://github.com/openstack/rally-openstack/tree/master/samples/deployments>`_
for other ways to describe the same cloud:

.. code-block:: json

    {
        "existing@openstack": {
            "auth_url": "http://example.net:5000/v3/",
            "region_name": "RegionOne",
            "endpoint_type": "public",
            "admin": {
                "username": "admin",
                "password": "myadminpass",
                "user_domain_name": "Default",
                "project_name": "admin",
                "project_domain_name": "Default"
            },
            "https_insecure": false,
            "https_cacert": ""
        }
    }

The *env create* command accepts such file via ``--spec`` argument:

.. code-block:: console

   $ rally env create --name=existing --spec existing.json
   Using environment: 87c1dada-de7b-4627-aa12-fb5f127da9fa
   +---------------------+------------------------------------------------+
   | uuid                | 87c1dada-de7b-4627-aa12-fb5f127da9fa           |
   | name                | existing                                       |
   | status              | READY                                          |
   | created_at          | 2025-01-18T00:11:38.059983                     |
   | updated_at          | 2025-01-18T00:11:38.071983                     |
   | description         |                                                |
   | extras              | {}                                             |
   | platform: openstack | {                                              |
   |                     |   "admin": {                                   |
   |                     |     "username": "admin",                       |
   |                     |     "password": "myadminpass",                 |
   |                     |     "user_domain_name": "Default",             |
   |                     |     "project_domain_name": "Default",          |
   |                     |     "tenant_name": "admin",                    |
   |                     |     "auth_url": "http://example.net:5000/v3/", |
   |                     |     "region_name": "RegionOne",                |
   |                     |     "endpoint_type": "public",                 |
   |                     |     "https_insecure": false,                   |
   |                     |     "https_cacert": "",                        |
   |                     |     "domain_name": null                        |
   |                     |   },                                           |
   |                     |   "users": []                                  |
   |                     | }                                              |
   +---------------------+------------------------------------------------+

Note the *"Using environment"* line in the output. It says that the just
created environment is now the default one; that means that all task or verify
commands are going to be run against it. Later in tutorial is described how to
use multiple environments.


Specifying versions of OpenStack APIs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

By default Rally discovers what version of each OpenStack API to use based on
the Keystone service catalog. The optional ``api_info`` subkey of the spec
allows to pin the version and the service type explicitly. This is what
``--from-sysenv`` did for *keystone* in the example above, and the same can be
done for any other client:

.. code-block:: json

    {
        "existing@openstack": {
            "auth_url": "http://example.net:5000/v3/",
            "admin": {
                "username": "admin",
                "password": "myadminpass",
                "user_domain_name": "Default",
                "project_name": "admin",
                "project_domain_name": "Default"
            },
            "api_info": {
                "cinder": {
                    "version": 3,
                    "service_type": "volumev3"
                }
            }
        }
    }

Whatever is listed in ``api_info`` applies to the whole environment. If
different subtasks of a single task should be executed against different API
versions, use `api_versions@openstack
<../../plugins/plugin_reference.html#api-versions-context>`_ context instead -
it accepts the same options, but is set per workload.


Checking the environment
~~~~~~~~~~~~~~~~~~~~~~~~

Finally, the *env check* command enables you to verify that your current
environment is healthy and ready to be tested:

.. code-block:: console

   $ rally env check
   Env `existing (87c1dada-de7b-4627-aa12-fb5f127da9fa)' :-)
   +-----------+-----------+---------+
   | Available | Platform  | Message |
   +-----------+-----------+---------+
   | :-)       | openstack | OK!     |
   +-----------+-----------+---------+

While *env check* only answers whether the cloud is reachable, the *env info*
command asks each platform for the details it can discover. For OpenStack it
is a list of services available in the cloud:

.. code-block:: console

   $ rally env info
   Env `existing (87c1dada-de7b-4627-aa12-fb5f127da9fa)'
   +--------------------+-------------------------------+-------+
   | platform           | info                          | error |
   +--------------------+-------------------------------+-------+
   | existing@openstack | {                             |       |
   |                    |   "services": [               |       |
   |                    |     {                         |       |
   |                    |       "type": "compute",      |       |
   |                    |       "name": "nova"          |       |
   |                    |     },                        |       |
   |                    |     {                         |       |
   |                    |       "type": "identity",     |       |
   |                    |       "name": "keystone"      |       |
   |                    |     },                        |       |
   |                    |     {                         |       |
   |                    |       "type": "image",        |       |
   |                    |       "name": "glance"        |       |
   |                    |     },                        |       |
   |                    |     {                         |       |
   |                    |       "type": "network",      |       |
   |                    |       "name": "neutron"       |       |
   |                    |     },                        |       |
   |                    |     {                         |       |
   |                    |       "type": "object-store", |       |
   |                    |       "name": "swift"         |       |
   |                    |     },                        |       |
   |                    |     {                         |       |
   |                    |       "type": "placement",    |       |
   |                    |       "name": "placement"     |       |
   |                    |     },                        |       |
   |                    |     {                         |       |
   |                    |       "type": "volumev3",     |       |
   |                    |       "name": "cinderv3"      |       |
   |                    |     }                         |       |
   |                    |   ]                           |       |
   |                    | }                             |       |
   +--------------------+-------------------------------+-------+


Running Rally Tasks
-------------------

Now that we have a working and registered environment, we can start testing
it. The sequence of subtask to be launched by Rally should be specified in a
*task input file* (either in *JSON* or in *YAML* format).
Let's try one of the task sample available in
`samples/tasks/scenarios
<https://github.com/openstack/rally/tree/master/samples/tasks/scenarios>`_,
say, the one that boots and deletes multiple servers
(*samples/tasks/scenarios/nova/boot-and-delete.json*):


.. code-block:: json

    {
        "version": 2,
        "title": "Boot and delete servers",
        "subtasks": [
            {
                "title": "Boot and delete a single server",
                "scenario": {
                    "NovaServers.boot_and_delete_server": {
                        "flavor": {
                            "name": "m1.tiny"
                        },
                        "image": {
                            "name": "^cirros.*-disk$"
                        },
                        "force_delete": false
                    }
                },
                "runner": {
                    "constant": {
                        "times": 10,
                        "concurrency": 2
                    }
                },
                "contexts": {
                    "users": {
                        "tenants": 3,
                        "users_per_tenant": 2
                    }
                }
            }
        ]
    }


To start a task, run the ``task start`` command (you can also add the
*-v* option to print more logging information):

.. code-block:: console

    $ rally task start samples/tasks/scenarios/nova/boot-and-delete.json
    --------------------------------------------------------------------------------
     Preparing input task
    --------------------------------------------------------------------------------

    Input task is:
    <Your task config here>

    --------------------------------------------------------------------------------
     Task  6fd9a19f-5cf8-4f76-ab72-2e34bb1d4996: started
    --------------------------------------------------------------------------------

    Running Task... This can take a while...

    To track task status use:

      rally task status
      or
      rally task detailed

    --------------------------------------------------------------------------------
     Task 6fd9a19f-5cf8-4f76-ab72-2e34bb1d4996: finished
    --------------------------------------------------------------------------------

    test scenario NovaServers.boot_and_delete_server
    args position 0
    args values:
    {u'args': {u'flavor': {u'name': u'm1.tiny'},
               u'force_delete': False,
               u'image': {u'name': u'^cirros.*-disk$'}},
     u'context': {u'users': {u'project_domain': u'default',
                             u'resource_management_workers': 30,
                             u'tenants': 3,
                             u'user_domain': u'default',
                             u'users_per_tenant': 2}},
     u'runner': {u'concurrency': 2, u'times': 10, u'type': u'constant'}}
    +--------------------+-----------+-----------+-----------+---------------+---------------+---------+-------+
    | action             | min (sec) | avg (sec) | max (sec) | 90 percentile | 95 percentile | success | count |
    +--------------------+-----------+-----------+-----------+---------------+---------------+---------+-------+
    | nova.boot_server   | 7.99      | 9.047     | 11.862    | 9.747         | 10.805        | 100.0%  | 10    |
    | nova.delete_server | 4.427     | 4.574     | 4.772     | 4.677         | 4.725         | 100.0%  | 10    |
    | total              | 12.556    | 13.621    | 16.37     | 14.252        | 15.311        | 100.0%  | 10    |
    +--------------------+-----------+-----------+-----------+---------------+---------------+---------+-------+
    Load duration: 70.1310448647
    Full duration: 87.545541048


    HINTS:
    * To plot HTML graphics with this data, run:
        rally task report 6fd9a19f-5cf8-4f76-ab72-2e34bb1d4996 --out output.html

    * To generate a JUnit report, run:
        rally task export 6fd9a19f-5cf8-4f76-ab72-2e34bb1d4996 --type junit-xml
         --to output.xml

    * To get raw JSON output of task results, run:
        rally task report 6fd9a19f-5cf8-4f76-ab72-2e34bb1d4996 --json --out output.json

    Using task: 6fd9a19f-5cf8-4f76-ab72-2e34bb1d4996


Note that the Rally input task above uses *regular expressions* to specify the
image and flavor name to be used for server creation, since concrete names
might differ from installation to installation. If this task fails, then the
reason for that might a non-existing image/flavor specified in the task.
To check what images/flavors are available in the environment, you might use
the following commands:

.. code-block:: console

   $ . openrc admin admin
   $ openstack image list
   +--------------------------------------+---------------------------------+--------+
   | ID                                   | Name                            | Status |
   +--------------------------------------+---------------------------------+--------+
   | 30dc3b46-4a4b-4fcc-932c-91fa87753902 | cirros-0.3.4-x86_64-uec         | active |
   | d687fc2a-75bd-4194-90c7-1619af255b04 | cirros-0.3.4-x86_64-uec-kernel  | active |
   | c764d543-027d-47a3-b46e-0c1c8a68635d | cirros-0.3.4-x86_64-uec-ramdisk | active |
   +--------------------------------------+---------------------------------+--------+

   $ openstack flavor list
   +----+-----------+-------+------+-----------+-------+-----------+
   | ID | Name      |   RAM | Disk | Ephemeral | VCPUs | Is Public |
   +----+-----------+-------+------+-----------+-------+-----------+
   | 1  | m1.tiny   |   512 |    1 |         0 |     1 | True      |
   | 2  | m1.small  |  2048 |   20 |         0 |     1 | True      |
   | 3  | m1.medium |  4096 |   40 |         0 |     2 | True      |
   | 4  | m1.large  |  8192 |   80 |         0 |     4 | True      |
   | 42 | m1.nano   |    64 |    0 |         0 |     1 | True      |
   | 5  | m1.xlarge | 16384 |  160 |         0 |     8 | True      |
   | 84 | m1.micro  |   128 |    0 |         0 |     1 | True      |
   +----+-----------+-------+------+-----------+-------+-----------+


Report generation
-----------------

One of the most beautiful things in Rally is its task report generation
mechanism. It enables you to create illustrative and comprehensive HTML reports
based on the task data. To create and open at once such a report for the last
task you have launched, call:

.. code-block:: bash

   rally task report --out=report1.html --open

This is going produce an HTML page with the overview of all the scenarios that
you've included into the last task completed in Rally (in our case, this is
just one scenario, and we will cover the topic of multiple scenarios in
one task in
:ref:`the next step of our tutorial <tutorial_step_2_input_task_format>`):

.. image:: ../../images/Report-Overview.png
   :align: center

This aggregating table shows the duration of the load produced by the
corresponding scenario (*"Load duration"*), the overall subtask execution time,
including the duration of context creation (*"Full duration"*), the number of
iterations of each scenario (*"Iterations"*), the type of the load used while
running the scenario (*"Runner"*), the number of failed iterations (*"Errors"*)
and finally whether the scenario has passed certain Success Criteria (*"SLA"*)
that were set up by the user in the input configuration file (we will cover
these criteria in
:ref:`one of the next steps <tutorial_step_4_adding_success_criteria_for_benchmarks>`).

By navigating in the left panel, you can switch to the detailed view of the
task results for the only scenario we included into our task, namely
**NovaServers.boot_and_delete_server**:

.. image:: ../../images/Report-Scenario-Overview.png
   :align: center

This page, along with the description of the success criteria used to check the
outcome of this scenario, shows more detailed information and statistics about
the duration of its iterations. Now, the *"Total durations"* table splits the
duration of our scenario into the so-called **"atomic actions"**: in our case,
the **"boot_and_delete_server"** scenario consists of two actions -
**"boot_server"** and **"delete_server"**. You can also see how the scenario
duration changed throughout its iterations in the *"Charts for the total
duration"* section. Similar charts, but with atomic actions detailed are on the
*"Details"* tab of this page:

.. image:: ../../images/Report-Scenario-Atomic.png
   :align: center

Note that all the charts on the report pages are very dynamic: you can change
their contents by clicking the switches above the graph and see more
information about its single points by hovering the cursor over these points.

Take some time to play around with these graphs
and then move on to :ref:`the next step of our tutorial <tutorial_step_2_input_task_format>`.
