.. _testing:

Testing mappyfile
=================

There are two main objectives to testing mappyfile. 

#. Check that mappyfile can parse all the test Mapfiles in the test suite. 
#. Check that mappyfile can parse the Mapfile, pretty print it to a new file, run the tests on the new file, and check that the output is the same
   as the original. 

The Mapfiles won't be identical as comments will be stripped, and whitespace will change. They should however produce the same results. 
MapServer has a full set of regression tests - `msautotest <https://mapserver.org/development/tests/autotest.html>`_
that includes 100s of Mapfiles. For example:

+ https://github.com/mapserver/mapserver/tree/main/msautotest/misc
+ https://github.com/mapserver/mapserver/tree/main/msautotest/wxs
+ https://github.com/mapserver/mapserver/tree/main/msautotest/renderers
+ https://github.com/mapserver/mapserver/tree/main/msautotest/gdal

These have been downloaded and added to the ``/tests/sample_maps`` folder and are tested as part of the automated pytest test suite. 
Includes are not currently included - extension names of include files differ - sometimes ``.map`` and sometimes ``.include``. In addition they are sometimes
stored in an ``includes`` folder, and sometimes in a ``data`` folder. These includes are currently added to an ``ignore_list`` in the
`test_sample_maps.py <https://github.com/geographika/mappyfile/blob/master/tests/test_sample_maps.py>`_ test file.

A second test file `test_msautotest.py <https://github.com/geographika/mappyfile/blob/master/tests/test_msautotest.py>`_ can be pointed to
a clone of the MapServer repository, and parses them, writes them to a new file, and then parses the new output. This is a good check that
the mappyfile pretty-print output remains valid. 

Running the Test Suite
----------------------

The test suite can be run to check that the mappyfile outputs not only pass the mappyfile parsing and validation steps, but also
pass being loaded by MapServer and run as part of msautotest. The steps below can be run on Windows using PowerShell.
Note however that due to image output differences between Windows and the expected reference images generated by Linux there will be lots
of failures.

.. code-block:: ps1

    $MSAUTOTEST_ROOT="D:\GitHub\mapserver\msautotest"
    C:\Python310\Scripts\virtualenv C:\VirtualEnvs\msautotest
    C:\VirtualEnvs\msautotest\Scripts\activate.ps1
    pip install lxml
    pip install future
    pip install pytest

    $env:PATH="C:\MapServer\bin;$env:PATH"
    $env:MAPSERVER_CONFIG_FILE="C:\MapServer\apps\mapserver.conf"

    cd $MSAUTOTEST_ROOT
    python python pymod/xmlvalidate.py -download_ogc_schemas

    cd $MSAUTOTEST_ROOT\gdal
    new-item -itemtype symboliclink -path . -name SCHEMAS_OPENGIS_NET -value $MSAUTOTEST_ROOT\SCHEMAS_OPENGIS_NET
    # can add -v for verbose output to the command below
    python.exe run_test.py

    cd $MSAUTOTEST_ROOT\misc
    new-item -itemtype symboliclink -path . -name SCHEMAS_OPENGIS_NET -value $MSAUTOTEST_ROOT\SCHEMAS_OPENGIS_NET
    python.exe run_test.py

    # now run on mappyfile generated Mapfiles - error count should match
    # switch to the copied output folder defined in test_msautotest.py
    cd D:\GitHub\mapserver\msautotest_mappyfile\misc
    new-item -itemtype symboliclink -path . -name SCHEMAS_OPENGIS_NET -value $MSAUTOTEST_ROOT\SCHEMAS_OPENGIS_NET
    python.exe run_test.py