A Python library to create, parse, modify, and format MapServer Mapfiles.
Pure Python - no MapServer dependencies
Open Source License (MIT)
An official OSGeo Community Project
An online formatter demonstrating the libraries capabilities can be found at: https://app.mapserverstudio.net/
mappyfile is available on PyPI (the Python Package Index), and can be installed using pip. It requires Python 3.8 or higher.
pip install mappyfile
To install the optional lark-cython library for better performance on CPython you can run the following command:
pip install mappyfile[lark_cython]
If lark-cython is installed it will be used automatically. If it is installed and you want to disable its
use in mappyfile you can set the
False (or any falsy value).
# Linux export MAPPYFILE_USE_CYTHON="False" # Windows PowerShell $env:MAPPYFILE_USE_CYTHON="False" # Windows Command Line set MAPPYFILE_USE_CYTHON=False
mappyfile is also available on conda. Install as follows:
conda install -c conda-forge mappyfile
To setup a Conda environment containing mappyfile from scratch on Windows download Miniconda3 Windows 64-bit and run the following commands:
conda create --name mappyfile-env conda activate mappyfile-env conda config --add channels conda-forge conda config --set channel_priority strict conda install mappyfile --yes mappyfile --help
This section details the basic use of the
mappyfile library. To see all functionality and examples see the mappyfile API documentation.
import mappyfile # open will accept a filename (mappyfile.loads will accept a string) mapfile = mappyfile.open("./docs/examples/raster.map") # print the map name print(mapfile["name"]) # "MyMap" # access layers layers = mapfile["layers"] layer2 = layers # access by index # access classes in a layer classes = layer2["classes"] for c in classes: print(c["name"])
# load will accept a filename (loads will accept a string) mapfile = mappyfile.open("./docs/examples/raster.map") # search for a layer by name layer = mappyfile.find(mapfile["layers"], "name", "sea") print(layer["name"]) # "sea" # search for all layers in a group for layer in mappyfile.findall(mapfile["layers"], "group", "my_group"): print(layer["name"])
# update the map name mapfile["name"] = "MyNewMap" # update a layer name layers = mapfile["layers"] layer = layers layer["name"] = "MyLayer" # update the error file path in the map config section # note key names can be lower or upper case mapfile["config"]["ms_errorfile"] = "/ms4w/tmp/ms_error.txt" # update the web metadata settings mapfile["web"]["metadata"]["wms_format"] = "image/png" print(mappyfile.dumps(mapfile["web"])) # print out just the WEB section # alternatively we can parse the Mapfile syntax and load it directly s = """ METADATA 'wms_enable_request' '*' 'wms_feature_info_mime_type' 'text/html' 'wms_format' 'image/jpg' END""" metadata = mappyfile.loads(s) mapfile["web"]["metadata"] = metadata print(mappyfile.dumps(mapfile))
Adding a new layer:
layers = mapfile["layers"] new_layer_string = """ LAYER NAME 'land' TYPE POLYGON DATA '../data/vector/naturalearth/ne_110m_land' CLASS STYLE COLOR 107 208 107 OUTLINECOLOR 2 2 2 WIDTH 1 END END END """ new_layer = mappyfile.loads(new_layer_string) layers.insert(0, new_layer) # can insert the new layer at any index
Adding a new class to a layer:
# find a layer using its name layer = mappyfile.find(mapfile["layers"], "name", "sea") new_class_string = """ CLASS NAME 'highlights' STYLE COLOR 107 208 107 OUTLINECOLOR 2 2 2 WIDTH 1 END END """ new_class = mappyfile.loads(new_class_string) layer["classes"].insert(1, new_class) # can insert the new class at any index print(mappyfile.dumps(mapfile))
Three command line tools are also available -
mappyfile format raster.map formatted_raster.map mappyfile validate D:\ms-ogc-workshop\ms4w\apps\ms-ogc-workshop\**\*.map mappyfile schema mapfile-schema-8-0.json --version=8.0
What is mappyfile?¶
mappyfile takes a Mapfile as input and parses it into an Abstract syntax tree (AST) using Lark a Python parsing library. mappyfile can then transform the AST into a dictionary structure, containing keys and values of dicts, and lists familiar to Python programmers. This structure can be edited directly. Alternatively new objects can be added by parsing further Mapfile text and inserting into the dictionary structure. mappyfile also includes a “pretty printer” to export this dictionary structure back to a Mapfile, with keyword formatting and indentation.
mappyfile assumes knowledge of the Mapfile format - a domain specific language (DSL) used by MapServer to generate map images. mappyfile is a possible alternative to using MapScript. The definitions of these (from the MapServer glossary) are shown below:
Mapfile is the declarative language that MapServer uses to define data connections, map styling, templating, and server directives. Its format is XML-like and hierarchical, with closing END tags, but the format is not XML.
MapScript is an alternative to the CGI application of mapserv that allows you to program the MapServer object API in many languages.
The diagram below shows the different elements of mappyfile, and how they are used to modify a Mapfile:
Easily generate development, staging, and production Mapfiles from the same source Mapfile
Create Mapfiles for different datasets from a single Mapfile
Create, manipulate, and test Mapfiles from within Python
The current alternative to building applications with MapServer is to use MapScript. This approach has a number of issues that resulted in the development of mappyfile:
MapScript needs to be compiled using SWIG and a C/C++ compiler
You need to create an empty log file or MapServer won’t open the map (or get
msSetErrorFile(): General error message. Failed to open MS_ERRORFILEerrors)
MapScript is only available for Windows on PyPI
It is necessary to set the working directory so that MapServer includes are found (this also applies to mappyfile, but there is no need to os.chdir and change the working directory for your script or application)
The MapScript API is not particularly “Pythonic”
One key difference is that mappyfile only deals with text, so you cannot retrieve features or connect to databases through layers as you can with MapScript. mappyfile’s approach is to build a Mapfile that then uses the mapserv program to handle these requirements. This design was influenced by Sean Gillies, the MapScript maintainer for several years (until 2006). A couple of his last blog posts on MapScript make a strong case for working with Mapfiles rather than MapScript:
“Cease, or at the very least, minimize your use of MapServer’s various language bindings. Instead, embrace MapServer’s domain-specific language (DSL) and write more of the declarative cartographic scripts known as mapfiles. Use the mapserv (or shp2img) program to compile these scripts into images. This is the path to happiness and prosperity.”
Sean Gillies - Stop using MapScript
A later post listed the benefits of this approach:
“the instructions encoded in a MapServer mapfile comprise a domain-specific language.. to embrace the map language is to benefit from simplicity, usability, and portability.”
Sean Gillies - Declarative Maps
The concept of the Mapfile as a DSL has been implemented a few times. A Python Mapfile builder written by Allan Doyle used an XML approach.
More recently the Node module node-mapserv provides support for declarative mapfile programming. As the author notes:
node-mapserv is not MapScript for Node. Instead it provides a simple declarative API for rendering mapserver mapfiles..most of what can be accomplished imperatively using mapscript can be done declaratively by custom generating new mapfiles and tweaking existing mapfiles
As an interesting footnote the MapScript “bindings” are available in several different languages thanks to SWIG which creates wrapper code for C. SWIG was developed by David Beazley, who then later built PLY on which mappyfile was originally based. PLY is an implementation of lex and yacc parsing tools for Python - the tools MapServer itself uses to parse Mapfiles in C.
Future development plans include:
Setup an easy way to plug in “linters” to check various Mapfile settings and rules (e.g. configured correctly for WFS)
Create a Jupyter Notebook demonstrating mappyfile usage
Add a plugins page to the docs
Add an example of creating Mapfiles using YAML
Create a new
prunefunction to remove redundant default settings from a Mapfile