mappyfile

A Python library to create, parse, and modify MapServer Mapfiles.

  • Python 2 and 3 compatible
  • Pure Python - no MapServer dependencies
_images/class_parsed.png

What is mappyfile?

mappyfile takes a Mapfile as input and parses it into an Abstract syntax tree (AST) using plyplus which in turn is built on PLY. mappyfile can then transform the AST into a dictionary structure, containing keys, values, dicts, and lists familiar to Python programmers. This structure can be edited directly. Alternatively new objects can be added by parsing further Mapfile text and inserting into the dictionary structure. mappyfile also includes a “pretty printer” to export this dictionary structure back to a Mapfile.

mappyfile assumes knowledge of the Mapfile format - a domain specific language (DSL) used by MapServer to generate map images. mappyfile is a possible alternative to using MapScript. The definitions of these from the MapServer glossary are shown below:

Mapfile is the declarative language that MapServer uses to define data connections, map styling, templating, and server directives. Its format is xml-like and hierarchical, with closing END tags, but the format is not xml.

MapScript is an alternative to the CGI application of mapserv that allows you to program the MapServer object API in many languages.

Why?

Some example use cases are:

  • Easily generate development, staging, and production Mapfiles from the same source
  • Create Mapfiles for different datasets from the same source
  • Create, manipulate, and test Mapfiles from within Python

The current alternative to building applications with MapServer is to use MapScript. This approach has a number of issues that resulted in the development of mappyfile:

  • When running on Windows any Python libraries using C/C++ require them to be built with the MS C/C++ VS2008 compiler, this means no applications using MapScript can take advantage of performance improvements in the MS C/C++ 2015 compiler
  • You need to create an empty log file or MapServer won’t open the map (or get “msSetErrorFile(): General error message. Failed to open MS_ERRORFILE” errors)
  • MapScript is not available through PyPI - the last version was uploaded in 2010 - https://pypi.python.org/pypi/mapscript/5.6.3.0
  • It is necessary to set the working directory so that MapServer includes are found (this also applies to mappyfile, but there is no need to os.chdir and change the working directory for your script or application)
  • The MapScript API is not particularly “Pythonic”

One key difference is that mappyfile only deals with text, so you cannot retrieve features or connect to databases through layers as you can with MapScript. mappyfile’s approach is to build a Mapfile that then uses the mapserv program to handle these requirements. This design was influenced by Sean Gillies, the MapScript maintainer for several years (until 2006). A couple of his last blog posts on MapScript make a strong case for working with Mapfiles rather than MapScript:

“Cease, or at the very least, minimize your use of MapServer’s various language bindings. Instead, embrace MapServer’s domain-specific language (DSL) and write more of the declarative cartographic scripts known as mapfiles. Use the mapserv (or shp2img) program to compile these scripts into images. This is the path to happiness and prosperity.”

Sean Gillies - Stop using MapScript

A later post listed the benefits of this approach:

“the instructions encoded in a MapServer mapfile comprise a domain-specific language.. to embrace the map language is to benefit from simplicity, usability, and portability.”

Sean Gillies - Declarative Maps

The concept of the Mapfile as a DSL has been implemented a few times. A Python Mapfile builder written by Norman Vine used an XML approach.

More recently the Node module node-mapserv provides support for declarative mapfile programming. As the author notes:

node-mapserv is not MapScript for Node. Instead it provides a simple declarative API for rendering mapserver mapfiles..most of what can be accomplished imperatively using mapscript can be done declaratively by custom generating new mapfiles and tweaking existing mapfiles

As an interesting footnote the MapScript “bindings” are available in several different languages thanks to SWIG which creates wrapper code for C. SWIG was developed by David Beazley, who then later built PLY on which mappyfile is based. PLY is an implementation of lex and yacc parsing tools for Python - the tools MapServer itself uses to parse Mapfiles in C.

API Examples

This section details the basic use of the mappyfile library.

Accessing Values

import mappyfile

# load will accept a filename (loads will accept a string)
mapfile = mappyfile.load("./docs/examples/raster.map")

# print the map name
print(mapfile["name"]) # outputs "MyMap"
       
# access layers
layers = mapfile["layers"]
layer2 = layers[1] # access by index
	
# access classes in a layer
classes = layer2["classes"]

for c in classes:
    print(c["name"])

Query

import mappyfile

# load will accept a filename (loads will accept a string)
mapfile = mappyfile.load("./docs/examples/raster.map")

# Search of a layer by its name
mappyfile.find(mapfile['layers'], 'name', 'my_layer')
       
# Search for all layers of a group
for layer in mappyfile.findall(mapfile['layers'], 'group', 'my_group'):
    print(layer['name'])

Modifying Values

# update the map name
mapfile["name"] = "MyNewMap"

# update the error file path in the map config section
# note key names will always need to be lower case

mapfile["config"]["ms_errorfile"] = "/ms4w/tmp/ms_error.txt"
mapfile["config"]["ON_MISSING_DATA"] = "IGNORE"

# currently will need to double-quote non-keyword properties
mapfile["web"]["metadata"]["wms_format"] = "'image/png'"

layers = mapfile["layers"]
layer = layers[0]
layer["name"] = "MyLayer"

print(mappyfile.dumps(mapfile))

# alternatively we can use the Mapfile syntax
# not currently working for CONFIG or METADATA

web = """WEB
        METADATA
            'wms_enable_request' '*'
            'wms_feature_info_mime_type' 'text/html'
            'wms_format' 'image/jpg'
        END
    END"""

web = mappyfile.loads(web)

mapfile["web"] = web
print(mappyfile.dumps(mapfile))

Adding Items

Adding a new layer:

layers = mapfile["layers"]

new_layer_string = """
LAYER
    NAME 'land'
    TYPE POLYGON
    DATA '../data/vector/naturalearth/ne_110m_land'
    CLASS
        STYLE
            COLOR 107 208 107
            OUTLINECOLOR 2 2 2
            WIDTH 1
        END
    END
END
"""

new_layer = mappyfile.loads(new_layer_string)
layers.insert(0, new_layer) # can insert the new layer at any index
assert(layers[0]['name'] == 'land')

Adding a new class to a layer:

# find a layer using its name
layer = mappyfile.find(mapfile["layers"], "name", "highlighted")

new_class_string = """
CLASS
    NAME "highlights"
    STYLE
        COLOR 107 208 107
        OUTLINECOLOR 2 2 2
        WIDTH 1
    END
END
"""

new_class = mappyfile.loads(new_class_string)
layer["classes"].insert(1, new_class) # can insert the new class at any index
assert(layer['classes'][1]['name'] == 'land')

print(mappyfile.dumps(mapfile))

Testing

Testing - there are many sample Mapfiles available in the testing suite of MapServer:

These have been downloaded and added to the /tests folder. This folder also contains a script to download these files again in the future.

Releases

Future Development