Osm Rendering Foss4g2010
-
Upload
tarokudo360 -
Category
Documents
-
view
232 -
download
0
Transcript of Osm Rendering Foss4g2010
-
7/23/2019 Osm Rendering Foss4g2010
1/51
OSM Rendering Toolchain - FOSS4G2010
Release 0.1
Dane Springmeyer
September 18, 2010
-
7/23/2019 Osm Rendering Foss4g2010
2/51
-
7/23/2019 Osm Rendering Foss4g2010
3/51
CONTENTS
1 Getting Started 1
1.1 Learning the Toolchain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Extra friends along the way . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 More Help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 Working with VirtualBox 3
2.1 A Virtual Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Start the Box! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3 Installing the Tools 5
3.1 Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.2 Ubuntu Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.3 Mac OS X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4 Configuring the Tools 11
4.1 PostGIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.2 QuantumGIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5 Downloading OSM Extracts 15
5.1 A Central Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5.2 A Sharable format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6 Importing into PostGIS 17
6.1 First Steps: Prep PostGIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6.2 Import with osm2pgsql . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6.3 View PostGIS data with Quantum GIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
7 Accessing the OSM Mapnik Cartography 19
7.1 Stylesheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
7.2 Downloading Shapefiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197.3 Getting the Latest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
7.4 Indexing Shapefiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
7.5 Reprojecting Shapefiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
8 Rendering with Mapnik 23
8.1 Mapnik Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.2 Bounding Boxes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.3 Using generate_xml.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.4 nik2img.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
i
-
7/23/2019 Osm Rendering Foss4g2010
4/51
8.5 TileLite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.6 generate_tiles.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
9 Serving Map Tiles 29
9.1 TileLite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
9.2 Apache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
9.3 mod_tile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
10 Making a Slippy Map with OpenLayers 31
10.1 Openlayers Map Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
10.2 Layer Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
11 TroubleShooting 33
11.1 Troubleshooting PostgreSQL Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
11.2 Troubleshooting PostgreSQL startup on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
11.3 Troubleshooting osm.xml rendering errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
12 Acknowledgements 35
13 Speakers Notes 37
13.1 Workshop Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3713.2 VM Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
13.3 VM Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
13.4 Ubuntu Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
14 Authors 43
14.1 Dane Springmeyer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
14.2 Ivn Snchez Ortega . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
15 Glossary 45
Index 47
ii
-
7/23/2019 Osm Rendering Foss4g2010
5/51
CHAPTER
ONE
GETTING STARTED
This workshop (http://2010.foss4g.org/workshop03.php) will walk you through getting raw OpenStreetMap data, im-
porting the data into PostGIS using osm2pgsql, rendering map tiles with Mapnik, and displaying them with OpenLay-
ers.
Note: If you are reading this documentation after September 2010, you should check for updated documentation
available at http://dbsgeo.com/foss4g2010/.
1.1 Learning the Toolchain
You will learn how to install and configure a mimic of the official OpenStreetMap toolchain, using the same tech-
nologies, that can be later customized to render different map styles or pull from other datasets. The OSM rendering
toolchain consists of the following technologies; all of them will be touched upon in the workshop:
1
http://2010.foss4g.org/workshop03.phphttp://dbsgeo.com/foss4g2010/http://dbsgeo.com/foss4g2010/http://2010.foss4g.org/workshop03.php -
7/23/2019 Osm Rendering Foss4g2010
6/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
1.2 Technologies
Planet.osm (http://wiki.openstreetmap.org/wiki/Planet.osm) - XML based dump format of the OSM database
PostGIS (http://postgis.refractions.net/) - Spatial Database (part of PostgreSQL)
osm2pgsql (http://wiki.openstreetmap.org/wiki/Osm2pgsql) - Tool for importing OSM data into PostGIS
Mapnik (http://mapnik.org/) - Rendering library
OpenLayers (http://openlayers.org/) - Javascript mapping interface
mod_tile (http://wiki.openstreetmap.org/wiki/Mod_tile) - high performance tile serving, on-demand rendering,
and caching
1.3 Extra friends along the way
We will also expose you to a few other useful tools:
QuantumGIS (http://qgis.org/) - User friendly data viewer with powerful python plugins
Nik2img (http://code.google.com/p/mapnik-utils/wiki/Nik2Img) - Command line stylesheet renderer
TileLite (http://bitbucket.org/springmeyer/tilelite/) - lightweight development tile server
Custom Mapnik stylesheets (Spreadnik (http://wiki.openstreetmap.org/wiki/Spreadnik), Quantum-
nik (http://bitbucket.org/springmeyer/quantumnik), and Cascadenik (http://code.google.com/p/mapnik-
utils/wiki/Cascadenik))
1.4 More Help
During the course of this workshop if you get stuck, you can find more details online at the OpenStreetMap/Mapnik
wiki page (http://wiki.openstreetmap.org/wiki/Mapnik). IRC (Internet Relay Chat) is also a great place to ask ques-tions. Try the #mapnik channel on irc://irc.freenode.net/mapnik and the #osm channel on irc://irc.oftc.net/osm.
1.4.1 Ready?
This plan sound good? Great, then lets begin by Installing the Tools (page 5).
If you are running linux first check out Working with VirtualBox (page 3).
2 Chapter 1. Getting Started
http://wiki.openstreetmap.org/wiki/Planet.osmhttp://postgis.refractions.net/http://wiki.openstreetmap.org/wiki/Osm2pgsqlhttp://mapnik.org/http://openlayers.org/http://wiki.openstreetmap.org/wiki/Mod_tilehttp://qgis.org/http://code.google.com/p/mapnik-utils/wiki/Nik2Imghttp://bitbucket.org/springmeyer/tilelite/http://wiki.openstreetmap.org/wiki/Spreadnikhttp://wiki.openstreetmap.org/wiki/Spreadnikhttp://bitbucket.org/springmeyer/quantumnikhttp://bitbucket.org/springmeyer/quantumnikhttp://code.google.com/p/mapnik-utils/wiki/Cascadenikhttp://wiki.openstreetmap.org/wiki/Mapnikhttp://wiki.openstreetmap.org/wiki/Mapnikhttp://wiki.openstreetmap.org/wiki/Mapnikhttp://wiki.openstreetmap.org/wiki/Mapnikhttp://code.google.com/p/mapnik-utils/wiki/Cascadenikhttp://bitbucket.org/springmeyer/quantumnikhttp://bitbucket.org/springmeyer/quantumnikhttp://wiki.openstreetmap.org/wiki/Spreadnikhttp://bitbucket.org/springmeyer/tilelite/http://code.google.com/p/mapnik-utils/wiki/Nik2Imghttp://qgis.org/http://wiki.openstreetmap.org/wiki/Mod_tilehttp://openlayers.org/http://mapnik.org/http://wiki.openstreetmap.org/wiki/Osm2pgsqlhttp://postgis.refractions.net/http://wiki.openstreetmap.org/wiki/Planet.osm -
7/23/2019 Osm Rendering Foss4g2010
7/51
CHAPTER
TWO
WORKING WITH VIRTUALBOX
For this workshop, some users will be running on Windows XP, but ideally the Ubuntu/Linux operating system will
be used for those comfortable with linux, or interested in learning. Linux offers great reward for those interested in
mapping with open source software.
2.1 A Virtual Machine
An easy way to run linux is from a Mac or Windows PC within VirtualBox (http://www.virtualbox.org/) (install it if it
is not already available).
The Mac or Windows operating system is the Host and linux becomes the guest running within VirtualBox.
3
http://www.virtualbox.org/http://www.virtualbox.org/ -
7/23/2019 Osm Rendering Foss4g2010
8/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
2.2 Start the Box!
To start the operating system open VirtualBox and start the system named osm-rendering.
Use osm for the username and password to log into the machine.
Note: If theres no system by that name, then you will have to create in in VirtualBox, using the ubuntu-lucid-
osm.vdi as the Hard Disk. This harddisk can be downloaded from http://dbsgeo.com/foss4g2010/.
2.2.1 Booted?
Once your linux machine is booted, then take a look at the guide for Installing the Tools (page 5).
Note: This VirtualBox Ubuntu machine already includes all necessary software pre-installed according to the instruc-
tions, but they are available to learn from and repeat if necessary.
4 Chapter 2. Working with VirtualBox
http://dbsgeo.com/foss4g2010/http://dbsgeo.com/foss4g2010/http://dbsgeo.com/foss4g2010/ -
7/23/2019 Osm Rendering Foss4g2010
9/51
CHAPTER
THREE
INSTALLING THE TOOLS
The stack of tools making up the OpenStreetMap rendering toolchain can run on Windows (mostly), Linux (nicely),
and Mac OS X.
This guide highly recommends running Ubuntu Lucid Linux (and installing apt packages), but also covers using
installers for Windows and Mac OS X.
This plan sound good? Great, then lets begin by Installing the Tools (page 5)
Choose your operating system: Windows (page 5), Ubuntu Linux (page 7), or Mac OS X (page 8).
When you are done installing, head next to Downloading OSM Extracts (page 15).
3.1 Windows
Note: For Windows you can find all the installers within the software folder on your Desktop.
3.1.1 Python
Mapnik works with both Python 2.5 and Python 2.6, but the installer below expects 2.5 so install Python2.5 from:
http://www.python.org/ftp/python/2.5.4/python-2.5.4.msi
3.1.2 PostgreSQL
Windows installers for the latest release of PostgreSQL:
http://downloads.enterprisedb.com/postgresql/postgresql-8.4.4-1-windows.exe
Run the installer:
Use osm as the postgres password
Use defaults for all other options, e.g. keep default port of 5432
Once finished, your computer should have a new user called postgres
You dont have to ever log in as this user, just leave it alone.
If you see an error about permission denied in a terminal window see Troubleshooting PostgreSQL
startup on Windows (page 33)
5
http://www.python.org/ftp/python/2.5.4/python-2.5.4.msihttp://downloads.enterprisedb.com/postgresql/postgresql-8.4.4-1-windows.exehttp://downloads.enterprisedb.com/postgresql/postgresql-8.4.4-1-windows.exehttp://www.python.org/ftp/python/2.5.4/python-2.5.4.msi -
7/23/2019 Osm Rendering Foss4g2010
10/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
3.1.3 PostGIS
The PostgreSQL windows installer can install PostGIS via the web using the Stackbuilder.
But, if you can also grab the installer directory:
http://postgis.refractions.net/download/windows/pg84/postgis-pg84-setup-1.5.1-1.exe
Note: Both Postgis 1.4.x and 1.5.x should be fine to use.
If installing from the Stackbuilder:
Go to Start Programs PostgreSQL 8.4 Application Stack Builder
In the first window from the drop down choose PostgreSQL 8.4 on port 5432
In the next window find and choose Spatial Extensions > PostGIS (any version is okay)
Choose any mirror, then move through the rest of the install keeping defaults
Once the installer is running:
Make sure to keep the blue checkbox checked that says Create Spatial Database
User: postgres Port: 5432
password for postgres user: osm
Choose Yes to the prompt about adding support for shp2pgsql
3.1.4 Mapnik & osm2pgsql
Standalone installer for Mapnik, osm2pgsql, and various python tools (nik2img,tilelite,cascadenik):
http://dbsgeo.com/mapnik/Mapnik_Installer.exe
3.1.5 mod_tile
Sorry, mod_tile is not available on Windows. An alternative is to use TileLite, which was just installed by the Mapnik
installer.
3.1.6 QuantumGIS
Standalone installer for QuantumGIS:
http://linfiniti.com/downloads/QGIS-1.4.0-1-No-GrassSetup.exe
Make sure to right click the installer and choose Run as Administrator
3.1.7 Unix Utilities
Other useful Unix utilities:
http://downloads.sourceforge.net/gnuwin32/wget-1.11.4-1-setup.exe
http://surfnet.dl.sourceforge.net/project/gnuwin32/bzip2/1.0.5/bzip2-1.0.5-setup.exe
http://downloads.sourceforge.net/tortoisesvn/TortoiseSVN-1.6.9.19725-win32-svn-1.6.12.msi?download
6 Chapter 3. Installing the Tools
http://postgis.refractions.net/download/windows/pg84/postgis-pg84-setup-1.5.1-1.exehttp://dbsgeo.com/mapnik/Mapnik_Installer.exehttp://linfiniti.com/downloads/QGIS-1.4.0-1-No-GrassSetup.exehttp://downloads.sourceforge.net/gnuwin32/wget-1.11.4-1-setup.exehttp://surfnet.dl.sourceforge.net/project/gnuwin32/bzip2/1.0.5/bzip2-1.0.5-setup.exehttp://downloads.sourceforge.net/tortoisesvn/TortoiseSVN-1.6.9.19725-win32-svn-1.6.12.msi?downloadhttp://downloads.sourceforge.net/tortoisesvn/TortoiseSVN-1.6.9.19725-win32-svn-1.6.12.msi?downloadhttp://surfnet.dl.sourceforge.net/project/gnuwin32/bzip2/1.0.5/bzip2-1.0.5-setup.exehttp://downloads.sourceforge.net/gnuwin32/wget-1.11.4-1-setup.exehttp://linfiniti.com/downloads/QGIS-1.4.0-1-No-GrassSetup.exehttp://dbsgeo.com/mapnik/Mapnik_Installer.exehttp://postgis.refractions.net/download/windows/pg84/postgis-pg84-setup-1.5.1-1.exe -
7/23/2019 Osm Rendering Foss4g2010
11/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
If finished, head along to Configuring the Tools (page 11).
3.2 Ubuntu Linux
Note: The workshop VirtualBox Image already has all these packages installed, but there is no harm re-installing
On ubuntu life is good because we can let the APT package manager (and the wisdom behind those who maintain it)
do all the hard work.
Update and upgrade package sources:
sudo apt-get update
sudo apt-get -y upgrade
Grab a bunch of base packages:
sudo apt-get install -y subversion python-setuptools \
build-essential wget curl vim unzip g++ cpp mercurial \
libboost1.40-dev libboost-filesystem1.40-dev \
libboost-iostreams1.40-dev libboost-program-options1.40-dev \
libboost-python1.40-dev libboost-regex1.40-dev \
libboost-thread1.40-dev libxml2 libxml2-dev \
libfreetype6 libfreetype6-dev libjpeg62 libjpeg62-dev \
libltdl7 libltdl-dev libpng12-0 libpng12-dev \
libgeotiff-dev libtiff4 libtiff4-dev libtiffxx0c2 \
libcairo2 libcairo2-dev python-cairo python-cairo-dev \
libcairomm-1.0-1 libcairomm-1.0-dev ttf-dejavu ttf-dejavu-core \
ttf-dejavu-extra python-nose libbz2-dev autoconf libsigc++-dev \
libsigc++0c2 libsigx-2.0-2 libsigx-2.0-dev libgdal1-dev \
libsqlite3-dev postgresql-8.4 postgresql-server-dev-8.4 \
postgresql-contrib-8.4 postgresql-8.4-postgis python-gdal \
apache2 apache2-threaded-dev apache2-mpm-prefork apache2-utils \
python-dev python-psycopg2 libapache2-mod-wsgi imagemagick php5-cli
Then compile the latest Mapnik:
svn co http://svn.mapnik.org/tags/release-0.7.1/ mapnik-0.7.1
cd mapnik-0.7.1
python scons/scons.py configure INPUT_PLUGINS=all OPTIMIZATION=3
sudo python scons/scons.py install
sudo ldconfig
cd ..
Warning: Make sure to uninstall any previous versions of Mapnik if present.
If you have installed Mapnik from packages do:
sudo apt-get remove libmapnik* python-mapnik
Or if you manually compiled do:
sudo rm /usr/local/lib/libmapnik.* # older mapnik versions will be in lib64
sudo rm -rf /usr/local/lib/mapnik/
sudo rm -rf /usr/local/include/mapnik/
sudo rm -rf /usr/lib/python2.6/dist-packages/mapnik/
Install a few python tools for working with Mapnik:
3.2. Ubuntu Linux 7
-
7/23/2019 Osm Rendering Foss4g2010
12/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
sudo easy_install cssutils PIL cascadenik nik2img tilelite werkzeug pygments
Compile the latest osm2pgsql:
svn co http://svn.openstreetmap.org/applications/utils/export/osm2pgsql osm2pgsql-trunk
cd osm2pgsql-trunk
./autogen.sh
./configure
make
sudo make install
cd ..
Compile the latest mod_tile:
svn co http://svn.openstreetmap.org/applications/utils/mod_tile mod_tile-trunk
cd mod_tile-trunk
make
sudo make install
sudo bash -c "echo LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so > /etc/apach
sudo a2enmod tile
sudo /etc/init.d/apache restart
Download Spreadnik:
svn co http://svn.openstreetmap.org/applications/utils/spreadnik/
Finally, grab QGIS:
sudo add-apt-repository ppa:ubuntugis/ubuntugis-unstable
sudo apt-get update
sudo apt-get install qgis
Note: QGIS should then appear in your menu under the Science catagory.
If finished, head along to Configuring the Tools (page 11).
3.3 Mac OS X
Great additional writeup for OS X is available at mojodna.net (http://mojodna.net/2009/12/05/the-os-x-spatial-
stack.html)
3.3.1 PostgreSQL & PostGIS
Mac installers for the latest releases of PostgreSQL and PostGIS:
http://www.kyngchaos.com/files/software/unixport/PostgreSQL-8.4.4-1.dmg
http://www.kyngchaos.com/files/software/unixport/PostGIS-1.5.1-1.dmg
3.3.2 Kyngchaos Frameworks
Install GDAL Complete linked from here (http://www.kyngchaos.com/software/frameworks):
http://www.kyngchaos.com/files/software/unixport/GDAL_Complete-1.7.dmg
8 Chapter 3. Installing the Tools
http://mojodna.net/2009/12/05/the-os-x-spatial-stack.htmlhttp://www.kyngchaos.com/files/software/unixport/PostgreSQL-8.4.4-1.dmghttp://www.kyngchaos.com/files/software/unixport/PostGIS-1.5.1-1.dmghttp://www.kyngchaos.com/software/frameworkshttp://www.kyngchaos.com/files/software/unixport/GDAL_Complete-1.7.dmghttp://www.kyngchaos.com/files/software/unixport/GDAL_Complete-1.7.dmghttp://www.kyngchaos.com/software/frameworkshttp://www.kyngchaos.com/files/software/unixport/PostGIS-1.5.1-1.dmghttp://www.kyngchaos.com/files/software/unixport/PostgreSQL-8.4.4-1.dmghttp://mojodna.net/2009/12/05/the-os-x-spatial-stack.html -
7/23/2019 Osm Rendering Foss4g2010
13/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
3.3.3 Mapnik & osm2pgsql
Standalone installer for Mapnik and osm2pgsql:
http://dbsgeo.com/downloads/(for both leopard and snow leopard)
Also install a few python tools for working with Mapnik:
sudo easy_install cssutils PIL cascadenik nik2img tilelite werkzeug pygments mercurial
3.3.4 mod_tile
Sorry, mod_tile is not available on Mac. An alternative is to use TileLite, which was just installed above using
easy_install.
3.3.5 Spreadnik
Grab from svn:
svn co http://svn.openstreetmap.org/applications/utils/spreadnik/
3.3.6 QuantumGIS
Standalone installer for QuantumGIS:
http://www.kyngchaos.com/software:qgis (for both leopard and snow leopard)
3.3.7 Next Steps
If finished, head along to Configuring the Tools (page 11).
3.3. Mac OS X 9
http://dbsgeo.com/downloads/http://www.kyngchaos.com/software:qgishttp://www.kyngchaos.com/software:qgishttp://dbsgeo.com/downloads/ -
7/23/2019 Osm Rendering Foss4g2010
14/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
10 Chapter 3. Installing the Tools
-
7/23/2019 Osm Rendering Foss4g2010
15/51
CHAPTER
FOUR
CONFIGURING THE TOOLS
4.1 PostGIS
To finish setting up PostGIS we need to create a database called: template_postgis. After creating this template, all
new databases that need spatial support can be cloned from this one.
Choose your operating system: Windows (page 11), Linux and Mac OS X (page 12)
4.1.1 Windows
Open the PostgreSQL shell, aka pqsl, by going to:
Start Programs PostgreSQL 8.4 SQL Shell (psql)
You will get a command prompt asking for input of connection parameters
Hit return to accept the default for all items, until you get to the password step where you should type the
password osm
Now, you should be inside the postgres shell!
Type:
CREATE DATABASE osm WITH TEMPLATE postgis;
Note: Make sure to finish the above CREATE statement with a semicolon otherwise the command will not
execute.
While we are inside the shell lets learn a few commands:
Switch into this new database called osm:
\c osm
List the tables inside this database:
\d
Get details about a specific table:
\d geometry_columns
Exit the database and shell:
\q
11
-
7/23/2019 Osm Rendering Foss4g2010
16/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
4.1.2 Linux and Mac OS X
First, become the postgres user:
sudo su - postgres
Then allow yourself to connect to postgres:
# allow your unix user to connect to postgres
createuser osm # or your username
# prompt: "Shall the new role be a superuser? (y/n)"
y
Then create the template_postgis:
# For PostGIS 1.5:
POSTGIS_SQL_PATH=pg_config --sharedir/contrib/postgis-1.5
# For PostGIS 1.4:
POSTGIS_SQL_PATH=pg_config --sharedir/contrib/
# Creating the template spatial database.
createdb -E UTF8 template_postgis
createlang -d template_postgis plpgsql # Adding PLPGSQL language support.
# Allows non-superusers the ability to create from this template
psql -q -d postgres -c "UPDATE pg_database SET datistemplate=true WHERE datname=template_p
# Loading the PostGIS SQL routines
psql -q -d template_postgis -f $POSTGIS_SQL_PATH/postgis.sql
psql -q -d template_postgis -f $POSTGIS_SQL_PATH/spatial_ref_sys.sql
# Enabling users to alter spatial tables.
psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"
# If running PostGIS 1.5:
psql -d template_postgis -c "GRANT ALL ON geography_columns TO PUBLIC;"
# now switch back to your normal user
exit
4.2 QuantumGIS
Install a few key plugins for working with Mapnik and OSM data:
Plugins Fetch Python Plugins Repositories tab
Click Add, then type:
Name: dbsgeo.com plugins
URL: http://qgis.dbsgeo.com
Clickokay
Click on the Plugins tab and you should should see two new plugins:
* Quantumnik
12 Chapter 4. Configuring the Tools
http://qgis.dbsgeo.com/http://qgis.dbsgeo.com/ -
7/23/2019 Osm Rendering Foss4g2010
17/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
* OSM Tools
Highlight Quantumnik and click Install Plugin
Restart QGIS
Enable the built-in OpenStreetMap viewer plugin:
Plugins Manage Plugins
Find the OpenStreetMap plugin and check the box to activate it.
Then a new panel of tools should appear with green square icons
You can drag this panel of tools wherever you like within the menu area
Here are a few recommended customizations of QGIS:
Plugins Fetch Python Plugins Options Check for updates on startup Everytime QGIS Starts
Settings Options:
CRS section Check prompt for CRS
General section Change the Icon Theme to GIS
Rendering and SVG section Check Make lines appear less jagged...
4.2.1 Next Steps
If finished, head along to Downloading OSM Extracts (page 15).
4.2. QuantumGIS 13
-
7/23/2019 Osm Rendering Foss4g2010
18/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
14 Chapter 4. Configuring the Tools
-
7/23/2019 Osm Rendering Foss4g2010
19/51
CHAPTER
FIVE
DOWNLOADING OSM EXTRACTS
5.1 A Central Database
OpenStreetMap data is stored in a highly tuned PostgreSQL database running in London. When you edit Open-
StreetMap data, whether from the Potlach web interface, or the JOSM desktop tool, you are interacting with this
server.
5.2 A Sharable format
But when you want to download anything more than a very small chunk of the OpenStreetMap database, you need
what is commonly called an Extract, which is OSM data serialized in XML. The file may have a .osm file extension,
or be compressed with bzip2 and have a .bz2 extension (most OSM software can easily read extracts in either format).
A complete extract of the whole planet is called Planet.osm (http://wiki.openstreetmap.org/wiki/Planet.osm). For the
purposes of this workshop, we will only work with a small extract for the country of Spain and for the city of Barcelona.
Note: The extracts are available in the workshop materials on your Desktop in the data/osm-extracts folder
5.2.1 Downloading the country of Spain
Extracts are produced by various individuals, companies, and other OpenStreetMap community members that keep an
up-to-date mirror of the OpenStreetMap database.
Two companies in particular offer a wide array of downloads broken out by country: Cloudmade and Geofabrik. We
find the extracts more reliable from Geofabrik (sometimes exported data has glitches that make imports fail), so lets
download Spain from Geofabrik:
http://download.geofabrik.de/osm/europe/spain.osm.bz2
5.2.2 Downloading the city of Barcelona
Geofabrik does not, however offer small extracts for every city in the world. Lucky for us, for small requests there are
other options. The OSM Xapi (http://wiki.openstreetmap.org/wiki/Xapi) is a great option, as it is protocol designed for
fast read-only requests of OSM data, and is currently running on a variety of nodes. In this case lets hit the OSMXAPI
at the telescience lab in San Diego (ofhttp://haiticrisismap.org/ fame). The trick is to send a bbox for Barcelona in the
url:
wget -O barcelona.osm http://osmxapi.hypercube.telascience.org/api/0.6/map?bbox=1.998653,41.3
15
http://wiki.openstreetmap.org/wiki/Planet.osmhttp://download.geofabrik.de/osm/europe/spain.osm.bz2http://wiki.openstreetmap.org/wiki/Xapihttp://haiticrisismap.org/http://haiticrisismap.org/http://wiki.openstreetmap.org/wiki/Xapihttp://download.geofabrik.de/osm/europe/spain.osm.bz2http://wiki.openstreetmap.org/wiki/Planet.osm -
7/23/2019 Osm Rendering Foss4g2010
20/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
Note: Mac OS X does not have the wget command installed by default but you can exchange the above command
with curl -O instead.
Note: If you need help finding the bounding box of an area, try this tool:
http://haiti.dbsgeo.com/maps/map_tools/get_extents.html
5.2.3 Accessing Extracts from Mapping Applications
There are many ways to get OSM data in small chunks. JOSM (http://josm.openstreetmap.de/) can download small
OSM extracts, as well a Merkaartor (http://merkaartor.be/), and even QuantumGIS (using the built-in OpenStreetMap
plugin or the OSM Tools plugin (http://qgis.dbsgeo.com/))
5.2.4 Next Steps
Great, now that youve got some OSM data, lets start Importing into PostGIS(page 17).
16 Chapter 5. Downloading OSM Extracts
http://haiti.dbsgeo.com/maps/map_tools/get_extents.htmlhttp://josm.openstreetmap.de/http://merkaartor.be/http://qgis.dbsgeo.com/http://qgis.dbsgeo.com/http://merkaartor.be/http://josm.openstreetmap.de/http://haiti.dbsgeo.com/maps/map_tools/get_extents.html -
7/23/2019 Osm Rendering Foss4g2010
21/51
CHAPTER
SIX
IMPORTING INTO POSTGIS
6.1 First Steps: Prep PostGIS
Create a PostGIS enabled database from template_postgis as the postgres user:
createdb barcelona -T template_postgis
Note: You may get see an error like Ident authentication failed. If so see: TroubleShooting (page 33)
Note: It is important that OSM databases be in UTF8 encoding (you can see which are by issuing the command
psql -l). You should note that in the Configuring the Tools (page 11) page we created our template_postgis in
UTF8. Because we cloned our barcelona database from that template it should also be in UTF8 encoding.
6.2 Import with osm2pgsql
osm2pgsql works so nicely youd think it were simple. But in fact its quite sophisticated because it handles two
important conversions:
1. Turning OSM key=value associations into rows and columns in the database
2. Turning OSM ways, nodes, and relations into proper OGC simple features (geometries GIS programs can read)
To import data into postgis do:
cd projects/osm-rendering/osm_extracts/
osm2pgsql --merc -d barcelona barcelona.osm
Note: The osm2pgsql program should automatically read what is called the default.style from
/usr/local/share/osm2pgsql/default.style, the location it should have been installed to when you compiled and installed
osm2pgsql. On windows you will need to pass the directory for where to find this file like style /path/to/default.style.
It can be downloaded from http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/default.style.
The merc command means you want to import the data in Spherical Mercator projection, but you can leave this
command off since that is the default.
6.2.1 Import data in Lat Long (WGS84)
For custom setups you may wish to import data in Lat Long (WGS84):
osm2pgsql --latlong -d barcelona barcelona.osm
17
http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/default.stylehttp://svn.openstreetmap.org/applications/utils/export/osm2pgsql/default.style -
7/23/2019 Osm Rendering Foss4g2010
22/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
Note: If you wish to import OSM data into PostGIS in Lat Long, make sure to also reproject and re-index your
shapefiles. See Reprojecting Shapefiles (page 20).
Note: Osmosis (http://wiki.openstreetmap.org/wiki/Osmosis) is a Java command line tool and the another powerful
tool for interfacing with OSM data and PostgeSQL - but it uses a different schema that osm2pgsql. Nevertheless you
should know about osmosis. You can use it to clip out smaller sections of osm files, or even merge multiple together.
While its not essential for the rendering toolchain its a good tool to know about. Once you get interested in keeping
your OSM server up to date then osmosis will quickly become crucial, as it is able to speak the language of .osc (osmchange files).
6.3 View PostGIS data with Quantum GIS
A great way to get a quick view of the OSM data now in PostgreSQL is to view it in QGIS.
Open QGIS
Go to Layers Add PostGIS Layer...
Click New
Then in the connections window type:
Name: OSM (or anything you like here)
Host: leave blank
* (normally you would type localhost but that requires allowing trust connections, see Trou-
bleShooting (page 33))
Database: barcelona
Port: 5432
Username: osm
Password: leave blank (on windows type osm)
ClickTest connect. And problems see: TroubleShooting (page 33)
You should should get a message that the connection was successful
Now you can choose layers to view
ClickAdd then close and they will appear in QGIS
6.3.1 Now Time for Style
Now that youve got data, we need style so lets move on to Accessing the OSM Mapnik Cartography (page 19)
18 Chapter 6. Importing into PostGIS
http://wiki.openstreetmap.org/wiki/Osmosishttp://wiki.openstreetmap.org/wiki/Osmosis -
7/23/2019 Osm Rendering Foss4g2010
23/51
CHAPTER
SEVEN
ACCESSING THE OSM MAPNIKCARTOGRAPHY
OSM cartographic styles are maintained by several skilled cartographers (notably Steve Chilton and Lennard Vor den
Dag) and the wider community.
7.1 Stylesheets
These are XML-based styling rules and layers. Dont confuse this XML (which is purely about cartography) with the
XML format of OSM data. These stylesheets reference PostGIS tables and shapefiles and apply styling logic to power
the maps at http://openstreetmap.org.
Several datasources are needed to be render-ready for Mapnik
OSM data in PostGIS: This import was done previously in Importing into PostGIS(page 17).
Shapefiles:
For rendering speed and ease of use a few datasets are processed from OSM data into shapefiles, like
coastlines.
In addition, a several Natural Earth (http://www.naturalearthdata.com/) shapefiles are used for world cities
and administrative boundaries.
7.2 Downloading Shapefiles
For this workshop we provide a single download of everything you need from:
http://dbsgeo.com/mapnik/stylesheets_with_shapefiles.zip
You will need to unzip the package of styles and shapefiles:
cd projects/osm-rendering
unzip stylesheets.zip
cd stylesheets/
It should already be here:
projects/osm-rendering/stylesheets.zip
But is also available for download like:
19
http://openstreetmap.org/http://openstreetmap.org/http://www.naturalearthdata.com/http://dbsgeo.com/mapnik/stylesheets_with_shapefiles.ziphttp://dbsgeo.com/mapnik/stylesheets_with_shapefiles.ziphttp://www.naturalearthdata.com/http://openstreetmap.org/ -
7/23/2019 Osm Rendering Foss4g2010
24/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
wget -O stylesheets.zip http://dbsgeo.com/mapnik/stylesheets_with_shapefiles.zip
7.3 Getting the Latest
If you would like to stay up to date on the latest styles, find them (along with links to the latest shapefiles) in theOpenStreetMap subversion server. Check this data out using subversion:
# xml stylesheets
svn co http://svn.openstreetmap.org/applications/rendering/mapnik stylesheets
# shapefiles needed by styles
cd stylesheets
wget http://tile.openstreetmap.org/world_boundaries-spherical.tgz # (50M)
wget http://tile.openstreetmap.org/processed_p.tar.bz2 # (227M)
wget http://tile.openstreetmap.org/shoreline_300.tar.bz2 # (46M)
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/10m
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/11
# unpack the shapefiles in the right placetar xzf world_boundaries-spherical.tgz # creates a world_boundaries folder which the styles
tar xjf processed_p.tar.bz2 -C world_boundaries
tar xjf shoreline_300.tar.bz2 -C world_boundaries
unzip -q 10m-populated-places.zip -d world_boundaries
unzip -q 110m-admin-0-boundary-lines.zip -d world_boundaries
7.4 Indexing Shapefiles
These shapefiles should come with indexes pre-built that are designed to speed up the ability of Mapniks shapefile
plugin to query features. But, just to be safe, well re-generate the proper indexes:
cd world_boundaries
shapeindex processed_p
shapeindex builtup_area
shapeindex shoreline_300
shapeindex 10m_populated_places
shapeindex 110m_admin_0_boundary_lines_land
7.5 Reprojecting Shapefiles
Optional: Some users may wish to serve maps in a different projection, like WGS 84. This is perfectly doable, but
for your maps to render quickly youll need to reproject and re-index your shapefiles (and Import data in Lat Long
(WGS84) (page 17)):
cd world_boundaries
ogr2ogr -t_srs EPSG:4326 processed_p_ll.shp -s_srs EPSG:900913 processed_p.shp
ogr2ogr -t_srs EPSG:4326 builtup_area_ll.shp builtup_area.shp
ogr2ogr -t_srs EPSG:4326 shoreline_300_ll.shp -s_srs EPSG:900913 shoreline_300.shp
shapeindex processed_p_ll
shapeindex builtup_area_ll
shapeindex shoreline_300_ll
20 Chapter 7. Accessing the OSM Mapnik Cartography
-
7/23/2019 Osm Rendering Foss4g2010
25/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
7.5.1 Finished?
Now that youve got the OSM styles its time to start Rendering with Mapnik(page 23)
7.5. Reprojecting Shapefiles 21
-
7/23/2019 Osm Rendering Foss4g2010
26/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
22 Chapter 7. Accessing the OSM Mapnik Cartography
-
7/23/2019 Osm Rendering Foss4g2010
27/51
CHAPTER
EIGHT
RENDERING WITH MAPNIK
8.1 Mapnik Overview
Mapnik is the rendering library and engine behind the familiar cartography of OSM. It can be used to render tiles or
serve WMS and is often found powering desktop and web applications.
As Mapnik has great python bindings, we will simply be interacting with the Mapnik library through a variety of
python tools in examples below.
Note: This guide focuses on Mapnik 0.7.1 (not yet Mapnik2)
8.2 Bounding Boxes
The next few sections are all going to depend on you understanding the basics of bounding boxes. For details we
recommend giving a read to: http://weait.com/content/map-tiles-and-bounding-boxes
Basically, Mapnik and most other GIS software accepts bounding boxes in any coordinate system but always expects
them to be composed of 4 numbers, made up of two pairs of x,y coordinates. The X is longitude, the Y is latitude, and
a bbox is made of up (minx,miny,maxx,maxy).
An example bounding box for the whole globe in WGS 84 is:
-180, -90, 180, 90
And an example bounding box for the whole globe in Google Mercator is:
-20037508.34, -20037508.34, 20037508.34, 20037508.34
Note: You will see the above global mercator bbox appear in the code of the Stylesheets (page 19) and in the
Openlayers Map Projection (page 31).
8.3 Using generate_xml.py
Now its time to set up the stylesheet and a script called generate_xml.py can help with this. Basically you need to pass
your database settings (database name and user/password) to be placed in the Mapnik XML stylesheets.
First head into the stylesheets directory (created when you uzipped the package in Accessing the OSM Mapnik Car-
tography (page 19)):
cd projects/osm-rendering/stylesheets/
23
http://weait.com/content/map-tiles-and-bounding-boxeshttp://weait.com/content/map-tiles-and-bounding-boxes -
7/23/2019 Osm Rendering Foss4g2010
28/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
Now check out the options (and defaults) the script knows about:
./generate_xml.py -h
You should receive output like:
Usage: generate_xml.py
Full help: $ generate_xml.py -h (or help for possible options)
Read osm.xml and print resulting xml to stdout: $ generate_xml.py osm.xml
Read template, save output xml, and pass variables as options $ generate_xml.py osm.xml
my_osm.xml dbname spain user postgres host
Options:
--version show programs version number and exit
-h, --help show this help message and exit
--inc=INC Includes dir (default: inc)
--accept-none Interpret lacking value as unneeded
--symbols=SYMBOLS Set value of symbols (default: symbols)
--epsg=EPSG Set value of epsg (default: 900913)
--world_boundaries=WORLD_BOUNDARIES Set value of world_boundaries
(default: world_boundaries)
--prefix=PREFIX Set value of prefix (default: planet_osm)
--password=PASSWORD Set value of password
--host=HOST Set value of host
--port=PORT Set value of port
--user=USER Set value of user
--dbname=DBNAME Set value of dbname
--estimate_extent=ESTIMATE_EXTENT Set value of estimate_extent (default:
false)
--extent=EXTENT Set value of extent (default: -20037508,-
19929239,20037508,19929239)
Because generate_xml.py already knows the right directory structure for the symbols and shapefiles all we need to do
is issue this command:
./generate_xml.py --dbname osm --user postgres --accept-none
On Windows all postgres users must have a password so do:
./generate_xml.py --dbname osm --user postgres --password osm --accept-none
8.3.1 Stylesheets in EPSG:4326
If you imported data into PostGIS using the latlong flag with osm2pgsql then youll need to do:
./generate_xml.py --dbname osm --extent -179,-89,179,89 --epsg 4326 --accept-none
24 Chapter 8. Rendering with Mapnik
-
7/23/2019 Osm Rendering Foss4g2010
29/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
And you will also need to customize the stylesheet which contains the references to the shapefiles which we reprojected
( see Reprojecting Shapefiles (page 20)):
Index: osm.xml
===================================================================
--- osm.xml (revision 22361)
+++ osm.xml (working copy)
@@ -6,7 +6,7 @@
-
+
&fontset-settings;
-
7/23/2019 Osm Rendering Foss4g2010
30/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
Index: inc/fontset-settings.xml.inc.template
===================================================================
--- inc/fontset-settings.xml.inc.template (revision 22361)
+++ inc/fontset-settings.xml.inc.template (working copy)
@@ -7,13 +7,13 @@
-
+
-
+
-
+
8.4 nik2img.py
Now that your osm.xml is set up, we can test a very simple rendering with nik2img.
This should create a blue and white world map:
nik2img.py osm.xml world.png
Warning: If you get an error starting with inc/entities.xml.inc:2: I/O warning : failed see: Troubleshooting
osm.xml rendering errors (page 34)
Note: You might wonder why the last image was for the entire world+. Well, nik2img is designed to zoom to the full
extent of the data referenced by the Stylesheets. While we have currently only imported a tiny chunk of Barcelona
into our PostGIS database, the stylesheets also reference global shapefiles, so these expand the cumulative extent to
the globe.
And this should create an image of barcelona:
nik2img.py osm.xml barcelona.png -e 216471.34 5052350.67 266818.65 5096245.2
Note: You might ask: How did we get those extents to pass to nik2img?
We grabbed them from the postgres tables using the command (just remove the middle comma):
$ psql barcelona -c "Select ST_Extent(way) from planet_osm_roads;"st_extent
-----------------------------------------------
BOX(216471.34 5052350.67,266818.65 5096245.2)
8.5 TileLite
Now head over and serve some tiles from this stylesheet:
26 Chapter 8. Rendering with Mapnik
-
7/23/2019 Osm Rendering Foss4g2010
31/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
cd ../tilelite
Note: If you dont yet have a tilelite directory, go ahead an use mercurial to check one out:
hg clone https://[email protected]/springmeyer/tilelite
cd tilelite
sudo python setup.py install
You can also download the code as a zip archive (http://bitbucket.org/springmeyer/tilelite/get/tip.zip)
Then launch the TileLite development server with the command:
liteserv.py ../stylesheets/osm.xml
Now youve got the server ready to respond to tile request on localhost. Open up firefox and go to:
http://localhost:8000 # you should see the TileLite welcome page
Finally to check your tiles on a map open the OpenLayers demo file at:
tilelite/demo/openlayers.html
# note: you can open this file from the command line with
xdg-open tilelite/demo/openlayers.html
You should be able to zoom into Barcelona and once you get close you should see your OSM data appearing.
Note: If you want more data to show up then youll need to import a bigger extract. Header over to Geofabrik
(http://www.geofabrik.de/data/download.html) for extracts.
8.6 generate_tiles.py
For fast tile serving its best to cache tiles. TileLite can do this by passing the caching flag to liteserv.py, but an even
easier way is to pre-generate the tiles and then just drop them on a webserver. This way you wont even need to installmapnik on your web host (or PostGIS).
Warning: This works for small areas, but for large areas it wont because the time and storage space needed to
cache (even a countries worth of tiles) is enormous. So, for large areas mod_tile is the best solution, because it can
cache on-demand only those tiles requested and clear out infrequently viewed tiles to save space.
First head into the stylesheets directory:
cd projects/osm-rendering/stylesheets
Then open up generate_tiles.py for editing:
gedit generate_tiles.py
This script is just a bit of python, meant to customize to your needs.
1. On line 13, change NUM_THREADS to 1.
2. On line 199 edit your bounding box in the script to:
bbox = (1.94459513295834,41.2706895358557,2.39687271383137,41.5663815729642)
3. Then delete all code below line 201
8.6. generate_tiles.py 27
http://bitbucket.org/springmeyer/tilelite/get/tip.ziphttp://www.geofabrik.de/data/download.htmlhttp://www.geofabrik.de/data/download.htmlhttp://bitbucket.org/springmeyer/tilelite/get/tip.zip -
7/23/2019 Osm Rendering Foss4g2010
32/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
Note: How did we get the bounding box for generate_tiles.py?
Well, generate_tiles.py excepts the bbox in EPSG:4326 (aka WGS 84), so we used postgres to fetch the extents of
barcelona and then translate them to that coordinate system:
$ psql barcelona -c "select ST_Extent(ST_Transform(way,4326)) from planet_osm_roads;"
st_extent
--------------------------------------------------------------------------BOX(1.94459513295834 41.2706895358557,2.39687271383137 41.5663815729642)
Now, lets create a directory for tiles:
mkdir tiles
And set a few environment variables the script expects, then run it:
export MAPNIK_MAP_FILE=osm.xml
export MAPNIK_TILE_DIR=tiles
./generate_tiles.py
Yahoo, you should now have tiles being created by the hundred!
8.6.1 Almost Done
At this point we are almost done. Depending on your interests you can now can now point OpenLayers at these tiles
instead of at TileLite (port 8000). For a hint about how to do that skip ahead to Making a Slippy Map with OpenLayers(page 31).
Or for more detail about on-demand rendering take a look at Serving Map Tiles (page 29).
28 Chapter 8. Rendering with Mapnik
-
7/23/2019 Osm Rendering Foss4g2010
33/51
CHAPTER
NINE
SERVING MAP TILES
9.1 TileLite
We have already seen TileLite in action in the TileLite (page 26) section. It is a simple tool designed for stylesheet
development and serving map tiles right out of the box. For a full fledged map serving system, the combination of
mod_tile and renderd is well worth checking out (it is what the official tile.openstreetmap.org site uses).
9.2 Apache
The fastest way weve found to deploy TileLite for production systems is using mod_wsgi
(http://code.google.com/p/modwsgi/) and Apache.
In addition, the fastest possible way to deploy a tileserver in production is mod_tile, which is written as an Apache
module.
So so youll need Apache installed before continuing. We skipped this step in the Installing the Tools (page 5) section
because there are a variety of ways to go about it.
One Windows we recommend installing Apache using OSGEO4W (http://trac.osgeo.org/osgeo4w/). On linux apacheshould already be installed using apt-get but you can confirm by running this command:
sudo apt-get install apache2 apache2-threaded-dev apache2-mpm-prefork apache2-utils libapache
On Mac OS X, apache is already installed and running (as 64 bit nonetheless!). To ensure it is turned on go to:
System Preferences Sharing Check **Web Sharing.
You should now be able to view a page at http://localhost
9.3 mod_tile
Detailed setup of mod_tile is best done in person. See you at Foss4g!
If you just cant wait see the mod_tile readme (http://svn.openstreetmap.org/applications/utils/mod_tile/readme.txt).
29
http://code.google.com/p/modwsgi/http://trac.osgeo.org/osgeo4w/http://localhost/http://svn.openstreetmap.org/applications/utils/mod_tile/readme.txthttp://svn.openstreetmap.org/applications/utils/mod_tile/readme.txthttp://localhost/http://trac.osgeo.org/osgeo4w/http://code.google.com/p/modwsgi/ -
7/23/2019 Osm Rendering Foss4g2010
34/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
30 Chapter 9. Serving Map Tiles
-
7/23/2019 Osm Rendering Foss4g2010
35/51
CHAPTER
TEN
MAKING A SLIPPY MAP WITHOPENLAYERS
OpenLayers is a excellent feature-rich javascript library for making map interfaces online. Please refer to the Open-
Layers documentation (http://docs.openlayers.org) for all the details.
For serving OSM tiles basically what you need to know boils down to two things: 1) How to create an OpenLayersMap in the right projection and 2) and how to create an OpenLayers Layer referencing your tiles by url.
10.1 Openlayers Map Projection
Your map must be in spherical mercator to support the standard tile schemes.
All the details youll even need on this projection should be available here
(http://docs.openlayers.org/library/spherical_mercator.html).
Basically what you should care about is that your OpenLayers map is in this projection, commonly
called the Google Projection (http://spatialreference.org/ref/sr-org/google-projection/ ), and referred to by
EPSG:900913 or EPSG:3857 (see the MapTiler docs (http://www.maptiler.org/google-maps-coordinates-tile-bounds-projection/) for more info).
To do this create a map object in OpenLayers like:
// Options for World Map in Google Spherical Mercator Projection
var options = {
maxResolution: 156543.0339,
units: m,
projection: new OpenLayers.Projection("EPSG:900913"),
displayProjection: new OpenLayers.Projection("EPSG:4326"),
maxExtent: new OpenLayers.Bounds(-20037508.34, -20037508.34, 20037508.34, 20037508.34)
};
map = new OpenLayers.Map("map", options);
10.2 Layer Creation
As of OpenLayers 2.8 this is now really easy as there is a specific layer type for OSM, that defaults to using the
common z/x/y.png tile scheme (aka Google/OSM tile scheme).
Create it like:
31
http://docs.openlayers.org/http://docs.openlayers.org/http://docs.openlayers.org/library/spherical_mercator.htmlhttp://spatialreference.org/ref/sr-org/google-projection/http://www.maptiler.org/google-maps-coordinates-tile-bounds-projection/http://www.maptiler.org/google-maps-coordinates-tile-bounds-projection/http://spatialreference.org/ref/sr-org/google-projection/http://docs.openlayers.org/library/spherical_mercator.htmlhttp://docs.openlayers.org/http://docs.openlayers.org/ -
7/23/2019 Osm Rendering Foss4g2010
36/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
var tiles = new OpenLayers.Layer.OSM("Mapnik", "http://localhost:8000/${z}/${x}/${y}.png");
10.2.1 Thats It!
This is the last page in the guide. If you missed the details on Serving Map Tiles (page 29), you might want to check
that out. Otherwise, feel free to email Dane or Ivn with ideas about what else youd like to learn. Find us at the
Authors (page 43) page.
32 Chapter 10. Making a Slippy Map with OpenLayers
-
7/23/2019 Osm Rendering Foss4g2010
37/51
CHAPTER
ELEVEN
TROUBLESHOOTING
11.1 Troubleshooting PostgreSQL Connections
You may recieve an error like:
$ createdb -T template_postgis osm # connecting as "osm" user by supplying no -U argument
could not connect to database postgres: FATAL: Ident authentication failed for user "osm"
or if trying to connect as the postgres user:
$ createdb -T template_postgis osm -U postgres could not connect to database postgres: FATAL: Ident
authentication failed for user postgres
If you get this it means you need to either allow your unix user to connect to a postgres database, or enable trust
connections for postgres (editing pg_hba.conf), or alternatively allow your user to connect to a postgres database.
The first approach is easier for a demo machine, the latter more secure.
Here is the trick for the easy, insecure method:
sudo su - postgres
createuser osm # or your username# prompt: "Shall the new role be a superuser? (y/n)"
y
Or trust connections, that will allow you to connect as the postgres user without a password:
sudo vim /etc/postgresql/8.4/main/pg_hba.conf
## Change
local all all ident sameuser
## To
local all all trust
## Then restart the database server
sudo su - postgres
/etc/init.d/postgresql-8.3 restartexit
11.2 Troubleshooting PostgreSQL startup on Windows
If on Windows Vista or Windows 7, you may need to give the postgres user administration privileges
Otherwise you will may see errors about permission denied when the PostgreSQL server automatically starts.
33
-
7/23/2019 Osm Rendering Foss4g2010
38/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
On Windows 7 (and vista):
Start Control Panel View by small icons User Accounts Manage another account, Pick
Postgres Change Account Type, make administrator
Then go back to User Accounts view
Change User Account Control Settings:
Lower the UAV settings to the lowest setting
Restart your machine, and login as your normal user (not the postgres user)
Now the PostgreSQL server should be automatically running
Go to Start Programs PostgreSQL 8.4 Restart Server (to confirm it is working)
If you get an error about access denied, then go back and repeat the above user account steps
If it works, then type to leave the window type:
\q
11.3 Troubleshooting osm.xml rendering errors
You may see an error like:
inc/entities.xml.inc:2: I/O warning : failed to load external entity "inc/settings.xml.inc"
%settings;
^
%settings;
^
I/O warning : failed to load external entity "inc/fontset-settings.xml.inc"
If so, you forgot to properly run generate_xml.py so return to the Rendering with Mapnik (page 23) page and in
particular the section on Using generate_xml.py (page 23)!
34 Chapter 11. TroubleShooting
-
7/23/2019 Osm Rendering Foss4g2010
39/51
CHAPTER
TWELVE
ACKNOWLEDGEMENTS
Great thanks go to a variety of sources of information for this guide:
Artem Pavelenko and Jon Burgess for writing nearly all the software used in this guide (or at least all the good
software!)
Steve Chilton and Lennard voor den Dag for the awesome work on the OSM Stylesheets
Richard Weait for great supporting guides - http://weait.com/
The osm wiki and all its many contributors
35
http://weait.com/http://weait.com/ -
7/23/2019 Osm Rendering Foss4g2010
40/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
36 Chapter 12. Acknowledgements
-
7/23/2019 Osm Rendering Foss4g2010
41/51
CHAPTER
THIRTEEN
SPEAKERS NOTES
13.1 Workshop Instructions
W-03: Setting up an OpenStreetMap rendering toolchain
We provide two options for students:
1) A VirtualBox (VDI) harddisk running Ubuntu Lucid with all software and data pre-installed. This is the PREFER-
ABLE method.
INSTRUCTIONS: Please copy this VDI (software/linux/ubuntu-lucid-osm.vdi.zip) to the Windows
Machine, Unzip it, install VirtualBox, and create a new Ubuntu 32bit VM, with the ubuntu-lucid-
osm.vdi attached as the SATA harddisk.
2. Windows installers for software (to run on the native OS - XP).
INSTRUCTIONS: Please copy the software/ and data/ directories to the windows XP desktop.
Students (with help from instructors) will be responsible for installing software on the spot. Note:
these should also work with Windows 7 if need be.
Contact us with any questions:
Dane Springmeyer , Ivn Snchez Ortega
13.2 VM Instructions
All the software and data needed for the OSM Workshop (w3) is included in this VirtualBox image (ubuntu-lucid-
osm.vdi).
Some information about the VirtualBox image:
Operating System: Ubuntu Lucid/Linux 32 bit
Network configuration through DHCP, the virtual server must use NAT
Admin user: osm/osm
This osm user has sudo
Tests to do upon booting the system:
The network interface is up
The PostgreSQL server is running
37
mailto:[email protected]:[email protected]:[email protected]:[email protected] -
7/23/2019 Osm Rendering Foss4g2010
42/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
13.3 VM Sizes
733 MB: original ubuntu-10.04-desktop-i386.iso download from canonical
3.61 GB: ubuntu-lucid-osm.vdi once installed and updated
4.12 GB: custom software through apt-get (384 MN more)
7.7 GB: After loading workshop data + running tests on applications
13.4 Ubuntu Setup
Installed all updates when system prompted (alt is to run sudo apt-get update and sudo apt-get upgrade)
Installed VBox guest additions
VBox Menu: Devices > Install Guest Additions > double-click autorun.sh (run in terminal)
Then right-click and eject disk mounted on desktop
Reboot machine, afterwords screen should now resize nicely
Added user logo as http://wiki.openstreetmap.org/w/images/3/30/Mag_map-120x120.png
Dragged terminal to menu bar for easy access
Dragged references to folders and web sites to menu bar
Ran setup.sh (see below):
## snippets to set up VM and Windows for tutorials ##
## SETUP UBUNTU MACHINE ##
cd Desktop
cat README.txtWelcome to the OSM Rendering workshop linux machine.
This is running Ubuntu Lucid (32 bit) and has pre-installed
all the necessary software for the workshop as well as sample
data and project files.
The user is osm and the password is osm.
See the projects/osm-rendering folder in your home directory
which is also available a shortcut on the Desktop.
The workshop will be walking you through how to import OSM data
and render it using the materials in the osm-rendering folder.
Sample data includes snapshots from the first week in July of
several osm extracts and the official stylesheets from OSM SVN.
Installations already on the machine include:
* PostgreSQL
* PostGIS
* osm2pgsql
* Mapnik
38 Chapter 13. Speakers Notes
http://wiki.openstreetmap.org/w/images/3/30/Mag_map-120x120.pnghttp://wiki.openstreetmap.org/w/images/3/30/Mag_map-120x120.png -
7/23/2019 Osm Rendering Foss4g2010
43/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
* mod_tile
* Quantum GIS
* Python-Mapnik tools:
* Quantumnik, Cascadenik, TileLite, nik2img
Please consult the workshop guide and materials for further info:
* In the "Guide" folder on your desktop
* Also available at: http://dbsgeo.com/foss4g2010/
Or contact the intructors:
* Dane Springmeyer
* Ivn Snchez Ortega
EOF
# from home directory...
cd
mkdir projects
ln -s pwd/projects pwd/Desktop/projectscd projects
mkdir osm-rendering
cd osm-rendering
# osm data
# for windows we put this in the data directory on the ftp
mkdir osm-extracts
cd osm-extracts
wget -O spain_shp.zip http://download.geofabrik.de/osm/europe/spain.shp.zip
wget http://download.geofabrik.de/osm/europe/spain.osm.bz2
wget -O barcelona.osm http://osmxapi.hypercube.telascience.org/api/0.6/map?bbox=1.998653,41.3
cd ../
# grab the osm2pgsql default style to have around:wget http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/default.style
# mapnik styles + shapefiles in one!
# for windows we put this in the data directory on the ftp
wget -O stylesheets.zip http://dbsgeo.com/mapnik/stylesheets_with_shapefiles.zip
# will have students unzip...
# tile lite to test tile rendering
# for windows we put this in the software directory on the ftp
#hg clone http://bitbucket.org/springmeyer/tilelite
wget -O tilelite.zip http://bitbucket.org/springmeyer/tilelite/get/tip.zip
unzip -o tilelite.zip
# setup offline demo.. with openlayers-local.htmlcd tilelite/demo
wget http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js
wget http://openlayers.org/download/OpenLayers-2.9.zip
unzip OpenLayers-2.9.zip
# grab spreadnik
svn co http://svn.openstreetmap.org/applications/utils/spreadnik/
# if we were to need the raw files rather than my bundle...
13.4. Ubuntu Setup 39
-
7/23/2019 Osm Rendering Foss4g2010
44/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
# mapnik styles
svn co http://svn.openstreetmap.org/applications/rendering/mapnik stylesheets
# shapefiles needed by styles
cd stylesheets
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/10m
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/11
wget http://tile.openstreetmap.org/world_boundaries-spherical.tgz
wget http://tile.openstreetmap.org/processed_p.tar.bz2
wget http://tile.openstreetmap.org/shoreline_300.tar.bz2
cd ../
## WINDOWS ##
# additional windows xp software for foss4g pre-loading
mkdir software
mkdir software/windows
cd software/windows
# grab version of VBox used (and tested) with the VM:# (actually only 3.2.2 was tested on Mac OSX 10.6, but this is the closest version for window
wget http://download.virtualbox.org/virtualbox/3.2.4/VirtualBox-3.2.4-62467-Win.exe
# postgis
wget http://downloads.enterprisedb.com/postgresql/postgresql-8.4.4-1-windows.exe
wget http://postgis.refractions.net/download/windows/pg84/postgis-pg84-setup-1.5.1-1.exe
# python 2.5
wget http://www.python.org/ftp/python/2.5.4/python-2.5.4.msi
# mapnik
wget http://dbsgeo.com/mapnik/Mapnik_Installer.exe
# qgiswget http://linfiniti.com/downloads/QGIS-1.4.0-1-No-GrassSetup.exe
# plugins from http://qgis.dbsgeo.com/
wget http://dbsgeo.com/quantumnik/0_3_7/quantumnik.zip
wget http://dbsgeo.com/osm_tools/0_1_4/osm_tools.zip
# utilities
wget http://downloads.sourceforge.net/gnuwin32/wget-1.11.4-1-setup.exe
wget http://surfnet.dl.sourceforge.net/project/gnuwin32/bzip2/1.0.5/bzip2-1.0.5-setup.exe
# skipped... spreadnik, mod_tile
## OTHER FTP UPLOADS ##cd printing
wget http://dbsgeo.com/foss4g2010/pdf/osm-rendering-foss4g2010.pdf
13.4.1 TODO: Reduce Size
https://wiki.ubuntu.com/ReducingDiskFootprint
a few commands to try to get rid of non-essentials:
40 Chapter 13. Speakers Notes
https://wiki.ubuntu.com/ReducingDiskFootprinthttps://wiki.ubuntu.com/ReducingDiskFootprint -
7/23/2019 Osm Rendering Foss4g2010
45/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
# this did nothing - actually grew... ugh
sudo apt-get remove aisleriot openoffice.org* empathy rythmbox transmission-gtk rhythmbox sud
13.4.2 FAQ:
Running your own OSM server
absolute minimum would be dual core with 4GB of RAM, but likely the weekly import taking around 24 hours
which may be too long
having multiple disks and/or more RAM may be more useful than lots of CPUs
single country: in that case DB probably all fits in RAM which makes the rendering CPU-bound
13.4. Ubuntu Setup 41
-
7/23/2019 Osm Rendering Foss4g2010
46/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
42 Chapter 13. Speakers Notes
-
7/23/2019 Osm Rendering Foss4g2010
47/51
CHAPTER
FOURTEEN
AUTHORS
14.1 Dane Springmeyer
Dane Springmeyer is a programmer and cartographer working to advance open source tools for collaborative mapping
and data visualization. Hes an active contributor to the OpenStreetMap project, a lead Mapnik developer, and works
on range of projects that use new technology to address social and environmental problems and aid in disasters.
Affiliation: Mapnik, OpenStreetMap US
Contact: http://dbsgeo.com
14.2 Ivn Snchez Ortega
Ivn is a vocational computer scientist, and has been active in the OpenStreetMap project since 2006, entering data,
developing software and, most of all, increasing awareness about the issue of map data licensing.
Affiliation: OpenStreetMap Espaa
Contact: http://twitter.com/RealIvanSanchez
43
http://dbsgeo.com/http://twitter.com/RealIvanSanchezhttp://twitter.com/RealIvanSanchezhttp://dbsgeo.com/ -
7/23/2019 Osm Rendering Foss4g2010
48/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
44 Chapter 14. Authors
-
7/23/2019 Osm Rendering Foss4g2010
49/51
CHAPTER
FIFTEEN
GLOSSARY
Mapnik2 The development branch (currently trunk) of Mapnik that has many new features but should only be used
by advanced Mapnik users until officially released.
45
-
7/23/2019 Osm Rendering Foss4g2010
50/51
OSM Rendering Toolchain - FOSS4G 2010, Release 0.1
46 Chapter 15. Glossary
-
7/23/2019 Osm Rendering Foss4g2010
51/51
INDEX
M
Mapnik2, 45