maptimeber

[maptimeber]

Data-driven Mapplications

In our June meetup we again had an exciting line up of speakers, and it was all about spatial data.

Mapping Fast Food & Discounters

Christian Kuntzsch (@DeEgge) started off by presenting his exploration of "Mapping Fast Food and Discounter Density in Germany". He used OpenStreetMap’s Overpass Turbo to collect the location data and combined it with Zensus Data. Check out his presentation PDF, read more about his project on Christian’s blog or go directly to the interactive version.

Processing Elevation Data

Next up, Soraya Kaiser (@toutfromage ) presented her "Implementation of a web-based tool for the analysis of digital elevation models". Due to their size the analysis of these datasets often proves to be time and memory consuming. The implementation of a tool that taps into the technology of tile-based web mapping is intended to provide a less time and memory consuming analysis by evaluating the topography dependent on its relevance to soil erosion.

Take a look at her presentation to learn more about her approach.

Geovisualization for Disaster Management

The third project was presented by Jordi Tost (@jorditost) who presented his project on how to analyse and communicate hazard and accessibility vulnerabilities by combining remote sensing and OpenStreetMap data in interactive visualizations. 

Beyond his research, Jordi introduced a technique for combining D3 and Mapbox GlJS, take a look at his tutorial to learn more.

The rest of his project’s work is available under an open source license as well.

Machine Learning and Geo Sciences

Machine Learning is everywhere lately, so is it in the geo community. Sebastian (@seb_meier) gave a brief introduction to "Machine Learning in Geo-Sciences". Discussing available tools and current applications of the technology.

Here are some interesting resource to get you started:

A brief history of neural nets

Deep learning 101

AI Playbook

A visual introduction to machine learning

TensorFlow Playground

Zach Lieberman & Landlines

Mario Klingemann

Convnet.js

Collection of Machine Learning Tools and Libraries

Weka - Desktop App

TensorFlow

His slides can be found here.

Source: https://m.xkcd.com/1838/

We would also like to use the opportunity to thank our sponsors, Das Büro am Draht for providing drinks and pizza and IXDS for providing us with a lovely space.

See you at our next Meetup on June 20th.

[maptimeber]

Never used the command line & where is the light?

In our April session we had a great talk about light pollution and a hands on tutorial about how to use GDAL/OGR via the command line. We started with our guest Helga Kuechly (@helsche), a Berlin & Potsdam based geo-ecologist. She explained to us, “where the light is“. Helga showed what nighttime satellite imagery can be used for and gave an introduction to her research about light pollution.

We uploaded here slides to our new 😃 slideshare account: https://www.slideshare.net/MaptimeBER/where-is-the-light-helga-kuechly-maptimeberlin

Several interesting nighttime image data sources and visualizations were mentioned in her talk.

Helga for example demonstrated a comparison between lamps in Berlin. Even nowadays, decades after the fall of the wall, clearly the former east-west division of Berlin can be seen  from sky.

Very impressive is NASA´s black marble visualization. Black marble images can be found at: http://www.space.com/19049-black-marble-earth-night-photos.html

For anyone who planes to work with nighttime data, Helga recommend some very interesting URLs:

- Berlin Mosaic 2010: http://dx.doi.org/10.1594/PANGAEA.785492

- DMSP & VIIRS data: https://ngdc.noaa.gov/eog/download.html

- Nightpod ISS data: https://eol.jsc.nasa.gov/

-Loss of the night app: http://www.verlustdernacht.de/Loss_of_the_Night_App_engl.html

It should not be unmentioned, that she and her joint researchers did some interesting publications in the field of light pollution:

Light pollution literature database: http://darkshy.org/resources/research/alan-database/

1. Kuechly, H. U., Kyba, C. C. M., Ruhtz, T., Lindemann, C., Wolter, C., Fischer, J., & Hölker, F. (2012). Aerial survey and spatial analysis of sources of light pollution in Berlin, Germany. Remote Sensing of Environment, 126, 39–50.

2. Kyba, C. C. M., Garz, S., Kuechly, H., de Miguel, A. S., Zamorano, J., Fischer, J., & Hölker, F. (2015). High-resolution imagery of earth at night: New sources, opportunities and challenges. Remote Sensing, 7(1), 1–23.

Helga works for the LUP GmbH (http://www.lup-umwelt.de/en/forschung-entwicklung/) located in Potsdam and as a freelancer, e.g. for NGOs like WWF, in the field of nature conservation.

Next to Helga Harald Schernthanner (@hatschito) and Nico Bellack (@bellackn) did a hands on tutorial about the command line in general and its great use, when combined with GDAL/OGR, the geospatial data abstraction library, which is  probably the most well known program library when it comes to geospatial applications. The word cloud below shows where GDAL/OGR is inside.

The final outcome of their tutorial was a heat map, showing all ice cream shops in Berlin, completely assembled just in the command line. Here you can find the slides of their talk: https://www.slideshare.net/MaptimeBER/gdal-ogr-intro2542017 They started with the very powerful wget command. A command to retrieve content from web servers. Wget was used to filter and download OpenStreetMap data. The following request retrieves all ice cream shops in Berlin via wget and the Overpass API: https://overpass-api.de. By the way, another great tool to filter the OpenStreetMap is Overpass Turbo: http://overpass-turbo.eu/

Next you can see the command to download Berlin's ice cream shops: wget -O ice_cream.osm https://overpass-api.de/api/xapi? node[amenity=ice_cream][bbox=12.93,52.45,13.71,52.65] Here we explain point by point, how the command is constructed. In the slides you can find further commands, explained point by point. 1. Wget = command to retrieve files via HTTP / HTTPS / FTP

2. -O = write output to document

3. URL = Url to the Overpass API

4. [amenity = ice_cream] = OpenStreetMap map features:

https://wiki.openstreetmap.org/wiki/Map_Features

5. [bbox=12.93,52.45,13.71,52.65] = Bounding box for the request in Lat / Long lower-left(SW) to upper right (NE) coordinate

Based on this first example, they turned to GDAL/OGR and at the end our we had our first shell script. Here you can see and copy our final small script for your own purpose. Feel free to copy & paste 😃.

######################################################

Our First Shell Script

#######################################################  #Shebang#!/bin/bash 

#wget download data via the Overpass APIwget -O ice_cream.osm https://overpass-api.de/api/xapi?node[amenity=ice_cream][bbox=12.93,52.45,13.71,52.65]

#Convert .osm to shape fileogr2ogr -f "ESRI Shapefile" ice ice_cream.osm -explodecollections

#Reproject to UTM 32ogr2ogr -t_srs EPSG:25833 icecream_shops_UTM33.shp ice/points.shp -overwrite

#Grid the datagdal_grid -l icecream_shops_UTM33 -a count:radius1=1000:radius2=1000:angle=0:min_points=0:nodata=0 -ot Float32 icecream_shops_UTM33.shp ice_cream_shops_4000_4000_mean.tif

To initiate the script, a so called Shebang has to be written: #!/bin/bash To execute the script, store it as .sh file and run it  in the terminal via the command bash + scriptname.sh  Finally, we get an almost automatically derived ice cream heat map of Berlin. Just some styling had to be done in QGIS.

Our Maptime ice-cream shop heat map

To round up our session, Nico briefly showed a web-based GDAL application prototype, he built during a study project at the University of Potsdam, automatically interpolating air pollution data of Berlin‘s BLUME sensor network. Below you can see a flowchart of the work-flow of his prototype:

The data from the Blume network can be downloaded here: http://www.stadtentwicklung.berlin.de/umwelt/luftqualitaet/luftdaten/index.shtml Harald works in the field of GIS and remote sensing at University of Potsdam and co-organizes MaptimeBER. Nico is a M.Sc. Student in the field of geoinformatics at University of Potsdam.

[maptimeber]

Mapping new frontiers

In our September Session we once again had a great lineup of speaks. First Numa Gremling (@Gremling89), working for geoSYS and teaches at gis-trainer.de, gave an interesting workshop and introduction to Turf.js:

Maptime Berlin Demo Data (340 KB) Maptime Berlin Presentation (1.7 MB) Turf.js Dashboard Demo

Andrea Naß working at the Germany's national aeronautics and space research centre (DLR).  http://www.dlr.de/pf/ gave an exciting talk on planetary cartography:

Presentation Andrea Naß

Last but not least, Nermin El-Sherif presented her work at the German University in Cairo and gave us an insight into her Master thesis that she is currently working on: http://nerminelsherif.wix.com/cairo

[maptimeber]

Visualizing Urban Data

For our latest meetup, we had the pleasure to welcome Till Nagel (@tillnm), Jonas Parnow (@zeto) and Sebastian Meier (@seb_meier).

Jonas Parnow is a freelance visualization expert and part time working for Golden Section Graphics. He introduced some of his latest visualization work and gave us a peak behind the scenes. Here are some of the Tools he talked about:

Gephi, D3, React, ScrapeSimilar

And of course Jonas’s helpful toolkit:

City Finder Merge Facet

Till Nagel is a guest professor at the University at Burg Giebichenstein. He is specialized in interactive (geo-) data visualization. One focus of his are interactive exhibits, one of his latest works cf - city flows, was exhibited at the Streams and Traces exhibition.

IN addition to the inspiring projects, Sebastian gave quick introduction to Postgresql and PostGIS:

Introduction to PostGIS (& Postgresql)

When you are working with GeoData, Maps and Geovisualization, at some point you will find yourself in the need of using a database, a database that supports spatial features. Either the data set you want to use comes as a Postgres dump or your data is just to complex and big to be handled in a json or csv file.

Postgresql (alternate documentation) and Postgis are handy combination to work with large spatial datasets. Many webservers support it natively and there are also comfortable installers for local setups: Mac http://postgresapp.com/de/, Windows: https://www.postgresql.org/download/windows/. If you want a GUI to access and query your data you can either use pgAdmin or on Mac Postico.

Before we can start you should have gdal and osm2pgsql installed: http://cl.ly/0j0E0N1J3z0z

Before we can import any data, we need to create a new database, to hold the data, you can use pgsql from command-line or use one of the previously highlighted GUIs:

CREATE DATABASE maptime;

Now we add the spatial capabilities by adding the PostGIS extension:

CREATE EXTENSION postgis;

Okey, we are all set, from here you can create spatial data and query it.

A good way to start exploring Postgres and Postgis is to download a OSM dataset and import it into our postgres server. As we are in Berlin, lets head over to Geofabrik.de and download the Berlin area: https://download.geofabrik.de/europe/germany/berlin-latest.osm.pbf

osm2pgsql -d NAME_OF_DATABASE /DIRECTORY/YOUR_OSM_FILE.pbf

This might take a while....

Now we have some data to work with. Let's take a look at the planet_osm_polygon table for a start. This table contains all polygons. The actual geometry of each item in the table is stored in the column "way". You will notice that the spatial data is stored in a binary format, which means you cannot directly read it (at least most of us cannot, i guess).

So the first command you want to remember is ST_AsText(column_name). This will return the column in a readable format. And if you want to take this data and put it in a web-map, an even more useful command is ST_AsGeoJson(column_name), which will return a well formatted geojson string, that you can use in your web-map.

If you want to insert additional data into your table, the easiest way is using the ST_GeomFromText command, which takes a string and converts it into geometry data, e.g. 'LINESTRING(-71.160281 42.258729,-71.160837 42.259113,-71.161144 42.25932)'; http://postgis.net/docs/manual-1.4/ST_GeomFromText.html

Before we start create a little more complex queries and use spatial functions, another important thing in postgis are projections. As we all know sptial data comes in a variety of projections, postgis natively support a bunch of projections, but if you are in need you can even add custom projections into the "spatial_ref_sys" table. This means every geo-column always has a projection, normally when you create a table manually you would define the projection. Here comes a first complication, you might think great the column knows it projection so i am safe, well... the projection is defined, but postgis simply expects that all data in the column, all data inserted into the column etc. has the projection, if the data has another projection it will still think the projection is set and everything will become messy, so it is really important when importing data etc. you need to convert it into the correct projection. And this job can easily be done by Postgis:

To force a column (independently from the projection set for this column) into a certain projection we can use ST_SetSRID(column_name, SRID_ID). We can then use ST_Transform(column_name, SRID_ID) to change the projection, all transformations are calculated by PostGIS. The great thing about those two function is, that we can also compare data from two tables (or columns) that are present in two different projections and PostGis will take care of the rest.

Lets look at some of those comparison/query features of PostGIS:

Probably the easiest one and most used feature is ST_Distance and ST_DWithin. ST_Distance simply calculates the distance between two features. ST_DWithin does the same and directly checks if distance is smaller than value X and returns true or false. Using this we can get all buildings that are near point X:

SELECT way FROM planet_osm_polygon WHERE building = 'yes' AND ST_DWithin(way, ST_Transform(ST_SetSRID(ST_GeomFromText('POINT(13.42277 52.49658)'),4326),900913), 100)

Note: the number defining the distance follows a weird logic, depending on the projection, it sometimes is meters, sometimes miles, sometimes something else, so be careful what to expect.

These kind of functions are useful, but you could almost calculate this manually, things that are more interesting are functions that combine two columns. For example ST_Contains, ST_Intersects, ST_Overlaps etc.

Those features allow you to find polygons, lines or points that intersect for example.

Especially for intersects and these kinds of things there are functions that can help, e.g. ST_Buffer, which creates a buffer around a point, a line or a polygon, allowing you to find e.g. points that are in a certain catchment area around your object.

So here is the IXDS building: 

SELECT ST_AsGeoJson(ST_Transform(way,4326)) FROM planet_osm_polygon WHERE building = 'yes' AND ST_DWithin(way, ST_Transform(ST_SetSRID(ST_GeomFromText('POINT(13.42277 52.49658)'),4326),900913), 20)

You can check by copying the code to geojson.io

Lets build a buffer around that building:

SELECT ST_AsGeoJson(ST_Buffer(ST_Transform(way,4326),0.0001)) FROM planet_osm_polygon WHERE building = 'yes' AND ST_DWithin(way, ST_Transform(ST_SetSRID(ST_GeomFromText('POINT(13.42277 52.49658)'),4326),900913), 20)

Lets now build a query that selects all the buildings nearby...

Get the geometry of the building:

SELECT ST_AsText(ST_Buffer(ST_Transform(way,4326),0.0001)) FROM planet_osm_polygon WHERE building = 'yes' AND ST_DWithin(way, ST_Transform(ST_SetSRID(ST_GeomFromText('POINT(13.42277 52.49658)'),4326),900913), 20)

And voila:

SELECT ST_AsGeoJson(ST_Transform(way,4326)) FROM planet_osm_polygon WHERE building = 'yes' AND ST_Intersects(way, ST_Transform(ST_GeomFromText('POLYGON((13.4222080327773 52.4962812870821,13.4221899276404 52.4962886926653,13.4221736105618 [...] ,13.4222080327773 52.4962812870821))', 4326), 900913))

Normally you would of course simply query the object directly and not copy paste the geometry, this is just to make it simpler.

Beyond this query features, postgis also offers a lot of features for calculations from ST_Distance to ST_Area to ST_Length and much more.

For more introductory content, check out mapbox tutorials for linux, windows and mac: https://www.mapbox.com/tilemill/docs/guides/osm-bright-mac-quickstart/

http://geojson.io/

https://eggerapps.at/postico/

http://postgresapp.com/

And the node.js script for connecting to your postgres database:

var Client = require('pg-native'),    http = require('http'),    express = require('express'),    app = express();

var client, pg_conf = { database:'maptime', user:'YOUR_USERNAME', password:'YOUR_PASSWORD', port:5432, host:"localhost", ssl:false };

client = new Client("postgres://"+pg_conf.user+":"+pg_conf.password+"@"+pg_conf.host+"/"+pg_conf.database); client.connectSync();

function locations(req, res){    res.header("Access-Control-Allow-Origin", "*");

   var json = [];    var rows = client.querySync("SELECT ST_AsGeoJSON(geom) FROM planet_osm_polygon LIMIT 1");    for(var i in rows){        json.push((JSON.parse(rows[i].st_asgeojson)).coordinates);    }

res.send(JSON.stringify(json)); }

//Setup HTTP Server (to handle external requests / communication) app.set('port', 10066);

//pages app.get('/locations', locations);

http.createServer(app).listen(app.get('port'));

[maptimeber]

Imaginary Maps

At our last session we had the pleasure to welcome Catalina @qriousvisualist. Berlin based illustrator and visualist Catalina gave an introductory presentation to her approach and perspective, before leaving our digital tools behind and getting hands on with an analog mapping workshop. Which led to an amazing set of personal cartographies of the city of Berlin.

Afterwards Sebastian from the infographics and mapping department of Bild.de @infobild gave us an insight into their everyday work life. While many projects presented at Maptime are in the makings for days if not weeks, it was interesting to see how the busy news life changes the way maps and graphics are produced.

Last but not least, another Sebastian @seb_meier shared some insights gathered during his latest project, which was a collaboration with Christopher Pietsch (@chrispiecom) and Till Nagel (@tillnm). They created the project http://www.breitband-berlin.de for the Technologie Stiftung Berlin (@TSBBerlin). We covered scraping technologies already a couple of times, Sebastian shared his experience scraping WMS image data and using Node.js + imagemagick to process large raster data files and turning them into raw data again. On the note of vector vs. pixel basemaps, they showed a usecase where pixel might actually be better than vector, e.g. when you need high details at low zoom levels, which would otherwise be too much data to load or too much processing power on the client side of things. 

High Level of Detail at low zoom level => PixelMaps

Generalize data at low zoom levels => VectorMaps

All the code behind the visualizations is available on github: https://github.com/technologiestiftung/breitband

[maptimeber]

Maptime visiting the German states map collection: Impressions of our guided tour

Author: @hatschito

Maptime BER has been invited to a guided tour to the German´s state library map collection, which is one of the most renowned map collections worldwide. The collection is located in the hoary historical building of the German´s state library, located at “Unter den Linden”. Due to space limitations only a group of 20 Maptimers could join the tour. We met at the back entrance of the library, where we were welcomed by our guides Mrs. Mittenzwei and Dr. Heinz. As cartographers and historians they are members of the scientific committee of the map collection. After short instructions (i.e. no camera´s flash and laptops allowed), we were guided to the elevators, taking us deep down into Berlin´s map vaults, where more than 1.3 million maps and atlases are stored. Usually  there is no public access to the map vaults, so we were extremely grateful to get this exclusive access to the archive.

Entering the map vault via two alarm secured locks

The map collection concentrates in the acquisition of atlases from the 18th century until today, on the collection of topographical maps and has a special focus on globes. In the vaults a constant temperature of 18 degrees Celsius has to be maintained, in order to archive the hidden treasures. Maps are stored in card cabinets, where they are laid down as plain sheets, as Mrs. Mittenzwei explained to us.

Plan laid map sheets

The job of Mrs. Mittenzwei and Dr. Heinz has turned them into passionate (map-)treasure hunters and from time to time they even have to attend auctions of maps and atlases.

As we learned inspecting some of the awesome globes,  the worlds biggest globe factories “Columbus” is located in Germany: http://www.columbus-globes.com/en/home/

We were especially thrilled about the  old inflatable leather-globe that we were shown

Dr. Heinz explaining the inflatable globe to Sebastian  ☺

Patrick standing next to a mobile card cabinet

Of course the question of white gloves was asked, but our  friendly guides told us, that white gloves only are used for show purposes. For example, the archduke of Luxemburg visited the map collection once and during his visit white glooves where used ☺.

Down in the map-vault we had really interesting discussions about the digitization of analogue maps. Digitization is a hot topic among the employees of the map collection, but many different aspects have to be considered and there seems to be no consensus on how this could be achieved soon. . For example, the incredible high number of analogue maps is one of the obstacles of scanning the historical maps. Another issues is, how the special character of an historical map, printed on parchment can be maintained in the digital age? Apparently, even Bill Gates has visited the map collection at one point and proposed to help digitize the maps, but his digitization strategy was not mature enough at that time.

One of the highlights of our tour was a 18th century atlas from a venetian manufacture, including a beautiful „Portolan“, a map drawn for the purpose of ship navigation.

Mrs. Mittenzwei explaining a venetian 18th  century atlas

Deep down in the map-vault

After visiting the map vault, we went to the beautiful map reading room, that is open to the public and where scientist and the interested public can work and explore different maps. Mrs. Mittenzwei explained some aspects of map search. For example, if you search for a map in their collection, you can use the so-called “Stabikat” online search tool: http://stabikat.de/

The problem is, that this search engine is designed for literature and the majority of maps do not have an author. The employees of the map collection support you in the search for  a map. Requests have to be as concise and correct as possible, in order to give perfect support. Even digitization can be requested for a fee of 10€ per requested map sheet.

The map reading room includes the whole analogue register of the map collection, has a section of literature about how to use and work with maps, even sections of no more existing states is part the collection.

No more existing states in the territory of Thuringia and Saxonia

At the end of our great 3-hourstour, some special pieces stored in the map reading room were  presented to us, for example a map of the no more existing Berlin fortress (where nowadays the Brandenburg gate is located) or very special map sheets of the first Prussian land survey, mapped for military and urban development purposes.One of these historical maps, the so-called “Schmettauische Karte”, is already completely accessible online and can be viewed using the “Brandenburg viewer”. Even an integration to a GIS software via WMS is possible: http://service.brandenburg.de/de/schmettau_online/307519

We spent a great and interesting time in Berlin’s map vaults! Again, thanks a lot to Mrs. Mittenzwei und Dr. Heinz for the great tour!

Map collection of the Germans state library: http://staatsbibliothek-berlin.de/die-staatsbibliothek/abteilungen/karten/

[maptimeber]

Maptime 2015-11 — Vector Tiles

Vector Maps

Creating your own Vector Maps

The former goto tool for creating web maps was TileMill. TileMill was creating image based tiled maps. Since most map hosters / providers are switching to vector maps MapBox has released an update of their map creation software called MapBox Studio. Sadly the latest release is an online tool, that is very much integrated into the whole MapBox ecosystem. But luckily there is MapBox Studio Classic, which will still allow us to create vector maps that we can use independently from the whole MapBox ecosystem and its open source on GitHub.

You find useful tutorials and advice on how to use MapBox Studio Classic on MapBox's website and elsewhere. If you need a simple data set to get going, here is a set of geojson files for Berlin's city districts. To import it, goto Styles & Sources > New Style or Source > Blank Source > New Layer > Select one of the geojsons from the ZIP file. Afterwards you might wonder what happened. You just need to zoom in, use the search box in the upper left and type in "Berlin" it will guide you to Berlin and now you should see the data you imported. Next you can save your new source, here you will also find an option to export as mbtiles. A big difference in vector tiles is that styles and data are separated.

If you want to take a look inside your mbtiles, use mbutil with this command:

mb-util YOUR_MBTILES_FILE.mbtiles DESTINATION_FOLDER

So after exporting your mbtiles you can start creating your own styles.

Thanks to some node.js packages (and now even a php library) you don't need MapBox to host your vector tiles. The actual hosting script is pretty straight forward, you just need to sort out the dependencies for the various required packages.

require('sqlite3').verbose();

var express = require('express'); var http = require('http'); var app = express(); var tilelive = require('tilelive'); require('mbtiles').registerProtocols(tilelive);

tilelive.load('mbtiles://'+ __dirname +'/tiles/Berlin_Districts_1.mbtiles', function(err, source) {

   if (err) {        throw err;    }    app.set('port', 10060);

   app.use(function(req, res, next) {        res.header("Access-Control-Allow-Origin", "*");        res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");        next();    });

   app.get(/^\/v2\/tiles\/(\d+)\/(\d+)\/(\d+).pbf$/, function(req, res){

       var z = req.params[0];        var x = req.params[1];        var y = req.params[2];

       console.log('get tile %d, %d, %d', z, x, y);

       source.getTile(z, x, y, function(err, tile, headers) {            if (err) {                res.status(404)                res.send(err.message);                console.log(err.message);            } else {              res.set(headers);              res.send(tile);            }        });    });

   http.createServer(app).listen(app.get('port'), function() {        console.log('Express server listening on port ' + app.get('port'));    }); });

More detailed explanations on how to host vector tiles can be found here:

http://gis.stackexchange.com/questions/125037/self-hosting-mapbox-vector-tiles http://www.scriptscoop.net/t/f486084bb4d1/self-hosting-mapbox-vector-tiles.html https://github.com/klokantech/vector-tiles-sample

Now that we got our tiles hosted on our server (or mapbox). We can start putting it on a website or in an app or what ever you plan on doing with your new basemap. Eventhough under the hood mapboxgl.js is quite different from the usual leaflet (alike) systems, the integration itself is very similar to what you know from common image based tile maps.

But as these are vector tiles you can do some cool stuff that you were not able to do with image tiles, like panning the map to create a 3d like appearance or rotate the map.

Here is an example GIST that should get you going.

The HKW is working on map project by refugees for refugees and is look for developers and designer. If you are interested get in touch with them. 

Jan Lindenberg @janlindenberg presented his exciting mapping projects:

Radiation Map for Japan

His Baumscheiben project in the Donau-Kiez in Neukölln.

And his Berliner Berge (Berlin’s Mountains) project:

Join them on facebook to climb the next mountain.

Last but not least Heike Otten @heikeotten_ and Lennart Hildebrandt @len_hil presented their project shifted maps which visualizes your personal traces using data from the moves app. Some people asked if it is possible to use some other kind of data. Right now this is not possible, but if you are interested, take a look at the shifted maps repo on github and write your own data importer.

Lennart also gave as a fast forward introduction to react.js. React is a javascript framework that helps you build complex web applications with a focus on the UI. You can find Lennart’s introduction as well as all the slides on Shifted Maps on SlideShare:

http://de.slideshare.net/HeikeOtten/shifted-maps-presentation-at-maptime-berlin-november-2015

You made it all to the end. So here is a little gift. Our December session will be something special. We have organize a guided tour through the maps section of the Staatsbibliothek which will guide us into the hidden vault of the Stabi.

We only have 20 seats for this tour so register now. We will inform everyone by the end of the week if you made it into the tour. Please only register if you really plan on coming, as seats are quite limited.

[maptimeber]

October 2015

Thanks again to Sascha Frohwerk (@saschafrohwerk) for his insightful talk on web scraping. Sascha presented his experience from more than three years of scraping data from websites. His technology stack builds upon python:

- Request Library - Beautiful Soup - JSON Library

- HttpFox (analyze files being requested by a website) - VisualJson (mac app for json prettifying) - MapQuest (routing, alternative OSMRouting) - Unfolding (Library for the visualizations)

If you are not a big python fan, there are several php and node.js libraries on github and some people have even written crazy tools to make use of google spreadsheets for scraping. Besides a hands on web scraping session, Sascha also showed us the insights he and his colleagues at civity extracted from scraping years of car sharing data. If you are interested check out their website or take a look at their collaboration with the mappable guys.

During the Q&A we had two things that i want to quickly follow up on. Someone asked about floating car data and the data available on car sharing cars. The data available on the web is usually the data that is required to build and populate the online interface, which more or less comes down to geo-location, some kind of identifier e.g. license plate and some car descriptors e.g. size, has navigation system, etc. Floating car data (FCD) or Extended floating car data (XFCD) is a term that often describes the data that is being collected by all the sensors a car has, which is a lot, really a a lot nowadays, which is then often transfered to a central server unit. Most modern cars have a port to access some of this data. You can even buy adapters to send this data to your smartphone and log the data.

Furthermore Harald Schernthanner (@hatschito) gave us a quick intro to the awesome osgeolive tool box. Nowadays virtualization has become a popular solution for creating secure and stable working environments. Especially if you are working with many machines that have no tools preinstalled, osgeolive takes care of worrying about dependencies and all the other stuff that steals your time and stops you from getting to work on your next geo project. And of course everything in osgeolive is open source. If you are interested in open source geo tools make sure to check out foss4g the biggest community on open source development of geo tools. By the way their next big conference is in Bonn, Germany next year.

Sebastian Meier (@seb_meier) showed some d3 tools for creating more advanced geo visualizations. D3.sketchy allows you to create visualizations with a hand drawn style, which can be of special interest if you are dealing with uncertainty visualizations. This tool can easily be combined with slippy maps, as Mike Bostock explains here. 

The second tool d3.layout.odmap is looking at more abstract but more insightful geographic representations. The tool allows to quickly generate various origin-destination visualization to show for example commuting behaviours. 

Last but not least Till Nagel (@tillnm) joined us to announce the exhibition of UCLab’s mapping exhibition, which opens upcoming Thursday night at the erstererster gallery in Prenzlauer Berg. The map projects will be on display for a week, so don't miss out.

Our next event in November will take a closer look at vector-maps and the mapboxGL.js library.

[maptimeber]

#9 - D3 & Maps

Thanks everyone for joining us on this warm summer night. In our 10th session we gave a short intro to making maps with D3, the deixis team present a map snippet editor and Alsino talked about his time at the OSM conference “state of the map US”.

First up D3. Most frameworks we presented so far are creating so called slippy maps. Those techniques can be found in commercial solutions such as Google, Bing or Mapbox as well as their open source counterparts leaflet and open layers. Each of those javascript frameworks / libraries allows you to embed a slippy map into website and connecting to a tile server. This is great because we can visualize the whole world in various zoom levels and detail, without the need to load everything at once, but we NEED a tile server.

With D3 we can build simple maps without the need for a tile server. But we still need data. D3 can handle geojson data as well as topojson through the topojson extension. We strongly recommend that you take a look at topojson as its a highly compressed version of geojson and therefore... well better. (TopoJson Command line tool)

You can visualize the geojson data in a wide range of projections (and even more).

You can even make it interactive, add zoom and pan features and so on.

And if you want to combine a slippy map and a complex d3 visualization you can even do that.

Here is a link to all the examples Seb showed throughout the tutorial.

If you are now interested in learning d3 we recommend you take a look at the d3 website. Mike Bostock has written a lot of tutorials and you find a long list of examples on the website as well, which you can pick up, add you data and modify it.

Thanks also to deixis for presenting their tool, which aims at helping multidisciplinary teams to work together on web-based map projects.

And if you are interested Alsino’s adventures at “state of the map US”, here is his presentation.

[maptimeber]

Recap: Maptime #8: Mapping housing & rents

So, after quite some time now, we finally managed to put together a thematic session on “Mapping housing and rents” in our last session on 16 May. We have been wanting to do this session for quite some time, as rising rents and housing issues are probably the most heated debate topic in Berlin at the moment and probably for some time to come. Coincidentally, the session took place only one day after the new official biannual rent index for Berlin, the ‘Mietspiegel 2015′, was released. This fact turned out to fuel the debate quite a bit after the lightning talks. Once again, we were kindly hosted by our friends from IXDS, who have been letting us use their beautiful space for quite some time now. Thanks guys! Also, we had free drinks this time, sponsored by our partners from HERE...Yeay!

Our lineup for the day was as follows: Harald Schernthanner did a nice presentation on why and how rental prices for appartments and houses can be estimated. He explained the difference between spatial and non-spatial estimation techniques and showed some of the visualization results of his current Phd research: His presentation can be found here: http://bit.ly/1AhAOuo. Next up was Stephan Plänitz, who presented his current prototype visualization and interface, which can be used to find yourself a new flat. His presentation is available here: http://bit.ly/1Be7zDu.

The third presentation was about the project airbnbvsberlin.de by Alsino, Jonas and Lucas, which had received a bit of attention in the German rent debate in the last few months. Besides showing off the maps featured in their work, they explained the idea, process and intention behind the project, as well as shared some insights on how it was received by the media and the public. Their presentation can be found here: http://bit.ly/1HwdqJh

Airbnb vs Berlin (airbnbvsberlin.de)

After that, Sebastian showed-off a kool 3D-map project, which he has made with three.js., a browser-based library for creating kool 3D graphics using Berlin buidling data in geojson. It will be exciting to see how this technology will be used for creative mapping in the future. See the live demo here: https://github.com/sebastian-meier/THREE.geojson. Other links:

http://threejs.org/ http://vr.chromeexperiments.com/

Geojson in three.js (by Sebastian Meier)

#maps #maptime #gis

[maptimeber]

And of course thanks to Martin for his presentation on the Isoscope Project he did in collaboration with Nokia Here. You can find is presentation-slides here:

https://dl.dropboxusercontent.com/u/4352238/isoscope_maptimeBER_small.pdf

[maptimeber]

Geodata, Tools & Sources

Summary of the 17.02.2015 event

GEODATA FORMATS

Shapefile

- electoral districts „Bundestagswahlen“: http://bit.ly/17iOk2A - Oregon GEO Spatial Data Library: http://bit.ly/orgeo

GeoJSON

- maptimes.geojson: http://bit.ly/maptimegj

- To learn more about GeoJSON's specificities, check out: http://bit.ly/1Bj5qer

- Webtool: http://www.geojson.io

KML

- Google placemarks examples: http://bit.ly/1CEDe0V

OSM

- OpenStreetMap Data Export as osm-file: http://www.osm.org

GeoTIFF

- examples (satellite images): http://bit.ly/1EJPDCU

- more infos about GeoTIFF format: http://trac.osgeo.org/geotiff/

more geodata formats (overview)

- Wikipedia: http://bit.ly/1EJOs6B

- GIS Centre: http://www.giscentrum.lu.se/english/whatisgisfileformat.htm

GEODATA TOOLS

Webtools: Data validation

- JSON: JSONLint http://www.jsonlint.com

- GeoJSON: GeoJSONLint http://www.geojsonlint.com

- KML: FEED Validator http://www.feedvalidator.org

Webtools: Data conversion

- OGRE: http://ogre.adc4gis.com

- GPSVisualizer/GPSBabel: http://www.gpsvisualizer.com/gpsbabel/

Desktop / command-line tools

- GPSBabel: http://www.gpsbabel.org

- ogr2ogr: http://www.gdal.org/ogr2ogr.html

- gdal (includes ogr2ogr and many other vector and raster data tools): http://www.gdal.org/

GEODATA SOURCES

Of course there are many sources, here only three, a general purpose source:

http://www.naturalearthdata.com/

And Berlin specific sources: http://data.berlin.de http://fbinter.stadt-berlin.de/fb/index.jsp

PIXEL DATA EXAMPLE

Source: http://fbinter.stadt-berlin.de/fb/berlin/service.jsp?id=a_luftbild2014@senstadt&type=FEED

We will use the command line interface (Terminal). You need to install GDAL first, see link above. First download all the ZIP files containing the aerial photography:

wget http://fbarc.stadt-berlin.de/FIS_Broker_Atom/DOP/dop20c_2014/dop20_{368..414}_{5798..5836}.ecw.ZIP

Unzip the files into a new folder:

unzip /Users/[YOURUSERNAME]/Desktop/BerlinWMS/*.ZIP -d /Users/sebastianmeier/Desktop/BerlinWMSRaw

Go into the folder:

cd /Users/[YOURUSERNAME]/Desktop/BerlinWMSRaw

Build a virtual file format (This is virtually stitching all images together):

gdalbuildvrt MergedRaster.vrt *.ecw

Now we can finally extract any portion of the image we like:

gdalwarp -ts 5000 4406 -t_srs EPSG:4326 -s_srs EPSG:25833 -te 13.079 52.3284 13.7701 52.6877 /Users/[YOURUSERNAME]/Desktop/BerlinWMSRaw/MergedRaster.vrt /Users/[YOURUSERNAME]/Desktop/BerlinWMSRaw/ZoomView_0.tiff

-ts: Represents the pixel width and height for your image

-t_srs / -s_srs: target and source projection/coordinate system

-te: Represents the extent of the rectangle that should be exported as an image (in target coordinate system)

VECTOR DATA EXAMPLE

Source: http://fbinter.stadt-berlin.de/fb/berlin/service.jsp?id=re_einwohnerdichte2013@senstadt&type=WFS&themeType=spatial

Now we need to install the ogr2ogr tool (see above). I suggest you first download the full dataset from the fis-broker from the dataset you want:

ogr2ogr -s_srs EPSG:25833 -t_srs WGS84 -f geoJSON einwohnerdichte2013_neu.geojson     WFS:"http://fbinter.stadt-berlin.de/fb/wfs/geometry/senstadt/re_einwohnerdichte2013" re_einwohnerdichte2013

If you just need a certain region you can now easily extract a rectangle or even shapes from the dataset. The fis-broker server is very slow so doing this next step locally is a lot faster:

ogr2ogr -s_srs WGS84 -t_srs WGS84 -spat 13.411557 52.487432 13.434817 52.500679 -f geoJSON einwohnerdichte2013_small.geojson einwohnerdichte2013_neu.geojson

And of course the link to Moritz example:

https://github.com/moklick/howto-fisbroker

[maptimeber]

[maptimeber]

CartoCSS & Mapbox Studio

In our last session we were looking into CartoCSS and Mapbox Studio. Mapbox Studio is the follow up tool of TileMill. The big difference between the tools is the tiling-technique: TileMill generates Pixel-Tiles and Mapbox Studio is using Vector-Tiles. One reason for us to choose Mapbox Studio is that its mostly open source. And you don't need to use the Mapbox service, you can also host the tiles yourself. You can also use the tiles to create mobile applications that offer offline maps. 

If you want to create maps to host on you own server, you need to get your own data into Mapbox Studio. For street data, i suggest you use OpenStreetMap data, for example from the geofabrik. Mapbox has a nice tutorial that explains how to get the data into Mapbox or Tilemill.

If you don't want to use MapBox's mbtile files you can also extract pngs from it with this tool here.

Other interesting sources for geo-data:

http://www.naturalearthdata.com

http://www.stadtentwicklung.berlin.de/geoinformation/fis-broker/

http://daten.berlin.de/

While going through the google results on ios and mbtile i discovered that i wrote an example for integrating mbtiles into ios apps a while back: http://sebastian-meier.github.io/OfflineMaps/index.html

And also thanks again to IXDS for hosting us and for the great presentations by Christian and Frank (Bettermaps).

[maptimeber]

Recap: Maptime No.3: Leaflet.js Intro, Stamen, HERE and more

So wow, this was actually already our third Maptime Berlin session on 4 Nov 2014. Who would have imagined? After our first two sessions at the Supermarkt in Wedding (thanks again for having us!!) we made our way back to Kreuzberg. This time IXDS had offered to be our host for the night and let us use their amazing open and cosy space in one of the industrial courtyards right along Paul-Linke Ufer (between U8 Kottbusser Tor and U8 Schönleinstrasse). We actually liked the location so much that we might stay there for a while ;)

We kicked-off at around 7pm with a short introduction about the session's roadmap. Unfortunately, already at this point we realized that the internet connection would not be our best friend that night. Already on loading the online-presentation, we came to realize that the connection would just not be able to handle that many people at once. "Not again...", I thought, remembering the shame of our previous internet fail at the previous session.

We started despite this inconvenience with our planned introduction to webmapping using the leaflet.js library. Harald began by explaining the fundamentals of mapping APIs and the different uses and providers. He then lead through a few hands-on exercises, explaining the different functionalities of the leaflet.js library. He bravely managed to run this tutorial despite the internet problems, at times even switching between his own computer and someone else's macbook ;). However, at the end most of us did manage to place a first marker on a map (YES! ;), which was also the intended result of this tutorial.

The second part of the night was all about inspirational presentations. Being on a stopover on her way back from the Istanbul Design Biennal to San Francisco, we had the pleasure to have Beth Schechter (Stamen Design) with us. Beth is one of the three original founders of Maptime (noooo way!!). She gave a super-nice talk about mapping as being a source of civic empowerment. Her talk was really inspirational and led to a few questions of how Maptime, and especially Maptime Berlin could contribute to that. She also showed-off some really cool Stamen projects, such as this neatly designed map on bird flight ranges in North America (http://bit.ly/1Anjlz0, gif below). 

Another one was this interesting "fake" transit-map depicting the inofficial bus routes of tech companies such as Google, Apple and Facebook in San Francisco, which have been subject of a heated debate recently (http://stamen.com/zero1/).

Our second guest was Markus Ort, Head of future map design at HERE (Nokia). Almost as per job description, Markus talked to us about the meaning of mapmaking in an institutional and business context such as HERE. He also gave us a glimpse into the future of digital maps by showing us all this impressive work HERE is doing with regard to designing 3D maps, geo-applications and navigation systems. This was real scy-fi ;)

This pretty much sums everything up for that night. Most of us enjoyed a diverse and fun session (despite the annoying internet...brrrr..which we will fix before next time -> promise) and we once again got the chance to meet new map-interested-people over here in Berlin. Afterwards, a few of us headed over to a nearby bar for drinks, where we had some more time to mingle and talk about the meaning of life...ehhhm..maps ;)

Yes, and here are all the session slides and resources:

1. Introduction presentation (Alsino; the presentation that never happened): http://bit.ly/maptimeixds

2. Leaflet.js tutorial (Harald): http://bit.ly/1zvoQv3­

3. Stamen projects (Beth) - http://stamen.com/zero1/ - http://climate.audubon.org/­ - http://prettymaps.stamen.com/­ - http://www.chesapeakebay.net/vi...­

4. Mapmaking HERE/Nokia (Markus Ort): - http://360.here.com/2014/08/29/...­ - http://360.here.com/2014/08/28/...­ - http://360.here.com/2014/10/02/...­ - http://360.here.com/2014/05/15/...­ - http://360.here.com/2014/06/23/...­ - http://360.here.com/2014/09/12/...­ - http://360.here.com/2014/05/14/...­ - https://www.youtube.com/watch?v=...­ - https://www.youtube.com/watch?v=...

A

[maptimeber]

Maptime Berlin #2 Introduction to CartoDB

[maptimeber]

Visualizing european migration data

During the european CodeWeek (codeweek.eu) MapTime Berlin (maptimeber.tumblr.com) and CartoDB (cartodb.com) joined forces to introduce beginners to visualizing spatial data with CartoDB. The CodeWeek is a non profit event across Europe mobilizing researchers and practitioners to share their knowledge in the local communities. As a sample use case for visualizing data, the MapTime team gathered migration data from EuroStat (http://epp.eurostat.ec.europa.eu/portal/page/portal/eurostat/home). CartoDB, as a tool, was chosen to explain participants, that visualizing spatial data is a powerful story telling tool. The big advantage of CartoDB is it's easy to use web interface, that doesn't require any coding skills. The low gateway hurdle allowed us to quickly introduce the visualization process to beginners. Within the two hour hands on workshop we were able to upload, format and visualize data. To highlight the power of the visual, we used the example of asylum requests for Germany. [IMAGE 1 (Positive Requests mainly from the middle east) & 2 (Negative Requests  ](http://cdb.io/1zhFjnd) With a few clicks we were able to focus on the peaks for the 2013 asylum requests and identify patterns within the data set through visual exploration.