OpenStreetMap and Pharo
1. OpenStreetMap and Pharo, part 1
We all have this issue of recovering significant map data to be able to know where to put sensors on (and obstacles, and etc...).
In a place like Jakarta, which is already well mapped, a solution is to use OpenStreetMap data (not tiles, the true "shape" data). How to to that? Just try the Overpass query language with the IDE here:
The language reference is there:
Ok. What can be done with overpass turbo? Enter Kampung Melayu as a search term, and select: Kampung Melayu, Daerah Khusus Ibukota Jakarta, Jawa, Indonesia.
It will center the map on Kampung Melayu, Jakarta. Zoom out a bit and you will see the Ciliwung river and a lot of small houses along Gang Pulo 1, Gang Pulo2, Gang Pulo 3 ...
Now, on the left pane, we can start selecting data: Let's try to select Gang Pulo 3: it should be a way with name "Gang Pulo 3". Try with this query:
Run it: You should see a blue line along Gang Pulo 3, and, in the data tab, a long json file containing a way, named Gang Pulo 3, and many nodes with their position.
Now, how to get the Ciliwung river? We need to understand how OpenStreetMap categorize things, and for that we can use OpenStreetMap.org itself.
Open http://openstreetmap.org, search for Ciliwung, select Ciliwung River. You see on the left that it has an attribute waterway=river (and that it is a way).
Go back to overpass-turbo, change the request so that we look for the waterway river with
<has-kv k="waterway" v="river"/>
We see the river drawn in blue... http://overpass-turbo.eu/s/58h
Part 2 coming tomorrow...
2. OpenStreetMap and Pharo, part 2
It's now time for the part 2 of our OpenStreetMap experience. So we know how to access a street and a river on our chunk of the Jakarta map, and we get the result as some json.
So, we know all the data is there. But how do we get all the different tags and names and stuff? Querying blindly in that doesn't give good results.
So we'll do a different kind of blind search, one more usefull: retrieve everything in one area and note the type of things we find. First, we will zoom a lot in the image, to reduce the number of things visible (but still keep interesting stuff, like the river, a street, and a few buildings). And we will write that query:
Everything is now displayed in blue and we have a lot of data. Let's look into the data and select interesting stuff.
I see many nodes: don't know what they are for. Higher level elements are more interesting: I see a way with tags:
Oh, this is the Ciliwung riverbank, then!
Another interesting way, a bit down:
Ok: a two level buildings, a school.
Another one: a way again:
Cool: this is a RW administrative boundary (how is that RT/RW stuff translation in english? A sub-sub-sub-district?). And look, it even has a tag "flood_prone:yes" !
(and we can probably keep on looking through that data with interest, but the last one gives me something I really want to query for!)
Let's unzoom to see Jakarta as a whole. Choose the wizard and enter the following query:
And execute the query. Yes, we have all the flood prones RW of Jakarta !
This is really cool. The people filling the data for Jakarta on OpenStreetMap have really done a great job. I didn't expect to be able to do such a powerfull query.
This is all for now; we know have fairly powerfull query abilities and json data out of OpenStreeMap. Stay tuned for part 3
3. OpenStreetMap and Pharo, Part 3
In the part 3 of this series, we will try to see if we can, via Pharo, access directly the OpenStreetMap data we want. First, with a bit of work from http://openstreetmap.org, we will select an area in Kampung Melayu.
I select more or less the same area as before (the Ciliwung river loop West of the Kampung Melayu name), get the center coordinates and select an area around that with the following coordinates
- 6.21970, 106.85923
- 6.22170, 106.86323
(They can be entered in OpenStreetMap to see the points)
Now, looking at the openstreetmap wiki, I saw that I can tap the following http address for queries:
Back on overpass-turbo.eu, I select that same area (more or less) and I use the wizard to generate a query for the riverbanks (
type:way and waterway=riverbank). Then I export, choose request and ask overpassQL, and I get the following (coordinates may not be exactly the same, I rewrote with the ones I chose above):
Now I fire up Pharo, version 3 and I will try to query directly OpenStreetMap. Where is the documentation for Zinc already? Oh, that's it: I need a ZnClient.
In Pharo, what I do is then the following:
Select this and inspect, and you will get, after a bit of time (the time needed for the request), a ZnResponse in an inspector. Choosing entity in there will give you... the osm data:
Note: the entity is long and won't be displayed entirely in the inspector.
Ok, now, what do we need: a JSON reader in Pharo. Go to the configuration browser, choose NeoJSON and select install stable. With NeoJSON, we will use the NeoJSONReader as a content reader for the ZnClient and change our request so:
And inspect the result. Tada! We now have a dictionary with all our elements nicely tagged. Inside elements, we will find our way(s), nodes, and everything...
Next step, displaying... This will be for tomorrow. Good night!
4. OpenStreetMap and Pharo, Part 4
Ok, from part 3, we had the data. Now, it's time to display it, with Roassal: the favorite tool for drawing and displaying data.
To load Roassal, do the following:
Ok. Now we're set, we have all the tools we need. But what are the things we do need now? We need to project the coordinates we have received in the OpenStreetMap data, rebuilt the object contained (the way(s)), and draw them.
To project the data, we will use the same projection as OpenStreetMap: the Mercator projection, with two blocks:
The problem is that this is a projection with coordinates between 0 and 1, and it doesn't work well when scaling in Roassal, so I spare you my experiments and tell you that we need to scale the results:
So we have our formulas. What we want to do now is to project the node coordinates. We will also create a Dictionary storing, for each node id, the projected coordinates.
And we will, for each object of type way, create a RTPolygon object with the node points, and add it to the Roassal view.
Special here: if the way is a building, then the inside color is gray, else the inside is blue (I will show you why). To finish that, we will ask the view to scale and center itself:
v canvas camera focusOnCenterScaled
Ok. Now I'll put it all together, with a slightly more complex query which retrieve the buildings and the river banks in our start area. This is a long script, enjoy!
With a screenshot, of course...