Mapping Cleveland’s Proposed Ward Boundaries of 2014

Monday March 25, 2013

Cleveland City Council President Martin Sweeney released the proposed ward boundaries for 2014. This is just one day before he presents them to be voted on in City Council.

City of cleveland issues this map. A JPEG. Not even georeferenced.
- it has no street names, all features (including rivers and railroads) are all styled the same. Nothing more.

This has sadly been characteristic of the City of Cleveland’s approach to open data, particularly spatial data…

Cleveland’s approach to open data, particularly in this instance, isn’t acceptable. Nor does it help foster a culture where civic hacking flourishes.

Great maps and other visualizations including this great slippy map of new Districts of NYC by WNYC (led by jkeefe) that inspired me to do this, shouldn’t be exclusive to the tech cultures that we usually hear (NYC, SF, CHI, Austin, SEA, PDX, on and on) or wherever Code for America stops in for the year.

Later that afternoon, Current Ward 14 Councilman Brian Cummins, had received PDF maps of most of the proposed ward and posted them on his blog.
A step above from what I had before. A Shapefile would be too much to ask.

(At least Kudos to him and my current councilman, Joe cimperman, for having twitter accounts and responding to their constituents on there. )

So I began by opening up a blank layer in JOSM, loaded in the Cleveland boundary from OpenStreetMap (less things to draw that way) and began simply tracing out the boundaries over Openstreetmap tiles. I was switching windows every couple minutes, looking at the JPEG boundary, then drawing the same lines in JOSM, repeat.

Had to be a better way, was going to take a couple hours (and it did).

(Side Question: What You use to draw geometries that you’ll later process in your maps/visualizations/analysis ? )

Behold: The georeferencing tool in qgis, which would let you load an image as a layer. This, I thought would be a shortcut. I could create the polygons of the wards by tracing right over the boundaries in the image, without hauving to switch windows.

The biggest problem was that I didn’t know the projection of the JPEG.
Unfortunately, these tutorials assume that your image is georeferenced.

To georeference in qgis, you should know what the projection of your original image is in before you start. If you don’t know, you’ll have to do some guessing and trial and error. I made a few guesses of the most popular projections (4326, 3857) and then tried several ohio ones. An hour or 2 later, none of the projections worked out.

So, I scrapped that idea, I began to draw the ways again in josm.
From there, I did my usual workflow which with I’m most comfortable into tilemill;

use osm2pgsql to convert my .osm file of boundaries (which were in the form of relations, specifically multipolygons) to load into a postgis enabled database.

(I’ve been meaning to become more comfortable with geoJSON and I would have tried to save my file in josm as json but I read there’s a bug in the json export of josm that doesn’t export relations correcty, I didn’t bother to verify this yet)

Next, I used Mapbox’s (which consists of data from OpenStreetmap) technique to create a custom map that I can use as my reference base layer.

Style my layer of proposed ward boundaries in Tilemill….

Then in mapbox.js, I simply put the two layers together. voila, as shown in my map at:

http://maps.jhfeichtnerfund.com/wards/index.html

Still more to do with this:

- finish up the documentation

- Tweak the colors
- Add in the census tracts so when a user hovers over an area, they can see the population of a particular place.
- add the existing boundaries too would be nice (as a separate layer for the online map)
- convert the 2014 ward boundaries (currently as a .osm) to a shapefile so others can use it.
Right now, it is available as an .osm in my github repo…

Follow this along in its github repo. https://github.com/skorasaurus/cleboundaries/

Life over the past 2 months

One of the big takeaways that I had from attending NACIS 2012 in October was to focus my efforts more on specific projects to more clearly demonstrate my mapping skills. So, I’ve taken
taking 2
Since December, I’ve spent most of my time on one of them, Cleveland Photography Society’s Scavenger Hunt Map

I realized that I wanted to take my maps to the next step, to offer interactivity, multi-zoom levels, I needed to learn how to code javascript (the language that mapping libraries like leaflet and mapbox.js are written).

It had taken me longer than I had planned to learn enough to reach my objectives for the Cleveland Photography Society’s Scavenger Hunt Map – http://maps.jhfeichtnerfund.com/
offering interaction (users seeing the picture when their mouse cursor is over the marking, a nice zooming action when someone clicks on the list of markers).

That said, I spent parts of December, January, and February, crashing head first into Javascript.

Initial, very brief thoughts on JS:
That there’s no standard for writing API documentation for javascript blew my mind.
As someone who’s learning, this can be frustrating at times.
However, I can understand (and don’t agree) the developer’s prospective: code becomes obsolete relatively quickly (within years), decreasing the incentive to write good documentation. Good documentation also means more than just writing enough notes for yourself so you could understand it months later. I am guilty of this myself sometimes for my projects.

- Best books so far to learn: Douglas Crockford’s “javascript, the good parts” and Marijn Haverbeke – http://eloquentjavascript.net

Also, mozilla’s js reference. https://developer.mozilla.org/en-US/docs/JavaScript/Reference

Most other online resources have been meh.

January, Cleveland has its first Open Geo meeting. Steve Mather and I organized it and it was a blast. I wrote a run down of it on my OpenStreetMap blog. http://www.openstreetmap.org/user/skorasaurus/diary/18494

In a nutshell, Cleveland was missing a place, really, a culture or ecosystem where people could informally talk about OpenStreetMap, open-source geospatial software, and other GIS-related projects [cool maps, data visualizations, analysis, etc] that they are working on. As the culture would develop, people would develop friendships, increase the size of their network, hare tips on using software, discuss geospatial news, share pointers on code, give career advice or share job leads, maybe even off or start ventures or even informal projects together. etc, all things that were missing or didn’t exist as much as they already did.

This culture isn’t going to develop overnight. But it’s starting. I’m excited to what will come out of it.

(Note to self, I do this more often, more and more things are coming into my head of things/projects that have been going on. )

How I tried to overlay a mask over multiple layers except for a specific region in Tilemill (and left with a buffalo tint)(and I liked it)

One thing about designing maps, you want to draw people to what they see (yes, ultimately the viewer will have their own subjective interpretation, but I digress…).
So, when designing a map to highlight the locations of 125 items that were to be photographed in the 2012 CPS Cleveland Photo Scavenger hunt, I wanted to focus on the contest area while giving viewers (likely participants who may have forgotten the exact boundaries or may not have done the scavenger hunt but would be at least familiar with downtown Cleveland) a little context of the surroundings.

How to accomplish this ? Well, I’m guessing it can be accomplished through a couple different ways: one, by, fading the surrounding features (this known as the buffalo tint) as shown in this demo by Bill Morris, wboykinm (he does inspiring works using Tilemill, pushes it to its capabilities, and blogs about it who does this buffalo tint over a separate base map, or by laying a gray mask over the surrounding area outside the border, on the same map.

Either of these techniques – Decreasing the opacity of the outside area or giving it a gray mask to make it a bit harder to read, drawing the viewer to the easier part to read and having the non-grayed part pop-out.

So, I started experimented in my project (based off originally osm-bright written by mapbox)

with map {
background-color: #999;
}

which would provide the gray mask, and one of the polygon comp-op function that allows you to do different effects on features in tilemill (using the gray mask)

#border {
opacity: 0.5;
polygon-comp-op: hard-light;
line-width: 4;
line-opacity: .65;
}

Unfortunately, all of the polygon-comp-op options that I tried including dst-in which had no effect or colored inside of the border. I uad only been able to lay the mask over the rest of the features inside of my polygon, (labels, roads, water, land), but not outside – what I’m intending to do.

So, hours later, I stepped away and thought how else to tackle this,
I came back, noticed, silly me, that

map {
background-color @water;
}

was called again in base.mss to provide Lake Erie with its blue sheen. I figured that might have something to do with it but that was a red harring. I eliminated that line and it left me with a gray lake.

Meanwhile, I hadn’t figured out yet or found anyone else who had done anything like this – overlay a mask over multiple layers except for a specific region in Tilemill.

(written on Nov. 28, 2012)
Flash forward 24 hours. This is why I love Tilemill. Yes, it’s free, open-source, supports linux (as well as win and osx). What you see is what you get – allowing you to code on one side while showing what your map looks like on the other. I’ve been a fan for a while, and I’m finally starting to make progress learning its intracies and operations 15 months later as its capabilities increase. It’s getting kudos by cartographers (Dane Springmeyer, lead developer of Tilemill, had the most attended presentation, by far at this year’s NACIS, the biggest annual mapping conference in North America).

I was a bit frustrated, and after a day of trying to understand the comp-op and wondering why all of the comp-op operations that I was trying had no visible effect or achieved the opposite of what I was intending to do, coloring inside the polygon, as shown above.

Only a few hours after I posted my query in TM support, Dane clarified some new tools that became available in mapnik 2.1 (just released 2 months ago) and how to go about doing it in Tilemill.

As of now, my code ended up as:
#border {
::outline {
line-color: #999;
line-width: 4;
line-opacity: .47;
line-join: round;
line-comp-op:multiply;
}
line-opacity: .95;
polygon-opacity:1;
opacity:.83;
image-filters:agg-stack-blur(10,10);
comp-op:dst-atop;
}

(for future reference, note: In this code, as opacity gets closer to zero, you are able to see more and more of the area outside of the border.

Resulting in:

As of now, I didn’t use the gray mask that I first intended to do (nor have I figured it out yet) but I’m really liking the results and I’m encouraged by my progress, to be able to do something that I hadn’t before.

Work on this isn’t done (I’d like to customize the colors a little more, maybe add interactivity) and you can follow the progress at its github repo and view the map.

Should I be a super user in postgis ? And other beginner’s questions about roles and users in postgis

I have a confession to make. Some reasons that I don’t post on here because:
a] I don’t want to appear too ignorant to others. Typing this out makes me realize the above thought is pretty sophomoric. We all have to start somewhere when we learn things including postgis, programming, cartography, and other things in the geospatial world.
b] Like others, I don’t always want to go back to my notes to write concise documentation. I just write enough to make sense for myself later on. Other tech writers do the same thing when they write documentation… leading to point C [SIDEBAR: I keep a textfile of every each problem, install issue, question that I have, named log-nameofprogram].
c] Most documentation for postgis and I’d extend this to linux, is written so that the reader is assumed to know a lot. This is a pet peeve of mine, and by not doing this myself, I’m a hypocrite. Below is an example…

Here is my journey along the way of learning all of this…. With that said…

What’s the difference between a “super user” in postgres and another user ? Should I use the ‘postgres’ user in my day to day postgis work ? Should my created user be a super user ?

All of my mapping work is locally done on this computer. I’m the only user. Occasionally, I’ll make dumps of my db’s and then store them in other places.

Until now, I had been just using postgres (the default user name, after you install postgres) as my user name. I knew that you could create a user name, but wasn’t sure whether I should or not in my case. What was the difference anyways ?

As a result, whenever I converting data into postgis databases, I would have to copy these data files (usually from OSM) into /tmp because postgres could access files from my /home . I was finding it to be a bit inefficient, and I was prompted to determine whether or not I couldn’t access my python virtual environment (where imposm is installed) while logged in as postgres..

So, as I was looking to create a new user, I was wondering whether or no to be a super user’

The Postgresql documentation was a bit helpful.
Stating:
“Only roles that have the LOGIN attribute can be used as the initial role name for a database connection. A role with the LOGIN attribute can be considered the same as a “database user”. To create a role with login privilege” use either:

CREATE ROLE name LOGIN;
CREATE USER name;


What does LOGIN priviledge mean exactly ? Oh, well. A couple paragraphs down, they mention that I can issue CREATE ROLE name CREATEDB.

postgresql up and running (the book) explained it differently, telling that you can create the users in pgadmin or by the command line and highly recommend to create a new user after ASAP after installation, by doing:

CREATE ROLE leo LOGIN PASSWORD 'lion!king'
CREATEDB VALID UNTIL 'infinity';

It raises a good point, do I want a password with my postgres user account ?
“If you wanted to create a user with super rights, meaning they can cause major
destruction to your database cluster and can create what we call untrusted language functions”
Not sure what untrusted language functions are, but I’ll trust them and create the user. ‘

Well, after running, logged in as postgres (you can log into postgres through different ways, including sudo -u postgres -i) launching psql in the terminal, I typed in:
CREATE USER skors
(skors – my user name, also the user name for my desktop as well. I forgot the ; but it’s no big deal because I was able to create a database simply by doing: createdb nameofdb)

So, I go to add postgis capabilities now to my newly created database using my user name by doing

psql -d cleve -c "CREATE EXTENSION postgis;"
ERROR: permission denied to create extension "postgis"

argh ! Why shouldn’t I be a super user if I have to be a super user to create a new database ?

Which leads me full circle back to my questions that I asked in the beginning still unanswered:

What’s the difference between a “super user” in postgres and another user ? Should I use the ‘postgres’ user in my day to day postgis work ? Should my created user be a super user ? Do I want a password with my postgres user account ?

Cleaning up .GPX files for OpenStreetMap and Visualizations.

While going through the rest of my GPS traces from the HOT exploratory mission in Senegal [some write-ups on the HOT blog], I had found one GPS trace that has 2 problems that I’ve occassionally experienced while collecting gps traces for OpenStreetMap.

- The GPS signal is temporarily lost and the distance between 2 points of your trace is abnormally lengthy,
and cuts across roads and features.

An example: Did I time-travel across that area ? Nope.

- And what I call the ‘scribble effect’ (is there another name for it?), where I accidentally forget to turn off the gps after I arrive at my destination and end up with multiple points near each other, looking like a child scribbled in a coloring book.
An example:

If you’re planning to make any visualizations with your traces, these two symptoms allow viewers to misinterpret where you’ve been and make your visualizations inaccurate.

The scribble effect also negatively affects OpenStreetMap because GPS traces that are uploaded in the future in the same area as your scribbled GPS trace, other users will be able to not be discern the newer gps trace from your scribbled gps trace. There’s also still a few places on earth (including some regions of Senegal) that don’t have any satellite imagery to (legally) use for drawing ways for OSM.

So, after you complete a trace, how do you eliminate these symptoms described above?

I’ve tried a combination of different filters with gpsbabel – (explained more here)

but have yet to find a single command that would solve both issues.

gpsbabel -t -i gpx -f gpx_louga_03062012-withmay.gpx -x track,pack,sdistance=0.6k -o gpx -F test7.gpx
- this had fixed the temporary signal loss … but it didn’t fix the scribble effect.

gpsbabel -t -i gpx -f gpx_louga_03062012-withway.gpx -x discard,fixnone,fixunknown -o gpx -F test.gpx
(resulted in a blank GPX File.

gpsbabel -t -i gpx -f gpx_louga_03062012-withway.gpx -x discard,fixnone -o gpx -F test.gpx
(no effect)

gpsbabel -t -i gpx -f gpx_louga_03062012-withway.gpx -x position,distance=3m -o gpx -F test3.gpx
(this removed the long points between the 2 segments, but didn’t solve the scribble effect)

gpsbabel -t -i gpx -f gpx_louga_03062012-withway.gpx -x discard,hdop=5,hdop -o gpx -F test6.gpx
(no effect)

gpsbabel -t -i gpx -f gpx_louga_03062012-withway.gpx -x position,distance=10m -o gpx -F test8.gpx
- removed 1100 of the trace’s 2400 points and the , but still left me with the scribble effect.

I’ll continue to explore cleaning up the GPS traces with GPSbabel filters and hope to find a way but if you know of a specific filter in gpsbabel or any other way to do this, besides manually deleting the offending points in gpsprune, I’d appreciate it.

Update: Dec. 1, 2012. I also shared this in an OpenStreetMap Diary Entry in Sept. Other OSM users have also experienced this problem and hadn’t found a solution besides what I’ve described above. So, I’ve found that the test3 example above and manually removing the points in gpsprune (from the scribble effect) is the easiest way to clean up your GPS points.

Best practices for styling your tilemill stylesheet

(edit Oct. 2012: A few months later, I’ve learned some of these tips and am sharing them here.)

Should I create separate layers for labels and then for the features (roadways, buildings, etc) themselves ? Yes. If you’re intending to display text of any feature (like the name of a building or road), create a separate layer for them. You can name it as ‘labels’ or whatever you’d like. Labels are not yet automatically set to render on top of every other feature (<a href="https://github.com/mapnik/mapnik/issues/87rparate layer, text will not display as you intend in some instances (Dane Springmayer gave me a more detailed explanation at State of the Map-US 2012).

Should each feature (roadways) have a separate layer ?
Do you have different layers for each zoom level ?

I haven’t found any Tilemill stylesheets that extensively use layers from postgis besides osm-bright and am curious to see what others do.

Remember to ask yourself, what do I specifically want to show at this zoom level ?
- (credit to J. Lord – State of the Map, US 2012)

If you want to see some carto in action, of how to write, manage your stylesheets, here’s a couple that I admire.

Osm-bright – Made by Mapbox.
Uses Postgis layers as their data sources;

openstreetmap presentation

Short notice, but I’ll be giving a presentation on using Openstreetmap and my work on Cleveland in OSM Tonight at BW’s ACM’s Chapter meeting if any readers are interested.

7:00 PM, Room 147, Math and Computer Science Building, Baldwin-Wallace College (This Room is in the Mathematics wing).

The Cleveland Neighborhood Map – Update and the past months in review

Behind the scenes, I’ve been relatively busy with the Cleveland Neighborhood Map.

Here’s the status of it… In Dec. 2011 while developing 5.14, I was soliciting for feedback from some professional cartographers over at cartotalk (a great place by the way). I received some good critiques, some of which I didn’t implement because I had decided I was already far enough in my workflow that it would delay the release by a couple weeks to implement the changes. So, I trucked ahead and released 5.14 on December 21st following my previous workflow.

Meanwhile during 2011, osmarender, the program that I had been using to style CNM, was no longer actively developed and other activity around it (people offering, seeking help using it, sharing map styles, fixing, reporting bugs, etc) dwindled. Its biggest user, the Openstreetmap, even dropped osmarender in March 2012 from its layer selection on its main website.

In the first couple weeks of 2012, I was devoting my time with CNM to soliciting price quotes for a printed copy of the CNM (Yes, A print version of the Cleveland Neighborhood Map is being implemented – more details to follow in a future post)had received some more feedback and then decided to implement a couple of the suggestions (increasing the width and style of the city borders, etc). While trying to alter the width and style of the city borders in osmarender, I had received a few errors; I solicited support, was falling on deaf ears.

Meanwhile, Tilemill and mapnik have come a long way in 2011. Mapnik has become the de facto map styling software library for openstreetmap. Tilemill is also becoming a great mapmaking program with an intuitive UI [although the docs could be better, and they've improved], and utilizes the mapnik libraries. Both also support other data formats (including postgis, geoJSON) besides OSM, unlike osmarender.
Additionally, I’ve changed my workflow to use store my data from OSM in a postgis which allows updates to be issued much more easier in the future.

All of this together, I started on a complete re-write of the Cleveland neighborhood map using tilemill and mapnik a couple weeks ago. I plan on releasing this in the coming weeks and currently looking for options to offer a printed copy for sale.

Cleveland City Planning Commission notes 2012.02.17 – No, you May not park there !

Quote of the Meeting: “I didn’t fall off the turnip truck last night” – Coyne.

(My comments in Italics)

In attendance: Special K, Pinkney, Coyne, Bowen, Lumpkin, Cleveland, Brown.

Surprisingly, the meeting started on time.

This Week’s agenda: http://planning.city.cleveland.oh.us/designreview/drcagenda/2012/02172012/index.php

Here’s a couple of the highlights:

• Dow’s rezoning plan:

Councilman Dow was sponsoring because he had recently made a comprehensive rezoning plan for the neighborhood(s) of Ward 7, (that map, courtesy of Cleveland GIS, the official GIS site of the City of Cleveland).. He wasn’t there to present it (and I forgot who did on his behalf, something I should have noted, so I’ll refer to them as ‘the rep’ here), it was supported by the city of Cleveland planning staff.

The rezoning’s Purpose was “to prevent nuisance-type retail from popping up”; the rep also said the few retail outlets that were there are ‘bad neighbors’ and ‘nuisances’ to the neighborhood.

One resident in attendance spoke up against the rezoning changes as the change to multi-family rezoning would encourage ‘riff-raff’ to the neighborhood.

Councilman Dow works with the Famicos Foundation and the St. Clair CDC in his Ward 7. (Here’s a map of Ward 7, courtesy of Cleveland GIS, the official GIS site of the City of Cleveland). Neither organization had made any statement of their position of the zoning changes to the CPC.

(My hunch is that there’s someone or something else that is supporting this.)(Why else did Dow come up with a comprehensive rezoning-plan for the neighborhood if the city already has a master zoning plan ?! He doesn’t have anywhere else to spend his time ? Was it driven by constituents ? I wonder when the city’s zoning plan was last updated)

Decision: Approval, on the condition that the CPC receives letters of support from Famicos and St. Clair CDC; and 603-11 specifically was approved on the that none of the changes conflicts with the master zoning plan. (typing from my notes, I don’t understand why only 603-11 was singled out that these zoning changes could conflict with master zoning plan).

• New parking garage, on the southeast corner of Ontario and Carnegie. Built for Tri-C… Across from Progressive Field, the existing surface parking lot was a huge money maker for Tri-C. With the New Innerbelt taking out some of the land of the existing surface lot, they decided to make a multi-level parking garage. On the northwest Corner, the archetect interesting included a small concrete pedestrian ‘gathering space’ at street-level.(scalpers now have a new prime location ! )(I generally prefer downtown land to be use in ways besides parking. On the positive, this is a parking garage rather than a surface lot, so it will decrease the amount of additional parking needed. It increases supply of parking spaces available, giving less incentive to land owners to use land as parking lots)

Decision: Passed by City Planning Commission

====

• The proposed Parking Garage in the May Company Building

Once the bastion of downtown Cleveland department store shopping, the May Company Building on Euclid (on the south side, between East 4th and Ontario) has sat dormant to the public eye for as long as I can remember.

Days before the meeting, the proposal to turn it into a parking garage received attention by local news- (Cleveland.com) and nationally at the Atlantic Cities blog.

One historical building demolished less than a year a go for parking, would another one bite the dust ?

In a nutshell, not anytime soon.

The CPC gave the strongest criticism of any project that I’ve seen covering their meetings for the past 13 months. Commission head Tony Coyne led the criticism, noted that the owner, in Florida and locally represented by Robert Zarzycki, ZM Architects, commented that the owner gave very little consideration of this building and its use and charged it as a ‘bottom-feeder approach.’ (Where was this criticism during the Columbia Building demolition?!)

Zarzycki tried to save face by noting that parking was only proposed on floors 2-5 of the 8 floors and although the building’s windows facing Euclid would be removed, the garage’s lighting would be minimally visible from the street. (Fortunately), Coyne and ‘Special K’ Norm Krumholz, weren’t buying it; Coyne replied that ‘he wasn’t born last night.’  and Norm noted it was ‘an incredibly bad land use.’ – No traffic study had been done and the archetect hastily mentioned that apartments wouldn’t work in the building because, who wants an apartment without windows (this building has 2 other buildings on its west and east sides) ?  – Coyne astutely mentioned that a light box could be added which also was done in other downtown conversions (the Bingham, National Terminal).

Decision : Rejected

Although it was rejected, the use of parking in the May Company in the future is still possible. As of now, the building is only occupied on the 6th Floor, as a ‘tech center’ (don’t know specifically what that is) and a portion of the 1st floor to a couple restaurants and bars. The architect noted that the other floors are sparse: most of its materials have been stripped and would require a bit of reinvestment before they could be used for anything.

Planning Director Brown said he wasn’t against it, although he said he wasn’t sure if the Planning Commission even has jurisdiction of over the use of the building (Me: If this is true, The fact that the CPC doesn’t have this power strikes me with only resigned surprise….), but jurisdiction over the windows and a traffic study that would need to be performed… Special K disagreed with him.

Chairman Coyne noted that if housing were a substantial part of the building in the future, he could see a floor or 2 being devoted to parking….

Downtown Cleveland Parking Map

Upon hearing about the (ill-fated) proposed parking garage for the May Company building a couple weeks ago, I wanted to highlight the existing amount of parking that already exists downtown.

So, here’s a map of Downtown Cleveland’s parking,
here’s a map of Downtown Cleveland parking broken down by garage, underground, and surface, hosted by mapbox. The map design is based on osm-bright and the source is available on my github page.

(Update: There are still a handful of lots that are exclusively for some downtown apartment complexes – particularly on West 9th, that are unmapped at the moment).