Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

Archive for the ‘Conferences’ Category

Boston Code Sprint

Posted by smathermather on March 31, 2013

I hung out this week at the Boston Code Sprint
http://wiki.osgeo.org/wiki/Boston_Code_Sprint_2013
, which is a “C-Tribe” code sprint for improving things like PostGIS, MapServer, and other GeoFOSS projects written in C. See Paul Ramsey’s posts on PostGIS and MapServer for more on what everyone was working on.

Being my first time at a code sprint, it’s been very interesting, and it’s a very warm and inviting group. I will have some forthcoming posts on the work I did on skeletonization, and also work on getting it up on GitHub. Toward this end, Voronoi (yipee!) should land in PostGIS in the future via two venues. The first venue to provide Voronoi will be from the recently welcomed SFCGAL (optional) dependency additions to PostGIS coming down the pike from the Oslandia representatives, Olivier Courtin and Hugo Mercier, who were at the sprint. The second venue will be the addition of Voronoi from within GEOS as a port from JTS, since JTS already has it implemented. That may be a few months off, as it is being proposed as a Google Summer of Code (GSoC) project which Sandro Santilli agreed to manage (after hunting me down in IRC for my overly broad GEOS ticket).

For my part, I cleaned up and worked on documenting my existing code base for routing skeletonization, played a bit with some additional filtering techniques appropriate to skeletonization, and ran some performance tests against the skeleton results that Olivier was producing from an ST_StraightSkeleton function leveraging the CGAL computational geometry library. It wasn’t a fair test, as my skeletonization was approximate, and Olivier’s exact, but it did give us the sense that ST_Voronoi from that codebase might be more performant for the skeletonization problem than exact skeletonization as performed by CGAL. As part of the skeletonization problem, I have been tasked by Paul Ramsey to write a C implementation of Dijkstra routing, to remove my pgRouting dependency from my skeletonization code, which I will start in on when I get a moment to breathe.

So, what will skeletonization look like when it’s complete? Nobody knows, but my projection is that we’ll have a few functions, dependent upon the kind of information you have about the geometry you want to skeletonize. So for example, if you had a stream network with known inputs and outputs, the algorithm would take advantage of that a priori knowledge and use it to clean up the final skeleton. Something like:


skeleton_geometry ST_Skeletonize(geometry polygon, geometry multipoint);

This is will create a nearly perfect skeleton for the cases where you have this a priori knowledge.

Stream Centerlines

The tougher problem, of course, is when you have less knowledge about the skeleton. And if you want to encourage PostGIS developers to kibitz about analytical geometry, pose this as a problem– believe me that hours will be spent generating new algorithms for solving it. I arrived with 4 ideas on skeletonization (one of them the above) and left with an additional 3 or 4 great ones from conversations with Stephen Woodbridge, Regina and Leo, Pierre Racine, and Bborie Park (hopefully I haven’t forgotten any). But, regardless of which of these pan out, the idea is simple, instead of two input geometries, these will take an input polygon geometry and one or more tuning parameters. Cheers to Paul too for helping to limit the depth of this rabbit hole for me.


skeleton_geometry ST_Skeletonize(geometry polygon, value double, skeletonType text);

These solutions will likely not be as good as the routing solution above, but will be tunable to a posteriori results. The use case here might be scanned maps of road networks, contours or similar from which we want to extract linear features, or similar. Thus these approaches are an important addition to enhancing conversion from PostGIS Raster data type.

Posted in Code Sprint, Conferences, Database, GIS, PostGIS, PostgreSQL, SQL | Tagged: , , | Leave a Comment »

EcoHackNYC, version 2.0

Posted by smathermather on April 22, 2012

I had the pleasure this weekend of attending EcoHackNYC for the second time.  It was a generous, brilliant, and fun group to join for a day of hacking.  Sadly, this time, as last, I had public transit issues, so I did not make it for the Friday night ignite talks.  There were a number of talks and topics; swing over to twitter and search for #ecohacknyc to get a tweety view of it.

I presented a project idea on Saturday AM re: putting together mobile-ready field forms for performing plant community assessment protocol (PCAP).  In short, plant community assessments allow us to make qualitative assessments of plant communities, so that we can distinguish between high quality, medium quality, and low quality habitats.  The process stems out of the tradition of wetland monitoring programs from the State of Ohio, recently adapted for terrestrial ecosystems as well.

The interesting thing about these protocols, relative to simple diversity indices, is that alike communities are compared.  So, for example in wetland assessment, the wetland hydrogeomorphic (HGM) class is considered, meaning a high-quality fen will be compared with other fens, but not to a coastal wetland.  This prevents high quality ecosystems of naturally lower diversity, lower plant sensitivity, etc. from being scored relative to systems that are naturally more diverse, and/or have more or a greater number of sensitive species.

What I proposed at EcoHackNYC is that a group of us work on putting together tablet-ready field sheets to help streamline the data collection process for these datasets.  Presenting at the last minute, as I had missed the ignite sessions, I got no nibbles on working on the project.

So, I sat with a group building this:

An interface for communicating local demand for community supported agriculture (CSA) through a simple map interface.  It’s meant to be a tool to aid in the development of new CSA drop-offs based on local demand.  You can follow the project on twitter @csabuilder.  It’s built on Heroku for hosting, MapBox for map tiles, Leaflet for the mapping interface, jQuery for most everything else, and leveraging CartoDB for PostGIS goodness.

In the interest of full disclosure, I wrote not a lick of code for this projects– I mostly just kibitzed and made snide remarks, while working on getting Open Data Kit (ODK) to bend to my will in developing field forms for the aforementioned PCAP project.  I may have a forthcoming post on ODK, which was fun and easy(ish) to set up and deploy for creating Android form management.

Posted in Conferences, EcoHackNYC, GIS, Mobile, ODK | Tagged: , , | Leave a Comment »

G. I. O.: A Real American Hero (FOSS4G-NA)

Posted by smathermather on April 14, 2012

Whew! What a conference! All the classic FOSS4G stuff was there, from why the breaks in version for PostGIS (serialization + more…) = 2.0 rather than 1.6, to “here are the new combinations of existing technology stacks”, to the “tribes” of Open Source GIS.

I will confess: when I went to FOSS4G 2011 in Denver, I was a little terrified to actually meet and interact with the minds and persons behind the ecosystem of Open Source tools that I use and follow. This time, FOSS4G North America, I was a little more comfortable in my own skin. I had some nice conversations with lead developers and other cognoscenti on some of my favorite projects, including GeoServer, JTS, PostGIS, and CartoDB. This remains a remarkably accessible and down-to-earth group, in spite of the rapid growth of Open Source Software in the GeoSpatial sector, with warm and knowledgeable personalities.

Subtitle: FOSS4G-NA– send in the Bureaucrats!

The thing that was different about this conference was the preponderance of bureaucrats. Making the list were Michael Byrne from the FCC and Keith Barber, from the NGA, plus a few from DOD that were at the vendor session on Thursday. Later I’ll create a blog, likely called Straw-Hat-Hacker on my thoughts on Peace, Social Justice, and Anabaptism and my thoughts on the interaction between technology, FOSS, FOSS4G and the military industrial complex. For now, I’ll just state there is a strong connection, and that connection was represented at the start and end of the conference.

Speaking-wise, the highlight from The Bureaucrats was Michael Byrne, the Geographic Information Officer (GIO) for the Federal Communications Commission. It was an entertaining, well structured, and informative presentation, advocating for RESTFul, transparent, paperless, as well as cost-effective governance, with tangible examples from his own work within the FCC, including getting his hands dirty with a MacBook Pro and MapBox TileMill for quickly provisioning great “enterprisey” products for decision-makers and the public. I look forward to re-watching it when the videos become available online.

Given all the politics we watch from outside The Beltway, it was really nice to see that genuine effort goes into informing decision makers by very talented and genuine characters like the FCC’s GIO. Bravo.

Posted in FOSS4G-NA | Leave a Comment »

EcoHackNYC– Cool projects, fun new ideas, human waste, CartoDB and other flotsam.

Posted by smathermather on November 7, 2011

I took a bus to New York City this weekend to enjoy the company of fellow hackers at EcoHackNYC, organized by Javier Torre and Andrew Hill of Vizzuality and Robin Kraft of REDD Metrics.  Due to delays in Pittsburg, I missed the ignite talks on Friday, arriving on NYU’s campus on Saturday morning. Several groups formed around topics and we started hacking. I worked on Robin Kraft’s crew– helping to put together a draft visualization of deforestation data for Indonesia. I suspect that something interesting will continue to evolve out of that project. Robin envisions visualizations for tropical deforestation on a monthly basis globally, and he’s not far away from that. It should be really great to see. We also got some wild-eyed data visualization/HTML5/data transfer protocol ideas out of that project, thinking about how to stuff any sort of data into a PNG tile.  The cool thing is that while our wild-eyed plans would require special data prep, they would not necessarily require changes to existing middlewhere/tile-rendering software initially, just client level magic. More on that later, unless we discover we’re re-inventing aluminium rims and someone has already built a hot rod.

It turns out, when I stayed in a hotel in Midtown Manhattan, had it been raining hard enough to activate the CSOs, my waste would have come out just upstream from the UN. True story.

For now though, I’ll highlight one of the more polished products presented at the end of the hacking event the Don’t Flush Me project (warning– potty humor and mild curse words involved). I did not work on this project. With so many pun opportunities I probably would have derailed the project with linguistic abuses of monumental puniness, so maybe it’s for the best. I’m holding back right now. I just want you all to appreciate that. I like potty puns so much, they are my first and second most favorite pun type. That said, the best pun of the night was by a gentleman named Francois who was with the big-carbon-footprint group (meaning they flew in from Brazil for the conference– it was nice to have the international contingent). I’m always particularly impressed with puns done in an individual’s second language. It shows true mastery. Of course, now I can’t remember the pun. Anyway, I digress substantially more than usual.

Quick 3rd party description of the project: Don’t Flush Me took the combined sewer overflow (CSO) pipe diagrams and outfall data for Manhattan and created an interface that geocodes from address or IP and returns a polygon that shows the shared sewershed and overflow location for the input location.

Additionally, it has the Wunderground API wrapped in, in order to report whether there has been rain in the last day for your sewershed. Further work on the project would model capacity vs. rainfall and report whether you should wait to flush, and other such features. There are some small browser-dependent geocoding api bugs, but that’s forgivable considering this was put together in fewer than 8 hours. Also, the code is reportedly not particularly well organized yet. But who cares– the functionality is awesome. As an aside, if memory serves me, the back end PostGIS services are hosted in a CartoDB instance.

I think Don’t Flush Me could serve as a great model for reporting mechanisms for Sewer Districts with CSOs. Brilliant, well-scoped, and well-executed work, also fun to use. If you are not in Manhattan, here are a couple of addresses you can feed into the geocoding engine:

11 West 53 Street  New York, NY 10019

1 Wall Street, New York, NY

and an homage to the kindness of strangers:

Chelsea Park, New York, NY.

Posted in Conferences, EcoHackNYC, Ecology, GIS | Tagged: , , , , | Leave a Comment »

Motivations behind Open Source

Posted by smathermather on September 20, 2011

Waxing philosophical tonight.  Apologies in advance.

Paul Ramsey had a great opening keynote opening keynote at FOSS4G on the business model/economics (punchline: business tactic) behind Open Source software, and Free and Open Source companies.  It was funny and convincing, began with absurdity, and made me laugh, as did his PostGIS for Power Users.  His presentation style is at once casual, clever, and passionate.  What caught my attention was the series of slides about the primary and secondary motivators behind making Open Source software: money, community, shared accomplishments, etc.  Is money or should money be the primary motivator behind Open Source software development?  I don’t mean for individual companies– for companies, money may be the primary motivator (if I may divorce individual choices from corporate motivation).  But for individuals working in the Open Source Geospatial community?

If RSAnimate’s video on “The surprising truth about what motivates us

(go ahead, watch it, I’ll wait) is a good indicator of what current research indicates motivates people, money becomes secondary once a certain level of comfort is met, and above that line (whatever that line is) more money actually makes us less productive, more risk averse, and well, less smart, of all things.  So, if Open Source is the superior development model, and we want great Open Source, do we want to be driving the model for Open Source development through the idea that money is the primary motivator for that development?  Money is important, and we want companies to understand that there are business models that include Open Source code as a tactic, but shouldn’t we be articulating carefully who we are and why we have the community, the tribes, of Open Source GeoSpatial developers and users we have?

So those intangibles Paul mentions in his keynote:

  • community
  • shared accomplishments
  • satisfaction of learning and exploring
  • freedom to build
  • membership in a global community

I argue those are the important factors in motivating great FOSS development.  The money.  That pays the bills, feeds the family, keeps the roof over out heads, etc., all vitally important.  But it’s the participation in projects larger and more important than me, the shared accomplishments, the great features and information I can provide to the public, it’s all these things that make this more than a job.  That’s what switches work from vital to integral.  Money can be made in many ways.  Which ways one chooses, that’s what distinguishes jobs from passions.  FOSS is great when it is our passion.

Posted in Conferences, FOSS4G 2011, GIS | Tagged: , | 2 Comments »

What is the center line of a complex polygon?

Posted by smathermather on September 16, 2011

Sat in on a great presentation yesterday by Tim Schaub and Justin Deoliveira on GeoScript. Definitely check out their tutorials. To practice what I learned, I thought I’d apply it to the problem of how to find the centerline of a polygon.

What do I mean by the centerline of a polygon? Well in the case of a stream that is drawn with both it’s banks, the centerline of a polygon is the effective flowline. Also, if you want to label such polygons, it would be useful to be able to calculate this. This could apply to road polygons as well, or labeling complex parcels, etc. etc. At first (and second) blush, there is no trivial solution. However, there are some excellent approximations.

There are some nice articles online about the derivation of centerlines from polygonized roads, e.g. this one. The basic approach is to densify the lines (if necessary), and run a Thiessen (Voronoi) algorithm, then grab the middle line from those polygons. So, let’s start this process, and take it through at least to deriving the Voronoi result using python/GeoScript:

First our stream line:


>>> from geoscript.geom import *
>>> from geoscript.style import *
>>> from geoscript.render import *
>>> poly = Polygon([(2130394.00006841,633268.4901058), ...
,(2130394.00006841,633268.4901058)])

Then to view it quickly:


>>> draw(poly, format='mapwindow')


Maybe fly with a little style:


>>> style = Stroke('blue', width=1) + Fill('#0000ff', opacity=1)
>>> draw(poly, style, format='mapwindow')


Ok, now that’s a complicated enough polygon. What do we get from calculating a Voronoi diagram against it?:


>>> polyv=voronoi(poly)
>>> draw([polyv, poly], format='mapwindow')

See the centerline running through the stream? It’s not perfect, but apply some densification before we calculate the polygons, then select only that centerline, and Bob’s your uncle, you have a very nice stream centerline for flowline calculations, labeling, what have you. In future steps, I’ll set this up to extract that line as well, and perhaps host it as a WPS service on GeoServer. If I can (I’m not sure the limitations to GeoScript, have to bug the GeoScript guys today…), I’ll also use it to extend the labeling capabilities for my GeoServer instance without having to dig deep into Java development. Ah, I do love a good API.

Posted in Conferences, FOSS4G 2011, GeoScript, GeoTools, GIS | Tagged: , , , , , | 7 Comments »

Presentation– GeoCamp in Portugal

Posted by smathermather on September 1, 2011

A blog entry so short, it’s practically a microblog:

I’ll be doing a presentation at GeoCamp in Portugal (remotely via skype) on September 24th.


http://geocamp.tumblr.com/post/9621852908/orador-confirmado-o-stephen-mather

It promises to be an interesting conference.

Posted in Conferences, GeoCamp | Leave a Comment »

FOSS4G 2011 Denver– Ideal schedule

Posted by smathermather on August 19, 2011

I spent a few hours winnowing down my list of desired talks at FOSS4G and came up with the following list. Path dependence (i.e. technologies related to the ones I’m already deploying) were at the top of this list, but I do hope to expand my knowledge of all the options as well. For almost every slot, there were anywhere from 2 to 4 presentations that really excite me. And finally, one of the most exciting (to me) presentations is the last in this list (Lifemapper Moves Forward: Bringing Together Geospatial and Biodiversity Informatics Tools to Save the World)– which I’ll miss because I’ll be flying out at that time. Alas, I’ll have to download the presentation when I get back to Ohio.

Date

Talk/Presentation

Presenter

Sept 13th

Developing OGC Compliant Web Applications With GeoExt

Andreas Hocevar

Sept 14th

State of PostGIS

Paul Ramsey

PostGIS 2.0, the new stuff

Regina Obe, Leo Hsu

Functional Coverages

Gennadii Donchyts, Deltares Fedor Baart

Tutorial: GeoServer Scripting with Python and RESTConfig

David Winslow

Introducing GXP: Webmapping made easy

Bart van den Eijnden

GeoScript – Spatial Capabilities for Scripting Languages

Justin Deoliveira, Jared Erickson

Scripting GeoServer with GeoScript

Justin Deoliveira, Tim Schaub

Sept 15th

Open Source tools for digital field mapping

ing Andrea Antonello, S.r.l., ing Silvia Franceschi

OpenLayers Mobile

Tim Schaub, Eric Lemoine

LiDAR Point Cloud Processing with libPC and libLAS

Howard Butler, Michael Gerlek

A web-based PostGIS educational tool

Javier Morales

The status of GeoServer WPS

Andrea Aime

Tutorial: Working with GeoScript

Tim Schaub, Justin Deoliveira

Sept 16th

HTML5 for Rich Geospatial applications on the web

Javier de la Torre

PostGIS Replication

Steven Singer

Store, manipulate and analyze raster data within the PostgreSQL/PostGIS spatial database

Pierre Racine

Once upon a time: JGrass

ing Andrea Antonello, HydroloGIS s.r.l., ing Silvia Franceschi, HydroloGIS

Lifemapper Moves Forward: Bringing Together Geospatial and Biodiversity Informatics Tools to Save the World

Aimee Stewart

Posted in Conferences, FOSS4G 2011, Other | Tagged: , , | Leave a Comment »

 
Follow

Get every new post delivered to your Inbox.

Join 425 other followers