Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

Muhahahaha!

Posted by smathermather on April 15, 2014

image

Stairway to PostGIS Heaven

Posted in Other | 2 Comments »

Inventorying linear assets– really high resolution orthos

Posted by smathermather on April 5, 2014

I have been contemplating all sorts of varied uses of Structure from Motion techniques. One of those outputs, in addition to using sUAVs (drones) is just to orthorectify and generate 3D meshes from ordinary photos. This has really great potential for linear assets like streams and rivers, trails and roads. We’ll have to being to contemplate how we’ll use (and summarize!) the incredible amount of data and information that will suddenly be available to us, e.g. this subset of an orthophoto of a multi-purpose trail:

Screen shot of orthophoto showing readable text: "Be courteous give notice when passing"

 

Posted in 3D, Drone, Image Processing, Optics, Photogrammetry, UAS | Tagged: , , , , | Leave a Comment »

Parks Data Cake, part deux

Posted by smathermather on April 2, 2014

Following up on my previous post, I have started to detail the how of our parks map rendering works, including a GitHub repository with all the code and data to build your own in TileStache. One of these days, we’ll port this to TileMill, but in the mean time, it works and works wonderfully.

Screenshot of trailsforthepeople-styles github readme.md

 

— Addendum —

Truth in advertising– the contours data were not included in the repository as they are 1.3 GB all by themselves

— End Addendum —

 

Posted in Cartography, Recreation, TileStache, Trail Curation, Trails | Tagged: , | Leave a Comment »

Packt Sale — 2 books for the price of one

Posted by smathermather on March 25, 2014

Now I feel like a salesman. I feel obliged to mention the Packt buy one get one free sale (ends tomorrow):

http://www.packtpub.com/?utm_source=referral&utm_medium=marketingPR&utm_campaign=2000thTitle

If you want a postgresql title to go with the PostGIS Cookbook (just sayin’):

http://www.packtpub.com/search?keys=postgresql&sort=0&types=0&forthcoming=1&available=1&count=20&op=Go

Ok. Enough. Back to fun reading:

An map of Mount Pleasant

 

Posted in PostGIS, PostGIS Cookbook | Tagged: | Leave a Comment »

Parks Data Cake

Posted by smathermather on March 24, 2014

Stamen has a great blog entry on mapping for parks on their blog. It’s a teaser for a deeper dive in mapping parks, and I’m staying tuned, as their write-ups tend to be detailed, thoughtful, and complete.

I thought I’d offer my own teaser– a bit of work done collaboratively with GreenInfo Network.  It started with their basemap– something they spent a few person-years refining from OSM data.

Example of GIN basemap

Example of GIN custom basemap

Custom basemap

We took it a little further by creating custom placed labels where the labels gave us some extra bang for the buck.

Custom text placement example

Custom text placement

And then we started to really play with the zoomed in versions, using nicely detailed streams, contours, and canopy boundaries (derived from LiDAR) to enrich the map, labeling natural features and major facilities with custom labels as well:

Image showing labeling, contours, vegetation, etc.

Detailed view of map

A couple of tricks we used here– first, the custom labels have shadows (convex hull) masks which match the background, ala this write up:

Example of masked text

Also, compositing is used heavily– the hillshade background is flattened significantly, and then overlayed with multiplied composites of park green, canopy green on top, contours, with all other layers being simple overlays. This results in a brightness and clarity to the information that allows for complexity without either muddling the map, or overwhelming the map reader.

Finally, since many of the parks are urban, major buildings are highlighted for context. In future revisions, we’ll add more (but still subdued) detail to the areas outside the parks, but this was a good start:

Example of major buildings showing on map near a park

In case you are wondering about technology, this is all done in TileStache (edit: and GeoServer).  In the future, this may well be replicated in Tilemill. More to come… .

Posted in Cartography, Recreation, TileStache, Trail Curation, Trails | Tagged: , | 4 Comments »

Custom projections using spatialreference.org and gdal

Posted by smathermather on March 17, 2014

Every now and then I get the urge to define my own projection. Usually, I sit down for a while, hit my head on the wall, and the urge passes. For a few years I have worked with the Lake Erie and Allegheny Partnership for Biodiversity on various projects. Now we are getting deep into region-wide data collection, and so I decided to define an Albers Equal Area projection for the task. Specifically, I sought to refine an existing projection to match the bounds and center of the LEAP region. Yes, I know. I will be cursed one day for this decision, but it sure beats the alternatives at the moment.

Map of LEAP Region

Beyond defining the projection, I wanted it as an ESRI WKT and in Proj4 format. Here are the steps I took. I used spatialreference.org to help me do the format translation.

http://spatialreference.org needs the projection input in a form called Well Known Text (WKT)– specifically the Open Geospatial Consortium’s for of WKT. First, I uploaded the description in ESRI’s WKT: this doesn’t work. So here’s ESRI’s WKT for the LEAP boundary as I created a few days ago:

PROJCS["LEAP_Contiguous_Albers_Equal_Area_Conic",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.017453292519943295]],PROJECTION["Albers"],PARAMETER["False_Easting",0.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-80.75],PARAMETER["Standard_Parallel_1",39],PARAMETER["Standard_Parallel_2",43],PARAMETER["Latitude_Of_Origin",41],UNIT["Meter",1.0]]

I changed 4 things from the North America version of this:

"Central_Meridian"
"Standard_Parallel_1"
"Standard_Parallel_2"
"Latitude_Of_Origin"

OGC WKT has different names for these, specifically:

"longitude_of_center"
"Standard_Parallel_1"
"Standard_Parallel_2"
"latitude_of_center"

So, we can take USA Contiguous Albers Equal Area Conic from http://spatialreference.org/ref/esri/102003/

Go to it’s OGC WKT page:

http://spatialreference.org/ref/esri/102003/ogcwkt/

And thus extract the OGC WKT we want to modify:

PROJCS["USA_Contiguous_Albers_Equal_Area_Conic",GEOGCS["GCS_North_American_1983",DATUM["North_American_Datum_1983",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Albers_Conic_Equal_Area"],PARAMETER["False_Easting",0],PARAMETER["False_Northing",0],PARAMETER["longitude_of_center",-96],PARAMETER["Standard_Parallel_1",29.5],PARAMETER["Standard_Parallel_2",45.5],PARAMETER["latitude_of_center",37.5],UNIT["Meter",1],AUTHORITY["EPSG","102003"]]

Modifying those parameters thusly:

PARAMETER["longitude_of_center",-80.75],
PARAMETER["Standard_Parallel_1",39],
PARAMETER["Standard_Parallel_2",43],
PARAMETER["latitude_of_center",41]

Resulting in the following:

PROJCS["USA_Contiguous_Albers_Equal_Area_Conic",GEOGCS["GCS_North_American_1983",DATUM["North_American_Datum_1983",SPHEROID["GRS_1980",6378137,298.257222101]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Albers_Conic_Equal_Area"],PARAMETER["False_Easting",0],PARAMETER["False_Northing",0],PARAMETER["longitude_of_center",-80.75],PARAMETER["Standard_Parallel_1",39],PARAMETER["Standard_Parallel_2",43],PARAMETER["latitude_of_center",41],UNIT["Meter",1]]

Which I have uploaded to a custom LEAP projection.

Ok. Final step. gdalwarp was the original tool I wanted to use. It requires that we define our projection in a format called Proj4 or using an EPSG code. Since we invented this projection, it’s has no EPSG code. Now that we have a definition for it loaded into http://spatialreference.org, we can use the web site to give us the proj4 definition:

http://spatialreference.org/ref/sr-org/7915/proj4/

This site now gives us the following:

+proj=aea +lat_1=39 +lat_2=43 +lat_0=41 +lon_0=-80.75 +x_0=0 +y_0=0 +ellps=GRS80 +datum=NAD83 +units=m +no_defs

The sample reprojection code from the gdalwarp website is as follows:

gdalwarp -t_srs '+proj=utm +zone=11 +datum=WGS84' raw_spot.tif utm11.tif

To use it for our own data, we’ll do something like this:

gdalwarp -t_srs '+proj=aea +lat_1=39 +lat_2=43 +lat_0=41 +lon_0=-80.75 +x_0=0 +y_0=0 +ellps=GRS80 +datum=NAD83 +units=m +no_defs ' input.tif output.tif

Boom! Data reprojected. And so another projection is born. Oops.

postscript– Yes I know about epsg.io– as far as I can tell, it does not yet have the ability to receive uploads.

Edit from Howard Butler:

ala:

gdalwarp -t_srs "http://spatialreference.org/ref/sr-org/7915/proj4/" input.tif output.tif

Brilliant!

Posted in GDAL | Tagged: , , , , , | 3 Comments »

OSM and licensing

Posted by smathermather on March 15, 2014

I don’t often wade into the flame war territory of vi vs. emacs, GPL vs. MIT (etc.) and the question of “more open” for a given definition of open. But, as State of the Map, US (SOTMUS) approaches, the question of the share-alike clause with it’s copyleft equivalency in OpenStreetMap seems to rise as a conversation piece. For example:

and of course this:

Create, Adapt, Share

Create Adapt Share

So, the copyleft. It’s an assertion of copyright, but with a radical twist. It’s a tool in the toolkit, and important. It’s used to good effect in Linux and PostGIS and other important projects. I’m no purist– there are undoubtedly times when copyleft prevents the growth and contribution of community. It’s important because it is a protection of that which we make public to remain public.

The same is true for share-alike. This is like enabling legislation is for governmental agencies, it’s a line in the sand that defines that which is a public good vs. private good.

Assuming it gets accepted as a pull request, you’ll be able to use CC0 or ODbL (share-alike) for data on GitHub in a drop-down soon… .

So, when does share-alike or copyleft make sense? Maybe when the thing needing protection is a core piece of data or software (PostGIS, Linux, etc.). And (from the perspective of a not being a nearly active enough OSM member) perhaps OpenStreetMap fits here too.

Let the argument begin. CC0! Share-alike! CC0! Share-alike!

Posted in OpenStreetMap | Tagged: , | Leave a Comment »

Cool write up on Structure from Motion (SfM)

Posted by smathermather on March 5, 2014

A really cool SfM workflow write up:

http://wedidstuff.heavyimage.com/

which includes texture mapping (yay!)… .

Pic from the end of the workflow:

Thanks to Fred Judson for the tip on this one.

Posted in 3D, Optics, Photogrammetry | Tagged: , | Leave a Comment »

PostGIS Cookbook(s)

Posted by smathermather on March 5, 2014

This is starting to feel real…

PostGIS Cookbook(s).  2904 pages.

6 PostGIS Cookbooks

Page count for a single book can be calculated as in the following code:

WITH distinction AS (
    SELECT DISTINCT( the_text) AS pages FROM postgis_cookbook
SELECT COUNT (pages) AS "Count of Pages" FROM distinction;

The preceding query will result in the following:

Count of Pages
--------------
484

Feeling real

A special thanks to my family, my co-authors, all the contributors to PostGIS, and to Regina Obe for helping me through the hardest bit of code in my part of the book. Check out Regina and Leo’s book, PostGIS in Action. Their second edition is already on pre-order. I got to read a bit of it last year, and it promises to be a really epic edition. I still reference the 1st edition myself. I wish I had had it when I was starting in PostGIS… .

Posted in Database, PostGIS, PostGIS Cookbook, PostgreSQL, SQL | Tagged: , | 2 Comments »

Short follow up: Photogrammetrically Derived Point Clouds

Posted by smathermather on February 5, 2014

In my previous post, http://smathermather.wordpress.com/2014/02/04/big-dmn-post-photogrammetrically-derived-point-clouds/, I briefly cover software for creating photogrammetrically derived point clouds.  I didn’t summarize, like this, but PDPCs can be created in three easy steps:

  1. Structure from Motion for unordered image collections
  2. Clustering Views for Multi-view Stereo
  3. Multi-view stereo (dense point cloud reconstruction)

But, unfairly, I gloss over some of the complications of creating meaningful data from PDPC processing. Truth told, it’s probably a 9 step process

  1. Optimize image order according to geography to optimize step 2
  2. Structure from Motion
  3. Clustering Views for Multi-view Stereo
  4. Multi-view stereo
  5. Georeference
  6. Create breaklines or disparity mapping or find some other process for refining our surface model
  7. Generate surface model (mesh)
  8. Texture mesh
  9. Render mesh as ortho

 

Posted in 3D, Bundler, Camera Calibration, Drone, Image Processing, Optics, Photogrammetry, PMVS, UAS | Tagged: , , , , , | Leave a Comment »

 
Follow

Get every new post delivered to your Inbox.

Join 645 other followers