Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

Archive for the ‘Conferences’ Category

And then Steve gets all sentimental…

Posted by smathermather on September 2, 2014

for sweetness in `ls *sappy*`;
do;

It is bittersweet as I leave you, Korea. I did not intend to stay this long, so I do miss my family dearly. As my son started kindergarten, I flew to you, and you welcomed me. You astonished me with hospitality, humor, understanding, and the deep beauty of your people, your language, your Seoul, and your mountains. Thank you for the time and conversation with Tongju who offered me grapes and words on Dobong mountain, for ‘Jim’ who fed me rice cakes, shared his spot in the creek for cooling off hot and tired feet in the valley

image

image

Thank you for the atmospheric scientist and programmer whose hospitality and grace were unmatched. May her purpose and direction continue to unfold before her.

Thanks for the statistician turned Open Source GIS person who listened, translated, and understood.

image

And for the centurion who I will see again soon.

image

It was with these three I ate and drank, talked to at length, and with whom I experienced that elevated hospitality you, Korea, are known for.  These three laughed and cheered as I ate the more difficult seafood (sea worms and urchin shell and all, live squid on ice…) , taught me the fine ceremonies of soju, and so very quickly I came to count as friends.

And so, as I long to be home with my loving family, the parting is bitter and sweet. Is it also sour and salty and umami, or would that strain the metaphor too far?

image

I hope not. Until next time…

done;

Posted in Conferences, FOSS4G Korea | Leave a Comment »

FOSS4G Korea 2014, poor GPS photos, and mapillary (part 2 of n)

Posted by smathermather on September 2, 2014

A classic and age old problem in GPS is collecting potentially wonderful data in the field, getting back the office, and realizing a lot of manual scrubbing, data massaging, and other such careful work will need to be done to make the GPS data useful and meaningful. This assumes we can even meaningfully correct it at all.

This is true too (maybe especially) for GPS enabled cameras in canyons and urban canyons. This is a problem we started to explore in http://smathermather.wordpress.com/2014/08/31/foss4g-korea-2014-poor-gps-photos-and-mapillary/

Let’s return the problem briefly — were the GPS readings to be consistent and accurate, we should see a relatively straight line, as the photos were taken along the length of sidewalk on Teheran-Ro in the Gangnam District of Seoul

Figure of raw data points showing anything other than a straight line

In addition to not looking straight, though it is supposed to follow a road, we demonstrated that there are a lot of points duplicated where, presumably, the camera was using cached GPS data rather than the latest available at the time of the photo. We can see this density of overlapping points even more clearly using the heatmap tool in QGIS:

Heatmap showing clumping of data points

The clump of red shows a clear issue of overlapping points. As these points are the GPS positions of photographs, we can match features within the photographs between photographs (using structure from motion) to map out the relative location of these photos to each other. The points in the below figure show the matched points in 3 or more photos, the blue shapes represent camera positions within the scene.

Image of sparse point cloud and relative camera positions

If we look at just the camera locations on a map, we see something like the following:

Figure of camera center points in relative space

For the astute student however, it should not be surprising that the coordinates of these points are not recognizable as any known coordinate system. For example let’s view the X, Y, and Z of the first three points:

id	X	Y	Z
1	-0.357585	-0.390081	-3.48026
2	-0.326079	-0.367529	-3.24815
3	-0.295885	-0.348935	-2.98469
4	-0.272306	-0.334949	-2.79409

This means we require some equation to convert between our unreferenced (but consistent) data to a known coordinate system. To build this equation, we just need to know four things about our data with some certainty — the start point and end point X and Y positions. We will ignore Z for this exercise.

Point 1:
X-Local: -0.357585
X-KUCS: 958632.326047712
Y-Local: 1.29161
Y-KUCS: 958744.221397964

If we remember our trigonometry (or google our trigonometry…) then we’ll be able to solve for our X and Y values independently. For example for X:

X1 = 67.8485 * X + 958657

With that and our Y equation:

Y1 = 27.2400 * Y + 19444469

Now we can transform our local coordinates into the Korean 2000 Unified Coordinate system, and get a much nicer result:

Figure showing corrected camera position points

If we perform a heat map on this output, we’ll see that we have spread out our duplicate geometries to their correct, non-overlapping spacing:

Figure showing corrected camera position heat map

Whew! Now to write some code which does this for us… .

Oh, wait! We forgot the final test. How do they look together (pre and post transformed — post transformed as stars of course):
Pre and post transformed points compared in single figure

But, as we know Google (or in the case of Korea, Naver) is the all knowing authority on where thing are. How does this bear out against satellite imagery?:

Pre and post transformed points compared in single figure with aerial for comparison

Woah! That works for me.

Posted in 3D, Analysis, Conference, Conferences, FOSS4G Korea | Tagged: , , , , | Leave a Comment »

FOSS4G Korea 2014 and the tale of the three Stephens

Posted by smathermather on August 31, 2014

Much is going on in Korea. It is and will be a place to watch for Open Source GeoSpatial, with the likes of Sanghee Shin and all his local chapter OSGeo compadres leading the charge. FOSS4G Korea was part of Smart Geospatial Expo 2014 this year, and during the expo, Sanghee was awarded a prize from the South Korean president for his work in promoting Open Source geospatial technologies. To hear Sanghee explain it, Korea is very much interested in growing its industries through the minds of their people. The great successes of South Korea in the recent past have not been dependent upon the natural resources of Korea, but the intellectual capital of firms such as LG and Samsung. With this in mind, Korea wants to grow it’s software industries from native seed. Sponsorship of Open Source GeoSpatial technologies will be part of this initiative. It seems that they have the talent, the energy, the love of topic, and now the financial resources to start leading. This will be really fun to watch.

But, for this post, I want to focus on the tale of three Stephens, perhaps an echo of narcissism, but an interesting filter for our post today. Not long before I left for this trip, I contacted Sanghee to ask if there were any parks + GIS folks I could connect with while here to talk. Photo of Mr. Yu, B.J. Jang, and Stephen Mather at Smart GeoSpatial Expo 2014 It turns out, Mr. Byeong-hyeok Yu was presenting at FOSS4G Korea on the use of QGIS for remote sensing for the Korean National Park Service. Mr. Yu is one of two GIS people working at KNPS, the other is with their research branch. Mr. Yu was good enough to invite me to headquarters, give me the overview on smartphone apps, QGIS analyses, UAS (drone) flight work, Google StreetView like trail work (in partnership with Naver maps, I believe), and other cutting edge initiatives they are working on. What FOSSGIS has enabled under Mr. Yu’s stewardship is the ability for not just Mr. Yu to do GIS, but also for him to democratize the process and allow a few hundred of KNPS park rangers to use QGIS on their desktops as well as to be the bearers of the equipment for the trail camera project. I like to think of Mr. Yu as a more bright and energetic version of me. After all, he is a FOSS4Geospatial parks guy. So, we’ll call Mr. Yu Stephen Mather number 2 (I did call this narcissism, right).

But here’s the real reason for the tale of three Stephens. Whoever has studied the history of the National Park Service in the US knows of Stephen Tyng Mather, the borax mining magnate and essential founder of the National Park System, who oversaw the development of 20 National Parks in his short tenure. While visiting the headquarters of KNPS, I was given the privilege of an audience and traditional Korean tea ceremony with tireless servant, Stephen Tyng Mather equivalent, and employee number 1 of Korea NPS, Mr. Young-Deck Park. Mr. Park has seen the KNPS from an idea to 21 parks covering more than 3-percent of Korea’s land area (more than 6% if you include the ocean refuges). In short, I was in the presence of a parks visionary and giant. Stephen Mather number 3. I won’t lie. I had to hold back tears. Oh, and the green tea was the best I have ever had.

To close this post, I’ll show you a view from KNPS’ most beloved, and least remote National Park — Bukhansan, which sits north of the Blue House (Korea’s equivalent of our White House) and partially inside the city of Seoul. At 11 million visitors a year and about 19,000 acres, it is quite popular. I hiked one of the ridges with one of their rangers. More on that in another post.

A view from Bukhansan National Park over Seoul

Posted in Conference, Conferences, FOSS4G Korea | Tagged: , , , , | 1 Comment »

FOSS4G Korea 2014, poor GPS photos, and mapillary

Posted by smathermather on August 31, 2014

As I have been moving around, whether traveling to Seoul or within Seoul, I have taken a lot of pictures. Some have GPS and I’ve processed to sent to Mapillary, like a few hundred I took on a day wandering Seoul:

Screen shot of Mapillary overview of SeoulI’ve taken a lot of strange videos too. I took a couple videos of my feet in the subway train just to get the music that plays to notify passengers of an approaching stop. Walking around Bukhansan National Park, I have taken many sign pictures. As I work for a park district, how signage and wayfinding are handled here is facinating, both from what I can understand, i.e. choice of material, color, frequency, how the letters are carved, to those elements that I cannot yet understand, i.e. exactly how the Korean Language wayfinding portions work.

But mostly I have been cataloging as much as I can in order to give my children a sense and feel for the city. I am realizing this imperative has given me a child-like view of the city. (Of course, my enthusiasm for the mundane does get me the occasional funny look from locals… . But hey! What could feel more like home than people thinking I am a little strange.)

This blog wouldn’t be mine without a technical twist to the narrative, so let’s dive into some geographic problems worth solving: The camera I have has built in GPS and compass, which makes it seemingly ideal for mapillary uploads. Except the GPS isn’t that accurate, doesn’t always update from photo to photo, struggles in urban areas in general, etc. etc.. And so it is that I am working on a little solution for that problem. First let me illustrate the problem better.

A sanity check on the GPS of the data can easily be done in QGIS using the Photo2Shape plugin:

Screen snapshot of photo2shape plugin install screen

Screenshot of distribution of camera GPS points in QGIS

Let’s do two things to improve our map. For old-time sake, we’ll add a little red-dot-fever, and use one of the native tile maps, Naver, via the TMS for Korea plugin.

Naver map with photo locations overlayed as red dots

We can see our points are both unevenly distributed and somewhat clumped. How clumped? Well, according to my fellow GeoHipsters on twitter, hex bin maps so 2013, so instead we’ll just use some feature blending (multiply) plus low saturation on our red (i.e. use pink) to show intensity of overlap:

Capture of map showing overlap of points with saturation of color increasing where overlaps exist.

Ok, that’s a lot of overlaps for pictures that were taken in a straight line series. Also, note the line isn’t so straight. Yes, I was sober. No, not even with soju can I walk though so many buildings.

Like all problems when I’m obsessed with a particular technology: “The solution here is to use <particular technology with which I am currently obsessed>”. In this case, we substitute <particular technology with which I am currently obsessed> with ‘Structure from Motion’ or OpenDroneMap. ODM would give us the correct relative locations of the original photos. Combined with the absolute locations (as bad as they are) perhaps we could get a better estimate. Here’s a start (confession — mocked up in Agisoft Photoscan. Sssh. Don’t tell) showing in blue the correct relative camera positions:

Image of sparse point cloud and relative camera positions

See how evenly spaced the camera positions are? You can also see the sparse point cloud which hints at the tall buildings of Gangnam and the trees in the boulevard.

  • Next step: Do this in OpenDroneMap.
  • Following Step: Find appropriate way to correlate with GPS positions.
  • Then: Correct model to match real world.
  • Finally: Update GPS ephemeris in original photos.

So, Korea has inspired another multi-part series. Stay tuned.

Posted in 3D, Analysis, Conference, Conferences, FOSS4G Korea | Tagged: , , , , | 2 Comments »

FOSS4G Korea 2014 and FOSS4G 2015 (Korea!)

Posted by smathermather on August 30, 2014

I have much to say about my experiences this past week at FOSS4G Korea, but I’ll keep this very short for now. If you live in North America or Europe and are thinking about not going to FOSS4G 2015 because it’s too far, rethink it. Waiting for you on the other side of the world is Korea, a complex mix of old and new, mountain and ocean, kind and fierce. I can’t say enough good about the hospitality I have received, the food I have eaten, the quality of the conference attendees, the energy and the love for OSGeo.

For the moment, I will leave you with this image of a temple entrance near Bukhansan National Park which effectively says, “Endeavor to explore the world”. It’s a geographer’s temple, I think.

Image of temple

Posted in Conferences, Conference, FOSS4G Korea | Tagged: , | Leave a Comment »

FOSS4G Korea 2014

Posted by smathermather on June 3, 2014

This year’s FOSS4G is in Portland, September 8th through the 14th. It should be a great conference.

Related and relavent: yours truly will keynote and offer a half day workshop in PostGIS at FOSS4G Korea, 2014.

I cannot put into words how excited I am. Sanghee, representative of OSGeo Korean Chapter put the relevant info on FOSS4G Korea 2014 succinctly:

“FOSS4G Korea 2014 will be held at COEX, Seoul, on 27th and 28th August in conjunction with Smart Geo Expo, the largest geospatial event in Korea.”

So, if you are in Seoul at the end of August, look me up.  (Oh, and I should mention FOSS4G 2015 will be in Seoul too).

Screen shot of DuckDuckGo search for Seoul

Posted in Conference, Conferences, FOSS4G Korea | Leave a Comment »

And on that farm he had a cat… #cfasummit

Posted by smathermather on November 4, 2013

Two of the sets of conversations I really enjoyed at the CfA Summit in the 3rd day unconference were on the topic of Chief level technology positions, from Chief Innovation Officers to Chief Technology Officers to Chief Information Officers.  It was interesting to hear others’ interests, frustrations with both ends of recruitment, and the realization that there wasn’t a central repository for such position descriptions available.  So with the help of people in those sessions plus a few people after the Summit, I started a little repo to centralize the PDs and other relevant info for C*O tech positions.

Screen shot 2013-11-04 at 10.09.20 PM

From the repo:

“A number of C-level technology related positions have been created in local governance in the last few years, from traditional Chief Information Officer and Chief Technology Officer, to newer descriptions including Chief Data Officer, Chief Innovation Officer and others. This repo is meant as a source of C*O position descriptions and other relevant information for the creation of these leadership tech position in the civic sector.”

https://github.com/smathermather/EIEIO

A number of people have been involved in the discussion, including the following folks at the Summit:

Steve Allen, David Burns, Jennifer Anastasoff, Matthew Barron, Noel Hidalgo, Brian Chatman, Zachary Vruwink, Jake Levitas (and a couple I unfortunately forgot the names of…)

as well as a number of people who weighed in post-Summit, including: Jonathan Reichental, Bill Haight, Deborah Acosta, Yiaway Yeh, Vakil Kuner, Mark Headd, and of course, CfA’s own Abhi Nemani.

If anything is wrong in the repo, I take credit.  Anything right?  Check the above list of names.  Oh, and fork it, so I’m not the only one being wrong… .

Posted in CfASummit, Conferences, OpenGov | Tagged: , , | Leave a Comment »

Favorite Slide, #cfasummit

Posted by smathermather on October 22, 2013

So, if I had to choose one iconic slide from the CfASummit (and I do as I really need to catch up on sleep, so long posts will wait until the end of the week) this would be it presented by our illustrious MC (David Eaves) for the event:

1381940090486

The gist of the slide is that CfA has gone from the simple website/front end design overhaul within governance to addressing rules, processes, and culture, and is aiming for the root of the government stack, processes of procurement and their impact on tech in government, and establishing and “open by default” culture for governance.

There are three things I like about this diagram:  the analogy of government and software stack, the idea that these changes are part of a culture of “play” (which by the way is a wonderful encapsulation for the process of experimentation and learning taking place), and finally that the plan is to address through thought leadership and practical solutions underlying issues of procurement and culture that are at the root of the disconnect between government and technology.

Speaking of government, technology, and procurement, it’s refreshing how on-point the media that I have caught on healthcare.gov is.  Nice that this has moved from a niche issue to something that is being commonly discussed.

Finally, a quick link to my only (and humble and nacent) post on open government:  https://smathermather.wordpress.com/2012/12/13/architecture-of-an-open-data-initiative-fed-style/

Posted in CfASummit, Conferences, OpenGov | Tagged: , , | Leave a Comment »

Building something really useful #CfASummit

Posted by smathermather on October 20, 2013

I just came back from the CfASummit, 2013 and thoroughly enjoyed it.  I met a whole bunch of really dedicated, intelligent, and idealistic people who have been blessed with a position to change the world, the presentations made me laugh and cry (really).  CfA represents many the changes I have been hoping for in the public sector since I entered it 5 years ago, and many I hadn’t thought about.  And the Kool-Aid was amazing.

I have a few really good posts regarding these experiences in the queue that I haven’t finished yet, but just saw a post by Steve Citron-Pousty called “Hey Civic Hackers! How about leaving the ninja skills at home and building really useful applications?“.  Steve had 4 core points:

  1. At the very least, the only programming languages you should use on the server are (in decreasing order of priority) PHP, Java, or Python.
  2. Next up, most civic hacks should use a popular framework.
  3. Document your project more than you think you need to document it.
  4. Finally, in truth, most civic hacks should really make a plugin for a platform like WordPress, Drupal, Django, or Magnolia.

The core idea being civic hackers should be providing tools that can be changed and adopted by the municipality.  Responses on twitter have varied, with a nuanced response from @spara:

Screen shot 2013-10-20 at 10.54.20 AM

Screen shot 2013-10-20 at 10.50.20 AM

———————

So, I want to be nuanced too.  I have a lot of affection and respect for the CfA Fellows who serve our neighbors and partners in Summit County.  They have built a simple and beautiful interface for finding trails in the park systema.  I celebrate what they built, and have enjoyed watching this product develop.

https://trailsy.herokuapp.com/

But, one frustration I have had with the process was the balkanization that resulted from the project–here all partners (I am not employed by a partner, but I’m a trouble-maker with the Brigade) involved were hoping for some regional product from the work of the Fellows.  There was already locally commissioned work complete built on (gasp) PHP/Codeignitor application with similar intents.

The Fellows application is compatible with the existing application in no single way– not employing a similar back end, not sharing any front end code, not even sharing API compatibilities.

So let’s return to Sophia’s comment on twitter above: “We are there to start process of changing everything rather than bolting on to entrenched process and infrastructure.”  This, I am afraid, right or wrong, is the language of colonization.

———————

So where does this leave us regionally?  Now we have two related applications, which share no heritage, front end, back end code, nor API, and an expectation from local leadership to avoid the mistakes of the past of balkanization, those very same entrenched processes that CfA aims to undo.

So, when civic hackers enter a space and see change for the good happening, is it better to build that rails race car that Steve’s post points to, or should we be, in addition to changing the existing civic tech space also be celebrating and enhancing good things that are in progress?

———————

From a practical standpoint, how do we put the pieces back together?  I want to emphasize again, the great and meaningful work work the Fellows did.  But they will leave us in a few weeks with a real challenge.  Thoughts?

Code for each project can be found (naturally) on GitHub.

https://github.com/danavery/trailsy

https://github.com/cleveland-metroparks/trailsforthepeople

Posted in CfASummit | Tagged: , | Leave a Comment »

North American Cartographic Information Society (NACIS) Conference (yay #NACIS!)

Posted by smathermather on October 14, 2013

In a short blog post, I won’t be able to do the NACIS conference justice, but if you haven’t gone and you are a map geek, then I recommend you attend next year’s conference in Pittsburg.  First:

The People:

What a collegial and warm group of people.  NACIS was a very welcoming community, an interesting mix of private industry geniuses (ahem, Mapbox, Stamen, Vizzuality etc.), academics, students, National Geographic cartographers, and typically some serious Federal representation (although largely absent this year, a notable exception being Mamata Akella from NPSMaps who is technically not directly employed by the Feds and so was allowed to attend anyway).  As someone who spent the better part of a decade in the academic sector, it was fascinating (and comforting) to be back among (largely) introverts.  I would, however, argue the Nat Geo Cartographers were generally an exception to the introversion, but really pleasant and interesting folks as well.

Also, Andrew Hill from Vizzuality was there, so I picked his brain on features coming down the pike (including better Torque support, including a GUI soon), and bugged him to add pgRouting to the back end of CartoDB… .  He seemed receptive to this, and suggested following up under CartoDB support.

Presentations:

All the presentations were great.  From a very tech-practical standpoint, I enjoyed Carl Sack’s presentation on D3.  It was a great intro to D3js, and really got me over the basic barrier to using D3– getting the data in.  The TL;DR– order matters in the use of the API, queue.js is mighty useful, and all the data manipulation needs to go in the callback function to ensure your D3 goodness isn’t sunk by asynchronous execution.

The Maps:

Oh, boy there were some nice maps in the map gallery.  One trail map was an excellent theft of Swiss cartographic techniques, all the maps were interesting and well polished, and it was fun to look around at the diversity of approaches and topics.  Two maps of note that I really enjoyed– one is a map/infographic of wool exports from New Zeland, knitted in wool on a maker space constructed knitting machine:

Wool map

Also a really nice alternative piece was a bathymetry map by Carolyn Rose, which can be viewed at her blog: http://bathymetricbook.blogspot.com/

Other:

My presentation went well.  There were lots of questions about a project we’ve been working on for our public web interface to help people find trails, parks, picnic areas and other parky amenities.  We just posted the code for the project to GitHub (https://github.com/cleveland-metroparks/trailsforthepeople) and I’ll have my slides posted from the presentation shortly.  It was really great to connect with others working and starting to work in similar spaces of parks and recreation mapping.  I am already enjoying the follow-up and hope this builds in to a large spatial/web community servicing this sector.

 

Posted in Cartography, Conference, NACIS | Tagged: , | Leave a Comment »

 
Follow

Get every new post delivered to your Inbox.

Join 718 other followers