Smathermather's Weblog

Remote Sensing, GIS, Ecology, and Oddball Techniques

BASH script for adding directional / compass arrows to image

Posted by smathermather on April 29, 2015

I got a request today to extract the compass bearing from a set of photos, and add a rotated north arrow to those images in visual form so end users can see if the picture was while the camera was pointed north, or east, or NNW, etc.. Below is a photo pointing mostly East.

With BASH and a little Imagemagick, this was a pretty easy build. First we extract the direction info, and transform it to Cartesian coordinates so our rotation numbers will be correct. Then we rotate our arrow, maintaining transparency of background. Finally we use the “composite” command to overlay our rotated arrow with the original image.


(code edited to remove transformation to cartesian — rotation in imagemagick is polar…)


for image in `ls *.JPG`
	# trim off leading and trailing spaces in image variable
	image=`echo $image | xargs`
	# get non-extension portion of name
	name=`echo $image | awk -F'.' '{print $1}'`
	# extract angle from EXIF, and turn into variable
	angle=`exif $image | grep 'Image Direction' | awk 'BEGIN { FS = "|" } ; { print $2 }' | sed 's/M//g' | xargs`

	# create rotated and scaled arrow for image
	convert -rotate $angle -transparent white -resize 300x300 arrow-hi.png arrow-hi_$angle.png
	# overlay that arrow on original image
	composite arrow-hi_$angle.png $image $name.png
	# perform a wee bit of cleanup

	rm arrow-hi_$angle.png

addendum — almost forgot the repo:

Posted in Other | Leave a Comment »

Moar kite flight pics

Posted by smathermather on April 27, 2015


Posted in 3D, Bundler, Image Processing, OpenDroneMap, OpenDroneMap, Optics, Photogrammetry, PMVS, UAS | Tagged: , , , , , , , , , | Leave a Comment »

Kite flight (too windy for balloons, ahem “aerostats”)

Posted by smathermather on April 20, 2015

Inflation of aerostat

The aerostat hanger.

The end of the string.

The 9-footer is just so stable. But not enough wind to lift the cameras this day.

And so we send up the 16-foot workhorse. See that little dot? That’s the camera array.

The 16-footer flew nice and vertical, but pulled really hard. Processed images to follow soon.

Canon S100s from Kaptery — the silver one is an NIR adapted one; the black one is RGB color.

Edit: forgot the camera array:

CIR image from balloon:

IR image from the flight.

IR image from the flight.

Posted in 3D, Bundler, Image Processing, OpenDroneMap, OpenDroneMap, Photogrammetry, PMVS, UAS | Tagged: , , , , , , , , , | Leave a Comment »

FOSS4gNA Photos, well, ok, just San Francisco

Posted by smathermather on March 13, 2015

Posted in Other | Tagged: | Leave a Comment »

North Carolina GIS Conference

Posted by smathermather on March 7, 2015


NCGIS was an excellent conference, and this year was my first attending / presenting. Bravo to the organizers, who themed it Mobile and Global, but confessed that Mobile, Global, Open would have been more apropos.

To catch screen captures and audio for the presentations, see this temporary location:

which is organized by room numbers, so check out the schedule: .

My favorite new ideas from the conference, aside from talks by Randale Hale, Howard Butler, and Paul Ramsey were local to NC State University, with Helena Mitasova from the Open Source Geospatial Research and Education Laboratory.

See especially their Tangible Landscape which ties a sandbox in to real-time GRASS DEM processing (flow accumulation, viewsheds, fire modeling, etc.):

Screen capture of tangible landscape video.

Posted in Other | Leave a Comment »

OpenDroneMap Video for FOSS4GNA

Posted by smathermather on March 6, 2015

Thanks to some brilliant work by our Interpretive Technology Technician, we now have a little video for FOSS4GNA on OpenDroneMap. Come see my presentation next week in San Francisco, Weds. at 13:30 (1:30PM):


Posted in 3D, Bundler, Camera Calibration, FOSS4G, Image Processing, OpenDroneMap, Optics, Photogrammetry, PMVS | Tagged: , , , , , , , | 6 Comments »

OpenDroneMap — GCP file format

Posted by smathermather on March 1, 2015


So much documentation to catch up on for OpenDroneMap. Probably the most important missing element is documenting the ground control point file format. This will be a file name gcp_list.txt to be placed in the root of the image directory that you are processing.


The ground control point file associates locations in the input raw images with geographic positions, allowing for the output of an orthophoto and georeferenced mesh. It can augment or replace the need for geographic coordinates in the exif headers of the input images.


I mention this file briefly in a previous post, but let’s get into the weeds a bit on how to create one. The format is a simple text file. The header, or first line, is a descriptor of the coordinate system. This is not a formal thing, so don’t worry about format for this line.

The subsequent lines contain the x, y, and z of the ground control location in your local coordinate system, the pixel and line number of the location in the image, and the imagename itself. A minimum of 3 GCPs is necessary:

x y z pixelx pixely imagename

So, for example, if I mock up a version for the Langley dataset, I get something like this:

544256.7 5320919.9 5 3044 2622 IMG_0525.jpg
544157.7 5320899.2 5 4193 1552 IMG_0585.jpg
544033.4 5320876.0 5 1606 2763 IMG_0690.jpg

Note that my elevations are all the same. I was lazy, and assumed that there’s minimal topography and Langley, and that we are 5 meters above sea level. It is far better to put real values here, derived from an elevation dataset.

More on the how:

So, how to get these numbers? I loaded my raw unprocessed jpegs into QGIS. As I mouse over the image, it gives me coordinates, like “3044 -2622″. These are pixel values. I ignore the sign on the Y value.

For the X and Y values, I just used a Leaflet slippy map with a lat/lon pop-up: It would be better if I had GPS values from the field (preferably map grade) or survey values, but this quick and dirty adaptation of will at least get my map in the correct part of the world. I plugged the lat/lon values I get into Montana State’s UTM converter, though this is a slow method if I am doing a lot of points.

Screenshot of coordinate collection process

Screenshot of coordinate collection process

Screenshot of coordinate collection process -- a total of 3 points have now been collected.

Screenshot of coordinate collection process — a total of 3 points have now been collected.

Finally, I save my text file as gcp_list.txt, save it in the directory that contains my raw images, and run the script, and sit back and get some coffee. Output will be a subdirectory with a name similar to reconstruction-with-image-size-2400-results.

And now I’ve added this to the wiki:

Posted in 3D, Bundler, Camera Calibration, FOSS4G, Image Processing, OpenDroneMap, Optics, Photogrammetry, PMVS | Tagged: , , , , , , , | Leave a Comment »

OpenDroneMap — Now with animated GIFs

Posted by smathermather on February 20, 2015

Ok. I confess, clickbait. OpenDroneMap won’t make the animated gifs for you. But come on! it can make the 3D models which you can view in Meshlab or where ever, and then create animated gifs. Ok ok. I promised animated gifs. These are from data contributed to our test dataset repo by  Aaron Racicot and Tomasz Nycz respectively.

View of 3D textured surface model of Langley, BC

View of 3D textured surface model of Langley, BC

3D of kite aerial photography dataset

3D view of farm from Kite Aerial Photography (revised description)


Posted in 3D, Bundler, Camera Calibration, FOSS4G, Image Processing, OpenDroneMap, Optics, Photogrammetry, PMVS | Tagged: , , , , , , , | 7 Comments »

Korean Phonetics and Hangul

Posted by smathermather on February 16, 2015


I learn topics better if I talk and write about them. If you have been following this blog for long, you’ll know that I am always willing to post half-learned knowledge here. Today, it will be the Korean alphabet, Hangul.

What I like about learning Korean (people, food, culture, letters, language) is that, like culture in general, it is a complicated, many faceted and subtle thing. It is a real challenge to get good at, and as I get better at it, I slowly get to know fellow thinkers in a FOSS, half a world away. It satisfies my love of people (in general and in particular) and my love of learning.

In addition, the better I learn Hangul and the Korean language, the easier it will be to navigate Seoul this September when I return for Free and Open Source for GeoSpatial.


In a previous post, I mentioned the value of learning a little bit of Hangeul (or Hangul) the Korean alphabet in advance of going to FOSS4G in Seoul this year. I will confess that, as much as there is English text on most signs in Seoul, flying into a nation that uses a completely different system of writing is an exercise in immediate illiteracy. At one point I spent 30 minutes comparing signs in the well labeled and signed Seoul Metropolitan Subway in order to get on the train going the right direction. I got it all figured out, looking carefully at the Hangul (which I couldn’t read), and after all my effort, still got on the train going the opposite direction of my intent. Fortunately, that particular line is a loop… .

But, you don’t have to be as ignorant on arrival as I was. A small amount of study will make you a pro. I’m currently working through Living Language: Korean Essentials, which does a fine job. Wikipedia is also not a bad place to start, and where I got all of my history which follows.

A little History:

Hangul is the official and native alphabet of the two Koreas. It’s called Hangul in South Korea meaning either “great script” or “Korean script”, and called Chosŏn’gŭl in North Korea, which (if my understanding is correct) means “Korean script” but also could be interpreted as “North Korean script”, as Choson is North Korea, but also a historical name for Korea.

Hangul was created during the Joseon Dynasty in the 15th century. It has mostly replaced the use of Chinese characters to write Korean, a borrowed writing system called Hanja.

Why Hangul:

The cool thing about Hangul, coming from a Roman alphabet, is that it is not much different. Unlike a Roman alphabet, syllables are blocks, much like Chinese and other writing systems, but the blocks are phonetic, so themselves can be decomposed into their requisite sounds.

There are a few rules for syllables.

  1. must always start with a consonant
  2. must always contain a vowel
  3. must be either consonant then vowel, or consonant then vowel then consonant (diphthongs and iotized vowels included here as vowels)

Let’s take my name, for example. My full first name is “Stephen” which is pronounced like “Steven”, or more specifically “Steevun” or “Steevin”. In Hangul, that converts approximately to:


Let’s break it down into it’s parts:

ㅅ is the character “siot”, which is the equivalent of the letter “s”. (So far so good). Note our third rule: we can’t have two consonants in a row in Hangul. The way around this is to use the character “eu” which is a simple line ㅡ (pronounced like i is in “bid” but further back in the mouth). In this case, the “eu” is unpronounced, it is just playing its role as the required vowel.

ㅌ is our next character. This is “tieut” and is pronounced like the “t” in “tip”. It is followed by ㅣwhich is pronounced like “ee” in “beep”. Putting these two character blocks together, we have “Stee”.

So far so good. Like with all languages, there is no one-to-one correspondence for all sounds. Korean doesn’t have a “v” sound, so the closest letter is used: ㅂ, which is called “bieup”, and sounds like the “b” in “boy”. What follows is another ㅡ or “eu”. In this case, I think it’s not acting in a silent capacity but as that short “i” like sound. Now we have “Steebi” with a short “i” sound.

Finally our “N” sound is fulfilled by “nieun” or ㄴ. On this syllable, I need to phone a friend — I’m not sure if the pronunciation of this get’s it’s own separate audible syllable or not. So, I’m not sure of the final sound is “Steebinin” or “Steebin”. For our sticklers for proper translation the proper way to Romanize this would be more like Stibeuneun (I think), or just translate it back to what it was in the first place “Stephen”.

Whew! Try this with your own name. Try going to or and see if you can get a Korean version of your own name translated. Common English names (anyway) do have common translations, and you can double check by translating it back, though I don’t know about other languages. Or better yet, contact a friend, if you have one that speaks Korean. If not, make sure to come to FOSS4G in Korea this year. You’ll find many.



I was informed by Sanghee Shin that the final 는 is the suffix that flags a word as the subject in the sentence.

Posted in FOSS4G, FOSS4G2015 | Tagged: , , | Leave a Comment »

Normalizing tables in PostgreSQL: experiments with Fulcrum

Posted by smathermather on February 12, 2015

Fulcrum by Spatial Networks is an interesting and very useful hosted service for collecting geo data in the field with whatever smartphone or tablet you already own.  It works well, so long as you just want to collect points, and the field building capabilities are quite extraordinary. Also, for many applications, it’s cheaper than custom solutions, so it hits that sweet spot well. If you’ve been paying attention, I prefer my hosted solutions to be (nearly) purely open source, so as to avoid vendor lock-in. Fulcrum is sufficiently compelling, we are using it carefully and with a bit of joy, in spite of not meeting my normal preferences.

Screen shot of Fulcrum's form builder

So, I thought I’d share some table normalization we are doing on our wetland inventory data that we’ll be collecting with Fulcrum. The State of Ohio specifies a rapid assessment method for classifying wetlands called Ohio Rapid Assessment Method for Wetlands (ORAM) (warning– pdf). This assessment method has force of law, and on paper is a one-page, two-sided form with a range of physical, water regime, land cover and other characterizations, and to date we’ve always filled it out on paper, transcribed that at the office, and then populated spreadsheets, or more recently a PostGIS/PostgreSQL database.

Where we run into some difficulty is that some of our form values can be multiple values. Fulcrum returns a single field for these, with comma delimited contents:


Since we need to do lookups and calculations on these fields, we really need some normalization. Enter regexp_split_to_table.

CREATE TABLE nr.oram_metrics AS
  SELECT oram_id, 'm1_wetland_area'::text AS metric, regexp_split_to_table(m1_wetland_area, ',')::text AS selection
    FROM nr.cm_oram_data;

Now in our case, we have one lookup table for ORAM. Perhaps it should be multiple tables, but it’s just one discrete (ht. Brian Timoney for grammar fix) thing, this ORAM form, so we keep it simple. This means for each of our columns that should be normalized, we can throw them all in one table. We use UNION ALL to accomplish this:

CREATE TABLE nr.oram_metrics AS
  SELECT oram_id, 'm1_wetland_area'::text AS metric, regexp_split_to_table(m1_wetland_area, ',')::text AS selection
    FROM nr.cm_oram_data


  SELECT oram_id, 'm2a_upland_buffer_width'::text AS metric, regexp_split_to_table(m2a_upland_buffer_width, ',')::text AS selection
    FROM nr.cm_oram_data;

Finally, we can use a FOREIGN KEY constraint with a table that has all our metrics and selections plus our lookup value to complete the exercise:

CREATE TABLE nr.oram_metrics AS
  SELECT oram_id, 'm1_wetland_area'::text AS metric, regexp_split_to_table(m1_wetland_area, ',')::text AS selection
    FROM nr.cm_oram_data


  SELECT oram_id, 'm2a_upland_buffer_width'::text AS metric, regexp_split_to_table(m2a_upland_buffer_width, ',')::text AS selection
    FROM nr.cm_oram_data;

ALTER TABLE nr.oram_metrics ADD CONSTRAINT "metric_constraint"
  FOREIGN KEY (metric, selection) REFERENCES nr.oram_score_lookup_all (metric, selection);​ 

Posted in Database, Database Optimization, Fulcrum, PostGIS, PostgreSQL, SQL | Tagged: , , , | Leave a Comment »


Get every new post delivered to your Inbox.

Join 1,360 other followers