GIS5103 - Module 7 - Exploring & Manipulating Spatial Data

Module 7 was definitely a challenge.   We covered a lot of material in two chapters (6 and 7) of our text.   We learned how to check for, describe, and list data.   Next, we worked with lists, tuples, and dictionaries.   Lists are used to facilitate batch processing and exist for different types of elements.  "For" loops can be used to iterate over the elements in a list.  Elements in a list can be modified through operations such as deleting, appending, and removing.  Tuples are similar to lists but their elements are immutable.  If you use an operation that would modify the elements in a tuple it returns another tuple.  Elements in lists and tuples can be identified by their index number or location in the list or tuple.  Dictionaries contain item pairs.  The pair matches a key to its corresponding value.  Dictionaries can be modified however "keys" have to be unique but "values" do not.   For example a dictionary containing city and state pairs might have a "key" of the city "Greenville".   The corresponding state "values" for the city of Greenville could be different, such as Alabama, North Carolina, South Carolina.

Chapter 7 introduced us to using cursors to access data.  Cursors work similarly to list functions and can be used to iterate over rows in a table.  There are three types of cursors.   Search cursors are used to retrieve rows.  Insert cursors are used to insert rows.  Update cursors are used to update and remove rows.  All cursor types have two required arguments: and input table and a list of field names.  A SQL query can be used to establish criteria for the optional "where_clause" parameter of the cursor object.

This assignment tasked us to use Python code to create an empty geodatabase (gdb) and then copy the shapefiles from our Module7/Data folder into the new gdb.  Using the "cities" feature class that was now in the gdb we needed to populate a newly created dictionary with the names and population of every "County Seat" city in the state of New Mexico.  To perform these tasks it was important to understand the examples provided in our text and module exercise.  Syntax was very important in creating the python code needed for the search cursor that would find the cities that were defined as "County Seats".   The syntax format differs depending on the feature class type.  Each beginning single quote, double quote, parenthesis, or bracket needs to have a corresponding ending.   Placing "print" statements within the script helped identify the location of errors when the script failed to execute entirely.

Learning how to update a dictionary was also new.   This step was the most difficult for me to understand.   I had to think about where the search cursor was in the table and think about what were the "key" and "value" pairs.   Using the examples in the text for printing rows in the table, helped me understand where the cursor was in the table and how to update the dictionary.  

Here's my flowchart for the assignment script and a screenshot of the results after running the script.

Screen Shot of output from Mod7_MThrush.py

GIS5103 - Module 6 - Geoprocessing within Python

The video, exercise, and textbook were all helpful in completing this assignment.   I began by opening Python Win and creating my script.   After setting the workspace and the overwrite of output files, I worked on the Buffer tool parts of the assignment first as I knew about that tool from my previous classes.  

I watched the video again and read ArcGIS Help to understand the AddXY_management tool.  I added my GetMessages and return lines ("\n").

I found a few problems/typos after using Check and Running the program.  After fixing my errors the script ran successfully.

Finally, I ran the script within ArcMap so I could visually see what happened to the hospitals.shp file.

I learned the following:

1. That it is not necessary to import arcpy within ArcMap’s Python window

2. Unless you close ArcMap you do not have to repeat code that has been run in the Python Window even if you close the Python Window or clear it.

3. When using ArcGIS Help it is useful to use Search Tab and then click on Locate Topic and then go to the Contents Tab to see where in the Contents this tool, function, or class is located.

Module 6 Flowchart for Script Assignment

Printed messages after successfully running Module 6 script

GIS5103 - Peer Review Assignment #1

GIS Modelling of Intertidal Wetland Exposure Characteristics

ARTICLE SUMMARY

Melissa Thrush

A study by Nathan Crowell, Timothy Webster, and Nelson J. O’Driscoll (2011) examined the use of GIS to create a model to capture exposure to solar radiation to use in conjunction with existing wetland mapping to provide information on ecosystem characteristics.

Calculating an ecosystem’s exposure to solar radiation and patterns of tidal inundation could enable scientists to gain insight into the components and processes that create a sustainable biodiverse ecosystem.

Accurate modelling and computation of solar radiation exposure is difficult because data is often captured at specific sampling sites which cannot consider dynamic tidal inundation patterns.

Crowell et. al. believe that GIS can help solve the difficulty in accurate modelling and capture of these data.  They hypothesize that the use of LIDAR derived high-resolution DEMs (digital elevation models) along with “existing wetland mapping information” can create a successful model for solar radiation and atmospheric exposure (Crowell, 2011).  Their model would then be able to provide information on “exposure characteristics for each 4-m² cell within the study area” (Crowell, 2011).

The Southern Bight portion of the Minas Basin located between Kings and Hants Counties in Nova Scotia, Canada served as the study area.  This particular study region experiences a semidiurnal tidal cycle, which is two high and two low tides of approximately equal size every lunar day.  The area was also listed as a Wetland of International Importance in 1987.  Marine flats located within the study area “sustain the largest number of mixed-species shoreline birds in all of North America during fall migrations” (Ginn and Hamilton, 2009).

The LIDAR digital elevation data was acquired by the Applied Geomatics Research Group in Middleton, Nova Scotia.  Data for the western extent of the study area was obtained in 2003 and for the eastern extent in 2007.  Airborne systems were used to survey the area at low tide to establish a baseline surface.

A tidal inundation model was created using a Visual Basic script within ArcGIS the researchers created a model using cell elevations from the LIDAR raster data and tidal prediction data obtained in 2009 “to identify high-risk areas for flood damage from storm surge events and sea level rise” (Webster and Stiff), 2008.

A solar exposure model was created using sunrise and sunset times from 2009 to create two classifications of solar exposure, exposed or shaded. 

Three distinct vegetation classes, low salt marshes, high salt marshes and marine flats, were created whose spatial extents were acquired from the Nova Scotia Wetlands and Coastal Habitats Inventory in 2000.  Using GIS these delineated vegetation classes were located in the study area.

Using a Python script, within ArcGIS, an intertidal analysis of each layer was performed that displayed solar and atmospheric exposure characteristics for areas of seashore that are covered at high tide and exposed at low tide.

The study results showed, pretty convincingly, that oxygen availability and solar exposure were strongly correlated to the intertidal elevation (Figure 2).  The greater the intertidal elevation the more atmospheric and solar exposure.

The use of GIS enabled the researchers to create solar and tidal models that created realistic layers based on tidal elevations.  Using GIS tools, they extracted submerged tidal areas and were able to add a layer depicting the three different wetland classes.   Then correlations between wetland classes and their exposure to oxygen and solar radiation could be made.  Vegetation classes existed in areas with the highest annual atmospheric and solar exposure (Figure 4).    Understanding the various factors that contribute to the survival of various vegetation species can help scientists make more accurate predictions and create more robust restoration initiatives.  The researchers also noted that their models and analysis methods could be “used to analyze any spatially relevant data without change to the underlying methodology” (Crowell, 2011). 

Figure 2 - GIS Modelling of Intertidal Wetland Exposure, Crowell (2011)

GIS Modelling of Intertidal Wetland Exposure Characteristics

GIS5103 - Module 5 - Geoprocessing in ArcGIS

"Geoprocessing in ArcGIS allows you to perform spatial analysis and modeling as well as automate GIS tasks".   This module showed us how to use geoprocessing tools but the main focus was how to make using geoprocessing tools easier.   Batch processing can be used to run the same geoprocessing tool on different inputs.  Model Builder can be used to create a model that runs a sequence of geoprocessing tools through a visual interface.  Model elements such as variables, tools, and connectors are used to build a model that can be run and produce results.  Scripting can also be used to run geoprocessing tools in sequence.  Scripting with the Python programming language is more advantageous than using a model when more advanced programming logic is required.  Scripts can also be run outside of ArcGIS and be scheduled to run without user intervention.

In Part 1 of the assignment we used Model Builder to create a model that used two provided data layers (soils and basin).  First, however, we had to create a Toolbox because models have to be saved within a Toolbox.  The new model used the Clip tool to clip the soils feature to the basin feature.   After running the model the output result was then used as input to the Select tool in order to select soils features that were classified only as "Not prime farmland".   The output selection result was then used by the Erase tool as the erase feature to remove from the original basin layer.   The resulting output is shown below.  The dark blue outline represents the basin boundary.  The soils features that are within the basin boundary that are not classified as "Not prime farmland" are shown in lime green.

Basin feature shapefile with Soils features that are "not prime farmland" removed

We then exported our model to create a Python script.  In order for this script to run stand-alone we had to make some modifications.   We had to insert code to "overwrite" existing output filenames.  Below is a flowchart of the Python script.

Flowchart for Module 5 script

Next we created a script tool from our script.  A custom script tool is a way to integrate Python scripts directly into ArcGIS.   The tool can then be used or called again.

Finally, we shared our custom Toolbox containing our model and script tool.   We had to create a zip file that compressed together the Toolbox and the actual Module 5 Python script.

I encountered a few hiccups in this assignment.  When initially creating my model I realized I had to run each tool prior to using its output as input for another tool.  

In Part 1 Step 3 I had saved my model but forgot to save it after setting variables as Model Parameters.   When I exported the model as Python script I did not have the “if” statements in script because I had failed to save the model first.

I was unable to compress together the entire Scripts folder and the Toolbox located in the Module5\Results folder so I had to place a copy of the Module 5 Python script file in the Results folder so the zip file would contain both.

UPDATE:  I moved my Toolbox to S:\GISProgramming\Module5 and then recreated my script tool as it could no longer find my python script.  I then was able to successfully compress together the Module5\Scripts folder with my Toolbox. 

GIS5103 - Module 4 - Debugging and Error Handling

Debugging and Error Handling techniques were covered in this Module.   Syntax errors, exceptions, and logic errors can prevent your script from running or produce undesired results.  Common syntax errors are spelling, punctuation, and inconsistent or incorrect indentation.  Often these types of errors can be caught by eye, but the Check button in PythonWin is an easy way to locate and find syntax errors.   Our first task was to edit Script 1 and correct errors that were preventing it from running and printing the field names in a specified shapefile.   Python is case sensitive so variable names have to be consistent.   The error free script will produce the following results:

Results from successful run of Script 1

The following is a flowchart for Script 1:

Flowchart for Script 1

Script 2 our second debugging task, was more difficult.  Because of my limited knowledge of Python I found Chapter 10 of our Zandbergen text to be very helpful in explaining the context of the mapping functions called in the provided script.   There were 8 errors in this script and I used different techniques to help me locate and correct them.   Reading the script and using the Check button helped identify syntax errors.   I also used the debugger tool in PythonWin to step through the program line by line.  I became more comfortable using this tool and understanding the step over and step out tools.  I also used a try-except statement, so I would understand how to 'handle' exceptions.   I chose to comment out the try-except as it was not needed to successfully run the script.  This script, once error free, printed the layer names for layers found in a data frame of a specified shapefile.

Results after successfully running Script 2

Script 3 had two parts.   There was an error in Part A of the script that we were asked not to correct.   I found the error, but used a try-except statement to 'handle' it, instead of correcting it.  The try-except statement enabled the script to successfully run through the second part of the script, Part B.  The 'except' statement "handled' the error and printed the error message for Part A.   When using a try-except statement it is imperative that you indent the entire block that is being 'trapped'.  Part B printed the data frame name, the spatial reference, and the map scale.

Results of Script 3 after applying a try-except statement to trap and handle the error in Part A

While I initially felt I wasn't prepared to handle this Module due to my minimal knowledge of the Python language, I now understand the importance of having this module relatively early in the semester.   Debugging and error handling will be necessary in all future scripting endeavors and a basic understanding of the various techniques I am sure will prove invaluable in future assignments.