GIS5103 - Module 6 - Geoprocessing within Python

The video, exercise, and textbook were all helpful in completing this assignment.   I began by opening Python Win and creating my script.   After setting the workspace and the overwrite of output files, I worked on the Buffer tool parts of the assignment first as I knew about that tool from my previous classes.  

I watched the video again and read ArcGIS Help to understand the AddXY_management tool.  I added my GetMessages and return lines ("\n").

I found a few problems/typos after using Check and Running the program.  After fixing my errors the script ran successfully.

Finally, I ran the script within ArcMap so I could visually see what happened to the hospitals.shp file.

I learned the following:

1. That it is not necessary to import arcpy within ArcMap’s Python window

2. Unless you close ArcMap you do not have to repeat code that has been run in the Python Window even if you close the Python Window or clear it.

3. When using ArcGIS Help it is useful to use Search Tab and then click on Locate Topic and then go to the Contents Tab to see where in the Contents this tool, function, or class is located.

Module 6 Flowchart for Script Assignment

Printed messages after successfully running Module 6 script

GIS5103 - Peer Review Assignment #1

GIS Modelling of Intertidal Wetland Exposure Characteristics

ARTICLE SUMMARY

Melissa Thrush

A study by Nathan Crowell, Timothy Webster, and Nelson J. O’Driscoll (2011) examined the use of GIS to create a model to capture exposure to solar radiation to use in conjunction with existing wetland mapping to provide information on ecosystem characteristics.

Calculating an ecosystem’s exposure to solar radiation and patterns of tidal inundation could enable scientists to gain insight into the components and processes that create a sustainable biodiverse ecosystem.

Accurate modelling and computation of solar radiation exposure is difficult because data is often captured at specific sampling sites which cannot consider dynamic tidal inundation patterns.

Crowell et. al. believe that GIS can help solve the difficulty in accurate modelling and capture of these data.  They hypothesize that the use of LIDAR derived high-resolution DEMs (digital elevation models) along with “existing wetland mapping information” can create a successful model for solar radiation and atmospheric exposure (Crowell, 2011).  Their model would then be able to provide information on “exposure characteristics for each 4-m² cell within the study area” (Crowell, 2011).

The Southern Bight portion of the Minas Basin located between Kings and Hants Counties in Nova Scotia, Canada served as the study area.  This particular study region experiences a semidiurnal tidal cycle, which is two high and two low tides of approximately equal size every lunar day.  The area was also listed as a Wetland of International Importance in 1987.  Marine flats located within the study area “sustain the largest number of mixed-species shoreline birds in all of North America during fall migrations” (Ginn and Hamilton, 2009).

The LIDAR digital elevation data was acquired by the Applied Geomatics Research Group in Middleton, Nova Scotia.  Data for the western extent of the study area was obtained in 2003 and for the eastern extent in 2007.  Airborne systems were used to survey the area at low tide to establish a baseline surface.

A tidal inundation model was created using a Visual Basic script within ArcGIS the researchers created a model using cell elevations from the LIDAR raster data and tidal prediction data obtained in 2009 “to identify high-risk areas for flood damage from storm surge events and sea level rise” (Webster and Stiff), 2008.

A solar exposure model was created using sunrise and sunset times from 2009 to create two classifications of solar exposure, exposed or shaded. 

Three distinct vegetation classes, low salt marshes, high salt marshes and marine flats, were created whose spatial extents were acquired from the Nova Scotia Wetlands and Coastal Habitats Inventory in 2000.  Using GIS these delineated vegetation classes were located in the study area.

Using a Python script, within ArcGIS, an intertidal analysis of each layer was performed that displayed solar and atmospheric exposure characteristics for areas of seashore that are covered at high tide and exposed at low tide.

The study results showed, pretty convincingly, that oxygen availability and solar exposure were strongly correlated to the intertidal elevation (Figure 2).  The greater the intertidal elevation the more atmospheric and solar exposure.

The use of GIS enabled the researchers to create solar and tidal models that created realistic layers based on tidal elevations.  Using GIS tools, they extracted submerged tidal areas and were able to add a layer depicting the three different wetland classes.   Then correlations between wetland classes and their exposure to oxygen and solar radiation could be made.  Vegetation classes existed in areas with the highest annual atmospheric and solar exposure (Figure 4).    Understanding the various factors that contribute to the survival of various vegetation species can help scientists make more accurate predictions and create more robust restoration initiatives.  The researchers also noted that their models and analysis methods could be “used to analyze any spatially relevant data without change to the underlying methodology” (Crowell, 2011). 

Figure 2 - GIS Modelling of Intertidal Wetland Exposure, Crowell (2011)

GIS Modelling of Intertidal Wetland Exposure Characteristics

GIS5103 - Module 5 - Geoprocessing in ArcGIS

"Geoprocessing in ArcGIS allows you to perform spatial analysis and modeling as well as automate GIS tasks".   This module showed us how to use geoprocessing tools but the main focus was how to make using geoprocessing tools easier.   Batch processing can be used to run the same geoprocessing tool on different inputs.  Model Builder can be used to create a model that runs a sequence of geoprocessing tools through a visual interface.  Model elements such as variables, tools, and connectors are used to build a model that can be run and produce results.  Scripting can also be used to run geoprocessing tools in sequence.  Scripting with the Python programming language is more advantageous than using a model when more advanced programming logic is required.  Scripts can also be run outside of ArcGIS and be scheduled to run without user intervention.

In Part 1 of the assignment we used Model Builder to create a model that used two provided data layers (soils and basin).  First, however, we had to create a Toolbox because models have to be saved within a Toolbox.  The new model used the Clip tool to clip the soils feature to the basin feature.   After running the model the output result was then used as input to the Select tool in order to select soils features that were classified only as "Not prime farmland".   The output selection result was then used by the Erase tool as the erase feature to remove from the original basin layer.   The resulting output is shown below.  The dark blue outline represents the basin boundary.  The soils features that are within the basin boundary that are not classified as "Not prime farmland" are shown in lime green.

Basin feature shapefile with Soils features that are "not prime farmland" removed

We then exported our model to create a Python script.  In order for this script to run stand-alone we had to make some modifications.   We had to insert code to "overwrite" existing output filenames.  Below is a flowchart of the Python script.

Flowchart for Module 5 script

Next we created a script tool from our script.  A custom script tool is a way to integrate Python scripts directly into ArcGIS.   The tool can then be used or called again.

Finally, we shared our custom Toolbox containing our model and script tool.   We had to create a zip file that compressed together the Toolbox and the actual Module 5 Python script.

I encountered a few hiccups in this assignment.  When initially creating my model I realized I had to run each tool prior to using its output as input for another tool.  

In Part 1 Step 3 I had saved my model but forgot to save it after setting variables as Model Parameters.   When I exported the model as Python script I did not have the “if” statements in script because I had failed to save the model first.

I was unable to compress together the entire Scripts folder and the Toolbox located in the Module5\Results folder so I had to place a copy of the Module 5 Python script file in the Results folder so the zip file would contain both.

UPDATE:  I moved my Toolbox to S:\GISProgramming\Module5 and then recreated my script tool as it could no longer find my python script.  I then was able to successfully compress together the Module5\Scripts folder with my Toolbox. 

GIS5103 - Module 4 - Debugging and Error Handling

Debugging and Error Handling techniques were covered in this Module.   Syntax errors, exceptions, and logic errors can prevent your script from running or produce undesired results.  Common syntax errors are spelling, punctuation, and inconsistent or incorrect indentation.  Often these types of errors can be caught by eye, but the Check button in PythonWin is an easy way to locate and find syntax errors.   Our first task was to edit Script 1 and correct errors that were preventing it from running and printing the field names in a specified shapefile.   Python is case sensitive so variable names have to be consistent.   The error free script will produce the following results:

Results from successful run of Script 1

The following is a flowchart for Script 1:

Flowchart for Script 1

Script 2 our second debugging task, was more difficult.  Because of my limited knowledge of Python I found Chapter 10 of our Zandbergen text to be very helpful in explaining the context of the mapping functions called in the provided script.   There were 8 errors in this script and I used different techniques to help me locate and correct them.   Reading the script and using the Check button helped identify syntax errors.   I also used the debugger tool in PythonWin to step through the program line by line.  I became more comfortable using this tool and understanding the step over and step out tools.  I also used a try-except statement, so I would understand how to 'handle' exceptions.   I chose to comment out the try-except as it was not needed to successfully run the script.  This script, once error free, printed the layer names for layers found in a data frame of a specified shapefile.

Results after successfully running Script 2

Script 3 had two parts.   There was an error in Part A of the script that we were asked not to correct.   I found the error, but used a try-except statement to 'handle' it, instead of correcting it.  The try-except statement enabled the script to successfully run through the second part of the script, Part B.  The 'except' statement "handled' the error and printed the error message for Part A.   When using a try-except statement it is imperative that you indent the entire block that is being 'trapped'.  Part B printed the data frame name, the spatial reference, and the map scale.

Results of Script 3 after applying a try-except statement to trap and handle the error in Part A

While I initially felt I wasn't prepared to handle this Module due to my minimal knowledge of the Python language, I now understand the importance of having this module relatively early in the semester.   Debugging and error handling will be necessary in all future scripting endeavors and a basic understanding of the various techniques I am sure will prove invaluable in future assignments.

GIS5103 - Module 3 - Python Fundamentals II

In Module 3 we were tasked with editing and completing a provided script.  After reading the required texts for this module I was initially weary of the assignment.   Many moons ago, programming did not come naturally to me.   In various work experiences I usually edited existing code instead of creating it.  So, I was happy to start the assignment by fixing/repairing some syntax issues in the provided code in Parts 1 and 2.   Part 1 emphasized the fact that a module has to be imported at the beginning of the script in order to use the functions of that module in the script.  Part 2 of the script was a dice game based on the length of the player's name.  The two syntax error corrections pointed out that case and type are important to producing an error free script.

Part 3 of the lab required us to generate a list of 20 numbers between 0 and 10 and it told us to use the "while" loop structure.   This made sense based on our reading of loop structures and the "while" loop requiring an exit condition; in our case the 20th number being generated.  At first I was stuck on how to put the generated numbers in a list but the assignment hint said to create an empty list first.   The Zandbergen text helped me figure out my next steps and the modules to use.

Flowchart for Step 3

Part 4 had us create a branching block and a loop structure using the list created in Part 3.  Again, the clues provided in the assignment steered me in the right direction.   A sketched flow chart helped me start the branching block and I also used the Zandbergen text to select the appropriate modules for both the branching and loop structures.  It was awesome seeing the results print on the interactive window.   I ran the script several times to make sure the correct result printed for each condition depicted in the script.   

I initially thought about receiving input from the script user to assign the "unlucky" number but quickly realized this was a little more complicated because I would have to take into account that the number received could only be between 0 and 10.   I would also have to create code to check for the length of the input and then truncate it to a value less than or equal to 2 in case a user typed in a value greater than 10.  I may still try to figure this out.

Flowchart for Step 4

Python code results for dice game and list creation and removal of unlucky number

GIS5103 - Module 2 - Python Fundamentals Part 1

In this week's module I found the lecture assignments and exercises very valuable in assisting with the lab assignment.  We were tasked with creating a script that given a string full name would then split the full name and print only the last name.  The script then had to calculate the length of the last name assigned in the original string and then multiply that length by three and print the value.   Because variable names were provided in the assignment, I had a better understanding of the script code.  My first script contained seven lines and produced the desired results.

Results of Mod2_MThrush.py running script in PythonWin

After creating the script and running it successfully, I created a flow chart.  I know this process should be reversed; create the flowchart and then create the script.  This is a habit I will need to form as scripts will become more complex.

Flow Chart of Mod2_MThrush.py which prints the last name of an assigned full name string and the value of three times the length of the last name.

While I was completing the Process Summary, I realized there was a way to reduce at least one line of code in my script.  After successfully running my second script, I found an additional line that I could remove.  My final script contained 5 lines of code and produced the same results.  There is  always more than one way to solve a problem or "skin the cat", especially with scripting/programming.   Editing and modifying my original script enabled me to get some experience using Python Win and to better understand Python syntax.

GIS5103 - Module 1 - Introduction to Python

In the first module of GIS Programming we were introduced to programming with Python.   We first learned about scripting versus programming and the layout of our Python textbook.   Next we were introduced to different Python editors.   Specifically we used Python's Integrated Development Environment (IDLE).   We learned about an interactive interpreter and its use to interpret and immediately carry out commands.   You cannot save scripts in an interactive interpreter.   In a script window you must first save the script before it can be run.   You can simultaneously show the interactive interpreter and the script window when creating a script.  Syntax highlighting is the use of different colors to highlight statements, strings, etc. as a means to error check as you write code.  We also were introduced to the PythonWin editor which we will use in this class along with the Python window within ArcMap.  We learned how to create pseudocode for given problems/objectives and to then translate into Python syntax.   We were provided a Python script that when Run created 12 module folders within a Programming folder on our S: drive.

CreateModuleFolders Python Script

I am a little nervous about this class as it has been quite some time since I took a programming course.   Programming does not come naturally for everyone and a quarter of a century ago, I found it difficult.  Hopefully, with some years and experiences behind me it will be easier this go around.