URISA GIS Capability Maturity Model (CMM):
A Self-assessment Tool For GIS Management
March 30, 2017
Summary of the Federal Highway Administration’s Quarterly Webinar: Applications of Geospatial Technologies in Transportation
These notes provide a summary of the presentation discussed during the webinar and detail the question and answer session that followed the presentation.
The presentation is available upon request from the webinar speaker, Allen Ibaugh (firstname.lastname@example.org).
- Allen Ibaugh, AICP
GISP Representative of Urban and Regional Information Systems Association (URISA) & CEO
Data Transfer Solutions, Orlando, Florida
- Mark Sarmiento
Federal Highway Administration, Office of Planning (HEPP), Washington D.C.
Approximately 76 participants attended this webinar.
Mr. Sarmiento provided background about organizational assessments and capability maturity models in the transportation and geospatial fields broadly. In 2011, the Every Day Counts (EDC) program started to highlight technologies/applications that are available but need visibility to be applied within the transportation communities. Geospatial data sharing—sharing the transportation data and the tools to make informed transportation planning decisions—was one of them. Having the resources to support these related activities and applying them efficiently and effectively, however, continues to be a challenge for State Departments of Transportation (DOTs). Through an organizational self-assessment, or a capability maturity model, State DOTs and other transportation agencies can identify their strengths and weaknesses in how GIS is handled. By using the Urban and Regional Information Systems Association (URISA) Capability Maturity Model and similar organizational assessments, States can address these challenges head on.
Mr. Sarmiento noted that there was a CMM Peer Exchange in September 2016, at which attendees used the CMM tool for self-assessments. Participants at that peer exchange spoke about their experience using the CMM at a workshop at the GIS-T Symposium in Phoenix, AZ on April 9, 2017.
Webinar participants were asked to complete a short poll about their organization and experience with the GIS in Transportation program. There results of the poll can be found in the Appendix.
Mr. Sarmiento then introduced the presenter, Mr. Allen Ibaugh, CEO of Data Transfer Solutions (DTS) and member of URISA.
The Ongoing Geospatial Revolution: A Need for GIS Capability Maturity Modeling (GIS CMM)
Mr. Ibaugh’s presentation provided an overview of what a GIS CMM is and how GIS has grown to become a ubiquitous tool across many disciplines such as government, private industry, agriculture, and academia.
The future holds enormous promise for GIS applications, but we must be able to measure our capability to know where we are headed. GIS CMMs allow practitioners to do so, and are crucial in creating a sustainable model for building an agency’s GIS.
What are Organizational Assessments and GIS Capability Maturity Models?
Each organization is different and has unique characteristics, levels of resources, and abilities. Organizational Assessments are tools used to define and score an organization’s capacity and ability to perform a given set of tasks. GIS Capability Maturity Models (CMMs) are surveys that ask questions about the makeup of a GIS division within an organization and how GIS services are administered.
The URISA GIS CMM tool is a specific CMM for GIS users primarily at the municipality level, but has been adopted at the MPO and State DOT levels. URISA’s GIS CMM seeks to evaluate and visualize the operational effectiveness on an agency’s GIS programs and provide a means for how an organization can optimize their GIS infrastructure and return on investment.
Development of the URISA CMM
A first step in the development of the GIS CMM was URISA’s GIS Management Column, a matrix that evaluates an agency’s “execution ability” across multiple GIS functions. From here, URISA took a step back and developed the Geospatial Management Competency Model for the U.S. Department of Labor. This new model showed how the agency was performing across an array of metrics not focused on GIS functions, but factors including human resources, leadership development, and Geospatial Project Management.
The “ah-ha!” moment came by bringing together these operations and management models. The GIS Management Column and the Geospatial Management Competency Model were both overlaid against the standing body of geospatial management best practices and standards known as the GIS Management Body of Knowledge. The end result was the GIS Capability Maturity Model (GIS CMM), which is an assessment using a broad spectrum for both technical GIS capabilities and non-technical work, and rates them against the Body of Knowledge.
How it Works
To best understand how the GIS CMM works, Mr. Ibaugh demonstrated what the model framework looks like, showing the framework of the GIS CMM. The framework consists of two components: “Enabling Capability” and “Execution Ability.” Enabling Capability is the technology, data, resources, and related infrastructure that can be acquired or developed to support GIS operations and includes GIS management and professional staff. The ability of the staff to utilize the enabling technology is the Execution Capability.
Figure 1. The GIS CMM Framework
Enabling Capability (EC) Component
- EC1. Framework GIS Data
- EC2. Framework GIS Data Maintenance
- EC3. Business GIS Data
- EC4. Business GIS Data Maintenance
- EC5. GIS Data Coordination
- EC6. Metadata
- EC7. Spatial Data Warehouse
- EC8. Architectural Design
- EC9. Technical Infrastructure
- EC10. Replacement Plan
- EC11. GIS Software Maintenance
- EC12. Data back-up and security
- EC13. GIS Application Portfolio
- EC14. GIS Application Portfolio Management
- EC15. GIS Application Portfolio O&M
- EC16. Professional GIS Management
- EC17. Professional GIS Operations Staff
- EC18. GIS Staff Training and Professional Development
- EC19. GIS Governance Structure
- EC20. GIS is Linked to Agency Strategic Goals
- EC21. GIS Budget
- EC22. GIS Funding
- EC23. GIS Financial Plan
Enabling Ability (EA) Component
- EA1. New Client Services Evaluation and Development
- EA2. User Support, Help Desk, and End-User Training
- EA3. Service Delivery Tracking and Oversight
- EA4. Service Quality Assurance
- EA5. Application Development or Procurement Methodology
- EA6. Project Management Methodology
- EA7. Quality Assurance and Quality Control
- EA8. GIS System Management
- EA9. Process Event Management
- EA10. Contract and Supplier Management
- EA11. Regional Collaboration
- EA12. Staff Development
- EA13. Operation Performance Management
- EA14. Individual GIS Staff Performance Management
- EA15. Client Satisfaction Monitoring and Assurance
- EA16. Resource Allocation Management
- EA17. GIS data sharing
- EA18. GIS Software License Sharing
- EA19. GIS data inter-operability
- EA20. Legal and policy affairs management
- EA21. Balancing minimal privacy with maximum data usage
- EA22. Service to the community and to the profession
Mr. Ibaugh went on to explain how each component has its own scoring system and that the GIS CMM questionnaire was created with flexible levels of granularity for user responses allowing users to be as specific as they can be.
Figure 2. Example Question for Enabling Capability Component
For each question in the 'Enabling Capability' section, read the brief description. Check the implementation category.
- [ ] 1.00 Fully implemented
- [ ] 0.80 In progress with full resources available to achieve the capability
- [ ] 0.60 In progress but with only partial resources available to achieve the capability
- [ ] 0.40 Planned and with resources available to achieve the capability
- [ ] 0.20 Planned but with no resources available to achieve the capability
- [ ] 0.00 This is desired, but is not planned
- [ ] Not Applicable (This is a non-numeric response that requires an explanation of why this component should not be considered in assessing the operation.
|EC1. Framework GIS Data
Complete assessment for each data layer:
- Geodetic Control
- Administrative Units
Does the agency have access to adequate framework GIS data to meet its business needs? For the GISCMM, framework data corresponds to jurisdiction-wide common base layers as defined by the agency to meet its business needs.
For reference, refer to the NSDI framework data layers (see https://www.fgdc.gov/resources/whitepapers-reports/annual%20reports/2008/web-version/AppendixC.html).
(See also EC2 below)
Figure 3. Example Question for Execution Ability Component
For each question in the 'Execution Ability' section, read the brief question and description. Check the implementation category that best describes your agency's current status. Feel free to include any clarifying comments or questions.
- [ ] Level 5: Optimized processes
- [ ] Level 4: Managed and measured processes
- [ ] Level 3: Defined processes
- [ ] Level 2: Repeatable processes
- [ ] Level 1: Ad-hoc processes
|EA1. New Client Services Evaluation and Development
How does the GIS operation evaluate new agency business needs for GIS services and develop plans to respond to new client service requests?
- This component should include a timeline/turn-around response focus.
- Are new services evaluated against the agency strategic plan?
- Are new services evaluated against ROI data criteria…does it make financial sense?
- Level 5 - optimized process - requires looking at existing services also and evaluating them to provide optimized services.
Online CMM Tool
URISA realized the limitations of a paper-format assessment and decided to provide an online-version of the Enterprise GIS CMM via an online survey instrument. This online tool provides the GIS CMM self-assessment, compiles benchmarking metrics, and provides a feedback report with benchmark analysis and development recommendations.
CMM Tool Demonstration
Mr. Ibaugh provided a live demonstration of the GIS CMM online tool, pointing out how the interface works and demonstrating helpful features such as the ability of an agency to use and manipulate the benchmarking models to compare themselves against similar organizations. He also explained how to generate reports for each component and how to read them. These reports provide a visual, spider diagram (see below) of where an organization is excelling or lacking in the framework of each component. Each variable in the framework has its own axis radiating from the center of the diagram. Points corresponding with an organization scores for a given variable are plotted and connected via a colored line. The farther the line is from the center, the better the organization is performing. The GIS CMM tool can then use the information from the report to generate recommendations for improvements.
Figure 4. GIS CMM Spider Diagram
The GIS CMM allows the organization to evaluate how mature they are across the assessment areas, where gaps can be filled, and how the organization can improve, grow, and move towards where their target level of maturity.
Mr. Sarmiento notified the attendees that FHWA has been releasing program area self-assessments, such as transportation performance management, or safety data.
Return to top
Questions and Answers
Can target States be set to see how an agency is progressing against a target State level of maturity?
It is not pre-set in the GIS CMM that way currently, but it could be a good future enhancement.
Can multiple people login to an account and share an assessment?
Yes. It just needs to be set up initially for multiple users and tie each section to a certain user.
With State IT consolidation, many DOTs don’t operate with their own Department of Information Services (DIS). How does that change the CMM?
In those IT categories, you’d be assessing how the external DIS is performing for you within your scope of work.
Is the assessment free?
There is a nominal fee (~$300) for URISA members. There has not been a specific State DOT GIS CMM created yet, and this would probably utilize the same nominal fee format.
Return to top
Before the presentation began, participants were asked to respond to a few questions to assess the population present in the webinar. The results of the poll are shown below in Table 1.
Results of poll asked of webinar participants
|Who do you work for?
|Metro/Regional Planning Organization
|How did you learn about today’s webinar?
|GIS-T Yahoo! Forum
|GIS State DOT Managers email listserv
|FHWA GIS in Transportation website posting
|Have you ever visited the FHWA GIS in
|How many previous GIS in Transportation
webinars have you participated in?
Return to top