 |
||||||||||||||||||
 |
 |
|
||||||||||||||||
 |
||||||||||||||||||
|
||||||||||||||||||
|
Countless acres of land, outdoor recreation opportunities, and fish, wildlife and plants are conserved, and environmental protection actions taken every day through the strategic efforts of people working together. In natural resource conservation, outdoor recreation, and environmental protection, it is commonly understood that working together “works.” From handshakes between a landowner and state biologist to national agreements among Federal agencies and industry, partnerships and collaborative activities are vital tools for doing business. People involved in partnerships and collaborative activities recognize the enormous power of their combined efforts and resources. But working together costs money, requires staff time, can result in unanticipated outcomes, and sometimes just plain doesn't work. Partnerships and collaborations require trust, commitments of resources, and a willingness of all parties to assume some level of risk. Working together often requires investment and with that investment come expectations and a need for accountability. When we work together, it is understood that the whole likely will be much greater than the sum of its parts, but how do we demonstrate that one plus one can actually equal three? This Web site can’t answer that question, but it offers a number of tools and resources to help you determine how best to demonstrate the leveraging power of partnerships and collaboration in your work. Some of these tools can also help people engaged in partnerships and collaborative efforts monitor how they are doing and improve their efforts during the process. Measuring and monitoring how, when and where collaboration and partnering work – and where they don’t – is an emerging art and science. You are encouraged to expand this body of knowledge by e-mailing your suggestions, contributions, ideas and concerns. Measuring & Monitoring Tools & References
Logic ModelsOne tool that may be useful to you in developing measures for collaboration and partnering is a “logic model.” A logic model is a visual model that shows the relationship between your work and your desired results. It communicates the performance story of your project, focusing attention on the most important connections between your actions and the results. A logic model can serve as a basic road map for the project, explaining where you are and where you hope to end up. The following Web Sites provide information on how to develop a logic model and how to use a logic model as a tool to develop your project measures. Bureau of Justice Assistance Planning the Evaluation: Working with Program Logic Models Center for Disease Control Guide for Developing and Using a Logic Model (20 page PDF) Center for Disease Control listing of Logic Model Training Guides National Fish and Wildlife Foundation logic framework examples University of Wisconsin Extension Service logic model examples University of Wisconsin Extension Service online course on enhancing program performance with logic models U.S. Institute for Environmental Conflict Resolution – ECR Evaluation Framework (Logic Model) (1page PDF) W. K. Kellogg Foundation Logic Model Development Guide (72 page PDF) — Measurement The Pollution Prevention Resource Exchange (P2Rx), managed by the Pollution Prevention Resource Center (a non-profit organization that is the Northwest’s leading source of high quality, unbiased pollution prevention information) has a topic hub for environmental measurement. It has a step-wise approach for measurement, and a list of resources Sustainable Measures: Indicators of Sustainability is a Web site that explores indicators of community sustainability—ways to measure how well a community is meeting the needs of present and future members. It explain what indicators are, how indicators relate to sustainability, how to identify good indicators of sustainability, and how indicators can be used to measure progress toward building a sustainable community. It may provide useful context to you as you work to measure project success U.S. Environmental Protection Agency Region 10: Measuring Environmental Results is a Web page with measurement information and tools. — Evaluation Horizon Research, Inc. Taking Stock: A Practical Guide to Evaluating Your Own Programs U.S. Environmental Protection Agency Evaluation of Environmental Programs provides a gateway to evaluation products, tools, resources, and other information related to program evaluation University of Arizona Community Health Worker Evaluation Tool Kit (2page PDF, 21 pages total), arranged in an e-textbook format. It includes sections on: using logic models for planning, evaluation, and action; case studies; grant-writing tips; an evaluation framework; tools; references; and other information. W.K. Kellogg Foundation Evaluation Toolkit: This toolkit is designed to provide the foundation’s grantees with guidance. It is targeted primarily at those grantees who will be working with an external evaluator, but is helpful for anyone seeking to design an effective, useful evaluation. Indicators/Measures2005 CARE Program Measures (2 page PDF): To develop its program measures, CARE convened a cross-agency, cross-program stakeholder group—the CARE Tracking Team. The Tracking Team developed a logic model of the program and then a set of draft measures, the final set of which are displayed in the attached table. These measures are meant to guide internal program management and to help the program managers respond to external information requests. Association of Public Health Observatories' Good Indicators Guide: Understanding How to Use and Choose Indicators is intended to be a short, practical resource for anyone in any health system who is responsible for using indicators to monitor and improve performance, systems or outcomes. However, its content is widely applicable to other systems. Blue Mountain Forest Plan Revision: Draft Criteria and Indicators (1 page PDF) and Blue Mountain Forest Plan Revision: Desired Conditions (38 page PDF). These documents provide an example of the Montreal Process and Criteria framework as developed by the Malheur, Umatilla, and Wallowa-Whitman National Forests. Indicators are provided for the following criteria: social well-being, ecological integrity, and economic well-being. Deliberative Democracy Evaluation Checklist (Ernest R. House and Kenneth R. Howe, October 2000): The purpose of this checklist is to guide evaluations from a deliberative democratic perspective. Such evaluation incorporates democratic processes within the evaluation to secure better conclusions. The evaluator is still responsible for unbiased data collection, analysis, and arriving at sound conclusions. Heritage Research Initiatives: Understanding and Assessing the Qualities of Heritage Areas (1 page PDF): This document presents data and bar charts of outcome metrics that National Heritage Areas are using to evaluate their partnership work. National Park Service has done much in the last 5 years to develop indicators for tracking results of National Heritage Area partnerships. In addition to collecting data for performance metrics of tangible outcomes, NPS has also invested in case studies on the process of collaborative conservation and partnership networks. Montreal Process Criteria & Indicators: Side by Side Comparison of 2010 versus 2003: A summary table of revised indicators (9 page PDF) and examples of forest-based community criteria and indicators that have been proposed to be changed or added to the original list. Public Involvement Evaluation Measures (4 page PDF): Presents an array of measures that key researchers use to quantify and understand the outcomes of public involvement activities. May assist individuals seeking to expand the array of public involvement measures that might be considered in developing feedback and evaluation approaches. Sustainable Rangeland Roundtable: Core Indicators (1 page PDF): This document provides further iteration of criteria and indicators relating to ecologic, economic, and social importance of ranching and rangelands. Survey Questionnaires/InstrumentsAssessing Your Collaboration: A Self Evaluation Tool: Scholars identified common factors influencing the collaborative process. Given the importance of these factors, a self-evaluation tool was developed to assist existing and forming groups. The tool is a self-assessment exercise allowing groups to rate their collaboration on key factors. Key factors examined include goals, communication, sustainability, evaluation, political climate, resources, catalysts, policies/laws/regulations, history, connectedness, leadership, community development, and understanding community. The tool is found on a Web site that promotes the well-being of children, families, and the communities in which they live, but the self-evaluation of collaborations can apply to any “process through which parties who see different aspects of a problem [or issue] can constructively explore their differences and search for solutions that go beyond their own limited vision of what is possible." Community-based Collaboration on Federal Lands Across the Intermountain West: An Evaluation of Participant Satisfaction (11 page PDF): This link summarizes the preliminary findings of a research project designed to: 1) determine the effectiveness of community-based collaboration from the perspective of the participants; 2) identify which criteria or indicators of success are important to participants; 3) move beyond the use of case studies and conduct an aggregate analysis of data from many cases; and 4) test and refine the Participant Satisfaction Scorecard as a state-of-the-art instrument. (Participant Satisfaction Scorecard can be found at the end of the paper on page 10.) Mattessich, Paul W., Murray-Close, Marta and Monsey, Barbara R. 2001. Collaboration: What Makes It Work (2nd Edition). Saint Paul, MN: Wilder Publishing Center. See this book for a discussion of the Wilder Collaboration Factors Inventory (5 page PDF): This inventory is an online questionnaire for assessing the factors that influence the success of collaboration. Twenty such factors have been identified, and practitioners are provided with summary scores for each of these factors upon completion of the inventory. Copies of the inventory, professionally packaged and designed for distribution to members of the collaboration are available from the Wilder Publishing Center (1-800-274-6024 or www.wilder.org). Groups who use the survey instrument (in any form) are encouraged to notify Wilder Research Center to list their use (research@wilder.org). The Partnership Self-Assessment Tool: The Center for the Advancement of Collaborative Strategies in Health presents a Partnership Self-Assessment Tool designed to assess how well collaborative processes are working and how they can be improved. The questionnaire, instructions for using the questionnaire, and an action-oriented report are available on this site. Report to Deliberative Democracy Consortium: Building a Deliberative Measurement Toolbox (51 page PDF), by Peter Muhlberger: This document is a report on a toolbox of measures for evaluating democratic deliberation, a toolbox of use to practitioners and researchers of deliberation. The actual Toolbox and Supporting Results can be found starting on page 28 of the document. U.S. Environmental Protection Agency Feedback and Evaluation: The U.S. Environmental Protection Agency has put together a suite of questionnaires for gathering feedback from participants in public involvement activities (e.g., Public Meetings, Stakeholder Negotiations, Federal Advisory Committees, and Community Advisory Groups). These instruments, along with an evaluation toolbox, are available on this site. U.S. Institute for Environmental Conflict Resolution – ECR Participant End-of-Process Questionnaire (16 page PDF): The U.S. Institute for Environmental Conflict Resolution, in partnership with several federal and state agencies, has created a system to evaluate environmental conflict resolution and collaborative problem solving processes (e.g., mediations and facilitations). The evaluation system facilitates (a) performance management and reporting, (b) diagnosis of what factors influence success, and (c) continual learning and improvement when evaluation information is gathered, analyzed, and shared with practitioners, users, and other appropriate audiences. Other ReferencesThe Community-Based Collaboratives Research Consortium national conference "Evaluating Methods and Environmental Outcomes of Community Based Collaborative Processes" held September 14-16, 2003, presented research findings and environmental outcomes concerning community based collaborative projects. The conference featured the results of Consortium funded research projects as well as emerging developments in the field of community based collaborative approaches to environmental management. Community Indicator Project: Final Report to the Round Table on Sustainable Forests (40 page PDF): This report is based on the Community Indicator project, which was undertaken to learn the kinds of social issues communities feel are important to track via a community indicator process, and to compare the existing social sustainability Montreal Process Criteria and Indicators to these efforts. An Evaluation of Community-Based Collaborative Approaches: Evaluating collaborative approaches to public participation and resource management is an important step in drawing lessons to improve collaborative processes. In fall 2001, The Hewlett Foundation awarded a grant to the Montana Consensus Council (MCC) and the Consensus Building Institute (CBI) to test and refine the “participant satisfaction scorecard,” a tool used to evaluate the success of community collaborations. The term community-based collaboration (CBC) was defined as the use of partnering, facilitation, mediation, consensus building, and other “alternative dispute resolution” techniques to prevent and resolve public interest conflicts (i.e. conflicts involving local, state, and/or federal governments as at least one party to the conflict). The goals of the project were to: 1) evaluate how successful CBCs are from the perspective of participants; 2) determine which criteria or indicators of success are most important to participants; and 3) refine the scorecard as an evaluation tool. Forest Sustainability Indicator (194 page PDF): This guide is designed to advance understanding of how local, regional, and national efforts to develop sustainability criteria and indicators can be connected. It is intended to serve as an indicator 'tool kit' for forest-based communities that are working on maintaining and enhancing their natural resources as a basis for long-term economic, social and environmental health. A key component of the tool kit is the Montreal Process Criteria and Indicators framework, which helps assess the ecological, economic and social aspects of forest resources. Making a Difference: A British guide to evaluating public participation in central government: This guide to evaluating public participation is intended to help those involved in planning, organizing or funding these activities to understand the different factors involved in creating effective public participation. It helps planners set and measure attainable objectives, evaluate impact, and identify lessons for future practice. Using clear language, simple instructions, illustrative case studies and a glossary, this guide is a valuable tool for anyone involved in running or commissioning public participation in central government and beyond. PriceWaterhouseCoopers Primer on Developing Metrics for Sharing Partnership Success: On the Web site for communicating successes of public-private partnerships, there are answers to basic questions as: why should my partnership develop and use metrics, what is a metric, what are characteristics of effective metrics, and how do partnerships create effective metrics? There are comprehensive, in-depth discussions on such key topics as developing metrics, customizing and collecting metrics, creating the infrastructure required to support metrics, how partnerships use metrics to communicate, internal versus eternal metrics, and how to use feedback for continuous improvement. Red Lodge Clearinghouse: Collaboration Handbook: The Collaboration Handbook provides common sense and practical advice about putting a collaborative effort together and making it work. One chapter provides the how and why to monitor and evaluate your collaboration. Rural Voices for Conservation Coalition (6 page PDF): This is a set of performance measures developed by a diverse group of stakeholders and represents their view on what is important in measuring government performance in collaboration, ecosystem management, and support for rural economies. These measures were developed by stakeholders out of concern "that current measures are insufficient to access federal agency progress towards inclusive and integrated land management priorities". Study Circles Resource Center Evaluation: How Are Things Going? (13 page PDF): This guide, prepared by the Study Circles Resource Center, presents a range of evaluation tools that will help practitioners collect information about the process of recruiting and training facilitators, the quality of the facilitation, and the facilitators’ experiences. U.S. Environmental Protection Agency: How to Evaluate Public Involvement (2 page PDF): The purpose of this short brochure is first to encourage the reader to do evaluation and then guide that individual through the things to consider in the process of planning for and carrying out participant feedback and evaluation of public involvement. What Do We Know and Need to Know about the Environmental Outcomes of Collaborative Management? (11 page PDF) (Tomas Koontz and Craig Thomas, Public Administration Review 2006): Many tout the benefits of collaborative environmental management as an alternative to centralized planning and command and control regulation, but the excitement over collaborative processes has not been matched by evidence that these processes actually improve the environment. This article discusses that future research on collaborative environmental management should demonstrate whether collaboration improves environmental conditions more than traditional processes and newer market-based processes. Contact UsPlease tell us about any additional Measuring and Monitoring tool related to collaboration and partnering. |
|||||||||||||||||
|
||||||||||||||||||
| Accessibility | Disclaimer | FOIA | Privacy | USA.gov | ||||||||||||||||||
