# data[].attributes.attachment_link.readable_text
New York -The Children's Aid Society and The Finance Project today announced the release of a case study whose promising results and companion guide will enable community schools around the country to demonstrate a significant social return on investment (SROI) for this proven education reform strategy, which brings together under one roof essential academic, health, parent engagement and social services.

In the first analysis of its kind of the community schools model, The Finance Project piloted a methodology to examine the SROI of two Children's Aid community school sites in Washington Heights, P.S. 5/Ellen Lurie Elementary School and its sister site, the Salomé Ureña de Henriquez Campus (which houses grades six to 12). The case study shows that every dollar invested in the schools over a three-year period produced a $10.30 and $14.80 return in social value respectively.

"Calculating social return on investment provides a sound and practical way to measure and communicate the value of community schools," said Cheryl D. Hayes, president of The Finance Project and co-author of the case study and guide.

The study identifies eight primary goals for the schools-among them readiness to enter school, academic success, health and safety, and family and school engagement-and analyzes more than 40 measurable outcomes required to achieve them. While community schools serve a wide range of stakeholders, the beneficiaries of the schools' activities are defined for the study as infants and young children, from birth to age five; students; families; and the school community.

After extensive research of third party sources, the authors assigned financial values to each benefit and gathered cost data to reflect the monetary value-in terms of improved outcomes, revenues generated and costs avoided-of the resources needed to run community schools. Next, the deadweight, or portion of the benefit that would have happened regardless, was subtracted from the total costs and benefits. Once the benefits were adjusted to reflect the value of past investments in current dollars, or net present value, the figure was divided by the investment costs and the resulting ratio serves as the SROI.

In the U.S., the concept of social return on investment was developed by venture philanthropy fund REDF (also known as The Roberts Enterprise Development Fund) in the 1990s. The approach, which is a modified cost-benefit analysis, has been used previously to evaluate investments in workforce development, prison alternatives and climate change interventions, among others.

The strong results from this case study bolster a decade of findings from independent evaluations of The Children's Aid Society's community schools. Previously, gains in math and reading, student and teacher attendance, and parent and caregiver involvement, as well as reports of a positive school climate, have all been shown to be higher in community schools than comparison schools. In addition, the schools' onsite health clinics have been demonstrated to prevent costly emergency room visits and provide more timely access to mental health services than traditional schools.

A companion publication provides a comprehensive, three-step guide, complete with tools and worksheets, to help community school leaders measure and communicate the social and economic value of their schools and programs.

The community schools chosen for the case study provide services in four major areas: expanded learning opportunities (including after-school and summer programs); onsite or school-linked health and mental health services; parent education and engagement; and other family support services. These services are integrated into the school day and provided to those most in need of academic and social boosts. The Children's Aid Society currently partners in 16 community schools in New York City.

With as many as 5,000 community schools currently operating in 44 states and the District of Columbia, serving an estimated 5.1 million students, the case study provides evidence of the effectiveness of a model that is proliferating in its third decade and gaining significant attention from lawmakers at the federal, state and local level.

A January report to the governor by the New York Education Reform Commission recommended reorganizing schools "by integrating social, health and other services through community schools to improve student performance." Community schools run by Children's Aid are cited in the report as a "homegrown success story" and "model of education-centered, comprehensive" programs. In New York City, the City Council speaker has endorsed the "strategic, coordinated" approach of community schools as part of a four-point strategy for the future of education here.

"It is a huge leap forward to have a standard for judging how community schools bring a solid return on investment and ensure student success," said Richard R. Buery, Jr., president and CEO of The Children's Aid Society. "With the release of these results and the dissemination of our guide, we hope to get one step closer to our goal of making every school a community school."

Download the case study here.
Download the companion guide here.

About The Children's Aid Society
The Children's Aid Society is an independent, not-for-profit organization established to serve the children of New York City. Our mission is to help children in poverty to succeed and thrive. We do this by providing comprehensive supports to children and their families in targeted high-needs New York City neighborhoods. Founded in 1853, it is one of the nation's largest and most innovative non-sectarian agencies, serving New York's neediest children. Services are provided in community schools, neighborhood centers, health clinics and camps. For additional information, please call Anthony Ramos at (212) 949-4938/ (917) 204-8214, email anthonyr@childrensaidsociety.org or visit www.childrensaidsociety.org.

About The Finance Project
The Finance Project is an independent nonprofit research, training, consulting, and technical assistance firm for public- and private-sector leaders nationwide. It specializes in helping leaders plan and implement financing and sustainability strategies for initiatives that benefit children, families, and communities. Through a broad array of tools, products, and services, The Finance Project helps leaders make smart investment decisions, develop sound financing strategies, and build solid partnerships. To learn more, visit http://www.financeproject.org.

# data[].attributes.attachment_link.title
Community Schools Model Shown to Offer Significant Social Return on Investment in Landmark Case Study

# data[].attributes.attachment_link.description
New York -The Children's Aid Society and The Finance Project today announced the release of a case study whose promising results and companion guide will enable community schools around the country to demonstrate a significant social return on investment (SROI) for this proven education reform strategy, which brings together under one roof essential academic, health, parent engagement and social services.

# data[].attributes.attachment_link.readable_text
By the time students poured back into schools this fall, the most disruptive impacts of the COVID-19 pandemic seemed to have finally receded. But the lingering effects on children and learning are unfortunately still very much with us.

New data from the National Center for Education Statistics show that the pandemic erased more than two decades of progress in reading and math for 9-year-old students. The effect was most profound for students from low-income communities-exacerbating the pre-pandemic achievement gap between those students and their higher-income peers.

​ Outside of school, the pandemic also magnified long-standing geographic and racial inequities in economic opportunity and overall health and well-being. A 2020 report from the Initiative for a Competitive Inner City (ICIC) found that approximately 78% of high-poverty neighborhoods in the U.S.-communities of color in particular-were highly vulnerable to the pandemic's economic impacts, including loss of jobs and income, compared to just 15% of low-poverty neighborhoods.

Federal relief funding is helping states and localities address these challenges. Large cities and counties have committed a significant amount of their State and Local Fiscal Recovery Funds toward projects in economically disadvantaged communities. And according to the National Conference of State Legislatures , over $9 billion has been allocated for K-12 education and related purposes, including after-school programs and programs for students' emotional and mental health needs.

But money alone isn't enough. Now is the time for local leaders to not only invest in families and communities, but to invest differently . Given the intricate relationship between neighborhood well-being and school performance, championing and investing in community schools -a model focused on leveraging and coordinating the resources and voices of the entire community to support a thriving educational environment-could be one of the best ways for mayors and other local officials to confront both types of challenges.

The relationship between schools and neighborhoods runs in both directions

Research consistently that a neighborhood's overall well-being and localized school performance go hand in hand.

Evidence demonstrates that children raised in disinvested communities where residents' overall well-being is poor-measured by factors such as high rates of , joblessness, physical disorder , and -generally have lower levels of academic achievement, including lower high school graduation rates , than peers from wealthier neighborhoods. Insofar as school quality is often measured by student achievement, a high concentration of low-achieving students translates into an overall low-performing neighborhood school.

Schools with low achievement levels can, in turn, impact the overall well-being of their neighborhoods. For one, we know that school performance affects families' decisions about whether or not to live in a particular neighborhood; a National Association of Realtors found that 53% of homebuyers with school-age children said that school district quality is an important factor when buying a home. Strong demand for communities with high-performing schools raises home values and generates wealth and investment in certain areas. But the reverse can also be true in areas where school quality is-or is perceived to be-low.

The cycle can be vicious. Low levels of academic achievement compromise students' long-term educational and workforce outcomes, which diminishes their income and wealth-building opportunities far into adulthood. Given low rates of neighborhood mobility in the U.S. among low-income residents (particularly low-income Black residents ), those individuals are then more likely to be concentrated in underinvested communities that often have their own underperforming schools .

Investing in a community schools approach for students, families, and neighborhoods

What if local leaders supported and scaled a different approach to K-12 education that fostered a feedback loop between schools and neighborhoods?

In 2022, four national partners-Brookings's Center for Universal Education, the Children's Aid National Center for Community Schools, the Institute for Educational Leadership's Coalition for Community Schools, and the Learning Policy Institute-collaborated with education practitioners, researchers, and leaders across the country to strengthen the community schools field in a joint project called the Community Schools Forward task force.

According to a forthcoming resource from the task force, a community schools strategy " transforms a school into a place where educators, local community members, families, and students work together to strengthen conditions for student learning and healthy development. Together, as partners, they organize in- and out-of-school resources, supports, and opportunities so that young people thrive." As part of their work, the task force has identified key practices that make for highly effective community schools:

Collaborative leadership, shared power, and voice among parents, students, teachers, principals, and community partners.

Expanded and enriched learning opportunities through after-school, weekend, and summer programs.

Integrated systems of supports such as health care, nutrition support, and housing assistance for families.

Rigorous, community-connected classroom instruction that emphasizes real-world learning and reflects lived experiences.

A culture of belonging, safety, and care that inspires trust and a sense of community.

Each of these practices supports learning and student development. But it is the synergy among them that makes the community schools approach effective. Such interdependence requires local leaders to undertake new ways of structuring and staffing schools and school systems to sustain effective governance, partnerships, and resources. Beyond the educational benefits, investing in such an approach can also contribute to the creation of more prosperous, vibrant, and inclusive neighborhoods in several key ways:

Advancing economic prosperity Investment in community schools can yield important economic outcomes for both individuals and communities. In addition to strong, high-quality learning environments, some community schools offer adult learning, training, and skills development opportunities both during and after school hours. Others provide convenient access to community health and wellness services make it easier for adult residents to successfully find and sustain quality employment. If designed intentionally in response to community-defined priorities, such programming can also strengthen the connections between neighborhood businesses and workers in ways that help fulfill mutual employment needs. The most effective community schools actively engage local organizations and businesses , involving them as valuable partners. Not only do such partnerships bring more resources into the school, but businesses and organizations may see benefits from an improved school and community environment as well.

Enhancing the physical environment. Community schools provide a familiar, convenient physical venue for expanded educational activities, community services, and recreational opportunities when school is not in session-if leaders choose to invest in them. Aside from limited fee-based arrangements with local sports or recreational leagues, traditional school buildings sit mostly empty during off hours and the summer, even as high-quality neighborhood recreational facilities are often lacking, particularly in low-income communities. A robust community schools model would provide wider access to classroom, auditorium, gymnasium, and outdoor space for both structured and unstructured community use, supported by funding for physical improvements, upkeep, supervision, and quality programming.

Increasing social capital. According to the Community Schools Playbook , the community schools approach requires strong family and community engagement for "fostering relationships of trust and respect, building the capacity of all stakeholders and the school, creating empowered decision-making processes, and leveraging local resources and expertise to address educational inequities. " By engaging with families around needs, assets, and interventions, the model aims to build trust between families and schools with the goal of improving student outcomes. Creating opportunities for families to forge social ties with one another can also have spillover effects for the broader community; even relatively weak social relationships can be helpful sources of information about employment or services, while stronger relationships provide a sense of community pride, connection, and stability that makes a neighborhood feel like a home.

Fostering civic involvement. Finally, the community schools approach to family engagement helps cultivate leadership and relationships that, in turn, can facilitate greater civic involvement. By becoming stronger advocates and decisionmakers in their children's education, parents and caregivers may then be more likely to engage or lead in advocacy efforts to advance broader community goals (e.g., infrastructure, housing), or may utilize new organizing and leadership skills to respond more effectively to community challenges or unwanted interventions (e.g., a toxic use of land, teardown of a culturally significant building). In a similar way, civic involvement can also be cultivated in older students. Projects such as the at the University of California, Berkeley's Center for Cities and Schools aim to further civic engagement by providing young people with the opportunity to investigate and address real-world community issues in areas such as housing, transportation, public space, and schools.

There are notable case studies that exemplify how community schools allow these four spheres to work in synergy. In Austin, Texas, Northeast Early College High School was one of several area community schools to create a family resource center and partner with local nonprofits to provide health services, parenting classes, and student after-school programs. School enrollment doubled after a year, and graduation rates rose from 48% to 98% in five years. In New York, the Children's Aid Society's community schools have a yearlong program that trains parents to proactively advocate for their children while also furthering their own education and skill-building. Parents can participate in advocacy events, take literacy and English classes, and learn skills ranging from interior design to cake-making.

As these examples illustrate, investment in community schools can have multiple and reinforcing positive outcomes for communities-a "win-win" that should galvanize school and city leaders to support and scale the model.

The vital role of mayors and local officials in supporting community schools

Last year, the Brookings Task Force on Next Generation Community Schools outlined the role of the federal and state governments in community schools with a series of recommended steps, including prioritizing leadership and staffing to focus inter-agency efforts; promoting policies and incentives for state and local government investment; pursuing research on innovative community schools strategies; and providing sustainable sources of funding to scale the model.

But while federal and state policies are critical, community schools can't succeed at scale without committed leadership-not only within school systems, but also from local officials more broadly.

First of all, mayors, city managers, and councilmembers must be of family-centered community-building , with the recognition that their cities' well-being is contingent on the well-being of the families that live there. One way this can play out is through the establishment of dedicated agencies or "cabinets" that bring cross-sector systems together around shared accountability for children and families' success. For example, the Baltimore Children's Cabinet is a collective effort by multiple agencies-including the public school system, the Department of Transportation, the Health Department, and many others-to improve outcomes for the city's youth.

Second, local leaders must support community school efforts by the involvement of other agency leaders (e.g., parks and recreation, public health, planning, public works, workforce) into this work-whether through a "cabinet" as described above or through organizational structures that create a permanent table for sustained, interdependent collaboration. Community schools naturally involve multiple city agencies with missions and mandates that may otherwise have little to do with education; thus, it is incumbent upon mayors and city managers to ensure that agency leaders and staff have an explicit responsibility to work together to support the model. In 2020, Philadelphia Mayor Jim Kenney did just that by establishing the Office of Children and Families , which now oversees the city's community schools program, the parks and recreation department, the library system, the Department of Human Services, and several other initiatives.

Finally, city officials need to ensure that community schools efforts are substantially and sustainably resourced . This means encouraging the local philanthropic and nonprofit community to directly fund grassroots and other organizations that work with community schools, while ensuring that city agencies' annual budgets include community schools. This need not necessarily be "new" money, but rather a repurposing of existing resources. Cities are already spending millions of dollars annually on neighborhood improvement as well as workforce development, community health, and parks and recreation. It makes good fiscal sense to utilize the social capital and physical infrastructure that schools provide-blending and braiding public funds in new ways to achieve the multiplicity of goals embedded in the community schools model. Evidence already shows how doing so can yield a positive return on investment: A found that support for community schools creates significant economic benefits in the form of higher earnings for students who graduate from high school, as well as from the fiscal savings that accrue from reduced use of publicly financed social services when those students reach adulthood.

The COVID-19 pandemic cast a harsh light on disparities in neighborhood and family well-being, revealing the interrelated nature of the economic, social, and environmental factors that created and sustain those disparities. And just as communities' challenges are multifaceted, so too must be the solutions. In this spirit, championing the creation of community schools can help meet the dual ambitions of advancing school performance and neighborhood well-being -and ultimately create more prosperous, equitable, and resilient cities and regions for everyone.

# data[].attributes.attachment_link.title
Why local leaders should champion 'community schools' to improve student, family, and neighborhood well-being

# data[].attributes.attachment_link.description
By the time students poured back into schools this fall, the most disruptive impacts of the COVID-19 pandemic seemed to have finally receded. But the lingering effects on children and learning are unfortunately still very much with us.

# data[].attributes.attachment_link.title
Publications – UCLA Center for Community Schooling – UCLA Center for Community Schooling

# data[].attributes.headline
A Practical Guide to Advance Community School Strategies for Youth Voice and Leadership

# data[].attributes.subject
A Practical Guide to Advance Community School Strategies for Youth Voice and Leadership

# data[].attributes.headline
Community schools promote equity: We need more of them 

# data[].attributes.subject
Community schools promote equity: We need more of them 

# data[].attributes.attachment_link.readable_text
For those of us in the trenches of the community school movement, an increased federal focus on community schools couldn't come at a better time.

City leaders and others are increasingly aware of the power of community schools as an equity strategy. And equity is needed now more than ever as schools face hardships exacerbated by the pandemic.

No wonder local calls for community schools are growing louder and more frequent. Community schools become hubs for students and communities. Often open evenings and over weekends and summers, they bring together families, students, teachers and local organizations to identify and provide health, social and out-of-school-time support.

Headlines promoting the effectiveness of community schools sit atop research studies, op-eds and other forms of media.

In recent weeks, the U.S. Department of Education has invited applications for $68 million in grants for community schools, representing almost four times more money than was made available in 2018. And the Biden administration said it aims to dedicate as much as $468 million to community schools in 2023.

Our current predominant school design is akin to a traditional telephone: a simple two-way system of delivery wherein teachers teach and students learn. Community schools are more like smartphones.

The more school systems that apply for these grants, the greater the benefit to our nation's education as a whole.

The ultimate goal of community schools is to ensure open pathways to academic success. Research from the RAND Corporation and the Learning Policy Institute and National Education Policy Center (LPI-NEPC) show that community schools are a good investment and an effective change strategy.

Some community school proponents say that our current predominant school design is akin to a traditional telephone: a simple two-way system of delivery wherein teachers teach and students learn.

Community schools are more like smartphones. They have collaborative leadership structures that help educators connect, pool resources and think more expansively. Ultimately, they help kids succeed in school and beyond.

And if community schools are like smartphones, promoting them citywide is like an upgrade to a stronger, more powerful operating system.
Related: OPINION: To the rescue - The schools we need now are community schools
As a former public school teacher in Boston and an advocate of community schools, I've seen positive results when school systems adopt this holistic approach.

Consider these examples from different parts of the country: Education and community leaders in Cincinnati have worked together over the past 20 years to build a system of community learning centers that now supports all the schools in the district. New York City houses the nation's largest community school initiative, with 317 of its schools working in long-term partnership with public and private community resources. Oakland is close to finalizing its vision of becoming the nation's first community school district.

All three districts have developed strong and sustained community engagement and systems of support at the district level with the help of university or nonprofit "backbone" partners (including, for example, the United Way and Children's Aid for New York City; the Community Learning Center Institute and United Way for Cincinnati; and the Urban Strategies Council and Alameda County Public Health Department for Oakland) to help build capacity, manage shared data and help maximize funding.

What the three districts also have in common is a shared community-wide vision of what their schools should look like - and success in improving outcomes for students and families.

Three elements are vital to achieving such community-wide visions, according to a study I conducted.

The first is the sharing of data with local leaders to build political will and clarify goals, including studies showing that strong community schools can lead to better student and school outcomes. This data, in combination with pressure from community leaders, generates momentum.

The second element, clear communication strategies shaped by multiple stakeholders, ensures that everyone understands those goals. Public testimony in school committee meetings, for example, is one great way to spread the message.

Finally, a local steering committee helps drive political will and spread that clear communication. A committee that gathers key stakeholders around the same table can also fill service gaps and establish policy. Members may include the superintendent, school leaders and community advisers, as well as the heads of the housing, public safety and health services departments.

Such committees are vital to ensuring coordination among traditional governmental entities in service of education. The coordination gives schools more time to do what matters, including time to build and foster relationships with families and nonprofit partners. In fact, one subcategory of the federal grant program asks states to discuss how they will use a steering committee to govern, and to outline how they will scale community school districts.

Developing these larger visions in pursuit of the community school grants will amplify positive outcomes for students, their families and their communities.

Paul Reville, a Harvard professor who formerly served as the Massachusetts secretary of education, recently noted that "the community school philosophy is absolutely what we need in this moment. The challenge is . . . to systematize this approach rather than work it one school at a time."

I could not agree more.
Emily Woods is head of education for the Richard K. Lubin Family Foundation. Her research focuses on school districts that have addressed the adoption of community school policy. This story about community schools was produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for Hechinger's newsletter.

# data[].attributes.attachment_link.title
OPINION: Community schools promote equity: We need more of them

# data[].attributes.attachment_link.description
For those of us in the trenches of the community school movement, an increased federal focus on community schools couldn't come at a better time. City leaders and others are increasingly aware of the power of community schools as an equity strategy. And equity is needed now more than ever as schools face hardships exacerbated by the pandemic.

# data[].attributes.headline
Teacher-Centred Guide  - creating equitable change in schools

# data[].attributes.subject
Teacher-Centred Guide  - creating equitable change in schools

# data[].attributes.attachment_link.description
Step 02: Understanding your Program Using Logic Modeling

# data[].attributes.attachment_link.title
Evaluation Toolkit for Community Youth Programmers - Bank of Ideas

# data[].attributes.attachment_link.description
Evaluation Toolkit for Community Youth Programmers - Bank of Read more about evaluation, programmers, toolkit, offord, studieshands and responses.

# data[].attributes.attachment_link.readable_text
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking "Accept All", you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.

# data[].attributes.attachment_link.description
Welcome to the Collective Impact Forum Explore resources Join an event We're here to help communities collaborate to reach their goals. We provide resources, host events, and offer coaching for people working to advance equity and achieve systems change using a collective impact approach to collaboration.

# data[].attributes.attachment_link.readable_text
BONUS138 adalah situs judi online terpercaya 2023 yang mempunyai game slot gacor hari ini dengan slot88 sebagai penyelenggara agen judi online. Di situs judi slot online terbaik BONUS138 terdapat berbagai slot dan tips menang maxwin slot dimana terdapat keuntungan semua member bisa bermain permainan judi slot online dengan mudah melalui komputer atau smartphone. Member cukup membuat satu akun saja untuk menikmati seluruh game judi online terlengkap 2023 yang disedikan.

Permainan Slot Online

Banyak masyarakat indonesia yang sudah tidak asing lagi dengan permainan slot yang saat ini sangat disukai di negara Indonesia. Sebelum menjadi sangat populer seperti sekarang, permainan judi slot telah banyak mengalami berbagai perkembangan dari waktu ke waktu. Dimulai dari awal tahun 1890an dimana kemunculan mesin slot tradisional yang bisa dimainkan melalui casino hingga tahun 2023 dimana kita bisa bermain melalui internet yang dinamakan judi slot online. Berikut merupakan penjelasan singkat mengenai sejarah perkembangan slot yang perlu para member ketahui.

Sejarah Mesin Slot

Mesin Slot pertama kali dibuat pada tahun 1895 yaitu diciptakan oleh seorang bernama Charles Fey. Charles Fey adalah tokoh utama pendiri slot dimana orang tersebut tinggal di Amerika Serikat. Fey membuat mesin slot pertama dengan menggunakan mesin mekanikal berputar untuk menampilkan 3 buah cetakan besi yang disertai dengan banyak gambar buah-buahan atau simbol kartu poker yang sangat umum dimainkan oleh masyarakat dunia. Adapun beberapa simbol tersebut seperti: Simbol Cherry, Simbol Seven, Simbol Spade dan berbagai Simbol juga terdapat dalam 1 mesin. Pada tahun 1970, permainan slot ini sudah banyak sekali digemari oleh masyarakat dunia hingga akhirnya banyak yang mendirikan yang kita sebut sebagai Kasino. Sehingga hal ini tentu saja menjadi ladang bagi masyarakat dunia untuk bermain dengan mudah dimana saja dan kapan saja. Sudah tersedia diberbagai kota dan bahkan berbagai negara. Beberapa tahun setelah 1970, hadirlah teknologi yang kita sebut sebagai internet yang membuat mesin slot ini sudah jarang dimainkan oleh para pemain judi. Sehingga terdapat teknologi baru untuk membawa permainan judi slot ke era internet yaitu menjadi game slot online. Banyak para pemain yang sangat senang karena slot ini sudah bisa dimainkan secara online tanpa perlu datang ke tempat Kasino.

Asal Mula Game Slot Online

Charles Fey memang benar membuat mesin slot pertama tapi sayangnya orangnya tidak cukup pintar sehingga dia tidak memantenkan buatannya tersebut kemudian hal itu tentu saja menjadi incaran orang-orang kaya untuk memantenkan mesin slot ini dengan mendirikan berbagai pabrik untuk menciptakan mesin slot tersebut. Hal ini mengakibatkan slot sangat terkenal hingga saat ini. Hingga pada tahun 1907 terdapat seseorang bernama Herbert Mills warga dari Amerika Serikat mempatenkan game judi slot ini, sehingga banyak sekali perusahaan yang harus membayar royalti kepada Herbert Mills ini ditambah terdapat banyak inovasi yang dia lakukan dalam mempatenkan mesin slot tersebut.

Mesin Slot Berasal Dari Mesin Poker

Sejarah terbentuknya mesin slot adalah berawal dari mesin poker pada tahun 1887. Ide pembuatan mesin slot tentu saja tidak mudah. Namun setelah melalui beberapa tahap lahirlah mesin slot ini. Berawal dari perusahaan di New York Amerika Serikat yang bernama Pitt dan Sittman yang berhasil membuat sebuah mesin slot dengan memiliki 3 roll yang didistribusikan diberbagai tempat hiburan dan tentunya ditempat kasino. Permainannya sendiri sangat mudah untuk memenangkan mesin slot online yaitu hanya mengabungkan ketiga buah yang sama untuk mendapatkan hadiah, pada saat itu hadiah yang diberikan cukup bervariasi seperti rokok, minuman bahkan sebuah koin untuk ditukerkan ke duit asli. Hal ini membuat seorang mekanik memikirkan sebuah ide untuk membuat mesin slot lebih baik ketimbang dengan yang sekarang yaitu membuat mesin slot yang mengeluarkan bayaran otomatis dalam bermain. Setelah beberapa tahun, seseorang bernama Charles Fey menemukan fitur baru dari permainan judi slot online yang nantinya akan memberikan bayaran otomatis saat bermain melalui dibawah mesin.

Slot Berasal Dari Fortune Coin

Pada tahun 1976 game slot mendapatkan perubahan yang cukup signifikan dimana perusahaan Fortuner koin telah mengeluarkan sebuah video judi slot yang bernama Fortune coin dimana mesin judi slot ini menggunakan sebuah layar TV yang berukuran sangat besar pada masanya yaitu seukuran 19 inch. Hal ini tentu saja menjadi sebuah inovasi terbesar pada masanya dimana membuat permainan slot meningkat pesat dalam hal keuntungan maupun pemainnya. Akan tetapi inovasi ini banyak ditentang oleh berbagai pihak karena bisa aja terdapat kecurangan yang bisa merugikan para pemainnya

Mesin Slot Dari Bally Company

Pada Tahun 1967 seseorang bernama Bally telah melakukan sebuah inovasi pada permainan judi slot yang dimana ia telah membuat sebuah mesin slot bertema semprot elektrik yang bernama Money Honey, hal ini menuai banyak perbincangan pada permainan judi slot. Walaupun begitu Mesin Slot dari Bally Company ini mulai hilang perlahan dalam perjudian slot.

Slot Online Slot88

Sekarang pada jaman 2023 sudah tersedia sebuah judi slot online yang bisa diakses diseluruh dunia, bahkan casino daratpun menggunakan sebagai dananya untuk membuat situs judi online yang bisa digunakan untuk para customer bermain online dirumah. Setelah semua orang bisa bermain slot pada platform online tentu saja banyak sekali developer atau kita bisa sebut sebagai pengembang slot online yang bermunculan dimana-mana dan salah satu yang kita mau bahas adalah Slot88. Slot88 merupakan judi slot online yang sampai sekarang memiliki berbagai perubahan dan beberapa penambahan fitur yang dapat menyesuaikan dengan para pemain misalnya jika dimainkan di smartphone android bisa, komputer maupun laptop dan hingga kini tersedia berbagai agen judi online yang menawarkan berbagai permainan slot juga.

Slot Online Trend di Indonesia

Pada jaman kemunculan internet pada tahun 2000-an tentu saja telah membaw banyak dampak yang terjadi di dunia ini. Salah satu dampak terbesar adalah pada dunia perjudian yang awalnya judi banyak dilakukan di kasino tapi dengan adanya internet maka bisa bermain dirumah dan permainan yang disediakan juga banyak seperti live casino online hingga inovasi terbaru yaitu beragam pilihan tema permainan slot online. Pada awalnya terbentuknya judi online cuma memiliki beberapa permainan saja, dan permainan yang ada di indonesia biasa cuma ada poker, domino dan lain-lain. Seiringnya waktu permainan judi online barat pun dibawa ke indonesia seperti Slot Online, Blackjack, dan Casino. Hingga pada akhirnya slot online merupakan salah satu game terbaik saat ini karena hanya dengan modal receh, sudah bisa mendapatkan uang yang sangat besar dalam bermain judi online. Untuk permainan slot sama seperti slot darat dimana pada umumnya tersedia mesin slot klasik yang ada di smartphone anda dengan beberapa tombol roll dan simbol yang berbeda-beda. Namun semakin bertambahnya jaman di era digital ini maka slot terus berkembang dan pada akhirnya mempunyai tampilan yang menarik, permainan yang cukup efesien hingga terdapat maxwin yang akurat.

Keuntungan Bermain Slot Online Di Situs BONUS138

Ada banyak sekali kelebihan yang akan dirasakan member ketika bermain pada situs judi online BONUS138. Situs judi online bonus138 ini telah memberikan keamanan untuk para member yang terdaftar. Selain itu bagi para member setia yang bermain di Bonus138 akan mendapatkan berbagai keuntungan dalam bermain, bisa dilihat dibawah ini:

Situs Judi Online BONUS138 Berlisensi Resmi Indonesia

Kelebihan yang paling pertama yang tersedia pada situs gacor BONUS138 merupakan platoform judi online yang berlisensi resmi dan sudah di akui aman dan terpercaya di Indonesia. Dengan adanya lisensi resmi ini tentunya kita tidak perlu khawatir lagi untuk bermain karena sudah pasti terjamin aman, tanpa ada curang dan terpercaya karena sudah menerapkan sistem fair play, adil serta maxwin yang tinggi.

Customer Service 24 jam

Kelebihan kedua adalah setiap member akan mendapatkan customer service 24 jam yang disediakan nonstop setiap harinya. Member setia situs gacor BONUS138 bisa mendapatkan pelayanan ini jika merasa ada masalah pada akun-nya atau bisa juga digunakan untuk menanyakan perihal permainan yang tersedia dalam situs bonus138.

Bonus Gede untuk Member

Keuntungan terbesar menjadi member di situs judi online BONUS138 adalah mendapatkan banyak sekali bonus yang tentu bisa didapatkan setiap minggu bahkan setiap harinya dan yang pastinya ini menguntungkan kita sebagai pemain. Di situs BONUS138 ini semua memberu baru akan mendapatkan banyak bonus juga.

BONUS138 Menjamin Privasi Keamanan Data dan Informasi Member

Dengan linsensi resmi, situs judi slot online BONUS138 juga menjamin privasi dan keamanan para member. Jadi kamu tidak perlu khawatir data kamu 100% aman di BONUS138 dan tidak akan disalahgunakan oleh pihak yang tidak bertanggung jawab.

Jackpot Slot Maxwin

Kelebihan yang terakhir adalah terdapat jackpot slot maxwin yang diberikan oleh BONUS138 dimana kesempatan ini bisa diberikan oleh semua member. Hal ini tentu merupakan hal yang bagus karena BONUS138 mempunyai berbagai jenis permainan slot dan hampir semuanya permainannya memiliki slot maxwin tinggi.

10 Daftar Situs Slot Online Terpercaya di Indonesia

Kali ini kami akan membahas sesuatu yang diinginkan oleh para member yaitu 10 daftar situs slot online terpercaya yang ada di BONUS138:

Slot Online Pragmatic Play

Pragmatic Play merupakan provider slot online yang sangat populer di kalangan pemain slot Indonesia pada tahun 2022-2023 ini. Dimana terdapat berbagai jackpot sehingga sangat menguntungkan dalam bermain judi slot online. Adapun dengan nilai RTP yang tinggi bisa mencapai 98% tentu menjadi pilihan terbaik untuk bermain slot.

Slot Online RTG Slot

Situs Slot Online RG Slot merupakan yang paling lama berdiri sejak tahun 2000 dan menjadi provider slot tertua yang masih beroperasi hingga saat ini. Seiring dengan perkembangan jaman, RTG berhasil mengembangkan banyak jenis permainan slot, salah satunya yang paling populer adalah Bonanza.

Slot Online Habanero

Provider slot satu ini memiliki pemain yang sangat besar bahkan hingga pada tahun 2023 sudah lebih dari ratusan ribu pemain aktif bermain Habanero slot. Alasan utama kenapa Habanero sangat tenar karena tersedia slot demo yang bisa dimainkan secara gratis untuk mencoba provider ini bisa bermain di BONUS138.

Slot Online Flow Gaming

Flow Gaming merupakan salah satu provider slot terpercaya karena slot online ini mempunyai lisensi resmi dari otoritas judi online dari Malta Gaming. Provider ini sering sekali dibilang menjadi mafia judi slot online di Asia terutama Indonesia, karena tim pengembang dari Flow Gaming merupakan salah satu warga Indonesia yang sudah lama di dunia judi online.

Slot Online PG Soft

PG Soft merupakan kepanjangan dari Pocket Game Soft dimana memiliki ciri khas tampilan yang cukup unik dan berbeda dibandingkan dengan situs slot online yang lain karena pada pocket game ini memberikan tampilan yang menghibur bagi para pemainnya, yaitu tampilan Starlight Princess yang banyak disukai orang.

Slot Online Microgaming

Microgaming merupakan salah satu perusahaan yang paling berkembang dalam permainan judi slot online ini bermarkas di negara Amerika serikat dan berdiri sejak tahun 2007. Microgaming juga merupakan provider slot online terbesar di dunia karena menyediakan jackpot tersebesar di dunia mencapai 900 juta rupiah dengan tema yang cukup unik yaitu Jurassic Park Slot.

Slot Online Spadegaming

Spadegaming merupakan salah satu penyedia situs slot online yang mempunyai pemain dunia yang sangat banyak karena provider ini telah berdiri sejak tahun 2008 dan merupakan salah satu penyedia judi yang menawarkan banyak turnover dan berbagai permainan judi slot online. Untuk turnover yang ditawarkan sendiri sangat banyak bisa mencapai jutaan rupiah setiap harinya. Dengan RTP yang tentunya sangat tinggi menjadikan Spadegaming disukai banyak masyarakat.

Slot Online PlayTech

Playtech bisa dibilang merupakan tempat bermain provider judi online yang baru saja didirikan oleh Playtech dan kini mempunyai member yang cukup besar. Sebelumnya Playtech ini adalah sebuah perusahaan dibagian financial trading atau bisa disebut dengan broker, setelah Playtech melihat potensi yang didapatkan menjadi agen slot online. Maka mereka merubah jalur perusahaan mereka menjadi situs judi slot online yang menjanjikan keamanan dari data dan privasi.

Slot Online Joker123

Joker123 merupakan tempat populer di negara Thailand tapi mereka sudah membesar dan hingga mencapai ke pasar judi Indonesia. Situs slot online Joker123 ini memberikan jackpot slot dengan x500 maxwin ditambah mempunyai nilai RTP yang cukup tinggi dengan tema slot online yang cukup unik yaitu bertema ala buddha cina.

Slot Online Slot88

SLOT88 menjadi provider slot online yang bisa dibilang paling terkenal saat ini karena ini merupakan salah satu provider slot online yang berdiri di Indonesia dan paling banyak dicari oleh masyarakat indonesia dengan bahasa indonesia dan mempunyai playerbase indonesia tertinggi saat ini.

Rekomendasi 5 Permainan Slot Gacor Hari ini

Seperti yang kita ketahui bahwa banyak sekali pemain Indonesia mencari game slot gacor karena hal ini mengacu dengan istilah "slot gacor" sendiri yang berarti permainan slot yang mudah menang atau "gacor" yang mengacu pada bahasa sunda yang artinya tidak akan pernah berhenti menang. Dalam bahasa gaul jakarta gacor sendiri bermakna keren, mantap, atau menang terus. Untuk saat ini kami akan memberikan rekomendasi tentang 5 slot gacor 2023 yang bisa kamu gunakan. Bisa lihat dibawah ini:

Slot Gacor Pragmatic Play

Slot Gacor paling pertama adalah Slot Gacor Pragmatic Play yang memiliki ciri khas permainan judi slot online yang mudah, dengan nilai RTP yang bisa mencapai 97% tentu menjadi nomor 1 diantara slot lainnya.

Slot Gacor Sweet Bonanza

Slot Gacor Sweet Bonanza sudah tidak perlu diragukan lagi, slot ini paling populer dijaman dulu maupun sekarang karena terdapat berbagai jackpot terbesar dalam bermain dimana terdapat RTP dari Sweet Bonanza yang mencapai 96%.

Slot Gacor Starlight Princess

Slot gacor berikutnya adalah Slot Gacor Starlight Princess. Game slot gacor ini menjadi salah satu yang fenomenal di tahun 2022 dimana ketika memenangkan jackpot pada slot gacor starlight princess kamu akan mendapatkan anime princess serta keuntungan yang banyak ditambah dengan bunyi yang cukup unik menjadikan game ini sangat paling gacor.

Slot Gacor Mahjong

Slot Gacor Mahjong ini menjadi yang cukup populer dikalangan para pemain slot gacor karena dalam segi gameplay yang cukup unik bertema mahjong ditambah RTP yang bisa mencapai 96% dengan jackpot pada slot ini juga menjadi salah satu pondasi dalam bermain slot gacor.

Trik Menang Slot Online & Maxwin Slot

Adapun untuk memenangkan judi slot online harus mempunyai banyak cara, akan tetapi disini kami akan mneyediakan beberapa trik terbaik kami dalam bermain judi slot online. bagi kamu yang ingin mengetahui apa saja triknya kamu bisa melihatnya dibawah ini.

Bermain dengan Tenang dan Santai

Trik yang paling bermutu adalah bermain dengan tenang dan santai, karena dalam bermain judi slot online tidak boleh tergesa-gesa, yang terpenting adalah harus menyiapkan waktu bermain dan istirahat terlebih dahulu jika mengalami kekalahan ditambah terdapat peluang untuk menang lebih tinggi jika kamu bermain dengan tenang.

Cek RTP Setiap Hari

Trik kedua adalah cek selalu RTP Slot Online yang bisa memberikan kenyamanan dalam bermain karena RTP atau yang disebut dengan Return To Percentange to Player dimana gunanya RTP ini akan memberikan keuntungan maksimal dalam bermain slot online.

Taruhan Rendah

Trik ketiga atau yang terakhir bisa dibilang menggunakan Taruhan Rendah, dimana taruhan rendah ini bisa digunakan untuk memenangkan permainan slot online dengan mudah, dengan bermain taruhan rendah tentu saja kamu bisa bermain slot sangat banyak dan tentu chance maxwin sangat tinggi.

Bocoran Pola Slot Gacor Hari ini di BONUS138

Sebagai situs judi penyedia slot gacor terbaik saat ini, BONUS138 mempunyai berbagai cara supaya para member bisa menang di provider slot. Sebagai bocoran, sebagian besar kemenangan slot online maxwin di BONUS138 didapat dari Bonanza.

Daftar Permainan Judi Online Terpercaya Selain Slot

Seperti yang sudah kita ketahui banyak masyarakat yang menyukai permainan judi online terutama pada slot. Tapi gak bisa dipungkiri terdapat masyarakat lain yang menyukai permainan selain slot online. Jika kamu termasuk salah satu pengemar judi online lain maka tidak perlu takut karena pada situs judi online BONUS138 menyediakan berbagai permainan judi online yang bisa digunakan, mau tau lebih lanjut bisa lihat dibawah ini:

Judi Online Bola

Taruhan olahraga atau judi bola sangat diminati oleh warga indonesia. Dari segi permainan yang menarik dan tentu saja berhubungan dengan bola tentu menjadi salah satu yang paling diminati dan tentu saja keuntungan yang didapat sangat tinggi seperti slot.

Judi Online Poker

Judi online poker adalah salah satu judi online yang mungkin kamu sudah tau dimana bermain poker online sendiri sudah banyak dimana-mana baik itu menggunakan taruhan asli maupun taruhan palsu. Oleh karena itu bermain di BONUS138 kamu dapat menggunakan uang asli dengan meraup untung yang cukup besar juga dalam bermain.

Judi Togel Online

Salah satu judi yang paling diminati oleh warga indonesia yaitu bermain togel. Biasa judi togel ini sangat diminati oleh warga indonesia terutama kaum chinese dimana dengan taruhan rendah, kamu akan mendapatkan keuntungan yang sangat tinggi dan Bonus138 menyediakan berbagai togel seperti togel Singapore, hongkong, macau dan lain-lainnya.

Judi Online Live Casino

Judi casino mungkin bisa dibilang masih awam buat masyarakat Indonesia, dimana permainan ini merupakan judi online versi barat, walaupun begitu permainan casino ini disediakan oleh situs gacor BONUS138 dan tentunya kami bisa membimbing kamu untuk bermain casino online dengan mudah.

BONUS138 Sebagai Situs Slot Gacor Terbaik 2023

Setelah kita ketahui tentang pembahasan jenis situs slot online terpercaya dan permainan judi online selain slot ada baiknya kamu mengetahui jika mau bermain, kamu harus mendaftarkan diri kamu terlebih dahulu di situs slot terpercaya BONUS138. Seperti yang kita ketahui bahwa BONUS138 merupakan situs terbaik dan terlengkap di Indonesia.

BONUS138 juga memberikan berbagai pelayanan baik itu customer service 24 jam hingga fitur maxwin slot yang bisa didapatkan dengan mudah. Untuk withdraw dan depositnya pun hanya hitungan menit lho.

Karena BONUS138 juga terlah berkerja sama dengan berbagai platform dompet digital seperti OVO, Dana, GoPay, dan banyak platform dompet digital lainnya. Jadi tidak perlu takut untuk bermain di Bonus138 karena segalanya dimudahkan untuk kamu bermain di situs judi slot online terbaik.

FAQ - Tanya Jawab Seputar Permainan Slot Online

Banyak orang yang bertanya bagaimana BONUS138 mendapatkan gelar situs judi slot online terbaik. Baik itu hal yang mendasar maupun pertanyaan mendetail mengenai teknis permainan slot.

Berikut ini kami rangkum pertanyaan umum yang biasa ditanyakan dalam bermain slot online, agar member baru bisa memahami dengan jelas mengenai permainan slot online.

Slot Online apa saja yang gacor?

Semua permainan diatur dalam ketentuan dimana bandar atau disebut dengan tuan rumah tidak bisa mengambil keuntungan sebanyak mungkin. Hal ini bisa dilihat dari nilai RTP atau rata-rata uang yang kembali ke pemain. Jadi perhatikan slot yang memiliki RTP tinggi.

Kapan waktu terbaik bermain judi slot online?

Bermain slot online lebih baik dilakukan pada saat malam hari, dikarenakan pada malam hari slot sedang ramai dan hal ini bisa memicu RTP lebih tinggi dan sangat akurat. Menurut kami sendiri jam terbang paling baik adalah pada saat jam 11 malam hingga 3 subuh.

Apa itu Permainan Slot Online?

Bermain slot online lebih baik dilakukan pada saat malam hari, dikarenakan pada malam hari slot sedang ramai dan hal ini bisa memicu RTP lebih tinggi dan sangat akurat. Menurut kami sendiri jam terbang paling baik adalah pada saat jam 11 malam hingga 3 subuh.

Apa itu RTP?

RTP atau disebut dengan Return of percentange player adalah angka rasio perhitungan jumlah taruhan uang yang kembali kepemain. RTP dihitung dari rentang waktu tertentu seperti 24 jam dengan perhitungan jumlah taruhan yang dibagi buat pemain slot juga.

Copyright © 2023 Slot Online Bonus138

# data[].attributes.attachment_link.title
BONUS138: Agen Judi Slot Online Terpercaya, Slot88 & Slot Gacor 2023

# data[].attributes.attachment_link.description
BONUS138 adalah situs judi online terpercaya 2023 yang mempunyai game slot gacor hari ini dengan slot88 sebagai penyelenggara agen judi online. Di situs judi slot online terbaik BONUS138 terdapat berbagai slot dan tips menang maxwin slot dimana terdapat keuntungan semua member bisa bermain permainan judi slot online dengan mudah melalui komputer atau smartphone.

# data[].attributes.attachment_link.readable_text
Environments and life experiences help shape our brains, which are changing and growing throughout our lives. A growing body of science supports the implications for education-that if we are able to create the right conditions for learning, we can help every student learn and thrive. Researchers can use this emerging knowledge to redesign a system in which all students have high-quality learning opportunities that ignite their curiosity and nurture their development.

This playbook points to principles to nurture innovations and effective school models that advance this change. It provides a framework-shown to the right-to guide the transformation of k-12 settings, illustrating how practitioners can implement structures and practices that support learning and development through its five components. These design principles do not suggest a single design or model for change, but rather illuminate the multiple ways that schools can be redesigned to support all learners.

Environments and life experiences help shape our brains, which are changing and growing throughout our lives. A growing body of science supports the implications for education-that if we are able to create the right conditions for learning, we can help every student learn and thrive. Researchers can use this emerging knowledge to redesign a system in which all students have high-quality learning opportunities that ignite their curiosity and nurture their development.

This playbook points to principles to nurture innovations and effective school models that advance this change. It provides a framework-shown to the right-to guide the transformation of k-12 settings, illustrating how practitioners can implement structures and practices that support learning and development through its five components. These design principles do not suggest a single design or model for change, but rather illuminate the multiple ways that schools can be redesigned to support all learners.

Explore the playbook, or hover over a principle in the pictured framework.

Structures and practices that promote healthy attachments, long-term relationships, and trust among children, educators, school staff, and families.

Structures and practices that cultivate supportive school environments to promote a sense of belonging and physical, emotional, and identity safety.

Structures and practices that promote personalized, authentic, and culturally responsive learning and provide opportunities for conceptual understanding and meaning.

Structures and practices that develop cognitive, emotional, and social skills-such as growth mindsets, social awareness, and self-direction-that enable learning and development.

Structures and practices that support student well-being with ready access to targeted supports to address learning barriers or personal obstacles, building resilience and agency.

When these principles are integrated in transformative, personalized, empowering, and culturally affirming ways, they enhance development and learning.

Guiding Principles for
Equitable Whole Child Design

# data[].attributes.attachment_link.description
Environments and life experiences help shape our brains, which are changing and growing throughout our lives. A growing body of science supports the implications for education-that if we are able to create the right conditions for learning, we can help every student learn and thrive.

# data[].attributes.attachment_link.readable_text
The Learning Policy Institute has assembled a team of researchers and policy experts to study community schools as a strategy to advance equity and support student and family well-being and academic success. A cornerstone of our work to date is an in-depth analysis of 143 research studies on the impact of community schools on student and school outcomes, produced in partnership with the National Education Policy Center. We also publish or produce with partners a variety of tools and resources to advance understanding and the effective implementation of this evidenced-based strategy, including infographics, blog posts, briefs, and a policy playbook.

# data[].attributes.attachment_link.description
The Learning Policy Institute has assembled a team of researchers and policy experts to study community schools as a strategy to advance equity and support student and family well-being and academic success. A cornerstone of our work to date is an in-depth analysis of 143 research studies on the impact of community schools on student and school outcomes, produced in partnership with the National Education Policy Center.

# data[].attributes.attachment_link.title
Dr.  Andreas Schleicher - Equity and excellence in learning in schools: Lessons from the OECD

# data[].attributes.headline
The concept of "after-school" has evolved and today is seen as a time when students either can get extra help or participate in a variety of enrichment activities.

# data[].attributes.subject
The concept of "after-school" has evolved and today is seen as a time when students either can get extra help or participate in a variety of enrichment activities.

# data[].attributes.attachment_link.description
Professional Development for After-School Staff

# data[].attributes.body
from Mark Cabaj - Tamarack

# data[].attributes.attachment_link.readable_text
Contribution analysis is particularly useful in situations where the programme is not experimental, i.e. not in trial projects but in situations where the programme has been funded on the basis of a relatively clearly articulated theory of change and where there is little or no scope for varying how the program is implemented. Contribution analysis helps to confirm or revise a theory of change; it is not intended to be used to surface or uncover and display a hitherto implicit or inexplicit theory of change. The report from a contribution analysis is not definitive proof, but rather provides evidence and a line of reasoning from which we can draw a plausible conclusion that, within some level of confidence, the program has made an important contribution to the documented results.

Steps​

Six steps are taken to produce a credible contribution story:

1: Set out the attribution problem to be addressed

Determine the specific questions being addressed. Not all cause-effect questions are useful to pursue. Contribution analysis is less suitable for traditional causality questions such as: Has the program caused the outcome? To what extent, quantitatively, has the program caused the outcome? These often are not that useful because they treat the program as a black box and don't get at the fact that there are usually many causes involved. Contribution analysis is more appropriate for contribution questions: Has the Program influenced the observed result? Has the program made an important contribution to the observed result? Why has the result occurred? What role did the intervention play? and for management questions: Is it reasonable to conclude that the program has made a difference? What does the preponderance of evidence say about how well the program is making a difference? What conditions are needed to make this type of program succeed?

2: Develop a theory of change and risks to it

Develop the program logic/results chain describing how the program is supposed to work. Identify as well the main external factors at play that might account for the outcomes observed. Based on the results chain, develop the theory of change upon which the program is based. This theory of change should lead to a plausible association between the activities of the program and the outcomes sought. The theory of change must include the assumptions made in the results chain and the inherent risks as well as external influences such as donor pressure, influences of peers and resourcing levels. Some links in the theory of change will be fairly well understood or accepted. Others will be less well understood, contested or subject to significant influence other than from the program. In this way you acknowledge that attribution is indeed a problem

3: Gather the existing evidence on the theory of change

It is useful to first use existing evidence such as from past related evaluations or research, and from prior monitoring, to test the theory of change. It sets out the intended results (outputs, intermediate and end outcomes). What evidence (information from performance measures and evaluations) is currently available about the occurrence of these various results? The links in the theory of change also need to be assessed. What evidence currently exists on the assumptions and risks behind these links? Which are strong (good evidence available, strong logic, or wide acceptance) and which are weak (little evidence available, weak logic, or little agreement among stakeholders)? What evidence exists about the identified other influencing factors and the contribution they may be making?

4: Assemble and assess the contribution story, or performance story, and challenges to it

With this information, you will be able to assemble your contribution story that expresses why it is reasonable to assume that the actions of the program have contributed (in some fashion, which you may want to try and characterize) to the observed outcomes. Now you have to assess it. How credible is the story? Do reasonable people agree with the story? Does the pattern of results observed validate the results chain? Where are the main weaknesses in the story? There always will be weaknesses. Weaknesses in the story point to where additional data or information is needed.

5: Seek out additional evidence

Having identified where the contribution story is less credible, additional evidence is now gathered to augment the evidence in terms of what results have occurred, how reasonable the key assumptions are, and what has been the role of external influences and other contributing factors. Augmenting evidence can include the collection of additional, new data such as from surveys, field visits, administrative data, focus groups, national statistical data, etc. as well as the synthesis of evidence from other research and evaluations.

6: Revise and, where the additional evidence permits, strengthen the contribution story

With the new evidence, you should be able to build a more substantive and so more credible story, one that a reasonable person will be more likely to agree with. It will probably not be foolproof, but the additional evidence will have made it stronger and more plausible.

Using a generative perspective on causality to infer that a program made an important contribution to an expected result that has been observed, contribution analysis argues that a reasonable contribution causal claim can be made if:

There is a reasoned theory of change for the intervention: the key assumptions behind why the intervention is expected to work make sense, are plausible, may be supported by evidence and/or existing research, and are agreed upon by at least some of the key players.

The activities of the intervention were implemented as set out in the theory of change.

The theory of change-or key elements thereof- is supported by and confirmed by evidence on observed results and underlying assumptions-the chain of expected results occurred. The theory of change has not been disproved.

Other influencing factors have been assessed and either shown not to have made a significant contribution or their relative role in contributing to the desired result has been recognized.

Issues

Some issues might arise when taking this approach with regards to:

Reducing uncertainty about the contribution the intervention is making to the observed results.

Inferring causality in real-life program evaluations.

Confirming or revising a programme's theory of change - including its logic model.

Risks and assumptions are labelled as [O] over which the intervention has no or very little influence, or [I], where the intervention can (should) have an influence, direct or indirect, or [C] where the intervention should be able to directly control.

Resources

Guides

Addressing attribution through contribution analysis: using performance measures sensibly. This article from the Canadian Journal of Program Evaluation outlines the key steps involved with using contribution analysis as a tool to discover the contribution a program has made to particular outcomes.

Contribution Analysis: An approach to exploring cause and effect: This brief from the Institutional Learning and Change Initiative(ILAC) explores contribution analysis and how it can be used to provide credible assessments of cause and effect.

Mayne, J. (2011). Addressing Cause and Effect in Simple and Complex Settings through Contribution Analysis. In Evaluating the Complex, R. Schwartz, K. Forss, and M. Marra (Eds.), Transaction Publishers.

Mayne, J. (2011) Contribution Analysis: Addressing Cause and Effect in Evaluating the Complex, K. Forss, M. Marra and R. Schwartz (Eds.), Transaction Publishers; Piscataway, New Jersey.

Source

# data[].attributes.attachment_link.description
Contribution analysis is particularly useful in situations where the programme is not experimental, i.e. not in trial projects but in situations where the programme has been funded on the basis of a relatively clearly articulated theory of change and where there is little or no scope for varying how the program is implemented.

# data[].attributes.attachment_link.title
Participatory | Participatory Action Research | Canada

# data[].attributes.attachment_link.description
Participatory methods and action research concepts for research, planning, monitoring and evaluation. SAS2 Dialogue, based in Ottawa, Canada

# data[].attributes.attachment_link.readable_text
In considering the key takeaways from the research on this program that other mentoring programs can apply to their work, it's useful to reflect on the features and practices that might have influenced its rating, as well as some of the more interesting aspects of the program's design.

Great target age, but also a missed opportunity?

Peer Group Connection (PGC) is a wonderfully targeted program, in spite of the fact that it is designed to be used with an entire class (or at least a very wide swath of students). Many school-based mentoring programs target students at key transition pointsfor example, the previously reviewed School-Based Mentoring Program for At-Risk Middle School Youth works specifically with youth right after the transition to middle school, once issues around attendance or grades appear. Others, like PGC seek to head off troubles before they even begin, working with an entire incoming class of ninth graders to orient them to the high school and teach them skills that will help them thrive in the new environment. In fact, the "wide swath" approach is part of the design, with the near peer and group meeting design intended to facilitate connections both among diverse classmates and with upperclassmen. The varying levels of risk of dropping out among the participants very possibly serve as a strength, allowing riskier students to connect with other students who can be supportive and lift them up.

According to the designers of the program note, the path to dropping out begins for most students in 9th grade, as the transition to the new environment and the challenges of the curriculum lead to often widespread disengagement from the school and peers. PGC should be commended for trying to influence this particularly critical period of time.

But one also wonders if there is a bit of a missed opportunity, at least in how results were reported in the evaluations that served as the basis for the review of the program's evidence base for CrimeSolutions.gov. In particular, the impact of program participation on the juniors and seniors who served as peer mentors/leaders was not reported much, if at all, in thse studies. These older students participated in a year-long leadership course, designed to teach them how to implement the program and what to teach the incoming freshmen. They also got a nice education in some academic and interpersonal skills, got to work closely with faculty advisors, and earned course credit. It would be nice to see more consideration of the impact of this experience on the peer leaders. One can imagine that this experience developed their interpersonal skills, built their confidence and competence in a number of areas, and prepared them for both postsecondary education and career experiences. This "dual benefit" of peer mentoring for older and younger students is one of the major draws of peer-led group mentoring like this. It is surprising that the impact on older students was not woven into the research design more. But given the quality of the PGC implementation, it seems likely that those older youth also benefitted from the program.

A nice example of a multi-layered program design

One of the aspects of PGC that becomes immediately clear is the careful thought that went into the program design. Not only is the program well-targeted, but it draws upon several areas of relevant research with a structure that allowed for strong implementation. The program builds on research that supports attention to Social Emotional Learning in school-based programs and provides a layered structure in doing so that allows for support at all levels. The peer leaders work as pairs with groups of up to 12 freshmendoing this as a pair means that they have some support from a peer in delivering the curriculum and leading the group sessions. The pair also has support from above in the form of a faculty advisor, each of whom goes through an 11-day training regimen to fill that role! They work closely with the peer leaders to plan and implement the program. This level of support for peer mentors nicely addresses one of the major concerns about peer mentoring: that the quality of the mentoring can suffer at peer mentors struggle to problem-solve, lead conversations, and deliver curriculum because they are, well, students themselves. These three levels of participants were then further supported by a stakeholder group of principals, other school administrators, and additional faculty, likely facilitating a deeper level of whole-school buy-in and access to resources and materials than is typical of that enjoyed by many mentoring programs operated within schools. Providing support and buy-in at every level seems likely to help explain why the program had strong implementation and advisors and peer leaders were both rated effective by the researchers (87% and 89% respectfully).

Activities are what turn theory into results

As noted above, the program was designed a series of meetings between freshmen and peer leaders that brought Social Emotional Learning (SEL) to life. The program developed a very nice curriculum and set of activities that addressed the academic, social, and emotional aspects of the student experience. A Family Night held mid-year also brought other caring adults and siblings into the program experience. But it was the activities that emphasized critical thinking, goal setting, decision-making, time management, teamwork, and other "soft skills" that really brought the SEL framework to life. The program should be commended for so intentionally selecting and delivering these activities, as the skills targeted have well-established connections to a broad array of positive school and life outcomes.

Outside help... helps.

Of course, in the case of PGC, it helps to have a team of talented researchers helping with program design, implementation, and even things like support of stakeholders and faculty advisors. And while most programs won't have this kind of access to researcher or subject matter expertise, there are opportunities to get help of this nature, especially with initial program design. Programs may want to consider tapping into the technical assistance offered by the NMRC or other entities. Local colleges and universities can also be good places to look for talented help in designing services or shoring up program weaknesses. No program should go it alone when it comes to making or improving its theory of change and accompanying services.

While PGC had an overall rating of "no effects" in the Crime Solutions framework, it's important to note that the program did have a statistically significant impact on the dropout rates of male students (a traditionally riskier group for dropping out). This may be an example of a whole population design being needed to reach a specific sub group of participants. It remains unclear if the program would have had this impact on male students if participants had only been male students.

We look forward to more research about this well-designed program and other peer mentoring efforts should keep an eye on this approach and what we can learn from the PGC program.
*Note: The National Mentoring Resource Center makes these "Insights for Mentoring Practitioners" available for each program or practice reviewed by the National Mentoring Resource Center Research Board. Their purpose is to give mentoring professionals additional information and understanding that can help them apply reviews to their own programs. You can read the full review on the Crime Solutions website. For more information on research-informed program practices and tools for implementation, be sure to consult the Elements of Effective Practice for Mentoring™ and the "Resources for Mentoring Programs" section of the National Mentoring Resource Center site.

# data[].attributes.attachment_link.title
Peer Group Connection - National Mentoring Resource Center

# data[].attributes.attachment_link.description
In considering the key takeaways from the research on this program that other mentoring programs can apply to their work, it’s useful to reflect on the features and practices that might have influenced its rating, as well as some of the more interesting aspects of the program’s design. Great target…

# data[].attributes.attachment_link.readable_text
Announcing our new YAB members!

We are so excited to share with you our newest Youth Advisory Board members! Follow us on Instagram @LetStudendsLead to learn more about each young leader impacting our work and the world.

Welcoming Angela Powell DeGraff

CSS is excited to share more about our new Interim CEO and the experience she brings to our organization, including over 30 years of supporting our work.

#LetStudentsLead

's Lessons in Adolescence podcast features a conversation with CSS Youth Consultants, Kiana Dixon and Janikaa Jackson, sharing their wisdom and experience building space for youth agency.

CSS helps schools become places where students want to be.

We help leverage the resources in schools to create more caring and vibrant learning communities.

# data[].attributes.attachment_link.description
We are so excited to share with you our newest Youth Advisory Board members! Follow us on Instagram @LetStudendsLead to learn more about each young leader impacting our work and the world. CSS is excited to share more about our new Interim CEO and the experience she brings to our organization, including over 30 years of supporting our work.

# data[].attributes.headline
Production and Use of Evidence/Data - Genvieve Lagare

# data[].attributes.subject
Production and Use of Evidence/Data - Genvieve Lagare

# data[].attributes.attachment_link.readable_text
Why Build a Scaled-Up System of Community Schools?

Community schools, with their emphasis on intentional partnerships, represent the most effective approach to addressing both the in-school and non-school factors that influence student achievement. The impact of poverty, changing demographics, chronic absence, health and many other factors simply cannot be ignored if students are to succeed. Of course our schools need qualified teachers and strong principals: and like all public institutions, schools must be accountable for performance. But communities must share responsibility for helping young people succeed.

Community schools are the vehicle for making this happen. They come in all shapes and sizes, yet each one is designed as a center of community-a place and a set of partnerships connecting school, family, and community. Community schools have a vision of students succeeding in school, graduating, and becoming productive parents, workers, and citizens. As they work toward this vision, community schools align school and community resources so that the essential conditions for learning are in place.

The Challenge: Growing Systems of Community Schools

There are several thousand community schools across the country and many others offer some variant of a community school strategy to better meet child and family needs-and they are making a difference in a wide range of indicators that lead to school success, including academics, attendance, and family participation. In order to wrap their arms around all their children, however, schools and communities need to expand, deepen, and sustain a scaled-up system of Community Schools.

Scaling up community schools is no pipe dream. A growing number of local initiatives are meeting the challenge and the readiness of schools and community partners to build a sustainable and coordinated system of community schools has never been greater. Scaling Up School and Community Partnerships: The Community Schools Strategy draws on the experience of these local school and community leaders. It lays out a vision-driven approach to building a scaled-up system focused on results.

What Does it Take to Build A Scaled-Up System of Community Schools?

A scaled-up system of community schools refers to a vertical network of schools from pre-kindergarten
through grade 12 in a single attendance area with all schools linked horizontally across one or more
school districts.Scaling Up School and Community Partnerships: The Community Schools Strategy
builds on both practice and research to describe the what, why, and how of system-wide expansion of
community schools. The Guide introduces a 6-stage spiraling process for moving toward a scaled-up system. Rather than providing a rigid formula or lockstep set of requirements, the Scale-Up Spiral helps school systems and communities at many different starting points begin building collaborative leadership and functional capacity-while staying focused on results. Each stage includes a set of milestones and end-state benchmarks.

To learn more and how to get support, send an email with the subject line "Scaling Up" to ccs@iel.org.

# data[].attributes.attachment_link.title
Scaling Up School and Community Partnerships: The Community Schools Strategy

# data[].attributes.attachment_link.description
Community schools, with their emphasis on intentional partnerships, represent the most effective approach to addressing both the in-school and non-school factors that influence student achievement. The impact of poverty, changing demographics, chronic absence, health and many other factors simply cannot be ignored if students are to succeed.

# data[].attributes.attachment_link.title
TOC Developing pathway outcomes activity

# data[].attributes.attachment_link.readable_text
A theory of change is a tool that shows a charity's path from needs to activities to outcomes to impact. It describes the change you want to make and the steps involved in making that change happen. Theories of change also depict the assumptions that lie behind your reasoning, and where possible, these assumptions are backed up by evidence.

A good theory of change can reveal:

whether you are doing the right activities to meet your goals;

whether there are things you do that do not help you achieve your goals;

which activities and outcomes you can achieve alone and which you cannot achieve alone; and

how to measure your impact.

A theory of change forces you to take a clear, simple view, crystallising your work into as few steps as possible to capture the key aspects of what you do.

This short paper introduces theory of change, explains the origins of the technique, and discusses how it can be used by charities to improve their work.

We show how a theory of change can be useful in three important ways. It is an excellent basis for a strategic plan because it works methodically from the need you are trying to address to the change you want to achieve. It provides a theoretical framework for measuring a charity's impact. And it can also be used to think more broadly about how different organisations within a sector are working together, and how they could achieve greater impact through collaboration.

We have since produced practical guidance to help charities think through the process of developing a theory of change.

# data[].attributes.attachment_link.title
Theory of change: The beginning of making a difference - NPC

# data[].attributes.attachment_link.description
A decade ago, the term 'theory of change' meant little to the UK charity sector. But today, more and more charities are using theories of change, and more and more funders are asking to see them. So what is a theory of change, and why is it so valuable?

# data[].attributes.headline
Practical information for facilitators - Actknowledge

# data[].attributes.subject
Practical information for facilitators - Actknowledge

# data[].attributes.body
Actknowledge - CLC training original TOC training

# data[].attributes.attachment_link.readable_text
Would you rather help one child a little bit today, or wait a few years and help five children even more? Every dollar spent on current programs is a dollar used to help today's children in need-a worthy cause. Yet every dollar spent on research today, in theory, is a dollar invested in helping tomorrow's children even more. Admittedly, this trade-off is complex, imprecise, and uncertain. But the promise of research that can help us do more good per dollar spent is enticing.

Yet here's one cautionary claim we can make for certain: Every dollar spent on poorly conceived research that does not help tomorrow's children is a dollar wasted.

Good impact evaluations-those that answer policy-relevant questions with rigor-have improved development knowledge, policy, and practice. For example, the NGO Living Goods conducted a rigorous evaluation to measure the impact of its community health model based on door-to-door sales and promotions. The evidence of impact was strong: Their model generated a 27-percent reduction in child mortality. This evidence subsequently persuaded policy makers, replication partners, and major funders to support the rapid expansion of Living Goods' reach to five million people. Meanwhile, rigorous evidence continues to further validate the model and help to make it work even better.

Of course, not all rigorous research offers such quick and rosy results. Consider the many studies required to discover a successful drug and the lengthy process of seeking regulatory approval and adoption by the healthcare system. The same holds true for fighting poverty: Innovations for Poverty Action (IPA), a research and policy nonprofit that promotes impact evaluations for finding solutions to global poverty, has conducted more than 650 randomized controlled trials (RCTs) since its inception in 2002. These studies have sometimes provided evidence about how best to use scarce resources (e.g., give away bed nets for free to fight malaria), as well as how to avoid wasting them (e.g., don't expand traditional microcredit). But the vast majority of studies did not paint a clear picture that led to immediate policy changes. Developing an evidence base is more like building a mosaic: Each individual piece does not make the picture, but bit by bit a picture becomes clearer and clearer.

How do these investments in evidence pay off? IPA estimated the benefits of its research by looking at its return on investment-the ratio of the benefit from the scale-up of the demonstrated large-scale successes divided by the total costs since IPA's founding. The ratio was 74x-a huge result. But this is far from a precise measure of impact, since IPA cannot establish what would have happened had IPA never existed. (Yes, IPA recognizes the irony of advocating for RCTs while being unable to subject its own operations to that standard. Yet IPA's approach is intellectually consistent: Many questions and circumstances do not call for RCTs.)

Even so, a simple thought exercise helps to demonstrate the potential payoff. IPA never works alone-all evaluations and policy engagements are conducted in partnership with academics and implementing organizations, and increasingly with governments. Moving from an idea to the research phase to policy takes multiple steps and actors, often over many years. But even if IPA deserves only 10 percent of the credit for the policy changes behind the benefits calculated above, the ratio of benefits to costs is still 7.4x. That is a solid return on investment.

Despite the demonstrated value of high-quality impact evaluations, a great deal of money and time has been wasted on poorly designed, poorly implemented, and poorly conceived impact evaluations. Perhaps some studies had too small of a sample or paid insufficient attention to establishing causality and quality data, and hence any results should be ignored; others perhaps failed to engage stakeholders appropriately, and as a consequence useful results were never put to use.

The push for more and more impact measurement can not only lead to poor studies and wasted money, but also distract and take resources from collecting data that can actually help improve the performance of an effort. To address these difficulties, we wrote a book, The Goldilocks Challenge, to help guide organizations in designing "right-fit" evidence strategies. The struggle to find the right fit in evidence resembles the predicament that Goldilocks faces in the classic children's fable. Goldilocks, lost in the forest, finds an empty house with a large number of options: chairs, bowls of porridge, and beds of all sizes. She tries each but finds that most do not suit her: The porridge is too hot or too cold, the bed too hard or too soft-she struggles to find options that are "just right." Like Goldilocks, the social sector has to navigate many choices and challenges to build monitoring and evaluation systems that fit their needs. Some will push for more and more data; others will not push for enough.

To create a right-fit evidence system, we need to consider not only when to measure impact, but when not to measure impact. Given all the benefits of impact measurement, it may seem irresponsible not to try to measure it. But there are situations in which an insistent focus on measuring impact can be counterproductive to collecting other important data.

Misplaced Priorities

How have we reached this point? If impact evaluation is so important, why are we advocating for limiting its use? The rapidly decreasing costs of data collection and analysis have certainly helped to heighten the appeal of impact measurement. Thirty years ago, frugal budgets restricted long-distance calls. Now free videoconferencing can connect people from multiple countries all at once. Previously, organizations might have argued that collecting data is too time-consuming and expensive. Today, the cost of collecting, storing, and analyzing data is much cheaper. We can process millions of data points and spit out analyses to field operators in mere minutes. And the pace of change remains rapid: Satellite imagery and a multitude of GPS monitoring devices, for example, are rapidly influencing the way programs are run and the richness of the questions that evaluators and researchers can ask. Naturally, quicker and cheaper data also makes organizations and stakeholders more willing to demand it.

At the same time, there have been more calls for accountability in the public and social sectors based on this ability to more easily measure results. Major donor organizations from the Bill & Melinda Gates Foundation to the UK's Department for International Development (DFID) are requiring evidence of impact. Social impact bonds and pay-for-success programs seek to fund effective initiatives by tying financing to proven results. And proponents of effective altruism seek to persuade philanthropists to give only to programs with strong evidence of effectiveness.

The trend toward impact measurement is mostly positive, but the push to demonstrate impact has also wasted resources, compromised monitoring efforts in favor of impact evaluation, and contributed to a rise in poor and even misleading methods of demonstrating impact. For instance, many organizations collect more data than they actually have the resources to analyze, resulting in wasted time and effort that could have been spent more productively elsewhere. Other organizations collect the wrong data, tracking changes in outcomes over time but not in a way that allows them to know whether the organization caused the changes or they just happened to occur alongside the program.

Bad impact evaluations can also provide misleading or just plain wrong results, leading to poor future decisions. Effective programs may be overlooked and ineffective programs wrongly funded. In addition to such social costs, poor impact evaluations have important opportunity costs as well. Resources spent on a bad impact evaluation could have been devoted instead to implementation or to needed subsidies or programs.

Much of such waste in pursuit of impact comes from the overuse of the word impact. Impact is more than a buzzword. Impact implies causality; it tells us how a program or organization has changed the world around it. Implicitly this means that one must estimate what would have occurred in the absence of the program-what evaluators call "the counterfactual." The term sounds technocratic, but it matters a great deal in assessing how best to spend limited resources to help individuals and communities.

When feasible, the most straightforward way to create a counterfactual is through a randomized controlled trial (RCT) in which participation in a program, or in some aspect of a program, is decided partly through random allocation. Without a counterfactual, we do not know whether the program caused a change to happen or whether some outside factor-such as weather, economic growth, or other government policy-triggered the change. We can't know whether those who participated in a program changed their lives because of the program or because of other factors. A rigorous counterfactual can change conventional but misplaced beliefs: For example, recent counterfactual-based impact evaluations of microcredit programs found much lower impact on household income than was previously claimed by microcredit advocates.

Good monitoring data are often collateral damage in the pursuit of measuring impact. Information on what the staff is doing, take-up and usage of program services, and what constituents think of operations can help create a better program and stronger organization. These data often get lost or overshadowed in the pursuit of impact evaluations. This is partly understandable: impact is the ultimate goal, and sloppy thinking often conflates management data with impact data. (Take-up of a product like microcredit, for example, is an important piece of management data but is not a measure of impact; statements such as "50,000 clients served" do not measure impact.)

The challenge for organizations is to build and use data collection strategies and systems that accurately report impact when possible, demonstrate accountability, and provide decision makers with timely and actionable operational data. The challenge for funders and other nonprofit stakeholders is to ask organizations to be accountable for developing these right-fit evidence systems and to demand impact evaluation only when the time is right.

In what follows, we offer 10 reasons for not measuring impact. We then provide a framework for right-fit monitoring and evaluation systems that help organizations stay consistently and appropriately attuned to the data needed for accountability, learning, and improvement.

The 10 Reasons

The 10 reasons not to measure impact fall into four categories: Not the Right Tool, Not Now, Not Feasible, and Not Worth It. For each reason, we also offer alternatives that fans of impact evaluation can adopt instead.

1. Not the Right Tool: Excellent question, wrong approach. Here are some excellent questions you may ask in evaluating a program: What is the story behind a successful or unsuccessful program recipient? Can we deliver the same services for less by improving our operating model? Are we targeting the people we said we would target? Are our constituents satisfied with the service we provide? Is there significant demand for the service we provide? Is the demand sustained-do people come back for more? Is the problem we are solving the most pressing in our context?

We could go on. These are the questions that key stakeholders often want answered. Some of these questions can be answered with data. Others are tougher to tackle. But-and this is the crucial point-their answers are not measures of impact.

Alternative: To answer these questions, data collection and analysis need to focus more precisely on the question being asked. Understanding constituent satisfaction requires feedback data. Improving the cost-effectiveness of program delivery requires detailed data on costs by site, as well as by product or service. All of this is important program monitoring data to collect, but none of it requires an impact evaluation.

2. Not Now: The program design is not ready. Thinking through the theory of change is the first step to planning out a monitoring or evaluation strategy. A theory of change articulates what goes into a program, what gets done, and how the world is expected to change as a result. Without it, staff may hold conflicting or muddled ideas about how or why a program works, which can result in large variations in implementation.

Articulating a clear theory of change is not merely an academic exercise for retreats and donors. A theory of change guides right-fit data collection by making clear what data to track to make sure an organization is doing what it says it does, to provide feedback and engagement data to guide program learning and improvement (neither of which requires a counterfactual), and to provide guidance for key outcomes to track in an impact assessment (which does require a counterfactual to be meaningful).

An untested theory of change likely contains mistaken assumptions. For example, hypothesized connections ("theory") between program elements may not hold. Assumptions may also be wrong empirically: Program outcomes may depend on everyone finishing the training part of the program. Do they? Good management data could help demonstrate this. Similarly, programs may assume that demand exists for their services (e.g., microcredit), but a good needs assessment might show that reasonable credit alternatives exist.

Large impact evaluations undertaken before key assumptions in the theory of change undergo examination are likely to be misguided and ultimately lead to conflict over interpretation. If the program is found not to work, implementers are likely to reject the results, arguing that the program evaluation doesn't reflect current implementation.

Alternative: Validating the initial steps in the theory of change is a critical step before moving on to measuring impact. Consider a program to deliver child development, health, and nutrition information to expectant mothers in order to improve prenatal care and early childhood outcomes. Starting an impact evaluation before knowing if expectant mothers will actually attend the training and adopt the practices makes little sense. First establish that there is a basic take-up of the program and that some immediate behaviors are being adopted. Before starting an impact evaluation of a program providing savings accounts, determine whether people will actually open a savings account when offered, and that they subsequently put money into the account. If not, the savings account design should be reconsidered.

If the theory of change has not been fully developed, then the obvious step is to develop the theory for the program, following the implementation step by step, examining the assumptions being made, and gathering data to test them. Then gather monitoring data on implementation and uptake before proceeding to an impact evaluation. Is the program reaching the people it targets? Are those individuals using the product or service? For how long and how intensively do they use the product or service? Based on this information, how can the program be improved?

When the program is still being adapted and implementation kinks worked out, it is probably too early to evaluate the program's impact. This is a tricky situation. We could craft some general principles for determining when a program is "ready" for evaluation, such as "Basic levels of demand are observed for the program," or "Constituents provide positive feedback." The challenge is then applying these principles to specific situations. Here reasonable people will no doubt disagree, and these principles cannot clearly resolve what to do for any given situation. The most sensible solution is to wait and let the program work out the implementation kinks. If women are not coming to the training or teachers are not following a new curriculum, wait, watch, try new tactics or incentives; and in the meantime, collect good monitoring data that informs progress.

3. Not Now: The program implementation is not ready. Even if a program's theory has been fully defined and basic assumptions tested, implementation may falter. An evaluation that finds no impact for a project with weak implementation is hard to interpret. Is the finding the result of poor implementation, the wrong partner, or outside circumstances (e.g., civil unrest or other disturbances)? Either way, when implementation is weak, impact evaluation is a bad choice.

To return to our previous example, a prenatal training program may have determined that mothers demand these services and will show up and complete the training in an "ideal" setting where the program was tested. But what if during program rollout the trainings are not implemented as planned? What if not all mothers complete the training? Basic implementation information is needed before moving to impact evaluation, so that stakeholders are satisfied that the program as designed is (roughly) the same as the program that is implemented. Otherwise, evaluation resources are wasted.

Alternative: Collect good monitoring data and use it to strengthen implementation. Evaluators can either work with program leadership to improve implementation or decide that a certain organization is not a good fit for an impact evaluation.

But what if the real world takes over and politics (or funding) mean you must evaluate now or never? If the program is still not ready, consider again carefully whether impact evaluation is the right step. Will the evaluation help answer theory-based questions under real-world implementation conditions? Will an evaluation now make an innovative or controversial program more likely to be accepted by constituents? Are the technical issues discussed below addressed, and can you construct a reliable comparison group? If you answer no to any of these questions, impact evaluation isn't the right step. But if you answer yes to all, an evaluation of a program that isn't quite ready can still inform important and timely policy-relevant decisions, especially if the evaluators work closely with the policy makers throughout the evaluation process.

4. Not Now: It is too late. The desire for impact measurement often comes after a program has already expanded and has no plans for further expansion. In these cases, it may be too late. Once a program has begun implementation, it is too late randomly to assign individuals or households or communities to treatment and control. Creating a non-randomized comparison group may be viable but is often hard to do and quite expensive. And the true comparability of this group may still be in question, thus rendering the evaluation less convincing.

Alternative: Plan for future expansions. Will the program be scaled up elsewhere? If so, read on to understand whether measuring impact is feasible. If the program has changed significantly as a result of organizational learning and improvement, timing may be perfect to then assess impact.

5. Not Feasible: Resources are too limited. Resource limitations can doom the potential for impact evaluation in two ways: The program scale may be too small, or resources may be too scarce to engage in high-quality measurement.

If a program is small, there simply will not be enough data to detect impact unless the impact is massive. Without sounding too sour, few initiatives have truly massive impact. And an impact evaluation with an ambiguous conclusion is worse than doing nothing at all. A lot of money is spent to learn absolutely nothing-money that could have been spent to help more people.

Similarly, if there is not enough money to do a good evaluation, consider not doing it at all. You may be forced to have too small a sample, cut too many corners on what you are measuring, or risk poor implementation of evaluation protocols.

Alternative: If your scale is limited, do not try to force an answer to the impact question. Consider other options. First, perhaps much is already known about the question at hand. What do other evaluations say about it? How applicable is the context under which those studies were done, and how similar is the intervention? Study the literature to see if there is anything that suggests your approach might be effective. If no other evaluations provide helpful insights, track implementation, get regular feedback, and collect other management data that you can use instead.

If money is limited, consider what is driving the cost of your evaluation. Data (especially household surveys) are a key cost driver for an evaluation. The randomization part of a randomized trial is virtually costless. Can you answer key impact questions with cheaper data, perhaps with administrative data? For example, if testing the impact of a savings program, no doubt many will want to know the impact on health and education spending, agricultural and enterprise investment, consumption of temptation goods, and so forth. But in many cases, just seeing increased savings in regulated financial institutions indicates some success.

If that alternative is not viable or satisfactory, then focus on tracking implementation and collecting other management data that you can put to use. Alternatively, of course, you can raise more money. If the knowledge gap on your issue is big enough-you have a widely implemented program that hasn't been tested, for example, or you're trying a new approach in a conflict setting-then funders may be interested in knowing the answer, too.

6. Not Feasible: Indirect effects are difficult to identify, yet critical to the theory of change. Many programs include indirect effects that are critical to their theory of change. A farming-information intervention, for example, teaches some farmers new techniques and hopes that they share this information with their neighbors and extended family. A health intervention protects individuals from an infectious disease and anticipates that those who come into contact with the treated individuals are also helped, because they will also not contract the disease.

In these cases, a simple question ought to be asked: Does one reasonably believe (and ideally have some evidence from elsewhere) that the indirect effects are significant enough that ignoring them may radically alter the policy implication of the results? If so, then ignoring them could lead to a deeply flawed study-one that should not be done at all.

Measuring such indirect effects correctly is critical to understanding a program's true impact. Take the example of deworming school children. Prior to Edward Miguel and Michael Kremer's 2004 study of deworming in Econometrica, studies that tested the impact of school-based deworming typically randomized within schools, with some children receiving deworming pills and others not. Program effects were evaluated by comparing children who received treatment with those who did not. Yet there was good reason to believe that there were indirect effects across children within the same schools-children playing barefoot in the same schoolyard pass infection from one to the other. So within any given school, the control group also got partially treated. Imagine that this indirect effect is big-so big that it is the same size as the direct effect. Even if treatment had huge effects on health or schooling outcomes, comparing treated and untreated children would lead to the conclusion that deworming has no effect at all. Miguel and Kremer's deworming study explicitly measured these indirect effects. Doing so fundamentally changed the cost-benefit calculation of deworming: With indirect effects included, the benefits of deworming turned out to be quite large.

Alternative: Measuring indirect effects can be a feature of a good impact evaluation, rather than an obstacle. Of course, if indirect effects are ignored, then the presence of such issues can introduce bias, and thus incorrect conclusions.

In considering the response to indirect effects, a first tack is to review existing studies and theory to predict how important these issues are. If they are significant, and therefore important to measure, then there are two potential approaches to take: First, indirect effects can be included in the experimental design-for example, by creating two control groups: one that is exposed indirectly to treatment and the other that is not. Second, data can be collected on indirect effects. Ask participants who they talk to, and measure social networks so that the path of indirect effects can be estimated. If indirect effects can't be accurately estimated, however, and they are likely to be large, then impact evaluation is not a good choice. Resources will be wasted if true impact is masked by indirect effects.

7. Not Feasible: Program setting is too chaotic. Some situations are not amenable to impact evaluation. Many disaster-relief situations, for example, would be difficult, if not impossible, to evaluate, since implementation is constantly shifting to adapt to evolving circumstances. Maintaining strict experimental protocols could be costly, compromising the quality of the implementation. Even if not costly in theory, such protocols are unlikely to be adhered to in a rapidly changing environment and could prevent assistance from going to those who need it most.

Alternative: Track implementation activities and collect other management data that you can use to strengthen the program. Consider also whether there are operational questions that could generate useful learning. Operational (sometimes called rapid-cycle or rapid-fire or A/B) experiments can help improve implementation: Will sending a text message to remind someone to do something influence short-run behavior? How frequently should that text message be sent, at what time of day, and what exactly should it say? Is transferring funds via cash or mobile money more effective for getting money to those affected? How will lump-sum versus spread-out transfers influence short-run investment choices? Such short-run operational questions may be amenable to evaluation.

8. Not Feasible: Implementation happens at too high a level. Consider monetary or trade policy. Such reforms typically occur for an entire country. Randomizing policy at the country level would be infeasible and ridiculous. Policies implemented at lower levels-say counties or cities-might work for randomization if there are a sufficient number of cities and spillover effects are not a big issue. Similarly, advocacy campaigns are often targeted at a high level (countries, provinces, or regions) and may not be easily amenable to impact evaluation.

Alternative: A clear theory of intended policy change is critical. Then track implementation, feedback, and management data on whether the changes implied by theory are occurring as expected.

9. Not Worth It: We already know the answer. In some cases, the answer about whether a program works might already be known from another study, or set of studies. In that case, little will be learned from another impact evaluation. But sometimes donors or boards push for this unnecessary work to check their investments. And organizations may not be sure if the existing evidence is sufficient, leading them to invest in unnecessary impact evaluations "just to be sure."

Alternative: Resist demands for impact measurement and find good arguments for why available evidence applies to your work. In " The Generalizability Puzzle," their Summer 2017 article for Stanford Social Innovation Review, Mary Ann Bates and Rachel Glennerster provide some guidance. In short, two main conditions are key to assessing the applicability of existing studies. First, the theory behind the evaluated program must be similar to your program-in other words, the program relies on the same individual, biological, or social mechanism. Second, the contextual features that matter for the program should be relatively clear and similar to the context of your work.

We also suggest that donors consider the more critical issue for scaling up effective solutions: implementation. Use monitoring tools to ask: Does the implementation follow what is known about the program model? Again, track the activities and feedback to know whether the implementation adheres to the evidence from elsewhere. A good example of this is the Catch Up program in Zambia, where the Ministry of General Education is scaling up the proven Teaching at the Right Level (TaRL) approach pioneered by the Indian NGO Pratham. With support from IPA and the Abdul Latif Jameel Poverty Action Lab (J-PAL), teams in Zambia are taking the TaRL program, mapping evidence to the Zambian context, supporting pilot implementation, and monitoring and assessing viability for scale-up.

10. Not Worth It: No generalized knowledge gain. An impact evaluation should help determine why something works, not merely whether it works. Impact evaluations should not be undertaken if they will provide no generalizable knowledge on the "why" question- that is, if they are useful only to the implementing organization and only for that given implementation. This rule applies to programs with little possibility of scale, perhaps because the beneficiaries of a particular program are highly specialized or unusual, or because the program is rare and unlikely to be replicated or scaled. If evaluations have only a one-shot use, they are almost always not worth the cost.

Alternative: If a program is unlikely to run again or has little potential for scale-up or replication, the best course of action is to measure implementation to make sure the program is running as intended. If some idea about the "why" is needed, a clear program theory and good implementation data (including data on early outcomes) can also help shed light on why something works. But an investment in measuring impact in this situation is misplaced.

Collecting the Right Data

As should now be clear, the allure of measuring impact distracts from the more prosaic but crucial tasks of monitoring implementation and improving programs. Even the best idea will not have an impact if implemented poorly. And impact evaluation should not proceed without solid data on implementation. Too often, monitoring data are undervalued because they lack connection to critical organizational decisions and thus do not help organizations learn and iterate. When data are collected and then not used internally, monitoring is wasted overhead that doesn't contribute to organizational goals.

External demands for impact undervalue information on implementation because such data often remain unconnected to a theory of change showing how programs create impact. Without that connection, donors and boards overlook the usefulness of implementation data. Right-fit systems generate data that show progress toward impact for donors and provide decision makers with actionable information for improvement. These systems are just as important as proving impact.

How can organizations develop such right-fit monitoring systems? In The Goldilocks Challenge, we develop what we call the CART principles-four rules to help organizations seeking to build these systems. CART stands for data that are Credible, Actionable, Responsible, and Transportable.
Credible: Collect high-quality data and analyze them accurately.
Credible data are valid, reliable, and appropriately analyzed. Valid data accurately capture the core concept that one is seeking to measure. While this may sound obvious, collecting valid data can be tricky.

Seemingly straightforward concepts such as schooling or medical care may be measured in quite different ways in different settings. Consider trying to measure health-seeking behavior: Should people be asked about visits to the doctor? A nurse? A traditional healer? How the question is framed affects the answer you get.

Credible data are also reliable. Reliability requires consistency; the data collection procedure should capture data in a consistent way. An unreliable scale produces a different weight every time one steps on it; a reliable one does not.

The final component of the credible principle is appropriate analysis. Credible data analysis requires understanding when to measure impact-and, just as important, when not to measure it. Even high-quality data to measure impact without a counterfactual can produce incorrect estimates of impact.
Actionable: Collect data you can commit to use.
Even the most credible data are useless if they end up sitting on a shelf or in a data file, never to be used to help improve programming. The pressure to appear "data-driven" often leads organizations to collect more data than anyone can be reasonably expected to use. In theory, more information seems better, but in reality, when organizations collect more data than they can possibly use, they struggle to identify the information that will actually help them make decisions.

The actionable principle aims to solve this problem by calling on organizations to collect only data they will use. Organizations should ask three questions of every piece of data that they want to collect: (1) Is there a specific action that we will take based on the findings? (2) Do we have the resources necessary to implement that action? (3) Do we have the commitment required to take that action?
Responsible: Ensure that the benefits of data collection outweigh the costs.
The increasing ease of data collection can lull organizations into a "more is better" mentality. Weighing the full costs of data collection against the benefits avoids this trap. Cost includes the obvious direct costs of data collection but also includes the opportunity costs, since any money and time spent collecting data could have been used elsewhere. This foregone "opportunity" is a real cost. Costs to respondents-those providing the data-are significant but often overlooked. Responsible data collection also requires minimizing risks to these constituents through transparent processes, protection of individuals' sensitive information, and proper research protocols.

While collecting data has real costs, the benefits must also be considered. We incur a large social cost by collecting too little data. A lack of data about program implementation could hide flaws that are weakening a program. And without the ability to identify a problem in the first place, it cannot be fixed. Too little data can also lead to inefficient programs persisting, and thus money wasted. And too little data can also mean that donors do not know whether their money is being used effectively. That money could be spent on programs with a greater commitment to learning and improvement, or those with demonstrated impact.
Transportable: Collect data that generate knowledge for other programs.
Valuable lessons generated from monitoring and evaluations should help build more effective programs. To be transportable, monitoring and evaluation data should be placed in a generalizable context or theory-they should address the question of why something works. Such theories need not always be complex, but they should be detailed enough to guide data collection and identify the conditions under which the results are likely to hold. Clarifying the theory underlying the program is also critical to understanding whether and when to measure impact, as we have argued.

Transportability also requires transparency-organizations must be willing to share their findings. Monitoring and evaluation data based on a clear theory and made available to others support another key element of transportability: replication. Clear theory and monitoring data provide critical information about what should be replicated. Undertaking a program in another context provides powerful policy information about when and where a given intervention will work. A lack of transparency has real social costs. Without transparency, other organizations cannot identify the lessons for their own programs.

Creating a Right-fit System

CART provides organizations with a set of principles to guide them in deciding which credible data are most critical to collect. But organizations need to do more than simply collect the right data. They need to integrate the data fully into what they do. They need to develop right-fit evidence systems.

Creating such systems should be a priority for all organizations. First, many organizations will be better served by improving their systems for monitoring and managing performance, rather than focusing on measuring impact. Right-fit evidence systems provide credible and actionable data that are far more valuable than the results of a poorly run impact evaluation. Second, society is better served when organizations develop right-fit evidence systems. High-quality management data help organizations learn and improve. Transparent data that are connected to theory help build our generalized knowledge of what works-and in what settings. Good programs can be replicated, poor ones retired. Impact evaluations are undertaken only when the conditions are right-avoiding waste and maximizing scarce resources.

The first step in moving toward right-fit evidence happens at the organizational level. To support program learning and improvement, evidence must be actionable-that is, it must be incorporated into organizational decision-making processes. An actionable system of data management does three things: collect the right data, report the data in useful formats in a timely fashion, and create organizational capacity and commitment to using data.

Organizations should collect five types of monitoring data. Two of these- financial and activity (implementation) tracking-are already collected by many organizations to help them demonstrate accountability by tracking program implementation and its costs. The other three- targeting, engagement, and feedback -are less commonly collected but are critical for program improvement.

The key to right-sized monitoring data is finding a balance between external accountability requirements and internal management needs. Consider financial data first. External accountability requirements often focus on revenues and expenses at the administrative and programmatic levels. To move beyond accountability to learning, organizations need to connect cost and revenue data directly to ongoing operations. This way they can assess the relative costs of services across programs and program sites.

Many organizations also collect monitoring data about program implementation, including outputs delivered (e.g., trainings completed). But such data are not clearly connected to a decision-making system based on a clear theory for the program. A clear and detailed theory of change supports organizations in pinpointing the key outputs of each program activity so that they can develop credible measures for them.

Targeting data answer the question: Who is actually participating in the program? They help organizations understand if they are reaching their target populations and identify changes (to outreach efforts or program design, for example) that can be undertaken if they are not. To be useful, targeting data must be collected and reviewed regularly, so that corrective changes can be made in a timely manner.

Engagement data answer the question: Beyond showing up, are people using the program? Once organizations have collected activity tracking data and feel confident that a program is being well delivered, the next step is to understand whether the program works as intended from the participant perspective. Engagement data provide important information on program quality. How did participants interact with the product or service? How passionate were they? Did they take advantage of all the benefits they were offered?

Feedback data answer the question: What do people have to say about your program? Feedback data give information about its strengths and weaknesses from participants' perspectives. When engagement data reveal low participation, feedback data can provide information on why. Low engagement may signal that more feedback is needed from intended beneficiaries in order to improve program delivery.

Empowering Data

Another fundamental challenge to creating an actionable data system is empowering decision makers to use the data to make decisions. Empowerment requires capacity and commitment. Building organizational commitment requires sharing data internally, holding staff members responsible for reporting on data, and creating a culture of learning and inquiry.

To do this, organizations first need the capacity to share the data they collect. This does not require big investments in technology. It can be as simple as a chalkboard or as fancy as a computerized data dashboard, but the goal should be to find the simplest possible system that allows everyone access to the data in a timely fashion.

Next, the organization needs a procedure for reviewing data that can be integrated into program operations and organizational routines. Again, this need not be complex. Data can be presented and discussed at a weekly or monthly staff meeting. The important thing is that data are reviewed on a regular basis in a venue that involves both program managers and staff.

But just holding meetings will not be enough to create organizational commitment and build capacity if accountability and learning are not built into the process. Program staff should be responsible for reporting the data, sharing what is working well, and developing strategies to improve performance when things are not. Managers can demonstrate organizational commitment by engaging in meetings and listening to program staff. Accountability efforts should focus on the ability of staff to understand, explain, and develop responses to data-in other words, focus on learning and improvement, not on punishment.

The final element of an actionable system is consistent follow-up. Organizations must return to the data and actually use it to inform program decisions. Without consistent follow-up, staff will quickly learn that data collection doesn't really matter and will stop investing in the credibility of the data.

To simplify the task of improving data collection and analysis, we offer a three-question test that an organization can apply to all monitoring data it collects:

Can and will the (cost-effectively collected) data help manage the day-to-day operations or design decisions for your program?

Are the data useful for accountability, to verify that the organization is doing what it said it would do?

Will your organization commit to using the data and make investments in organizational structures necessary to do so?

If you cannot answer yes to at least one of these questions, then you probably should not be collecting the data.

Maybe this seemingly new turn away from impact evaluation is all a part of our plan to make rigorous evaluations even more useful to decision makers at the right time. When organizations or programs aren't ready for an impact evaluation, they still need good data to make decisions or improve the implementation of their model. And when a randomized evaluation (or six) shows that something works and it is ready for scale, a good monitoring system based on a sound theory of change is the critical link to ensuring quality implementation of the program as it scales.

In the interim, our plan is to shift the focus to evidence strategies that build learning and improvement. If this stratagem ultimately leads to more effective impact evaluations, so much the better.

# data[].attributes.attachment_link.title
Ten Reasons Not to Measure Impact-and What to Do Instead (SSIR)

# data[].attributes.attachment_link.description
Impact evaluations are an important tool for learning about effective solutions to social problems, but they are a good investment only in the right circumstances. Open access to this article is made possible by Kellogg School of Management at Northwestern University.

# data[].attributes.attachment_link.readable_text
Contact American Educator

Articles may be reproduced for noncommercial personal or educational use only; additional permission is required for any other purpose. For copies of articles not posted online, please email us at ae@aft.org.

Write to us! We welcome comments on American Educator articles. Send letters to ae@aft.org or American Educator, 555 New Jersey Ave. N.W., Washington, DC 20001. Letters selected for publication may be edited for length, clarity, and style; please include your email address so we may contact you if necessary.

# data[].attributes.attachment_link.description
Articles may be reproduced for noncommercial personal or educational use only; additional permission is required for any other purpose. For copies of articles not posted online, please email us at ae@aft.org. Write to us! We welcome comments on American Educator articles. Send letters to ae@aft.org or American Educator, 555 New Jersey Ave.

# data[].attributes.attachment_link.readable_text
Contact American Educator

Articles may be reproduced for noncommercial personal or educational use only; additional permission is required for any other purpose. For copies of articles not posted online, please email us at ae@aft.org.

Write to us! We welcome comments on American Educator articles. Send letters to ae@aft.org or American Educator, 555 New Jersey Ave. N.W., Washington, DC 20001. Letters selected for publication may be edited for length, clarity, and style; please include your email address so we may contact you if necessary.

# data[].attributes.attachment_link.description
Articles may be reproduced for noncommercial personal or educational use only; additional permission is required for any other purpose. For copies of articles not posted online, please email us at ae@aft.org. Write to us! We welcome comments on American Educator articles. Send letters to ae@aft.org or American Educator, 555 New Jersey Ave.