https://padlet.com/ilana_s_horn/2oz1bfu7bqm4 en-us 2018-04-02 20:37:48 UTC 2023-03-22 03:23:47 UTC hello@padlet.com https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934031 VAM in a lot of ways seems more fair than mere achievement scores used for teacher evaluation, in that it moves away from the incentive to teach the students most likely to score well. At least in theory, it could actually incentivize teachers to teach the "lowest-performing" students because they may be seen as "easier" to move. I see this as one affordance, given the ways I saw teachers fight to not teach these students under our testing regime in Oklahoma (literally, lying about students' disabilities or English abilities to get them out of their classes so they wouldn't count against the teacher on district benchmark exam results--the results of which were publicly circulated by our principal). However, I fear that VAMs have a great deal of potential to be misused, especially given the noisiness and imprecision Goldstein (and others) pointed out. And in particular in our current policy climate-- we can see a cautionary tale in the way that A Nation at Risk played out. It recommended several policy initiatives that were immediately skipped in favor of the easiest and most politically-viable one: blame teachers. Without strict commitment to VAMs only as ONE PART of a teacher evaluation system, and without attention to and commitment to equitable funding / resourcing / desegregating of public schooling, VAMs will remain a dangerous element of teacher evaluation. ]]> 2018-04-02 20:41:48 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934031 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934050 Scoring (bias, systematic error) - VAMs measures not only how well they teach but whom and where they teach
Generalization (reliability, random error) - VAMs are noisy
Extrapolation (relation with other measures) - VAMs conflict with observation data/ student climate surveys, other reading/math tests. standardized tests are not really indicative of rigorous standards, standardized tests ignore the importance oc noncognitive tests
Implication (uses and consequences) appropriate uses: "for researchers compairing large groups of teachers to investigate the effects of teacher training approaches or educational policies, or simply to investigate the size and importance of long-term teacher effects." there is the assumption that researchers have "the training and expertise to respect their limitations"]]> 2018-04-02 20:41:52 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934050 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934075 I like the idea of measuring/getting an idea about what students are learning/know, but I don't know if this should be used to measure teacher effectiveness. VAM doesn't take into account a number of factors (contexts, peers, LIFE). I maybe could be convinced that it should be a small part of the overall picture... but only maybe. 

Do other jobs in the human improvement fields even have a measure like this to determine effectiveness? (shout out to social workers) .]]> 2018-04-02 20:41:56 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934075 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934083 Although they seem like a logical way to think about teachers, I am unconvinced of their ability to provide give an accurate picture of teaching. How they are used as a determiner of pay is problematic because it doesn't promote the use of them for formative purposes. They do not allow for nuance in measuring based on the context. They also are problematic because of the ease of seeing their results hides the bigger more influential issues at stake in students' experiences at schools. This is an easy talking point that does not adequately facilitate improvement of teaching.]]> 2018-04-02 20:42:00 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934083 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934087 VAM should not be used in isolation but as part of multi-measure system]]> 2018-04-02 20:42:02 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934087 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934110 I can understand the want of VAMs in thinking about both teacher evaluation, but I worried about how quickly and ubiquitously agregate formulations of quantitative data are taken up by policy makers. 
As a formative measure, I think there are thoughtful ways to use and think about VAMs, but I think these may be difficult and have a steep learning curve. ]]> 2018-04-02 20:42:07 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934110 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934136 VAM does help with defining teachers' expectations in someway. It lacks the consideration of students' individual gains in learning as well as the lack of multiple measurements in teachers' evaluation. I agree to implement VAM as a evaluating system, but on the condition that it is based on the different contexts and students in those contexts. VAM makes learning and teaching become more goal-driven, but if designed in a situated way, it should play an important role in teachers' evaluation. ]]> 2018-04-02 20:42:12 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934136 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934140 Although I'm not opposed to the idea of assessing teaching practice for the purpose of professional growth (or, more broadly, ensuring that students' access to "good" teachers is distributed randomly or at least in ways that are not dependent on their race, class, language, ability, and other social markers), I'm not convinced that VAM can be a nuanced enough tool on a practical level (in terms of how much time it takes to design and validate surveys or standardized assessments or evaluation rubrics, to actually conduct and document meaningful observations, etc.) to be worth the many, many potential risks (not the least of which is the time and money we spend arguing about what is worth measuring and how we should measure it and how well it's measured by the way we thought we should measure it). I'd be more open to a random sampling evaluation than evaluating every single teacher every single year, because the drastically reduced burden would open up the possibility for less "scalable" and thus more intensive and hopefully sensitive/nuanced metrics. ]]> 2018-04-02 20:42:15 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934140 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934153 It seems like VAMs really hinge on the reliability and meaningfulness of the data they use, the models used to make sense of the data, and the subsequent interpretstions. These all require huge amounts of knowledge in disparate fields (e.g., statistics, test-making, content knowledge, etc.) that would be difficult to bring together. From our readings and discussion today, I think VAMs are an interesting way to model the world and have the potential to provide insight into student and teacher growth, but I do not think they should be used beyond an exploratory/ theory building manner.]]> 2018-04-02 20:42:18 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934153 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934205 Seems that taking into account ideas about teacher collaboration (including things like the idea that collective commitment to students is predictive of better outcomes for students, even when taking into account factors such as the usual dip in scores from early-career teachers, etc.) might be useful in considering VAM-type criteria at a different scale. Can some of the problems with VAM be ameliorated by this kind of change-of-scale/change-of-consequences? I agree that it seems like historically they haven't, and maybe they really just are subject to Campbell's Law, and high-quality qualitative research is too costly and cumbersome, but we can't really be saying that we want to abandon any ideas of being able to evaluate teachers in any way that carries from one place to another, can we? Can we propose something like an ongoing conversation rather than the centralized locus of authority implied by doing anything "at scale"?

]]> 2018-04-02 20:42:36 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934205 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934218 I was pretty convinced by the quote Janna read from the Haertel article (p. 25) in which it essentially said VAM's aren't inherently evil, however they need to be used in moderation (like any healthy diet) in the context of other information in settings in which teachers and administrators can act autonomously. In addition, Hartel argued that it is imperative that those interpreting/using VAM's actually understand them and know how to interpret them, similarly to how Lani knew how to interpret the meaningless box and whiskers plot she got of her own son's "academic achievement." However, similarly to Hartel and Goldstein the unreliability and validity issues seem to also be highly problematic, however if used in moderation this might help them be used more as Brette would want them to be used, as check engine lights and not as gas indicators. In general, any system that privileges evaluation of individual teachers, and locates problems in the American institution of education within individuals, is destined to fail. Without any sort of acknowledgement, understanding, and/or evaluation of the social and political systems at play things will never change for the better :(]]> 2018-04-02 20:42:41 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934218 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934245 I believe that there could be productive uses of VAMs as an aspect (and only an aspect) of teacher evaluation. In an ideal situation, they might even be extremely useful. But I'm skeptical about the way we would, in practice, implement them. It's too easy to take that one snapshot and give it inordinate weight in the evaluation of one's teaching practice. 

]]> 2018-04-02 20:42:49 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934245 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934433 It seems the underlying tests themselves are the problem - if we had valid, reliable, and worthy tests on which to calculate VAM scores, I think it would make much more sense as both an evaluative and formative tool

]]> 2018-04-02 20:43:49 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934433 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934466 I am not convinced that VAMs are designed in accordance with what defines good (or to use Fenstermacher's term) or quality teaching. If what is intended is to measure whether or not teachers are completing specific tasks, that is one thing; but to truly get at what students are learning is a far more complicated and subjective topic that cannot be assessed within such a narrow scope. ]]> 2018-04-02 20:44:01 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247934466 ilana_s_horn https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247935051 2018-04-02 20:47:25 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247935051 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247935550 Is it just me or is anyone else starting to think about the VMA's after looking at the letters VAM's all afternoon?]]> 2018-04-02 20:50:38 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247935550 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247935583 2018-04-02 20:50:51 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247935583 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247935828 2018-04-02 20:52:10 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247935828 https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247936202 2018-04-02 20:54:13 UTC https://padlet.com/ilana_s_horn/2oz1bfu7bqm4/wish/247936202