Saturday, October 31, 2015

Software to Indict Plagiarists and the edTPA

Remember that memo that Deputy Commissioner John D’Agati of New York State’s Education Department wrote back in July about what would happen to candidates of questionable moral character caught cheating on the edTPA? Here’s a piece to refresh your memory:
“As part of the edTPA scoring process, originality detection software is employed to compare all edTPA submissions nationwide against all other edTPA submissions received, including outside written sources and other sources of material. The software reports any substantial degree of matching between submitted edTPA portfolios. In cases where there is substantial matching, a specially trained portfolio reviewer may then elect to seek enforcement action against the candidate(s) involved and/or refer the candidate(s) to the State Education Department for enforcement action.” (7-23-15)
Now there’s another memo released a few days ago, and the originality detection software used at my college, called Safe Assign, picked up a 46% match! See if you can figure out why:

“It is important that candidates be made aware that, as part of the edTPA submission process, originality detection software is employed to compare all edTPA submissions against all new submissions, including outside written sources and other sources of material. In cases where there is substantial matching, a candidate’s edTPA score may be voided, and the candidate may not be eligible for the edTPA Safety Net. In cases when candidates have already received their teaching certificate, the Department may seek revocation of the certificate. Candidates will be given the opportunity to appeal a decision to void their scores, and that process will be explained in any communication they would receive if their score is voided.” (10-28-15)
All that’s really new in the latest memo is an acknowledgement that candidates should not be prevented from collaborating with each other pursuant to SCALE’s guidelines regarding acceptable forms of support, which was revised in April of 2014 after some questioned the extent to which peers were permitted to help each other through the edTPA process (I wrote about that too). In that document, SCALE stated the obvious: “within their coursework and key program assignments and activities, candidates receive feedback from instructors and fellow candidates.” Professor Laura Davies, in a thoughtful essay on the first D’Agati memo and the questioning of originality in the edTPA process from September 23rd  raises important issues about the harmful unintended consequences of generating so much fear regarding collaboration, a prominent feature of the work of teaching in a profession that suffers from teachers’ isolation from each other.
Recently the first significant empirical study of the edTPA rollout in New York and Washington was published, and among the findings of Meuwissen and Choppin is confirmation that candidates, desperate for clarity and help navigating the complex handbook instructions and rubric guides to ensure a passing score on the edTPA, have found clever secretive ways to get and give support to each other, mostly through social networking. Candidates interviewed in the study also mediated support tensions in their placements in schools, which can place considerable constraints on the teaching they do for the edTPA. Just working out the logistics of which students will be videotaped, what curriculum will be used, and when and where to do the taping is a source of stress and despair more often than not in my own experience with student teachers.
Now imagine the candidate that finally submits an edTPA portfolio, believing to have properly cited the use of the school’s curriculum in the lesson plans and commentary, turning up a high percentage of matching in the originality detection software used by scorers at Pearson. Theoretically such a portfolio is flagged and sent to a specialist to determine the degree of culpable intentionality in the presumed crime. Similarly, imagine a candidate being flagged by the software for using common phrases and routine descriptions in the commentaries that are similarly flagged by the software as matching other text. Alan Singer addressed this problem here. The software doesn’t judge, that’s up to a specialist at Pearson. But how exactly does that specialist make a judgment, one that could jeopardize employment, either due to delays in receiving a score, or due to a guilty verdict where none is merited?
What’s more, why would ANY candidate agree to allow the edTPA portfolio to be used by SCALE, Pearson, or the college or university, when there is a risk that at some point some stranger could plagiarize from that portfolio without the knowledge of the author, and that author could be subjected to having his or her license revoked?
While we’re considering what is legally unfair about all of this, have a look at the fine print regarding candidate’s compliance with edTPA rules (italics are mine):
From edTPA.com site:
“edTPA Rules of Assessment Participation
COMPLIANCE WITH ASSESSMENT RULES
By registering for edTPA, you are agreeing to abide by the current Rules of Assessment Participation for edTPA and all rules, procedures, and policies contained on the current edTPA.com website and/or in the current edTPA Handbook for the content area for which you have registered.
For the purpose of these Rules of Assessment Participation, the following definitions apply:
       "Evaluation Systems." Evaluation Systems, a business of NCS Pearson, Inc. (referred to throughout as Evaluation Systems).
       "Program." The educator preparation program in which an edTPA candidate is enrolled as indicated by the candidate at the time of registration.
       "State Agency." The educator certification agency in any state in which an edTPA candidate is pursuing certification with this assessment.
Rules of Assessment Participation for edTPA
1     PURPOSE OF ASSESSMENT: I understand that this assessment is administered for the purpose of fulfilling a program requirement and/or a state teacher licensure requirement and is only to be taken by individuals to fulfill such requirement(s). I certify that I am taking this assessment for which I have registered, to fulfill a program requirement or for the purpose of teacher licensure.
2     ORIGINALITY OF SUBMISSION: I understand that by submitting my edTPA materials, I am confirming that I am the person who has completed the assessment, that I have primary responsibility for teaching the students/class during the learning segment profiled in this assessment, that the video clip(s) submitted show me teaching the students/class profiled, that the student work included in the documentation is that of my students and completed during the learning segment documented in this assessment, that I am the sole author of the commentaries and other written responses to prompts and other requests for information in this assessment, and that I have cited all materials in the assessment whose sources are from published text, the internet, or other educators.
3     PERMISSIONS AND CONFIDENTIALITY: I understand that I am responsible for obtaining appropriate permissions from the parents/guardians of my students and from adults that appear on the video clip(s) I submit. I agree to produce such permissions if requested after I submit my assessment. I have ensured confidentiality of individuals appearing in the video clip(s) I submit by uploading the video only to the designated Pearson ePortfolio system, an integrated third party edTPA Platform Provider system, or other secure system designated by my program. I understand that I may use my assessment materials according to the parameters of the release forms obtained for children and/or adults who appear in the video. Because parents/guardians and/or adults have not typically granted permission for public use of the videos in which they or their children appear, I will not display videos publicly (i.e., personal websites, YouTube, Facebook) without expressed permission for this purpose from those featured in the video.
4     ASSESSMENT MATERIALS: I acknowledge that I am not permitted to reproduce or share any of the information or materials from edTPA handbooks or support materials (Making Good Choices or other materials with Stanford copyright) for commercial purposes. If I do reproduce information or materials from the edTPA handbooks or related materials for personal use, I will properly attribute the copyright of such materials to Stanford University.
5     USE OF ASSESSMENT: I agree that my edTPA submission, including text, graphics, digital files and video or audio recordings, without the use of my name or other identifying information, may be used by Stanford University and/or Evaluation Systems edTPA program development and implementation, including scorer training associated with the program. If I provided consent as part of my response to registration questions, my submission, without use of my name or other identifying information, may also be used for continued edTPA program activities conducted by Stanford University and/or Evaluation Systems such as future validity and reliability studies of the edTPA. Stanford University and Evaluation Systems will not show candidate materials publicly, make them available in a non-secure way, or use them as exemplars for marketing purposes.
6     SCORE REPORTING AND CANCELLATION: I understand that my results will be reported to me; to the program authorized by me during registration and/or State Agency, if applicable; and to any other institution, entity, or person authorized or required by law to receive this information. edTPA results are anonymously provided to SCALE with candidate responses to registration background questions which address edTPA placement context and demographics of edTPA candidates, including educator preparation program and state affiliation, for the purpose of edTPA assessment analyses and assessment development. edTPA results received by SCALE will not include candidate name or other personally identifying information such as date of birth or partial social security number. I understand that any information provided as part of registration may be used to report scores or to contact me regarding assessment- or program-related issues. Once I submit my assessment, I cannot cancel the scoring or score reporting.
7     CONFORMITY WITH PROCEDURES: I understand that if my submitted artifacts, videos, and/or related documentation do not conform to the current rules, requirements and polices as specified in the edTPA Assessment Handbook, and the edTPA website, my submission or portions thereof may not be scored, my score may be voided and other actions as described in Rule 11 may be taken as deemed appropriate by Evaluation Systems, my program, and/or State Agency. If my complete submission or portions thereof cannot be scored because it does not conform to requirements, no refund of my fee will be issued, and no portion of my fee can be applied to the cost of any future edTPA registration or associated services. If my submission cannot be scored due to a system error occurring after submission, I will have the opportunity to resubmit my portfolio without paying additional fees. I understand that my submitted assessment materials, or a portion thereof, may be reviewed by authorized individuals at the program in which I am enrolled, as indicated at the time of registration, and/or the relevant State Agency responsible for educator certification in order to investigate compliance with the Rules of Assessment Participation, as needed.
8     RIGHTS AND OBLIGATIONS REGARDING edTPA: I understand and agree that liability for assessment activities, including but not limited to the adequacy or accuracy of assessment materials, of the registration processes, of scoring, of score reports, of information provided to me in connection with edTPA and the adequacy of protection of candidate information, will be limited to score correction or edTPA retake at no additional fee. I understand and agree that liability for data loss or file corruption associated with my edTPA submission will be limited to an additional edTPA submission at no additional fee. I waive any and all rights to all other claims, specifically including but not limited to claims for negligence arising out of any acts or omissions of Stanford University, Evaluation Systems, and/or the state or program which is requiring completion of the edTPA (including the agents, employees, contractors, or professional advisors of Stanford University, Evaluation Systems, or such entity).
9     PROGRAM CHANGES: I understand that the edTPA assessment and associated policies and procedures are subject to change at the sole discretion of Stanford University and Evaluation Systems. State Agencies or individual programs may make changes to their policies and requirements related to the edTPA at their discretion.
10    OBJECTION TO PROCEDURES: If, for any reason, I object to the procedures presented in these Rules of Assessment Participation, I must advise Evaluation Systems, in writing, of the basis of my objection at least six (6) weeks before I plan to register for edTPA for my objection to be taken under consideration. If my objection is not honored, I will not be registered for edTPA.
11     COMPLIANCE: I understand that if I fail to comply with the rules, requirements, and policies specified or referenced on the current edTPA website, including these Rules of Assessment Participation, or if I take any prohibited actions, my results may be voided, no refund will be issued, no portion of the assessment fee can be applied toward the cost of any future assessment fees, my registration may be canceled, I may be prohibited in the future from registering for edTPA. Legal proceedings and actions may be pursued as well as other remedies deemed appropriate by Evaluation Systems, my program or State Agency, as appropriate. In addition, I understand that assessment fraud may be grounds for denial, revocation and/or suspension of a teaching license.
RULES: I understand that should any of these rules or any other requirement or provision contained on the current edTPA.com website be declared or determined by any court to be illegal or invalid, the remaining rules, requirements, and provisions will not be affected and the illegal or invalid rule, requirement, or provision shall not be deemed a part of the current edTPA website. The headings of each of the Rules of Assessment Participation for edTPA are for convenient reference only. They are not a part of the rules themselves; they do not necessarily reflect the entire subject matter of each rule; and they are not intended to be used for the purpose of modifying, interpreting, or construing any of these Rules of Assessment Participation for edTPA. I agree that any legal action arising in connection with my registration for or participation in edTPA shall be brought in the state and federal courts governing St. Paul, Minnesota, and I consent to the personal jurisdiction of such courts.

Voiding of Scores:
If you violate one of the Rules of Assessment Participation or if doubts are raised about the validity or legitimacy of your registration or your scores, Evaluation Systems may notify the educator preparation program and/or State Agency you identified during the registration process, as applicable.
Evaluation Systems reserves the right to void your scores if in their sole opinion, or after consultation with the educator preparation program and/or State Agency you identified during the registration process (as appropriate), there is adequate reason to question your scores' validity or legitimacy, due to misconduct including, but not limited to, a violation of the rules set forth on the current edTPA website, including the Rules of Assessment Participation.
Further action may be taken, including remedies deemed appropriate by Evaluation Systems, your educator preparation program or State Agency, as applicable.
Please note that software may be employed to screen submissions for originality of content. Submissions determined to violate edTPA rules regarding the originality of the submitted material will be subject to actions described above.”


We’re always warned to read the fine print, but that doesn’t really mean we have much power to do anything about it, especially when it is a requirement of employment. Now, imagine that candidate I described above, waiting since last spring to receive an edTPA score, getting nowhere with Pearson, SCALE, or the program, and knowing only that the portfolio is in a limbo of administrative review. Maybe that’s not hypothetical. Maybe that is reality.

34 comments:

  1. This describes the perfect storm.

    The corporations did it for tort reform and screwed the people by taking away a victims right the take a corporation to court when the services or products they sell damage/injur the customer.

    They are doing it with prison reform.

    They are doing it with the reform of the legal system: courts, police, etc.

    And they are doing it with education reform.

    The small print always strips the people of their constitutional rights to justice.
    If you buy a mobile phone, the small print that you agree to strips you of your right to take, for instance, AT&T to court for alleged fraud.

    I recently read that the use of small print in almost every service contract that we sign up for started in the 1970s and as profits have soared into orbit for corporation, the fine print has spread like terminal cancer and it had paralleled the corporate war on labor unions.

    ReplyDelete
  2. Battle in Illinois Over the “McDonaldization” of Teacher Education – Alan Singer’s Latest Huffington Post

    http://www.huffingtonpost.com/alan-singer/battle-in-illinois-over-t_b_8509212.html

    ReplyDelete
  3. This is exactly what I am going through now with Pearson and it got to the point where I re-did the whole portfolio and submitted a brand new one that is now also administrative review.

    ReplyDelete
    Replies
    1. That is a nightmare! I hope that when this is all behind you, you might consider going public with your experience. I'd be happy to help you do so through my blog. I exchanged messages with someone on Twitter who also redid the edTPA after being in administrative review, even though she cited the Ready Gen lesson/unit clearly in her commentary. There is zero transparency on the software detection and it is clearly a problem causing grave injustice, undue anxiety, and it is blatantly unfair. I am so sorry you are going through this TWICE. Hang in there.

      Delete
  4. I recently found out that my edTPA is in administrative review.. what does this mean? after spending a year working on the portfolio, and another month and a half waiting for my score, I am sickened by this. when I called the company, they told me to re-read their website and that they have zero additional information. I have completed all other parts of my NYS dual certification and am losing sleep over this edTPA situation.

    ReplyDelete
    Replies
    1. I just found out this same thing today-have you gotten any response from this situation?? I am not sure what to expect

      Delete
    2. Kaitlin, see my comment to Erica below. See if your college can contact the state on your behalf.

      Delete
    3. did you receive scores or just a letter saying under administrative review?

      Delete
    4. Never got scores. My portfolio never even made it to a scorer.

      Delete
    5. I need help with this too. Mine is under administrative review and I got a letter saying to mail a written submission of any additional information regarding the originality of my submission. I have limited time, What do I do?

      Delete
  5. Erica, I am sorry you are stuck in limbo hell. All I can suggest is that you make a lot of noise and complain about Pearson and the state taking so long. I think what happens is they run the portfolio through a software detection program for plagiarism, and then it is supposed to be reviewed by an administrator (whether at Pearson, the state, or both is unclear) to determine the outcome. I don't know what you do to appeal the decision once they send it to you but I think it is outrageous that you have been held up all these weeks. Maybe see if your college can pursue it with the state.

    ReplyDelete
  6. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. Kuku your story is horrifying. I don't know what else to say.

      Delete
  7. Is this something that typically happens with the commentary sections? Like planning commentary, assessment commentary and instructional commentary? I'm a little worried because my lesson I modified for my student but I got many of the ideas and strategies from other lessons. However, the materials, assessments, and supports are different. EdTPA makes me really nervous. You can't plagiarize a lesson, if you taught it. And as teachers we borrow ideas from other teachers and modify them to fit our students. This is teaching.

    ReplyDelete
    Replies
    1. It's hardto know what triggers an admin review, because the process is not transpatent. There is no way to know what they suspect you of doing, beyond general non- compliance. The best protection is to thoroughly (i.e. APA cites) give credit where it's due.

      Delete
    2. What happened to me was another cohort student and I were in the same school, same grade. So we collaborated on our lessons and were encouraged to do so. When you submit to edTPA, and we did not know this at the time, it goes through a plagiarism program similar to turnitin.com to see if there are any other portfolios similar. It automatically caught her's and mine and we were put on Administrative Review. When your portfolio is put on that, the only way to contact the company is through fax or USPS. So we mailed them a letter asking what the problem was and received a letter back about 2 weeks later. All it told us was that we were on Administrative Review (which we already knew). It gave us no new information other than that it looked like we cheated. So, we got our program's Dean involved and many other people. At that time, we decided to be proactive and REDID our ENTIRE edTPA while we were still with our students. We knew we couldn't just sit there and wait around. About another 2-3 weeks after that, we got another letter saying what part seemed to be "copied"-and it was a a commentary section. So, then, the Dean actually vouched for us and many others by writing rebuttal letters fighting it. Unfortunately, it did no good. So, after receiving a final letter that said the portfolio was void, we resubmitted what we worked on the second time. That other girl and I did not work on ANYTHING together this time and we went to totally different grades. We actually got our new scores back this week and passed! It was the WORST thing that I have ever been through, so I am just glad it is over.

      Delete
    3. Kaitlin, thank you for sharing your story. It deeply saddens me that the message is not to collaborate, for that is a foundational idea in teaching, and as it is teaching can be very isolating. Teachers have to actively work on collaboration. That the edTPA process sends the opposite message, and actually harshly punishes those who do collaborate, is one of the many reasons why I hope for its demise. I wish you all the best as you move forward into your first year of teaching.

      Delete
    4. Thank you for sharing your story. I am currently a little worried. I'm supposed to get scores back June 1, but it seems as though many people from my college collaborated with one another and everyone's really worried because we were unaware we could do so. Additionally, our lesson plans kind of look very similar to past edTPAs so everyone's really worried, including myself. It was the most grueling thing I've ever done so the thought of having to do it again is driving me nuts. However, I rather they tell me the submission is simply voided than remove my certificate, you know. I'm so incredibly sorry, this happened to you and trust me when I say I wouldn't even wish that on the worse people. But I am so happy you are on the other side now, and all of that is behind you. What I'm getting is that the commentary sections are looked at under a microscope. My commentary section is very very different, and everything I used I did cite. I'm just worried about the lesson plan as I did use gradual release model and reciprocal teaching, very similar to other lesson plans and even with similar justifications. However, the results and materials are all different and cited. I agree with Alexandra. Collaboration is a huge part of teaching, and it sucks that edTPA expects people to completely reinvent the wheel. However, I'm incredibly happy that you've passed and able to enjoy your summer. Hopefully, I'll be able to do the same. I'll keep you both updated.

      Delete
    5. Im in the middle of Kaitlins situation. They want a mailed response back with any additionaly information regarding the originality of my submission. Is there anything I can say to make them pass it and not make it void? That would put me back quite a bit.

      Delete
  8. If a score is voided do you have to complete an entirely new lesson or can you use the same one and just retype everything?

    ReplyDelete
  9. No one could ever give us an actual answer to that question. We didn't want to risk it so redid all. I think someone else did just retyped the part they said was invalid, I think. Not completely positive.

    ReplyDelete
  10. Did it actually take them 60 days to respond to you?

    ReplyDelete
    Replies
    1. I was told today that mine is under review & I am having complete anxiety about it. This is going to push me back from getting a job in September if they take 60 days to let me know if it has been voided.

      Delete
    2. Nope! It won't take that long, they just say that to cover their behinds.

      Delete
  11. I was suppose to receive my score yesterday. The website still says pending and I didn't receive an email. Does this mean I am under review?

    ReplyDelete
    Replies
    1. Most likely.... that's how it seems to happen

      Delete
  12. This comment has been removed by the author.

    ReplyDelete
  13. You can resubmit at any time. We redid everything, no one could give us an answer about having to redo the part that was under scrutiny or the entire thing. It was difficult, but we got through it.

    ReplyDelete
  14. I read that I cannot register to resubmit until the score is voided? Is this true? I am on a time crunch.

    ReplyDelete
  15. And thank you for your help! There does not seem to be much info available and they are of no help.

    ReplyDelete
  16. And how long did it take for you to get an answer? I was going to send a letter...is it even worth it?

    ReplyDelete
  17. I should remember better than I do, but I think you're right. As soon as you get the voided score, you can resubmit. We just went ahead and redid all tasks while waiting on the voided score. Once we did receive it, we resubmitted immediately. I would not send a letter, it won't do any good. If it'll make you feel better you can, just to defend yourself.

    ReplyDelete
  18. edTPA is a pile of s**t to begin with and this administrative review malarkey is even worse. I was supposed to get my score yesterday and received no notification whatsoever. I knew something was up when I didn't get the email the day before about my score report but now it seems I will have to reach out to Pearson to confirm whether my submission is in administrative review. It doesn't seem fair that they can do this without sending something (a generic email perhaps) letting you know that your score will not be available on the reporting date. Yes, we agree to the terms and all that blah blah blah we know they can delay it for administrative review but why not let candidates know instead of making us wait for the score report date and when we don't get them by email and the website just says "pending"? They can send you an email to say that you will get your score tomorrow but can't let you know if you're in administrative review?

    ReplyDelete