{"id":6387,"date":"2018-08-02T17:30:09","date_gmt":"2018-08-03T00:30:09","guid":{"rendered":"http:\/\/dev.essaypop.com\/?p=6387"},"modified":"2020-01-18T14:09:54","modified_gmt":"2020-01-18T22:09:54","slug":"part-two-the-skeptics","status":"publish","type":"post","link":"https:\/\/blog.essaypop.com\/features\/part-two-the-skeptics","title":{"rendered":"AI to the Rescue? (Part 2)"},"content":{"rendered":"

If you listen to the executives at <\/span>Pearson<\/b> an and the other big edtech companies who are using AI to assess student writing, you begin to believe that we at the dawning of a new era. \u00a0We have reached a point where we can feed our students\u2019 essays into an algorithmic software like Pearson\u2019s <\/span>Intelligent Essay Assessor<\/b> and they can can get feedback that is not only instantaneous, but substantive as well. \u00a0The gradebots are here and they want to help! <\/span><\/p>\n

Most of my English-teaching colleagues, as I mentioned in <\/span>part one<\/span> of this post, are absolutely open to this kind of technological assistance. \u00a0Sure, some have expressed the fear that algorithm-based assessment may one day alleviate the need for teachers, and I understand their concerns, but don\u2019t necessarily share them. \u00a0<\/span><\/p>\n

I see the AI that is being developed as something that savvy educators will use take off some of the pressure (as would be the case with automated essay assessment), and something that that would enhance student engagement. \u00a0I\u2019m just one of those who believe that the role of the organic, flesh-and-blood teacher could never really be replaced. <\/span><\/p>\n

I don\u2019t fear AI assessment. I\u2019m just skeptical about whether or not it really works. I\u2019ve assessed and provided feedback on tens of thousands of papers in my 25 years as an English teacher, and I know that there are infinite ways to express oneself in writing. Every writer has a different, approach, a different take, a different voice. \u00a0Writing is a strange, organic, often nonlinear thing. Sure, there are grammar, spelling and syntax conventions which I imagine a computer could identify and, if necessary, correct. And certainly, paragraph structure and overall coherence of thought is critical to good writing, but so much more goes into it.<\/span><\/p>\n

I like what the people at <\/span>Hubert ai<\/b>, an edtech company out of Sweden say about this:<\/span><\/p>\n

The big question is just how much of a poet a computer is capable of becoming in order to recognize small but significant nuances the can mean the difference between a good essay and a great essay. \u00a0Can it capture essentials of written communication: reasoning, ethical stance, argumentation, clarity?<\/span><\/p><\/blockquote>\n

This is precisely it for me and why, at my core, I am skeptical when it comes to the claims made by those who announce that that the assessment bots have arrived, and that they\u2019re just as good, if not better than human graders. \u00a0Which brings me to the views of an even greater skeptic of AI assessment, <\/span>Les Perelman, former director of undergraduate writing at MIT.<\/span><\/p>\n

Perlman in his studies has found that AI writing assessment software is light years away from being an effective or reliable means of judging student work. \u00a0He even demonstrated this in a very humorous way by developing a software, called <\/span>Babel Generator<\/b>, which constructs complete essays within seconds based on keywords. \u00a0The essays use rich vocabulary, complex sentence construction and flawless syntax, but, when read, are completely meaningless. \u00a0Check out this nugget I generated from Babel Generator, using the keywords <\/span>hybrid<\/span><\/i>, <\/span>climate<\/span><\/i> and <\/span>future<\/span><\/i>. <\/span><\/p>\n

\u201cAs I have learned in my theory of knowledge class, humanity will always incarcerate time to come. The same brain may emit two different neutrinoes to reproduce. Despite the fact that gravity counteracts plasmas, the same brain may receive two different neurons of accessions.\u201d<\/span><\/p>\n

Wow! What a mess, and yet, the kind of \u00a0written expression that a software like Pearson\u2019s <\/span>Intelligent Essay Assessor<\/b> might score a proficient or beyond-proficient level. Perlman has also made intelligible, but totally outrageous claims and been rewarded by automated grading systems. \u00a0On an essay he submitted to <\/span>e-Rater<\/b>, the automated grader developed by ETS (Educational Testing Service) and very similar to Person\u2019s technology in response to a prompt asking respondents to discuss some of the challenges facing education, he argued that the main challenge is that teachers assistants are paid six times as much as college presidents and receive extra benefits such as south seas vacations and trips on private jets, thus causing financial stress on universities. He received a 6 out of 6 on the essay. \u00a0He has gamed and fooled a number of systems, and now companies that produce such software won\u2019t even talk to him. I can\u2019t blame them.<\/span><\/p>\n

Perlman\u2019s work supports what I\u2019ve sort of suspected all along. Any experienced English teacher knows that assessing student style, reasoning, critical thought, voice, et al is uniquely human ability. <\/span>\u00a0An essay is at its core a dialogue between a person with something to say and a human audience that cares to spend some time listening. \u00a0It is not an interaction between a person and a machine. The machine doesn\u2019t care. It can\u2019t care. As I say this, I can\u2019t help but be reminded of the movie, <\/span>Her<\/b> in which Joaquin Phoenix \u00a0forges an \u201cintimate\u201d relationship with the voice on his phone which seems somewhat human, but clearly falls short in the end. \u00a0I see the automated grading software as something \u201clike\u201d a real teacher, but not quite the real thing. <\/span><\/p>\n

This is not to say that computer scientists won\u2019t someday crack the essay-assessment code. \u00a0AI is changing the way we interact with the world in breathtaking ways, and, as I\u2019ve said (and I believe I speak for many an overburdened English teacher), I would welcome an essay bot that could provide meaningful and substantive feedback to my students quickly. \u00a0I would love to dedicate all of my time to the fun part, <\/span>the teaching part<\/b>. \u00a0I just don\u2019t think that we\u2019re there yet. \u00a0<\/span><\/p>\n

Despite what companies like Pearson and ETS claim, real assessment and real feedback still lies in the hands of real teachers. \u00a0What remains is for us to determine ways that we can take the pressure off of them that smartly leverages technology in a way that maximizes the teachers\u2019 ability to bring their expertise, experience and <\/span>human<\/b> judgement in the equation.<\/span><\/p>\n

 <\/p>\n\n

\n <\/div>\n\n","protected":false},"excerpt":{"rendered":"

Has artificial intelligence reached the point where it can assess and provide feedback on student essays?<\/p>\n","protected":false},"author":6,"featured_media":7014,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15],"tags":[188,187,189],"_links":{"self":[{"href":"https:\/\/blog.essaypop.com\/wp-json\/wp\/v2\/posts\/6387"}],"collection":[{"href":"https:\/\/blog.essaypop.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.essaypop.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.essaypop.com\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.essaypop.com\/wp-json\/wp\/v2\/comments?post=6387"}],"version-history":[{"count":9,"href":"https:\/\/blog.essaypop.com\/wp-json\/wp\/v2\/posts\/6387\/revisions"}],"predecessor-version":[{"id":8910,"href":"https:\/\/blog.essaypop.com\/wp-json\/wp\/v2\/posts\/6387\/revisions\/8910"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.essaypop.com\/wp-json\/wp\/v2\/media\/7014"}],"wp:attachment":[{"href":"https:\/\/blog.essaypop.com\/wp-json\/wp\/v2\/media?parent=6387"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.essaypop.com\/wp-json\/wp\/v2\/categories?post=6387"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.essaypop.com\/wp-json\/wp\/v2\/tags?post=6387"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}