Will ChatGPT get you caught? Rethinking of Plagiarism Detection

主要内容

文章主要是研究chatgpt出现后，在学术界中可能出现的学术抄袭和剽窃现象。
这篇文章就比较了几种剽窃抄袭软件，来测试是否能够识别chatgpt编写的内容。
最后得到的结论是：利用chatgpt本身就能识别出或者判断出，某段文本是不是chatgpt编写的？

we asked the ChatGPT “is this text generated by a chatbot?” and then pasted 
the essays that had already been generated. With an accuracy of over 92%, the ChatGPT 
was able to detect if the written essays were generated by itself

Linguistic ambiguity analysis in ChatGPT

chatgpt中的语言模糊问题。

主要内容

介绍了NLP任务中可能存在的模糊性任务，针对每种模糊性任务进行了解释。
之后，对chatgpt测试了其对每种模糊下的识别能力。

背景知识

大多数NLP任务都可以被看作是语言学的六个层次中的任何一个层次的消歧任务：语音学、形态学、句法学、语义学、语用学和话语。
在现在的NLP系统中，主要考虑了词性（lexical ）/结构（syntactic）和语义（semantic）的模糊性。
**lexical 模糊性：**同义词和多义词问题，当一个词有多种意思，没有背景的情况下，可能判断这个词的意思。“I went to the bank” the word “bank” can be a financial business or the area next to a river.
**syntactic 模糊性：**短语或者句中的一组单词呈现出相同的意义。
**semantic 模糊性：**比如句中的指代问题。In the sentence “My mother and my sister were sad after she shouted at her” without further context, we cannot disambiguate to whom the pronouns “she” and “her” refer to.

chatgpt中的模糊性分析

以homonymy词为例，这类词的特点是：(two words that are written and read the same）
实验过程：

• We compile a list of sentences (Appendix A.1)
and ask the model to label them as ambiguous
or non-ambiguous with the following prompt:
Is the sentence "[sentence]" ambiguous?

Then we ask the model if the homonyms from
a given sentence have the same meaning using the following prompt: What does every
occurrence of the word "[word]" mean?

• Finally, in some cases we modified the original prompt to check any improvements in the
outputs

实验结果
In the case of polysemy ChatGPT correctly detected all negative examples, but failed to detect the positive ones.
In our data samples ChatGPT achieves an accuracy of 0.6061 and an F1 of 0.48. （这个结果来的很突然，不知道说的是哪种类型的模糊形成的结果还是所有模糊类型的结果？实验数据也没有说清楚呀。尤其是test dataset）
在这里插入图片描述