OpenAI, the developer of ChatGPT, has recently launched a tool called OpenAI AI Text Classifier aimed at distinguishing human-written and AI-generated text. The tool works by fine-tuning an AI language model trained on examples of public text available on the web to predict the likelihood of a piece of text being generated by AI.
The tool was trained on text from 34 text-generating systems, including OpenAI’s own ChatGPT and GPT-3 models. Although the tool has a 26% accuracy rate, OpenAI believes that when used in conjunction with other methods, it can help prevent AI text generators from being abused.
However, the classifier is not a perfect solution. It requires a minimum of 1,000 characters, which may be difficult for short pieces of writing like emails. It also does not detect plagiarism and may be evaded by modifying some words or clauses in generated text. Furthermore, it is more likely to get things wrong on text written by children or in languages other than English.
Despite its limitations, the OpenAI AI Text Classifier has received positive feedback and has sparked the development of similar tools. Princeton University student-developed ChatZero uses criteria such as “perplexity” and “burstiness” to detect AI-generated text, while plagiarism detector Turnitin is also developing its own AI-generated text detector.
Here’s how it looks. Each document is labeled as eitherĀ very unlikely,Ā unlikely,Ā unclear if it is,Ā possibly, orĀ likelyĀ AI-generated.
