Identifying 🤖 AI Articles?

Generative AI articles seem to be common now and detecting them can be difficult. I even found an AI-generated article about detecting AI-generated articles. It was created by an AI-based SEO service, which I am not linking it because I do not think it should receive more SEO benefit. This article is mostly my thoughts on using and identifying AI content, but I’m very interested to hear how others detect/determine AI generated content.

A Reminder

Dev’s Code of Conduct includes a section on AI-assisted articles which every poster should be familiar with and strive to adhere to.

This article contains AI-generated content, which is specifically called out at the end.

Risk vs. Reward

Generative AI is an easy way to build a portfolio of work and get community involvement. Using AI to help structure an article or flesh out an argument can speed up content creation and communication, but leaning on AI to provide information which we do not know comes with risks. Unknowingly disseminating outdated or incorrect information can lead others to question our capability or competency. In an online article it might be shrugged off by the masses, but there can be consequences to using AI. Still, the allure is there.

If we do use AI content, I think it is important to be forthcoming about that use. Using a spell checker is standard. Microsoft Word has had a grammar check since Microsoft Word in the 1990’s, and leveraging this new tool to improve our writing is fine, but misusing it can be unethical or harmful.

🔍 Detection Tools

There are tools that attempt to identify generated content, but they are not perfect. Even OpenAI shut down their own AI-generated content detector for low reliability.

Tools like QuillBot’s AI Detector will attempt to identify generated content, but they are not perfect and adding or removing a sentence can make the calculation shift by tens of percent. In fact, QuillBot considered this paragraph to be AI-written when I fed it part of this article. While this article contains some AI-generated content, it is specifically called out, below.

Common AI Patterns

One repeated feature I’ve found is the “listicle”. These were popular before generative AI, but they are much easier to produce, now. Articles containing numbered entries, each with a single sentence or short paragraph, alert me to be on the lookout for the possibility of generated content. When there is a lack of deeper information about the details, I tend to stop reading. It’s occasional that a surface-read of a topic is useful, but it’s easy to get that from a fleshed out article, where a generated high-level summary offers nothing to build on for the parts which interest you.

Once I suspect a generated article, I move on to looking for links to source material. If there are links, are they to relevant and authoritative sources? I see generated content used for SEO and link-farming which typically contain one or two links back to a site/company, often with a very SEO-focused URLs.

If you suspect a specific piece of content uses generative AI, it could be relevant to compare the voice/style to other content known to come from the same author.

While outside the scope of this article, I recall reading about using Markov chains to help identify and validate authorship of content in the early 2000s. I could not readily locate the article I read back then, but I did find abstracts of research papers from 2002 and 2006, on the off chance anyone is interested in this wild tangent.

📢🤖 The “Generative AI Voice”

By “voice” I mean the wording and tone of the article. While it can be hard to articulate, some wording can trigger an “AI Alarm” in my head. Providing unexpected alternatives is one that I recognize. If an article presents a specific solution, offering alternatives later on could be thoroughness, but it often comes off feeling like an unnatural shift of direction.

Summary

So there are a few obvious signs, like shallow listicles lacking links or detail, and long “authoritative” content with no sources.

What tips can you share about identifying AI-generated content?
Are there specific things that set off your “AI Alarm”?

🧐 Reference Content

I’ve asked ChatGPT to produce some articles on how to identify AI-generated articles. These examples are included below. Some of the suggestions may be helpful, but for me the more important question is about the tone or pattern. Does the style of these example articles seem familiar? I’ve added emphasis in a few places that seemed relevant to me. I hope you enjoy!

AI generated content below this line (emphasis mine)

How to Identify Articles Written by AI

As artificial intelligence (AI) continues to advance, distinguishing between human-written and AI-generated content has become increasingly challenging. However, there are several indicators you can look for to help identify articles written by AI:

1. Consistency in Tone and Style

AI-generated content often exhibits a highly consistent tone and writing style throughout the article. While human writers naturally vary their tone and style, AI tends to maintain a uniform approach, which can sometimes feel mechanical or overly formal.

2. Repetition and Redundancy

AI models may produce content that frequently repeats phrases or ideas. This redundancy occurs because the AI is trying to ensure it covers the topic thoroughly, but it can result in repetitive language that feels unnatural.

3. Lack of Deep Insight or Personal Touch

AI lacks personal experiences and emotions, which means it often produces content that lacks deep insight or a personal touch. Human writers typically infuse their work with anecdotes, opinions, and unique perspectives that AI cannot replicate.

4. Overly Perfect Grammar and Syntax

While human writers make occasional grammatical errors or typos, AI-generated content tends to have perfect grammar and syntax. This precision can be a giveaway, especially if the text lacks the usual small mistakes found in human writing.

5. Pattern Recognition

Experienced readers may notice certain patterns in AI-generated content. For example, AI might use specific sentence structures or transitional phrases more frequently than a human writer would.

6. Absence of Complex Emotions

AI struggles with conveying complex emotions and nuanced opinions. Articles written by AI may lack the depth and emotional range that a human writer can provide, making them feel flat or unemotional.

7. Fact-Checking Inconsistencies

While AI can generate accurate information, it may also produce content with factual inconsistencies or outdated information. Human writers typically cross-reference sources and update their knowledge base more effectively.

8. Use of Generic Examples

AI often uses generic examples and scenarios that lack specificity. Human writers, on the other hand, tend to provide more detailed and context-rich examples based on their experiences and research.

Conclusion

Identifying AI-generated content requires a keen eye for detail and an understanding of the nuances in writing styles. By paying attention to consistency, repetition, insight, grammar, patterns, emotional depth, factual accuracy, and specificity, you can better discern whether an article was written by a human or an AI.

As AI technology continues to improve, these distinctions may become less apparent, making it even more crucial to stay informed about the capabilities and limitations of AI in content creation.

How to Identify AI-Written Articles

Distinguishing between human and AI-generated content is becoming harder. Here are key indicators:

1. Consistent Tone and Style

AI content often has a uniform tone and style, which can feel mechanical.

2. Repetition

AI may repeat phrases or ideas to cover a topic thoroughly, resulting in unnatural repetition.

3. Lack of Insight

AI lacks personal experiences, leading to content without deep insight or a personal touch.

4. Perfect Grammar

AI-generated text usually has flawless grammar and syntax, unlike human writing with occasional errors.

5. Recognizable Patterns

AI often uses specific sentence structures or transitional phrases more frequently.

6. No Complex Emotions

AI struggles with conveying complex emotions, making the content feel flat.

7. Fact-Checking Issues

AI may produce content with factual inconsistencies or outdated information.

8. Generic Examples

AI tends to use generic examples that lack specificity.

Summary

To identify AI-written articles, look for consistency, repetition, lack of insight, perfect grammar, recognizable patterns, absence of complex emotions, fact-checking issues, and generic examples. As AI advances, these distinctions may blur, so staying informed is crucial.