Science

3072 readers

1 users here now

General discussions about "science" itself

Be sure to also check out these other Fediverse science communities:

https://lemmy.ml/c/science

https://beehaw.org/c/science

founded 2 years ago

MODERATORS

Sal@mander.xyz

fossilesque@mander.xyz

‘ChatGPT detector’ catches AI-generated papers with unprecedented accuracy (www.nature.com)

submitted 10 months ago by throws_lemy@lemmy.nz to c/science@mander.xyz

9 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] Doombot1@lemmy.one 11 points 10 months ago (1 children)

Interesting that the article ends with “The new ChatGPT catcher even performed well with introductions from journals it wasn’t trained on”. Isn’t that the whole point? If you just judge a model based on what it was trained on, you just get a biased model. I can’t remember the exact word for it but it’s essentially over-relying on your own dataset. So of course it will get near-100% accuracy on what it was trained with. I’d be curious to see what the accuracy on other papers is.

[–] MagosInformaticus@sopuli.xyz 4 points 10 months ago (1 children)

Overfitting is the normal term.

[–] Doombot1@lemmy.one 1 points 10 months ago

There we go, thanks for the addition! I did a lot of ML/DL stuff about 2 years ago but just couldn’t remember the term.