168
Chat GPT Did NOT Like My Memory Test
(beehaw.org)
I was trying to do a memory test to see how far back 3.5 could recall information from previous prompts, but it really doesn't seem to like making pseudorandom seeds. ๐
I don't know why you would expect a pattern-recognition engine to generate pseudo-random seeds, but the reason OpenAI disliked the prompt is that it caused GPT to start repeating itself, and this might cause it to start printing training data verbatim.
Because it literally will. It just clunks out when they get long. The point isn't their randomness, though. The point is for gpt to be able to forget them.
That way I could track roughly how much it can keep track of at once before it forgets.
I can get around protection in chatgpt4 and it will repeat the same word forever and spew random things. The protection is not working the way you described.
the article states that they were using version 3.5 during the study, I'd assume it would be patched in later iterations