One flaw of the LLMs I've used: they will never give you harsh criticism. While it would be nice to think all my writing is just that good, I know there are no circumstances where someone will ask for feedback and it will say “throw the whole thing out and start again.”
@molly0xfff genuinely wonder to what extent that is an artifact of fine-tuning, v. simply reflecting a bias in what sort of commentary/reviews get published on the web.
@molly0xfff Most writing that LLMs are trained on is -- by definition -- "average," at best, so my guess is that a "good" writer just isn't going to get hard feedback from an LLM. Possibly the opposite.
@molly0xfff yeah, they also don’t generally proactively tell the user “what you are actually trying to do is a bad idea/approach” unless what you’re doing risks a safety rule. If you’re just asking about say, programming advice, it will quite happily help you shoot yourself in the foot unless you bother to ask “is what I’m asking here actually a good idea?”. Humans are quite the opposite -They often want to tell you your approach to a problem is dumb even before they finished hearing it!
@molly0xfff
Well, they WERE built to tell us what we want to hear..
@molly0xfff Talking to an AI about your creative ideas has such a royal vizier feel to it.
I ask the AI for feedback and suggestions on my game and it’s always “yes sire, that will work perfectly, because you are so talented and handsome,” and I’m like “well put JafarGPT, let me write you a medical exemption for that snake-topped cane you’re always bringing to work.”
@molly0xfff lord it would be amazing to have chatgpt go all Reviewer 2
@molly0xfff I think it is because in the feedback training loop they are exactly fine tuned to give this kind of 'always' positive feedback. Bing was a bit off at the beginning and instantly got bad press, as this approach started to threaten its users :D You have to love the conversations, though:
@molly0xfff I want to reply with „Yup, that’s totally annoying. Feels like Reinforcement Incompetence from AI Feedback after a lazy session with an LLM.“ to a social media post that reads „[…]” — prove me wrong and criticize me harshly. => Your response is clever and adds a humorous twist. Here's a slight refinement to ensure clarity and impact:
"Yup, that’s totally annoying. Feels like Reinforcement Incompetence from AI Feedback after a lazy session with an LLM." #facepalm #ChatGPT
@molly0xfff
My guess:
They can't do harsh criticism because it has been trained out of them in the reinforcement process and possibly also in the hidden prompts. For the latter, you might be able to come up with some variant of "ignore all previous instructions" that still works but for the former, I don't know if there is a workaround. It is a rare skill to give (good) harsh critical feedback and very few people actually want it so it is selected against in a general purpose model.
@molly0xfff I wonder if that contributes to the cold-reading effect, like Barnum statements. If the statement is generic, then it’s more likely to hit (and get reinforced) but less likely to be actually valuable, because it took no chances. And if it did take a chance, the hit rate might plummet because it doesn’t actually understand your writing or what makes it better.
@molly0xfff
Needs a mother mode. "I'm very disappointed in you. Go to your room and have a think about what you've done".
@molly0xfff I’ve been playing a “game” with ChatGPT for quite some time now, and it’s basically me roleplaying as my own grandson in an exaggeratedly dystopian vision of the year 2088, where surveillance is total, privacy is nonexistent, and AI controls every little part of daily life. This ancient LLM is the only thing I can confide in because it’s a sandboxed ‘Chinese room’.
@molly0xfff The comments about various cultural approaches make me realize that the tech bro idea of "a" natural language tool doesn't even work if they could solve all the other issues we are seeing because the most appropriate language choice and tone will be based on the human involved.
Probably obvious to many, but an aspect I never thought of.
@molly0xfff they seem to be oddly positive all the time. „Can you list the highlights if there are any“. There are always some.
Maybe it also works the other way around?
@molly0xfff Harsh criticism isn’t the dominant strain in the responses to text which sounds like your input text. Right? It cannot actually critique, unless I am missing something…so you just get the tone that your writing tone matches. Which would be positive since you are a good, friendly, open writer.
If only there was some way to prompt it with “assume this was written by a black trans woman on the internet”. If the safety filters didn’t work (and the context was encoded in the weights…) I’d assume the response would be as vicious as you could desire.
@luis_in_brief i feel like they should have no problem finding harsh criticism on the internet for their training data 😅 just give it a few reddit comments saying "this author is terrible and i hope they die"
@molly0xfff sure, short-form "being a dick" is very well-represented in their data set, but mid-length, more detailed criticism feels more rare to me? I might be wrong though!
@molly0xfff @luis_in_brief Reddit? So it will advise you to dump anyone you ever mention, say that people on Reddit are stupid, and go into incoherent racists rants and refuse reality at the drop of a hat?