[HN Gopher] Bypassing Gmail's spam filters with ChatGPT
___________________________________________________________________
Bypassing Gmail's spam filters with ChatGPT
Author : neelc
Score : 83 points
Date : 2023-01-22 17:18 UTC (5 hours ago)
(HTM) web link (neelc.org)
(TXT) w3m dump (neelc.org)
| wolverine876 wrote:
| Think what can be done with an AI trained on all the data that's
| been collected about you.
| b1n wrote:
| Spear-fishing now scales.
| Tagbert wrote:
| AFAIK all it takes to bypass Gmails spam filters is to resend the
| same scammy email that was Flagged but using a different email
| address. I get the same kind of scammy emails, flag them as spam,
| and then a couple days later, Gmail lets the same email in though
| it is coming from a different 123134r12345124@blahblah.com
| address.
| orlp wrote:
| In my experience the new strategy is to just send an email with
| no body, no title, optionally with an image of a conventionally
| attractive lady, in the hope people respond after which they're
| automatically whitelisted (as there's now a 'conversation').
| mfi wrote:
| > I'm no AI or machine learning expert so I don't know how it
| works. But I am also worried that spammers could use ChatGPT to
| get around Gmail and Outlook's spam filters.
|
| This will not only increase the spam-problem, but will most
| likely be used to scale and do targeted phishing attack as well.
| I wrote an extensive article[0] where I analyzed this. And to no
| surprise, GPT-3 can be used to generate dynamic phishing
| campaigns on the fly in multiple languages, classify email
| responses, improve email thread hijacking attacks etc.
|
| [0] https://www.xorlab.com/en/blog/why-ai-powered-phishing-
| will-...
| [deleted]
| shadowgovt wrote:
| On the other hand, ChatGPT becomes a very convenient tool for
| generating example data to train the ML component of the spam
| filter on.
| aitball wrote:
| something.. something... arms race
| barking_biscuit wrote:
| something... something... technology is not neutral.
| HyperSane wrote:
| Maybe the Amish were right after all.
| steveBK123 wrote:
| Interpretations I've seen of ChatGPT type tools, and playing
| around with it, I would sum up as "reducing the marginal cost of
| creating BS to $0". Great for content farms, spam,
| disinfo/propaganda campaigns.
|
| Stuff where it doesn't have to be correct, have a high hit rate,
| or even be edited. Just need to produce plausible enough
| sounding, human-like content.
| sirsinsalot wrote:
| If we thought the signal vs noise problem was bad, wait until 90%
| of data is banal AI drivel drowning out any semblance of
| authenticity.
|
| Boring, brown, homogeneous noise
| jeffbee wrote:
| People are overestimating the importance of the message body in
| spam classification. The stuff that appears in your spam label on
| gmail is what google considered marginal, almost ham. The vast,
| vast majority of what they think is spam is rejected with
| temporary failure codes at SMTP time and never gets delivered
| with any label. IP reputation and other related metadata features
| are the key features in spam classification, and repeatedly
| sending different messages is not a valid test of whether the
| body looks spammy or not.
| walrus01 wrote:
| I concur on this for the most part because I would say that my
| custom postfix + spamassassin + opendkim setup, on my self run
| MX, correctly classifies 75%+ of the spam or outright rejects
| it for SMTP transfer just based on:
|
| a) invalid rdns of other mx
|
| b) invalid spf
|
| c) invalid DKIM / no DKIM signature
|
| d) failed RBL list check - I subscribe to and feed it a few
| different common sense SMTP RBLs
|
| Rejecting as spam things in the above category before it even
| looks at the content.
|
| Adding a high score for invalid rdns, spf or dkim before
| something generally similar to spamassassin or a more advanced
| message subject line/body analyzing system begins classifying
| things help.
|
| And then additional score is added of course for text spam
| content in message subject line and body.
| topicseed wrote:
| > and other related metadata features
|
| Such as? Actually curious and doing a lot of sales myself, I'm
| interested!
| jeffbee wrote:
| Every email operator considers their classification features
| to be trade secrets. The closest you will get to advice from
| Google on this topic is
| https://support.google.com/mail/answer/81126?hl=en
|
| But anyway it sounds like you intend to send spam. I
| recommend doing literally anything else.
| labria wrote:
| The last line reads like something Clippy would say :)
| DwnVoteHoneyPot wrote:
| Yeah, I'm starting to hear and understand that more myself.
| There was a extremely long twitter thread (i think from former
| Reddit CEO) that said the key to content moderation is
| moderating bad behaviour, not bad content.
| natch wrote:
| I don't think ChatGPT is really needed here.
|
| Gmail's spam filters are Google's weakest tech. At least I don't
| know of anything worse.
| teruakohatu wrote:
| > Gmail's spam filters are Google's weakest tech. At least I
| don't know of anything worse.
|
| I get a fair amount obvious spam coming through the filter, but
| the issue with any sort of classification is the tradeoff
| between False Positives and False Negatives.
|
| The occasional False Negative causes a lot less damage (2
| seconds to delete or report as spam) verses the damage of a
| False Positive (not seeing an important email for two weeks or
| ever).
| 4g wrote:
| Using these tools to mitigate spam is as likely a scenario as
| this, imagine that every spam mail receives a masterfully crafted
| response showing utter fascination and interest in SEO, or
| helping out a Nigerian prince. Every phone call to an
| unregistered number is answered by an artificial, frail, and
| forgetful lady that is trying her best to register gift cards.
|
| When reporting an e-mail as spam it will not only block the
| address but waste the spammers time, rendering the actions
| unprofitable.
| Waterluvian wrote:
| Oh my god a future where you are never ever quite sure if your
| online circle of friends are human or not.
| InCityDreams wrote:
| And you're confident right now?
| ben_w wrote:
| My online friends circle includes three dragons, two
| coyotes, a raven, a squirrel, and a wasp.
|
| All totally real. But then, I am a furry, I've met many of
| them IRL also.
| Waterluvian wrote:
| I'd be okay with this option too. As long as people are
| who they claim to be and it doesn't turn out that I'm in
| some horrible non-VR Matrix.
| Waterluvian wrote:
| I... but... oh no.
| jimkleiber wrote:
| I wonder if that inspires new platforms or types of tech that
| verify someone was typing it in vs pasting. But then do bots
| get better at typing it into the input boxes? Ugh.
| kibwen wrote:
| At the same time, it's not necessarily pleasant to consider the
| prospect of an internet where 99.9% of traffic is generated by
| AI-powered spambots engaged in adversarial games with AI-
| powered anti-spambots.
| sharkweek wrote:
| But it does make for an interesting idea as a plot device in
| speculative fiction!
| atorodius wrote:
| I thought we are already nearly there? I remember reading
| 90%+ of emails are spam and this was a while back
| ThrowawayTestr wrote:
| Honestly the most likely source of computers becoming self-
| aware.
| nwellnhof wrote:
| That's basically the plot of the novel Avogadro Corp by
| William Hertling.
| ben_w wrote:
| And a plausible way for them -- on either team -- to take
| over completely. Both treating us not even as pets but as
| grass. In this analogy the good AI are gardeners, the bad
| AI are cow farmers.
| [deleted]
| honkler wrote:
| dead internet theory:
| https://dailyfreepress.com/2021/10/21/the-dead-internet-
| theo...
| 5e92cb50239222b wrote:
| If as a (hypothetical) Nigerian prince spammer I get a
| "masterfully crafted response" from a mark, it would be obvious
| right away that I am talking to a bot. The kind of people who
| respond to such bait would hardly be able to write anything
| like that.
| Kamq wrote:
| A "masterfully crafted" response to a nigerian prince spammer
| would probably sound a lot like a person who can barely
| write, possibly with what sounds like the beginning of
| dementia setting in.
| jahewson wrote:
| Fun idea but impersonating someone else, especially your
| customer, sounds like a way to land in hot water. Also LLMs are
| not exactly cheap.
| miketery wrote:
| I think for next few years that will be cost prohibitive for
| 95% of Americans.
| rglullis wrote:
| Already a reality for phone spammers/scammers:
| https://jollyrogertelephone.com/
| mimimi31 wrote:
| >imagine that every spam mail receives a masterfully crafted
| response showing utter fascination and interest in SEO
|
| That would necessitate reliably detecting the emails as spam in
| the first place though. False positives in particular could be
| devastating. Imagine a chat bot coming up or going along with
| business proposals in your name for example.
| crummy wrote:
| You could do it with humans - every time you click "mark as
| spam" it doesn't just trash the email, it begins a long and
| drawn-out chatGPT conversation with the spammer, stringing
| them along.
| jawr wrote:
| I'm pretty sure most spam senders black hole any response,
| the money is in the target clicking a link and no where
| else.
| crummy wrote:
| Don't most of the "nigerian-prince" type scams involve
| some kind of back-and-forth?
| just_boost_it wrote:
| There's lots of legitimate email traffic that would find itself
| stuck in here. I could see business questions being answered
| and those answers actioned on. Or legitimate sales prospecting
| resulting in actual orders being placed. If you choose to let a
| tool do your communication for you by impersonating you to the
| extent that another person would reasonably expect that they're
| talking to you, then I'm not sure you can just say "lol, that
| just was my spam bot" as a way of getting out of it.
| walrus01 wrote:
| Sure, turn loose these tools to answer the actual spammers/UCE.
| But:
|
| Speaking as an ISP, if somebody turns loose what is clearly an
| AUTOMATED tool shitting up the contents of my abuse@ispname.com
| inbox with reports from some software script, I can guarantee
| you it goes to /dev/null
|
| At some point we will just block their MX at the SMTP transfer
| point and call it a day.
|
| 98% of that already is abusive DMCA rights holders who are
| ignoring our federally designated DMCA-agent address for
| copyright violation complaints. With their automated 3rd party
| things complaining about people torrenting Yellowstone or
| whatever.
|
| Actual reports that are clearly written by a human saying "hey
| it looks like this /32 of an IP address is compromised as some
| sort of botnet" will get a thousand times more attention. Or
| the very rare cases where we have a network-engineering
| emergency escalation and somebody calls me on the phone.
|
| Anything generated by chatGPT or similar will be clearly
| obvious enough that it matches a similar pattern and comes from
| an automated script.
| awb wrote:
| James Vitech did this, but manually and with humorous results:
|
| https://youtu.be/4o5hSxvN_-s
|
| https://youtu.be/IUjpoauJcKo
| wolverine876 wrote:
| So the two AIs will be talking to each other, trying to suss
| out if the other is fake (a sort of Turing test), trying to con
| the other to keep talking or to really buy in?
| abraae wrote:
| Sounds absurd all right. But what's to prevent this from
| being the future of the internet?
| londons_explore wrote:
| I don't think message content is weighted very heavily in modern
| spam filters...
|
| Things like IP reputation, sender reputation, and various SPF-
| like headers are far more important.
| narrator wrote:
| As I have said before, the future will have two kinds of AI
| everywhere.
|
| _Their AI_ to get you to buy something, do something, believe
| something, or in a warzone to kill you and _Your AI_ to protect
| you from _Their AI_. Reality may even become so dangerous and
| illusory that humans lose a lot of their agency to _Your AI_.
| thyrox wrote:
| All this can be done since GPT3 API has been available.
|
| I see a lot of people thinking chatgpt is something new capable
| of such stuff but GPT3 is far less restrictive and has been able
| to do all this for almost an year now.
| puffybuf wrote:
| But you must pay to use these APIs. They did give me $20 free
| trial though. You could make a bunch of accounts and abuse the
| free trial I guess. It must be cost effective for the scammers.
|
| You can change up the prompt to change the writing style so
| spam filters will have trouble catching this new world of spam.
| dhruval wrote:
| ChatGPT also throttles the number of requests an hour and has
| various measures to prevent bots (though not that hard to
| bypass, easier to pay for API)
| marginalia_nu wrote:
| Anecdotally, almost every email I see in my gmail inbox is
| advertisement of some sort. There's newsletters I never signed up
| for, special offers from companies I've never had dealings with,
| it never ends.
|
| Some of it hasn't even been sent to me as an email, but shows up
| in the inbox as though it was an email.
|
| Granted, there's fewer scam emails than in my non-gmail inbox,
| but man is there a lot of spam.
| sorry_outta_gas wrote:
| This is about all its' good for, it won't even write sexy fan
| fiction for me
| Crystalin wrote:
| well, you don't need ChatGPT for that. I receive daily scam in my
| gmail with 99% the same content each time (and i mark it spam
| each time)... the worst part ? the server serving it is google
| (but the email attached to it is always a different obscure
| email)
| jokoon wrote:
| only way to make email work is to let the user choose what domain
| they trust, and maybe to have a feature where the user must
| whitelist addresses.
| Avamander wrote:
| In addition to further emphasis on "trusted senders" in the
| form of contact books, we'll also hopefully see a rise in
| identity-validated S/MIME. Though I get the feeling it'll hurt
| really bad before either gets deployed to a sufficient extent.
| fIREpOK wrote:
| Then everyone would choose "gmail.com" and Google will get its
| true monopoly?
| killingtime74 wrote:
| Give google half a chance and they will make "Gmail" domain
| only opt-out, won't even ask you to choose
| 867-5309 wrote:
| over 90% of the world's smartphones are onboarding with
| gmail
| deno wrote:
| I can't find it but Gmail actually had some sort of
| whitepaper or something once about Gmail-originated SPAM and
| how it became a huge problem which was partially the reason
| they started doing phone verification.
|
| The parent's comment is valid. Any modern email peer is doing
| domain based reputation which is possible thanks to SPF and
| DKIM, and if you don't have those configured you'll have a
| bad time. Then it's the job of the domain owner or email
| operator (postmaster) to make sure you're not blasting out
| SPAM and respond to abuse feedback. If you think about it,
| this is the only sane way for email to function without
| preauthentication.
|
| The only major outlier to this is Outlook, which is still
| doing IP based reputation. And of course a long tail of small
| server operators that rely on legacy SPAM lists from decade
| ago and reject only legitimate emails and pass through plenty
| of Viagra ads.
| [deleted]
___________________________________________________________________
(page generated 2023-01-22 23:00 UTC)