For years we told people to watch for typos and bad grammar. AI has made that advice dangerous.
The standard phishing advice — look for spelling mistakes, clumsy grammar, generic greetings — worked because attackers were often careless or writing in a second language. Generative AI has erased all of that. Today's phishing emails are flawless, perfectly on-brand, and personalised. The old red flags are gone, and relying on them now gives a false sense of safety.
What has AI changed about phishing?
AI phishing is phishing that generative AI writes, personalises or even voices, and it changes four things at once: the quality of the writing, the depth of personalisation, the channels attackers can fake, and the sheer volume of campaigns. For Melbourne businesses, the warning signs staff were trained to spot have largely gone.
Perfect writing
AI produces grammatically flawless, professional emails in seconds, in any tone.
Personalisation at scale
Attackers scrape public information, plus details leaked in past data breaches, and have AI craft messages to your name, role, and even recent activity. A free dark web scan shows which staff addresses are already circulating after a breach.
Voice and video
AI voice cloning means a phone call "from the boss" authorising a payment may not be the boss at all. Deepfake phishing against a business tends to target whoever approves payments; our step-by-step look at a voice-cloning CEO scam shows how it unfolds.
Speed
Convincing campaigns that once took skill now take minutes, so there are far more of them.
Why doesn't "spot the fake" work any more?
You cannot reliably tell a good AI-written phishing email from a real one by reading it carefully — that is the whole point. When the content is perfect, scrutinising the wording is no longer a defence. The signal has moved from how the message is written to what it is asking and whether you expected it.
What are the new warning signs of AI phishing?
The new warning signs sit in the request, not the wording: anything unexpected, urgent, or involving money, a login or sensitive data. Teach staff to pause on any message that:
Changes payment details
New bank details on a supplier invoice are the hallmark of a payment redirection scam, and AI now makes the email around them read perfectly.
Applies pressure or secrecy, like "before 3pm" or "keep this between us".
Sends you to a sign-in page through a link or QR code, rather than you going to the site yourself.
Switches channels to "confirm", such as an email followed by a text or call, so the request feels already checked.
Asks you to type in a code to open a shared document, sign something or hear a voicemail. In device code phishing, entering that code on Microsoft's real sign-in page approves the attacker's own device into your Microsoft 365 account, and the Australian Signals Directorate says criminals now use automated systems and AI to request fresh codes the moment someone clicks.
A flawless message can also come from the real, compromised mailbox of someone you know, so the sender name proves nothing; ringing them on a number you already have does.
What actually protects you now?
The controls and habits that do not depend on anyone spotting anything: MFA, out-of-band verification, technical filtering and monitoring, and a culture of checking.
Multi-factor authentication
Even if a perfect email steals a password, MFA stops it being enough to get in. This is the single most important control, and a core part of the Essential Eight. That said, some phishing kits now relay codes through fake login pages in real time, so move towards phishing-resistant passkeys in Microsoft 365, which will not sign in to a lookalike site. Passkeys do not stop the code trick above, though, because the victim signs in on Microsoft's genuine page; a Conditional Access policy that blocks the device code sign-in flow does.
Verify out of band
Any unexpected request involving money, payment details or credentials gets verified by a known phone number — never the contact details in the message.
Email security and monitoring
Technical filtering catches much of it, and a SOC catches the account compromise that follows a successful one. On your own domain, enforced SPF, DKIM and DMARC records make your exact email address much harder to forge in messages to your clients and suppliers, though a lookalike domain one character off still needs filtering and alert staff.
A culture of "check, don't trust."
Staff who feel safe pausing to verify, rather than rushing to comply, are your strongest defence. Key IT's free phishing simulation demo shows, anonymously, how your team would respond to a realistic attack.
What's the mindset shift?
The shift is simple: stop teaching people to spot fakes and start teaching them to verify requests. The question is no longer "does this email look suspicious?" but "am I about to do something involving money or access that I did not expect — and have I confirmed it through a separate channel?" That shift, backed by MFA and monitoring, is what defends a business in the age of AI phishing. It sits at the centre of Key IT's managed cybersecurity work with Melbourne businesses.
Frequently asked questions
What is AI phishing?
AI phishing is a scam email, text or phone call that uses generative AI to write, personalise or voice the message. It turns out polished, on-brand messages and even cloned voices at scale, so the typos and odd phrasing that once gave phishing away are gone.
Why are phishing emails harder to spot now?
AI writes flawless, personalised emails, removing the typos and clumsy grammar that used to give phishing away.
Does the old "check for spelling mistakes" advice still work?
No. AI-written phishing has perfect spelling and grammar. Focus instead on whether a request is expected and verify it independently.
What is the best protection against AI phishing?
Multi-factor authentication, out-of-band verification of money and credential requests, email security, and monitoring — together.
How can you tell if a phishing email was written by AI?
Usually you cannot, and you do not need to. Judge the request instead: is it unexpected, urgent, or about money, logins or bank details? If so, verify it through a phone number you already hold, never the contact details in the message.
Does phishing awareness training still work against AI-generated phishing?
Yes, if it teaches the right habit. Drilling typo-spotting is out of date; teaching staff to verify unexpected requests and report anything suspicious quickly still works, and regular simulated phishing keeps that habit fresh.
Can attackers fake a phone call from my boss?
AI voice cloning makes this possible. Verify unusual requests through a known, separate channel, even if the voice sounds right.
What should you do if someone entered their password on a phishing page?
Treat the account as compromised, even with MFA switched on, because some phishing kits relay the real sign-in and keep the session. If money has moved, ring your bank first. Then, from a clean device, block sign-in, reset the password, sign out every session, and remove any MFA methods, app consents and inbox rules the attacker added. Our first-hour checklist for a hacked Microsoft 365 account walks through each step and who to tell in Australia.
Would your team spot the next one? A free phishing simulation demo shows, safely, how many of your team would click.
Request the Free Demo