What Boards Need to KNOW and DO About the Latest AI Threats

Advances in AI are making cyber attacks faster, more effective, and easier to execute. 

In partnership with the Australian Institute of Company Directors, the Australian Signals Directorate recommends the following key actions:

• Review how advances in frontier AI models could affect your organisation’s cyber security posture, including new AI-enabled cyber threats.

  
  

• Assess the risk of relying on AI providers, including cyber supply chain risks and concerns about foreign ownership, control and influence.

  
  

• Make sure your organisation is prepared to prevent, detect, respond to and recover from cyber security incidents in an agentic AI-driven threat environment.

  
  

• Strengthen cyber security by improving governance, processes, and capabilities to manage AI use within your organisation.

  
  

What do boards need to know?

Cyber threats are rapidly evolving due to the constant advances of “frontier” AI models. Frontier AI models are typically characterised by advanced reasoning, software development capabilities, natural language processing or multimodal capabilities, and are trained at scale using state-of-the-art techniques. Such AI models can perform complex tasks that exceed the capabilities of conventional automated tools. Frontier AI models have been widely available since early 2023 and continue to rapidly evolve.

These AI models can:

• identify vulnerabilities and rapidly weaponise them;

  
  

• chain together multiple low severity vulnerabilities into high-impact compromises; and

  
  

• perform malicious cyber activities with little to no human oversight.

  
  

Newer AI models are also dramatically increasing the speed, scale and ease at which vulnerabilities and weaknesses in systems and cyber supply chains can be identified and exploited, including by malicious actors that previously lacked the knowledge or skills to perform such activities. Combined with malicious actors exploiting weaknesses in proprietary AI model guardrails and illegally distilling such AI models, these developments enable broader access to advanced capabilities and amplify traditional attacker-defender asymmetries. This may rapidly invalidate organisations’ current risk tolerance. Overall, AI models are currently, and will continue to, fundamentally transform both offensive and defensive cyber capabilities.

Boards should also be conscious that the implications of AI models extend beyond their effect on cyber security operations. For example, boards should consider cyber supply chain risks associated with their reliance on particular AI vendors and the foreign ownership, control or influence risks this may pose.

These dynamics have clear implications for how boards should oversee proactive steps to enhance cyber security and resilience within their organisations. Organisations that fail to act now are exposing their organisations to greater business risk and losses.

What threshold questions should boards ask?

Boards should quiz management on the following:

• How vulnerable is our organisation to AI-enabled attacks?

  
  

• What assumptions underpin our current risk assessments, and how might AI threats invalidate those assumptions?

  
  

• Have we reviewed our risk tolerance in light of AI threats and is that risk tolerance still appropriate?

  
  

• If AI models were used to identify and exploit weaknesses across our organisation, what areas of our business would be most exposed?

  
  

• Where could minor weaknesses in our systems and cyber supply chain be combined into a major cyber security incident?

  
  

• Are there known weaknesses in our systems that could become materially more dangerous if identified and exploited at machine speed?

  
  

• Are we relying on vendors and service providers without sufficient governance and oversight, including an understanding of their foreign ownership, control or influence?

  
  

• Do we have visibility of the security and resilience posture of third and fourth-party suppliers within our cyber supply chain?

  
  

• Do we have control over our cyber security fundamentals, such as adherence with a recognised cyber security framework?

  
  

• What legacy technology risks are we carrying and do we have a plan to remediate these risks?

  
  

• Are we delaying addressing any weaknesses due to a perceived low exposure or severity?

  
  

• Would our cyber security operations remain effective if cyber attacks became more frequent, more targeted and more automated?

  
  

• Can we keep the business operational during a serious cyber security incident?

  
  

• If one of our critical systems was compromised, could we continue operating our most important services?

  
  

• If attacks moved from taking days to hours, could we still detect and respond to them effectively?

  
  

• Have our incident response and business continuity plans been updated and tested to capture AI threats?

  
  

What should boards focus on?

AI has the potential to significantly reduce vulnerability discovery and exploitation timelines from days to hours while significantly lowering the skill and knowledge barrier for malicious actors. Organisations should enhance their cyber security fundamentals now and boards should oversee and challenge management on how their organisations are implementing the following strategies. 

Immediate priorities

• Secure attack surfaces: Systems are securely configured to approved and maintained baselines to reduce system exposure, including by reducing attack surfaces and attack paths, with configurations continually monitored and consistently enforced.

  
  

• Reduce software vulnerabilities: Vulnerabilities in systems are identified, documented, validated and prioritised for remediation or mitigation in a timely manner, with all remediation and mitigation actions verified for effectiveness.

  
  

Short-term priorities

• Replace legacy systems: Systems that aren’t capable of meeting cyber security requirements are managed using compensating controls, along with enhanced monitoring and assurance activities, to maintain an acceptable level of residual risk until they can be decommissioned or replaced.

  
  

• Reinforce identity, credential and access management: Robust and secure identity, credential and access management is used to establish, maintain and control access to systems and to support effective detection of identity and credential misuse.

  
  

• Restrict unnecessary privileges: Personnel and services, including AI agents, are granted the minimum access to systems required to undertake their duties.

  
  

• Prepare for cyber security incidents: Cyber security incident response, business continuity and disaster recovery plans for systems support continued business operations during cyber security incidents, and the resumption of normal business operations following cyber security incidents.

  
  

Medium-term priorities

• Adopt AI for cyber defence purposes: Artificial intelligence is used for cyber defence and is secure, controllable, human-supervised and used in an ethical and accountable manner.

  
  

Longer-term priorities

• Modernise for the future: Systems are planned, designed, developed, tested, deployed, maintained and decommissioned according to business criticality ratings and security and resilience requirements using Secure by Design and Secure by Default principles and practices.

  
  

What practical actions can boards oversee?

Boards are encouraged to engage with, and where appropriate challenge, management on their implementation as their organisation develops its response to AI threats. This should include regular reporting and assurance.

Secure attack surfaces

• Define approved configuration baselines that disable or remove unnecessary services, insecure settings, and unsupported or weak communication protocols.

  
  

• Deploy approved configuration baselines on systems, including infrastructure, operating systems and applications.

  
  

• Detect and respond to configuration drift from approved configuration baselines through automation or continuous technical assessments.

  
  

Reduce software vulnerabilities

• Identify known software and configuration weaknesses of assets through continuous and repeatable assessment activities, including vulnerability scanning activities.

  
  

• Apply security patches or updates within defined risk-based timeframes or, where operational impact is low, automatically.

  
  

• Remediate or mitigate vulnerabilities exceeding defined risk thresholds within defined risk-based timeframes based on their severity and exposure.

  
  

Replace legacy systems

• Establish and maintain a process for identifying, assessing and managing the cyber security risk associated with legacy systems until they can be removed or replaced.

  
  

Reinforce identity, credential and access management

• Identify and disable or remove default and unused accounts on a regular basis.

  
  

• Store authentication keys, secrets and tokens in approved secure repositories with access controls, including multi-factor authentication.

  
  

• Regularly scan codebases, configuration files, file shares and user collaboration platforms to identify unsecured keys, secrets and credentials.

  
  

• Use phishing-resistant multi-factor authentication for users authenticating to systems, including from untrusted, external or high-risk locations.

  
  

• Disable legacy authentication protocols that bypass or weaken modern authentication controls.

  
  

• Ensure that user, machine, system and service credentials are unique per system or account, particularly in high-privilege contexts.

  
  

Restrict unnecessary privileges

• Periodically review and update access privileges in alignment with current roles, duties and business requirements.

  
  

• Restrict permissions to create, modify or delete accounts, roles or access to privileged functions or applications to authorised administrative roles.

  
  

• Provision user, administrative and non-person accounts with only the minimum access and privileges required to perform defined operational functions.

  
  

Prepare for cyber security incidents

• Review cyber security incident response, business continuity and disaster recovery plans to ensure they remain fit for purpose.

  
  

• Regularly exercise cyber security incident response, business continuity and disaster recovery plans.

  
  

Adopt AI for cyber defence purposes

• Deploy suitable AI models for augmenting software development activities, such as identifying and remediating vulnerabilities and weaknesses for software before its release and periodically throughout its life.

  
  

• Deploy suitable AI models for augmenting security assessment activities, such as performing vulnerability scanning and vulnerability assessments.

• 
  

• Deploy suitable AI models for augmenting security monitoring activities, such as identifying and triaging cyber security events.

• 
  

Modernise for the future

• Design systems to provide visibility, enforce authorised access, protect data, minimise required and default privileges, limit exposed services, and remove unnecessary functionality.

  
  

• Secure build and deployment processes to enforce controls consistently across systems.

  
  

How can MFT help mitigate the AI threat?

Managed File Transfer (MFT) mitigates AI cyber risks by imposing strict governance over how sensitive data moves, shutting down the very leakage paths that AI‑enhanced attackers exploit. 

By centralising all file flows, MFT prevents shadow‑AI behaviours - such as staff feeding regulated data into external LLMs - through granular access controls, encryption, and content inspection that blocks sensitive prompts before they reach uncontrolled endpoints. 

While AI adversaries can discover vulnerabilities and weaponise exploits at machine speed, MFT counters this by enforcing hardened, vendor‑supported transfer channels, rapid patching, and continuous anomaly detection that flags abnormal transfer patterns and stops malicious exfiltration in real time. Critically, MFT maintains immutable audit trails for all AI‑related data flows, giving organisations forensic visibility and compliance evidence as AI models become autonomous actors in enterprise systems

Here to Help

At Generic Systems Australia, we are the nation’s most experienced and dedicated provider of the industry’s leading MFT solution, GoAnywhere.  We’re right here and on hand locally to help your organisation protect itself against malicious AI incursions.

As the ASD recommends: don’t leave yourself exposed to the risk from AI. Boards should press management to act now to ensure their organisation does not fall victim to current and emerging AI threats, including by supporting targeted investments to enhance cyber security and resilience.

Acknowledgement: this article substantially reproduces (and expands upon) excellent advice provided by the ASD in partnership with the AICD.