Introduction

As organisations deploy self-hosted LLMs, RAG systems, and agentic AI, they create new security risks. Learn how practical training, SIEM monitoring, and SOAR response can help protect enterprise AI.

Artificial intelligence is moving beyond isolated experiments.

Large Language Models are now being used in customer support, internal knowledge bases, software development, document processing, security operations, and business decision-making.

At the same time, many organisations are testing agentic AI. Unlike a traditional chatbot, an AI agent can retrieve information, use tools, access applications, and complete multi-step tasks.

This shift creates clear business value. However, it also changes the security landscape.

Once an LLM connects to internal documents, databases, APIs, email, cloud services, or automation tools, it becomes more than a chatbot. It becomes part of the organisation’s infrastructure.

Therefore, it also becomes part of the attack surface.

For security teams, the challenge is no longer deciding whether LLM security matters. Instead, teams must build the controls, visibility, testing processes, and response capabilities needed to protect AI systems before those systems receive access to sensitive data or business-critical actions.

Self-Hosted LLMs Do Not Automatically Provide Better Security

Many organisations are exploring private or self-hosted LLM deployments. This approach can offer more control over sensitive data, model customisation, operating costs, availability, and data residency. In addition, a private deployment may reduce the amount of information shared with an external AI provider. However, self-hosting also transfers more security responsibility to the organisation.

When a business operates its own LLM infrastructure, it is responsible for:

• Model and dependency integrity
• Authentication and authorisation
• API security
• Container and host hardening
• Secrets management
• Network segmentation
• Data ingestion
• Vector databases
• Model gateways
• Monitoring and logging
• Patch management
• Vulnerability management
• Incident detection and response

As a result, the organisation must secure much more than the model itself. For example, a self-hosted model can still be exposed through weak APIs, excessive permissions, compromised packages, poisoned documents, insecure output handling, stolen credentials, or vulnerable infrastructure. Furthermore, internal systems often receive a higher level of trust simply because they operate inside the corporate network. That trust can increase the impact of an attack. Self-hosted is a deployment model. It is not a security control.

Agentic AI Raises the Potential Impact

Depending on their tools and permissions, AI agents may search internal document repositories, read emails and calendars, query databases, create or modify files, execute scripts, open support tickets, send messages, call internal APIs, trigger workflows, interact with cloud services, and communicate with other agents. These capabilities make agents valuable because they allow them to perform work rather than simply provide information.

Therefore, these capabilities make agents useful because they can perform work instead of only discussing it.

Nevertheless, the same capabilities increase risk. A manipulated chatbot may generate an unsafe response. By contrast, a manipulated AI agent may take an unsafe action. For example, an agent could retrieve confidential data, call a privileged API, modify a file, update its memory, send an email, or start an automated workflow.

OWASP describes this type of risk as Excessive Agency. It appears when an LLM application receives more functionality, permissions, or independence than it needs. Consequently, a successful attack may affect the confidentiality, integrity, or availability of connected systems.This is the key security shift. A malicious prompt is no longer limited to changing what the model says. It may also change what the system does.

The OWASP Top 10 for LLM Applications

The OWASP Top 10 for LLM Applications gives organisations a practical starting point for assessing AI security risks.

The list includes:

1. Prompt Injection
2. Sensitive Information Disclosure
3. Supply Chain
4. Data and Model Poisoning
5. Improper Output Handling
6. Excessive Agency
7. System Prompt Leakage
8. Vector and Embedding Weaknesses
9. Misinformation
10. Unbounded Consumption

As a result, these risks show that LLM security is not only a model-security problem.

Instead, it involves applications, identities, data, infrastructure, third-party components, APIs, tools, and human decisions.

For that reason, responsibility cannot sit with one team alone.

A secure enterprise AI programme requires cooperation across the organisation. SOC analysts, security and detection engineers, application-security teams, AI and machine-learning engineers, DevOps and platform teams, identity and access-management teams, data owners, governance teams, and compliance teams must work together. Therefore, the complete security boundary includes every system that supplies data to the model, receives its output, or allows it to take action.

Therefore, the complete security boundary includes every system that supplies data to the model, receives output from it, or allows it to take action.

Prompt Injection May Begin Outside the Chat Window

Prompt injection is one of the most widely discussed LLM security risks.In a direct prompt injection attack, a user enters instructions designed to override the model’s intended behaviour. However, the malicious instruction does not always come directly from the user. During an indirect prompt injection attack, the instruction is hidden inside external content that the model processes. For instance, malicious instructions may be hidden inside emails, documents, calendar invitations, support tickets, web pages, source-code comments, product reviews, knowledge-base articles, images, data returned by another application, or content produced by another AI agent. Consequently, the user may never see the hidden instruction.

Consequently, the user may never see the hidden instruction. Instead, the model retrieves or reads the content while performing a normal task. It may then treat the attacker’s text as a trusted instruction and as a result, the model could disclose sensitive information, reveal internal instructions, misuse a connected tool, or trigger an unauthorised action.

RAG and fine-tuning can improve the usefulness of an LLM. However, they do not remove the risk of prompt injection. Recent security research has shown how external content can affect later agent behaviour, manipulate trusted information, trigger unintended actions, or influence persistent AI memory.

Although individual weaknesses may affect different products, the wider issue is architectural. LLMs do not always separate trusted instructions from untrusted data as clearly as traditional software separates code from input. Therefore, organisations must assume that any content processed by an LLM could attempt to influence its behaviour.

RAG Should Be Treated as a Data Supply Chain

Retrieval-Augmented Generation, or RAG, allows an LLM to retrieve information from an external knowledge source before producing a response.

This approach helps organisations build AI assistants that use internal policies, product documentation, customer records, security information, or operational knowledge.

However, RAG also introduces several security-sensitive components. These components may include source-document repositories, data-ingestion and transformation pipelines, embedding models, vector databases, retrieval services, access-control rules, document permissions, metadata, prompt-building logic, model inference services, caches, and conversation memory. Because each component affects the information supplied to the model, an attacker may not need to compromise the model directly.

Therefore, an attacker may not need to compromise the model directly. Instead, the attacker could target the information that the model retrieves. For example, an adversary might upload a poisoned document into an indexed repository. Later, the retrieval system may select that document and send it to the LLM.

Other possible scenarios include:

• Changing content in a trusted knowledge base
• Hiding malicious instructions inside retrieved text
• Exploiting weak permissions between vector collections
• Manipulating document metadata
• Causing restricted information to be retrieved for the wrong user
• Repeatedly adding poisoned content through an ingestion service
• Compromising retrieval-pipeline credentials
• Modifying the ranking or relevance of indexed documents
• Inserting misleading data into a trusted source

Consequently, protecting RAG requires more than scanning user prompts and security teams must monitor the complete retrieval process, from the original document to the final model response.

What Should Security Teams Monitor?

In order to, investigate suspicious LLM activity, defenders need visibility across several parts of the AI environment. Monitoring only the final prompt and response is not enough. On the contrary, teams should collect and connect events from the model, RAG pipeline, AI agents, identities, applications, and underlying infrastructure.

Model and Application Telemetry

Useful model and application events may include:

• Prompt and response metadata
• Model versions
• Application versions
• Policy and guardrail versions
• Input classifications
• Output classifications
• Rejected requests
• Guardrail decisions
• System-prompt access attempts
• Token usage
• Latency
• Model errors
• Conversation identifiers
• Request identifiers
• User identities
• Service identities
• Tool-selection decisions

However, organisations should not automatically log every prompt and response in full. Prompts may contain personal data, credentials, confidential business information, or regulated records. Furthermore, the logs themselves could become a target. For that reason, logging policies should support data minimisation, access control, masking, retention limits, and regulatory requirements.

RAG and Data-Pipeline Telemetry

In addition, security teams should monitor the data entering and leaving the RAG pipeline.

Relevant events may include:

• Documents added to a repository
• Documents modified or removed
• The identity responsible for each change
• Document origin
• Data classification
• Ingestion jobs
• Embedding jobs
• Changes to vector collections
• Retrieval queries
• Retrieved document identifiers
• Permission-filtering results
• Access to restricted knowledge sources
• Large retrieval operations
• Repeated access to sensitive records
• Changes in document ranking
• Changes in retrieval relevance

Data provenance is especially important. Defenders should be able to determine where a document came from, when it entered the system, who changed it, and which user or agent later retrieved it. Since, without this context, a poisoned document may look like a legitimate source.

Agent and Tool Telemetry

Agentic systems require an additional layer of monitoring because they can use tools, access data, and perform actions across connected systems. Organisations should log which tools are available to each agent, which tools are selected for each task, the parameters sent to those tools, API calls, database queries, files read or modified, commands executed, memory updates, approval requests, denied actions, actions performed without confirmation, agent-to-agent communication, and any changes to agent configuration or tool permissions.

The original prompt may appear suspicious, but the agent’s actions after receiving it are often more important. Security teams should determine whether the agent accessed restricted documents, updated persistent memory, called unexpected APIs, executed commands, or communicated with external endpoints. Viewed separately, these events may appear harmless. However, when correlated, they may reveal a complete attack chain.

Infrastructure Telemetry

Similarly, self-hosted AI environments should be monitored like any other critical production system.

Relevant data sources include:

• Authentication logs
• API gateway logs
• Identity-provider events
• Operating-system logs
• Container events
• Kubernetes audit events
• Process execution
• File-integrity monitoring
• Network traffic
• DNS queries
• Firewall logs
• Cloud audit logs
• Database access
• Secrets-manager events
• GPU and resource usage
• Model-server health
• Patch status
• Vulnerability findings

Infrastructure telemetry helps defenders understand whether an attack remained inside the application layer or moved into the wider environment. For example, a suspicious prompt may be followed by a shell process, a secrets-manager request, and an outbound network connection. That sequence is far more important than the prompt alone.

Correlating AI Security Events with Logstail SIEM

An individual AI event may not provide enough context to identify an attack. A document update may be normal. Likewise, a retrieval request may be expected. A tool call may also be part of a legitimate workflow. The risk becomes visible when these events are connected.

Consider the following sequence:

1. An external document is added to an indexed repository.
2. An embedding job processes the document.
3. A user asks the AI agent an unrelated question.
4. The poisoned document is retrieved.
5. The agent calls a privileged tool.
6. The model-serving environment creates a new outbound connection.

Individually, each event may appear harmless. Together, they form a high-priority security investigation.

Logstail SIEM can provide a central visibility layer for logs, alerts, detection rules, dashboards, infrastructure metrics, and reporting.

By collecting telemetry from the LLM application and the systems around it, security teams can build detections for scenarios such as:

• Prompt-injection indicators followed by sensitive retrieval
• Unexpected changes to indexed documents
• Retrieval from an unauthorised vector collection
• Memory updates triggered by untrusted content
• Repeated attempts to extract system instructions
• Unusual access to secrets
• Agents using tools outside their normal behaviour
• High-impact actions performed without approval
• Model-server processes starting command-line tools
• New outbound connections after an agent action
• Unusual token or resource consumption
• AI configuration changes outside approved deployment windows

Of course, SIEM alone cannot prevent every LLM attack.

Nevertheless, it provides the visibility and correlation needed to determine whether a manipulation attempt affected the model, the connected tools, or the wider infrastructure.

Responding to AI Threats with Logstail SOAR

However, detection is only the first step. After identifying suspicious activity, the security team may need to contain the incident before the agent performs more actions or the poisoned content is retrieved again.

Depending on the available integrations, Logstail SOAR playbooks can help coordinate actions such as:

• Creating and enriching an incident case
• Notifying the SOC
• Disabling an affected service account
• Revoking sessions
• Revoking API tokens
• Blocking a malicious source
• Isolating a host or container
• Disabling a high-risk agent tool
• Quarantining a suspicious document
• Removing content from the RAG ingestion queue
• Preserving relevant logs
• Collecting evidence
• Requesting analyst approval
• Escalating the incident to privacy, legal, AI, or governance teams

However, automated response must be designed carefully. An incorrect response action could interrupt a legitimate AI service or remove important business data. Therefore, high-impact playbooks can be built to include risk checks, approval stages, and clear rollback procedures. Automation should support the analyst, not remove control from the analyst.

LLM Security Must Be Practised

Reading about prompt injection is useful. Still, theory alone does not show how an attack works in a real environment. A written explanation cannot fully demonstrate how malicious instructions influence an LLM. Similarly, a RAG architecture diagram cannot show what poisoned retrieval looks like inside security telemetry. A checklist also cannot teach an analyst how to connect model events, identity logs, API calls, and network activity during an investigation.

The AI Security Monitoring with Logstail course provides guided, hands-on training based on common LLM security risks and the OWASP Top 10 for LLM Applications. Participants work with an intentionally vulnerable LLM environment.

First, learners follow guided exercises to perform attacks against the system. Afterwards, they examine the activity from the defender’s perspective. The generated telemetry can then be investigated through the Logstail platform.

Through this approach, learners can understand:

• How common LLM attacks are constructed
• Why simple prompts can become security incidents
• How prompt injection affects model behaviour
• How RAG data can influence an LLM
• What evidence an attack leaves behind
• Which telemetry sources provide the most value
• How detection rules identify suspicious behaviour
• How SOC analysts can investigate AI incidents
• How red-team activity can become blue-team detection logic

Most importantly, the training connects offensive testing with defensive monitoring. As a result, learners do not only discover how an LLM can be attacked. They also learn how to detect the attack, investigate its impact, and respond through a security operations workflow.

Building a More Secure AI Environment

LLMs will continue to gain access to more data, tools, applications, and workflows, while AI agents will become more independent and capable of performing more complex tasks.

As these systems gain authority, the security boundary around them must grow as well. Organisations should begin by mapping their AI environment.

Teams need to identify every LLM and AI-agent deployment, the applications and tools connected to them, their data sources and vector databases, the service accounts and API permissions they use, the business workflows they support, their external dependencies, and any gaps in logging or monitoring. Next, teams should apply least privilege and create clear approval boundaries. In parallel, RAG ingestion pipelines and vector stores also need protection. In addition, organisations should record data provenance and monitor changes to trusted knowledge sources.

Furthermore centralised telemetry is equally important, security teams should collect events from the LLM application, AI agents, RAG components, identities, APIs, endpoints, containers, networks, and cloud environments.

Finally, organisations should test their controls. Controlled attack exercises can reveal gaps that architecture reviews and policy documents may miss and they also help SOC, development, infrastructure, and AI teams understand how they must work together during an incident.

From AI Adoption to AI Defence

The goal is to ensure that innovation does not create an unmonitored layer of infrastructure with access to sensitive data and business-critical systems. LLM security is becoming part of application security, data security, identity security, cloud security, infrastructure security, and security operations.Therefore, the organisations best prepared for the next stage of AI adoption will combine:

• Secure architecture
• Least-privilege access
• Protected RAG pipelines
• Continuous telemetry
• SIEM correlation
• SOAR response
• Controlled security testing
• Personel Hands-on training

AI systems will continue to evolve.

Security teams must evolve with them.

Turn LLM Attacks into Detectable Security Events

Explore the Securing AI Systems: Attacks, Leakage, Agents & Defense and gain practical experience attacking an intentionally vulnerable LLM.

Then, analyse the resulting telemetry and detect the attack through the Logstail platform.

Built by defenders, for defenders—because securing AI starts with understanding how it can be attacked.

Contact Our Experts or Sign Up for Free