No potential LLM Jacking attacks detected across all Azure API Management instances

apim_threat_detection_llm_jacking

Service: apimSeverity: critical

OfficialBuilt-inVerified

Categories:gen-ailoggingthreat-detection

Resource Type:microsoft.apimanagement/service

API Management diagnostic logs in Log Analytics are analyzed for LLM-related operations. Requests are grouped by caller IP, the number of distinct monitored actions (e.g., ChatCompletions_Create, ImageGenerations_Create) within a configurable minutes window is measured, and that ratio is compared to a threshold to surface anomalous multi-action patterns.

Prowler prowler-provider-azure

Risk

Concentrated LLM activity from one IP indicates automation or leaked credentials.

Availability/cost: rapid token burn and quota exhaustion
Confidentiality: exposure of prompts/completions and model details
Integrity: abuse of deployment/model actions enabling unauthorized changes or mass content generation

Run this check with Prowler

Run in Prowler Cloud

Run

prowler azure --checks apim_threat_detection_llm_jacking

Recommendation

Adopt defense in depth for LLM APIs:

Enforce least privilege; isolate management from inference
Prefer managed identities over keys; rotate secrets
Apply quotas, rate limiting, and IP allowlisting; use private access
Alert on anomalous action diversity; review logs

Tune threshold and minutes for your environment.

Remediation

Terraform

# Global APIM policy that blocks a specific IP
resource "azurerm_api_management_policy" "<example_resource_name>" {
  api_management_id = "<example_resource_id>"

  # Critical: Policy XML that blocks the offending IP by returning 403
  xml_content = <<XML
<policies>
  <inbound>
    <base />
    <choose>
      <!-- Critical: Block offending IP to stop LLM jacking -->
      <when condition="@(context.Request.IpAddress == \"<OFFENDING_IP>\")">
        <return-response>
          <set-status code="403" reason="Forbidden" />
        </return-response>
      </when>
    </choose>
  </inbound>
  <backend><base /></backend>
  <outbound><base /></outbound>
  <on-error><base /></on-error>
</policies>
XML
}

NativeIaC

// Blocks a specific IP at the global (service) policy level for APIM
param apimName string
param blockedIp string

resource apim 'Microsoft.ApiManagement/service@2023-05-01-preview' existing = {
  name: apimName
}

resource apimPolicy 'Microsoft.ApiManagement/service/policies@2023-05-01-preview' = {
  parent: apim
  name: 'policy'
  properties: {
    value: '<policies><inbound><base /><choose><!-- Critical: Block offending IP to stop LLM jacking -->
      <when condition="@(context.Request.IpAddress == "${blockedIp}")">
        <return-response><set-status code="403" reason="Forbidden" /></return-response>
      </when>
    </choose></inbound><backend><base /></backend><outbound><base /></outbound><on-error><base /></on-error></policies>' // Critical: Policy XML that blocks the offending IP
    format: 'xml' // Critical: Apply policy as XML
  }
}

Other

1In the Azure portal, open your API Management instance

2Go to APIs > All APIs

3Click Policies (Inbound processing)

4Add a when block to block the offending IP:

5- Condition: @(context.Request.IpAddress == "<OFFENDING_IP>")

6- Action: return-response with status 403 Forbidden

7Save the policy

8Re-run the scan after the detection window elapses to confirm PASS

Check Code

from typing import List

from prowler.lib.check.models import Check, Check_Report_Azure
from prowler.providers.azure.services.apim.apim_client import apim_client
from prowler.providers.azure.services.apim.apim_service import LogsQueryLogEntry


class apim_threat_detection_llm_jacking(Check):
    def execute(self):
        findings = []

        # Get configuration from audit config with defaults
        threshold = float(
            getattr(apim_client, "audit_config", {}).get(
                "apim_threat_detection_llm_jacking_threshold", 0.1
            )
        )
        threat_detection_minutes = getattr(apim_client, "audit_config", {}).get(
            "apim_threat_detection_llm_jacking_minutes", 1440
        )
        monitored_actions = getattr(apim_client, "audit_config", {}).get(
            "apim_threat_detection_llm_jacking_actions",
            [
                # OpenAI API endpoints
                "ImageGenerations_Create",
                "ChatCompletions_Create",
                "Completions_Create",
                "Embeddings_Create",
                "FineTuning_Jobs_Create",
                "Models_List",
                # Azure OpenAI endpoints
                "Deployments_List",
                "Deployments_Get",
                "Deployments_Create",
                "Deployments_Delete",
                # Anthropic endpoints
                "Messages_Create",
                "Claude_Create",
                # Google AI endpoints
                "GenerateContent",
                "GenerateText",
                "GenerateImage",
                # Meta AI endpoints
                "Llama_Create",
                "CodeLlama_Create",
                # Other LLM endpoints
                "Gemini_Generate",
                "Claude_Generate",
                "Llama_Generate",
            ],
        )

        for subscription, instances in apim_client.instances.items():
            subscription_name = apim_client.subscriptions.get(
                subscription, subscription
            )
            all_llm_logs: List[LogsQueryLogEntry] = []
            for instance in instances:
                if instance.log_analytics_workspace_id:
                    logs = apim_client.get_llm_operations_logs(
                        subscription, instance, threat_detection_minutes
                    )
                    all_llm_logs.extend(logs)

            # Analyze logs only within the current subscription to avoid
            # cross-subscription attribution when scanning multiple subscriptions.
            potential_llm_jacking_attackers = {}
            for log in all_llm_logs:
                operation_name = log.operation_id
                caller_ip = log.caller_ip_address

                if operation_name in monitored_actions and caller_ip:
                    # Use IP address as the principal identifier
                    if caller_ip not in potential_llm_jacking_attackers:
                        potential_llm_jacking_attackers[caller_ip] = set()
                    potential_llm_jacking_attackers[caller_ip].add(operation_name)

            # 3. Check each principal against the threshold and report failures
            found_potential_llm_jacking_attackers = False
            for (
                principal_ip,
                distinct_actions,
            ) in potential_llm_jacking_attackers.items():
                action_ratio = round(len(distinct_actions) / len(monitored_actions), 2)

                if action_ratio > threshold:
                    found_potential_llm_jacking_attackers = True
                    # Build Identity resource for the report
                    resource = {
                        "name": principal_ip,
                        "id": principal_ip,
                    }
                    # Report against the subscription, identifying the offending principal (IP)
                    report = Check_Report_Azure(self.metadata(), resource=resource)
                    report.subscription = subscription
                    report.status = "FAIL"
                    report.status_extended = f"Potential LLM Jacking attack detected from IP address {principal_ip} in subscription {subscription_name} ({subscription}) with an action ratio of {action_ratio}, above the configured threshold of {threshold}."
                    findings.append(report)

            # If no threats were found after checking all principals, create a single PASS report.
            if not found_potential_llm_jacking_attackers:
                report = Check_Report_Azure(self.metadata(), resource={})
                report.resource_name = subscription_name
                report.resource_id = f"/subscriptions/{subscription}"
                report.subscription = subscription
                report.status = "PASS"
                report.status_extended = f"No potential LLM Jacking attacks detected across monitored APIM instances in subscription {subscription_name} ({subscription}) in the last {threat_detection_minutes} minutes."
                findings.append(report)

        return findings

References

https://learn.microsoft.com/en-us/azure/api-management/monitor-api-management