Back to Checks
aws

SageMaker endpoint configuration has all production variants with at least two initial instances

sagemaker_endpoint_config_prod_variant_instances
Service: sagemakerSeverity: medium
OfficialBuilt-inVerified
Categories:resiliencegen-ai
Resource Type:Other

Amazon SageMaker endpoint configurations are evaluated to ensure each production variant uses an initial instance count of at least two. Variants with InitialInstanceCount less than two in instance-based endpoints are identified, indicating no built-in multi-AZ redundancy.

Prowler logoProwlerprowler-provider-aws

Risk

A single-instance variant is a single point of failure. If the instance or its Availability Zone fails, inference becomes unavailable, leading to service outages, SLA breaches, and cascading failures in dependent systems. This primarily degrades availability and reliability.

Run this check with Prowler

Run in Prowler Cloud
Run
prowler aws --checks sagemaker_endpoint_config_prod_variant_instances

Recommendation

Apply the resilience principle:

  • Run each production variant with InitialInstanceCount >= 2
  • Spread capacity across AZs with health-based scaling
  • Use rolling/canary updates to avoid downtime
  • Right-size capacity to tolerate node loss

For bursty workloads, consider serverless or autoscaling for elasticity.

Remediation

CLI

aws sagemaker delete-endpoint-config --endpoint-config-name <endpoint-config-name>

Terraform

# SageMaker Endpoint Configuration with InitialInstanceCount > 1
resource "aws_sagemaker_endpoint_configuration" "<example_resource_name>" {
  production_variants {
    variant_name           = "<example_resource_name>"
    model_name             = "<example_resource_name>"
    instance_type          = "ml.m5.large"
    initial_instance_count = 2 # Critical: set >=2 so the variant passes the check
  }
}

NativeIaC

# CloudFormation: EndpointConfig with InitialInstanceCount > 1
Resources:
  <example_resource_name>:
    Type: AWS::SageMaker::EndpointConfig
    Properties:
      ProductionVariants:
        - VariantName: <example_resource_name>
          ModelName: <example_resource_name>
          InstanceType: ml.m5.large
          InitialInstanceCount: 2  # Critical: set >=2 so all variants start with at least two instances

Other

1In the AWS Console, go to SageMaker > Inference > Endpoint configurations
2Click Create endpoint configuration and add each Production variant with Initial instance count set to 2
3If any endpoints use the old configuration, go to SageMaker > Inference > Endpoints, select the endpoint, choose Update, and select the new endpoint configuration
4Return to Endpoint configurations and delete the noncompliant configuration

Check Code

from prowler.lib.check.models import Check, Check_Report_AWS
from prowler.providers.aws.services.sagemaker.sagemaker_client import sagemaker_client


class sagemaker_endpoint_config_prod_variant_instances(Check):
    def execute(self):
        findings = []
        for endpoint_config in sagemaker_client.endpoint_configs.values():
            report = Check_Report_AWS(
                metadata=self.metadata(), resource=endpoint_config
            )
            report.status = "PASS"
            report.status_extended = f"Sagemaker Endpoint Config {endpoint_config.name} has all production variants with more than one initial instance."
            non_compliant_production_variants = []
            for production_variant in endpoint_config.production_variants:
                if production_variant.initial_instance_count <= 1:
                    non_compliant_production_variants.append(production_variant.name)

            if non_compliant_production_variants:
                report.status = "FAIL"
                report.status_extended = f"Sagemaker Endpoint Config {endpoint_config.name}'s production variants {', '.join(non_compliant_production_variants)} with less than two initial instance."

            findings.append(report)

        return findings

References