Back to Checks
aws

Amazon SageMaker training job volume has KMS encryption enabled

sagemaker_training_jobs_volume_and_output_encryption_enabled
Service: sagemakerSeverity: high
OfficialBuilt-inVerified
Categories:encryptiongen-ai
Resource Type:Other

Amazon SageMaker training jobs use KMS encryption for their attached ML storage volumes via VolumeKmsKeyId.

The finding identifies training jobs where the volume encryption key is not configured.

Prowler logoProwlerprowler-provider-aws

Risk

Missing CMEK leaves training data, checkpoints, and logs on the volume without tenant-controlled encryption. This reduces confidentiality, enables data exposure via snapshots or privileged access, and limits control over key policies, rotation, and emergency revocation.

Run this check with Prowler

Run in Prowler Cloud
Run
prowler aws --checks sagemaker_training_jobs_volume_and_output_encryption_enabled

Recommendation

Encrypt training volumes with customer-managed KMS keys and apply the same to S3 input/output.

Use least privilege on key policies, enable rotation, restrict grants, and prevent storage of unencrypted artifacts to achieve defense in depth.

Remediation

Terraform

# SageMaker Training Job with EBS volume KMS encryption enabled
resource "aws_sagemaker_training_job" "<example_resource_name>" {
  name     = "<example_resource_name>"
  role_arn = "<example_role_arn>"

  algorithm_specification {
    training_image      = "<example_ecr_image_uri>"
    training_input_mode = "File"
  }

  output_data_config {
    s3_output_path = "s3://<example_bucket>"
  }

  resource_config {
    instance_type     = "ml.m5.large"
    instance_count    = 1
    volume_size_in_gb = 10
    volume_kms_key_id = "<example_kms_key_arn>" # CRITICAL: Enables KMS encryption for the training EBS volume
  }

  stopping_condition {
    max_runtime_in_seconds = 3600
  }
}

NativeIaC

# CloudFormation: SageMaker TrainingJob with EBS volume KMS encryption enabled
Resources:
  <example_resource_name>:
    Type: AWS::SageMaker::TrainingJob
    Properties:
      TrainingJobName: <example_resource_name>
      RoleArn: <example_role_arn>
      AlgorithmSpecification:
        TrainingImage: <example_ecr_image_uri>
        TrainingInputMode: File
      OutputDataConfig:
        S3OutputPath: s3://<example_bucket>
      ResourceConfig:
        InstanceType: ml.m5.large
        InstanceCount: 1
        VolumeSizeInGB: 10
        VolumeKmsKeyId: <example_kms_key_arn>  # CRITICAL: Encrypts the training EBS volume with the specified KMS key
      StoppingCondition:
        MaxRuntimeInSeconds: 3600

Other

1In the AWS console, go to SageMaker > Training > Training jobs
2Select the non-encrypted job and click Clone to create a new job
3In Resource configuration, set Volume encryption key to your KMS key (Customer managed key)
4Complete required fields and click Create training job
5Verify the new job shows the KMS key under Volume encryption

Check Code

from prowler.lib.check.models import Check, Check_Report_AWS
from prowler.providers.aws.services.sagemaker.sagemaker_client import sagemaker_client


class sagemaker_training_jobs_volume_and_output_encryption_enabled(Check):
    def execute(self):
        findings = []
        for training_job in sagemaker_client.sagemaker_training_jobs:
            report = Check_Report_AWS(metadata=self.metadata(), resource=training_job)
            report.status = "PASS"
            report.status_extended = f"Sagemaker training job {training_job.name} has KMS encryption enabled."
            if not training_job.volume_kms_key_id:
                report.status = "FAIL"
                report.status_extended = f"Sagemaker training job {training_job.name} has KMS encryption disabled."

            findings.append(report)

        return findings

References