OpenSearchIngestion / Client / start_pipeline

start_pipeline¶

OpenSearchIngestion.Client.start_pipeline(**kwargs)¶

Starts an OpenSearch Ingestion pipeline. For more information, see Starting an OpenSearch Ingestion pipeline.

Request Syntax

response = client.start_pipeline(
    PipelineName='string'
)

Parameters:

PipelineName (string) –

[REQUIRED]

The name of the pipeline to start.

Return type:

dict

Returns:

Response Syntax

{
    'Pipeline': {
        'PipelineName': 'string',
        'PipelineArn': 'string',
        'MinUnits': 123,
        'MaxUnits': 123,
        'Status': 'CREATING'|'ACTIVE'|'UPDATING'|'DELETING'|'CREATE_FAILED'|'UPDATE_FAILED'|'STARTING'|'START_FAILED'|'STOPPING'|'STOPPED',
        'StatusReason': {
            'Description': 'string'
        },
        'PipelineConfigurationBody': 'string',
        'CreatedAt': datetime(2015, 1, 1),
        'LastUpdatedAt': datetime(2015, 1, 1),
        'IngestEndpointUrls': [
            'string',
        ],
        'LogPublishingOptions': {
            'IsLoggingEnabled': True|False,
            'CloudWatchLogDestination': {
                'LogGroup': 'string'
            }
        },
        'VpcEndpoints': [
            {
                'VpcEndpointId': 'string',
                'VpcId': 'string',
                'VpcOptions': {
                    'SubnetIds': [
                        'string',
                    ],
                    'SecurityGroupIds': [
                        'string',
                    ],
                    'VpcAttachmentOptions': {
                        'AttachToVpc': True|False,
                        'CidrBlock': 'string'
                    },
                    'VpcEndpointManagement': 'CUSTOMER'|'SERVICE'
                }
            },
        ],
        'BufferOptions': {
            'PersistentBufferEnabled': True|False
        },
        'EncryptionAtRestOptions': {
            'KmsKeyArn': 'string'
        },
        'VpcEndpointService': 'string',
        'ServiceVpcEndpoints': [
            {
                'ServiceName': 'OPENSEARCH_SERVERLESS',
                'VpcEndpointId': 'string'
            },
        ],
        'Destinations': [
            {
                'ServiceName': 'string',
                'Endpoint': 'string'
            },
        ],
        'Tags': [
            {
                'Key': 'string',
                'Value': 'string'
            },
        ],
        'PipelineRoleArn': 'string'
    }
}

Response Structure

(dict) –
- Pipeline (dict) –
  
  Information about an existing OpenSearch Ingestion pipeline.
  - PipelineName (string) –
    
    The name of the pipeline.
  - PipelineArn (string) –
    
    The Amazon Resource Name (ARN) of the pipeline.
  - MinUnits (integer) –
    
    The minimum pipeline capacity, in Ingestion Compute Units (ICUs).
  - MaxUnits (integer) –
    
    The maximum pipeline capacity, in Ingestion Compute Units (ICUs).
  - Status (string) –
    
    The current status of the pipeline.
  - StatusReason (dict) –
    
    The reason for the current status of the pipeline.
    - Description (string) –
      
      A description of why a pipeline has a certain status.
  - PipelineConfigurationBody (string) –
    
    The Data Prepper pipeline configuration in YAML format.
  - CreatedAt (datetime) –
    
    The date and time when the pipeline was created.
  - LastUpdatedAt (datetime) –
    
    The date and time when the pipeline was last updated.
  - IngestEndpointUrls (list) –
    
    The ingestion endpoints for the pipeline, which you can send data to.
    - (string) –
  - LogPublishingOptions (dict) –
    
    Key-value pairs that represent log publishing settings.
    - IsLoggingEnabled (boolean) –
      
      Whether logs should be published.
    - CloudWatchLogDestination (dict) –
      
      The destination for OpenSearch Ingestion logs sent to Amazon CloudWatch Logs. This parameter is required if IsLoggingEnabled is set to true.
      - LogGroup (string) –
        
        The name of the CloudWatch Logs group to send pipeline logs to. You can specify an existing log group or create a new one. For example, /aws/vendedlogs/OpenSearchService/pipelines.
  - VpcEndpoints (list) –
    
    The VPC interface endpoints that have access to the pipeline.
    - (dict) –
      
      An OpenSearch Ingestion-managed VPC endpoint that will access one or more pipelines.
      - VpcEndpointId (string) –
        
        The unique identifier of the endpoint.
      - VpcId (string) –
        
        The ID for your VPC. Amazon Web Services PrivateLink generates this value when you create a VPC.
      - VpcOptions (dict) –
        
        Information about the VPC, including associated subnets and security groups.
        
        SubnetIds (list) –
        
        A list of subnet IDs associated with the VPC endpoint.
        
        (string) –
        
        SecurityGroupIds (list) –
        
        A list of security groups associated with the VPC endpoint.
        
        (string) –
        
        VpcAttachmentOptions (dict) –
        
        Options for attaching a VPC to a pipeline.
        
        AttachToVpc (boolean) –
        
        Whether a VPC is attached to the pipeline.
        
        CidrBlock (string) –
        
        The CIDR block to be reserved for OpenSearch Ingestion to create elastic network interfaces (ENIs).
        
        VpcEndpointManagement (string) –
        
        Defines whether you or Amazon OpenSearch Ingestion service create and manage the VPC endpoint configured for the pipeline.
  - BufferOptions (dict) –
    
    Options that specify the configuration of a persistent buffer. To configure how OpenSearch Ingestion encrypts this data, set the EncryptionAtRestOptions. For more information, see Persistent buffering.
    - PersistentBufferEnabled (boolean) –
      
      Whether persistent buffering should be enabled.
  - EncryptionAtRestOptions (dict) –
    
    Options to control how OpenSearch encrypts buffer data.
    - KmsKeyArn (string) –
      
      The ARN of the KMS key used to encrypt buffer data. By default, data is encrypted using an Amazon Web Services owned key.
  - VpcEndpointService (string) –
    
    The VPC endpoint service name for the pipeline.
  - ServiceVpcEndpoints (list) –
    
    A list of VPC endpoints that OpenSearch Ingestion has created to other Amazon Web Services services.
    - (dict) –
      
      A container for information about VPC endpoints that were created to other services
      - ServiceName (string) –
        
        The name of the service for which a VPC endpoint was created.
      - VpcEndpointId (string) –
        
        The unique identifier of the VPC endpoint that was created.
  - Destinations (list) –
    
    Destinations to which the pipeline writes data.
    - (dict) –
      
      An object representing the destination of a pipeline.
      - ServiceName (string) –
        
        The name of the service receiving data from the pipeline.
      - Endpoint (string) –
        
        The endpoint receiving data from the pipeline.
  - Tags (list) –
    
    A list of tags associated with the given pipeline.
    - (dict) –
      
      A tag (key-value pair) for an OpenSearch Ingestion pipeline.
      - Key (string) –
        
        The tag key. Tag keys must be unique for the pipeline to which they are attached.
      - Value (string) –
        
        The value assigned to the corresponding tag key. Tag values can be null and don’t have to be unique in a tag set. For example, you can have a key value pair in a tag set of project : Trinity and cost-center : Trinity
  - PipelineRoleArn (string) –
    
    The Amazon Resource Name (ARN) of the IAM role that the pipeline uses to access AWS resources.

start_pipeline¶

Request Syntax

Response Syntax

Response Structure

Exceptions