Advanced Terraform Project 2: Automated ASG Lifecycle Management with Integrated SNS and Lambda Functions for Seamless Instance Termination.

Introduction:

This project is designed to manage AWS infrastructure using Terraform, leveraging Auto Scaling Groups (ASG) and Application Load Balancers (ALB) to dynamically handle server loads. Additionally, SNS and Lambda functions are integrated to perform actions before instance termination, ensuring a smooth and automated infrastructure management process.

Objective:

The objective of this project is to automate the scaling of server instances based on load conditions while ensuring pre-termination actions are executed to maintain application stability and performance.

Agenda:

Set up a robust infrastructure using Terraform. Implement auto-scaling for dynamic load handling. Ensure pre-termination tasks are executed using SNS and Lambda. Provide a structured and maintainable codebase for infrastructure management.

Use Cases:

1. Dynamic Load Management: Automatically scale instances based on traffic and load, ensuring optimal resource utilization.

2. Pre-Termination Tasks: Execute specific actions before terminating instances to maintain application consistency.

3. Automated Infrastructure: Use Terraform to automate the deployment and management of AWS resources.

Advantages:

1. Scalability: Automatically adjusts to traffic changes, ensuring high availability.

2. Automation: Reduces manual intervention through automated deployment and scaling.

3. Cost Efficiency: Optimizes resource usage, reducing unnecessary costs.

4. Maintainability: Structured Terraform codebase for easier maintenance and updates.

Project Structure

1. VPC Module

   Purpose: Creates a Virtual Private Cloud (VPC) with two subnets, one public and one private, to segment and manage network traffic efficiently.

   Configuration: Includes CIDR block allocation, DNS support, and tagging for identification and management.

   Resources: AWS VPC and Subnet resources.

2. Security Groups Module

   Purpose: Defines security groups to manage inbound and outbound traffic for the Application Load Balancer (ALB) and EC2 instances.

   Configuration: Specifies rules for traffic filtering based on protocols, ports, and IP ranges.

   Resources: Separate security groups for ALB and EC2.

3. EC2 Module

   Purpose: Configures EC2 instances with specific instance profiles for role-based access.

   Configuration: Defines instance types, AMI IDs, key pairs, and security group associations, along with IAM instance profiles.

   Resources: AWS IAM Role, IAM Instance Profile, and EC2 Launch Template.

4. Target Groups Module

   Purpose: Sets up target groups to manage and distribute traffic between instances based on load conditions.

   Configuration: Defines target groups for day and night loads, with port and protocol specifications.

   Resources: AWS LB Target Group resources.

5. ALB Module

   Purpose: Configures the Application Load Balancer and its listeners to distribute traffic across target groups.

   Configuration: Specifies load balancing algorithms, SSL policies, and listener rules for traffic distribution based on conditions.

   Resources: AWS LB, LB Listener, and LB Listener Rule resources.

7. Auto Scaling Module

   Purpose: Manages auto-scaling groups to handle dynamic scaling of instances based on load and predefined policies.

   Configuration: Defines scaling policies, lifecycle hooks for pre-termination actions, and instance launch configurations.

   Resources: AWS Auto Scaling Group, Lifecycle Hook, IAM Role, and IAM Policy.

8. SNS Module

   Purpose: Creates SNS topics and subscriptions to send notifications for lifecycle events and trigger Lambda functions.

   Configuration: Specifies SNS topic names, subscription endpoints, and notification protocols.

   Resources: AWS SNS Topic and SNS Topic Subscription.

9. Lambda Function Module

   Purpose: Defines Lambda functions to execute custom actions, such as running scripts before instance termination.

   Configuration: Includes IAM roles and policies for Lambda execution and the function code itself.

   Resources: AWS Lambda Function, IAM Role, and IAM Policy.

Lambda Function:

import boto3
import logging
import json
import time

# Setup logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
    logger.info("Running custom termination script")
    logger.info(f"Event received: {event}")

    # Extract the SNS message
    try:
        sns_message = event['Records'][0]['Sns']['Message']
        message_dict = json.loads(sns_message)
        instance_id = message_dict['EC2InstanceId']
    except KeyError as e:
        logger.error(f"Key error: {e}")
        return {
            'statusCode': 400,
            'body': 'Event does not contain expected keys'
        }
    except Exception as e:
        logger.error(f"Error: {e}")
        return {
            'statusCode': 500,
            'body': 'Internal server error'
        }

    # Optional delay to ensure the instance is in the correct state
    time.sleep(10)  # Adjust the delay as needed

    # Execute SSM command
    ssm_client = boto3.client('ssm')

    try:
        response = ssm_client.send_command(
            InstanceIds=[instance_id],
            DocumentName="AWS-RunShellScript",
            Parameters={'commands': ['sudo sh /tmp/apach_tomcat_restart.sh']}
        )
        logger.info(f"SSM command response: {response}")
    except Exception as e:
        logger.error(f"Error executing SSM command: {e}")

    # Complete the lifecycle action
    client = boto3.client('autoscaling')

    try:
        response = client.complete_lifecycle_action(
            LifecycleHookName=message_dict['LifecycleHookName'],
            AutoScalingGroupName=message_dict['AutoScalingGroupName'],
            LifecycleActionToken=message_dict['LifecycleActionToken'],
            LifecycleActionResult='CONTINUE',
            InstanceId=instance_id
        )
        logger.info(f"Lifecycle action response: {response}")
    except Exception as e:
        logger.error(f"Error completing lifecycle action: {e}")

    return {
        'statusCode': 200,
        'body': 'Success'
    }

This Lambda function is part of an Auto Scaling Group's lifecycle hook setup. When an instance is about to be terminated, the function ensures that a specific script runs on the instance before it is terminated. The function achieves this by:

1. Extracting necessary information from the SNS message.

2. Running a custom script on the instance using AWS Systems Manager.

3. Completing the lifecycle action to allow the instance to terminate properly.

This process ensures that necessary cleanup or preparatory tasks are executed before the instance is terminated, maintaining the stability and integrity of the application.

Conclusion:

This project demonstrates a comprehensive approach to automating AWS infrastructure management using Terraform, ASG, ALB, SNS, and Lambda. By following the detailed documentation and structured codebase, you can achieve scalable, cost-efficient, and automated infrastructure that adapts to dynamic load conditions. If you have any further questions or need additional details, feel free to ask!

…