> For the complete documentation index, see [llms.txt](https://docs.caf.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.caf.io/caf-api/all-id/production-guidance/ecs-terraform.md).

# AWS ECS (Terraform)

Complete guide for deploying All ID on AWS ECS Fargate using Terraform.

## Overview

This guide explains how to deploy All ID using Terraform Infrastructure as Code.

The deployment requires:

* **All ID Application Terraform** (`src/terraform/allid/`) - Application services (ECS, RDS, ALB)
* **Existing VPC** - You must provide a VPC with the required subnet structure

{% hint style="info" %}
**Network Requirements**: The All ID Terraform modules require an existing VPC with public, private, and isolated subnets. If you don't have a VPC yet, Certta provides a reference Terraform project (`src/terraform/shared/`) that creates a production-ready network infrastructure.
{% endhint %}

{% hint style="success" %}
Certta will provide you with a package containing:

* All ID Terraform project (required)
* Reference network infrastructure Terraform project (optional, use if you need to create a new VPC)
* Configuration examples and documentation

**Note**: The project uses a default prefix `teste-tf` for resource naming. You should change this to your project name in the `terraform.tfvars` file via the `prefix` variable.
{% endhint %}

## Prerequisites

Before deploying, ensure you have:

**Tools**:

* Terraform >= 1.5
* AWS CLI configured with credentials

**AWS Account**:

* AWS account with appropriate permissions
* Permissions to create: ECS, RDS, ALB, CloudFormation, IAM roles, Security Groups
* AWS credentials configured (`aws configure`)

**Network Infrastructure** - You have three options:

1. **Option A (Recommended)**: Deploy new VPC using the Certta reference Terraform project (`src/terraform/shared/`)
2. **Option B**: Use existing VPC by manually creating SSM Parameters (see step 2B)
3. **Already have SSM Parameters**: Skip directly to step 3 if parameters are already configured

Required VPC structure (regardless of option):

* Public subnets (minimum 2 AZs) for Application Load Balancer
* Private subnets with NAT (minimum 2 AZs) for ECS services
* Isolated subnets (minimum 2 AZs) for RDS database
* Internet Gateway and NAT Gateway configured

**Container Images**:

* Access to Certta's private container registry
* ECR repository URIs for Peer and Facematch images
* Image versions/tags to deploy

{% hint style="warning" %}
**Important**: Contact your Certta technical account manager to obtain:

* Terraform project files (All ID + optional network infrastructure)
* Container registry credentials and image URIs
* Router Service certificates for mTLS communication
  {% endhint %}

## Makefile Automation

This project includes Makefiles that simplify Terraform operations. Instead of running `terraform` commands directly, you can use `make` commands that automatically handle environment-specific configurations.

**Benefits**:

* ✅ **Consistent execution**: Same commands work across all environments
* ✅ **Automatic navigation**: No need to `cd` to environment directories
* ✅ **Built-in safety**: Confirmation prompts for destructive operations
* ✅ **Easy multi-environment**: Switch environments with `ENV` parameter
* ✅ **Self-documenting**: Run `make help` to see all available commands
* ✅ **Error prevention**: Validates environment names and required parameters

{% hint style="info" %}
Throughout this guide, you'll see `make` commands instead of direct `terraform` commands. This is the recommended approach for managing this infrastructure.

**Alternative**: If you prefer using Terraform directly, you can still do so by navigating to the environment directory (e.g., `cd environments/dev`) and running standard `terraform` commands.
{% endhint %}

**Quick example**:

```bash
# Instead of:
cd environments/dev && terraform plan

# Use:
make plan ENV=dev
```

**Safety example**:

```bash
# Destroy command requires typing environment name to confirm
make destroy ENV=dev
# Output: "Are you sure? Type 'dev' to confirm:"
```

## Project structure

Certta will provide you with Terraform project files. The package structure:

```
allid-ecs-quickstart/
│
├── src/terraform/allid/              # All ID application (required)
│   ├── modules/
│   │   ├── ecs-cluster/              # ECS cluster + Service Discovery
│   │   ├── database/                 # Aurora MySQL Serverless v2
│   │   ├── load-balancer/            # Application Load Balancer
│   │   ├── peer-service/             # Peer Service (used 3 times)
│   │   └── facematch-service/        # Facematch Service
│   ├── environments/
│   │   ├── dev/
│   │   │   ├── main.tf               # Main configuration
│   │   │   ├── variables.tf          # Variable definitions
│   │   │   ├── outputs.tf            # Output definitions
│   │   │   └── terraform.tfvars      # Environment-specific values
│   │   ├── stg/
│   │   └── prd/
│   ├── Makefile                      # Automation commands
│   └── README.md
│
└── src/terraform/shared/             # Network infrastructure (optional)
    ├── modules/
    │   └── network/                  # VPC, subnets, NAT, IGW
    ├── environments/
    │   ├── dev/
    │   ├── stg/
    │   └── prd/
    ├── Makefile                      # Automation commands
    └── README.md
```

{% hint style="info" %}
**Shared infrastructure is optional**: Only deploy if you need to create a new VPC. If you already have a VPC, skip the `src/terraform/shared/` project and configure the VPC information via SSM Parameters.
{% endhint %}

### Main Files per Environment

Each environment directory contains:

* **`main.tf`**: Main configuration with provider settings and module invocations
* **`variables.tf`**: All variable definitions with types and defaults
* **`outputs.tf`**: Outputs of created resources (ALB DNS, database endpoint, etc.)
* **`terraform.tfvars.example`**: Template with example values and comments
* **`terraform.tfvars`**: Real values (not versioned in git, contains secrets)
* **`ssm.tf`** (shared only): SSM Parameters for cross-project references

## Step 1: Extract Terraform project

Extract the Terraform project files provided by Certta:

```bash
# Extract the package
tar -xzf allid-ecs-quickstart.tar.gz
cd allid-ecs-quickstart
```

## Step 2: Network infrastructure (Option A)

{% hint style="info" %}
**Choose your option**:

* **This step (Option A)**: Deploy a new VPC using the Certta reference Terraform project
* [**Step 2B (Option B)**](#step-2b-using-existing-vpc-option-b): Use your existing VPC by creating SSM Parameters manually
  {% endhint %}

### Option A: Deploy new VPC with Terraform

If you need to create a new VPC, deploy the reference network infrastructure:

```bash
# Navigate to shared project
cd src/terraform/shared

# Configure variables (optional, defaults are usually fine)
cp environments/dev/terraform.tfvars.example environments/dev/terraform.tfvars
# Edit environments/dev/terraform.tfvars if needed

# Initialize Terraform
make init ENV=dev

# Set environment variables
export AWS_REGION=us-east-1

# Review what will be created
make plan ENV=dev

# Deploy network infrastructure
make apply ENV=dev
```

**What gets deployed**:

* VPC with environment-specific CIDR blocks
* 2 Public subnets (across 2 availability zones)
* 2 Private subnets with NAT (across 2 availability zones)
* 2 Isolated subnets (across 2 availability zones) for databases
* Internet Gateway for public internet access
* NAT Gateway for private subnet outbound access
* SSM Parameters for cross-project references

**Deployment time**: \~5-10 minutes

After deployment, the VPC information is automatically stored in SSM Parameters:

```bash
# View created SSM Parameters
make output ENV=dev

# Or check directly
aws ssm get-parameter --name "/shared/dev/vpc-id" --query "Parameter.Value" --output text
```

The following SSM Parameters are created:

* `/shared/dev/vpc-id` - VPC ID
* `/shared/dev/vpc-cidr` - VPC CIDR block
* `/shared/dev/public-subnet-ids` - Public subnet IDs (comma-separated)
* `/shared/dev/private-egress-subnet-ids` - Private subnet IDs with NAT (comma-separated)
* `/shared/dev/private-isolated-subnet-ids` - Isolated subnet IDs (comma-separated)

{% hint style="info" %}
The All ID Terraform project automatically reads these SSM Parameters - no manual configuration needed!
{% endhint %}

## Step 2B: Using Existing VPC (Option B)

{% hint style="info" %}
**Skip this step if you chose Option A** (deploying new VPC with terraform-shared).
{% endhint %}

If you **already have a VPC** and **don't want to use terraform-shared**, you can create the SSM Parameters manually so the All ID Terraform project can find your VPC.

**Prerequisites**: Your VPC must have:

* Public subnets (with route to Internet Gateway)
* Private subnets with egress (with route to NAT Gateway)
* Private isolated subnets (no internet route, for databases)

### Get your VPC and subnet IDs

Use AWS CLI to identify your VPC and subnets:

```bash
# List VPCs
aws ec2 describe-vpcs --query 'Vpcs[*].[VpcId,CidrBlock,Tags[?Key==`Name`].Value|[0]]' --output table

# List Subnets from a specific VPC
aws ec2 describe-subnets --filters "Name=vpc-id,Values=vpc-xxxxx" \
  --query 'Subnets[*].[SubnetId,CidrBlock,AvailabilityZone,Tags[?Key==`Name`].Value|[0]]' \
  --output table

# View routes of a subnet to identify its type
aws ec2 describe-route-tables --filters "Name=association.subnet-id,Values=subnet-xxxxx" \
  --query 'RouteTables[*].Routes' --output table
```

### Identify each subnet type

* **Public Subnets**: Has route `0.0.0.0/0 → igw-xxxxx` (Internet Gateway)
* **Private Egress Subnets**: Has route `0.0.0.0/0 → nat-xxxxx` (NAT Gateway)
* **Private Isolated Subnets**: NO route to `0.0.0.0/0`

### Create SSM Parameters

Once you've identified your VPC and subnets, create the SSM Parameters:

```bash
# Configure variables with your real values
ENV=dev
VPC_ID=vpc-xxxxx                    # Your VPC ID
VPC_CIDR=172.31.0.0/16             # Your VPC CIDR
PUB_SUBNETS=subnet-111,subnet-222   # Public subnets (comma-separated, no spaces)
PRIV_EGRESS=subnet-333,subnet-444   # Private subnets with NAT
PRIV_ISOLATED=subnet-555,subnet-666 # Private isolated subnets

# Create all required SSM Parameters
aws ssm put-parameter --name "/shared/$ENV/vpc-id" --value "$VPC_ID" --type String --overwrite && \
aws ssm put-parameter --name "/shared/$ENV/vpc-cidr" --value "$VPC_CIDR" --type String --overwrite && \
aws ssm put-parameter --name "/shared/$ENV/public-subnet-ids" --value "$PUB_SUBNETS" --type StringList --overwrite && \
aws ssm put-parameter --name "/shared/$ENV/private-egress-subnet-ids" --value "$PRIV_EGRESS" --type StringList --overwrite && \
aws ssm put-parameter --name "/shared/$ENV/private-isolated-subnet-ids" --value "$PRIV_ISOLATED" --type StringList --overwrite && \
echo "✅ SSM Parameters created successfully!"
```

{% hint style="danger" %}
**Important**:

* Use subnet IDs separated by comma **WITHOUT SPACES** (e.g., `subnet-111,subnet-222`)
* Ensure subnets are in different AZs for high availability
* Subnet types (public, private-egress, private-isolated) must correspond to configured routes
  {% endhint %}

### Verify parameters were created

```bash
# List all parameters
aws ssm get-parameters-by-path --path "/shared/dev" --recursive

# View individual values
aws ssm get-parameter --name "/shared/dev/vpc-id"
aws ssm get-parameter --name "/shared/dev/public-subnet-ids"
```

### Delete parameters (if you need to redo)

```bash
ENV=dev
aws ssm delete-parameters --names \
  "/shared/$ENV/vpc-id" \
  "/shared/$ENV/vpc-cidr" \
  "/shared/$ENV/public-subnet-ids" \
  "/shared/$ENV/private-egress-subnet-ids" \
  "/shared/$ENV/private-isolated-subnet-ids"
```

{% hint style="success" %}
**Which option to choose?**

* ✅ **Option A (terraform-shared)**: Recommended for new projects or when you want to manage the VPC with Terraform
* 🔧 **Option B (Manual)**: Use if you already have a VPC and don't want to migrate to Terraform yet
  {% endhint %}

## Step 3: Configure application variables

Navigate to the application directory:

```bash
cd src/terraform/allid
```

Copy the example variables file and configure your environment-specific values:

```bash
cp environments/dev/terraform.tfvars.example environments/dev/terraform.tfvars
```

Edit `environments/dev/terraform.tfvars` and configure the following values:

```hcl
# General Configuration
aws_region  = "us-east-1"
environment = "dev"
prefix      = "allid"  # Change this to your project name (default: teste-tf)

# Container Images
peer_ecr_repository_uri      = "211125355658.dkr.ecr.us-east-1.amazonaws.com/peer-v2"
peer_version                 = "v2.5.0"
facematch_ecr_repository_uri = "211125355658.dkr.ecr.us-east-1.amazonaws.com/facematch"
facematch_version            = "7d88b2c4803add026000f97ea7913f1297f6e786"

# Router Configuration (varies by environment)
# dev: https://mtls.us.dev.caf.io/v1/biometrics/facial-validation
# stg: https://mtls.us.stg.caf.io/v1/biometrics/facial-validation  
# prd: https://mtls.us.prd.caf.io/v1/biometrics/facial-validation
router_rest_url = "https://mtls.us.prd.caf.io/v1/biometrics/facial-validation"

# Note: Router mTLS certificates are NOT configured here
# They will be added to AWS Secrets Manager after deployment (see Step 5)

# Service Configuration
desired_count = 1

# Database Configuration
database_name                   = "db"
database_username               = "dbadmin"
database_min_capacity           = 0
database_max_capacity           = 2
database_backup_retention_period = 7
database_deletion_protection    = false

# Load Balancer Configuration
alb_allowed_cidr_blocks = [
  "3.218.90.124/32",     # Certta Router IPs
  "44.219.96.170/32",
  "18.235.54.162/32",
  "18.228.123.114/32",
  "54.232.24.70/32",
  "54.94.8.234/32"
]

# Logging Configuration
log_retention_days = 7
enable_ecs_exec    = true
```

**Configuration values to update**:

| Field                          | Description                                | How to obtain                                    |
| ------------------------------ | ------------------------------------------ | ------------------------------------------------ |
| `peer_ecr_repository_uri`      | Peer Service container registry            | Provided by Certta                               |
| `peer_version`                 | Peer image tag to deploy                   | Provided by Certta (e.g., `latest`, commit hash) |
| `facematch_ecr_repository_uri` | Facematch container registry               | Provided by Certta                               |
| `facematch_version`            | Facematch image tag to deploy              | Provided by Certta (e.g., `latest`, commit hash) |
| `desired_count`                | Number of tasks per peer service           | Set to 0 for dev (cost saving), 1+ for stg/prd   |
| `router_rest_url`              | Certta Router endpoint for your region/env | Provided by Certta                               |

{% hint style="info" %}
**About `desired_count`**: This variable controls the number of tasks for **all three Peer services** simultaneously (default, client-a, client-b). This ensures consistency across all peers.

**Recommended values**:

* **Development**: `0` (no tasks running - saves costs when not in use)
* **Staging**: `1` (one task per peer for testing)
* **Production**: `2+` (multiple tasks per peer for high availability)

**Note**: Facematch service is hardcoded to 2 tasks. To change, edit the `facematch` module in `environments/<env>/main.tf`.
{% endhint %}

{% hint style="info" %}
**Router mTLS Certificates**: The certificates are NOT configured in `terraform.tfvars`. Terraform creates empty secrets in AWS Secrets Manager, and you will populate them with actual certificates after deployment in Step 5.

**Alternative approach**: You can include certificates in `terraform.tfvars` if preferred, but be aware:

* Values will be stored in Terraform state files (even if marked as sensitive)
* State files must be encrypted and access-controlled
* This approach is optional and not recommended for security reasons
  {% endhint %}

{% hint style="info" %}
**Multiple environments**: Create separate `terraform.tfvars` files for each environment (dev, stg, prd) with environment-specific values.
{% endhint %}

## Step 4: Deploy All ID application

Deploy the All ID application services:

```bash
# Make sure you're in the correct directory
cd src/terraform/allid

# Initialize Terraform (first time only)
make init ENV=dev

# Review what will be created
make plan ENV=dev

# Deploy all application resources
make apply ENV=dev
```

Type `yes` when prompted to confirm the deployment.

**What gets deployed**:

1. **ECS Cluster + Service Discovery** (`ecs-cluster` module)
   * ECS Fargate cluster
   * Cloud Map private namespace (`allid.local`)
   * Container Insights enabled
2. **Database** (`database` module)
   * Aurora MySQL Serverless v2 cluster
   * Database security group
   * Secrets Manager for credentials (auto-generated)
   * CloudWatch Logs for error, general, and slow query logs
   * Performance Insights enabled
3. **Load Balancer** (`load-balancer` module)
   * Application Load Balancer (internet-facing)
   * HTTP listener (port 80)
   * Security group with IP whitelisting
   * Default action: 403 Access Denied
4. **Facematch Service** (`facematch-service` module)
   * ECS Task Definition (CPU: 1024, Memory: 2048 MB)
   * Fargate Service with 2 instances
   * Cloud Map service registration (`facematch.allid.local`)
   * Security group for internal communication
   * IAM roles for execution and task
5. **Peer Services** (`peer-service` module - instantiated 3 times)
   * Creates 3 peer instances (default, client-a, client-b)
   * Each peer has:
     * ECS Task Definition (CPU: 256, Memory: 512 MB)
     * Fargate Service with configurable instance count
     * ALB target group with path-based routing
     * Cloud Map service registration
     * Database connection to RDS
     * Router mTLS certificate stored in Secrets Manager
     * IAM roles for execution and task
     * CloudWatch Log Group

**Deployment time**: \~15-20 minutes

{% hint style="info" %}
Terraform automatically handles resource dependencies and deploys them in the correct order. You can monitor the progress in the terminal output.
{% endhint %}

{% hint style="warning" %}
**Important**: After deployment, complete Step 5 (configure certificates) and Step 6 (initialize databases) to make the Peer Services operational.
{% endhint %}

## Step 5: Configure Router mTLS certificates

After deploying the infrastructure, you need to populate the Router mTLS certificates in AWS Secrets Manager. Terraform creates empty secrets that must be filled with actual certificate content.

{% hint style="info" %}
**Why certificates are not in Terraform**:

* Keeps sensitive credentials out of Terraform state files
* Allows certificate rotation without Terraform changes
* Follows security best practices for secrets management

**Alternative**: You can include certificates in `terraform.tfvars` if your security policies allow it, but this will store them in the Terraform state file.
{% endhint %}

### Configure certificates for each peer

**Option 1: AWS Console** (recommended)

1. Navigate to Secrets Manager Console → Secrets
2. Find and click on each secret:
   * `{prefix}-peer-default-router-certificate`
   * `{prefix}-peer-client-a-router-certificate`
   * `{prefix}-peer-client-b-router-certificate`
3. Click "Retrieve secret value" → "Edit"
4. Update the JSON with your actual certificates:

```json
{
  "private-key": "-----BEGIN PRIVATE KEY-----\nMIIE...content...\n-----END PRIVATE KEY-----",
  "certificate": "-----BEGIN CERTIFICATE-----\nMIID...content...\n-----END CERTIFICATE-----"
}
```

5. Save changes

**Option 2: AWS CLI**

```bash
# Update default peer certificate (replace with actual content)
aws secretsmanager update-secret \
  --secret-id {prefix}-peer-default-router-certificate \
  --secret-string '{
    "private-key": "-----BEGIN PRIVATE KEY-----\nYOUR_KEY_HERE\n-----END PRIVATE KEY-----",
    "certificate": "-----BEGIN CERTIFICATE-----\nYOUR_CERT_HERE\n-----END CERTIFICATE-----"
  }'

# Update client-a peer certificate
aws secretsmanager update-secret \
  --secret-id {prefix}-peer-client-a-router-certificate \
  --secret-string '{
    "private-key": "-----BEGIN PRIVATE KEY-----\nYOUR_KEY_HERE\n-----END PRIVATE KEY-----",
    "certificate": "-----BEGIN CERTIFICATE-----\nYOUR_CERT_HERE\n-----END CERTIFICATE-----"
  }'

# Update client-b peer certificate
aws secretsmanager update-secret \
  --secret-id {prefix}-peer-client-b-router-certificate \
  --secret-string '{
    "private-key": "-----BEGIN PRIVATE KEY-----\nYOUR_KEY_HERE\n-----END PRIVATE KEY-----",
    "certificate": "-----BEGIN CERTIFICATE-----\nYOUR_CERT_HERE\n-----END CERTIFICATE-----"
  }'
```

{% hint style="warning" %}
**Important**: Replace `{prefix}` with your actual prefix value (default is `teste-tf` or what you configured in `terraform.tfvars`).
{% endhint %}

{% hint style="success" %}
After updating the secrets, the Peer Services will automatically detect and use the certificates. No restart is required.
{% endhint %}

## Step 6: Initialize database

The Peer Service requires a database schema to be initialized. Certta will provide a SQL dump file that must be restored to the Aurora MySQL database.

{% hint style="danger" %}
**Critical**: The Peer Services may fail to start if the database schema is not initialized. Complete this step to allow the services to become operational.
{% endhint %}

{% hint style="info" %}
**Multi-tenant setup**: Each peer instance uses its own database. The database names follow the pattern `peer-{name}`:

* `peer-default` (for default peer)
* `peer-client-a` (for client-a peer)
* `peer-client-b` (for client-b peer)
  {% endhint %}

### Database access methods

The Aurora MySQL database is deployed in isolated subnets with no direct internet access. You need to establish a secure connection to access it.

**Option 1: Bastion Host (Recommended for production)**

Deploy a bastion host (EC2 instance) in a public subnet to act as a jump server:

1. Launch an EC2 instance in a public subnet of your VPC
2. Configure security groups to allow:
   * SSH access from your IP to the bastion host
   * MySQL access from bastion host to the RDS security group
3. Connect to the database through SSH tunnel:

```bash
# SSH tunnel to bastion host
ssh -i your-key.pem -L 3306:DATABASE_ENDPOINT:3306 ec2-user@BASTION_IP

# In another terminal, connect to database via localhost
mysql -h 127.0.0.1 -u DB_USERNAME -p
```

**Option 2: VPN Connection**

If you have a VPN connection configured to your VPC:

1. Connect to your VPN
2. Access the database directly using its private endpoint

**Option 3: AWS Systems Manager Session Manager**

Use Session Manager for secure access without exposing SSH ports:

1. Ensure your bastion host has SSM agent installed
2. Grant necessary IAM permissions
3. Create port forwarding session:

```bash
aws ssm start-session \
  --target INSTANCE_ID \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{"portNumber":["3306"],"localPortNumber":["3306"],"host":["DATABASE_ENDPOINT"]}'
```

### Get database endpoint and credentials

**Option 1: Terraform outputs**

```bash
# From the allid directory
cd src/terraform/allid

# Get database endpoint and secret ARN
make output ENV=dev

# Or navigate to environment directory for specific outputs
cd environments/dev
SECRET_ARN=$(terraform output -raw database_secret_arn)
aws secretsmanager get-secret-value --secret-id $SECRET_ARN --query 'SecretString' --output text | jq
```

**Option 2: AWS Console**

1. Navigate to RDS Console → Databases
2. Find the Aurora cluster (look for name with `allid-database`)
3. Copy the **Writer endpoint** (e.g., `allid-database-cluster.cluster-xxx.us-east-1.rds.amazonaws.com`)
4. Navigate to Secrets Manager Console → Secrets
5. Find the database secret (look for name with `allid-database`)
6. Click "Retrieve secret value" to see username and password

**Option 3: AWS CLI**

```bash
# List RDS clusters to find your database
aws rds describe-db-clusters --query 'DBClusters[*].[DBClusterIdentifier, Endpoint, Port]' --output table

# List secrets to find database credentials
aws secretsmanager list-secrets --query 'SecretList[?contains(Name, `allid-database`)].[Name, ARN]' --output table

# Get specific secret value (replace SECRET_ARN with actual ARN)
aws secretsmanager get-secret-value --secret-id SECRET_ARN --query 'SecretString' --output text | jq
```

### Create and initialize databases

You can use any of the following tools to create databases and restore the dump:

#### Option A: MySQL Command Line Client

Best for automation and CI/CD pipelines:

```bash
# Connect to Aurora MySQL (replace with your endpoint, username, and password from above)
mysql -h {DB_ENDPOINT} -u {DB_USERNAME} -p

# Create databases
CREATE DATABASE IF NOT EXISTS `peer-default`;
CREATE DATABASE IF NOT EXISTS `peer-client-a`;
CREATE DATABASE IF NOT EXISTS `peer-client-b`;
EXIT;

# Restore the dump to each database (Certta will provide the SQL dump file)
mysql -h {DB_ENDPOINT} -u {DB_USERNAME} -p peer-default < allid-dump.sql
mysql -h {DB_ENDPOINT} -u {DB_USERNAME} -p peer-client-a < allid-dump.sql
mysql -h {DB_ENDPOINT} -u {DB_USERNAME} -p peer-client-b < allid-dump.sql
```

#### Option B: DBeaver (GUI Tool)

Recommended for visual database management:

1. **Create a new connection**:
   * Database: MySQL
   * Host: `{DB_ENDPOINT}` (from above)
   * Port: 3306
   * Username: `{DB_USERNAME}` (from Secrets Manager)
   * Password: `{DB_PASSWORD}` (from Secrets Manager)
2. **Create databases**:

   * Right-click on connection → SQL Editor → New SQL Script
   * Execute:

   ```sql
   CREATE DATABASE IF NOT EXISTS `peer-default`;
   CREATE DATABASE IF NOT EXISTS `peer-client-a`;
   CREATE DATABASE IF NOT EXISTS `peer-client-b`;
   ```
3. **Restore dump**:
   * Right-click on each database → Tools → Execute Script
   * Select the `allid-dump.sql` file provided by Certta
   * Click "Start" to execute

#### Option C: MySQL Workbench (GUI Tool)

Alternative GUI tool for MySQL management:

1. **Create a new connection**:
   * Connection Name: `All ID Database`
   * Hostname: `{DB_ENDPOINT}`
   * Port: 3306
   * Username: `{DB_USERNAME}`
   * Password: Store in Keychain/Vault
2. **Create databases**:

   * Open connection → Query tab
   * Execute:

   ```sql
   CREATE DATABASE IF NOT EXISTS `peer-default`;
   CREATE DATABASE IF NOT EXISTS `peer-client-a`;
   CREATE DATABASE IF NOT EXISTS `peer-client-b`;
   ```
3. **Import dump**:
   * Server → Data Import
   * Select "Import from Self-Contained File"
   * Choose `allid-dump.sql` file
   * Select target database (repeat for each: peer-default, peer-client-a, peer-client-b)
   * Click "Start Import"

{% hint style="info" %}
**Connection troubleshooting**: If you cannot connect to the database, ensure:

* You have established proper access (bastion host, VPN, or Session Manager)
* The RDS security group allows connections from your source
* The database endpoint and credentials are correct
  {% endhint %}

{% hint style="danger" %}
**Critical**: Each Peer Service will fail to start if its corresponding database is not initialized. Contact your Certta technical account manager to obtain the SQL dump file.
{% endhint %}

{% hint style="success" %}
**Services will auto-recover**: Once you initialize the databases, the Peer Services will automatically restart and become healthy within a few minutes. ECS will detect that tasks are failing and restart them automatically.
{% endhint %}

## Step 7: Retrieve endpoints

After deployment completes, get the application endpoint:

```bash
# View all outputs
make output ENV=dev

# Get specific outputs (navigate to environment directory first)
cd environments/dev
terraform output alb_dns_name
terraform output database_endpoint
terraform output database_secret_arn
terraform output ecs_cluster_name
```

**Important outputs**:

| Output                      | Description                            | Example                                                          |
| --------------------------- | -------------------------------------- | ---------------------------------------------------------------- |
| `alb_dns_name`              | Load Balancer DNS name                 | `allid-load-balancer-123456789.us-east-1.elb.amazonaws.com`      |
| `database_endpoint`         | Aurora MySQL cluster endpoint          | `allid-database-cluster.cluster-xxx.us-east-1.rds.amazonaws.com` |
| `database_secret_arn`       | Secrets Manager ARN for DB credentials | `arn:aws:secretsmanager:...`                                     |
| `ecs_cluster_name`          | ECS cluster name                       | `allid-cluster`                                                  |
| `peer_default_service_name` | Peer default service name              | `allid-peer-default`                                             |
| `facematch_service_name`    | Facematch service name                 | `allid-facematch`                                                |

{% hint style="info" %}
The deployment creates 3 peer instances with path-based routing for multi-tenant support. Each peer has its own isolated endpoint with path prefix.
{% endhint %}

**API endpoints** (3 peer instances by default):

```
# Default peer
http://{alb-dns}/default/v1/biometric-validation-responder

# Client A peer
http://{alb-dns}/client-a/v1/biometric-validation-responder

# Client B peer
http://{alb-dns}/client-b/v1/biometric-validation-responder
```

**Health check endpoints**:

```
http://{alb-dns}/default/status
http://{alb-dns}/client-a/status
http://{alb-dns}/client-b/status
```

{% hint style="warning" %}
To add/remove peers, edit the environment `main.tf` file and add/remove peer modules. Remember to create the corresponding database and provide Router certificates for new peers BEFORE deploying.
{% endhint %}

## Step 8: Validate deployment

Test that all services are healthy:

**Test peer endpoints** (using ALB DNS from Step 7):

```bash
# Get ALB DNS from outputs (from src/terraform/allid/environments/dev directory)
cd environments/dev
ALB_DNS=$(terraform output -raw alb_dns_name)

curl -f http://${ALB_DNS}/default/status
curl -f http://${ALB_DNS}/client-a/status
curl -f http://${ALB_DNS}/client-b/status
```

All endpoints should return HTTP 200 with status information.

**Check ECS services status**:

**AWS Console**:

1. Navigate to ECS Console → Clusters → `allid-cluster`
2. Click on "Services" tab
3. Verify all services show "Running" status and desired count matches running count

**AWS CLI**:

```bash
# Get cluster name from Terraform
CLUSTER_NAME=$(terraform output -raw ecs_cluster_name)

# List all services in the cluster
aws ecs list-services --cluster $CLUSTER_NAME --output table

# Check detailed status
aws ecs describe-services \
  --cluster $CLUSTER_NAME \
  --services allid-peer-default allid-peer-client-a allid-peer-client-b allid-facematch \
  --query 'services[*].[serviceName, runningCount, desiredCount, deployments[0].status]' \
  --output table
```

All services should show `runningCount` matching `desiredCount` and deployment status `PRIMARY`.

**Check CloudWatch Logs** (if services fail to start):

**AWS Console**:

1. Navigate to CloudWatch Console → Log groups
2. Find log groups: `/ecs/allid/peer-default`, `/ecs/allid/facematch`, etc.
3. Check recent log streams for errors

**AWS CLI**:

```bash
# List log groups
aws logs describe-log-groups --log-group-name-prefix /ecs/allid --output table

# Tail specific log (replace log group name)
aws logs tail /ecs/allid/peer-default --follow
```

## Makefile Commands

Both `src/terraform/allid/` and `src/terraform/shared/` include a Makefile with automation commands for easier infrastructure management.

{% hint style="info" %}
**Getting Started**: Run `make help` in either project directory to see all available commands and their descriptions.
{% endhint %}

### Available Commands

```bash
# Show help and available commands
make help

# Initialize Terraform
make init ENV=dev

# Validate Terraform configuration
make validate ENV=dev

# Plan changes
make plan ENV=dev

# Apply changes
make apply ENV=dev

# Destroy infrastructure (requires confirmation)
make destroy ENV=dev

# Show outputs
make output ENV=dev

# Format Terraform files
make fmt

# Check Terraform file formatting
make fmt-check

# Clean Terraform state and cache
make clean ENV=dev

# Clean all environments
make clean-all

# Upgrade Terraform providers
make upgrade ENV=dev

# List resources in Terraform state
make state-list ENV=dev

# Show a specific resource from state
make state-show ENV=dev RESOURCE=module.database.aws_rds_cluster.main

# Taint a resource to force recreation
make taint ENV=dev RESOURCE=module.peer_default.aws_ecs_service.main

# Import existing infrastructure
make import ENV=dev RESOURCE=module.database.aws_rds_cluster.main ID=cluster-id
```

### Environment Selection

All commands support the `ENV` parameter to specify the target environment:

```bash
# Development
make plan ENV=dev

# Staging
make plan ENV=stg

# Production
make plan ENV=prd
```

{% hint style="info" %}
**Default environment**: If you don't specify `ENV`, the Makefile defaults to `dev`.
{% endhint %}

### Shared Infrastructure Commands

The shared infrastructure Makefile includes an additional command to view SSM parameters:

```bash
# Show SSM parameters created by shared infrastructure
cd src/terraform/shared
make ssm-params ENV=dev
```

### Workflow Example

Here's a typical workflow using Makefile commands:

```bash
# 1. Deploy shared infrastructure (VPC)
cd src/terraform/shared
make init ENV=dev
make plan ENV=dev
make apply ENV=dev

# 2. Deploy All ID application
cd ../allid
make init ENV=dev
make plan ENV=dev
make apply ENV=dev

# 3. View outputs
make output ENV=dev

# 4. Later: Update and redeploy
# Edit environments/dev/terraform.tfvars (e.g., change image versions)
make plan ENV=dev
make apply ENV=dev

# 5. Scale services
# Edit environments/dev/terraform.tfvars or main.tf
make plan ENV=dev
make apply ENV=dev

# 6. Cleanup when done
make destroy ENV=dev
cd ../shared
make destroy ENV=dev
```

## Quick Reference

### Most Common Commands

```bash
# View help
cd src/terraform/allid  # or src/terraform/shared
make help

# Basic workflow
make init ENV=dev
make plan ENV=dev
make apply ENV=dev
make output ENV=dev

# Update infrastructure
# Edit environments/dev/terraform.tfvars
make plan ENV=dev
make apply ENV=dev

# View specific output
cd environments/dev
terraform output alb_dns_name

# Validate configuration
make validate ENV=dev

# Format Terraform files
make fmt

# List resources in state
make state-list ENV=dev

# View resource details
make state-show ENV=dev RESOURCE=module.database.aws_rds_cluster.this

# Cleanup
make destroy ENV=dev
```

### Environment Management

```bash
# All commands support ENV parameter
make plan ENV=dev     # Development
make apply ENV=stg    # Staging  
make output ENV=prd   # Production

# Default is dev if not specified
make plan            # Same as: make plan ENV=dev
```

### Common Tasks

**Deploy to new environment**:

```bash
cd src/terraform/shared
make init ENV=stg
make apply ENV=stg

cd ../allid
cp environments/dev/terraform.tfvars environments/stg/terraform.tfvars
# Edit environments/stg/terraform.tfvars with environment-specific values
make init ENV=stg
make apply ENV=stg
```

**Update container images**:

```bash
cd src/terraform/allid
# Edit environments/dev/terraform.tfvars
# Update: peer_version = "new-version"
make plan ENV=dev
make apply ENV=dev
```

**Scale services**:

```bash
cd src/terraform/allid
# Edit environments/dev/terraform.tfvars
# Update: desired_count = 2
make plan ENV=dev
make apply ENV=dev
```

**View logs**:

```bash
# Peer logs
aws logs tail /ecs/allid/peer-default --follow

# Facematch logs
aws logs tail /ecs/allid/facematch --follow

# Filter by pattern
aws logs tail /ecs/allid/peer-default --follow --filter-pattern "ERROR"
```

## Architecture overview

### Network layers

The deployment creates a three-tier network architecture:

```
┌─────────────────────────────────────────┐
│ Public Subnets (2 AZs)                  │
│ • Application Load Balancer             │
│ • Internet Gateway                      │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│ Private Subnets (2 AZs)                 │
│ • Peer Service (ECS Fargate)            │
│ • Facematch Service (ECS Fargate)       │
│ • NAT Gateway (outbound internet)       │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│ Isolated Subnets (2 AZs)                │
│ • Aurora MySQL Database                 │
│ • No internet access                    │
└─────────────────────────────────────────┘
```

### Service communication

```
Internet → ALB (HTTP:80) → Peer (8080) → Facematch (8080)
                              ↓
                         Database (3306)
```

### Path-based routing (multi-tenant)

The ALB uses path-based routing to support multiple peer instances:

```
/default/*   → peer-default   (Target Group 1)
/client-a/*  → peer-client-a  (Target Group 2)
/client-b/*  → peer-client-b  (Target Group 3)
```

Each peer instance:

* Has its own ECS service
* Has its own isolated database (peer-default, peer-client-a, peer-client-b)
* Operates independently from other peers
* Shares the same Facematch service pool

## Configuration management

The Terraform modules automatically configure environment variables and secrets for all services.

**Environment variables** are defined in the `peer-service` module and include:

* Database connection (host, port, database name)
* Facematch service endpoint
* Router Service URL
* Feature flags (RabbitMQ, Redis, Router communication)

**Secrets** are injected via ECS task secrets and include:

* Database credentials (auto-generated by Terraform)
* Router mTLS certificates (provided via terraform.tfvars)

{% hint style="info" %}
See [Configuration](/caf-api/all-id/configuration.md) for complete list of environment variables, required values, and configuration details.
{% endhint %}

### Secrets management

Terraform automatically creates secrets in AWS Secrets Manager:

**Secret naming convention**:

* Database credentials: `{prefix}-database-secret` (e.g., `allid-database-secret`)
  * Auto-generated by Terraform during deployment
* Router certificates: `{prefix}-peer-{name}-router-certificate` (e.g., `allid-peer-default-router-certificate`)
  * Created empty by Terraform, populated manually after deployment (see [Step 5](#step-5-configure-router-mtls-certificates))

{% hint style="info" %}
**Router certificates security approach**:

By default, Router certificate secrets are created empty and populated manually after deployment. This approach:

* ✅ Keeps sensitive credentials out of Terraform state files
* ✅ Allows certificate rotation without Terraform changes
* ✅ Follows security best practices for secrets management

**Alternative**: You can include certificates in `terraform.tfvars` if needed, but be aware that they will be stored in Terraform state files (even if marked as sensitive). If you choose this approach:

* Store state files in encrypted S3 buckets with restricted access
* Use state locking with DynamoDB
* Never commit state files to version control
* Limit access to state files to authorized personnel only
  {% endhint %}

## Security

Terraform automatically configures security groups and IP whitelisting following least-privilege principles.

**Security groups**:

* ALB accepts traffic only from whitelisted IPs
* Peer Services accept traffic only from ALB and internal VPC
* Facematch accepts traffic only from Peer Services
* Database accepts traffic only from Peer Services

**IP whitelisting**:

* ALB is configured to accept traffic only from Certta Router IP addresses
* Configure additional IPs in `terraform.tfvars` via the `alb_allowed_cidr_blocks` variable

{% hint style="info" %}
See [Security Best Practices](/caf-api/all-id/security-best-practices.md) for:

* Complete list of Certta Router IP addresses
* Network segmentation recommendations
* Secrets management best practices
* Additional security hardening options
  {% endhint %}

## Updating the deployment

### Update container images

1. Update image versions in `terraform.tfvars`:

```hcl
peer_version      = "new-commit-hash"
facematch_version = "new-commit-hash"
```

2. Apply the changes:

```bash
cd src/terraform/allid

# Review changes
make plan ENV=dev

# Apply changes
make apply ENV=dev
```

Terraform will update the ECS task definitions and trigger a rolling deployment.

### Update environment variables

1. Edit the `peer-service` module in `src/terraform/allid/modules/peer-service/main.tf`
2. Modify the environment variables in the task definition
3. Apply the changes:

```bash
cd src/terraform/allid
make plan ENV=dev
make apply ENV=dev
```

This will update all peer instances (default, client-a, client-b) with the new configuration.

### Rotate Router mTLS certificates

If you need to rotate the Router mTLS certificates (certificate expiration, security incident, etc.), follow the same process as the initial configuration in [Step 5](#step-5-configure-router-mtls-certificates).

{% hint style="info" %}
**When to rotate certificates**:

* Certificate expiration/renewal
* Security incident requiring certificate replacement
* Moving between Certta environments (dev/stg/prd)
* Compliance requirements for periodic rotation

The application will automatically pick up the new certificates from Secrets Manager without requiring a service restart.
{% endhint %}

### Scale services

**Scale Peer Services**: Edit `environments/dev/terraform.tfvars` and change the `desired_count` variable:

```hcl
desired_count = 2  # Change from 1 to 2
```

Then apply:

```bash
cd src/terraform/allid
make plan ENV=dev
make apply ENV=dev
```

**Scale Facematch Service**: Edit the `facematch` module in `environments/dev/main.tf`:

```hcl
module "facematch" {
  # ... other configuration ...
  desired_count = 4  # Change from 2 to 4
}
```

Then apply:

```bash
cd src/terraform/allid
make plan ENV=dev
make apply ENV=dev
```

### Add/remove peer instances

To add a new peer instance:

1. **Edit the environment `main.tf`** file (e.g., `src/terraform/allid/environments/dev/main.tf`):

```hcl
# Add new peer module
module "peer_client_c" {
  source = "../../modules/peer-service"

  prefix                   = var.prefix
  peer_name                = "client-c"
  vpc_id                   = local.vpc_id
  vpc_cidr_block           = local.vpc_cidr_block
  subnet_ids               = local.private_egress_subnet_ids
  cluster_id               = module.ecs_cluster.cluster_id
  namespace_id             = module.ecs_cluster.namespace_id
  namespace_name           = module.ecs_cluster.namespace_name
  listener_arn             = module.load_balancer.http_listener_arn
  alb_security_group_id    = module.load_balancer.security_group_id
  database_host            = module.database.cluster_endpoint
  database_port            = module.database.cluster_port
  database_name            = module.database.database_name
  database_secret_arn      = module.database.secret_arn
  ecr_repository_uri       = var.peer_ecr_repository_uri
  image_version            = var.peer_version
  router_rest_url          = var.router_rest_url
  router_certificate       = var.router_certificate
  router_private_key       = var.router_private_key
  cpu                      = 256
  memory                   = 512
  desired_count            = var.desired_count
  priority_base            = 190  # Use next available priority block
  enable_private_endpoints = false
  enable_ecs_exec          = var.enable_ecs_exec
  anonymization_enabled    = true
  cache_enabled            = true
  log_retention_days       = var.log_retention_days

  tags = {
    environment = var.environment
  }

  depends_on = [module.facematch]
}
```

2. **Add output for the new peer** in `outputs.tf`:

```hcl
output "peer_client_c_service_name" {
  description = "Name of the Peer Client C ECS service"
  value       = module.peer_client_c.service_name
}
```

3. **Apply the changes**:

```bash
cd src/terraform/allid
make plan ENV=dev
make apply ENV=dev
```

4. **Create and initialize the database** for the new peer (see Step 6):

```bash
# Connect to database
mysql -h {DB_ENDPOINT} -u {DB_USERNAME} -p

# Create database
CREATE DATABASE IF NOT EXISTS `peer-client-c`;
EXIT;

# Restore the dump
mysql -h {DB_ENDPOINT} -u {DB_USERNAME} -p peer-client-c < allid-dump.sql
```

{% hint style="info" %}
The new Peer Service will automatically have its Router certificate secret created by Terraform using the same certificate from `terraform.tfvars`. The service will become healthy after you complete database initialization.
{% endhint %}

{% hint style="warning" %}
**Priority ranges**: Each peer must use a unique `priority_base` value for ALB listener rules. Use ranges of 30 (e.g., 100-129, 130-159, 160-189, 190-219) to avoid conflicts.
{% endhint %}

## Troubleshooting

### Error: SSM Parameter not found

**Cause**: Network infrastructure (src/terraform/shared) has not been deployed, or SSM Parameters were not created.

**Solution**:

1. **Verify if parameters exist**:

```bash
aws ssm get-parameter --name "/shared/dev/vpc-id" --query "Parameter.Value" --output text
```

2. **If parameters don't exist**, deploy the shared infrastructure:

```bash
cd src/terraform/shared
make init ENV=dev
make apply ENV=dev
```

3. **If you have an existing VPC**, create SSM Parameters manually:

```bash
# Replace with your actual values
aws ssm put-parameter --name "/shared/dev/vpc-id" --value "vpc-xxxxx" --type "String"
aws ssm put-parameter --name "/shared/dev/vpc-cidr" --value "10.0.0.0/16" --type "String"
aws ssm put-parameter --name "/shared/dev/public-subnet-ids" --value "subnet-aaa,subnet-bbb" --type "String"
aws ssm put-parameter --name "/shared/dev/private-egress-subnet-ids" --value "subnet-ccc,subnet-ddd" --type "String"
aws ssm put-parameter --name "/shared/dev/private-isolated-subnet-ids" --value "subnet-eee,subnet-fff" --type "String"
```

### Peer Service won't start

**Check CloudWatch Logs**:

```bash
# Tail logs for specific peer
aws logs tail /ecs/allid/peer-default --follow
```

**Common causes**:

1. **Database not initialized**: See Step 6 (database initialization and dump restoration)
2. **Database connection failed**: Check security groups and RDS endpoint
3. **Facematch service unavailable**: Check Facematch service status

**Check task stopped reason**:

**AWS Console**:

1. ECS Console → Clusters → allid-cluster
2. Click on service (e.g., allid-peer-default)
3. Go to "Tasks" tab → Click on stopped tasks
4. Check "Stopped reason" field

**AWS CLI**:

```bash
CLUSTER_NAME=$(terraform output -raw ecs_cluster_name)

aws ecs describe-tasks \
  --cluster $CLUSTER_NAME \
  --tasks $(aws ecs list-tasks --cluster $CLUSTER_NAME --service-name allid-peer-default --query 'taskArns[0]' --output text) \
  --query 'tasks[0].stoppedReason'
```

### Health check failures

**Check ECS service status**:

```bash
CLUSTER_NAME=$(terraform output -raw ecs_cluster_name)

# Check service status
aws ecs describe-services \
  --cluster $CLUSTER_NAME \
  --services allid-peer-default allid-peer-client-a allid-peer-client-b allid-facematch \
  --query 'services[*].[serviceName, runningCount, desiredCount, healthCheckGracePeriodSeconds]' \
  --output table
```

**Check ALB target health**:

**AWS CLI**:

```bash
# List target groups
aws elbv2 describe-target-groups \
  --query 'TargetGroups[?contains(TargetGroupName, `allid`)][TargetGroupName, TargetGroupArn]' \
  --output table

# Check health for a specific target group (replace TARGET_GROUP_ARN)
aws elbv2 describe-target-health --target-group-arn TARGET_GROUP_ARN
```

**Common issues**:

* Security groups blocking traffic between ALB and Peer Services
* Service not registered in Cloud Map (DNS resolution fails)
* Database connection errors (check credentials secret)
* Facematch service not responding
* Health check grace period not long enough

### No tasks running / Services showing 0/0

**Cause**: `desired_count` is set to 0 in `terraform.tfvars`.

**Solution**:

This is often intentional for development environments to save costs. To start tasks:

1. Edit `environments/dev/terraform.tfvars`:

```hcl
desired_count = 1  # Change from 0 to 1
```

2. Apply changes:

```bash
cd src/terraform/allid
make plan ENV=dev
make apply ENV=dev
```

3. Verify tasks are starting:

```bash
cd environments/dev
CLUSTER_NAME=$(terraform output -raw ecs_cluster_name)
aws ecs list-tasks --cluster $CLUSTER_NAME
```

{% hint style="info" %}
**Cost saving tip**: Set `desired_count = 0` when not actively using development environments to stop all ECS tasks and reduce infrastructure costs.
{% endhint %}

### Can't access ALB

**Check if your IP is whitelisted**:

```bash
ALB_DNS=$(terraform output -raw alb_dns_name)
curl -v http://${ALB_DNS}/default/status
```

If you get connection timeout or 403 Forbidden, your IP is not whitelisted.

**Add your IP to the allowlist**:

1. Edit `environments/dev/terraform.tfvars`:

```hcl
alb_allowed_cidr_blocks = [
  "3.218.90.124/32",     # Certta Router IPs
  "44.219.96.170/32",
  # ... other IPs ...
  "YOUR_IP_HERE/32",     # Add your IP
]
```

2. Apply changes:

```bash
cd src/terraform/allid
make plan ENV=dev
make apply ENV=dev
```

{% hint style="info" %}
See [Security Best Practices](/caf-api/all-id/security-best-practices.md) for the complete list of Certta Router IP addresses that should be whitelisted.
{% endhint %}

## Cleanup

To remove all resources and stop incurring costs:

```bash
# Destroy All ID application
cd src/terraform/allid
make destroy ENV=dev

# Destroy shared infrastructure (only if you deployed it)
cd ../shared
make destroy ENV=dev
```

{% hint style="info" %}
**Targeted destruction**: If you need to destroy specific resources in order, you can navigate to the environment directory and use targeted destroy:

```bash
cd src/terraform/allid/environments/dev
terraform destroy -target=module.peer_default
terraform destroy -target=module.peer_client_a
terraform destroy -target=module.peer_client_b
terraform destroy -target=module.facematch
terraform destroy -target=module.load_balancer
terraform destroy -target=module.database
terraform destroy -target=module.ecs_cluster
```

{% endhint %}

{% hint style="danger" %}
**Warning**: This permanently deletes all resources including:

* Aurora MySQL database and all data
* CloudWatch logs
* Secrets Manager secrets (database credentials, Router certificates)
* ECS services and tasks

Ensure you have backups before destroying. Database snapshots are NOT automatically created during destroy.
{% endhint %}

**Create database snapshot before cleanup** (optional):

```bash
aws rds create-db-cluster-snapshot \
  --db-cluster-identifier allid-database-cluster \
  --db-cluster-snapshot-identifier allid-final-snapshot-$(date +%Y%m%d)
```

## Environment Standardization

All environments (dev, stg, prd) follow the same base structure and configuration patterns. This standardization simplifies management and reduces errors when promoting changes across environments.

### Standardized Components

1. **Module Structure**: All environments use the same modules with the same parameters
2. **Tag Naming**: Lowercase with hyphens (e.g., `environment`, `managed-by`, `business-unity-id`)
3. **Default Values**: Consistent defaults across environments, customizable via `terraform.tfvars`
4. **Resource Naming**: `{prefix}-{resource-type}` pattern

### Environment-Specific Customization

Customize each environment by adjusting values in `environments/<env>/terraform.tfvars`:

**Development (dev)**:

```hcl
prefix                 = "allid-dev"
desired_count          = 0  # Save costs when not in use
enable_ecs_exec        = true  # Enable debugging
database_min_capacity  = 0  # Scale to zero when idle
database_max_capacity  = 2
log_retention_days     = 7
database_deletion_protection = false
```

**Staging (stg)**:

```hcl
prefix                 = "allid-stg"
desired_count          = 1  # One task per peer
enable_ecs_exec        = false
database_min_capacity  = 0
database_max_capacity  = 2
log_retention_days     = 7
database_deletion_protection = true
```

**Production (prd)**:

```hcl
prefix                             = "allid-prd"
desired_count                      = 2  # High availability
enable_ecs_exec                    = false
database_min_capacity              = 0.5  # Always active
database_max_capacity              = 4
database_backup_retention_period   = 30
database_deletion_protection       = true
alb_enable_deletion_protection     = true
log_retention_days                 = 30
```

### Default Tags

All resources are automatically tagged via AWS provider `default_tags`:

```hcl
default_tags {
  tags = {
    environment       = var.environment      # dev, stg, prd
    managed-by        = "Terraform"
    business-unity-id = "allid"
    workload-id       = "allid"
    cost-center       = "engineering"
  }
}
```

Additional resource-specific tags can be added through module parameters.

### Promoting Changes Across Environments

Best practice for promoting changes from dev → stg → prd:

1. **Test in Development**:

   ```bash
   cd src/terraform/allid
   # Edit environments/dev/terraform.tfvars
   make plan ENV=dev
   make apply ENV=dev
   # Test thoroughly
   ```
2. **Promote to Staging**:

   ```bash
   # Copy tested changes to staging
   # Edit environments/stg/terraform.tfvars with same changes
   make plan ENV=stg
   make apply ENV=stg
   # Verify
   ```
3. **Deploy to Production**:

   ```bash
   # Apply to production with proper review
   # Edit environments/prd/terraform.tfvars
   make plan ENV=prd > plan-output.txt
   # Review plan-output.txt carefully
   make apply ENV=prd
   ```

{% hint style="warning" %}
**Production Deployment Checklist**:

* [ ] Changes tested in dev and stg
* [ ] Terraform plan reviewed and approved
* [ ] Database backup completed (if applicable)
* [ ] Rollback plan documented
* [ ] Team notified of deployment window
* [ ] Monitoring and alerts ready
  {% endhint %}

## Detailed File Structure

### All ID Application (`src/terraform/allid/`)

```
src/terraform/allid/
├── README.md                           # Project documentation
├── Makefile                            # Command automation
├── modules/                            # Reusable modules
│   ├── ecs-cluster/
│   │   ├── main.tf                     # ECS Cluster + Service Discovery
│   │   ├── variables.tf
│   │   └── outputs.tf
│   ├── database/
│   │   ├── main.tf                     # Aurora MySQL Serverless V2
│   │   ├── variables.tf
│   │   └── outputs.tf
│   ├── load-balancer/
│   │   ├── main.tf                     # ALB + Listener + Security Group
│   │   ├── variables.tf
│   │   └── outputs.tf
│   ├── peer-service/
│   │   ├── main.tf                     # Task Definition + Service + Secrets
│   │   ├── variables.tf
│   │   └── outputs.tf
│   └── facematch-service/
│       ├── main.tf                     # Task Definition + Service
│       ├── variables.tf
│       └── outputs.tf
└── environments/
    ├── dev/
    │   ├── main.tf                     # Environment configuration
    │   ├── variables.tf                # Variable definitions
    │   ├── outputs.tf                  # Environment outputs
    │   ├── terraform.tfvars.example    # Example values
    │   └── terraform.tfvars            # Real values (not versioned)
    ├── stg/
    └── prd/
```

### Shared Infrastructure (`src/terraform/shared/`)

```
src/terraform/shared/
├── README.md                           # Project documentation
├── Makefile                            # Command automation
├── modules/
│   └── network/
│       ├── main.tf                     # VPC, Subnets, IGW, NAT
│       ├── variables.tf
│       └── outputs.tf
└── environments/
    ├── dev/
    │   ├── main.tf                     # Environment configuration
    │   ├── ssm.tf                      # SSM Parameters for cross-reference
    │   ├── variables.tf                # Variable definitions
    │   ├── outputs.tf                  # Environment outputs
    │   ├── terraform.tfvars.example    # Configuration template
    │   └── terraform.tfvars            # Real values (not versioned)
    ├── stg/
    └── prd/
```

### Key Files Explained

**`main.tf`**:

* Terraform and AWS provider configuration
* Module invocations with parameters
* Data sources for SSM Parameters (allid project)
* Identical structure across environments

**`variables.tf`**:

* All variable definitions with types and defaults
* Standardized defaults across environments
* Inline documentation for each variable

**`outputs.tf`**:

* Outputs of created resources
* Used for retrieving values after deployment
* Referenced by dependent modules or external systems

**`terraform.tfvars.example`**:

* Template with all configurable values
* Explanatory comments for each variable
* Environment-specific example values

**`terraform.tfvars`** (not versioned):

* Real environment values
* Contains secrets (certificates, private keys)
* Must be created from `.example` template
* **Never commit this file to git**

**`ssm.tf`** (shared only):

* Creates SSM Parameters for VPC and subnet IDs
* Enables other projects to lookup resources
* Format: `/shared/{environment}/{parameter-name}`

## Next steps

* Review [Technical Requirements](/caf-api/all-id/technical-requirements.md) for resource specifications
* Review [Configuration](/caf-api/all-id/configuration.md) for environment variable details
* Review [Security Best Practices](/caf-api/all-id/security-best-practices.md) for hardening


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.caf.io/caf-api/all-id/production-guidance/ecs-terraform.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.