AWS ECS (Terraform)
Complete guide for deploying All ID on AWS ECS Fargate using Terraform.
Overview
This guide explains how to deploy All ID using Terraform Infrastructure as Code.
The deployment requires:
All ID Application Terraform (
src/terraform/allid/) - Application services (ECS, RDS, ALB)Existing VPC - You must provide a VPC with the required subnet structure
Network Requirements: The All ID Terraform modules require an existing VPC with public, private, and isolated subnets. If you don't have a VPC yet, Certta provides a reference Terraform project (src/terraform/shared/) that creates a production-ready network infrastructure.
Certta will provide you with a package containing:
All ID Terraform project (required)
Reference network infrastructure Terraform project (optional, use if you need to create a new VPC)
Configuration examples and documentation
Note: The project uses a default prefix teste-tf for resource naming. You should change this to your project name in the terraform.tfvars file via the prefix variable.
Prerequisites
Before deploying, ensure you have:
Tools:
Terraform >= 1.5
AWS CLI configured with credentials
AWS Account:
AWS account with appropriate permissions
Permissions to create: ECS, RDS, ALB, CloudFormation, IAM roles, Security Groups
AWS credentials configured (
aws configure)
Network Infrastructure - You have three options:
Option A (Recommended): Deploy new VPC using the Certta reference Terraform project (
src/terraform/shared/)Option B: Use existing VPC by manually creating SSM Parameters (see step 2B)
Already have SSM Parameters: Skip directly to step 3 if parameters are already configured
Required VPC structure (regardless of option):
Public subnets (minimum 2 AZs) for Application Load Balancer
Private subnets with NAT (minimum 2 AZs) for ECS services
Isolated subnets (minimum 2 AZs) for RDS database
Internet Gateway and NAT Gateway configured
Container Images:
Access to Certta's private container registry
ECR repository URIs for Peer and Facematch images
Image versions/tags to deploy
Important: Contact your Certta technical account manager to obtain:
Terraform project files (All ID + optional network infrastructure)
Container registry credentials and image URIs
Router Service certificates for mTLS communication
Makefile Automation
This project includes Makefiles that simplify Terraform operations. Instead of running terraform commands directly, you can use make commands that automatically handle environment-specific configurations.
Benefits:
✅ Consistent execution: Same commands work across all environments
✅ Automatic navigation: No need to
cdto environment directories✅ Built-in safety: Confirmation prompts for destructive operations
✅ Easy multi-environment: Switch environments with
ENVparameter✅ Self-documenting: Run
make helpto see all available commands✅ Error prevention: Validates environment names and required parameters
Throughout this guide, you'll see make commands instead of direct terraform commands. This is the recommended approach for managing this infrastructure.
Alternative: If you prefer using Terraform directly, you can still do so by navigating to the environment directory (e.g., cd environments/dev) and running standard terraform commands.
Quick example:
Safety example:
Project structure
Certta will provide you with Terraform project files. The package structure:
Shared infrastructure is optional: Only deploy if you need to create a new VPC. If you already have a VPC, skip the src/terraform/shared/ project and configure the VPC information via SSM Parameters.
Main Files per Environment
Each environment directory contains:
main.tf: Main configuration with provider settings and module invocationsvariables.tf: All variable definitions with types and defaultsoutputs.tf: Outputs of created resources (ALB DNS, database endpoint, etc.)terraform.tfvars.example: Template with example values and commentsterraform.tfvars: Real values (not versioned in git, contains secrets)ssm.tf(shared only): SSM Parameters for cross-project references
Step 1: Extract Terraform project
Extract the Terraform project files provided by Certta:
Step 2: Network infrastructure (Option A)
Choose your option:
This step (Option A): Deploy a new VPC using the Certta reference Terraform project
Step 2B (Option B): Use your existing VPC by creating SSM Parameters manually
Option A: Deploy new VPC with Terraform
If you need to create a new VPC, deploy the reference network infrastructure:
What gets deployed:
VPC with environment-specific CIDR blocks
2 Public subnets (across 2 availability zones)
2 Private subnets with NAT (across 2 availability zones)
2 Isolated subnets (across 2 availability zones) for databases
Internet Gateway for public internet access
NAT Gateway for private subnet outbound access
SSM Parameters for cross-project references
Deployment time: ~5-10 minutes
After deployment, the VPC information is automatically stored in SSM Parameters:
The following SSM Parameters are created:
/shared/dev/vpc-id- VPC ID/shared/dev/vpc-cidr- VPC CIDR block/shared/dev/public-subnet-ids- Public subnet IDs (comma-separated)/shared/dev/private-egress-subnet-ids- Private subnet IDs with NAT (comma-separated)/shared/dev/private-isolated-subnet-ids- Isolated subnet IDs (comma-separated)
The All ID Terraform project automatically reads these SSM Parameters - no manual configuration needed!
Step 2B: Using Existing VPC (Option B)
Skip this step if you chose Option A (deploying new VPC with terraform-shared).
If you already have a VPC and don't want to use terraform-shared, you can create the SSM Parameters manually so the All ID Terraform project can find your VPC.
Prerequisites: Your VPC must have:
Public subnets (with route to Internet Gateway)
Private subnets with egress (with route to NAT Gateway)
Private isolated subnets (no internet route, for databases)
Get your VPC and subnet IDs
Use AWS CLI to identify your VPC and subnets:
Identify each subnet type
Public Subnets: Has route
0.0.0.0/0 → igw-xxxxx(Internet Gateway)Private Egress Subnets: Has route
0.0.0.0/0 → nat-xxxxx(NAT Gateway)Private Isolated Subnets: NO route to
0.0.0.0/0
Create SSM Parameters
Once you've identified your VPC and subnets, create the SSM Parameters:
Important:
Use subnet IDs separated by comma WITHOUT SPACES (e.g.,
subnet-111,subnet-222)Ensure subnets are in different AZs for high availability
Subnet types (public, private-egress, private-isolated) must correspond to configured routes
Verify parameters were created
Delete parameters (if you need to redo)
Which option to choose?
✅ Option A (terraform-shared): Recommended for new projects or when you want to manage the VPC with Terraform
🔧 Option B (Manual): Use if you already have a VPC and don't want to migrate to Terraform yet
Step 3: Configure application variables
Navigate to the application directory:
Copy the example variables file and configure your environment-specific values:
Edit environments/dev/terraform.tfvars and configure the following values:
Configuration values to update:
peer_ecr_repository_uri
Peer Service container registry
Provided by Certta
peer_version
Peer image tag to deploy
Provided by Certta (e.g., latest, commit hash)
facematch_ecr_repository_uri
Facematch container registry
Provided by Certta
facematch_version
Facematch image tag to deploy
Provided by Certta (e.g., latest, commit hash)
desired_count
Number of tasks per peer service
Set to 0 for dev (cost saving), 1+ for stg/prd
router_rest_url
Certta Router endpoint for your region/env
Provided by Certta
About desired_count: This variable controls the number of tasks for all three Peer services simultaneously (default, client-a, client-b). This ensures consistency across all peers.
Recommended values:
Development:
0(no tasks running - saves costs when not in use)Staging:
1(one task per peer for testing)Production:
2+(multiple tasks per peer for high availability)
Note: Facematch service is hardcoded to 2 tasks. To change, edit the facematch module in environments/<env>/main.tf.
Router mTLS Certificates: The certificates are NOT configured in terraform.tfvars. Terraform creates empty secrets in AWS Secrets Manager, and you will populate them with actual certificates after deployment in Step 5.
Alternative approach: You can include certificates in terraform.tfvars if preferred, but be aware:
Values will be stored in Terraform state files (even if marked as sensitive)
State files must be encrypted and access-controlled
This approach is optional and not recommended for security reasons
Multiple environments: Create separate terraform.tfvars files for each environment (dev, stg, prd) with environment-specific values.
Step 4: Deploy All ID application
Deploy the All ID application services:
Type yes when prompted to confirm the deployment.
What gets deployed:
ECS Cluster + Service Discovery (
ecs-clustermodule)ECS Fargate cluster
Cloud Map private namespace (
allid.local)Container Insights enabled
Database (
databasemodule)Aurora MySQL Serverless v2 cluster
Database security group
Secrets Manager for credentials (auto-generated)
CloudWatch Logs for error, general, and slow query logs
Performance Insights enabled
Load Balancer (
load-balancermodule)Application Load Balancer (internet-facing)
HTTP listener (port 80)
Security group with IP whitelisting
Default action: 403 Access Denied
Facematch Service (
facematch-servicemodule)ECS Task Definition (CPU: 1024, Memory: 2048 MB)
Fargate Service with 2 instances
Cloud Map service registration (
facematch.allid.local)Security group for internal communication
IAM roles for execution and task
Peer Services (
peer-servicemodule - instantiated 3 times)Creates 3 peer instances (default, client-a, client-b)
Each peer has:
ECS Task Definition (CPU: 256, Memory: 512 MB)
Fargate Service with configurable instance count
ALB target group with path-based routing
Cloud Map service registration
Database connection to RDS
Router mTLS certificate stored in Secrets Manager
IAM roles for execution and task
CloudWatch Log Group
Deployment time: ~15-20 minutes
Terraform automatically handles resource dependencies and deploys them in the correct order. You can monitor the progress in the terminal output.
Important: After deployment, complete Step 5 (configure certificates) and Step 6 (initialize databases) to make the Peer Services operational.
Step 5: Configure Router mTLS certificates
After deploying the infrastructure, you need to populate the Router mTLS certificates in AWS Secrets Manager. Terraform creates empty secrets that must be filled with actual certificate content.
Why certificates are not in Terraform:
Keeps sensitive credentials out of Terraform state files
Allows certificate rotation without Terraform changes
Follows security best practices for secrets management
Alternative: You can include certificates in terraform.tfvars if your security policies allow it, but this will store them in the Terraform state file.
Configure certificates for each peer
Option 1: AWS Console (recommended)
Navigate to Secrets Manager Console → Secrets
Find and click on each secret:
{prefix}-peer-default-router-certificate{prefix}-peer-client-a-router-certificate{prefix}-peer-client-b-router-certificate
Click "Retrieve secret value" → "Edit"
Update the JSON with your actual certificates:
Save changes
Option 2: AWS CLI
Important: Replace {prefix} with your actual prefix value (default is teste-tf or what you configured in terraform.tfvars).
After updating the secrets, the Peer Services will automatically detect and use the certificates. No restart is required.
Step 6: Initialize database
The Peer Service requires a database schema to be initialized. Certta will provide a SQL dump file that must be restored to the Aurora MySQL database.
Critical: The Peer Services may fail to start if the database schema is not initialized. Complete this step to allow the services to become operational.
Multi-tenant setup: Each peer instance uses its own database. The database names follow the pattern peer-{name}:
peer-default(for default peer)peer-client-a(for client-a peer)peer-client-b(for client-b peer)
Database access methods
The Aurora MySQL database is deployed in isolated subnets with no direct internet access. You need to establish a secure connection to access it.
Option 1: Bastion Host (Recommended for production)
Deploy a bastion host (EC2 instance) in a public subnet to act as a jump server:
Launch an EC2 instance in a public subnet of your VPC
Configure security groups to allow:
SSH access from your IP to the bastion host
MySQL access from bastion host to the RDS security group
Connect to the database through SSH tunnel:
Option 2: VPN Connection
If you have a VPN connection configured to your VPC:
Connect to your VPN
Access the database directly using its private endpoint
Option 3: AWS Systems Manager Session Manager
Use Session Manager for secure access without exposing SSH ports:
Ensure your bastion host has SSM agent installed
Grant necessary IAM permissions
Create port forwarding session:
Get database endpoint and credentials
Option 1: Terraform outputs
Option 2: AWS Console
Navigate to RDS Console → Databases
Find the Aurora cluster (look for name with
allid-database)Copy the Writer endpoint (e.g.,
allid-database-cluster.cluster-xxx.us-east-1.rds.amazonaws.com)Navigate to Secrets Manager Console → Secrets
Find the database secret (look for name with
allid-database)Click "Retrieve secret value" to see username and password
Option 3: AWS CLI
Create and initialize databases
You can use any of the following tools to create databases and restore the dump:
Option A: MySQL Command Line Client
Best for automation and CI/CD pipelines:
Option B: DBeaver (GUI Tool)
Recommended for visual database management:
Create a new connection:
Database: MySQL
Host:
{DB_ENDPOINT}(from above)Port: 3306
Username:
{DB_USERNAME}(from Secrets Manager)Password:
{DB_PASSWORD}(from Secrets Manager)
Create databases:
Right-click on connection → SQL Editor → New SQL Script
Execute:
Restore dump:
Right-click on each database → Tools → Execute Script
Select the
allid-dump.sqlfile provided by CerttaClick "Start" to execute
Option C: MySQL Workbench (GUI Tool)
Alternative GUI tool for MySQL management:
Create a new connection:
Connection Name:
All ID DatabaseHostname:
{DB_ENDPOINT}Port: 3306
Username:
{DB_USERNAME}Password: Store in Keychain/Vault
Create databases:
Open connection → Query tab
Execute:
Import dump:
Server → Data Import
Select "Import from Self-Contained File"
Choose
allid-dump.sqlfileSelect target database (repeat for each: peer-default, peer-client-a, peer-client-b)
Click "Start Import"
Connection troubleshooting: If you cannot connect to the database, ensure:
You have established proper access (bastion host, VPN, or Session Manager)
The RDS security group allows connections from your source
The database endpoint and credentials are correct
Critical: Each Peer Service will fail to start if its corresponding database is not initialized. Contact your Certta technical account manager to obtain the SQL dump file.
Services will auto-recover: Once you initialize the databases, the Peer Services will automatically restart and become healthy within a few minutes. ECS will detect that tasks are failing and restart them automatically.
Step 7: Retrieve endpoints
After deployment completes, get the application endpoint:
Important outputs:
alb_dns_name
Load Balancer DNS name
allid-load-balancer-123456789.us-east-1.elb.amazonaws.com
database_endpoint
Aurora MySQL cluster endpoint
allid-database-cluster.cluster-xxx.us-east-1.rds.amazonaws.com
database_secret_arn
Secrets Manager ARN for DB credentials
arn:aws:secretsmanager:...
ecs_cluster_name
ECS cluster name
allid-cluster
peer_default_service_name
Peer default service name
allid-peer-default
facematch_service_name
Facematch service name
allid-facematch
The deployment creates 3 peer instances with path-based routing for multi-tenant support. Each peer has its own isolated endpoint with path prefix.
API endpoints (3 peer instances by default):
Health check endpoints:
To add/remove peers, edit the environment main.tf file and add/remove peer modules. Remember to create the corresponding database and provide Router certificates for new peers BEFORE deploying.
Step 8: Validate deployment
Test that all services are healthy:
Test peer endpoints (using ALB DNS from Step 7):
All endpoints should return HTTP 200 with status information.
Check ECS services status:
AWS Console:
Navigate to ECS Console → Clusters →
allid-clusterClick on "Services" tab
Verify all services show "Running" status and desired count matches running count
AWS CLI:
All services should show runningCount matching desiredCount and deployment status PRIMARY.
Check CloudWatch Logs (if services fail to start):
AWS Console:
Navigate to CloudWatch Console → Log groups
Find log groups:
/ecs/allid/peer-default,/ecs/allid/facematch, etc.Check recent log streams for errors
AWS CLI:
Makefile Commands
Both src/terraform/allid/ and src/terraform/shared/ include a Makefile with automation commands for easier infrastructure management.
Getting Started: Run make help in either project directory to see all available commands and their descriptions.
Available Commands
Environment Selection
All commands support the ENV parameter to specify the target environment:
Default environment: If you don't specify ENV, the Makefile defaults to dev.
Shared Infrastructure Commands
The shared infrastructure Makefile includes an additional command to view SSM parameters:
Workflow Example
Here's a typical workflow using Makefile commands:
Quick Reference
Most Common Commands
Environment Management
Common Tasks
Deploy to new environment:
Update container images:
Scale services:
View logs:
Architecture overview
Network layers
The deployment creates a three-tier network architecture:
Service communication
Path-based routing (multi-tenant)
The ALB uses path-based routing to support multiple peer instances:
Each peer instance:
Has its own ECS service
Has its own isolated database (peer-default, peer-client-a, peer-client-b)
Operates independently from other peers
Shares the same Facematch service pool
Configuration management
The Terraform modules automatically configure environment variables and secrets for all services.
Environment variables are defined in the peer-service module and include:
Database connection (host, port, database name)
Facematch service endpoint
Router Service URL
Feature flags (RabbitMQ, Redis, Router communication)
Secrets are injected via ECS task secrets and include:
Database credentials (auto-generated by Terraform)
Router mTLS certificates (provided via terraform.tfvars)
See Configuration for complete list of environment variables, required values, and configuration details.
Secrets management
Terraform automatically creates secrets in AWS Secrets Manager:
Secret naming convention:
Database credentials:
{prefix}-database-secret(e.g.,allid-database-secret)Auto-generated by Terraform during deployment
Router certificates:
{prefix}-peer-{name}-router-certificate(e.g.,allid-peer-default-router-certificate)Created empty by Terraform, populated manually after deployment (see Step 5)
Router certificates security approach:
By default, Router certificate secrets are created empty and populated manually after deployment. This approach:
✅ Keeps sensitive credentials out of Terraform state files
✅ Allows certificate rotation without Terraform changes
✅ Follows security best practices for secrets management
Alternative: You can include certificates in terraform.tfvars if needed, but be aware that they will be stored in Terraform state files (even if marked as sensitive). If you choose this approach:
Store state files in encrypted S3 buckets with restricted access
Use state locking with DynamoDB
Never commit state files to version control
Limit access to state files to authorized personnel only
Security
Terraform automatically configures security groups and IP whitelisting following least-privilege principles.
Security groups:
ALB accepts traffic only from whitelisted IPs
Peer Services accept traffic only from ALB and internal VPC
Facematch accepts traffic only from Peer Services
Database accepts traffic only from Peer Services
IP whitelisting:
ALB is configured to accept traffic only from Certta Router IP addresses
Configure additional IPs in
terraform.tfvarsvia thealb_allowed_cidr_blocksvariable
See Security Best Practices for:
Complete list of Certta Router IP addresses
Network segmentation recommendations
Secrets management best practices
Additional security hardening options
Updating the deployment
Update container images
Update image versions in
terraform.tfvars:
Apply the changes:
Terraform will update the ECS task definitions and trigger a rolling deployment.
Update environment variables
Edit the
peer-servicemodule insrc/terraform/allid/modules/peer-service/main.tfModify the environment variables in the task definition
Apply the changes:
This will update all peer instances (default, client-a, client-b) with the new configuration.
Rotate Router mTLS certificates
If you need to rotate the Router mTLS certificates (certificate expiration, security incident, etc.), follow the same process as the initial configuration in Step 5.
When to rotate certificates:
Certificate expiration/renewal
Security incident requiring certificate replacement
Moving between Certta environments (dev/stg/prd)
Compliance requirements for periodic rotation
The application will automatically pick up the new certificates from Secrets Manager without requiring a service restart.
Scale services
Scale Peer Services: Edit environments/dev/terraform.tfvars and change the desired_count variable:
Then apply:
Scale Facematch Service: Edit the facematch module in environments/dev/main.tf:
Then apply:
Add/remove peer instances
To add a new peer instance:
Edit the environment
main.tffile (e.g.,src/terraform/allid/environments/dev/main.tf):
Add output for the new peer in
outputs.tf:
Apply the changes:
Create and initialize the database for the new peer (see Step 6):
The new Peer Service will automatically have its Router certificate secret created by Terraform using the same certificate from terraform.tfvars. The service will become healthy after you complete database initialization.
Priority ranges: Each peer must use a unique priority_base value for ALB listener rules. Use ranges of 30 (e.g., 100-129, 130-159, 160-189, 190-219) to avoid conflicts.
Troubleshooting
Error: SSM Parameter not found
Cause: Network infrastructure (src/terraform/shared) has not been deployed, or SSM Parameters were not created.
Solution:
Verify if parameters exist:
If parameters don't exist, deploy the shared infrastructure:
If you have an existing VPC, create SSM Parameters manually:
Peer Service won't start
Check CloudWatch Logs:
Common causes:
Database not initialized: See Step 6 (database initialization and dump restoration)
Database connection failed: Check security groups and RDS endpoint
Facematch service unavailable: Check Facematch service status
Check task stopped reason:
AWS Console:
ECS Console → Clusters → allid-cluster
Click on service (e.g., allid-peer-default)
Go to "Tasks" tab → Click on stopped tasks
Check "Stopped reason" field
AWS CLI:
Health check failures
Check ECS service status:
Check ALB target health:
AWS CLI:
Common issues:
Security groups blocking traffic between ALB and Peer Services
Service not registered in Cloud Map (DNS resolution fails)
Database connection errors (check credentials secret)
Facematch service not responding
Health check grace period not long enough
No tasks running / Services showing 0/0
Cause: desired_count is set to 0 in terraform.tfvars.
Solution:
This is often intentional for development environments to save costs. To start tasks:
Edit
environments/dev/terraform.tfvars:
Apply changes:
Verify tasks are starting:
Cost saving tip: Set desired_count = 0 when not actively using development environments to stop all ECS tasks and reduce infrastructure costs.
Can't access ALB
Check if your IP is whitelisted:
If you get connection timeout or 403 Forbidden, your IP is not whitelisted.
Add your IP to the allowlist:
Edit
environments/dev/terraform.tfvars:
Apply changes:
See Security Best Practices for the complete list of Certta Router IP addresses that should be whitelisted.
Cleanup
To remove all resources and stop incurring costs:
Targeted destruction: If you need to destroy specific resources in order, you can navigate to the environment directory and use targeted destroy:
Warning: This permanently deletes all resources including:
Aurora MySQL database and all data
CloudWatch logs
Secrets Manager secrets (database credentials, Router certificates)
ECS services and tasks
Ensure you have backups before destroying. Database snapshots are NOT automatically created during destroy.
Create database snapshot before cleanup (optional):
Environment Standardization
All environments (dev, stg, prd) follow the same base structure and configuration patterns. This standardization simplifies management and reduces errors when promoting changes across environments.
Standardized Components
Module Structure: All environments use the same modules with the same parameters
Tag Naming: Lowercase with hyphens (e.g.,
environment,managed-by,business-unity-id)Default Values: Consistent defaults across environments, customizable via
terraform.tfvarsResource Naming:
{prefix}-{resource-type}pattern
Environment-Specific Customization
Customize each environment by adjusting values in environments/<env>/terraform.tfvars:
Development (dev):
Staging (stg):
Production (prd):
Default Tags
All resources are automatically tagged via AWS provider default_tags:
Additional resource-specific tags can be added through module parameters.
Promoting Changes Across Environments
Best practice for promoting changes from dev → stg → prd:
Test in Development:
Promote to Staging:
Deploy to Production:
Production Deployment Checklist:
Detailed File Structure
All ID Application (src/terraform/allid/)
Shared Infrastructure (src/terraform/shared/)
Key Files Explained
main.tf:
Terraform and AWS provider configuration
Module invocations with parameters
Data sources for SSM Parameters (allid project)
Identical structure across environments
variables.tf:
All variable definitions with types and defaults
Standardized defaults across environments
Inline documentation for each variable
outputs.tf:
Outputs of created resources
Used for retrieving values after deployment
Referenced by dependent modules or external systems
terraform.tfvars.example:
Template with all configurable values
Explanatory comments for each variable
Environment-specific example values
terraform.tfvars (not versioned):
Real environment values
Contains secrets (certificates, private keys)
Must be created from
.exampletemplateNever commit this file to git
ssm.tf (shared only):
Creates SSM Parameters for VPC and subnet IDs
Enables other projects to lookup resources
Format:
/shared/{environment}/{parameter-name}
Next steps
Review Technical Requirements for resource specifications
Review Configuration for environment variable details
Review Security Best Practices for hardening
Last updated

