Terraform Github Actions Pipeline

---

Table of Contents

• Table of Contents
• Overview
• Features
• Project Structure
  • Project Index
• Getting Started
  • Prerequisites
  • Installation
  • Usage
  • Testing
• Roadmap
• Contributing
• License
• Acknowledgments

---

Overview

This project demonstrates Infrastructure as Code (IaC) automation using Terraform and GitHub Actions. It provides automated workflows for managing cloud infrastructure using AWS. The repository implements CI/CD best practices for infrastructure deployment, including automated planning and applying of Terraform configurations through GitHub Actions workflows integrated with HCP Terraform (formerly Terraform Cloud) for remote state management.

---

Features

• Automated Infrastructure Deployment: GitHub Actions workflows automatically validate and deploy infrastructure changes
• HCP Terraform Integration: API-driven workflow with remote state management and execution
• Remote State Management: Terraform Cloud configuration for collaborative state management
• Modular Terraform Code: Well-organized Terraform modules with separated concerns (network, main, variables, outputs)
• Pull Request Integration: Automated Terraform plan on pull requests for review before merging
• Multi-Job Pipeline: Separated unit tests and planning jobs for better visibility and control
• Security Scanning: Automated security checks using Checkov with results uploaded to GitHub Advanced Security
• Code Quality Checks: Automated validation and formatting checks for Terraform code
• Smart Change Detection: Automatically detects modified Terraform directories and runs workflows only for changed workspaces
• Matrix Strategy: Parallel execution for multiple workspace deployments
• Workspace Auto-Discovery: Automatically resolves workspace names from Terraform configuration files
• Secure Deployment: GitHub Actions workflows with proper authentication and secure credential management

---

Project Structure

 Terraform-Github-Actions-Pipeline/
    ├── .github
    │   └── workflows
    │       ├── terraform-apply.yaml       # Automated deployment workflow with change detection
    │       └── terraform-plan.yaml        # Three-stage workflow: change detection, unit tests, and planning
    └── Template            
        ├── .terraformignore               # Files excluded from Terraform uploads
        ├── backend.tf                     # HCP Terraform backend configuration
        ├── main.tf                        # Core infrastructure resources
        ├── network.tf                     # Network infrastructure (VPC, subnets, routing)
        ├── output.tf                      # Output values for resource attributes
        ├── provider.tf                    # Cloud provider configuration
        └── variables.tf                   # Input variable definitions

Project Index

Terraform-Github-Actions-Pipeline/

Template

Template

File Name	Summary
network.tf	❯ Defines VPC, subnets, internet gateway, and routing tables for network infrastructure
.terraformignore	❯ Specifies files and directories to exclude from Terraform configuration uploads
main.tf	❯ Core infrastructure resources including compute instances, storage, and application services
variables.tf	❯ Input variable definitions for customizing infrastructure deployment parameters
output.tf	❯ Output values exposing resource attributes for external consumption and reference
provider.tf	❯ Cloud provider configuration and authentication settings for Terraform operations
backend.tf	❯ Remote state backend configuration for Terraform Cloud workspace integration

.github

⦿ .github

workflows

⦿ .github.workflows

File Name	Summary
terraform-apply.yaml	❯ Multi-job deployment workflow with smart change detection, parallel matrix execution, and automatic workspace resolution
terraform-plan.yaml	❯ Three-stage PR workflow: change detection, parallel unit tests (validation, formatting, security scanning), and speculative plan generation

---

Getting Started

Prerequisites

This project requires the following dependencies:

• Terraform: Version 1.0 or higher (Download)
• HCP Terraform Account: Sign up here
• AWS Account: With IAM user with sufficient permissions
• Git: For version control
• GitHub Account: For GitHub Actions workflows

Installation & Setup

This project utilizes HCP Terraform (formerly Terraform Cloud) for remote state management and execution, driven by GitHub Actions via the API-driven workflow.

1. Prerequisites

• HCP Terraform Account: Sign up here.
• AWS Account: Ensure you have an IAM user with AdministratorAccess or sufficient permissions.

2. Configure HCP Terraform (Backend)

1. Create an Organization: Log in to HCP Terraform and create a new organization.
2. Create a Workspace:
   • Select "API-driven workflow".
   • Name it (e.g., terraform-github-actions-pipeline).
3. Add Environment Variables:
   • Navigate to your Workspace > Variables.
   • Add the following as Environment Variables (mark as Sensitive):
     • AWS_ACCESS_KEY_ID
     • AWS_SECRET_ACCESS_KEY
   • (Note: These are stored in HCP Terraform so the remote runner can authenticate with AWS.)

3. Configure GitHub Repository

To allow GitHub Actions to trigger runs in HCP Terraform:

1. Generate a Team Token:
   • In HCP Terraform, go to Organization Settings > Teams.
   • Create a team (e.g., "GitHub Actions") or use an existing one.
   • Go to Organization Settings > API Tokens > Create a team token.
   • Copy this token.
2. Add Secret to GitHub:
   • In this GitHub repository, go to Settings > Secrets and variables > Actions.
   • Click New repository secret.
   • Name: TF_API_TOKEN
   • Value: (Paste the Team Token you generated).
3. Enable GitHub Advanced Security (for security scanning):
   • In this GitHub repository, go to Settings > Code security and analysis.
   • Under Code scanning, click Set up > Default or Advanced.
   • This enables CodeQL and allows the unit test workflow to upload security scan results.
   • (Note: This feature requires GitHub Advanced Security for private repositories, but is free for public repositories).

4. Workflow Usage

The repository includes two GitHub Actions workflows in .github/workflows/:

Terraform Plan Workflow (terraform-plan.yaml)

Triggers on Pull Requests and includes three jobs:

1. Detect Changes: Identifies modified Terraform directories using file change detection
2. Unit Tests (runs in parallel for each workspace):
   • Terraform initialization (backend-less mode)
   • Terraform validation
   • Terraform format checking
   • Security scanning with Checkov (results uploaded to GitHub Advanced Security)
3. Plan Execution (runs after unit tests pass):
   • Automatically discovers workspace name from Terraform configuration
   • Uploads configuration to HCP Terraform
   • Creates a speculative plan (non-destructive)
   • Displays plan output with resource changes

Terraform Apply Workflow (terraform-apply.yaml)

Triggers on Push to Main branch and includes:

1. Detect Changes: Identifies added/modified and deleted directories
2. Apply Execution (runs in parallel for multiple workspaces):
   • Automatically resolves workspace names
   • Uploads configuration to HCP Terraform
   • Creates and executes apply runs
   • Provisions or updates infrastructure automatically

Key Benefits:

• Only affected workspaces are processed (efficient change detection)
• Parallel matrix execution for multiple workspaces
• Automatic workspace name resolution
• Separation of concerns with dedicated unit test jobs

5. Local Development (Optional)

If you wish to run Terraform locally before pushing:

1. Clone the repository:

   git clone https://github.com/Travis-Houston/Terraform-Github-Actions-Pipeline.git
   cd Terraform-Github-Actions-Pipeline

2. Authenticate with HCP Terraform:

   terraform login

3. Navigate to Template directory and initialize:

   cd Template
   terraform init
   terraform plan

6. Using the Pipeline

Development Workflow:

1. Create a Feature Branch:

   git checkout -b feature/your-feature-name

2. Make Infrastructure Changes:

   • Modify Terraform files in your workspace directory
   • Ensure proper formatting: terraform fmt -recursive
   • Validate locally (optional): terraform validate

3. Create a Pull Request:

   • Push your changes to GitHub
   • Open a pull request against the main branch
   • The Terraform Plan workflow automatically triggers with three stages:

   Stage 1: Change Detection

   • Identifies which Terraform directories have been modified
   • Creates a matrix of workspaces to test

   Stage 2: Unit Tests (parallel execution per workspace)

   • ✅ Terraform initialization
   • ✅ Terraform validation
   • ✅ Format checking (terraform fmt -check)
   • 🔒 Security scanning with Checkov
   • Results uploaded to GitHub Advanced Security

   Stage 3: Terraform Plan (after unit tests pass)

   • Generates speculative execution plan
   • Shows resources to be added, changed, or destroyed
   • Review plan output in workflow logs

4. Review and Address Feedback:

   • Check the Actions tab for workflow results
   • Review security scan findings in the Security tab
   • Fix any format issues or security concerns
   • Push additional commits to update the PR (workflow re-runs automatically)

5. Merge to Main:

   • Once approved and all checks pass, merge the PR
   • The Terraform Apply workflow automatically triggers:
     • Detects changed workspaces
     • Uploads configurations to HCP Terraform
     • Executes apply runs in parallel for affected workspaces
     • Provisions/updates infrastructure
   • Monitor deployment progress in the Actions tab

Best Practices:

• Always format code before committing: terraform fmt -recursive
• Review security scan results and address critical findings
• Test changes in a development workspace before production
• Use meaningful commit messages for clear audit trails
• Monitor workflow logs for detailed execution information

---

Roadmap

• Task 1: Implement automated Terraform plan workflow for pull requests
• Task 2: Implement automated Terraform apply workflow for main branch
• Task 3: Add security scanning with Checkov
• Task 4: Implement smart change detection for modified workspaces
• Task 5: Add parallel matrix execution for multiple workspaces
• Task 6: Separate unit tests into dedicated job for better visibility
• Task 7: Add automatic workspace name resolution
• Task 8: Add PR comment with Terraform plan summary
• Task 9: Implement multi-environment deployment (dev/staging/prod)
• Task 10: Add infrastructure testing with Terratest
• Task 11: Implement cost estimation in PR comments
• Task 12: Add drift detection scheduled workflow

---

Contributing

• 💬 Join the Discussions: Share your insights, provide feedback, or ask questions.
• 🐛 Report Issues: Submit bugs found or log feature requests for the Terraform-Github-Actions-Pipeline project.
• 💡 Submit Pull Requests: Review open PRs, and submit your own PRs.

Contributing Guidelines

1. Fork the Repository: Start by forking the project repository to your github account.
2. Clone Locally: Clone the forked repository to your local machine using a git client.

   git clone https://github.com/Travis-Houston/Terraform-Github-Actions-Pipeline

3. Create a New Branch: Always work on a new branch, giving it a descriptive name.

   git checkout -b new-feature-x

4. Make Your Changes: Develop and test your changes locally.
5. Commit Your Changes: Commit with a clear message describing your updates.

   git commit -m 'Implemented new feature x.'

6. Push to github: Push the changes to your forked repository.

   git push origin new-feature-x

7. Submit a Pull Request: Create a PR against the original project repository. Clearly describe the changes and their motivations.
8. Review: Once your PR is reviewed and approved, it will be merged into the main branch. Congratulations on your contribution!

Contributor Graph

---

License

This project is protected under the MIT License. For more details, refer to the LICENSE file.

---

Acknowledgments

• Terraform by HashiCorp for infrastructure as code
• GitHub Actions for CI/CD automation
• The open-source community for IaC best practices and examples
• Contributors who help improve this project

---

Contact Information

• Project Owner: Travis Houston
• Repository: Terraform-Github-Actions-Pipeline
• Issues: Report a bug or request a feature

For questions or support, please open an issue in the repository or reach out through GitHub discussions.

---