CI/CD Pipelines - Comprehensive DevOps Skill

A unified skill for CI/CD pipeline design, DevOps automation, infrastructure as code, GitOps deployment automation, security scanning, and enterprise pipeline readiness across major platforms.

When to Use This Skill

Use this skill when:

CI/CD Pipeline Development

• Creating new CI/CD workflows (GitHub Actions, GitLab CI, Jenkins)
• Debugging pipeline failures or flaky tests
• Optimizing slow builds or test suites
• Implementing caching strategies
• Setting up deployment workflows
• Implementing matrix builds or test sharding
• Configuring multi-environment deployments

DevSecOps & Security

• Securing pipelines (secrets, OIDC, supply chain)
• Implementing security scanning (SAST, DAST, SCA)
• Container image vulnerability scanning inside a pipeline
• Secret detection and management
• Enterprise readiness assessment (OpenSSF compliance)

Infrastructure as Code

• Terraform module development
• CloudFormation/CDK templates
• Deploying Kubernetes manifests and Helm charts from CI/CD
• GitOps workflows (ArgoCD, Flux)

Container Pipeline Integration

• Building and signing container images from an existing Dockerfile
• Publishing to registries from CI/CD
• Scanning container images with tools such as Trivy or Snyk
• Deploying containerized services through GitOps

For Dockerfile design, Docker Compose, local container environments, or container runtime architecture, switch to docker-containerization.

Release Management

• Semantic versioning automation
• Changelog generation
• GitHub/GitLab release creation
• Artifact signing and provenance

Quick Start

1. Creating a New Pipeline

Decision tree:

What are you building?
+-- Node.js/Frontend --> templates/github-actions/node-ci.yml | templates/gitlab-ci/node-ci.yml
+-- Python --> templates/github-actions/python-ci.yml | templates/gitlab-ci/python-ci.yml
+-- Go --> templates/github-actions/go-ci.yml | templates/gitlab-ci/go-ci.yml
+-- Container image pipeline --> templates/github-actions/docker-build.yml | templates/gitlab-ci/docker-build.yml
+-- Security Scanning --> templates/github-actions/security-scan.yml | templates/gitlab-ci/security-scan.yml

Basic pipeline structure:

# 1. Fast feedback (lint, format) - <1 min
# 2. Unit tests - 1-5 min
# 3. Integration tests - 5-15 min
# 4. Build artifacts
# 5. E2E tests (optional, main branch only) - 15-30 min
# 6. Deploy (with approval gates)

2. Optimizing Pipeline Performance

Quick wins checklist:

• [ ] Add dependency caching (50-90% faster builds)
• [ ] Remove unnecessary needs dependencies
• [ ] Add path filters to skip unnecessary runs
• [ ] Use npm ci instead of npm install
• [ ] Add job timeouts to prevent hung builds
• [ ] Enable concurrency cancellation for duplicate runs

Analyze existing pipeline:

# Use the pipeline analyzer script
python3 scripts/pipeline_analyzer.py --platform github --workflow .github/workflows/ci.yml

3. Securing Your Pipeline

Essential security checklist:

• [ ] Use OIDC instead of static credentials
• [ ] Pin actions/includes to commit SHAs
• [ ] Use minimal permissions
• [ ] Enable secret scanning
• [ ] Add vulnerability scanning (dependencies, containers)
• [ ] Implement branch protection
• [ ] Separate test from deploy workflows

OIDC Authentication (GitHub Actions to AWS):

permissions:
  id-token: write
  contents: read

steps:
  - uses: aws-actions/configure-aws-credentials@v4
    with:
      role-to-assume: arn:aws:iam::123456789:role/GitHubActionsRole
      aws-region: us-east-1

Core Capabilities

Architecture Patterns

CI/CD Pipeline Flow

Code Commit --> Build --> Test --> Security Scan --> Package
                                                       |
Monitor <-- Release Staging <-- Smoke Tests <-- Deploy Dev
                 |
         Manual Approval
                 |
         Deploy Production

GitOps Architecture

App Repo --CI--> Config Repo --ArgoCD--> K8s Cluster
                     ^                        |
                     +----Continuous Sync-----+

Reference Documentation

CI/CD & Pipeline Design

• references/best_practices.md - Pipeline design patterns, testing strategies, deployment workflows
• references/optimization.md - Caching strategies, parallelization, build performance
• references/troubleshooting.md - Common issues, debugging, platform-specific problems
• references/cicd-github-actions.md - GitHub Actions workflows, runners, secrets

Security & DevSecOps

• references/security.md - Secrets management, OIDC, supply chain security
• references/devsecops.md - SAST, DAST, SCA, container scanning guide
• references/devsecops-basics.md - Security best practices, shift-left security
• references/security-hardening.md - TLS enforcement, input validation, headers

Infrastructure & Cloud Platforms

• references/terraform-eks-module.tf - Production EKS cluster Terraform
• references/kubernetes-deployment.yaml - Example manifest deployed by pipelines
• references/kubernetes-basics.md - Kubernetes concepts needed to understand deployment jobs
• references/docker-basics.md - Pipeline-adjacent container concepts; use docker-containerization for Dockerfile design
• references/docker-compose.md - Pipeline-adjacent compose references; use docker-containerization for local container environments
• references/aws-overview.md - AWS fundamentals, IAM, services
• references/gcloud-platform.md - GCP overview, gcloud CLI
• references/cloudflare-workers-basics.md - Edge computing, Workers

Enterprise Readiness

• references/general.md - Universal enterprise readiness checks
• references/github.md - GitHub-specific enterprise requirements
• references/openssf-badge-silver.md - Silver badge criteria
• references/openssf-badge-gold.md - Gold badge criteria
• references/signed-releases.md - Artifact and tag signing
• references/reproducible-builds.md - Deterministic build patterns

Release Artifact References

• references/signed-releases.md - Artifact and tag signing
• references/reproducible-builds.md - Deterministic build patterns

Templates

GitHub Actions

GitLab CI

Enterprise Templates

Scripts

Pipeline Analysis

Security & Compliance

Infrastructure

Anti-Patterns

Script injection fix:

# DANGEROUS
- run: echo "Title: ${{ github.event.issue.title }}"

# SAFE
- name: Process issue
  env:
    TITLE: ${{ github.event.issue.title }}
  run: echo "Title: $TITLE"

Quick Reference Commands

GitHub Actions

gh workflow list                    # List workflows
gh run list --limit 20              # View recent runs
gh run view <run-id>                # View specific run
gh run rerun <run-id> --failed      # Re-run failed jobs
gh run view <run-id> --log > logs.txt  # Download logs
gh workflow run ci.yml              # Trigger workflow manually

GitLab CI

gl project-pipelines list           # View pipelines
gl project-pipeline get <id>        # Pipeline status
gl project-pipeline retry <id>      # Retry failed jobs
gl project-pipeline cancel <id>     # Cancel pipeline

Docker

docker build -t myapp .             # Build image
docker run -p 3000:3000 myapp       # Run container
docker compose up -d                # Start multi-container app
docker scout cves myapp             # Scan for vulnerabilities

Kubernetes

kubectl apply -f deployment.yaml    # Apply manifest
kubectl get pods,services           # Check status
kubectl logs -f <pod>               # Stream logs
kubectl rollout status deployment/app  # Check rollout

Terraform

terraform init                      # Initialize
terraform plan                      # Preview changes
terraform apply                     # Apply changes
terraform state list                # List resources

Quality Checklist

[ ] All secrets in secret management (not in code)
[ ] Resource limits defined for all containers
[ ] Health checks configured (liveness, readiness)
[ ] Horizontal pod autoscaling enabled
[ ] Security contexts set (non-root, read-only)
[ ] Monitoring and alerting configured
[ ] Rollback strategy documented
[ ] Multi-environment support (dev, staging, prod)
[ ] Concurrency controls in CI pipelines
[ ] Remote state backend for Terraform
[ ] Vulnerability scanning in pipeline
[ ] Version pinning for all dependencies
[ ] Branch protection enabled
[ ] Code review required before merge

Platform Selection Guide

Getting Started

1. New pipeline: Start with a template from templates/
2. Add security scanning: Use DevSecOps templates or add security stages
3. Optimize existing: Run scripts/pipeline_analyzer.py
4. Debug issues: Check references/troubleshooting.md
5. Improve security: Review references/security.md and references/devsecops.md
6. Enterprise readiness: Follow references/general.md checklist
7. Release notes/versioning: switch to git-workflow for changelog, semantic version, and commit convention workflows

Source Skills

This curated skill combines content from the following legacy skills (now part of cicd-pipelines):

• Pipeline design, DevSecOps, optimization
• IaC, Kubernetes, deployment automation
• Cloud platforms, Docker, Cloudflare
• Multi-cloud, FinOps, comprehensive DevOps
• OpenSSF compliance, security assessment
• Artifact signing, provenance, and pipeline gates

Resources

• GitHub Actions: https://docs.github.com/actions
• GitLab CI: https://docs.gitlab.com/ee/ci/
• Terraform: https://developer.hashicorp.com/terraform
• Kubernetes: https://kubernetes.io/docs
• ArgoCD: https://argo-cd.readthedocs.io
• OpenSSF Scorecard: https://securityscorecards.dev/
• SLSA Framework: https://slsa.dev/