Automating Next.js Multiple Git Branch Deployment with PM2 and Nginx using Python

Introduction

Managing multiple branches of a Next.js application for deployment can be a tedious task if done manually. To make this process smoother, we can automate service management using PM2 and reverse proxy setup with Nginx. However, the cloning of branches should be handled through CI/CD pipelines, leaving the script responsible for setting up services and ensuring they run efficiently.

In this case study, we’ll walk through how to:

• Set up PM2 to manage services.
• Configure Nginx to serve each branch under its own subdomain.
• Continuously monitor services to ensure that everything runs smoothly.

How It Works

1. CI/CD for Cloning: The initial cloning of all branches is done via your CI/CD pipeline. Once cloned, the branches are placed in designated folders.
2. Service Management with PM2: The script then starts services for each branch, manages them, and monitors their health.
3. Nginx Reverse Proxy: For each branch, Nginx is configured to route traffic through a subdomain.

The Code

Here’s a breakdown of the script that will manage services and configure Nginx after the branches have been cloned by CI/CD.

1. Logging Setup

import logging

LOG_FILE = "/path/to/logs/web_folder_monitor.log"

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s",
    handlers=[
        logging.FileHandler(LOG_FILE),
        logging.StreamHandler()
    ]
)

Logging is crucial for monitoring what happens during execution. Logs are stored both in a file and printed to the console.

2. Define Directories and Config Paths

WEB_DIRECTORY = "/path/to/web"
NGINX_CONFIG_PATH = "/path/to/nginx/sites-available"
NGINX_ENABLED_PATH = "/path/to/nginx/sites-enabled"
PROCESSED_FOLDERS_FILE = "/path/to/processed_folders.json"
NGINX_TEMPLATE = """
server {{
    listen 80;
    server_name {domain};

    location / {{
        proxy_pass http://127.0.0.1:{port};
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }}
}}
"""

Here, the WEB_DIRECTORY is where the branches are stored after they’ve been cloned by the CI/CD pipeline. The other paths are for Nginx configuration and a JSON file that tracks processed folders.

3. Generate a Unique Port for Each Branch

def generate_port_from_name(branch_name):
    """Generate a consistent port number from the branch name."""
    unique_string = f"{branch_name}-web"
    hash_object = hashlib.md5(unique_string.encode())
    hash_value = int(hash_object.hexdigest(), 16)
    port = 3002 + (hash_value % (65000 - 3002))
    logging.info(f"Generated port {port} for branch {branch_name}")
    return port

We generate a unique port for each branch to avoid conflicts when serving multiple branches simultaneously.

4. Load and Save Processed Folders

def load_processed_folders():
    """Load the processed folders from a JSON file."""
    if os.path.exists(PROCESSED_FOLDERS_FILE):
        with open(PROCESSED_FOLDERS_FILE, 'r') as f:
            return set(json.load(f))
    return set()

def save_processed_folders(processed_folders):
    """Save the processed folders to a JSON file."""
    with open(PROCESSED_FOLDERS_FILE, 'w') as f:
        json.dump(list(processed_folders), f)

These functions load and save the list of processed folders, so the script knows which branches have already been handled.

5. Start PM2 Service for a Branch

def start_pm2_service(folder_name):
    """Start PM2 service for a folder."""
    try:
        service_name = f"{folder_name}-web"
        logging.info(f"Starting PM2 service for folder: {folder_name}")

        port = generate_port_from_name(folder_name)
        project_path = os.path.join(WEB_DIRECTORY, folder_name)

        subprocess.run(
            ["pm2", "start", "npm", "--name", service_name, "--", "run", "start", "-p", str(port)],
            cwd=project_path,
            check=True
        )

        logging.info(f"PM2 service started for {folder_name} on port {port}")
    except subprocess.CalledProcessError as e:
        logging.error(f"Error starting PM2 service for {folder_name}: {str(e)}")

This function starts a PM2 service for each branch, ensuring that the branch is served on a unique port.

6. Nginx Configuration for Each Branch

def create_nginx_configuration(folder_name):
    """Create Nginx configuration file and reload Nginx."""
    try:
        logging.info(f"Creating Nginx config for folder: {folder_name}")
        port = generate_port_from_name(folder_name)
        domain = f"{folder_name}.web.example.com"
        config_file_path = os.path.join(NGINX_CONFIG_PATH, f"{folder_name}-web.conf")

        # Write Nginx configuration
        nginx_config = NGINX_TEMPLATE.format(domain=domain, port=port)
        with open(config_file_path, "w") as config_file:
            config_file.write(nginx_config)

        # Create symlink in sites-enabled
        symlink_path = os.path.join(NGINX_ENABLED_PATH, f"{folder_name}-web.conf")
        subprocess.run(["ln", "-s", config_file_path, symlink_path], check=True)

        # Reload Nginx
        subprocess.run(["nginx", "-s", "reload"], check=True)
        logging.info(f"Nginx reloaded for {folder_name}")

    except subprocess.CalledProcessError as e:
        logging.error(f"Error creating Nginx config for {folder_name}: {str(e)}")

For each branch, a new Nginx configuration is created to route traffic to the right port using a subdomain like branchname.web.example.com.

7. Putting It All Together

def create_nginx_and_run_pm2(folder_name):
    """Ensure Nginx and PM2 services are running correctly."""
    create_nginx_configuration(folder_name)
    start_pm2_service(folder_name)

if __name__ == "__main__":
    logging.info("Starting folder monitoring and deployment service for web.")
    processed_folders = load_processed_folders()

    # After CI/CD clones the branches, check and process them
    for folder_name in os.listdir(WEB_DIRECTORY):
        if os.path.isdir(os.path.join(WEB_DIRECTORY, folder_name)) and folder_name not in processed_folders:
            create_nginx_and_run_pm2(folder_name)
            processed_folders.add(folder_name)
            save_processed_folders(processed_folders)

    # Continue monitoring for changes and redeploying services as needed
    while True:
        logging.info("Monitoring for changes and checking PM2 services...")
        time.sleep(300)

This script:

1. Processes each branch cloned by the CI/CD pipeline.
2. Sets up a PM2 service for each branch and configures Nginx to serve the branch via a subdomain.
3. Continuously monitors services, ensuring they are always online.

Conclusion

This approach automates the deployment process for your Next.js branches after they’ve been cloned by a CI/CD pipeline. By using PM2 to manage services and Nginx to route traffic, you can ensure that your applications are always available and running efficiently.

While the CI/CD pipeline handles the branch cloning, this script manages everything post-cloning, ensuring that services are started, monitored, and served via the correct subdomains.

Feel free to adapt this solution to suit your specific deployment needs!