CalejoControl/DEPLOYMENT.md

# Calejo Control Adapter - Deployment Guide

This guide provides step-by-step instructions for deploying the Calejo Control Adapter to production, staging, and test environments.

## Table of Contents

1. [Prerequisites](#prerequisites)
2. [Environment Setup](#environment-setup)
3. [SSH Key Configuration](#ssh-key-configuration)
4. [Configuration Files](#configuration-files)
5. [Deployment Methods](#deployment-methods)
6. [Post-Deployment Verification](#post-deployment-verification)
7. [Troubleshooting](#troubleshooting)

## Prerequisites

### Server Requirements

- **Operating System**: Ubuntu 20.04+ or CentOS 8+
- **Docker**: 20.10+
- **Docker Compose**: 2.0+
- **Disk Space**: Minimum 10GB
- **Memory**: Minimum 4GB RAM
- **Network**: Outbound internet access for package updates

### Local Development Machine

- Python 3.11+
- Git
- SSH client
- Required Python packages: `pyyaml`, `paramiko`

## Environment Setup

### 1. Clone the Repository

```bash
git clone http://95.111.206.201:3000/calejocontrol/CalejoControl.git
cd CalejoControl
```

### 2. Install Required Dependencies

```bash
pip install -r requirements.txt
pip install pyyaml paramiko
```

## SSH Key Configuration

### 1. Generate SSH Key Pairs

For each environment, generate dedicated SSH key pairs:

```bash
# Generate production key
ssh-keygen -t ed25519 -f deploy/keys/production_key -C "calejo-production-deploy" -N ""

# Generate staging key  
ssh-keygen -t ed25519 -f deploy/keys/staging_key -C "calejo-staging-deploy" -N ""

# Set proper permissions
chmod 600 deploy/keys/*
```

### 2. Deploy Public Keys to Target Servers

Copy the public keys to the target servers:

```bash
# For production
ssh-copy-id -i deploy/keys/production_key.pub root@95.111.206.155

# For staging
ssh-copy-id -i deploy/keys/staging_key.pub user@staging-server.company.com
```

### 3. Configure SSH on Target Servers

On each server, ensure the deployment user has proper permissions:

```bash
# Add to sudoers (if needed)
echo "calejo ALL=(ALL) NOPASSWD: /usr/bin/docker-compose, /bin/systemctl" | sudo tee /etc/sudoers.d/calejo
```

## Configuration Files

### Production Configuration

Edit `deploy/config/production.yml` with your actual values:

```yaml
# SSH Connection Details
ssh:
  host: "95.111.206.155"
  port: 22
  username: "root"
  key_file: "deploy/keys/production_key"
  
# Deployment Settings
deployment:
  target_dir: "/opt/calejo-control-adapter"
  backup_dir: "/var/backup/calejo"
  log_dir: "/var/log/calejo"
  config_dir: "/etc/calejo"
  
# Application Configuration
app:
  port: 8080
  host: "0.0.0.0"
  debug: false
  
# Database Configuration
database:
  host: "localhost"
  port: 5432
  name: "calejo_production"
  username: "calejo_user"
  password: "${DB_PASSWORD}"  # Will be replaced from environment
  
# SCADA Integration
scada:
  opcua_enabled: true
  opcua_endpoint: "opc.tcp://scada-server:4840"
  modbus_enabled: true
  modbus_host: "scada-server"
  modbus_port: 502
  
# Optimization Integration
optimization:
  enabled: true
  endpoint: "http://optimization-server:8081"
  
# Security Settings
security:
  enable_auth: true
  enable_ssl: true
  ssl_cert: "/etc/ssl/certs/calejo.crt"
  ssl_key: "/etc/ssl/private/calejo.key"
  
# Monitoring
monitoring:
  prometheus_enabled: true
  prometheus_port: 9090
  grafana_enabled: true
  grafana_port: 3000
  
# Backup Settings
backup:
  enabled: true
  schedule: "0 2 * * *"  # Daily at 2 AM
  retention_days: 30
```

### Environment Variables

Create environment files for different environments:

```bash
# Copy template
cp .env.example .env.production

# Edit production environment
nano .env.production
```

Example `.env.production`:
```
DB_PASSWORD=your-secure-password
SECRET_KEY=your-secret-key
DEBUG=False
ALLOWED_HOSTS=your-domain.com,95.111.206.155
```

## Deployment Methods

### Method 1: Python SSH Deployment (Recommended)

This method uses the Python-based deployment script with comprehensive error handling and logging.

#### Dry Run (Test Deployment)

```bash
python deploy/ssh/deploy-remote.py -c deploy/config/production.yml --dry-run
```

#### Actual Deployment

```bash
python deploy/ssh/deploy-remote.py -c deploy/config/production.yml
```

#### Deployment Steps

The Python deployment script performs the following steps:

1. **Connect to target server** via SSH
2. **Check prerequisites** (Docker, Docker Compose)
3. **Create directories** for application, backups, logs, and configuration
4. **Transfer deployment package** containing the application code
5. **Extract and setup** the application
6. **Configure environment** and copy configuration files
7. **Start services** using Docker Compose
8. **Run health checks** to verify deployment

### Method 2: Shell Script Deployment

For simpler deployments, use the shell script:

```bash
./deploy/deploy-onprem.sh
```

### Method 3: Manual Deployment

For complete control over the deployment process:

```bash
# 1. Copy files to server
scp -r . root@95.111.206.155:/opt/calejo-control-adapter/

# 2. SSH into server
ssh root@95.111.206.155

# 3. Navigate to application directory
cd /opt/calejo-control-adapter

# 4. Set up environment
cp .env.example .env
nano .env  # Edit with actual values

# 5. Start services
docker-compose -f docker-compose.production.yml up -d

# 6. Verify deployment
docker-compose -f docker-compose.production.yml logs -f
```

## Post-Deployment Verification

### 1. Run Health Checks

```bash
# From the deployment server
./deploy/validate-deployment.sh
```

### 2. Test Application Endpoints

```bash
# Health check
curl http://95.111.206.155:8080/health

# API endpoints
curl http://95.111.206.155:8080/api/v1/discovery/pump-stations
curl http://95.111.206.155:8080/api/v1/safety/emergency-stop
```

### 3. Check Service Status

```bash
# Check Docker containers
docker-compose -f docker-compose.production.yml ps

# Check application logs
docker-compose -f docker-compose.production.yml logs app

# Check database connectivity
docker-compose -f docker-compose.production.yml exec db psql -U calejo_user -d calejo_production -c "SELECT version();"
```

### 4. Run End-to-End Tests

```bash
# Run comprehensive tests
python tests/integration/test-e2e-deployment.py

# Or use the test runner
python run_tests.py --type integration --verbose
```

## Monitoring and Maintenance

### 1. Set Up Monitoring

```bash
# Deploy monitoring stack
./deploy/setup-monitoring.sh
```

### 2. Backup Configuration

```bash
# Generate monitoring secrets
./deploy/generate-monitoring-secrets.sh
```

### 3. Regular Maintenance Tasks

- **Log rotation**: Configure in `/etc/logrotate.d/calejo`
- **Backup verification**: Check `/var/backup/calejo/`
- **Security updates**: Regular `apt update && apt upgrade`
- **Application updates**: Follow deployment process for new versions

## Troubleshooting

### Common Issues

#### SSH Connection Failed
- Verify SSH key permissions: `chmod 600 deploy/keys/*`
- Check if public key is deployed: `ssh -i deploy/keys/production_key root@95.111.206.155`
- Verify firewall settings on target server

#### Docker Not Available
- Install Docker on target server: `curl -fsSL https://get.docker.com | sh`
- Add user to docker group: `usermod -aG docker $USER`

#### Application Not Starting
- Check logs: `docker-compose logs app`
- Verify environment variables: `cat .env`
- Check database connectivity

#### Port Conflicts
- Change application port in `deploy/config/production.yml`
- Verify no other services are using ports 8080, 9090, 3000

### Debug Mode

For detailed debugging, enable verbose output:

```bash
python deploy/ssh/deploy-remote.py -c deploy/config/production.yml --verbose
```

### Rollback Procedure

If deployment fails, rollback to previous version:

```bash
# Stop current services
docker-compose -f docker-compose.production.yml down

# Restore from backup
cp -r /var/backup/calejo/latest/* /opt/calejo-control-adapter/

# Start previous version
docker-compose -f docker-compose.production.yml up -d
```

## Security Considerations

- **Never commit private keys** to version control
- **Use different SSH keys** for different environments
- **Set proper file permissions**: `chmod 600` for private keys
- **Regularly rotate SSH keys** and database passwords
- **Enable firewall** on production servers
- **Use SSL/TLS** for all external communications
- **Monitor access logs** for suspicious activity

## Support

For deployment issues:

1. Check the logs in `/var/log/calejo/`
2. Review deployment configuration in `deploy/config/`
3. Run validation script: `./deploy/validate-deployment.sh`
4. Contact the development team with error details

---

**Last Updated**: 2025-11-06  
**Version**: 1.0  
**Maintainer**: Calejo Control Team