CalejoControl/README.md

# Calejo Control Adapter

**Multi-protocol integration adapter for municipal wastewater pump stations with comprehensive safety and security framework**

[![Python Version](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker](https://img.shields.io/badge/docker-ready-blue)](https://www.docker.com/)

## Overview

The Calejo Control Adapter translates optimized pump control plans from Calejo Optimize into real-time control signals for municipal wastewater pump stations. It supports diverse SCADA systems with minimal configuration through automatic discovery and multiple protocol support.

### Implementation Status

✅ **Phase 1**: Core Infrastructure
- Database connection pooling with FlexibleDatabaseClient
- Auto-discovery of pump stations and pumps
- Safety framework with limit enforcement
- Emergency stop management
- Optimization plan management

✅ **Phase 2**: Multi-Protocol Servers
- OPC UA server implementation
- Modbus TCP server implementation
- REST API server implementation
- Database watchdog for failsafe operation
- Alert management system

✅ **Phase 3**: Setpoint Management
- Setpoint Manager for real-time control with three calculator types:
  - `DIRECT_SPEED`: Direct speed control
  - `LEVEL_CONTROLLED`: Level-based control with feedback
  - `POWER_CONTROLLED`: Power-based control with feedback
- Integration with all safety components
- Unified main application
- 15 comprehensive unit tests for SetpointManager

✅ **Phase 4**: Security Layer
- JWT-based authentication with bcrypt password hashing
- Role-based access control (RBAC) with four user roles
- TLS/SSL encryption with certificate management
- Compliance audit logging for IEC 62443, ISO 27001, and NIS2
- 56 comprehensive security tests (24 auth/authz, 17 TLS, 15 audit)

✅ **Phase 5**: Protocol Server Enhancements
- Enhanced OPC UA server with node caching and performance monitoring
- Optimized Modbus TCP server with connection pooling and industrial features
- Enhanced REST API with OpenAPI documentation, response caching, and compression
- Protocol-specific security enhancements and performance optimizations
- 31 comprehensive tests for protocol enhancements (23 unit + 8 integration)

⏳ **Phase 6**: Integration and Testing (Pending)
- End-to-end testing
- Validation with real SCADA systems

⏳ **Phase 7**: Production Hardening (Pending)
- Performance optimization
- Monitoring and alerting

**Current Status**: All 220 tests passing (100% success rate)

**Recent Updates**:
- SetpointManager fully integrated with main application
- Added start/stop methods for SetpointManager
- Fixed configuration settings and database pool parameters
- Updated protocol server initializations
- Verified main application starts and stops gracefully
- **Fixed main application integration with enhanced protocol servers**
- **All 220 tests passing (100% success rate)**

### Key Features

- **Multi-Protocol Support**: OPC UA, Modbus TCP, and REST API simultaneously
- **Auto-Discovery**: Automatically discovers pump stations and pumps from database
- **Safety Framework**: Multi-layer limits, watchdogs, emergency stop, failsafe mechanisms
- **Security**: Authentication, authorization, encryption, audit logging
- **Compliance**: IEC 62443, NIS2 Directive, ISO 27001
- **High Availability**: Caching, failover, health monitoring

## Architecture

```
┌─────────────────────────────────────────────────────────┐
│  Calejo Optimize Container (Existing)                   │
│  - Optimization Engine                                  │
│  - PostgreSQL Database (pump plans)                     │
└─────────────────────────────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────┐
│  Calejo Control Adapter (IMPLEMENTED)                   │
│                                                          │
│  ┌────────────────────────────────────────────────┐    │
│  │  Core Components:                               │    │
│  │  1. Auto-Discovery Module ✅                    │    │
│  │  2. Safety Framework ✅                         │    │
│  │  3. Emergency Stop Manager ✅                   │    │
│  │  4. Optimization Plan Manager ✅                │    │
│  │  5. Setpoint Manager ✅                         │    │
│  │  6. Database Watchdog ✅                        │    │
│  │  7. Alert Manager ✅                            │    │
│  │  8. Multi-Protocol Server ✅                    │    │
│  │     - OPC UA Server                             │    │
│  │     - Modbus TCP Server                         │    │
│  │     - REST API                                  │    │
│  └────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────┘
                          ↓
                  (Multiple Protocols)
                          ↓
        ┌─────────────────┼─────────────────┐
        ↓                 ↓                  ↓
   Siemens WinCC    Schneider EcoStruxure  Rockwell FactoryTalk
```

## Project Structure

```
calejo-control-adapter/
├── src/
│   ├── core/
│   │   ├── auto_discovery.py     # Auto-discovery module
│   │   ├── safety.py             # Safety framework
│   │   ├── emergency_stop.py     # Emergency stop manager
│   │   ├── optimization_manager.py # Optimization plan manager
│   │   └── setpoint_manager.py   # Setpoint manager
│   ├── protocols/
│   │   ├── opcua_server.py       # OPC UA server
│   │   ├── modbus_server.py      # Modbus TCP server
│   │   └── rest_api.py           # REST API server
│   ├── database/
│   │   ├── flexible_client.py    # Flexible database client
│   │   └── models.py             # Data models
│   ├── monitoring/
│   │   ├── watchdog.py           # Database watchdog
│   │   └── alerts.py             # Alert manager
│   └── main.py                   # Main application
├── tests/
├── config/
│   ├── settings.py               # Application settings
│   └── docker-compose.yml        # Docker configuration
├── docs/
│   ├── ARCHITECTURE.md           # Comprehensive system architecture
│   ├── SAFETY_FRAMEWORK.md       # Multi-layer safety architecture
│   ├── SECURITY_COMPLIANCE.md    # Security controls and compliance
│   ├── PROTOCOL_INTEGRATION.md   # OPC UA, Modbus, REST API integration
│   ├── INSTALLATION_CONFIGURATION.md # Installation and configuration guide
│   ├── specification.txt         # Full implementation specification
│   ├── optimization_plan_management.md  # Optimization system documentation
│   └── alert_system_setup.md     # Alert system configuration guide
├── requirements.txt              # Python dependencies
├── Dockerfile                    # Docker container definition
└── README.md                     # This file
```

## 🚀 Simplified Deployment

### One-Click Setup

**Run one script, then configure everything through the web dashboard.**

```bash
# Run the setup script (auto-detects configuration from deploy/ directory)
./setup-server.sh

# For local development
./setup-server.sh -h localhost

# Preview what will be done
./setup-server.sh --dry-run
```

The script automatically reads from existing deployment configuration files and handles everything:
- Server provisioning and dependency installation
- Application deployment and service startup
- SSL certificate generation
- Health validation

### Web-Based Configuration

After setup, access the dashboard at `http://your-server:8080/dashboard` to configure:
- SCADA protocols (OPC UA, Modbus TCP)
- Pump stations and hardware
- Safety limits and emergency procedures
- User accounts and permissions
- Monitoring and alerts

**No manual configuration files or SSH access needed!**

---

### Default Credentials

After deployment:
- **Grafana Dashboard**: admin / admin (http://localhost:3000)
- **Prometheus Metrics**: No authentication required (http://localhost:9091)
- **PostgreSQL Database**: calejo / password (localhost:5432)
- **Main Dashboard**: http://localhost:8080/dashboard

**Security Note**: Change default passwords after first login!

---

### Traditional Installation (Alternative)

If you prefer manual setup:

#### Prerequisites
- Python 3.11+
- PostgreSQL 14+
- Docker (optional)

#### Manual Installation

1. **Clone the repository**
   ```bash
   git clone http://95.111.206.201:3000/calejocontrol/CalejoControl.git
   cd calejo-control-adapter
   ```

2. **Install dependencies**
   ```bash
   pip install -r requirements.txt
   ```

3. **Set up environment variables**
   ```bash
   cp config/.env.example config/.env
   # Edit config/.env with your database and security settings
   ```

4. **Run the application**
   ```bash
   python -m src.main
   ```

#### Docker Deployment

```bash
# Build the container
docker build -t calejo-control-adapter .

# Run the container
docker run -p 8080:8080 -p 4840:4840 -p 502:502 calejo-control-adapter
```

## Configuration

Key configuration options:

- `DATABASE_URL`: PostgreSQL connection string
- `OPC_UA_ENDPOINT`: OPC UA server endpoint (default: opc.tcp://0.0.0.0:4840)
- `MODBUS_PORT`: Modbus TCP port (default: 502)
- `REST_API_PORT`: REST API port (default: 8080)
- `SAFETY_TIMEOUT_SECONDS`: Database watchdog timeout (default: 1200)

### Documentation

Comprehensive documentation is available in the `docs/` directory:

- **[System Architecture](docs/ARCHITECTURE.md)**: Complete system architecture and component interactions
- **[Safety Framework](docs/SAFETY_FRAMEWORK.md)**: Multi-layer safety architecture and emergency procedures
- **[Security & Compliance](docs/SECURITY_COMPLIANCE.md)**: Security controls and regulatory compliance framework
- **[Protocol Integration](docs/PROTOCOL_INTEGRATION.md)**: OPC UA, Modbus TCP, and REST API integration guide
- **[Installation & Configuration](docs/INSTALLATION_CONFIGURATION.md)**: Step-by-step installation and configuration guide
- **[Alert System Setup](docs/alert_system_setup.md)**: Alert system configuration (email, SMS, webhook)
- **[Optimization Plan Management](docs/optimization_plan_management.md)**: Optimization plan processing and management

### Alert System Configuration

For detailed alert system setup (email, SMS, webhook integration), see:
[Alert System Setup Guide](docs/alert_system_setup.md)

## Safety Framework

The adapter implements a comprehensive three-layer safety architecture:

1. **Layer 1**: Physical Hard Limits (PLC/VFD) - 15-55 Hz
2. **Layer 2**: Station Safety Limits (Database) - 20-50 Hz (enforced by adapter)
3. **Layer 3**: Optimization Constraints (Calejo Optimize) - 25-45 Hz

### Safety Features

- **Hard Operational Limits**: Speed, level, power, and flow limits
- **Rate of Change Limits**: Prevent sudden speed changes
- **Database Watchdog**: Reverts to safe defaults if updates stop
- **Emergency Stop**: Manual override capability
- **Failsafe Mode**: Automatic fallback to default setpoints

## Security & Compliance

- **Authentication**: JWT tokens and certificate-based authentication
- **Authorization**: Role-based access control (RBAC)
- **Encryption**: TLS/SSL for all communications
- **Audit Logging**: Immutable audit trail for compliance
- **Standards**: IEC 62443, ISO 27001, NIS2 Directive

## Development

### Running Tests

```bash
pytest tests/
```

### Code Quality

```bash
flake8 src/ tests/          # Linting
mypy src/                   # Type checking
black src/ tests/           # Code formatting
```

### Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

## License

This project is licensed under the MIT License - see the LICENSE file for details.

## Support

For support and documentation, refer to the specification in `docs/specification.txt` or contact the development team.