CalejoControl/docs/optimization_plan_management.md

# Optimization Plan Management

## Overview

The Calejo Control Adapter implements a sophisticated version-based optimization plan management system that enables real-time synchronization between optimization intervals and control execution. This system supports both pump-specific plans and generic actuator optimization plans.

## Database Schema

### Version-Based Pump Plans (`pump_plans`)

```sql
CREATE TABLE pump_plans (
    plan_id SERIAL PRIMARY KEY,
    station_id VARCHAR(50) NOT NULL,
    pump_id VARCHAR(50) NOT NULL,
    interval_start TIMESTAMP NOT NULL,
    interval_end TIMESTAMP NOT NULL,
    
    -- Optimization outputs
    target_flow_m3h DECIMAL(10, 2),
    target_power_kw DECIMAL(10, 2),
    target_level_m DECIMAL(5, 2),
    suggested_speed_hz DECIMAL(5, 2),
    
    -- Version-based update metadata
    plan_created_at TIMESTAMP DEFAULT NOW(),
    plan_updated_at TIMESTAMP DEFAULT NOW(),
    plan_version INTEGER DEFAULT 1,
    optimization_run_id INTEGER,
    
    -- Status tracking
    plan_status VARCHAR(20) DEFAULT 'ACTIVE',  -- 'ACTIVE', 'SUPERSEDED', 'CANCELLED'
    superseded_by INTEGER,  -- Points to plan_id that superseded this plan
    
    FOREIGN KEY (station_id, pump_id) REFERENCES pumps(station_id, pump_id),
    FOREIGN KEY (superseded_by) REFERENCES pump_plans(plan_id)
);
```

### Generic Optimization Plans (`optimization_plans`)

```sql
CREATE TABLE optimization_plans (
    plan_id SERIAL PRIMARY KEY,
    station_id VARCHAR(50) NOT NULL,
    resource_id VARCHAR(50) NOT NULL,
    resource_type VARCHAR(20) NOT NULL DEFAULT 'PUMP',  -- 'PUMP', 'VALVE', 'BLOWER', etc.
    
    interval_start TIMESTAMP NOT NULL,
    interval_end TIMESTAMP NOT NULL,
    
    -- Generic optimization targets (JSON for flexibility)
    optimization_targets JSONB NOT NULL,
    
    -- Version-based update metadata
    plan_created_at TIMESTAMP DEFAULT NOW(),
    plan_updated_at TIMESTAMP DEFAULT NOW(),
    plan_version INTEGER DEFAULT 1,
    optimization_run_id INTEGER,
    plan_priority INTEGER DEFAULT 1,
    
    -- Status tracking
    plan_status VARCHAR(20) DEFAULT 'ACTIVE',
    superseded_by INTEGER,
    
    FOREIGN KEY (station_id, resource_id) REFERENCES pumps(station_id, pump_id),
    FOREIGN KEY (superseded_by) REFERENCES optimization_plans(plan_id),
    
    -- Ensure resource_type is valid
    CONSTRAINT valid_resource_type CHECK (resource_type IN ('PUMP', 'VALVE', 'BLOWER', 'COMPRESSOR', 'GATE'))
);
```

## Version-Based Update Strategy (Strategy B)

### Core Principles

1. **Immutable Plan History**: Each plan update creates a new version while preserving the old version
2. **Status Tracking**: Plans can be `ACTIVE`, `SUPERSEDED`, or `CANCELLED`
3. **Audit Trail**: Complete history of all plan changes with timestamps
4. **Real-time Synchronization**: Support for optimization systems that update more frequently than control intervals

### Update Process

```python
# Example update sequence
plan_id = 123
updates = {
    'target_flow_m3h': 325.0,
    'target_power_kw': 66.5,
    'suggested_speed_hz': 42.8
}

# This creates a new version and marks the old one as SUPERSEDED
success = db_client.update_pump_plan(plan_id, updates)
```

### Query Strategy

```sql
-- Get latest active plans for current time interval
SELECT DISTINCT ON (station_id, pump_id)
    station_id, pump_id, target_flow_m3h, target_power_kw,
    target_level_m, suggested_speed_hz, interval_start, interval_end,
    plan_version, plan_created_at, plan_updated_at
FROM pump_plans
WHERE interval_start <= NOW() AND interval_end >= NOW()
  AND plan_status = 'ACTIVE'
ORDER BY station_id, pump_id, plan_version DESC;
```

## Optimization Targets JSON Structure

### Pump Optimization Targets

```json
{
    "target_type": "SPEED",
    "target_value": 42.3,
    "secondary_targets": {
        "power_kw": 65.2,
        "flow_m3h": 320.5,
        "level_m": 2.5
    },
    "constraints": {
        "max_rate_of_change": 5.0,
        "deadband": 0.2
    }
}
```

### Valve Optimization Targets

```json
{
    "target_type": "POSITION",
    "target_value": 75.0,
    "secondary_targets": {
        "flow_m3h": 200.0,
        "pressure_bar": 1.5
    },
    "constraints": {
        "max_rate_of_change": 10.0,
        "deadband": 1.0
    }
}
```

### Blower Optimization Targets

```json
{
    "target_type": "SPEED",
    "target_value": 35.0,
    "secondary_targets": {
        "power_kw": 25.0,
        "pressure_bar": 1.2
    },
    "constraints": {
        "max_rate_of_change": 3.0,
        "deadband": 0.5
    }
}
```

## Real-Time Plan Management

### OptimizationPlanManager Class

The `OptimizationPlanManager` provides real-time monitoring and synchronization:

```python
# Initialize manager
manager = OptimizationPlanManager(
    db_client=db_client,
    refresh_interval_seconds=30
)

# Start monitoring
await manager.start_monitoring()

# Register for plan updates
manager.register_plan_update_callback(handle_plan_update)

# Get current plans
current_plan = manager.get_current_pump_plan('STATION_001', 'PUMP_001')
```

### Plan Update Detection

The manager automatically detects plan updates by:

1. **Periodic Refresh**: Configurable refresh interval (default: 30 seconds)
2. **Version Comparison**: Compares plan versions to detect updates
3. **Callback Notification**: Notifies registered callbacks of plan changes
4. **Status Tracking**: Maintains cache of current active plans

### Update Types

- `NEW_PUMP_PLAN`: New plan detected for a pump
- `PUMP_PLAN_UPDATE`: Version update for existing pump plan
- `NEW_GENERIC_PLAN`: New plan for generic actuator
- `GENERIC_PLAN_UPDATE`: Version update for generic actuator plan

## Integration with Control System

### Safety Enforcement Integration

```python
# Get current optimization plan
current_plan = optimization_manager.get_current_pump_plan(station_id, pump_id)

if current_plan:
    # Extract suggested speed
    suggested_speed = current_plan['suggested_speed_hz']
    
    # Apply safety enforcement
    enforced_speed, violations = safety_enforcer.enforce_setpoint(
        station_id, pump_id, suggested_speed
    )
    
    # Send to control system
    await send_to_scada(station_id, pump_id, enforced_speed)
```

### Real-Time Plan Updates

```python
async def handle_plan_update(updates):
    """Handle real-time optimization plan updates."""
    for update in updates:
        if update['type'] == 'PUMP_PLAN_UPDATE':
            logger.info(
                "pump_plan_updated_realtime",
                station_id=update['station_id'],
                pump_id=update['pump_id'],
                old_version=update['old_version'],
                new_version=update['new_version']
            )
            
            # Immediately apply updated plan
            await apply_updated_plan(update['plan'])
```

## Configuration

### Settings

```python
# Auto-discovery settings
auto_discovery_enabled: bool = True
auto_discovery_refresh_minutes: int = 60

# Optimization plan management settings
optimization_monitoring_enabled: bool = True
optimization_refresh_seconds: int = 30
```

## Benefits

### For Optimization System

1. **Flexible Update Frequency**: Optimization can update plans more frequently than control intervals
2. **Audit Trail**: Complete history of all optimization decisions
3. **Rollback Capability**: Can revert to previous plan versions if needed
4. **Multi-Actuator Support**: Unified system for pumps, valves, blowers, etc.

### For Control System

1. **Real-time Synchronization**: Immediate application of updated optimization plans
2. **Safety Integration**: All optimization targets pass through safety enforcement
3. **Status Monitoring**: Real-time visibility into optimization plan status
4. **Error Recovery**: Automatic handling of optimization system failures

### For Operations

1. **Transparency**: Complete visibility into optimization decisions and changes
2. **Compliance**: Audit trail for regulatory requirements
3. **Troubleshooting**: Historical data for performance analysis
4. **Flexibility**: Support for various optimization strategies and intervals

## Extensibility

The system is designed to support future extensions:

1. **Additional Actuator Types**: Easy to add new resource types
2. **Advanced Optimization Strategies**: Support for complex multi-actuator coordination
3. **Machine Learning Integration**: Real-time model updates and predictions
4. **Distributed Optimization**: Support for multiple optimization engines

## Performance Considerations

1. **Indexed Queries**: Efficient retrieval of current plans
2. **Caching**: In-memory cache of active plans for fast access
3. **Partial Indexes**: Only index active plans for better performance
4. **Connection Pooling**: Efficient database access patterns