CalejoControl/docs/PROTOCOL_MAPPING_IMPLEMENTA...

514 lines
21 KiB
Markdown
Raw Normal View History

# Protocol Mapping - Phase 1 Implementation Plan
## Overview
This document outlines the detailed implementation plan for Phase 1 of the Protocol Mapping UI feature, supporting Modbus, OPC UA, and other industrial protocols.
## 🎯 Phase 1 Goals
- Enable basic configuration of database-to-protocol mappings through unified dashboard interface
- Replace hardcoded protocol mappings with configurable system
- Support multiple protocols (Modbus, OPC UA) through single Protocol Mapping tab
- Provide protocol-specific validation within unified interface
- Implement protocol switching within single dashboard tab
## 📋 Detailed Task Breakdown
### Task 1: Extend Configuration Manager with Protocol Mapping Support
**Priority**: High
**Estimated Effort**: 3 days
#### Implementation Details:
```python
# File: src/dashboard/configuration_manager.py
class ProtocolMapping(BaseModel):
"""Protocol mapping configuration for all protocols"""
id: str
protocol_type: str # modbus_tcp, opcua, custom
station_id: str
pump_id: str
data_type: str # setpoint, status, power, etc.
protocol_address: str # register address or OPC UA node
db_source: str
transformation_rules: List[Dict] = []
# Protocol-specific configurations
modbus_config: Optional[Dict] = None
opcua_config: Optional[Dict] = None
class ConfigurationManager:
def __init__(self):
self.protocol_mappings: List[ProtocolMapping] = []
def add_protocol_mapping(self, mapping: ProtocolMapping) -> bool:
"""Add a new protocol mapping with validation"""
def get_protocol_mappings(self,
protocol_type: str = None,
station_id: str = None,
pump_id: str = None) -> List[ProtocolMapping]:
"""Get mappings filtered by protocol/station/pump"""
def validate_protocol_mapping(self, mapping: ProtocolMapping) -> Dict[str, Any]:
"""Validate mapping for conflicts and protocol-specific rules"""
```
### Task 2: Create Protocol Mapping API Endpoints
**Priority**: High
**Estimated Effort**: 2 days
#### Implementation Details:
```python
# File: src/dashboard/api.py
@dashboard_router.get("/protocol-mappings")
async def get_protocol_mappings(
protocol_type: Optional[str] = None,
station_id: Optional[str] = None,
pump_id: Optional[str] = None
):
"""Get all protocol mappings"""
@dashboard_router.post("/protocol-mappings")
async def create_protocol_mapping(mapping: ProtocolMapping):
"""Create a new protocol mapping"""
@dashboard_router.put("/protocol-mappings/{mapping_id}")
async def update_protocol_mapping(mapping_id: str, mapping: ProtocolMapping):
"""Update an existing protocol mapping"""
@dashboard_router.delete("/protocol-mappings/{mapping_id}")
async def delete_protocol_mapping(mapping_id: str):
"""Delete a protocol mapping"""
@dashboard_router.post("/protocol-mappings/validate")
async def validate_protocol_mapping(mapping: ProtocolMapping):
"""Validate a protocol mapping without saving"""
# Protocol-specific endpoints
@dashboard_router.get("/protocol-mappings/modbus")
async def get_modbus_mappings():
"""Get all Modbus mappings"""
@dashboard_router.get("/protocol-mappings/opcua")
async def get_opcua_mappings():
"""Get all OPC UA mappings"""
```
### Task 3: Build Multi-Protocol Configuration Form UI
**Priority**: High
**Estimated Effort**: 3 days
#### Implementation Details:
```html
<!-- File: static/dashboard.js - Add to existing dashboard -->
// Add Protocol Mapping section to dashboard
function createProtocolMappingSection() {
return `
<div class="protocol-mapping-section">
<h3>Protocol Mapping Configuration</h3>
<div class="protocol-selector">
<button class="protocol-btn active" onclick="selectProtocol('modbus')">Modbus</button>
<button class="protocol-btn" onclick="selectProtocol('opcua')">OPC UA</button>
<button class="protocol-btn" onclick="selectProtocol('custom')">Custom</button>
</div>
<div class="mapping-controls">
<button onclick="showMappingForm()">Add Mapping</button>
<button onclick="exportMappings()">Export</button>
</div>
<div id="mapping-grid"></div>
<div id="mapping-form" class="modal hidden">
<!-- Multi-protocol configuration form implementation -->
</div>
</div>
`;
}
```
### Task 4: Implement Protocol Mapping Grid View
**Priority**: Medium
**Estimated Effort**: 2 days
#### Implementation Details:
```javascript
// File: static/dashboard.js
function renderMappingGrid(mappings) {
const grid = document.getElementById('mapping-grid');
grid.innerHTML = `
<table class="mapping-table">
<thead>
<tr>
<th>Protocol</th>
<th>Station</th>
<th>Pump</th>
<th>Data Type</th>
<th>Address</th>
<th>Database Source</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
${mappings.map(mapping => `
<tr class="protocol-${mapping.protocol_type}">
<td><span class="protocol-badge">${mapping.protocol_type}</span></td>
<td>${mapping.station_id}</td>
<td>${mapping.pump_id}</td>
<td>${mapping.data_type}</td>
<td>${mapping.protocol_address}</td>
<td>${mapping.db_source}</td>
<td>
<button onclick="editMapping('${mapping.id}')">Edit</button>
<button onclick="deleteMapping('${mapping.id}')">Delete</button>
</td>
</tr>
`).join('')}
</tbody>
</table>
`;
}
```
### Task 5: Add Protocol-Specific Validation Logic
**Priority**: High
**Estimated Effort**: 2 days
#### Implementation Details:
```python
# File: src/dashboard/configuration_manager.py
class ConfigurationManager:
def validate_protocol_mapping(self, mapping: ProtocolMapping) -> Dict[str, Any]:
"""Validate protocol mapping configuration"""
errors = []
warnings = []
# Protocol-specific validation
if mapping.protocol_type == 'modbus_tcp':
# Modbus validation
try:
address = int(mapping.protocol_address)
if not (0 <= address <= 65535):
errors.append("Modbus register address must be between 0 and 65535")
except ValueError:
errors.append("Modbus address must be a valid integer")
# Check for address conflicts
for existing in self.protocol_mappings:
if (existing.id != mapping.id and
existing.protocol_type == 'modbus_tcp' and
existing.protocol_address == mapping.protocol_address):
errors.append(f"Modbus address {mapping.protocol_address} already used by {existing.station_id}/{existing.pump_id}")
elif mapping.protocol_type == 'opcua':
# OPC UA validation
if not mapping.protocol_address.startswith('ns='):
errors.append("OPC UA Node ID must start with 'ns='")
# Check for node conflicts
for existing in self.protocol_mappings:
if (existing.id != mapping.id and
existing.protocol_type == 'opcua' and
existing.protocol_address == mapping.protocol_address):
errors.append(f"OPC UA node {mapping.protocol_address} already used by {existing.station_id}/{existing.pump_id}")
return {
'valid': len(errors) == 0,
'errors': errors,
'warnings': warnings
}
```
### Task 6: Integrate Configuration Manager with Protocol Servers
**Priority**: High
**Estimated Effort**: 2 days
#### Implementation Details:
```python
# File: src/protocols/modbus_server.py
class ModbusServer:
def __init__(self, setpoint_manager, configuration_manager):
self.setpoint_manager = setpoint_manager
self.configuration_manager = configuration_manager
async def _update_registers(self):
"""Update registers using configured mappings"""
modbus_mappings = self.configuration_manager.get_protocol_mappings('modbus_tcp')
for mapping in modbus_mappings:
try:
# Get value from database/setpoint manager
value = await self._get_mapped_value(mapping)
# Apply transformations
transformed_value = self._apply_transformations(value, mapping.transformation_rules)
# Write to register
self._write_register(mapping.protocol_address, transformed_value, mapping.modbus_config['register_type'])
except Exception as e:
logger.error(f"Failed to update mapping {mapping.id}: {str(e)}")
# File: src/protocols/opcua_server.py
class OPCUAServer:
def __init__(self, configuration_manager):
self.configuration_manager = configuration_manager
async def update_nodes(self):
"""Update OPC UA nodes using configured mappings"""
opcua_mappings = self.configuration_manager.get_protocol_mappings('opcua')
for mapping in opcua_mappings:
try:
# Get value from database/setpoint manager
value = await self._get_mapped_value(mapping)
# Apply transformations
transformed_value = self._apply_transformations(value, mapping.transformation_rules)
# Write to node
await self._write_node(mapping.protocol_address, transformed_value)
except Exception as e:
logger.error(f"Failed to update mapping {mapping.id}: {str(e)}")
```
### Task 7: Create Database Schema for Protocol Mappings
**Priority**: Medium
**Estimated Effort**: 1 day
#### Implementation Details:
```sql
-- File: database/schema.sql
CREATE TABLE IF NOT EXISTS protocol_mappings (
id VARCHAR(50) PRIMARY KEY,
protocol_type VARCHAR(20) NOT NULL, -- modbus_tcp, opcua, custom
station_id VARCHAR(50) NOT NULL,
pump_id VARCHAR(50) NOT NULL,
data_type VARCHAR(50) NOT NULL,
protocol_address VARCHAR(200) NOT NULL, -- register address or OPC UA node
db_source VARCHAR(200) NOT NULL,
transformation_rules JSONB,
-- Protocol-specific configurations
modbus_config JSONB,
opcua_config JSONB,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (station_id, pump_id) REFERENCES pumps(station_id, pump_id)
);
CREATE INDEX idx_protocol_mappings_type ON protocol_mappings(protocol_type);
CREATE INDEX idx_protocol_mappings_station_pump ON protocol_mappings(station_id, pump_id);
CREATE INDEX idx_protocol_mappings_address ON protocol_mappings(protocol_address);
```
### Task 8: Add Protocol-Specific Unit Tests
**Priority**: Medium
**Estimated Effort**: 1.5 days
#### Implementation Details:
```python
# File: tests/unit/test_protocol_mapping.py
class TestProtocolMapping(unittest.TestCase):
def test_modbus_address_conflict_detection(self):
"""Test that Modbus address conflicts are properly detected"""
config_manager = ConfigurationManager()
mapping1 = ProtocolMapping(
id="test1", protocol_type="modbus_tcp", station_id="STATION_001", pump_id="PUMP_001",
data_type="setpoint", protocol_address="40001", db_source="pump_plans.speed_hz"
)
mapping2 = ProtocolMapping(
id="test2", protocol_type="modbus_tcp", station_id="STATION_001", pump_id="PUMP_002",
data_type="setpoint", protocol_address="40001", db_source="pump_plans.speed_hz"
)
config_manager.add_protocol_mapping(mapping1)
result = config_manager.validate_protocol_mapping(mapping2)
self.assertFalse(result['valid'])
self.assertIn("Modbus address 40001 already used", result['errors'][0])
def test_opcua_node_validation(self):
"""Test OPC UA node validation"""
config_manager = ConfigurationManager()
mapping = ProtocolMapping(
id="test1", protocol_type="opcua", station_id="STATION_001", pump_id="PUMP_001",
data_type="setpoint", protocol_address="invalid_node", db_source="pump_plans.speed_hz"
)
result = config_manager.validate_protocol_mapping(mapping)
self.assertFalse(result['valid'])
self.assertIn("OPC UA Node ID must start with 'ns='", result['errors'][0])
```
### Task 9: Add Single Protocol Mapping Tab to Dashboard
**Priority**: Low
**Estimated Effort**: 0.5 days
#### Implementation Details:
```javascript
// File: static/dashboard.js
// Update tab navigation - Add single Protocol Mapping tab
function updateNavigation() {
const tabButtons = document.querySelector('.tab-buttons');
tabButtons.innerHTML += `
<button class="tab-button" onclick="showTab('protocol-mapping')">Protocol Mapping</button>
`;
}
// Add Protocol Mapping tab content
function addProtocolMappingTab() {
const tabContainer = document.querySelector('.tab-container');
tabContainer.innerHTML += `
<!-- Protocol Mapping Tab -->
<div id="protocol-mapping-tab" class="tab-content">
<h2>Protocol Mapping Configuration</h2>
<div class="protocol-selector">
<button class="protocol-btn active" onclick="selectProtocol('modbus')">Modbus</button>
<button class="protocol-btn" onclick="selectProtocol('opcua')">OPC UA</button>
<button class="protocol-btn" onclick="selectProtocol('all')">All Protocols</button>
</div>
<div id="protocol-mapping-content">
<!-- Unified protocol mapping interface will be loaded here -->
</div>
</div>
`;
}
// Protocol switching within the single tab
function selectProtocol(protocol) {
// Update active protocol button
document.querySelectorAll('.protocol-btn').forEach(btn => btn.classList.remove('active'));
event.target.classList.add('active');
// Load protocol-specific content
loadProtocolMappings(protocol);
}
```
### Task 10: Implement Protocol Discovery Features
**Priority**: Medium
**Estimated Effort**: 2 days
#### Implementation Details:
```python
# File: src/dashboard/api.py
@dashboard_router.post("/protocol-mappings/modbus/discover")
async def discover_modbus_registers():
"""Auto-discover available Modbus registers"""
try:
# Scan for available registers
discovered_registers = await modbus_client.scan_registers()
return {"success": True, "registers": discovered_registers}
except Exception as e:
logger.error(f"Failed to discover Modbus registers: {str(e)}")
raise HTTPException(status_code=500, detail=f"Discovery failed: {str(e)}")
@dashboard_router.post("/protocol-mappings/opcua/browse")
async def browse_opcua_nodes():
"""Browse OPC UA server for available nodes"""
try:
# Browse OPC UA server
nodes = await opcua_client.browse_nodes()
return {"success": True, "nodes": nodes}
except Exception as e:
logger.error(f"Failed to browse OPC UA nodes: {str(e)}")
raise HTTPException(status_code=500, detail=f"Browse failed: {str(e)}")
```
## 🔄 Integration Points
### Existing System Integration
1. **Configuration Manager**: Extend existing class with unified protocol mapping support
2. **Protocol Servers**: Inject configuration manager and use configured mappings (Modbus, OPC UA)
3. **Dashboard API**: Add unified protocol mapping endpoints alongside existing configuration endpoints
4. **Dashboard UI**: Add single Protocol Mapping tab with protocol switching
5. **Database**: Add unified table for persistent storage of all protocol mappings
### Data Flow Changes
```
Current: Database → Setpoint Manager → Hardcoded Mapping → Protocol Servers
New: Database → Setpoint Manager → Unified Configurable Mapping → Protocol Servers
Unified Configuration Manager
```
### Dashboard Integration
```
┌─────────────────────────────────────────────────────────────────┐
│ DASHBOARD NAVIGATION │
├─────────────────────────────────────────────────────────────────┤
│ [Status] [Config] [SCADA] [Signals] [Protocol Mapping] [Logs] │
└─────────────────────────────────────────────────────────────────┘
Within Protocol Mapping Tab:
┌─────────────────────────────────────────────────────────────────┐
│ PROTOCOL MAPPING │
├─────────────────────────────────────────────────────────────────┤
│ [Modbus] [OPC UA] [All Protocols] ← Protocol Selector │
│ │
│ Unified Mapping Grid & Configuration Forms │
└─────────────────────────────────────────────────────────────────┘
```
## 🧪 Testing Strategy
### Test Scenarios
1. **Protocol Configuration Validation**: Test address conflicts, data type compatibility across protocols
2. **Integration Testing**: Test that configured mappings are applied correctly to all protocol servers
3. **Protocol-Specific Testing**: Test Modbus register mapping and OPC UA node mapping separately
4. **Performance Testing**: Test impact on protocol server performance
### Test Data
- Create test mappings for different protocols and scenarios
- Test edge cases (address boundaries, data type conversions, protocol-specific rules)
- Test cross-protocol conflict scenarios
## 📊 Success Metrics
### Functional Requirements
- ✅ Users can configure database-to-protocol mappings through dashboard
- ✅ System uses configured mappings for all supported protocols
- ✅ Protocol-specific validation prevents configuration conflicts
- ✅ Mappings are persisted across application restarts
- ✅ Support for multiple protocols (Modbus, OPC UA) with unified interface
### Performance Requirements
- ⏱️ Mapping configuration response time < 500ms
- ⏱️ Protocol server update performance maintained
- 💾 Memory usage increase < 15MB for typical multi-protocol configurations
## 🚨 Risk Mitigation
### Technical Risks
1. **Performance Impact**: Monitor protocol server update times, optimize if needed
2. **Configuration Errors**: Implement comprehensive protocol-specific validation
3. **Protocol Compatibility**: Ensure consistent behavior across different protocols
### Implementation Risks
1. **Scope Creep**: Stick to Phase 1 requirements only
2. **Integration Issues**: Test thoroughly with existing protocol servers
3. **Data Loss**: Implement backup/restore for mapping configurations
## 📅 Estimated Timeline
**Total Phase 1 Effort**: 18.5 days
| Week | Tasks | Deliverables |
|------|-------|--------------|
| 1 | Tasks 1-3 | Configuration manager, API endpoints, multi-protocol UI |
| 2 | Tasks 4-6 | Grid view, protocol-specific validation, server integration |
| 3 | Tasks 7-10 | Database schema, tests, navigation, discovery features |
## 🎯 Next Steps After Phase 1
1. **User Testing**: Gather feedback from operators on multi-protocol interface
2. **Bug Fixing**: Address any issues discovered in production
3. **Phase 2 Planning**: Begin design for enhanced features (drag & drop, templates, bulk operations)
---
*This implementation plan provides a detailed roadmap for delivering Phase 1 of the Protocol Mapping feature, supporting multiple industrial protocols with a unified interface. Each task includes specific implementation details and integration points with the existing system.*