The REST API server implementation uses uvicorn.Server(config).serve() which blocks the event loop, preventing the test from proceeding. This requires architectural refactoring to make the server testable. All other integration tests (5/5 optimization-to-SCADA, 5/5 safety workflows) are now passing successfully. |
||
|---|---|---|
| config | ||
| database | ||
| docs | ||
| src | ||
| tests | ||
| .coverage | ||
| .gitignore | ||
| Dockerfile | ||
| FINAL_TEST_SUMMARY.md | ||
| FLEXIBLE_DATABASE_CLIENT_SUMMARY.md | ||
| IMPLEMENTATION_PLAN.md | ||
| LICENSE | ||
| PHASE5_ACTUAL_VERIFICATION.md | ||
| PHASE5_SUMMARY.md | ||
| PHASE5_VERIFICATION.md | ||
| PHASE_2_COMPLETION_SUMMARY.md | ||
| PHASE_3_COMPLETION_SUMMARY.md | ||
| README.md | ||
| TEST_INVESTIGATION_SUMMARY.md | ||
| TEST_RESULTS_SUMMARY.md | ||
| package.json | ||
| pytest.ini | ||
| requirements.txt | ||
| run_tests.py | ||
| run_tests_by_system.py | ||
| run_tests_detailed.py | ||
| run_tests_with_db.sh | ||
README.md
Calejo Control Adapter
Multi-protocol integration adapter for municipal wastewater pump stations with comprehensive safety and security framework
Overview
The Calejo Control Adapter translates optimized pump control plans from Calejo Optimize into real-time control signals for municipal wastewater pump stations. It supports diverse SCADA systems with minimal configuration through automatic discovery and multiple protocol support.
Implementation Status
✅ Phase 1: Core Infrastructure
- Database connection pooling with FlexibleDatabaseClient
- Auto-discovery of pump stations and pumps
- Safety framework with limit enforcement
- Emergency stop management
- Optimization plan management
✅ Phase 2: Multi-Protocol Servers
- OPC UA server implementation
- Modbus TCP server implementation
- REST API server implementation
- Database watchdog for failsafe operation
- Alert management system
✅ Phase 3: Setpoint Management
- Setpoint Manager for real-time control with three calculator types:
DIRECT_SPEED: Direct speed controlLEVEL_CONTROLLED: Level-based control with feedbackPOWER_CONTROLLED: Power-based control with feedback
- Integration with all safety components
- Unified main application
- 15 comprehensive unit tests for SetpointManager
✅ Phase 4: Security Layer
- JWT-based authentication with bcrypt password hashing
- Role-based access control (RBAC) with four user roles
- TLS/SSL encryption with certificate management
- Compliance audit logging for IEC 62443, ISO 27001, and NIS2
- 56 comprehensive security tests (24 auth/authz, 17 TLS, 15 audit)
✅ Phase 5: Protocol Server Enhancements
- Enhanced OPC UA server with node caching and performance monitoring
- Optimized Modbus TCP server with connection pooling and industrial features
- Enhanced REST API with OpenAPI documentation, response caching, and compression
- Protocol-specific security enhancements and performance optimizations
- 31 comprehensive tests for protocol enhancements (23 unit + 8 integration)
⏳ Phase 6: Integration and Testing (Pending)
- End-to-end testing
- Validation with real SCADA systems
⏳ Phase 7: Production Hardening (Pending)
- Performance optimization
- Monitoring and alerting
Current Status: All 164 tests passing (100% success rate)
Recent Updates:
- SetpointManager fully integrated with main application
- Added start/stop methods for SetpointManager
- Fixed configuration settings and database pool parameters
- Updated protocol server initializations
- Verified main application starts and stops gracefully
Key Features
- Multi-Protocol Support: OPC UA, Modbus TCP, and REST API simultaneously
- Auto-Discovery: Automatically discovers pump stations and pumps from database
- Safety Framework: Multi-layer limits, watchdogs, emergency stop, failsafe mechanisms
- Security: Authentication, authorization, encryption, audit logging
- Compliance: IEC 62443, NIS2 Directive, ISO 27001
- High Availability: Caching, failover, health monitoring
Architecture
┌─────────────────────────────────────────────────────────┐
│ Calejo Optimize Container (Existing) │
│ - Optimization Engine │
│ - PostgreSQL Database (pump plans) │
└─────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────┐
│ Calejo Control Adapter (IMPLEMENTED) │
│ │
│ ┌────────────────────────────────────────────────┐ │
│ │ Core Components: │ │
│ │ 1. Auto-Discovery Module ✅ │ │
│ │ 2. Safety Framework ✅ │ │
│ │ 3. Emergency Stop Manager ✅ │ │
│ │ 4. Optimization Plan Manager ✅ │ │
│ │ 5. Setpoint Manager ✅ │ │
│ │ 6. Database Watchdog ✅ │ │
│ │ 7. Alert Manager ✅ │ │
│ │ 8. Multi-Protocol Server ✅ │ │
│ │ - OPC UA Server │ │
│ │ - Modbus TCP Server │ │
│ │ - REST API │ │
│ └────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────┘
↓
(Multiple Protocols)
↓
┌─────────────────┼─────────────────┐
↓ ↓ ↓
Siemens WinCC Schneider EcoStruxure Rockwell FactoryTalk
Project Structure
calejo-control-adapter/
├── src/
│ ├── core/
│ │ ├── auto_discovery.py # Auto-discovery module
│ │ ├── safety.py # Safety framework
│ │ ├── emergency_stop.py # Emergency stop manager
│ │ ├── optimization_manager.py # Optimization plan manager
│ │ └── setpoint_manager.py # Setpoint manager
│ ├── protocols/
│ │ ├── opcua_server.py # OPC UA server
│ │ ├── modbus_server.py # Modbus TCP server
│ │ └── rest_api.py # REST API server
│ ├── database/
│ │ ├── flexible_client.py # Flexible database client
│ │ └── models.py # Data models
│ ├── monitoring/
│ │ ├── watchdog.py # Database watchdog
│ │ └── alerts.py # Alert manager
│ └── main.py # Main application
├── tests/
├── config/
│ ├── settings.py # Application settings
│ └── docker-compose.yml # Docker configuration
├── docs/
│ ├── specification.txt # Full implementation specification
│ ├── optimization_plan_management.md # Optimization system documentation
│ └── alert_system_setup.md # Alert system configuration guide
├── requirements.txt # Python dependencies
├── Dockerfile # Docker container definition
└── README.md # This file
Getting Started
Prerequisites
- Python 3.11+
- PostgreSQL 14+
- Docker (optional)
Installation
-
Clone the repository
git clone http://95.111.206.201:3000/calejocontrol/CalejoControl.git cd calejo-control-adapter -
Install dependencies
pip install -r requirements.txt -
Set up environment variables
cp config/.env.example config/.env # Edit config/.env with your database and security settings -
Run the application
python -m src.main
Docker Deployment
# Build the container
docker build -t calejo-control-adapter .
# Run the container
docker run -p 8080:8080 -p 4840:4840 -p 502:502 calejo-control-adapter
Configuration
Key configuration options:
DATABASE_URL: PostgreSQL connection stringOPC_UA_ENDPOINT: OPC UA server endpoint (default: opc.tcp://0.0.0.0:4840)MODBUS_PORT: Modbus TCP port (default: 502)REST_API_PORT: REST API port (default: 8080)SAFETY_TIMEOUT_SECONDS: Database watchdog timeout (default: 1200)
Alert System Configuration
For detailed alert system setup (email, SMS, webhook integration), see: Alert System Setup Guide
Safety Framework
The adapter implements a comprehensive three-layer safety architecture:
- Layer 1: Physical Hard Limits (PLC/VFD) - 15-55 Hz
- Layer 2: Station Safety Limits (Database) - 20-50 Hz (enforced by adapter)
- Layer 3: Optimization Constraints (Calejo Optimize) - 25-45 Hz
Safety Features
- Hard Operational Limits: Speed, level, power, and flow limits
- Rate of Change Limits: Prevent sudden speed changes
- Database Watchdog: Reverts to safe defaults if updates stop
- Emergency Stop: Manual override capability
- Failsafe Mode: Automatic fallback to default setpoints
Security & Compliance
- Authentication: JWT tokens and certificate-based authentication
- Authorization: Role-based access control (RBAC)
- Encryption: TLS/SSL for all communications
- Audit Logging: Immutable audit trail for compliance
- Standards: IEC 62443, ISO 27001, NIS2 Directive
Development
Running Tests
pytest tests/
Code Quality
flake8 src/ tests/ # Linting
mypy src/ # Type checking
black src/ tests/ # Code formatting
Contributing
- Fork the repository
- Create a feature branch
- Make your changes
- Add tests
- Submit a pull request
License
This project is licensed under the MIT License - see the LICENSE file for details.
Support
For support and documentation, refer to the specification in docs/specification.txt or contact the development team.