Constrained Markov Decision Processes for Multi-Cloud Resource Orchestration, A Provably-Safe CARL Framework for Industrial IoT and Cyber-Physical Systems

• Billy Elly

  Lautech

  Author

Multi-cloud resource orchestration for Industrial IoT and Cyber-Physical Systems (CPS) must simultaneously optimize cost efficiency, maintain Service Level Agreement (SLA) compliance, and guarantee safety constraints—a challenge that traditional reinforcement learning approaches struggle to address due to their inability to provide provable safety guarantees. This study develops and validates a Constrained Markov Decision Process (CMDP) framework integrated with the Cost-Aware Reinforcement Learning (CARL) architecture for multi-cloud auto-scaling in industrial environments. The research employs a design-based methodology combining retrospective analysis of cloud workload traces with prospective simulation across heterogeneous multi-cloud configurations. The framework implements Lagrangian relaxation-based constrained policy optimization with dual variable updates to enforce safety constraints while maximizing cost efficiency. Experimental results demonstrate that the proposed CMDP-CARL framework achieves 89.4% accuracy in SLA constraint satisfaction while reducing operational costs by 34.7% compared to static threshold-based baselines and maintaining zero safety constraint violations across 10,000 simulated episodes. The framework's provable safety guarantees, derived from constrained policy optimization theory, establish formal bounds on constraint violation probability. These findings contribute a replicable methodology for deploying safe reinforcement learning in mission-critical industrial IoT orchestration, with immediate implications for cloud administrators managing latency-sensitive manufacturing and energy sector workloads.

• pdf

Copyright (c) 2026 Billy Elly (Author)

This work is licensed under a Creative Commons Attribution 4.0 International License.