AI-Driven Cloud Optimization: Enhancing Cost Prediction, Resource Scheduling and Fault Resilience in Cloud Environments

Cloud computing has the benefits of scalability and flexibility, yet poses long-term problems of cost estimation, efficient scheduling of resources, and fault tolerance. In this paper, an AI-driven framework is proposed that can reconcile these drawbacks by combining cost prediction, dynamic task scheduling, and fault detection into a user-friendly visualization dashboard. Cost prediction makes use of supervised machine learning algorithms such as Linear Regression, Random Forest, and XGBoost to predict the costs of a task based on synthetic workloads created with iFogSim. The prediction accuracy is also improved after hyperparameter optimization using Optuna. Task scheduling employs Deep Reinforcement Learning (DRL) with a Deep Q-Network (DQN) structure that maximizes job placement on heterogeneous virtual machines (VMs) and has benchmark comparisons with First-Come-First-Serve (FCFS) and Round-Robin schedules. The scheduling logic is trained and tested on the Kaggle Cloud Task Scheduling dataset. The fault detection mechanism uses the Isolation Forest algorithm to detect anomalous system behavior such as CPU usage behavior or long execution time. Evaluation metrics, reward curves, anomaly plots, and interpretability graphs, are displayed as part of a Streamlit-based dashboard on Render. The framework is a modular automation constructed to stage each aspect on demand, making it flexible, reproducible, and resilient in deployment. Experimental results show that such a technique makes cost estimation more accurate, minimizes delays in scheduling, and increases fault tolerance. This makes the proposed framework holistic and practical, since predictive analytics is combined with reinforcement learning along with anomaly detection, to optimise operations in multi-cloud environments. The outcome of this research can be of interest for real-life cloud management applications.