AITurbo: Unified Compute Allocation for Partial Predictable Training in Commodity Clusters

As the scale and complexity of deep learning models continues to grow, model training is becoming an expensive job and only a small number of well-financed organizations can afford. Are the resources in commodity clusters well utilized for training? or how much potential space are still there for further improving the training efficiency in commodity clusters? is an urgent question to answer.