Ambulance offload delays occur when emergency medical service (EMS) personnel are unable to promptly transfer patients to overwhelmed emergency departments. These delays postpone necessary care for the patient and hinder the EMS system from attending to new emergencies. This study introduces a real-time multi-priority patient transfer policy aimed at reducing these delays. We model the patient transfer problem as a stochastic dynamic program based on post-decision states, and develop approximate dynamic programming approaches to overcome the curse of dimensionality. Additionally, we derive a lower bound for the optimal solution through information relaxation. We evaluate the proposed method using both synthetic and real EMS data from St. Paul, Minnesota. Our solution is close to the lower bound, and the resulting policy results in a triple-win situation. For patients, our policy reduces hospital wait times by 80%; for hospitals, our policy alleviates emergency department congestion, and we find that even partial compatibility leads to notable system-wide improvements; for the EMS system, ambulance utilization is reduced by more than 30%, and ambulances could handle more calls per day, with a reduction in lost calls exceeding 80%.