To prevent respiratory infections between patients and medical workers, the transmission risk of airborne pollutants in hospital wards must be mitigated. The ventilation modes, which are regarded as an important strategy to minimize the infection risk, is challenging to systematically design. Studies have considered the effect of ventilation openings (inlets/outlets) or infected source locations on the airflow distribution, pollutant removal, and infection risk mitigation. However, the relationship (such as relative distance) between ventilation openings and infected sources is critical because it affects the direct exhaust of exhaled pollutants, which has not been thoroughly studied. To explore pollutant removal and infection prevention in wards, different ventilation modes (with varying ventilation openings) and infected patient locations must be jointly considered. This study investigated displacement ventilation (DV), downward ventilation (DWV), and stratum ventilation (SV) with 4, 6, and 10 scenarios of ventilation openings, respectively. The optimal ventilation mode and relative distance between outlets and infected patients were analyzed based on the simulated pollutant concentration fields and the evaluated infection risk. The pollutant removal effect and infection risk mitigation of SV in the ward were largely improved by 67% and 59% compared with DV and DWV, respectively. The average infection risk was reduced below 7% when a non-dimensional relative distance (a ratio of the actual distance to the cubic root of the ward volume) was less than 0.25 between outlets and infected patient. This study can serve as a guide for the systematic ventilation system design in hospitals during the epidemic.