Shipping emissions potentially contribute to the degradation of air quality in port cities. In this study, PM2.5 samples were collected from two sites at different distances from the shipping channel of Xiamen Port in southeastern China between November 2015 and May 2018 and analyzed for their chemical compositions, which included water-soluble ions, carbonaceous species, and elements. The average annual PM2.5 mass concentrations were 55.8 ± 22.7 µg m–3 and 56.5 ± 24.5 µg m–3 at the urban and suburban sites, respectively, with the lowest values in summer and the highest in winter/spring. Significantly higher values for vanadium (V) and nickel (Ni) were found at the urban site due to the shorter distance between this location and the shipping channel. Using a PMF model, six source factors were identified: sulfate and shipping emissions (16.6–20.9%), secondary nitrate and chloride (14.7–17.3%), fugitive dust (16.9–23.0%), industrial emissions (5.5–7.0%), primary organic aerosol (14.1–14.8%), and traffic emissions and biomass burning (23.8–24.6%). Potential source contribution function analysis indicated that air masses from the South China Sea contributed significantly to the shipping emissions. The PMF-based method did not distinguish primary shipping emissions from secondary sulfate. When a V-based method was used, the primary PM2.5 from shipping emissions plus its associated secondary sulfate was shown to contribute 5.5–8.9% of the ambient PM2.5 on average. The results from the V-based method exhibited strong positive correlations with those of the PMF-based method. Considering the potential negative effect on air quality and the expanding international maritime trade in the long term, our research indicates that policies and regulations for controlling shipping emissions are necessary in all major port cities, including those outside the domestic emission control areas of China.