Xcode stuck at `Compile Swift source files` stage

I'm trying to adopt the Xcode new build system in our project, which contain a development pod. After performing a clean build, I modify one of the swift file in the pod (just adding a empty line), and perform a Incremental build. In the Xcode report navigator, I check the latest building progress, and found out something is weird:

  1. Xcode stuck at Compile Swift source files for a few seconds without compiling any swift file
  2. Xcode compile the changed swift file, merge swiftmodule, and stuck for addition nearly 60 seconds
  3. Xcode continue the building process

Any idea why Xcode stuck for such a long time?