I eventually managed to do it.
-
Drag the package folder which contains the Package.swift into your Xcode project
-
Click the Plus button in the "Link Binary with Libraries" section, locate the package in the modal dialog, select the gray library icon inside the package, and add this one.
-
Build your Xcode project and move on to your next interesting task
Trouble is that Xcode lets you drag the brown package icon to the "Link Binary with Libraries" section of the "Build Phases" tab of your Xcode target. Don't do that: this has no (useful) effect.