Somehow I think this is something coming from the OS and not from the environment. This problem wasn't there initially. I think (cannot say for certain) that it started with the last MacOS update. (I'm now on 10.14.5)
I checked my environment variables and don't see anything suspect.
My guess is that it has to do with the support of system libraries in the generated xcodeproj file. The package SwiftGD has a dependency on an externally defined library:
.systemLibrary(name: "gd", pkgConfig: "gdlib", providers: [.apt(["libgd-dev"]), .brew(["gd"])]),
The compiler errors I get are symbols that should come from the gd.h file coming from that package. Interestingly enough there is no error/warning for the "import gd" statement, it just complains about unknown types.