Can't profile Metal shaders within a package

AFAIK, I can't pass compiler options to the Metal compiler from Package.swift, so I can't enable debug symbols for metal and use the shader debugger/profiler on shaders within a Swift Package. So currently, whenever I want to debug/profile my shaders, I move all the relevant code out of the package.

How could this be improved? Xcode seems to pass debug flags to SPM for Swift or ObjC files. I suppose the same could be done for Metal?

thanks!

5 Likes

I have the same problem in Xcode 12. I gave up and use .frameworks instead. In Xcode 13 there are new features for shader compiling and debugging. Now the compiler can generate a .metallibsym file (see here at about 16:45). I have not tried to get this work with Swift Package Manager yet. Has anyone else tried?

1 Like

We encountered this recently as well and I asked a Swift Package Manager engineer about it during WWDC last week. They confirmed it is not supported.

Our work around is to create a runscript that runs just after the file compilation phase of our project. We do this only when we detect a package that contains metal files has been dragged into our workspace as a local package, thus replacing our production, remote package.

The script finds the local reference path in our XCWorkspace file, finds the metal files in the package at the referenced path and then compiles them like this (I left out the specifics of finding the metal files... this is just the compile and link step for metal):

compile_air() {
        export MTL_ENABLE_DEBUG_INFO=INCLUDE_SOURCE 
        $(xcrun --sdk iphoneos -f metal) -c -target air64-apple-ios13.0 -gline-tables-only -MO \
        -I${OUTPUT_DIR}/include \
        -F${OUTPUT_DIR} \
        -isysroot $(xcrun --show-sdk-path --sdk iphoneos) \
        -ffast-math \
        -o $2 \
        $1
}

//do the compile_air for each metal file
compile_air $RENDERER_METAL $RENDERER_AIR

echo "#### LINKING AIR"
$(xcrun --sdk iphoneos -f metal) \
-target air64-apple-ios13.0 \
-MO \
-o ${OUTPUT_DIR}/MyPackage.bundle/default.metallib \
$RENDERER_AIR //can add more air files here if you have them

This overwrites the .metallib generated by SwiftPM with one that can be debugged.

Edit: Also note, the OUTPUT_DIR is passed to the script and is just the Xcode BUILT_PRODUCTS_DIR

3 Likes

That looks like a great work around. I'm able to compile the .air files, but getting this when linking:

air-lld: error: library not found for -lmetal_rt_ios_air2.3

EDIT: got it, had to set SDKROOT

Ah! Yes that makes sense. I did that too, but neglected to paste it in.

Just need export SDKROOT=$(xcrun --show-sdk-path --sdk iphoneos) before the compiler_air step

2 Likes

cc @abertelrud

Unfortunately, I can't get it to work. Maybe you or somebody can point me in the right direction.
This is the script in my build phase:

#!/bin/zsh
compile_air() {
        export MTL_ENABLE_DEBUG_INFO=INCLUDE_SOURCE 
        $(xcrun --sdk iphoneos -f metal) -c -target air64-apple-ios13.0 -gline-tables-only -MO \
        -I${OUTPUT_DIR}/include \
        -F${OUTPUT_DIR} \
        -isysroot $(xcrun --show-sdk-path --sdk iphoneos) \
        -ffast-math \
        -o $2 \
        $1
}
AIRFILES=()
for file in $(find "$PROJECT_DIR" -iname '*.metal')
do
    filename=$file:t:r
    filepath="$BUILT_PRODUCTS_DIR/$filename.air"
    compile_air $file $filepath
    AIRFILES+=$filepath
done

echo "#### LINKING AIR"
$(xcrun --sdk iphoneos -f metal) \
-target air64-apple-ios13.0 \
-MO \
-o ${BUILT_PRODUCTS_DIR}/SPMPackage_SPMPackage.bundle/default.metallib \
${AIRFILES}

The phase runs just fine but when trying to profile the metal shader it can't find the metal source file.

Did you set the SDKROOT as I suggested in my follow up message?

Yes, I tried that as well. When trying to profile the metal shader I get this error:

Task Failed: Compiling source "Shader.metal" for target "1".

metal: error: no such file or directory: '/private/var/folders/7k/y3sqtzxd0ldby388_vfhcffm0000gn/T/1632836607.447792/dfae43e651e5ab2ee08bf79c7c061e29f25a38483f0b185da3757a5486f15180/source/1/Applications/Xcode_13_RC.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.0.sdk
/private/var/folders/7k/y3sqtzxd0ldby388_vfhcffm0000gn/T/1632836607.447792/dfae43e651e5ab2ee08bf79c7c061e29f25a38483f0b185da3757a5486f15180/source/1/Users/user/project/Shaders/Shader.metal'
metal: error: no input files

It seems like the path to the metal files is prefixed with the BUILT_PRODUCTS_DIR path. Where do you compile the air files to?

Thanks for the help!

i got it to work by removing this line: -isysroot $(xcrun --show-sdk-path --sdk iphoneos) \

1 Like

How do you detect that the local package has been added to the project and find the metal files?

I pass the path to my apps workspace file to the script and use something like this to determine if my package is local or not:

WS_CONTENTS=$WORKSPACE_DIR/contents.xcworkspacedata
HAS_LOCAL_PACKAGE=$(grep "mypackage" $WS_CONTENTS | awk '{print $3}' | awk -F ":" '{ print $2 }' | awk -F '"' '{print $1}')

For the record, I have not been able to figure out how to recompile the metal files since Xcode 13 came out. Currently, our devs are just dragging their package repos into the top level of the workspace, then dragging the .metal files to our main project as a reference (not copying the actual file). This allows Xcode to pick it up properly. They have to make sure not to commit those changes though, so not ideal, but it is working.

2 Likes

Any better solutions for this in 2023? Perhaps some improvements were introduced in Xcode 15 beta?

1 Like

Any better solutions for this in 2024 :upside_down_face:? Perhaps some improvements were introduced in Xcode 16?

1 Like

There's this swift package plugin happens, which is quite useful to get things work (i.e. to compile debug.metallib library with debug symbols included). MetalCompilerPlugin - Swift Package Registry (works only with Xcode/xcodebuild)

Disclaimer: I use it with a slightly different approach (to enable shaders logging on a pure metal target) and thus I was unable to make it work to set breakpoint on a shader source.