Performance Test Baselines not available for Swift Packages

Using Swift
1

So I read online that performance test baselines are stored in xcshareddata/xcbaselines for xcodeproj projects, but since there is no equivalent for Swift Package Testing targets, I think there may be a bug where baselines are not being cached. When I try to save baseline metrics even though the IDE looks to have updated it (tapping the grey button in the line number column shows a value for baseline) , I get a diagnostic on the next runs like No baseline average for Clock Monotonic Time. The same applies to the default measure block metric as well.

Just wanted to confirm if this is a bug or I am missing something. Thanks for the help.

EDIT: I found some baselines stored in the .swiftpm folder, and these seem correct (they have values for a lcoal baseline). So not sure now why do I get a diagnostic in this case?

UPDATE: Going to radar this one, got nothing to lose I suppose. :joy:

2 Likes
2

I also started playing with this and get the same problem with Xcode 13.4 as you saw - it's a bit of a bummer if SPM can't be used for performance testing (having it on Linux is another topic altogether, but just having it working from both Xcode and the command line and proper reflection in UI would be great).

Changing a performance tests so it runs twice slower than the recorded baseline will still not break my test, so even if baseline is stored, it isn't used properly.

Did you get any radar feedback at all?

1 Like
3

No radar feedback unfortunately :joy:

2 Likes