__counted_by__ regression issue on Swift 6.1 for Linux

Using Swift
,
1

Try to bumping a C library from Swift 6.0 toolchain to Swift 6.1.

And it just stop compiling for some message related with __counted_by__. (Can be build with Swift 6.0 toolchain)

I have tried to explicitly set a cLanguageStandard for different values. And it does not seem to work.

In file included from <redacted>/OpenGraph/Sources/OpenGraph_SPI/Attribute/AttributeID.cpp:8:
In file included from <redacted>/OpenGraph/Sources/OpenGraph_SPI/Attribute/AttributeID.hpp:12:
In file included from <redacted>/OpenGraph/Sources/OpenGraph_SPI/include/OGAttribute.h:12:
<redacted>/OpenGraph/Sources/OpenGraph_SPI/include/OGGraph.h:108:70: error: use of undeclared identifier 'count'; did you mean 'round'?
  108 | bool OGGraphAnyInputsChanged(const OGAttribute *inputs OG_COUNTED_BY(count), size_t count);
      |                                                                      ^~~~~
      |                                                                      round
<redacted>/OpenGraph/Sources/OpenGraph_SPI/include/OGBase.h:54:39: note: expanded from macro 'OG_COUNTED_BY'
   54 | #define OG_COUNTED_BY(N) __counted_by(N)
      |                                       ^
<redacted>/OpenGraph/Sources/OpenGraph_SPI/include/OGBase.h:48:55: note: expanded from macro '__counted_by'
   48 | #define __counted_by(N) __attribute__((__counted_by__(N)))
      |                                                       ^
/usr/lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/math.h:107:12: note: 'round' declared here
  107 | using std::round;
      |            ^
In file included from <redacted>/OpenGraph/Sources/OpenGraph_SPI/Attribute/AttributeID.cpp:8:
In file included from <redacted>/OpenGraph/Sources/OpenGraph_SPI/Attribute/AttributeID.hpp:12:
In file included from <redacted>/OpenGraph/Sources/OpenGraph_SPI/include/OGAttribute.h:12:
<redacted>/OpenGraph/Sources/OpenGraph_SPI/include/OGGraph.h:108:56: warning: '__counted_by__' attribute ignored [-Wignored-attributes]
  108 | bool OGGraphAnyInputsChanged(const OGAttribute *inputs OG_COUNTED_BY(count), size_t count);
      |                                                        ^
<redacted>/OpenGraph/Sources/OpenGraph_SPI/include/OGBase.h:54:26: note: expanded from macro 'OG_COUNTED_BY'
   54 | #define OG_COUNTED_BY(N) __counted_by(N)
      |                          ^
<redacted>/OpenGraph/Sources/OpenGraph_SPI/include/OGBase.h:48:40: note: expanded from macro '__counted_by'
   48 | #define __counted_by(N) __attribute__((__counted_by__(N)))
      |                                        ^
1 warning and 1 error generated.
error: fatalError

The code related with it is as the following

#if !defined(__counted_by)
#if __has_attribute(__counted_by__)
#define __counted_by(N) __attribute__((__counted_by__(N)))
#else
#define __counted_by(N)
#endif
#endif

#define OG_COUNTED_BY(N) __counted_by(N)

bool OGGraphAnyInputsChanged(const OGAttribute *inputs OG_COUNTED_BY(count), size_t count);

See detail OG_COUNTED_BY issue on Linux with Swift 6.1 compiler · Issue #130 · OpenSwiftUIProject/OpenGraph · GitHub

2

A minimal package to reproduce the issue (Success with 6.0 compiler and fails with 6.1 compeller via swift build on Ubuntu 22.04)

.
├── Package.swift
└── Sources
    └── DemoKit
        ├── demo.c
        └── include
            └── demo.h

Package.swift

// swift-tools-version: 6.0

import PackageDescription

let package = Package(
    name: "DemoKit",
    products: [
        .library(name: "DemoKit",targets: ["DemoKit"]),
    ],
    targets: [
        .target(name: "DemoKit"),
    ]
)

demo.c

#include "demo.h"

void hello(const void *inputs OG_COUNTED_BY(count), size_t count) {

}

demo.h

#ifndef demo_h
#define demo_h

#include <stdio.h>

#if !defined(__counted_by)
#if __has_attribute(__counted_by__)
#define __counted_by(N) __attribute__((__counted_by__(N)))
#else
#define __counted_by(N)
#endif
#endif

#define OG_COUNTED_BY(N) __counted_by(N)

void hello(const void *inputs OG_COUNTED_BY(count), size_t count);

#endif /* demo_h */
1 Like
3

@Kyle-Ye Thanks for reporting this.

The __counted_by attribute supports being late parsed so that parameters that are declared later can be referred to by __counted_by. However, for various reasons this is not clang's default behavior. It is technically possible to manually enable it with -fexperimental-late-parse-attributes but you SHOULD NOT DO THIS. There is a much more fundamental problem here which I'll get into now.

Normally, adding the __counted_by attribute to C++ files is not supposed to have any affect on compilation by default because when the attribute used from the ptrcheck.h header it is a macro that expands to nothing unless the code is built with -fexperimental-bounds-safety-attributes (not the default behavior).

The guards in the ptrcheck.h header effectively look like this:

#if defined(__has_feature) && (__has_feature(bounds_attributes)) || __has_feature(bounds_safety_attributes))
  #define __counted_by(N) __attribute__((__counted_by__(N)))
#else
  // define to nothing
  #define __counted_by(N) 
#endif

Why have you bypassed using the ptrcheck.h header? If you are going to bypass the header you need to mimic what ptrcheck.h does, otherwise you can run into problems such as the one you've hit. I highly recommend not bypassing the header though because it contains internal compiler implementation details that might change overtime.

There are a few other things worth noting.

  • Releases prior to Swift 6.2 have known bugs related to parsing the attribute in C++ code. Again this isn't supposed to be a problem because if you use __counted_by from the ptrcheck.h header the __counted_by macro expands to nothing unless you build with -fbounds-safety (for C) or -fexperimental-bounds-safety-attributes (for C++), neither of which are the default and thus shouldn't affect your build.
  • Your minimal reproducer technically isn't valid because __counted_by is not allowed on void pointers because __counted_by needs to know the size of the pointee. Replacing void with char is enough to reproduce the problem though. Note on macOS the stdio.h include has to be removed to reproduce because that implicitly includes ptrcheck.h so __counted_by expands to nothing (as it's supposed to).

I hope one day we can get to the point where __attribute__((__counted_by__(N)) can be used directly from C or C++ in Clang and "the right thing" happens but that is not the current state of things. Until that happens you should use the ptrcheck.h header to avoid breaking your builds.

Does that make sense?

2 Likes
4

Thanks for your detail explanation on this.

First, let's forgot about C++ issue and only focus on C.

On Darwin platform, my #define will conflict with SDK's ptrcheck.h after bumping into Xcode 16.3 / Swift 6.1 toolchain. And I can easily workaround it by something like this. :down_arrow:

#if __has_include(<ptrcheck.h>) // Fix conflict define issue of the SDK
#include <ptrcheck.h>
#define OG_COUNTED_BY(N) __counted_by(N)
#else
...
#endif

On Linux platform, I can build the code with Swift 6.0 compiler is due to __has_attribute(__counted_by__) will be evaluated as false.

But with Swift 6.1, it will be true so I hit the order issue.

On Linux Swift toolchain's clang, I checked the full build argument via swift build --verbose.

The argument does not contain "-fexperimental-late-parse-attributes" so my code will fail to build.

/home/kyle/.local/share/swiftly/toolchains/6.1.2/usr/bin/clang -target aarch64-unknown-linux-gnu -O0 -DSWIFT_PACKAGE=1 -DDEBUG=1 -fblocks -index-store-path /Users/kyle/Workspace/OP/DemoKit/.build/aarch64-unknown-linux-gnu/debug/index/store -I /Users/kyle/Workspace/OP/DemoKit/Sources/DemoKit/include -fPIC -g -fno-omit-frame-pointer -MD -MT dependencies -MF /Users/kyle/Workspace/OP/DemoKit/.build/aarch64-unknown-linux-gnu/debug/DemoKit.build/demo.c.d -c /Users/kyle/Workspace/OP/DemoKit/Sources/DemoKit/demo.c -o /Users/kyle/Workspace/OP/DemoKit/.build/aarch64-unknown-linux-gnu/debug/DemoKit.build/demo.c.o

On macOS platform. I also checked the full build argument via swift build --verbose.

clang -fobjc-arc -target arm64-apple-macosx10.13 -O0 -DSWIFT_PACKAGE=1 -DDEBUG=1 -fblocks -index-store-path /Users/kyle/tmp/DemoKit/.build/arm64-apple-macosx/debug/index/store -fmodules -fmodule-name=DemoKit -fmodules-cache-path=/Users/kyle/tmp/DemoKit/.build/arm64-apple-macosx/debug/ModuleCache -I /Users/kyle/tmp/DemoKit/Sources/DemoKit/include -isysroot /Applications/Xcode-16.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.4.sdk -F /Applications/Xcode-16.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -F /Applications/Xcode-16.3.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/PrivateFrameworks -fPIC -g -MD -MT dependencies -MF /Users/kyle/tmp/DemoKit/.build/arm64-apple-macosx/debug/DemoKit.build/demo.c.d -c /Users/kyle/tmp/DemoKit/Sources/DemoKit/demo.c -o /Users/kyle/tmp/DemoKit/.build/arm64-apple-macosx/debug/DemoKit.build/demo.c.o

It does not contain "-fexperimental-late-parse-attributes" either. But it can build successfully. Also both of the clang version is 17.0

It is technically possible to manually enable it with -fexperimental-late-parse-attributes but you SHOULD NOT DO THIS . There is a much more fundamental problem here which I'll get into now.

So my guess is the clang shipped with Xcode's Swift toolchain has enabled -fexperimental-late-parse-attributes by default so there is no build issue related with order on Darwin platform.

Other questions:

  • Is ptrcheck.h available as part of the Apple platform SDK or does it also available on Linux platform as part of Swift toolchain?
  • Is there any hope we get an aligned behavior of -fexperimental-late-parse-attributes or a list where Clang shipped with Xcode(Swift toolchain on Apple platform) has difference feature set on/off?
5

Where is the conflict? This on it's own doesn't have a conflict.

What you should be doing is this.

#if __has_include(<ptrcheck.h>)
#include <ptrcheck.h>
#define OG_COUNTED_BY(N) __counted_by(N)
#else
// define to nothing because the feature is not available
#define OG_COUNTED_BY(N)
#endif

You should not try to define __counted_by(N) yourself. You should use the ptrcheck.h header which provides it. The header is not part of the SDK headers. It's part of the toolchain shipped with Clang.

6

That's a good guess but I don't think that's what's happening. On Apple platforms the Libc headers will automatically trigger an include of ptrcheck.h. In your example I think stdio.h will trigger this so __counted_by gets defined. See the sys/cdefs.h header inside the SDK. I believe that's the one that includes ptrcheck.h.

If we consider your example

#include <stdio.h>
// after including this `__counted_by` is already defined on Apple platforms
// after including this `__counted_by` is not defined on Linux with glibc (and probably any other C library)

#if !defined(__counted_by)
#if __has_attribute(__counted_by__)
#define __counted_by(N) __attribute__((__counted_by__(N)))
#else
#define __counted_by(N)
#endif
#endif

The code guarded by #if !defined(__counted_by) is never parsed because !defined(__counted_by) is false. So in this case __counted_by is defined to be empty and thus doesn't do anything (unless building with -fbounds-safety or -fexperimental-bounds-safety-attributes)

For Linux however that uses a different C library which doesn't implicitly include the ptrcheck.h header. So for Linux the code guarded under #if !defined(__counted_by) is parsed because !defined(__counted_by) is true. So in this case the source code then does.

#define __counted_by(N) __attribute__((__counted_by__(N)))

This defines __counted_by and is the wrong thing to do because this is not what ptrcheck.h does. The ptrcheck.h header only defines __counted_by when building with -fbounds-safety or -fexperimental-bounds-safety-attributes, otherwise the macro is defined to be empty. Your code is not matching that behavior which is why you are running into problems.

1 Like
7

The header is included as part of the Swift toolchain. If the header is missing entirely (which it will be for some older releases) then feature isn't there and you shouldn't try to use it.

There isn't a difference between -fexperimental-late-parse-attributes between toolchains. What you're observing is a difference between __counted_by being defined and not defined.

1 Like
8

Got it. Now I have more understanding of the whole picture and how this feature is working.

This defines __counted_by and is the wrong thing to do because this is not what ptrcheck.h does. The ptrcheck.h header only defines __counted_by when building with -fbounds-safety or -fexperimental-bounds-safety-attributes , otherwise the macro is defined to be empty. Your code is not matching that behavior which is why you are running into problems.

So I guess the suggested way to properly use it is

  1. include <ptrcheck.h> (Most of time optional on Darwin platform since it was already included)
  2. Use __counted_by provided by ptrcheck header instead of dealing with (__counted_by__) or __has_attribute ourselves.
9

Yep. Also if you're building code that needs to built by other toolchains you can do something like this so that your code can build with the Swift toolchain and other toolchains that don't have the ptrcheck.h header file.

#if __has_include(<ptrcheck.h>)
#include <ptrcheck.h>
#ifndef __counted_by
  #error __counted_by is not defined but should be
#endif
#else
// Feature not available so define attributes to be empty to avoid breaking the build
#define __counted_by(N)
#define __sized_by(N)
#define __counted_by_or_null(N)
#define __sized_by_or_null(N)
// TODO: Add others here...
#endif

As I said. Hopefully one day you'll be able to use __attribute__((__counted_by__(N))) directly but Clang currently doesn't properly support that.