Correspondence between swift-clang and Apple's Clang end user releases

Hello,

I would like to know when a feature
(https://github.com/apple/swift-clang/commit/14bfc9e99e6e6903b09480a22c153032be77ae4e\)
was introduced in a binary version of Clang shipped by Apple.

I tried the latest source code version from
https://opensource.apple.com, which is clang-800.0.42.1, but the feature
is not there.

Then I found a public repository, swift-clang. However, even though I
see that the commit is on 4.0 branch, I have no idea how to tell the
precise version when the commit was cherry-picked onto the release which
gets shipped to the end user (which I assume doesn't happen in this repo).

I am also feel a bit frustrated due to the lack of correspondence with
usual Clang, which is currently at 4.0.1 and has browsable tags
corresponding to releases at https://llvm.org/svn/llvm-project/cfe/tags/\.

Regards,
Oleksii Vilchanskyi

Hello,

I would like to know when a feature
(https://github.com/apple/swift-clang/commit/14bfc9e99e6e6903b09480a22c153032be77ae4e\)
was introduced in a binary version of Clang shipped by Apple.

I tried the latest source code version from
https://opensource.apple.com, which is clang-800.0.42.1, but the feature
is not there.

I believe that what gets uploaded to that site is the clang that is used to build the latest
macOS release. So it doesn't get uploads until macOS is actually released, and the upload
is of a slightly different clang anyway from the Xcode release — typically it's a earlier tag of
the same release branch.

Then I found a public repository, swift-clang. However, even though I
see that the commit is on 4.0 branch, I have no idea how to tell the
precise version when the commit was cherry-picked onto the release which
gets shipped to the end user (which I assume doesn't happen in this repo).

Apple believes strongly in the LLVM/Clang/Swift open-source process. While the
ultimate, canonical Xcode release branches (i.e. clang-NNN) are indeed held in
Apple-internal repositories, those branches are continually and automatically merged
from the open-source release branches in the github repositories (i.e. swift-X.Y-branch).
When we want to ensure that an open-source LLVM or Clang commit ends up in an Xcode
release, we don't cherry-pick it directly into our internal release branches, we cherry-pick it
into the github repositories and let it get auto-merged. There are just so many advantages
to doing it this way that, frankly, we would do it even if we were actively hostile to open-source. :)
So the github release branches should always be a reliable point of truth for whether an
LLVM/clang commit will be present in the release.

The clang binary in a toolchain is built from the same source as the clang components of the
swiftc binary in that toolchain.

I am also feel a bit frustrated due to the lack of correspondence with
usual Clang, which is currently at 4.0.1 and has browsable tags
corresponding to releases at https://llvm.org/svn/llvm-project/cfe/tags/\.

The swift-4.0-branch is actually ultimately based on the LLVM's project 4.0 branch, although
the numbering is just a coincidence. This is something we've done for (I think) the last two
releases and plan to continue doing going forward. Since the Swift branch is usually targeted for
a significantly later release and does require a substantial amount of specific Clang support, we
do cherry-pick a fair amount into our release that LLVM does not cherry-pick into their release.

John.

···

On Aug 7, 2017, at 3:47 PM, Oleksii Vilchanskyi via swift-dev <swift-dev@swift.org> wrote:

Hello,

I would like to know when a feature
(https://github.com/apple/swift-clang/commit/14bfc9e99e6e6903b09480a22c153032be77ae4e\)
was introduced in a binary version of Clang shipped by Apple.

I tried the latest source code version from
https://opensource.apple.com, which is clang-800.0.42.1, but the feature
is not there.

I believe that what gets uploaded to that site is the clang that is used to build the latest
macOS release. So it doesn't get uploads until macOS is actually released, and the upload
is of a slightly different clang anyway from the Xcode release — typically it's a earlier tag of
the same release branch.

Minor correction: it used to be that way a while back, but more recently, the compiler sources on opensource.apple.com have matched the Xcode releases. For some reason, that page has lagged behind and the latest release posted is from Xcode 8.2.1.

···

On Aug 7, 2017, at 1:55 PM, John McCall via swift-dev <swift-dev@swift.org> wrote:

On Aug 7, 2017, at 3:47 PM, Oleksii Vilchanskyi via swift-dev <swift-dev@swift.org> wrote:

Then I found a public repository, swift-clang. However, even though I
see that the commit is on 4.0 branch, I have no idea how to tell the
precise version when the commit was cherry-picked onto the release which
gets shipped to the end user (which I assume doesn't happen in this repo).

Apple believes strongly in the LLVM/Clang/Swift open-source process. While the
ultimate, canonical Xcode release branches (i.e. clang-NNN) are indeed held in
Apple-internal repositories, those branches are continually and automatically merged
from the open-source release branches in the github repositories (i.e. swift-X.Y-branch).
When we want to ensure that an open-source LLVM or Clang commit ends up in an Xcode
release, we don't cherry-pick it directly into our internal release branches, we cherry-pick it
into the github repositories and let it get auto-merged. There are just so many advantages
to doing it this way that, frankly, we would do it even if we were actively hostile to open-source. :)
So the github release branches should always be a reliable point of truth for whether an
LLVM/clang commit will be present in the release.

The clang binary in a toolchain is built from the same source as the clang components of the
swiftc binary in that toolchain.

I am also feel a bit frustrated due to the lack of correspondence with
usual Clang, which is currently at 4.0.1 and has browsable tags
corresponding to releases at https://llvm.org/svn/llvm-project/cfe/tags/\.

The swift-4.0-branch is actually ultimately based on the LLVM's project 4.0 branch, although
the numbering is just a coincidence. This is something we've done for (I think) the last two
releases and plan to continue doing going forward. Since the Swift branch is usually targeted for
a significantly later release and does require a substantial amount of specific Clang support, we
do cherry-pick a fair amount into our release that LLVM does not cherry-pick into their release.

John.
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

The clang binary in a toolchain is built from the same source as the clang components of the
swiftc binary in that toolchain.

Just wanted to verify that this is still true. Does the clang binary shipped as part of XCode use the same driver? The reason I'm asking is that it appears the behavior of the driver w.r.t. the handling of -fmodule parameter differs. In particular the defaults used in RenderModulesOptions [lib/Driver/ToolChains/Clang.cpp] for OPT_fcxx_modules/OPT_fno_cxx_modules appears to be swapped. The default in XCode's Clang appears to not allow modules in CXX files unless the OPT_fcxx_modules is set. The default in swift-clang is to allow modules unless the OPT_fno_cxx_modules is set.

If there is indeed a different driver for the toolchain's clang is the source code available?

Thanks.

Apple disables modules in C++ in our internal branch out of caution. That branch is not open source. We do try to minimize the differences, but for a variety of reasons, open source compilers do not and will never perfectly replicate the behavior of Apple's officially-released compilers.

Exactly what I needed to know. Thanks for the fast response John!