Pitch: Importing Forward Declared Objective-C Classes and Protocols

Evolution Pitches
, ,
1

Hi everyone, we would like to propose some improvements to Swift - Objective-C interop, specifically with respect to Objective-C forward declarations. A draft PR of the proposed change can be found here: Import objc forward declarations by NuriAmari · Pull Request #61606 · apple/swift · GitHub

Introduction:

This proposal seeks to improve the usability of existing Objective-C libraries from Swift by reducing the negative impact forward declarations have on API visibility from Swift. We wish to start synthesizing placeholder types to represent forward declared Objective-C interfaces and protocols in Swift.

Motivation:

Forward declarations are very common in many existing Objective-C code bases, used often to break cyclic dependencies or to improve build performance. Unfortunately, when it comes to "importability" into Swift they are quite detrimental.

As it stands, the ClangImporter will fail to import any declaration that references a forward declared type in many common cases. This means a single forward declared type can render larger portions of an Objective-C API unusable from Swift. For example, the following Objective-C API from the proposal PR is empty from the Swift perspective (Swift textual interface generated via swift-ide-test):

Objective-C

#import <Foundation/Foundation.h>

@class ForwardDeclaredInterface;
@protocol ForwardDeclaredProtocol;

@interface IncompleteTypeConsumer1 : NSObject
@property id<ForwardDeclaredProtocol> propertyUsingAForwardDeclaredProtocol1;
@property ForwardDeclaredInterface *propertyUsingAForwardDeclaredInterface1;
- (id)init;
- (NSObject<ForwardDeclaredProtocol> *)methodReturningForwardDeclaredProtocol1;
- (ForwardDeclaredInterface *)methodReturningForwardDeclaredInterface1;
- (void)methodTakingAForwardDeclaredProtocol1:
    (id<ForwardDeclaredProtocol>)param;
- (void)methodTakingAForwardDeclaredInterface1:
            (ForwardDeclaredInterface *)param;
@end

ForwardDeclaredInterface *CFunctionReturningAForwardDeclaredInterface1();
void CFunctionTakingAForwardDeclaredInterface1(
    ForwardDeclaredInterface *param);

NSObject<ForwardDeclaredProtocol> *CFunctionReturningAForwardDeclaredProtocol1();
void CFunctionTakingAForwardDeclaredProtocol1(
    id<ForwardDeclaredProtocol> param);

Swift

class IncompleteTypeConsumer1 : NSObject {
  init!()
}

We wish to make the experience of consuming an Objective-C API with forward declarations from Swift more consistent with the experience of consuming the API from Objective-C.

Proposed Solution

As a first iteration, we propose the following representation for forward declared Objective-C interfaces and protocols in Swift:

// @class Foo turns into
@available(*, unavailable, message: “This Objective-C class has only been forward declared; import its owning module to use it”)
class Foo : NSObject {}

// @protocol Bar turns into
@available(*, unavailable, message: “This Objective-C protocol has only been forward declared; import its owning module to use it”)
protocol Bar : NSObjectProtocol {}

The idea is to introduce the minimal change that will make Objective-C APIs usable in a predictable safe manner.

You will be able to use Objective-C and C declarations that refer to these types without issue. You will be able to pass around instances of these incomplete types from Swift to Objective-C and vice versa. You’ll also be able to call the set of methods provided by NSObject or NSObjectProtocol on instances. The aforementioned Objective-C API with this change looks like this from Swift:

@available(*, unavailable, message: "This Objective-C class has only been forward-declared; import its owning module to use it")
class ForwardDeclaredInterface {
}
@available(*, unavailable, message: "This Objective-C protocol has only been forward-declared; import its owning module to use it")
protocol ForwardDeclaredProtocol : NSObjectProtocol {
}
class IncompleteTypeConsumer1 : NSObject {
  var propertyUsingAForwardDeclaredProtocol1: ForwardDeclaredProtocol!
  var propertyUsingAForwardDeclaredInterface1: ForwardDeclaredInterface!
  init!()
  func methodReturningForwardDeclaredProtocol1() -> ForwardDeclaredProtocol!
  func methodReturningForwardDeclaredInterface1() -> ForwardDeclaredInterface!
  func methodTakingAForwardDeclaredProtocol1(_ param: ForwardDeclaredProtocol!)
  func methodTakingAForwardDeclaredInterface1(_ param: ForwardDeclaredInterface!)
}
func CFunctionReturningAForwardDeclaredInterface1() -> ForwardDeclaredInterface!
func CFunctionTakingAForwardDeclaredInterface1(_ param: ForwardDeclaredInterface!)
func CFunctionReturningAForwardDeclaredProtocol1() -> ForwardDeclaredProtocol!
func CFunctionTakingAForwardDeclaredProtocol1(_ param: ForwardDeclaredProtocol!)

More usage examples can be found in these tests introduced here: Import objc forward declarations by NuriAmari · Pull Request #61606 · apple/swift · GitHub

However, that is the limit of the functionality. You will not be able to directly use the synthesized type itself in Swift in any manner. This is to keep the impact of the change small and to prevent unsound declarations, such as declaring a new Swift class that inherits from or conforms to such a type. You will also not be able to create new instances of these types in Swift.

If the complete definition for one of these types is imported, either in the Objective-C library or the consuming Swift environment, these changes will have no effect. That is, no new declaration will be synthesized at all.

Source Compatibility

This change may introduce name conflicts for users that have created accessible declarations with the same name as any newly imported Clang declarations. Thus we suggest keeping the change off by default initially.

Alternatives Considered

The unavailable attribute attached to synthesized declarations might be more restrictive than necessary. We considered the introduction of a new attribute to denote an incomplete type. This attribute would be used to prevent operations that are not suitable for an incomplete type (ie. declaring a Swift type that conforms to an incomplete protocol).

Allowing more use of the type however increases the complexity of the problem, and the likelihood of source compatibility issues.

Feedback

We are looking for input on if these might be reasonable stand-in types and any possible edge cases we might not be thinking of. In particular, input from any of @allevato, @xwu, @hborla, @beccadax and other members of the language workgroup would be great!

7 Likes
2

This certainly looks to be a straightforward improvement; if @available(*, unavailable) is too strict, then that can certainly be relaxed later.

So far as feedback is concerned, though, it'd be great to as many people in the community as possible who can weigh in to do so. I'm not a high-demand user of Obj-C APIs myself, so others will be much better suited to commenting on the nitty gritty details.

1 Like
3

This has been a point of contention in many of our internal builds. The recommended style for Obj-C was to forward declare when possible in header files and only import headers in implementation files, because this resulted in a lot less compiler/filesystem work during regular Obj-C compilations, but we had to start telling users to add #imports that were otherwise not "necessary" so that methods referencing those types wouldn't be dropped completely (and without diagnosis) by Swift.

So just to make sure I understand what you're proposing, if I have the following:

// Both only forward declare @class Foo in their headers
// and define interfaces with properties of type Foo
import FooUser1
import FooUser2

// This works because I'm treating it as an opaque reference
someTypeFromFooUser1.foo = someTypeFromFooUser2.foo

// ERROR: This doesn't work because we don't have the full definition
someTypeFromFooUser1.foo.someMethod()

But if I add the import for Foo, this works:

import FooUser1
import FooUser2
// This declares @interface Foo
import FooModule

// Then both work without conflict:
someTypeFromFooUser1.foo = someTypeFromFooUser2.foo
someTypeFromFooUser1.foo.someMethod()

That sounds ideal and a great way to balance our concerns!

I don't have enough experience with ClangImporter/type checking to know whether there would be issues from having this fake opaque type vs. the full type in different contexts, but if that doesn't cause any problems, I'm excited to see this come to fruition.

3 Likes
4

That's correct. You can also call methods provided by NSObject or NSObjectProtocol on the opaque reference. Once you import the declaring module, you will have access to the complete interface, in addition to the restricted functionality you already had with the forward declaration. The semantics are analogous to within Objective-C/C/C++, importing the full definition just augments the available functionality, nothing should regress.

We had similar concerns, but it in our testing at least, it didn't seem to be an issue. When Clang loads the imported Clang modules, it associates forward declarations with each other and a complete definition if one exists. This association seems to work even accross modules. Within a given swift::ASTContext the ClangImporter always operates on the same ClangNode, using the definition if it exists, and the "most recently enountered" forward declaration otherwise. This means there is only a single Swift representation, there is no mismatch between the "incomplete" and "complete" versions. Since the type itself is not available in Swift, I see no way for a given synthesized declaration (either complete or incomplete) to escape the ASTContext and conflict with another. In the attached PR, there are a few tests to make sure no conflicts occurs within the same file. That said, these are exactly the edge cases we would like to make sure we've got covered and welcome any more suggestions.

1 Like
5

The reason this wasn’t implemented like this to begin with is because it messes with REPLs and with LLDB: if you import a forward declaration and then later import the real @interface, how does the compiler reconcile that in the AST? (The way this comes up in LLDB is if one module in the program has the definition and another does not, and then LLDB tries to set up a context that accommodates both of them.)

It’s possible the unavailability means that nothing could depend on the initial shapes of those types, and so this would work out in practice, but I thought I’d bring it up.

EDIT: this may also be source-breaking, for the stupid reason that somebody may be using a method name or selector that the importer is currently omitting.

1 Like
6

Thank you, that's very helpful, I'll take a look at this.

EDIT: this may also be source-breaking, for the stupid reason that somebody may be using a method name or selector that the importer is currently omitting.

Yes I suspected as much. Related to this, I've been taking a look at this patch you wrote years ago: [Serialization] Mark decls that can never be cross-referenced by jrose-apple · Pull Request #17223 · apple/swift · GitHub . If you recall, was there a reason the unavailable was not enough to prevent such ambiguities? Unavailable seems like it should be sufficient for the introduced test at least? Should cross referencing these forward declarations be forbidden, even now that the compiler no longer ignores them? I don't have much insight into the implications for SourceKit.

7

Deserialization deliberately doesn’t look at availability to resolve references; for example, an unavailable function can be defined using an unavailable type in order to produce the most useful diagnostics in client modules. Usually this is something that comes up with per-platform availability, not unconditional or Swift-specific availability, but I’d hesitate to get that specific when resolving references and possibly cause even more subtle behavior. This is doubly true if you’re planning to make methods using these types available, because then serialized SIL will have to reference the unavailable types if only to immediately upcast them.

The reference to SourceKit is because SourceKit already uses a mode like what you describe, but I don’t know when they do it other than cataloguing the contents of an Objective-C module on its own. Anything more than that is lost to time, sorry.

1 Like
8

Hi everyone,

I have made a couple changes to the implementation and created a PR for Swift evolution. The implementation changes are as follows:

  • Disable the new functionality in the REPL

    • Outside of introducing a proper notion of forward declaration into the language, there does not seem to be a reasonable way to support the REPL. The unavailability of the synthesized types means they should not escape file scope, meaning issues in LLDB seem impossible. However, if import statements are made in the correct order, at the correct time within a single REPL session, one can end up with two different incompatible representations for the same Objective-C type (one representing the incomplete version, the other the complete version). This can be quite confusing, so we've opted to just disable the feature all together. We think given the relative niche of the REPL, this is worth it.

  • Added new diagnostics to diagnose missing members on the incomplete representation

    • Users may attempt to access members only available on the complete version of the type. These diagnostics help explain such cases:
swift-client-fwd-declared.swift:6:7: error: value of type 'Foo' has no member 'sayHello'
myFoo.sayHello()
~~~~~ ^~~~~~~~
<unknown>:0: note: class 'Foo' is a placeholder for a forward declared Objective-C interface and may be missing members; import the definition to access the complete interface
foo-bar-consumer.h:3:1: note: interface 'Foo' forward declared here
@class Foo;
^

More details can be found in the Swift Evolution PR: NNNN-importing-forward-declared-objc-interfaces-and-protocols.md by NuriAmari · Pull Request #1884 · apple/swift-evolution · GitHub

If there are any remaining concerns, please let me know. Thanks!

3 Likes
9

I hate to say it, but I don’t think “disable the functionality” in the REPL is a good way to go. The REPL is the expression evaluator in LLDB; while it’s not as reliable as it should be, we should not have language features that exclude certain APIs from LLDB, or change their types.

2 Likes
10

Obviously I would prefer to have feature parity with the REPL, but does there come a point where it is a cost worth paying? Is there a level of reduced functionality in the REPL that is acceptable?

The problematic scenario in the REPL looks something like this:

import Incomplete
let foo = FunctionReturningFoo()
import Complete
FunctionTakingAFoo(foo) // error: FunctionTakingAFoo expects "Complete.Foo" not "Incomplete.Foo"

As long as we allow both versions of Foo to coexist, we are going to have issues like this. I've made some attempts at making the two types more "compatible", but it seems very difficult to get 100% equivalence. I can expand on my attempts if there is interest. Are we willing to accept some level of "incompatibility" between the two versions of the type in the REPL? It seems to me that's the only reasonably achievable alternative.

If it reduces concern, I also think this could remain behind a more experimental style flag. Generally if there are compromises we can make to move forward, I'd like to identify them. The forward declaration issue seems quite significant, I think it'd be a shame not to address it for the sake of this REPL issue.

11

To be honest, having to write extra imports in LLDB in order to get access to the types that are right there in the module I’m debugging is par for the course, so I don’t imagine that this limitation will even be very noticeable.

12

Hi @jayton,

This proposal is under active review, so for best visibility comments are best made over in the review thread.