And you've landed on exactly one of the benefits of having attached macros use the same naming conventions as property wrappers—allowing for migration from property wrappers to macros without having to end up in an awkward state where something is using a different convention than if it had been built a certain way from first principles.
2 Likes
As someone who is working on her own cross-platform COM implementation (Nucom), which has recently grown a sizable Swift codebase since I’m rewriting the IDL compiler in Swift (the rest of it has stalled until I get that done), I’m very interested in seeing this. I’m eventually going to have, or already partially have, all the pieces you list under Windows platform integration in one way or another, so it would be nice to be able to integrate with that via Swift attributes in the headers or something, but I’d already be very happy with just the core object model stuff/layer 1 because what I had in mind so far with Swift support in Nucom would have been way less clean.
I have a couple more thoughts about this which I can hopefully write down later when I’m at my computer.
4 Likes
Alright, first of all, I love this a lot, and I love that it recognizes that COM has merit outside of Win32 and intends to support those cases – I guess isn't the first time, Apple has already used some form of COM on their own platforms with CFPlugIn and apparently IOKit as well after all. The layered design should help a lot with decoupling different parts of COM: the platform-independent core object ABI and base interfaces, the C/C++ headers and other compile-time stuff like IDL, and the runtime library.
Also, another important COM implementation to verify this works with is Wine/Winelib and widl, especially since it's probably the closest to Windows's!
FWIW the vtable pointer(s) could be anywhere in the object, at least as far as COM is concerned, and not having this restriction might make this more composable. What's the benefit from putting it in front of the Swift object header like this?
I think everything else I thought of yesterday was cleared up when I re-read the thread, but here's some more:
This is probably obvious, but I just want to note: Contrary to Obj-C, a well-formed COM object (or at least, one that is non-[local]) returns HRESULT for all COM interface calls except for AddRef and Release. If you always map that to throws in Swift, that means that every call on a COM interface needs to be called with try. With Swift's error handling this shouldn't be too annoying though, and it's the right thing to do, and at least for the DCOM case, matches with what Distributed Actors do at a quick glance.
Speaking of try/throws, how do you plan on mapping HRESULT to Error and the other way around? A combination of something like Foundation.POSIXError and a protocol to turn Error into HRESULT?
The other thing wrt error handling is that COM methods should (must?) not throw exceptions. This is at least the case for C++ exceptions, but I don't know in which way it applies to Swift if at all (thinking about stuff like fatalError or try!).
Lastly, you wrote this is intended to work with XPCOM which is quite different to Win32 at source level (and Nucom is also different enough to be incompatible but not quite as much) so I'm curious as to how you intend to support the different APIs.
It is a bit unclear which bit you are referring to vtable[-1] or the self[-...]. The former is just the virtual adjustment offset, which is for the conversions. We need to stash that somewhere, and the vtable layout is mandated and variable, so stashing that preceding is convenient.
As to the layout of the COM vtables preceding the object layout - that is again the same reasoning. Preceding the object is easier because the Swift object layout is already well defined and variable. It is easier to scan backwards for a known layout than to have to have a variable sized adjustment forwards.
The current design makes it significantly easier to compose and manage the necessary runtime and compiler support.
Correct - because it is surfacing the reality: every single call in COM can fail. You have network transparency: you do not know if you are IP, OoP, or remote. If it is not in InProc, the server may have crashed or your network connection may have been lost. You need to handle that, even if it terminating the application.
I don't think that I mention anything of mapping from an some Error to HRESULT. The conversion is one way only HRESULT to some Error. Generally speaking, the idea is that the HREUSLT is stashed along with a IErrorInfo is applicable. We lazily materialise the error description to avoid the possible unnecessary cost. So, effectively, you are simply storing up to ~2-registers worth of data (an unsigned long and a void *).
It applies vacuously: there are no exceptions in Swift. If you throw a C++ exception, it cannot cross the FFI boundary, and you are well within UB lands.
To be clear, I do not personally intend to heavily invest in the XPCOM path, but the design itself should lend to a pretty easy support for alternative implementations. You would need to change the shape of COMError (aka, the previously stated some Error), and some additional types may need type definitions.
Right, I meant the latter.
Ah, I suppose this is related to how you intend to implement QueryInterface without per-class codegen then. I think I get how that would work now.
So that's the error type that a COM method (implicitly?) throws? What I was thinking of is a COM method implemented in Swift where the bare throws looked like it would throw any Error, i.e.
[object, local]
interface ISomething: IUnknown {
HRESULT DoFoo();
HRESULT DoBar();
}
@COM(...)
class Something: ISomething {
func doFoo() throws {
try doBar() // I assume this will just work
do {
try doBar()
} catch {
let x = error // then I assume the type of this is COMError instead of any Error
print(x.code, x.description) // and you can get its parts something like this
throw x // and I assume propagating throwing it from the binding also works
}
// ^= return S_OK
}
func doBar() throws {
throw E_NOTIMPL // or whatever
}
}
Is that around what you have in mind?
Oh, also, I suppose this is pretty important, how do you intend on mapping different success values (for example, S_OK vs S_FALSE), both for returning and consuming?
Are all types of Errors allowed, or only COMError? If all failable COM functions are annotated as throws(COMError) it would ensure that error information isn't implicitly lost before it reaches the other end.
Yeah, the vision is that:
[object, local]
interface ISomething: IUnknown {
HRESULT DoFoo();
HRESULT DoBar();
}
would be imported as:
@COM
public protocol ISomething {
func doFoo() throws(COMError)
func doBar() throws(COMError)
}
Yeah, that is where you need to actually have the desugared form (I believe that the design notes have additional details here, but might need to revisit them). We cannot disambiguate between the S_FALSE and an error with the sugared form.
The initial implementation would be entirely desugared, but the vision is to get to the point where we can use sugar with explicit opt-outs to allow for the extended forms.
I think that within the confines of the language as it is today, it would need to be limited to COMError as otherwise you end up paying for boxing the error. This might be something that we can refine further as the implementation progresses. An option might be to use a dependent type (typealias ErrorType = Never) which the user can override to specify a secondary Error type which would enable us to use a conjunction for the error type to offer some flexibility.
1 Like
Some COM methods return non-S_OK HRESULT values to indicate special statuses like S_FALSE. How would those be handled?
There was an opt-out of the conversion (I don't remember the fully details off the top of my head, currently working on bootstrapping the compiler, so I've somewhat paged out the details). That would bring the functions in as func doFoo() -> HRESULT.
Instead of directly returning an HRESULT, would it be better to map them to a COMResult enum with .succeeded(ok: Bool) and .failed(HRESULT) cases, to prevent direct comparison to S_OK?
I think that I would prefer to avoid the COMResult approach. That becomes overlay verbose to chain versus the throwing methods. If you have a series of say 4 calls, you would need to unwrap at each call rather than being able to use a do block.
I meant as an alternative only when opting out of the throwing projection. (Who controls that opt-out, by the way?)
Ah, interesting thought on returning COMError instead of HRESULT for the opt-out. The sugaring would occur in ClangImporter, which means that the caller would have control over the sugaring.
On a per-import basis? It would be unfortunate to need to opt out an entire imported COM module just because I need to distinguish whether one method I call returns S_OK or S_FALSE. Especially if that also means all the [out] params have to be modeled as OutputSpan arguments or something instead of normal return values.
Would it be possible to make the unsugared version available as an overload?
1 Like
Urk, yet more COM esoterica: there are apparently cases where functions return a failure HRESULT for “partial successes”. So it is imperative that these APIs have an escape hatch that does not map all failing HRESULTs to a thrown error.
1 Like
It may well be the case that they do throw in Swift, and the caller is just going to have to catch specific HRESULT values of interest.
The problem is that it is valid for an implementation to do this:
HRESULT IWacky::GetFlag(/* [out, retval] */ boolean *pfRet) { hr
*pfRet = _GetFlagImpl(this);
return WACKY_E_TRYAGAIN;
}
If the bridge turns this into a thrown error, the client can’t see what value was returned via pfRet (which will be mapped to the projected function’s return value).
4 Likes
As for prior art related to HRESULT-to-exception mapping, in .NET, as far as I can tell this seems to be able to be controlled either by an attribute [PreserveSig] when writing a declaration of the interface in the .NET language (for example C#). When this is applied, it will not transform the HRESULT into throws and it will not turn the retval parameter into the return value of the method.
Interestingly, I also saw this.
By default, the Tlbexp.exe (Type Library Exporter) ensures that a call that returns an HRESULT of S_OK is transformed such that the [out, retval] parameter is used as the function return value. The S_OK HRESULT is discarded. For HRESULTs other than S_OK, the runtime throws an exception and discards the [out, retval] parameter. When you apply the PreserveSigAttribute to a managed method signature, the managed and unmanaged signatures of the attributed method are identical.
— PreserveSigAttribute Class (System.Runtime.InteropServices) | Microsoft Learn
That would mean returning S_FALSE would also throw an exception with the mapping as done by these tools. I don't know whether this is actually what they meant, and I can't quickly test it since I don't have a Windows machine.
I have to say I'm not a fan of this (either throwing on successful HRESULTs, or only S_OK) since this transformation doesn't always map nicely to the COM model and seems annoying to circumvent when it doesn't (not that I've used any language that does this transformation before together with COM so I can't say from experience). But clearly Microsoft considers it good enough, and perhaps it is good enough for most cases.
I have two initial main ideas on how I would design this myself, which are sort of variations on what has already been said:
- Always expose both the unsugared and sugared version of the method both on the public interface and when implementing it (you have to implement exactly one variant), so you can call or implement whichever based on your needs. This would naturally need additional compiler support. However, this does not match what Objective-C importing does as far as I can tell so even though this one is my preferred option, it's probably undesired.
- Always return a composite type like the following.
struct COMReturnValue<ReturnValue> {
var result: HRESULT
var returnValue: ReturnValue // see below [1]
var asTuple: (result: HRESULT, returnValue: ReturnValue) { (result, returnValue) }
var throwing: ReturnValue {
get throws {
guard result.succeeded else { throw COMError(result) }
return returnValue
}
}
}
so you can pick whether you want to access the raw result and return value, or throw.
With this return value type, chained COM calls would look thus.
let val = try someObject.getOtherObject().throwing.frobnicate().throwing
And accessing the HRESULT value along with it would be something like the following.
let (hr, otherObject) = someObject.getOtherObject().asTuple
switch hr {
case S_OK: ...
case S_FALSE: ... // we can distinguish between different success values
case WACKY_E_TRYAGAIN: ... // and we can access otherObject here
default: ...
}
let val = try otherObject.frobnicate().throwing
It's not ultra clean like the "pretend the only success value is S_OK" way, but I'd consider it usable and you do not need to special-case anything.
I still don't really like having to go through this for every COM call, however, because the .throwing is noise regardless, and at this point we might as well not treat retval in a special way compared to out parameters in the first place.
[1] Having not thought about this for too long, this should be fine, nullability notwithstanding, assuming the compiler initializes the memory for the retval parameter beforehand. Even if IIRC the general wisdom is to always initialize out parameters in the method implementation, as far as I can tell, the client may still receive uninitialized memory in out parameters for DCOM objects if the RPC call failed, assuming the DCOM layer on top of DCE/RPC doesn't initialize it, which I couldn't find anything conclusive on so I don't assume it does. This is what is written in the DCE/RPC spec.
Parameter semantics depend on the IDL directional attributes as follows:
- The value of a parameter with the in attribute is marshalled by the client stub when the client invokes the call. It is unmarshalled by the server stub and passed to the server manager on invocation by the stub.
- The value of a parameter with the out attribute is marshalled by the server stub when the server manager routine returns. It is unmarshalled by the client stub and passed to the client when the call returns.
- The value of a parameter with the attributes in, out is passed from the client to the server as an in parameter and passed from the server to the client as an out parameter.
In the event of an abnormal end-of-call, resulting from either an exception condition on the server (which may be reported through fault_status parameter) or a communications failure (which may be reported through comm_status parameter) the values of out parameters are undefined.
— C706, section 5.1.1
1 Like