Understanding runtime entry points

Development Compiler
1

Hello,

I'm trying to better understand how calls are made between the Swift standard library code and the runtime entry points. I've read through most of the documentation in the repo but still have some questions.

More specifically, here's an example: the '_EnumMirror' struct below represents a mirror reflecting a value whose type is an enum.

···

------

struct _EnumMirror : _MirrorType {
  let data: _MagicMirrorData
  var value: Any { return data.value }
  var valueType: Any.Type { return data.valueType }
  // ... more stuff

  var caseName: UnsafePointer<CChar> {
    @_silgen_name("swift_EnumMirror_caseName")get
  }
  // ... (more stuff)
}

------

The 'caseName' property represents the name of the enum value's case (e.g. "FirstCase" in Foo.FirstCase) as a C string. This property's getter calls into a C++ runtime function named "swift_EnumMirror_caseName", which is reproduced below (from Reflection.mm):

extern "C"
const char *swift_EnumMirror_caseName(HeapObject *owner,
                                      const OpaqueValue *value,
                                      const Metadata *type) {
  if (!isEnumReflectable(type))
    return nullptr;

  const auto Enum = static_cast<const EnumMetadata *>(type);
  const auto &Description = Enum->Description->Enum;

  unsigned tag;
  getEnumMirrorInfo(value, type, &tag, nullptr, nullptr); // effectively, same as "tag = type->vw_getEnumTag(value);"
  return getFieldName(Description.CaseNames, tag);
}

Now, I had a few questions about exactly how this interoperation works, because I'd like to be able to get the name of an enum case using this entry point from a different context (not from within an _EnumMirror property).

* swift_EnumMirror_caseName takes three arguments, but the Swift call site doesn't seem to specify what gets passed into the function. Is there a convention that is implicitly passing properties on _EnumMirror as arguments into the C++ function when it's being called? I did note that there were other runtime entry points (like "swift_MagicMirrorData_summary") where the number of arguments in the Swift function matched the number of arguments in the C++ function, but in those cases the Swift function was a free function and not a method.

* What I really want to do is to get the tag of an enum. I wrote a different entry point that omits the unused "owner" property and simply calls swift_EnumMirror_caseName with nullptr as the first argument. This other C++ function takes 'value' (an OpaqueValue*) and 'type' (a Metadata*). I've surmised that 'type' should be the Swift metatype of the enum instance (e.g. myEnum.dynamicType), and I do get the case names table. However, if I pass in the enum instance itself as 'value', my tag is always retrieved as 0. I noticed that there's some sort of indirection in the form of "vw_getEnumTag", which goes through something called the "value witness". Is there somewhere I can read up about the value witness concept? I assume the reason the 'original' code worked was because it was passing in a different object as 'value', maybe one that could serve as a value witness for the reflected-upon instance's type.

Thanks a lot for your time.

Best,
Austin

2

Hi Austin,

Hello,

I'm trying to better understand how calls are made between the Swift standard library code and the runtime entry points. I've read through most of the documentation in the repo but still have some questions.

More specifically, here's an example: the '_EnumMirror' struct below represents a mirror reflecting a value whose type is an enum.

------

struct _EnumMirror : _MirrorType {
let data: _MagicMirrorData
var value: Any { return data.value }
var valueType: Any.Type { return data.valueType }
// ... more stuff

var caseName: UnsafePointer<CChar> {
   @_silgen_name("swift_EnumMirror_caseName")get
}
// ... (more stuff)
}

------

The 'caseName' property represents the name of the enum value's case (e.g. "FirstCase" in Foo.FirstCase) as a C string. This property's getter calls into a C++ runtime function named "swift_EnumMirror_caseName", which is reproduced below (from Reflection.mm):

extern "C"
const char *swift_EnumMirror_caseName(HeapObject *owner,
                                     const OpaqueValue *value,
                                     const Metadata *type) {
if (!isEnumReflectable(type))
   return nullptr;

const auto Enum = static_cast<const EnumMetadata *>(type);
const auto &Description = Enum->Description->Enum;

unsigned tag;
getEnumMirrorInfo(value, type, &tag, nullptr, nullptr); // effectively, same as "tag = type->vw_getEnumTag(value);"
return getFieldName(Description.CaseNames, tag);
}

Now, I had a few questions about exactly how this interoperation works, because I'd like to be able to get the name of an enum case using this entry point from a different context (not from within an _EnumMirror property).

* swift_EnumMirror_caseName takes three arguments, but the Swift call site doesn't seem to specify what gets passed into the function.

The three arguments together form the 'self' value of the call. That is, an EnumMirror is a struct containing a pointer to the owner object, a pointer to the value being reflected, and runtime type information for the value. You can see this if you look at how the _MagicMirrorData struct is defined on the swift side.

Is there a convention that is implicitly passing properties on _EnumMirror as arguments into the C++ function when it's being called? I did note that there were other runtime entry points (like "swift_MagicMirrorData_summary") where the number of arguments in the Swift function matched the number of arguments in the C++ function, but in those cases the Swift function was a free function and not a method.

Right.

* What I really want to do is to get the tag of an enum. I wrote a different entry point that omits the unused "owner" property and simply calls swift_EnumMirror_caseName with nullptr as the first argument. This other C++ function takes 'value' (an OpaqueValue*) and 'type' (a Metadata*). I've surmised that 'type' should be the Swift metatype of the enum instance (e.g. myEnum.dynamicType), and I do get the case names table. However, if I pass in the enum instance itself as 'value', my tag is always retrieved as 0.

The value should indeed be a pointer to the enum value itself. Not sure why it's not working for you, maybe you can share more code?

I noticed that there's some sort of indirection in the form of "vw_getEnumTag", which goes through something called the "value witness". Is there somewhere I can read up about the value witness concept? I assume the reason the 'original' code worked was because it was passing in a different object as 'value', maybe one that could serve as a value witness for the reflected-upon instance's type.

The value witness table is a member of the type metadata. It contains entry points for runtime manipulation of values of that type. The value witness table is used for runtime generics (when I have a generic parameter 'T' and a value of type 'T', the value witness functions are used for copying/moving/destroying/etc values of type 'T'). They are also used for reflection.

They're not really documented anywhere except for in the source code itself -- see here:

include/swift/Runtime/Metadata.h
lib/IRGen/ValueWitness.h

Slava

···

On Jan 1, 2016, at 10:58 PM, Austin Zheng via swift-dev <swift-dev@swift.org> wrote:

Thanks a lot for your time.

Best,
Austin

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

3

Hello,

I'm trying to better understand how calls are made between the Swift standard library code and the runtime entry points. I've read through most of the documentation in the repo but still have some questions.

The particular functions in question aren't so much runtime entry points as Swift methods that happen to be written in C++, in a way that makes them incidentally ABI-compatible with Swift methods.

More specifically, here's an example: the '_EnumMirror' struct below represents a mirror reflecting a value whose type is an enum.

------

struct _EnumMirror : _MirrorType {
let data: _MagicMirrorData
var value: Any { return data.value }
var valueType: Any.Type { return data.valueType }
// ... more stuff

var caseName: UnsafePointer<CChar> {
   @_silgen_name("swift_EnumMirror_caseName")get
}
// ... (more stuff)
}

------

The 'caseName' property represents the name of the enum value's case (e.g. "FirstCase" in Foo.FirstCase) as a C string. This property's getter calls into a C++ runtime function named "swift_EnumMirror_caseName", which is reproduced below (from Reflection.mm):

extern "C"
const char *swift_EnumMirror_caseName(HeapObject *owner,
                                     const OpaqueValue *value,
                                     const Metadata *type) {
if (!isEnumReflectable(type))
   return nullptr;

const auto Enum = static_cast<const EnumMetadata *>(type);
const auto &Description = Enum->Description->Enum;

unsigned tag;
getEnumMirrorInfo(value, type, &tag, nullptr, nullptr); // effectively, same as "tag = type->vw_getEnumTag(value);"
return getFieldName(Description.CaseNames, tag);
}

Now, I had a few questions about exactly how this interoperation works, because I'd like to be able to get the name of an enum case using this entry point from a different context (not from within an _EnumMirror property).

* swift_EnumMirror_caseName takes three arguments, but the Swift call site doesn't seem to specify what gets passed into the function. Is there a convention that is implicitly passing properties on _EnumMirror as arguments into the C++ function when it's being called? I did note that there were other runtime entry points (like "swift_MagicMirrorData_summary") where the number of arguments in the Swift function matched the number of arguments in the C++ function, but in those cases the Swift function was a free function and not a method.

Swift structs are (currently) passed by passing each stored property individually. _EnumMirror is defined as a struct with owner/value/type fields, and the 'self' parameter to its methods gets broken down this way.

* What I really want to do is to get the tag of an enum. I wrote a different entry point that omits the unused "owner" property and simply calls swift_EnumMirror_caseName with nullptr as the first argument. This other C++ function takes 'value' (an OpaqueValue*) and 'type' (a Metadata*). I've surmised that 'type' should be the Swift metatype of the enum instance (e.g. myEnum.dynamicType), and I do get the case names table. However, if I pass in the enum instance itself as 'value', my tag is always retrieved as 0. I noticed that there's some sort of indirection in the form of "vw_getEnumTag", which goes through something called the "value witness". Is there somewhere I can read up about the value witness concept? I assume the reason the 'original' code worked was because it was passing in a different object as 'value', maybe one that could serve as a value witness for the reflected-upon instance's type.

'value' is a pointer to the value in memory, not the value itself. Are you passing the enum's literal representation by value instead of passing a pointer to it?

-Joe

···

On Jan 1, 2016, at 10:58 PM, Austin Zheng via swift-dev <swift-dev@swift.org> wrote:

4

Thanks Joe, this is incredibly helpful, especially the note about how enums
are passed in 'piecewise'.

re. your question: I was indeed calling the C++ function from Swift and
directly passing in the enum (in pseudo-Swift):
let myEnum : SomeEnumType = ...
cplusplusFunction(value: myEnum, type: SomeEnumType.self) // incorrect

How would I pass a pointer to the enum's value from Swift? I don't think
Swift exposes any mechanism to get the memory address of a Swift object.
Perhaps I should create an UnsafeMutablePointer<SomeEnumType>, set its
memory to (a copy of) the enum value, and pass that pointer object in
instead?

Austin

···

On Wed, Jan 6, 2016 at 12:04 PM, Joe Groff <jgroff@apple.com> wrote:

> On Jan 1, 2016, at 10:58 PM, Austin Zheng via swift-dev < > swift-dev@swift.org> wrote:
>
> Hello,
> (...)

Swift structs are (currently) passed by passing each stored property
individually. _EnumMirror is defined as a struct with owner/value/type
fields, and the 'self' parameter to its methods gets broken down this way.

> * What I really want to do is to get the tag of an enum. I wrote a
different entry point that omits the unused "owner" property and simply
calls swift_EnumMirror_caseName with nullptr as the first argument. This
other C++ function takes 'value' (an OpaqueValue*) and 'type' (a
Metadata*). I've surmised that 'type' should be the Swift metatype of the
enum instance (e.g. myEnum.dynamicType), and I do get the case names table.
However, if I pass in the enum instance itself as 'value', my tag is always
retrieved as 0. I noticed that there's some sort of indirection in the form
of "vw_getEnumTag", which goes through something called the "value
witness". Is there somewhere I can read up about the value witness concept?
I assume the reason the 'original' code worked was because it was passing
in a different object as 'value', maybe one that could serve as a value
witness for the reflected-upon instance's type.

'value' is a pointer to the value in memory, not the value itself. Are you
passing the enum's literal representation by value instead of passing a
pointer to it?

-Joe

5

Hey Slava,

Thanks a lot for your detailed responses; it definitely helps to understand
how structs are passed to the C++ function.

In a separate email, Joe Groff mentioned that there was a difference
between passing the enum value and passing a pointer to it. I think that
might be the root of my problem. I'll try a few things and send over a
better code sample tonight, if there are still issues.

Best,
Austin

···

On Wed, Jan 6, 2016 at 11:37 AM, Slava Pestov <spestov@apple.com> wrote:

Hi Austin,

> On Jan 1, 2016, at 10:58 PM, Austin Zheng via swift-dev < > swift-dev@swift.org> wrote:
>
> Hello,
>
> I'm trying to better understand how calls are made between the Swift
standard library code and the runtime entry points. I've read through most
of the documentation in the repo but still have some questions.
>
> More specifically, here's an example: the '_EnumMirror' struct below
represents a mirror reflecting a value whose type is an enum.
>
> ------
>
> struct _EnumMirror : _MirrorType {
> let data: _MagicMirrorData
> var value: Any { return data.value }
> var valueType: Any.Type { return data.valueType }
> // ... more stuff
>
> var caseName: UnsafePointer<CChar> {
> @_silgen_name("swift_EnumMirror_caseName")get
> }
> // ... (more stuff)
> }
>
> ------
>
> The 'caseName' property represents the name of the enum value's case
(e.g. "FirstCase" in Foo.FirstCase) as a C string. This property's getter
calls into a C++ runtime function named "swift_EnumMirror_caseName", which
is reproduced below (from Reflection.mm):
>
> extern "C"
> const char *swift_EnumMirror_caseName(HeapObject *owner,
> const OpaqueValue *value,
> const Metadata *type) {
> if (!isEnumReflectable(type))
> return nullptr;
>
> const auto Enum = static_cast<const EnumMetadata *>(type);
> const auto &Description = Enum->Description->Enum;
>
> unsigned tag;
> getEnumMirrorInfo(value, type, &tag, nullptr, nullptr); //
effectively, same as "tag = type->vw_getEnumTag(value);"
> return getFieldName(Description.CaseNames, tag);
> }
>
> Now, I had a few questions about exactly how this interoperation works,
because I'd like to be able to get the name of an enum case using this
entry point from a different context (not from within an _EnumMirror
property).
>
> * swift_EnumMirror_caseName takes three arguments, but the Swift call
site doesn't seem to specify what gets passed into the function.

The three arguments together form the 'self' value of the call. That is,
an EnumMirror is a struct containing a pointer to the owner object, a
pointer to the value being reflected, and runtime type information for the
value. You can see this if you look at how the _MagicMirrorData struct is
defined on the swift side.

> Is there a convention that is implicitly passing properties on
_EnumMirror as arguments into the C++ function when it's being called? I
did note that there were other runtime entry points (like
"swift_MagicMirrorData_summary") where the number of arguments in the Swift
function matched the number of arguments in the C++ function, but in those
cases the Swift function was a free function and not a method.

Right.

>
> * What I really want to do is to get the tag of an enum. I wrote a
different entry point that omits the unused "owner" property and simply
calls swift_EnumMirror_caseName with nullptr as the first argument. This
other C++ function takes 'value' (an OpaqueValue*) and 'type' (a
Metadata*). I've surmised that 'type' should be the Swift metatype of the
enum instance (e.g. myEnum.dynamicType), and I do get the case names table.
However, if I pass in the enum instance itself as 'value', my tag is always
retrieved as 0.

The value should indeed be a pointer to the enum value itself. Not sure
why it's not working for you, maybe you can share more code?

> I noticed that there's some sort of indirection in the form of
"vw_getEnumTag", which goes through something called the "value witness".
Is there somewhere I can read up about the value witness concept? I assume
the reason the 'original' code worked was because it was passing in a
different object as 'value', maybe one that could serve as a value witness
for the reflected-upon instance's type.

The value witness table is a member of the type metadata. It contains
entry points for runtime manipulation of values of that type. The value
witness table is used for runtime generics (when I have a generic parameter
'T' and a value of type 'T', the value witness functions are used for
copying/moving/destroying/etc values of type 'T'). They are also used for
reflection.

They're not really documented anywhere except for in the source code
itself -- see here:

include/swift/Runtime/Metadata.h
lib/IRGen/ValueWitness.h

Slava

>
> Thanks a lot for your time.
>
> Best,
> Austin
>
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

6

Hey Slava,

Thanks a lot for your detailed responses; it definitely helps to understand how structs are passed to the C++ function.

In a separate email, Joe Groff mentioned that there was a difference between passing the enum value and passing a pointer to it. I think that might be the root of my problem. I'll try a few things and send over a better code sample tonight, if there are still issues.

Yeah, that sounds like it could be the problem.

In general, expecting the Swift calling convention to line up with C++ in this manner is really not something we want to officially support, and I believe JoeG is currently working on refactoring the runtime reflection interface to be a bit more minimal and orthogonal on the runtime side.

It would be great to see what you're working on, because I suspect we should be able to fold your use-case into the general reflection API.

Slava

···

On Jan 6, 2016, at 12:57 PM, Austin Zheng <austinzheng@gmail.com> wrote:

Best,
Austin

On Wed, Jan 6, 2016 at 11:37 AM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:
Hi Austin,

> On Jan 1, 2016, at 10:58 PM, Austin Zheng via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>
> Hello,
>
> I'm trying to better understand how calls are made between the Swift standard library code and the runtime entry points. I've read through most of the documentation in the repo but still have some questions.
>
> More specifically, here's an example: the '_EnumMirror' struct below represents a mirror reflecting a value whose type is an enum.
>
> ------
>
> struct _EnumMirror : _MirrorType {
> let data: _MagicMirrorData
> var value: Any { return data.value }
> var valueType: Any.Type { return data.valueType }
> // ... more stuff
>
> var caseName: UnsafePointer<CChar> {
> @_silgen_name("swift_EnumMirror_caseName")get
> }
> // ... (more stuff)
> }
>
> ------
>
> The 'caseName' property represents the name of the enum value's case (e.g. "FirstCase" in Foo.FirstCase) as a C string. This property's getter calls into a C++ runtime function named "swift_EnumMirror_caseName", which is reproduced below (from Reflection.mm):
>
> extern "C"
> const char *swift_EnumMirror_caseName(HeapObject *owner,
> const OpaqueValue *value,
> const Metadata *type) {
> if (!isEnumReflectable(type))
> return nullptr;
>
> const auto Enum = static_cast<const EnumMetadata *>(type);
> const auto &Description = Enum->Description->Enum;
>
> unsigned tag;
> getEnumMirrorInfo(value, type, &tag, nullptr, nullptr); // effectively, same as "tag = type->vw_getEnumTag(value);"
> return getFieldName(Description.CaseNames, tag);
> }
>
> Now, I had a few questions about exactly how this interoperation works, because I'd like to be able to get the name of an enum case using this entry point from a different context (not from within an _EnumMirror property).
>
> * swift_EnumMirror_caseName takes three arguments, but the Swift call site doesn't seem to specify what gets passed into the function.

The three arguments together form the 'self' value of the call. That is, an EnumMirror is a struct containing a pointer to the owner object, a pointer to the value being reflected, and runtime type information for the value. You can see this if you look at how the _MagicMirrorData struct is defined on the swift side.

> Is there a convention that is implicitly passing properties on _EnumMirror as arguments into the C++ function when it's being called? I did note that there were other runtime entry points (like "swift_MagicMirrorData_summary") where the number of arguments in the Swift function matched the number of arguments in the C++ function, but in those cases the Swift function was a free function and not a method.

Right.

>
> * What I really want to do is to get the tag of an enum. I wrote a different entry point that omits the unused "owner" property and simply calls swift_EnumMirror_caseName with nullptr as the first argument. This other C++ function takes 'value' (an OpaqueValue*) and 'type' (a Metadata*). I've surmised that 'type' should be the Swift metatype of the enum instance (e.g. myEnum.dynamicType), and I do get the case names table. However, if I pass in the enum instance itself as 'value', my tag is always retrieved as 0.

The value should indeed be a pointer to the enum value itself. Not sure why it's not working for you, maybe you can share more code?

> I noticed that there's some sort of indirection in the form of "vw_getEnumTag", which goes through something called the "value witness". Is there somewhere I can read up about the value witness concept? I assume the reason the 'original' code worked was because it was passing in a different object as 'value', maybe one that could serve as a value witness for the reflected-upon instance's type.

The value witness table is a member of the type metadata. It contains entry points for runtime manipulation of values of that type. The value witness table is used for runtime generics (when I have a generic parameter 'T' and a value of type 'T', the value witness functions are used for copying/moving/destroying/etc values of type 'T'). They are also used for reflection.

They're not really documented anywhere except for in the source code itself -- see here:

include/swift/Runtime/Metadata.h
lib/IRGen/ValueWitness.h

Slava

>
> Thanks a lot for your time.
>
> Best,
> Austin
>
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org <mailto:swift-dev@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-dev

7

Thanks Joe, this is incredibly helpful, especially the note about how enums are passed in 'piecewise'.

To clarify, what you're seeing in EnumMirror is not the enum itself, but the _EnumMirror structure, which is itself designed to be bitwise-compatible with a generic (owner, type, pointer to value) triple that can reference an arbitrary value in memory while keeping it alive if reference-counted. The 'value' field in this structure is always a pointer to the value of the 'type' type in memory.

re. your question: I was indeed calling the C++ function from Swift and directly passing in the enum (in pseudo-Swift):
let myEnum : SomeEnumType = ...
cplusplusFunction(value: myEnum, type: SomeEnumType.self) // incorrect

How would I pass a pointer to the enum's value from Swift? I don't think Swift exposes any mechanism to get the memory address of a Swift object. Perhaps I should create an UnsafeMutablePointer<SomeEnumType>, set its memory to (a copy of) the enum value, and pass that pointer object in instead?

If all you want is the enum tag, you could implement a more straightforward interface. In Swift:

@_silgen_name("_swift_stdlib_getEnumTag")
func _getEnumTag<T>(value: T) -> Int32

In C++:

/// Return the tag number for an enum, or crash if the value is not of enum type.
int32_t _swift_stdlib_getEnumTag(OpaqueValue *value, const Metadata *type) {
  assert(type->getKind() == MetadataKind::Enum);
  auto tag = type->vw_getEnumTag(value);
  type->vw_destroy(value);
  return tag;
}

-Joe

···

On Jan 6, 2016, at 12:46 PM, Austin Zheng <austinzheng@gmail.com> wrote:

8

Hi Slava,

I took a closer look at how reflectAny() and MagicMirror's C++ constructors work. It appears that instances of value types are being boxed at the time the mirror is created, which would explain the difference in semantics between MagicMirrorData's 'ptr' property and the raw value I'm passing into my function. (In retrospect, this should have been obvious: the 'ptr' property is of type Builtin.RawPointer.)

To step back and provide some more context: I'm working on SR-88 (https://bugs.swift.org/browse/SR-88\). This task involves moving the standard library APIs off the old reflection API (using _reflect(), _Reflectable, _MirrorType, etc) and onto the CustomReflectable API. This work is going to precede Joe Groff designing a new runtime reflection API, like you mentioned.

Part of this involves rewriting the _adHocPrint() function in OutputStream.swift. (This is the function that the standard library uses to print out any type that doesn't conform to the various string convertible or output stream protocols.) Currently, that function uses _reflect() to create a mirror of the object to print out, and then matches on its type to determine how to print it. For enums (which generate an _EnumMirror), the case name is printed out by using the _EnumMirror instance's 'caseName' property, which in turn calls into the "swift_EnumMirror_caseName" function in reflection.mm.

Since we don't want to rely upon _reflect(), and want to delete as many of the uses of the underscored mirror APIs as possible, my goal is to figure out how to call into the runtime to get the enum case name without needing to create an _EnumMirror. This is sort of transitory work, admittedly, because once the new runtime reflection API is in place it should provide that functionality instead. It's possible that it might just be easier to create an _EnumMirror and use it to get the case name in that one specific situation, without resorting to calling into the runtime.

I got something extremely rudimentary working. It needs to be cleaned up (if it goes in at all), it probably leaks memory, but the following seems to produce the proper enum cases if the variable passed into the Swift function isn't of type Any:

SWIFT:
@_silgen_name("swift_EnumCaseName")
func _getEnumCaseName<T>(value: T, _ type: Any.Type) -> UnsafePointer<CChar>

C++:
extern "C"
const char *swift_EnumCaseName(OpaqueValue *value, const Metadata *type) {
  if (!isEnumReflectable(type))
    return nullptr;

  // Put the enum in a box so we can get a pointer to it
  BoxPair box = swift_allocBox(type);
  type->vw_initializeWithCopy(box.second, value);
  
  const auto Enum = static_cast<const EnumMetadata *>(type);
  const auto &Description = Enum->Description->Enum;

  const auto tag = type->vw_getEnumTag(box.second);
  //type->vw_destroy(value);
  return getFieldName(Description.CaseNames, tag);
}

Thanks for your help, and if you have any questions let me know.

Best,
Austin

···

On Jan 6, 2016, at 1:11 PM, Slava Pestov <spestov@apple.com> wrote:

On Jan 6, 2016, at 12:57 PM, Austin Zheng <austinzheng@gmail.com <mailto:austinzheng@gmail.com>> wrote:

Hey Slava,

Thanks a lot for your detailed responses; it definitely helps to understand how structs are passed to the C++ function.

In a separate email, Joe Groff mentioned that there was a difference between passing the enum value and passing a pointer to it. I think that might be the root of my problem. I'll try a few things and send over a better code sample tonight, if there are still issues.

Yeah, that sounds like it could be the problem.

In general, expecting the Swift calling convention to line up with C++ in this manner is really not something we want to officially support, and I believe JoeG is currently working on refactoring the runtime reflection interface to be a bit more minimal and orthogonal on the runtime side.

It would be great to see what you're working on, because I suspect we should be able to fold your use-case into the general reflection API.

Slava

Best,
Austin

On Wed, Jan 6, 2016 at 11:37 AM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:
Hi Austin,

> On Jan 1, 2016, at 10:58 PM, Austin Zheng via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>
> Hello,
>
> I'm trying to better understand how calls are made between the Swift standard library code and the runtime entry points. I've read through most of the documentation in the repo but still have some questions.
>
> More specifically, here's an example: the '_EnumMirror' struct below represents a mirror reflecting a value whose type is an enum.
>
> ------
>
> struct _EnumMirror : _MirrorType {
> let data: _MagicMirrorData
> var value: Any { return data.value }
> var valueType: Any.Type { return data.valueType }
> // ... more stuff
>
> var caseName: UnsafePointer<CChar> {
> @_silgen_name("swift_EnumMirror_caseName")get
> }
> // ... (more stuff)
> }
>
> ------
>
> The 'caseName' property represents the name of the enum value's case (e.g. "FirstCase" in Foo.FirstCase) as a C string. This property's getter calls into a C++ runtime function named "swift_EnumMirror_caseName", which is reproduced below (from Reflection.mm):
>
> extern "C"
> const char *swift_EnumMirror_caseName(HeapObject *owner,
> const OpaqueValue *value,
> const Metadata *type) {
> if (!isEnumReflectable(type))
> return nullptr;
>
> const auto Enum = static_cast<const EnumMetadata *>(type);
> const auto &Description = Enum->Description->Enum;
>
> unsigned tag;
> getEnumMirrorInfo(value, type, &tag, nullptr, nullptr); // effectively, same as "tag = type->vw_getEnumTag(value);"
> return getFieldName(Description.CaseNames, tag);
> }
>
> Now, I had a few questions about exactly how this interoperation works, because I'd like to be able to get the name of an enum case using this entry point from a different context (not from within an _EnumMirror property).
>
> * swift_EnumMirror_caseName takes three arguments, but the Swift call site doesn't seem to specify what gets passed into the function.

The three arguments together form the 'self' value of the call. That is, an EnumMirror is a struct containing a pointer to the owner object, a pointer to the value being reflected, and runtime type information for the value. You can see this if you look at how the _MagicMirrorData struct is defined on the swift side.

> Is there a convention that is implicitly passing properties on _EnumMirror as arguments into the C++ function when it's being called? I did note that there were other runtime entry points (like "swift_MagicMirrorData_summary") where the number of arguments in the Swift function matched the number of arguments in the C++ function, but in those cases the Swift function was a free function and not a method.

Right.

>
> * What I really want to do is to get the tag of an enum. I wrote a different entry point that omits the unused "owner" property and simply calls swift_EnumMirror_caseName with nullptr as the first argument. This other C++ function takes 'value' (an OpaqueValue*) and 'type' (a Metadata*). I've surmised that 'type' should be the Swift metatype of the enum instance (e.g. myEnum.dynamicType), and I do get the case names table. However, if I pass in the enum instance itself as 'value', my tag is always retrieved as 0.

The value should indeed be a pointer to the enum value itself. Not sure why it's not working for you, maybe you can share more code?

> I noticed that there's some sort of indirection in the form of "vw_getEnumTag", which goes through something called the "value witness". Is there somewhere I can read up about the value witness concept? I assume the reason the 'original' code worked was because it was passing in a different object as 'value', maybe one that could serve as a value witness for the reflected-upon instance's type.

The value witness table is a member of the type metadata. It contains entry points for runtime manipulation of values of that type. The value witness table is used for runtime generics (when I have a generic parameter 'T' and a value of type 'T', the value witness functions are used for copying/moving/destroying/etc values of type 'T'). They are also used for reflection.

They're not really documented anywhere except for in the source code itself -- see here:

include/swift/Runtime/Metadata.h
lib/IRGen/ValueWitness.h

Slava

>
> Thanks a lot for your time.
>
> Best,
> Austin
>
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org <mailto:swift-dev@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-dev

9

Hi Slava,

I took a closer look at how reflectAny() and MagicMirror's C++ constructors work. It appears that instances of value types are being boxed at the time the mirror is created, which would explain the difference in semantics between MagicMirrorData's 'ptr' property and the raw value I'm passing into my function. (In retrospect, this should have been obvious: the 'ptr' property is of type Builtin.RawPointer.)

To step back and provide some more context: I'm working on SR-88 (https://bugs.swift.org/browse/SR-88\). This task involves moving the standard library APIs off the old reflection API (using _reflect(), _Reflectable, _MirrorType, etc) and onto the CustomReflectable API. This work is going to precede Joe Groff designing a new runtime reflection API, like you mentioned.

Part of this involves rewriting the _adHocPrint() function in OutputStream.swift. (This is the function that the standard library uses to print out any type that doesn't conform to the various string convertible or output stream protocols.) Currently, that function uses _reflect() to create a mirror of the object to print out, and then matches on its type to determine how to print it. For enums (which generate an _EnumMirror), the case name is printed out by using the _EnumMirror instance's 'caseName' property, which in turn calls into the "swift_EnumMirror_caseName" function in reflection.mm.

Since we don't want to rely upon _reflect(), and want to delete as many of the uses of the underscored mirror APIs as possible, my goal is to figure out how to call into the runtime to get the enum case name without needing to create an _EnumMirror. This is sort of transitory work, admittedly, because once the new runtime reflection API is in place it should provide that functionality instead. It's possible that it might just be easier to create an _EnumMirror and use it to get the case name in that one specific situation, without resorting to calling into the runtime.

I got something extremely rudimentary working. It needs to be cleaned up (if it goes in at all), it probably leaks memory, but the following seems to produce the proper enum cases if the variable passed into the Swift function isn't of type Any:

SWIFT:
@_silgen_name("swift_EnumCaseName")
func _getEnumCaseName<T>(value: T, _ type: Any.Type) -> UnsafePointer<CChar>

You don't need the type parameter here; the generic T parameter is sufficient. The boxing you do is wasteful as well. Did you try the implementation I sent you in my previous email?

-Joe

···

On Jan 6, 2016, at 11:38 PM, Austin Zheng via swift-dev <swift-dev@swift.org> wrote:

C++:
extern "C"
const char *swift_EnumCaseName(OpaqueValue *value, const Metadata *type) {
  if (!isEnumReflectable(type))
    return nullptr;

  // Put the enum in a box so we can get a pointer to it
  BoxPair box = swift_allocBox(type);
  type->vw_initializeWithCopy(box.second, value);
  
  const auto Enum = static_cast<const EnumMetadata *>(type);
  const auto &Description = Enum->Description->Enum;

  const auto tag = type->vw_getEnumTag(box.second);
  //type->vw_destroy(value);
  return getFieldName(Description.CaseNames, tag);
}

Thanks for your help, and if you have any questions let me know.

Best,
Austin

On Jan 6, 2016, at 1:11 PM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:

On Jan 6, 2016, at 12:57 PM, Austin Zheng <austinzheng@gmail.com <mailto:austinzheng@gmail.com>> wrote:

Hey Slava,

Thanks a lot for your detailed responses; it definitely helps to understand how structs are passed to the C++ function.

In a separate email, Joe Groff mentioned that there was a difference between passing the enum value and passing a pointer to it. I think that might be the root of my problem. I'll try a few things and send over a better code sample tonight, if there are still issues.

Yeah, that sounds like it could be the problem.

In general, expecting the Swift calling convention to line up with C++ in this manner is really not something we want to officially support, and I believe JoeG is currently working on refactoring the runtime reflection interface to be a bit more minimal and orthogonal on the runtime side.

It would be great to see what you're working on, because I suspect we should be able to fold your use-case into the general reflection API.

Slava

Best,
Austin

On Wed, Jan 6, 2016 at 11:37 AM, Slava Pestov <spestov@apple.com <mailto:spestov@apple.com>> wrote:
Hi Austin,

> On Jan 1, 2016, at 10:58 PM, Austin Zheng via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:
>
> Hello,
>
> I'm trying to better understand how calls are made between the Swift standard library code and the runtime entry points. I've read through most of the documentation in the repo but still have some questions.
>
> More specifically, here's an example: the '_EnumMirror' struct below represents a mirror reflecting a value whose type is an enum.
>
> ------
>
> struct _EnumMirror : _MirrorType {
> let data: _MagicMirrorData
> var value: Any { return data.value }
> var valueType: Any.Type { return data.valueType }
> // ... more stuff
>
> var caseName: UnsafePointer<CChar> {
> @_silgen_name("swift_EnumMirror_caseName")get
> }
> // ... (more stuff)
> }
>
> ------
>
> The 'caseName' property represents the name of the enum value's case (e.g. "FirstCase" in Foo.FirstCase) as a C string. This property's getter calls into a C++ runtime function named "swift_EnumMirror_caseName", which is reproduced below (from Reflection.mm):
>
> extern "C"
> const char *swift_EnumMirror_caseName(HeapObject *owner,
> const OpaqueValue *value,
> const Metadata *type) {
> if (!isEnumReflectable(type))
> return nullptr;
>
> const auto Enum = static_cast<const EnumMetadata *>(type);
> const auto &Description = Enum->Description->Enum;
>
> unsigned tag;
> getEnumMirrorInfo(value, type, &tag, nullptr, nullptr); // effectively, same as "tag = type->vw_getEnumTag(value);"
> return getFieldName(Description.CaseNames, tag);
> }
>
> Now, I had a few questions about exactly how this interoperation works, because I'd like to be able to get the name of an enum case using this entry point from a different context (not from within an _EnumMirror property).
>
> * swift_EnumMirror_caseName takes three arguments, but the Swift call site doesn't seem to specify what gets passed into the function.

The three arguments together form the 'self' value of the call. That is, an EnumMirror is a struct containing a pointer to the owner object, a pointer to the value being reflected, and runtime type information for the value. You can see this if you look at how the _MagicMirrorData struct is defined on the swift side.

> Is there a convention that is implicitly passing properties on _EnumMirror as arguments into the C++ function when it's being called? I did note that there were other runtime entry points (like "swift_MagicMirrorData_summary") where the number of arguments in the Swift function matched the number of arguments in the C++ function, but in those cases the Swift function was a free function and not a method.

Right.

>
> * What I really want to do is to get the tag of an enum. I wrote a different entry point that omits the unused "owner" property and simply calls swift_EnumMirror_caseName with nullptr as the first argument. This other C++ function takes 'value' (an OpaqueValue*) and 'type' (a Metadata*). I've surmised that 'type' should be the Swift metatype of the enum instance (e.g. myEnum.dynamicType), and I do get the case names table. However, if I pass in the enum instance itself as 'value', my tag is always retrieved as 0.

The value should indeed be a pointer to the enum value itself. Not sure why it's not working for you, maybe you can share more code?

> I noticed that there's some sort of indirection in the form of "vw_getEnumTag", which goes through something called the "value witness". Is there somewhere I can read up about the value witness concept? I assume the reason the 'original' code worked was because it was passing in a different object as 'value', maybe one that could serve as a value witness for the reflected-upon instance's type.

The value witness table is a member of the type metadata. It contains entry points for runtime manipulation of values of that type. The value witness table is used for runtime generics (when I have a generic parameter 'T' and a value of type 'T', the value witness functions are used for copying/moving/destroying/etc values of type 'T'). They are also used for reflection.

They're not really documented anywhere except for in the source code itself -- see here:

include/swift/Runtime/Metadata.h
lib/IRGen/ValueWitness.h

Slava

>
> Thanks a lot for your time.
>
> Best,
> Austin
>
> _______________________________________________
> swift-dev mailing list
> swift-dev@swift.org <mailto:swift-dev@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-dev

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev