SwiftReflectionTest small differences between platforms

Development LLDB
1

@vedantk @Adrian_Prantl @Joe_Groff @Slava_Pestov @Mike_Ash @dcci

Hi everyone!
This is a followup to Adapting SwiftReflectionTest to work on Linux.

Right now, I got SwiftReflectionTest to output metadata values for our tests, but there are 11 tests that still fail (not counting the ones that rely on Obj-C interop and Foundation).

The issue I'm facing is with the expected outputs of the tests versus my actual results. Basically, on Linux, it seems that the output is a mix of what's expected in a 64 bit and 32 bit systems. For one, the num_extra_inhabitants field on Linux equals that which is expected on a 32 bit system, but all the other fields match a 64 bit one.
For example, reflect_Array.swift expects the following:

// CHECK-64: Type info:
// CHECK-64: (class_instance size=24 alignment=8 stride=24 num_extra_inhabitants=0 bitwise_takable=1
// CHECK-64:   (field name=t offset=16
// CHECK-64:     (struct size=8 alignment=8 stride=8 num_extra_inhabitants=2147483647 bitwise_takable=1
// (unstable implementation details omitted)

// CHECK-32: Type info:
// CHECK-32: (class_instance size=12 alignment=4 stride=12 num_extra_inhabitants=0 bitwise_takable=1
// CHECK-32:   (field name=t offset=8
// CHECK-32:     (struct size=4 alignment=4 stride=4 num_extra_inhabitants=4096 bitwise_takable=1
// (unstable implementation details omitted)

And the output I get is:

(class_instance size=24 alignment=8 stride=24 num_extra_inhabitants=0 bitwise_takable=1
  (field name=t offset=16
    (struct size=8 alignment=8 stride=8 num_extra_inhabitants=4096 bitwise_takable=1
      (field name=_buffer offset=0
        (struct size=8 alignment=8 stride=8 num_extra_inhabitants=4096 bitwise_takable=1
          (field name=_storage offset=0
            (reference kind=strong refcounting=native)))))))

Notice that the size, alignment and stride match that of a 64 bit system. But num_extra_inhabitants matches that of a 32 bit system (4096).

Another thing that is happening: some structures which are only expected in 32 bit systems are being outputed by my implementation. For example in existential.swift, starting at line 207, we expect a reflection to existentials.MyStruct only on 32 bit systems, but my implementation outputs that as well. (EDIT: this actually doesn't happen, I got confused by the diffs)

There's also some difference in the metadata produced by Linux vs macOS. For the reflect_Array test, on macOS, the pointer to super is set, and there are 2 reserved fields + the rodata fields which are missing on Linux. Also, the class object size is different (136 on macOS and 112 on Linux, in reflect_Array). I'm not sure if this influences anything, but I thought I should mention it as well.

So, overall, this leaves me wondering if I have a bug, or if this is an expected difference between the platforms. @Adrian_Prantl suggested that we might want to split the tests cases between objc interop and non objc interop, in order to deal with this. What do you think?

2

Exciting progress!

The number of extra inhabitants actually depends to some extent on the OS. Apple platforms reserve a 4GB "page zero" on 64 bit platforms in order to help detect 32/64 bit errors. That means that every pointer less than 4GB is invalid, so we get that many extra inhabitants. (There's a separate limitation of 2G extra inhabitants for any data structure, hence the value you see here.). (Background: "Extra inhabitant" in this context just means "invalid value" -- Swift encodes enums that have exactly one payload case by using invalid values to represent the non-payload cases. This avoids needing extra memory for things like Optional<pointerType>)

But Linux only reserves a 4K page zero, so there are only 4096 invalid pointer values.

We could either duplicate the tests or try to make this particular magic value something that can be set and/or calculated otherwise. FileCheck has some support for defining constants for use in the pattern matching; I experimented with unifying the 32-bit and 64-bit cases using that but it got rather involved as there are a lot of difference. You might have better luck unifying the 64-bit cases for different platforms.

2 Likes
3

Nice! So that's where the difference comes from!

I didn't know that! I'll try defining constants before duplicating the tests.

4

@augusto2112 Really nice work! You might be able to define a lit substitution like "%num_extra_inhabitants_64bit" in test/lit.cfg, then forward that to FileCheck with a -D option.

1 Like