SwiftReflectionTest small differences between platforms

@vedantk @Adrian_Prantl @Joe_Groff @Slava_Pestov @Mike_Ash @dcci

Hi everyone!
This is a followup to Adapting SwiftReflectionTest to work on Linux.

Right now, I got SwiftReflectionTest to output metadata values for our tests, but there are 11 tests that still fail (not counting the ones that rely on Obj-C interop and Foundation).

The issue I'm facing is with the expected outputs of the tests versus my actual results. Basically, on Linux, it seems that the output is a mix of what's expected in a 64 bit and 32 bit systems. For one, the num_extra_inhabitants field on Linux equals that which is expected on a 32 bit system, but all the other fields match a 64 bit one.
For example, reflect_Array.swift expects the following:

// CHECK-64: Type info:
// CHECK-64: (class_instance size=24 alignment=8 stride=24 num_extra_inhabitants=0 bitwise_takable=1
// CHECK-64:   (field name=t offset=16
// CHECK-64:     (struct size=8 alignment=8 stride=8 num_extra_inhabitants=2147483647 bitwise_takable=1
// (unstable implementation details omitted)

// CHECK-32: Type info:
// CHECK-32: (class_instance size=12 alignment=4 stride=12 num_extra_inhabitants=0 bitwise_takable=1
// CHECK-32:   (field name=t offset=8
// CHECK-32:     (struct size=4 alignment=4 stride=4 num_extra_inhabitants=4096 bitwise_takable=1
// (unstable implementation details omitted)

And the output I get is:

(class_instance size=24 alignment=8 stride=24 num_extra_inhabitants=0 bitwise_takable=1
  (field name=t offset=16
    (struct size=8 alignment=8 stride=8 num_extra_inhabitants=4096 bitwise_takable=1
      (field name=_buffer offset=0
        (struct size=8 alignment=8 stride=8 num_extra_inhabitants=4096 bitwise_takable=1
          (field name=_storage offset=0
            (reference kind=strong refcounting=native)))))))

Notice that the size, alignment and stride match that of a 64 bit system. But num_extra_inhabitants matches that of a 32 bit system (4096).

Another thing that is happening: some structures which are only expected in 32 bit systems are being outputed by my implementation. For example in existential.swift, starting at line 207, we expect a reflection to existentials.MyStruct only on 32 bit systems, but my implementation outputs that as well. (EDIT: this actually doesn't happen, I got confused by the diffs)

There's also some difference in the metadata produced by Linux vs macOS. For the reflect_Array test, on macOS, the pointer to super is set, and there are 2 reserved fields + the rodata fields which are missing on Linux. Also, the class object size is different (136 on macOS and 112 on Linux, in reflect_Array). I'm not sure if this influences anything, but I thought I should mention it as well.

So, overall, this leaves me wondering if I have a bug, or if this is an expected difference between the platforms. @Adrian_Prantl suggested that we might want to split the tests cases between objc interop and non objc interop, in order to deal with this. What do you think?

Exciting progress!

The number of extra inhabitants actually depends to some extent on the OS. Apple platforms reserve a 4GB "page zero" on 64 bit platforms in order to help detect 32/64 bit errors. That means that every pointer less than 4GB is invalid, so we get that many extra inhabitants. (There's a separate limitation of 2G extra inhabitants for any data structure, hence the value you see here.). (Background: "Extra inhabitant" in this context just means "invalid value" -- Swift encodes enums that have exactly one payload case by using invalid values to represent the non-payload cases. This avoids needing extra memory for things like Optional<pointerType>)

But Linux only reserves a 4K page zero, so there are only 4096 invalid pointer values.

We could either duplicate the tests or try to make this particular magic value something that can be set and/or calculated otherwise. FileCheck has some support for defining constants for use in the pattern matching; I experimented with unifying the 32-bit and 64-bit cases using that but it got rather involved as there are a lot of difference. You might have better luck unifying the 64-bit cases for different platforms.

2 Likes

Nice! So that's where the difference comes from!

I didn't know that! I'll try defining constants before duplicating the tests.

@augusto2112 Really nice work! You might be able to define a lit substitution like "%num_extra_inhabitants_64bit" in test/lit.cfg, then forward that to FileCheck with a -D option.

1 Like
Terms of Service

Privacy Policy

Cookie Policy