"For" loop iterator and "enum"

samding · October 24, 2017, 3:32pm

Hi,

It is found that "for" loop iterate and "enum" does not work on s390x.
Here is a sample code:

      enum IntKey : Int {
          case a = 3
          case b = 4 // Implicitly 4
          case c = 1
      }
      for ( a1, a2) in [(IntKey.a, 1)] {
         print ("+++++ test \n")
      }

The for loop inifinitly iterates on s390x. Note that it works fine if
replacing the enum with normal tuple in the above code.
When debugging this code, after one iteration, the logic gets a "nil" in "
IndexingIterator.next()" as follows:

      ....
       frame #0: 0x000003fffd5b4088 libswiftCore.so`IndexingIterator.next
   (self=0x000003fffffff460) at Collection.swift:411
      408 @inline(__always)
      409 public mutating func next() -> Elements.Element? {
      410 if _position == _elements.endIndex {
   -> 411 return nil
      412 }
      413 let element = _elements[_position]
   ..
Then comes "Enum.cpp:211" (swift::swift_storeEnumTagSinglePayload) and no
difference on both s390x and x86_64.
However, It continues goes into the loop body on s390x, and terminates on
x86_64, respectively.

Have anyones know well this implementation and give us a hint for the
problems?

Thanks,

Sam Ding,
Linux on z Systems Open Source Ecosystem
IBM Toronto Lab,
email: samding@ca.ibm.com
phone: (905)413-2947

Joe_Groff · October 27, 2017, 9:33pm

At a binary level, Swift is going to represent the result of the next() operation as an value of the element type wrapped in an Optional, `(IntKey, Int)?`, and the compiler is going to recognize that it can represent the `nil` case by using unused representations of IntKey. Our code for doing enum layout was originally written in a very little-endian-centric way, and if s390x is big-endian, there's a possibility that the code in IRGen or the runtime that encodes the `nil` result is not correct, or that the runtime and IRGen do not agree on the representation of `nil`. That could lead to the infinite loop you're seeing.

-Joe

···

On Oct 24, 2017, at 8:32 AM, Sam Ding via swift-dev <swift-dev@swift.org> wrote:

Hi,

It is found that "for" loop iterate and "enum" does not work on s390x.
Here is a sample code:
enum IntKey : Int {
case a = 3
case b = 4 // Implicitly 4
case c = 1
}
for ( a1, a2) in [(IntKey.a, 1)] {
print ("+++++ test \n")
}

The for loop inifinitly iterates on s390x. Note that it works fine if replacing the enum with normal tuple in the above code.
When debugging this code, after one iteration, the logic gets a "nil" in "IndexingIterator.next()" as follows:
....
frame #0: 0x000003fffd5b4088 libswiftCore.so`IndexingIterator.next(self=0x000003fffffff460) at Collection.swift:411
408 @inline(__always)
409 public mutating func next() -> Elements.Element? {
410 if _position == _elements.endIndex {
-> 411 return nil
412 }
413 let element = _elements[_position]
..
Then comes "Enum.cpp:211" (swift::swift_storeEnumTagSinglePayload) and no difference on both s390x and x86_64.
However, It continues goes into the loop body on s390x, and terminates on x86_64, respectively.

Have anyones know well this implementation and give us a hint for the problems?

Thanks,

samding · April 4, 2018, 5:05pm

@Joe_Groff

Could you please let me know where /what IRGen or runtime code that encodes the nil?

Thanks,

Sam

Joe_Groff · April 4, 2018, 5:26pm

In IRGen, the representations are determined in SinglePayloadEnumImplStrategy in GenEnum.cpp. In the runtime, it's handled by swift_storeEnumTagSinglePayload and swift_getEnumCaseSinglePayload.

samding · April 4, 2018, 6:16pm

Thank @Joe_Groff

Just found that the example swift code above works if it is compiled with -O, but infinite looping without -O on s390x platform. -O is optimization option, why does it make difference on the behavior? In the compiler swift or cpp code, is there any difference by -O?

By the way, the same happens on swift both v4.0.3 and v4.1 releases.

Thanks,

Joe_Groff · April 4, 2018, 6:22pm

I suspect that that indicates an inconsistency between the IRGen and runtime representations used for nil.

samding · April 4, 2018, 7:34pm

Thank @Joe_Groff

When I debugging the example code with debugger (lldb or gdb), it does not stop at stdlib/public/runtime/Enum.cpp:102 (swift_getEnumCaseSinglePayload),
how the runtime code is called?
Is there any way to debug these code (as well as IRGen code)?

Thanks,

samding · April 5, 2018, 7:36pm

@Joe_Groff Do you know what functions in class SinglePayloadEnumImplStrategy are related to the nil representations?
Checking the runtime swift_storeEnumTagSinglePayload and swift_getEnumCaseSinglePayload, both have special BIG_ENDIAN process, but none inside of SinglePayloadEnumImplStrategy

Thanks,

Joe_Groff · April 5, 2018, 8:34pm

You may want to look at the code for how we form extra inhabitant bit patterns in the various getFixedExtraInhabitantValue implementations in IRGen to make sure they make sense for a big-endian platforms, as well as the code in the EnumPayload struct that turns those bit patterns into integer LLVM values.

samding · April 6, 2018, 5:31pm

@Joe_Groff
On v4.1 even for in without enum becomes infinite loop on s390x, see issue #11657 ( Infinite loop issue of a "for x in <list>" on big_endian platform),
The same for in example in #11657 is working on v4.0.3 release.
What is the changes from v4.0.3 to v4.1 related to this part?

By the way, more information:

if I define a optional Int and check if it is == nil, it works on the both v4.0.3 and v4.1

    var ii:Int? = 10
    if  (ii == nil)   {
       print("+++ 1")
    }
    ii = nil
   if  (ii == nil)   {
      print("+++ 2")
   }

In the #11657 example, if change strings to optional explicitly, the loops works correctly.

Does this mean no problem on encode of nil on s390x?

Thanks

rishi · April 10, 2018, 7:28pm

Hi Joe - can you help me understand what would EnumPayload struct look for "var intarray = [11,12]"?

I am particularly interested in knowing for this declaration what would be corresponding :

Payload
Tagbits
ExtraInhabitantValue
Optional struct for Iterator.next

I have enabled breakpoints in getEnumTagSinglePayloadImpl and storeEnumTagSinglePayloadImpl (EnumImpl.h) but they are never hit.

How do you guys normally debug compiler runtime code in lldb? One way I can debug it is by "lldb myswiftcodeexe" and then set Swift code breakpoint and then "si" at instruction level hoping it may take me to runtime code but it is very tedious.

Thanks.

Joe_Groff · April 10, 2018, 7:43pm

Int doesn't have any extra inhabitants, so when iteration forms an Optional<Int> it should do so by adding a tag bit, so you'd end up with a nine-byte value with zero in the bottom word and one in the trailing tag bit. In big endian, I'd expect to see these values:

value	payload	tag
.some(0x1122_3344_5566_7788)	11 22 33 44 • 55 66 77 88	• 00
.none	00 00 00 00 • 00 00 00 00	• 01

For the tuple (IntKey, Int), I would expect us to use one byte for the IntKey value, and pad to pointer alignment for the following Int value. Since IntKey only has three valid values, we ought to use the byte value 3 as an extra inhabitant for the representation of Optional<(IntKey, Int)>, like this:

value	.0	padding	.1
.some((.a, 0x1122_3344_5566_7788))	00	00 00 00 • 00 00 00 00	• 11 22 33 44 • 55 66 77 88
.some((.c, 0x1122_3344_5566_7788))	02	00 00 00 • 00 00 00 00	• 11 22 33 44 • 55 66 77 88
.none	03	00 00 00 • 00 00 00 00	• 00 00 00 00 • 00 00 00 00

Setting a breakpoint on a runtime entry point normally works for me. @Arnold recently changed the codegen for single-payload enums in unspecialized contexts, so maybe we don't use those runtime entry points anymore.

Arnold · April 10, 2018, 7:56pm

Unspecialized single pay load enums use the value witnesses GetEnumTagSinglePayload/StoreEnumTagSinglePayload of the payload.

This code is generated by the compiler using the TypeInfo functions getEnumTagSinglePayload/storeEnumTagSinglePayload.

Or if the value witness is defined in the runtime, ultimately getEnumTagSinglePayloadImpl/storeEnumTagSinglePayloadImpl (this function is used in several places in the runtime that define the value witness).

Joe_Groff · April 10, 2018, 7:59pm

Thanks @Arnold! In that case, we'd want to check the runtime implementation of those value witnesses for tuple type metadata in the runtime.

rishi · April 10, 2018, 8:08pm

Thanks so much - this really helps!

Perhaps another fundamental question - "iteration forms an Optional " and " tuple (IntKey, Int)" -> Where exactly in compiler code this is happening? IRGen phase? I must be missing it.

samding · April 10, 2018, 8:10pm

@Joe_Groff
yes, we found the same on x86_64 when an array element is Int.
.none 00 00 00 00 • 00 00 00 00 • 01
But on Z(s390x) it is
.none 00 00 00 00 • 00 00 00 00 • 00
so it makes infinite loop. Want to know what compiler code writing the tag.
Is it the GenEnum.cpp:getNoPayloadCaseValue?
Thanks,

Joe_Groff · April 10, 2018, 8:12pm

The expansion of for into an iterator loop happens during type checking. for x in y { ... } is basically sugar for var iterator = y.makeIterator(); while let x = iterator.next() { ... }. If the optimizer is not run, the call to next() may be a generic call that uses runtime functions to form Optional values.

Joe_Groff · April 10, 2018, 8:17pm

Yeah, in code where the compiler statically knows what type it's constructing, I would expect it to use the bit pattern returned by getNoPayloadCaseValue. For little-endian platforms we treat the APInts in this code as bit vectors, where the LSB of the APInt is the LSB of the first byte in memory and subsequent bits in the APInt correspond to the following bits in increasing significance and byte address. That's not really a natural representation for a big-endian platform, maybe. In unspecialized code where we're working with something like Optional<T> for unknown T, we would use the value witness functions for T to get the no-payload bitmask, as Arnold noted. If the IRGen and runtime implementations don't agree on bit patterns, or they're inconsistent with the bit patterns switch codegen expects, then you might see this problem. It sounds like the problem may be in either the IRGen or runtime code generating these bit patterns.

rishi · April 10, 2018, 9:44pm

Thanks ... In my case Optimizer is not run so it likely uses runtime functions to form Optional values. I suspect this Optional formation happens in some cpp code (EnumImpl.h/Enum.cpp?) and would be a good place to start.

rishi · April 13, 2018, 6:42pm

@Arnold I assume "Unspecialized" referes to known data types (Int/Strings..).

For array in question "var intarray = [11,12]" - I notice that swift::swift_initEnumValueWitnessTableSinglePayload is called during runtime. I believe this populates value witness tables. However, getEnumTagSinglePayloadImpl/storeEnumTagSinglePayloadImpl methods never get called. I would have thought (from your second comment) that these would be called to get/store payloads since they get defined in runtime by initEnumValueWitness call.

It is getting really tricky to nail down where this Tag byte for none is being set. You had indicated that the code for Unspecialized single pay load enums are generated by the compiler using the TypeInfo functions - get/storeEnumTagSinglePayload -> I think that code is found in ..\swift\lib\IRGen\GenType.cpp. However, I am unable to spot where this tag bit is generated with value of "1". Any pointers will greatly help.

Joe had indicated if optimzer is not run then call to Iterator.next() may be a generic call that uses runtime to form Optional values. To catch where this happens is proving difficult in runtime. I would have expected this to happen somewhere in Metadata.cpp or Enum.cpp but I can only catch it if I specifically define Optionals like so "var intarray:[Int?] = [11,12]". It would be nice to know how these values are formed at runtime.

Thanks.