Swift 4.0 bug in concurrent array access

I'm on macOS Sierra 10.12.6, and using Swift 4.0Tried to access the array concurrently to see whether is it synchronized or no
import Foundation let global = DispatchQueue.global() var array: [Int] = [ - Pastebin.com

  > >

   >

  >

  >

import Foundation let global = DispatchQueue.global() var array: [Int...
   > >

  >

  >

When tried to run it using the terminalThis crash happened
swift(69836,0x700002dc2000) malloc: *** error for object 0x7ffe4be237e0: pointer - Pastebin.com
  

  > >

   >

  >

  >

swift(69836,0x700002dc2000) malloc: *** error for object 0x7ffe4be237e0: po...
   > >

  >

  >

Avoided this crash using Serial Dispatch Queue
let otherDispatch = DispatchQueue(label: "another.dispatch.com",
qos: .default)
func appending(_ i: Int) {
otherDispatch.sync { array.append(i) }
}

Also tried to file this issue on bugs.swift.org, and it appears your site is down (under maintainence)
Sorry

That sounds expected to me. While Array values are thread-safe, writing to the global variable 'array' from two different threads would be a race. You can see this properly diagnosed if you enable Thread Sanitizer.

Jordan

P.S. bugs.swift.org should be back up now, if you find any other issues.

···

On Jan 8, 2018, at 11:02, ⁨‫Fadi Botros‬ ‫ via swift-users⁩ <⁨swift-users@swift.org⁩> wrote:

I'm on macOS Sierra 10.12.6, and using Swift 4.0
Tried to access the array concurrently to see whether is it synchronized or no
import Foundation let global = DispatchQueue.global() var array: [Int] = [ - Pastebin.com <https://pastebin.com/PnvyM3Qn&gt;

import Foundation let global = DispatchQueue.global() var array: [Int...
<https://pastebin.com/PnvyM3Qn&gt;

When tried to run it using the terminal
This crash happened

swift(69836,0x700002dc2000) malloc: *** error for object 0x7ffe4be237e0: pointer - Pastebin.com <https://pastebin.com/vrdKTfrP&gt;

swift(69836,0x700002dc2000) malloc: *** error for object 0x7ffe4be237e0: po...
<https://pastebin.com/vrdKTfrP&gt;

Avoided this crash using Serial Dispatch Queue

let otherDispatch = DispatchQueue(label: "another.dispatch.com",
                                  qos: .default)
func appending(_ i: Int) {
  otherDispatch.sync { array.append(i) }
}

Also tried to file this issue on bugs.swift.org, and it appears your site is down (under maintainence)

Sorry
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

1st: How to invoke the thread sanitizer?

Check out this article on developer.apple.com: https://developer.apple.com/documentation/code_diagnostics/thread_sanitizer/enabling_the_thread_sanitizer

2nd: I think making something like ConcurrentModificationException of Java should be a little better experience (if it will not make, or make a small performance penalty, also if it makes performance penalty, it would be activated only when you make a debug version or non-optimized version)

I don't exactly disagree, but my understanding is that the work to do such a thing is nearly equivalent to having thread sanitizer on all the time. But maybe there's a simpler model that we could still turn on in debug builds. Can you file a bug report requesting this now that bugs.swift.org is back up?

Thanks!
Jordan

···

On Jan 8, 2018, at 11:47, ⁨‫Fadi Botros‬ ‫⁩ <⁨botros_fadi@yahoo.com⁩> wrote:

من: Jordan Rose <jordan_rose@apple.com>
إلى: ‫Fadi Botros‬ ‫ <botros_fadi@yahoo.com>
نسخة كربونية: "swift-users@swift.org" <swift-users@swift.org>
تاريخ الإرسال: الإثنين 8 يناير، 2018‏ 9:14 م
الموضوع: Re: [swift-users] Swift 4.0 bug in concurrent array access

That sounds expected to me. While Array values are thread-safe, writing to the global variable 'array' from two different threads would be a race. You can see this properly diagnosed if you enable Thread Sanitizer.

Jordan

P.S. bugs.swift.org <Issues · apple/swift · GitHub; should be back up now, if you find any other issues.

On Jan 8, 2018, at 11:02, ⁨‫Fadi Botros‬ ‫ via swift-users⁩ <⁨swift-users@swift.org <mailto:swift-users@swift.org>⁩> wrote:

I'm on macOS Sierra 10.12.6, and using Swift 4.0
Tried to access the array concurrently to see whether is it synchronized or no
import Foundation let global = DispatchQueue.global() var array: [Int] = [ - Pastebin.com <https://pastebin.com/PnvyM3Qn&gt;

import Foundation let global = DispatchQueue.global() var array: [Int...
<https://pastebin.com/PnvyM3Qn&gt;

When tried to run it using the terminal
This crash happened

swift(69836,0x700002dc2000) malloc: *** error for object 0x7ffe4be237e0: pointer - Pastebin.com <https://pastebin.com/vrdKTfrP&gt;

swift(69836,0x700002dc2000) malloc: *** error for object 0x7ffe4be237e0: po...
<https://pastebin.com/vrdKTfrP&gt;

Avoided this crash using Serial Dispatch Queue

let otherDispatch = DispatchQueue(label: "another.dispatch.com <USA TODAY - Breaking News and Latest News Today,
                                  qos: .default)
func appending(_ i: Int) {
  otherDispatch.sync { array.append(i) }
}

Also tried to file this issue on bugs.swift.org <Issues · apple/swift · GitHub, and it appears your site is down (under maintainence)

Sorry
_______________________________________________
swift-users mailing list
swift-users@swift.org <mailto:swift-users@swift.org>
https://lists.swift.org/mailman/listinfo/swift-users

Something like ConcurrentModificationException ought to be cheaper than the thread sanitizer. The thread sanitizer works hard to detect every concurrency error. ConcurrentModificationException typically does something simple and cheap that catches errors sometimes but makes no attempt to be exhaustive.

Objective-C's fast enumeration protocol includes a mechanism like this. The collection can have a simple mutation counter, and the enumerator captures the counter's value at the start and checks that the value is unchanged as the enumeration proceeds.

···

On Jan 8, 2018, at 12:51 PM, Jordan Rose via swift-users <swift-users@swift.org> wrote:

On Jan 8, 2018, at 11:47, ⁨‫Fadi Botros‬ ‫⁩ <⁨botros_fadi@yahoo.com <mailto:botros_fadi@yahoo.com>⁩> wrote:

1st: How to invoke the thread sanitizer?

Check out this article on developer.apple.com <http://developer.apple.com/&gt;: https://developer.apple.com/documentation/code_diagnostics/thread_sanitizer/enabling_the_thread_sanitizer

2nd: I think making something like ConcurrentModificationException of Java should be a little better experience (if it will not make, or make a small performance penalty, also if it makes performance penalty, it would be activated only when you make a debug version or non-optimized version)

I don't exactly disagree, but my understanding is that the work to do such a thing is nearly equivalent to having thread sanitizer on all the time. But maybe there's a simpler model that we could still turn on in debug builds. Can you file a bug report requesting this now that bugs.swift.org <Issues · apple/swift · GitHub; is back up?

--
Greg Parker gparker@apple.com <mailto:gparker@apple.com> Runtime Wrangler

A simple mutation counter works in the single-threaded case

for i in x {
  print(i)
  if middle_of_x { x.add(y) } // bang!
}

The add while enumerating should change the mutation counter, and the implicit getting of next value for the for-loop should see the current mutation counter doesn't equal the initial cached value of the mutation counter.

In the multi-threaded case, the mutation counter needs to be thread-safe, otherwise comparing its current value to a cached value could fail from timing issues.

// thread one
for i in x { print(i) }

// thread two
x.add(y) // bang?

···

On Jan 8, 2018, at 2:59 PM, Greg Parker via swift-users <swift-users@swift.org> wrote:

On Jan 8, 2018, at 12:51 PM, Jordan Rose via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

On Jan 8, 2018, at 11:47, ⁨‫Fadi Botros‬ ‫⁩ <⁨botros_fadi@yahoo.com <mailto:botros_fadi@yahoo.com>⁩> wrote:

1st: How to invoke the thread sanitizer?

Check out this article on developer.apple.com <http://developer.apple.com/&gt;: https://developer.apple.com/documentation/code_diagnostics/thread_sanitizer/enabling_the_thread_sanitizer

2nd: I think making something like ConcurrentModificationException of Java should be a little better experience (if it will not make, or make a small performance penalty, also if it makes performance penalty, it would be activated only when you make a debug version or non-optimized version)

I don't exactly disagree, but my understanding is that the work to do such a thing is nearly equivalent to having thread sanitizer on all the time. But maybe there's a simpler model that we could still turn on in debug builds. Can you file a bug report requesting this now that bugs.swift.org <Issues · apple/swift · GitHub; is back up?

Something like ConcurrentModificationException ought to be cheaper than the thread sanitizer. The thread sanitizer works hard to detect every concurrency error. ConcurrentModificationException typically does something simple and cheap that catches errors sometimes but makes no attempt to be exhaustive.

Objective-C's fast enumeration protocol includes a mechanism like this. The collection can have a simple mutation counter, and the enumerator captures the counter's value at the start and checks that the value is unchanged as the enumeration proceeds.

--
Greg Parker gparker@apple.com <mailto:gparker@apple.com> Runtime Wrangler

_______________________________________________
swift-users mailing list
swift-users@swift.org <mailto:swift-users@swift.org>
https://lists.swift.org/mailman/listinfo/swift-users

Does ObjC synchronize the mutation counter? My understanding is that that counter was used to detect mutation while iteration on a single thread, not across threads.

-Chris

···

On Jan 8, 2018, at 2:59 PM, Greg Parker via swift-users <swift-users@swift.org> wrote:

On Jan 8, 2018, at 12:51 PM, Jordan Rose via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

On Jan 8, 2018, at 11:47, ⁨‫Fadi Botros‬ ‫⁩ <⁨botros_fadi@yahoo.com <mailto:botros_fadi@yahoo.com>⁩> wrote:

1st: How to invoke the thread sanitizer?

Check out this article on developer.apple.com <http://developer.apple.com/&gt;: https://developer.apple.com/documentation/code_diagnostics/thread_sanitizer/enabling_the_thread_sanitizer

2nd: I think making something like ConcurrentModificationException of Java should be a little better experience (if it will not make, or make a small performance penalty, also if it makes performance penalty, it would be activated only when you make a debug version or non-optimized version)

I don't exactly disagree, but my understanding is that the work to do such a thing is nearly equivalent to having thread sanitizer on all the time. But maybe there's a simpler model that we could still turn on in debug builds. Can you file a bug report requesting this now that bugs.swift.org <Issues · apple/swift · GitHub; is back up?

Something like ConcurrentModificationException ought to be cheaper than the thread sanitizer. The thread sanitizer works hard to detect every concurrency error. ConcurrentModificationException typically does something simple and cheap that catches errors sometimes but makes no attempt to be exhaustive.

Objective-C's fast enumeration protocol includes a mechanism like this. The collection can have a simple mutation counter, and the enumerator captures the counter's value at the start and checks that the value is unchanged as the enumeration proceeds.

The mutation counter is not synchronized; this is safe for correct code. An un-synchronized counter can still sometimes catch mutations from other threads.

···

On Jan 8, 2018, at 5:20 PM, Chris Lattner <clattner@nondot.org> wrote:

On Jan 8, 2018, at 2:59 PM, Greg Parker via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

On Jan 8, 2018, at 12:51 PM, Jordan Rose via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

On Jan 8, 2018, at 11:47, ⁨‫Fadi Botros‬ ‫⁩ <⁨botros_fadi@yahoo.com <mailto:botros_fadi@yahoo.com>⁩> wrote:

1st: How to invoke the thread sanitizer?

Check out this article on developer.apple.com <http://developer.apple.com/&gt;: https://developer.apple.com/documentation/code_diagnostics/thread_sanitizer/enabling_the_thread_sanitizer

2nd: I think making something like ConcurrentModificationException of Java should be a little better experience (if it will not make, or make a small performance penalty, also if it makes performance penalty, it would be activated only when you make a debug version or non-optimized version)

I don't exactly disagree, but my understanding is that the work to do such a thing is nearly equivalent to having thread sanitizer on all the time. But maybe there's a simpler model that we could still turn on in debug builds. Can you file a bug report requesting this now that bugs.swift.org <Issues · apple/swift · GitHub; is back up?

Something like ConcurrentModificationException ought to be cheaper than the thread sanitizer. The thread sanitizer works hard to detect every concurrency error. ConcurrentModificationException typically does something simple and cheap that catches errors sometimes but makes no attempt to be exhaustive.

Objective-C's fast enumeration protocol includes a mechanism like this. The collection can have a simple mutation counter, and the enumerator captures the counter's value at the start and checks that the value is unchanged as the enumeration proceeds.

Does ObjC synchronize the mutation counter? My understanding is that that counter was used to detect mutation while iteration on a single thread, not across threads.

--
Greg Parker gparker@apple.com <mailto:gparker@apple.com> Runtime Wrangler

What about double checking?

Something like that
if counter != 0 { throw exc }
counter += 1if counter != 1 { throw exc }counter += 1

I think the performance penalty is just a check and an increment
It could be disabled after optimization or in release version to increase performance

      من: Greg Parker <gparker@apple.com>
إلى: Chris Lattner <clattner@nondot.org>
نسخة كربونية: Jordan Rose <jordan_rose@apple.com>; swift-users <swift-users@swift.org>; ‫Fadi Botros‬ ‫ <botros_fadi@yahoo.com>
تاريخ الإرسال: الثلاثاء 9 يناير، 2018‏ 3:47 ص
الموضوع: Re: [swift-users] بخصوص: Swift 4.0 bug in concurrent array access

···

On Jan 8, 2018, at 5:20 PM, Chris Lattner <clattner@nondot.org> wrote:

On Jan 8, 2018, at 2:59 PM, Greg Parker via swift-users <swift-users@swift.org> wrote:

On Jan 8, 2018, at 12:51 PM, Jordan Rose via swift-users <swift-users@swift.org> wrote:

On Jan 8, 2018, at 11:47, ⁨‫Fadi Botros‬ ‫⁩ <⁨botros_fadi@yahoo.com⁩> wrote:
1st: How to invoke the thread sanitizer?

Check out this article on developer.apple.com: https://developer.apple.com/documentation/code_diagnostics/thread_sanitizer/enabling_the_thread_sanitizer

2nd: I think making something like ConcurrentModificationException of Java should be a little better experience (if it will not make, or make a small performance penalty, also if it makes performance penalty, it would be activated only when you make a debug version or non-optimized version)

I don't exactly disagree, but my understanding is that the work to do such a thing is nearly equivalent to having thread sanitizer on all the time. But maybe there's a simpler model that we could still turn on in debug builds. Can you file a bug report requesting this now that bugs.swift.org is back up?

Something like ConcurrentModificationException ought to be cheaper than the thread sanitizer. The thread sanitizer works hard to detect every concurrency error. ConcurrentModificationException typically does something simple and cheap that catches errors sometimes but makes no attempt to be exhaustive.
Objective-C's fast enumeration protocol includes a mechanism like this. The collection can have a simple mutation counter, and the enumerator captures the counter's value at the start and checks that the value is unchanged as the enumeration proceeds.

Does ObjC synchronize the mutation counter? My understanding is that that counter was used to detect mutation while iteration on a single thread, not across threads.

The mutation counter is not synchronized; this is safe for correct code. An un-synchronized counter can still sometimes catch mutations from other threads.

-- Greg Parker gparker@apple.com Runtime Wrangler

It's up to the programmer-user to impose thread-safety around an array or other data structure. The compiler doesn't know if you want to allow multiple threads to write and read multiple array members, or prevent all but one thread to read and write multiple array members, or allow multiple readers when there are no writers, etc. (And we have queues precisely for these scenarios.)

The mutation counter detects if modifications happen while iterating over a collection, modifications like adding or removing members. When this happens, it's probably a programmer error. It's a bonus if this can be detected in a multiple thread scenario.

···

--
C. Keith Ray

* What Every Programmer Needs To… by C. Keith Ray [PDF/iPad/Kindle] <- buy my book?
* http://www.thirdfoundationsw.com/keith_ray_resume_2014_long.pdf
* http://agilesolutionspace.blogspot.com/

On Jan 8, 2018, at 6:25 PM, ‫Fadi Botros‬ ‫ via swift-users <swift-users@swift.org> wrote:

What about double checking?

Something like that

if counter != 0 { throw exc }
counter += 1
if counter != 1 { throw exc }
counter += 1

I think the performance penalty is just a check and an increment
It could be disabled after optimization or in release version to increase performance

من: Greg Parker <gparker@apple.com>
إلى: Chris Lattner <clattner@nondot.org>
نسخة كربونية: Jordan Rose <jordan_rose@apple.com>; swift-users <swift-users@swift.org>; ‫Fadi Botros‬ ‫ <botros_fadi@yahoo.com>
تاريخ الإرسال: الثلاثاء 9 يناير، 2018‏ 3:47 ص
الموضوع: Re: [swift-users] بخصوص: Swift 4.0 bug in concurrent array access

On Jan 8, 2018, at 5:20 PM, Chris Lattner <clattner@nondot.org> wrote:

On Jan 8, 2018, at 2:59 PM, Greg Parker via swift-users <swift-users@swift.org> wrote:

On Jan 8, 2018, at 12:51 PM, Jordan Rose via swift-users <swift-users@swift.org> wrote:

On Jan 8, 2018, at 11:47, ⁨‫Fadi Botros‬ ‫⁩ <⁨botros_fadi@yahoo.com⁩> wrote:

1st: How to invoke the thread sanitizer?

Check out this article on developer.apple.com: https://developer.apple.com/documentation/code_diagnostics/thread_sanitizer/enabling_the_thread_sanitizer

2nd: I think making something like ConcurrentModificationException of Java should be a little better experience (if it will not make, or make a small performance penalty, also if it makes performance penalty, it would be activated only when you make a debug version or non-optimized version)

I don't exactly disagree, but my understanding is that the work to do such a thing is nearly equivalent to having thread sanitizer on all the time. But maybe there's a simpler model that we could still turn on in debug builds. Can you file a bug report requesting this now that bugs.swift.org is back up?

Something like ConcurrentModificationException ought to be cheaper than the thread sanitizer. The thread sanitizer works hard to detect every concurrency error. ConcurrentModificationException typically does something simple and cheap that catches errors sometimes but makes no attempt to be exhaustive.

Objective-C's fast enumeration protocol includes a mechanism like this. The collection can have a simple mutation counter, and the enumerator captures the counter's value at the start and checks that the value is unchanged as the enumeration proceeds.

Does ObjC synchronize the mutation counter? My understanding is that that counter was used to detect mutation while iteration on a single thread, not across threads.

The mutation counter is not synchronized; this is safe for correct code. An un-synchronized counter can still sometimes catch mutations from other threads.

--
Greg Parker
gparker@apple.com Runtime Wrangler

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users