How to fast fill a buffer

I did:

let uint8Array: [UInt8] = []
uint8Array.reserveCapacity(moreThanEnough)	
then a few 100 000 times:
	let shortAsciiString = ... 
	uint8Array.append(contentsOf: shortAsciiString.utf8.reversed())
finally:
let result = String(bytes: uint8Array.reversed(), encoding: .utf8) ?? "Error"	

Filling the array was reasonably fast.
But the reversed() at the end really killed the performance.

So I tried:

a few 100 000 times:
	let shortAsciiString = ... 
	uint8Array.insert(contentsOf: shortAsciiString.utf8, at: 0)
finally:
let result = String(bytes: uint8Array, encoding: .utf8) ?? "Error"	

Creating the result was now really fast, but filling the array was abysmally slow. Clearly arrays prefer to get appended.

Now I would like to try something like:

let buffer = .. something with Unsafe .. capacity: moreThanEnough
defer { buffer.deallocate() }
var bufPtr = end of buffer
bufPtr.pointee = 0	//	cStrings need a final zero
bufPtr -= 1
a few 100 000 times:
	let shortAsciiString = ... 
	bufPtr -= shortAsciiString.count
	copy shortAsciiString.utf8 to bufPtr
finally:
let result = String(cString: buffer)

But I am absolutely clueless, how to make this work.
Any help greatly appreciated. Or any other ideas, how make this perform.

Gerriet.

1 Like

What are you actually trying to do? Where are your shortAsciiStrings coming from? What are you doing with the final result of this operation?

1 Like

Well, my question is really: how does one in Swift do things which are quite easy in C (admittedly also easy to get wrong), like:

char * buffer = malloc(...)
char *bufPtr = buffer + 12
*bufPtr = 32
memcpy(bufPtr, source, sizeOfSource)

etc.

But not to shy away from answering your question:

My current problem arises while converting a huge integer (way more than 64 bits) into decimal.

Which leads me to another question:

Is there an official BigInteger in Swift?
If not, has anybody compared the different BigInteger packages, which can be found in the internet, to speed, bugginess, features?

Kind regards,

Gerriet.

Well, philosophically, C makes a design choice to not just expose and encourage the use of unsafe pointers but actually to effectively force you to use them all the time while providing no safeguards against mis-using them. That choice means that achieving simple things with pointers is often straightforward in C, but it's also inextricably bound up with it being, as you say, easy to get wrong in a way we didn't want to bring over to Swift. A lot of the extra ceremony around getting a pointer in the first place in Swift is intended to force, or at least encourage, safer patterns for pointer use.

4 Likes

FWIW, a perfectly valid literal translation of this into Swift is straightforward:

var buffer = UnsafeMutableBufferPointer<CChar>.allocate(capacity: ...)
var bufPtr = buffer.baseAddress! + 12
bufPtr.pointee = 32
bufPtr.assign(from: source, count: sizeOfSource)

This is, of course, nearly as unsafe as the C implementation you started with.

1 Like

Works perfectly.
But to copy Strings, I have to use this code:

let string = ...
let array = Array(string.utf8)
bufPtr.assign(from: array, count: array.count)

Can this intermediate step of creating an Array be avoided?
if I try to, compiler complaints:
"Cannot convert value of type 'String.UTF8View' to expected argument type 'UnsafePointer’"

Anyway, thanks a lot for your help!

Gerriet.

UTF8View is a Sequence, so you can call withContiguousStorageIfAvailable(_:) on it to get a pointer to the underlying buffer.

2 Likes

I tried this:

let capacity = 22
let buffer = UnsafeMutableBufferPointer<UInt8>.allocate(capacity: capacity) 
defer { buffer.deallocate() }

guard let bufferBase = buffer.baseAddress else { return }
var bufPtr = bufferBase + 5

let string = "หมี"
string.utf8.withContiguousStorageIfAvailable
{	some in	 
		
	print("some \(type(of: some))”)	//	UnsafeBufferPointer<UInt8>
	bufPtr.assign(from: some, count: array.count)	← problem with “some"		
}

Xcode tells me:
"Cannot convert value of type 'UnsafeBufferPointer<String.UTF8View.Element>' (aka 'UnsafeBufferPointer') to expected argument type 'UnsafePointer’”

Gerriet.

Shouldn't you be passing in some.baseAddress!?

I think this is just:

UnsafeMutableBufferPointer(rebasing: buffer[5...]).initialize(from: string.utf8)

It's unfortunate that the init(rebasing:) call seems to be necessary.

1 Like

@guillaume points out that the recently-accepted SE-0370 adds methods like initialize to Slice<UnsafeMutableBufferPointer>, so in Swift 5.8 (hopefully) you will be able to do:

let endIndex = buffer[5...].initialize(fromContentsOf: string.utf8)
6 Likes