The lower four bytes (the right half of the digits) are clearly not all zero, so the memory isn't aligned to 1 << 32.
I understand that there is and must be an upper limit for the alignment, and that it probably shouldn't trap (like it does for a negative alignment) if the result isn't aligned as requested.
But there is nothing telling me that the resulting alignment may differ from the one I specified, Here is the documentation:
/// Allocates uninitialized memory with the specified size and alignment.
///
/// You are in charge of managing the allocated memory. Be sure to deallocate
/// any memory that you manually allocate.
///
/// The allocated memory is not bound to any specific type and must be bound
/// before performing any typed operations. If you are using the memory for
/// a specific type, allocate memory using the
/// `UnsafeMutablePointer.allocate(capacity:)` static method instead.
///
/// - Parameters:
/// - byteCount: The number of bytes to allocate. `byteCount` must not be negative.
/// - alignment: The alignment of the new region of allocated memory, in
/// bytes.
/// - Returns: A pointer to a newly allocated region of memory. The memory is
/// allocated, but not initialized.
public static func allocate(byteCount: Int, alignment: Int) -> UnsafeMutableRawPointer
Also, is there a way to (programmatically) check what alignments are actually possible (given the current hardware)?
UnsafeMutableRawPointer.allocate(byteCount:alignment) ends up calling posix_memalign(3), which doesn't seem to like 1 << 32. It does everything up 1 << 31, though.
Just to clarify, the above example program compiles (on my machine) and does not crash or anything. (It's only meant as an example to show that the resulting (successfully allocated) memory isn't necessarily always aligned as requested.)
But I did some experiments and they show that any alignment within the closed range 0 ... (1 << 31) seems to be respected by UnsafeMutableRawPointer.allocate(byteCount: alignment: ).
The problem is that there is nothing (apart from my personal experiments on my particular machine) telling me about this, ie there is nothing that tells me that the resulting memory may have an alignment that is different from the one I requested, and there is nothing telling me what given alignments will be respected or not.
The bug fix was actually to start using posix_memalign.
See AlignedAlloc
int res = posix_memalign(&r, align, size);
If posix_memalign has a limitation, we may need an assert.
This was the actual fix:
commit 7239dd61b90b52644cc71bf26e74770e129c6639
Author: Mike Ash <mikeash@apple.com>
Date: Fri Feb 2 14:13:34 2018
[Runtime] Fix swift_slowAlloc to respect its alignMask parameter.
Instead of calling malloc, call AlignedAlloc. That calls posix_memalign on platforms where it's available. The man page cautions to use it judiciously, but Apple OSes and Linux implement it to call through to malloc when the alignment is suitable. Presumably/hopefully other OSes do the same.
So this confirms that 1 << 32 is the first value which doesn't work. The man page doesn't seem to mention this limitation, so I'm thinking it might be a at minimum a documentation bug:
POSIX_MEMALIGN(3) BSD Library Functions Manual POSIX_MEMALIGN(3)
NAME
posix_memalign -- aligned memory allocation
SYNOPSIS
#include <stdlib.h>
int
posix_memalign(void **memptr, size_t alignment, size_t size);
DESCRIPTION
The posix_memalign() function allocates size bytes of memory such that
the allocation's base address is an exact multiple of alignment, and
returns the allocation in the value pointed to by memptr.
The requested alignment must be a power of 2 at least as large as
sizeof(void *).
Memory that is allocated via posix_memalign() can be used as an argument
in subsequent calls to realloc(3), reallocf(3), and free(3). (Note how-
ever, that the allocation returned by realloc(3) or reallocf(3) is not
guaranteed to preserve the original alignment).
NOTES
posix_memalign() should be used judiciously as the algorithm that real-
izes the alignment constraint can incur significant memory overhead.
RETURN VALUES
The posix_memalign() function returns the value 0 if successful; other-
wise it returns an error value.
ERRORS
The posix_memalign() function will fail if:
[EINVAL] The alignment parameter is not a power of 2 at least
as large as sizeof(void *).
[ENOMEM] Memory allocation error.
SEE ALSO
free(3), malloc(3), realloc(3), reallocf(3), valloc(3),
malloc_zone_memalign(3)
STANDARDS
The posix_memalign() function conforms to IEEE Std 1003.1-2001
(``POSIX.1'').
BSD April 9, 2008 BSD
I think it's a proper bug. It looks like the POSIX spec doesn't allow any limit on alignment, other than explicitly failing with EINVAL or ENOMEM. I'm not sure what, if any, measures Swift ought to take to work around a lower-level bug like this.
FWIW, I just tried my program in a virtual machine and it appears that Linux does this too. I'm not sure if this is a deviation from POSIX or it's just that posix_memalign just gives up after 1 << 32.
I agree, also Swift's current Pointer API documentation doesn't mention anything at all about what happens if an allocation fails. Again, the only way for me to find out is by experimentation:
// I'd like to allocate 1 PiB, please:
let byteCount = 1024 * 1024 * 1024 * 1024 * 1024
let p = UnsafeMutableRawPointer.allocate(byteCount: byteCount,
alignment: 64)
p.deallocate()
Running that program results in:
PointerExperiment(14449,0x100a29340) malloc: *** mach_vm_map(size=1125899906842624) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
(lldb)
This is a macOS Command Line App, and I guess it might be handled differently in a Cocoa(Touch) App.
Makes me wonder what is the recommended cross platform way to check (programmatically, at runtime) whether an allocation of a certain number of bytes is possible or not? For example, my app might prefer to allocate 1 GB if possible, but be fine with 128 MB otherwise.
Even calling malloc won't necessarily work, because malloc is usually lazy. You might be able to allocate a huge amount of memory, then die when you try to use too much of it.
If you can throw data away, I think your best bet is to allocate memory with an API that tells the kernel it can be thrown away, such as vm_allocate and purgeable memory on Apple platforms, or ASHMEM on Android), or register for memory pressure notifications and manually free memory before the OS comes after you. Unfortunately, neither approach is fun if your goal is cross platform compatibility, since the APIs are platform-specific and may not even exist on some platforms.
Oh! Well then, purgeable memory, or memory pressure notifications, or a higher-level API that uses those (like NSPurgeableData or NSCache) is the way to go if you can fit into what they provide.