`FileManager.copyItem()` on Linux fails after copying the files

sebsto · January 18, 2025, 5:42pm

FileManager.copyItem() on Linux fails after copying the files.

The error is error: Error Domain=NSCocoaErrorDomain Code=513 "You don't have permission."

Should I report this on FoundationEssentials ? Or is it a know issue ?

Context (using Ubuntu 24):
The Swift Lambda Runtime archive plugin copies files using FileManager.default.copyItem

Source files are created by docker and are owned by root

ls -lah .build/release/ResourcesPackaging_
MyLambda.resources/
total 12K
drwxr-xr-x  2 root root 4.0K Jan 18 17:11 .
drwxr-xr-x 45 root root 4.0K Jan 18 17:12 ..
-rw-rw-r--  1 root root   12 Jan 18 17:11 hello.txt

After the copy operation, the destination files are copied correctly and they are owned by ubuntu

ls -alh .build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/MyLambda/ResourcesPackaging_MyLambda.resources/
total 12K
drwxr-xr-x 2 ubuntu ubuntu 4.0K Jan 18 17:12 .
drwxrwxr-x 3 ubuntu ubuntu 4.0K Jan 18 17:12 ..
-rw-rw-r-- 1 ubuntu ubuntu   12 Jan 18 17:12 hello.txt

The copy operation throws the error after the files have been copyied.

                        try FileManager.default.copyItem(
                            atPath: artifactURL.path(),
                            toPath: relocatedResourcesDirectory.path()
                        )

error: Error Domain=NSCocoaErrorDomain Code=513 "You don't have permission."

An easy workaround is to wrap try FileManager.default.copyItem in a do ... catch ... block but this might hide other legitimate errors.

ANy idea what the FileManager does after copying the files and that fail ?

Finagolfin · January 18, 2025, 6:21pm

This looks like the error I was getting on Android when the new Foundation was trying to copy over files' extended attributes, which is blocked by SELinux on most recent versions of Android. Is it likely the same in your linux runtime?

If so, maybe there is some way to query SELinux first before attempting to copy those. If that's not your problem, I suggest you compile and test a debug build of the new Foundation repo in your runtime, which is easy to do now that it is just a Swift package, and see if any of its tests fail. That is how I caught those Android issues.

sebsto · January 18, 2025, 9:08pm

Thank you for the pointers and references.

I observe this on a standard Ubuntu 24.04
SE Linux is not enabled by default.

$ getenforce
Disabled

I downloaded and run swift test on Swift foundation and indeed, one FileManager test failed :-) but it looks like it is just a matter of order of the checks.

github.com/swiftlang/swift-foundation

Tests/FoundationEssentialsTests/FileManager/FileManagerTests.swift

e9d59b606


      
                      "other_file"
                  }.test(captureDelegateCalls: true) {
                      XCTAssertTrue($0.delegateCaptures.isEmpty)
                      try $0.copyItem(atPath: "dir", toPath: "dir2")
                      XCTAssertEqual(try $0.subpathsOfDirectory(atPath: ".").sorted(), ["dir", "dir/bar", "dir/foo", "dir2", "dir2/bar", "dir2/foo", "other_file"])
                      XCTAssertEqual($0.contents(atPath: "dir/foo"), data)
                      XCTAssertEqual($0.contents(atPath: "dir2/foo"), data)
          #if os(Windows)
                      XCTAssertEqual($0.delegateCaptures.shouldCopy, [.init("dir", "dir2"), .init("dir/bar", "dir2/bar"), .init("dir/foo", "dir2/foo")])
          #else
                      XCTAssertEqual($0.delegateCaptures.shouldCopy, [.init("dir", "dir2"), .init("dir/foo", "dir2/foo"), .init("dir/bar", "dir2/bar")])
                      
                      // Specifically for non-Windows (where copying directory metadata takes a special path) double check that the metadata was copied exactly
                      XCTAssertEqual(try $0.attributesOfItem(atPath: "dir2")[.posixPermissions] as? UInt, 0o777)
          #endif
                      XCTAssertThrowsError(try $0.copyItem(atPath: "does_not_exist", toPath: "dir3")) {
                          XCTAssertEqual(($0 as? CocoaError)?.code, .fileReadNoSuchFile)
                      }
                      
                      try $0.copyItem(atPath: "dir", toPath: "other_file")
                      XCTAssertTrue($0.delegateCaptures.shouldProceedAfterCopyError.contains(.init("dir", "other_file", code: .fileWriteFileExists)))

Looking into this

sebsto · January 19, 2025, 3:13pm

Here is a minimal code to reproduce the problem

import FoundationEssentials

do {
    try FileManager.default.copyItem(
        atPath: "src",
        toPath: "dst"
    )
    print("Files copied successfully.")
} catch let error as CocoaError{
    print("Cocoa Error copying files: \(error)")
} catch {
    print("Error copying files: \(error)")
}

and to launch the test:

#!/bin/bash 

sudo mkdir src 
sudo touch src/toto 

swift main.swift

ls -al dst

This causes this error, after the file is copied.

Cocoa Error copying files: CocoaError(code: FoundationEssentials.CocoaError.Code(rawValue: 513), userInfo: ["NSUnderlyingError": AnyHashable(FoundationEssentials.POSIXError(code: Glibc.POSIXErrorCode.EPERM)), "NSFilePath": AnyHashable("dst"), "NSURL": AnyHashable(dst -- file:///home/ubuntu/swift-aws-lambda-runtime/_REPRO/)])

Finagolfin · January 19, 2025, 3:51pm

You have a couple options, depending on what that runtime allows:

Attach a debugger to the failing executable and try to step through where you think the error may be.
Build a self-contained executable against a patched swift-foundation, which dumps logging output in the likely error locations in FoundationEssentials, and try to pinpoint where that executable that is statically linked against swift-foundation fails, by examining the log.

My guess is that SELinux is turned on in the AWS runtime and it is disallowing setting extended attributes just like on Android, but could be something different for you.

sebsto · January 20, 2025, 7:32am

I filed an issue on the Foundation project.

github.com/swiftlang/swift-foundation

FileManager.copyItem() throws a permission error after having copied files (Ubuntu)

opened 07:31AM - 20 Jan 25 UTC

sebsto

When copying an entire directory where files are owned by root, `FileManager.cop…yItem()` throws a `CocoaError.fileWriteNoPermission` **after** having copied the files. It works on macOS and it is reproductible on Ubuntu. Version : Swift 6.0.3 on Ubuntu 24.04 Minimal code to reproduce ```swift import FoundationEssentials do { try FileManager.default.copyItem( atPath: "src", toPath: "dst" ) print("Files copied successfully.") } catch let error as CocoaError{ print("Cocoa Error copying files: \(error)") } catch { print("Other error copying files: \(error)") } ```` and this shell script ```sh #!/bin/bash sudo mkdir src sudo touch src/toto ls -al src swift main.swift ls -al dst ``` This produces ```txt total 8 drwxr-xr-x 2 root root 4096 Jan 20 07:24 . drwxrwxr-x 3 ubuntu ubuntu 4096 Jan 20 07:24 .. -rw-r--r-- 1 root root 0 Jan 20 07:24 toto Cocoa Error copying files: CocoaError(code: FoundationEssentials.CocoaError.Code(rawValue: 513), userInfo: ["NSURL": AnyHashable(dst -- file:///home/ubuntu/swift-aws-lambda-runtime/_REPRO/), "NSFilePath": AnyHashable("dst"), "NSUnderlyingError": AnyHashable(FoundationEssentials.POSIXError(code: Glibc.POSIXErrorCode.EPERM))]) total 8 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 20 07:24 . drwxrwxr-x 4 ubuntu ubuntu 4096 Jan 20 07:24 .. -rw-r--r-- 1 ubuntu ubuntu 0 Jan 20 07:24 toto ``` Note that the source files have perm 0o644, they are readable by non-root users. Copying them must succeed and the target files must be owned by the user running the swift script. This is the case here, but `FileManager.copyItem()` throws an error **after** the copy operation.

This is 100% reproductible on a standard Ubuntu 24.04 with SELinux disabled.

jmschonfeld · January 21, 2025, 5:06pm

Thanks for filing an issue, I'll take a look into this. I suspect that given src/ and src/toto are created by the root user (via sudo) and the swift application is run by the current user (presumably not root?) that this is indeed an issue with copying over the permissions after copying the files over themselves. I'll see if I can add some logging to figure out what's going on and see if we have an ordering issue with how we apply the attributes post-copy or if this is indeed a "correct" failure because we don't have the right permissions to copy over the attributes.

sebsto · January 22, 2025, 9:42am

Thank you @jmschonfeld for having a look.

Yes, I confirm the script is run from a regular user, ubuntu in my case.
I also agree that FileManager.copyItem() probably tries to change the permissions on the files after the copy.

However, the default behaviour when copying files owned by another user (when they are readable) is to copy them, change ownership to the user doing the copy and to not change permissions.

Try this

$ id
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),4(adm),24(cdrom),27(sudo),30(dip),105(lxd),113(docker)

$ sudo mkdir src && sudo touch src/toto 

$ ls -al src 
total 8
drwxr-xr-x 2 root   root   4096 Jan 22 09:38 .
drwxrwxr-x 3 ubuntu ubuntu 4096 Jan 22 09:38 ..
-rw-r--r-- 1 root   root      0 Jan 22 09:38 toto

$ cp -r src dst 

$ ls -al dst 
total 8
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 22 09:39 .
drwxrwxr-x 4 ubuntu ubuntu 4096 Jan 22 09:39 ..
-rw-r--r-- 1 ubuntu ubuntu    0 Jan 22 09:39 toto

The cp command doesn't fail and the files indst are owned by the user issuing the cp command.

tim1724 · January 24, 2025, 3:04am

On Linux my first step to debugging something like this would usually be to just run it using the strace utility to see what system call is failing and what arguments it's being called with.

sebsto · January 25, 2025, 8:58am

Root cause has been identified here

github.com/swiftlang/swift-foundation

FileManager.copyItem() throws a permission error after having copied files (Ubuntu)

opened 07:31AM - 20 Jan 25 UTC

sebsto

When copying an entire directory where files are owned by root, `FileManager.cop…yItem()` throws a `CocoaError.fileWriteNoPermission` **after** having copied the files. It works on macOS and it is reproductible on Ubuntu. Version : Swift 6.0.3 on Ubuntu 24.04 Minimal code to reproduce ```swift import FoundationEssentials do { try FileManager.default.copyItem( atPath: "src", toPath: "dst" ) print("Files copied successfully.") } catch let error as CocoaError{ print("Cocoa Error copying files: \(error)") } catch { print("Other error copying files: \(error)") } ```` and this shell script ```sh #!/bin/bash sudo mkdir src sudo touch src/toto ls -al src swift main.swift ls -al dst ``` This produces ```txt total 8 drwxr-xr-x 2 root root 4096 Jan 20 07:24 . drwxrwxr-x 3 ubuntu ubuntu 4096 Jan 20 07:24 .. -rw-r--r-- 1 root root 0 Jan 20 07:24 toto Cocoa Error copying files: CocoaError(code: FoundationEssentials.CocoaError.Code(rawValue: 513), userInfo: ["NSURL": AnyHashable(dst -- file:///home/ubuntu/swift-aws-lambda-runtime/_REPRO/), "NSFilePath": AnyHashable("dst"), "NSUnderlyingError": AnyHashable(FoundationEssentials.POSIXError(code: Glibc.POSIXErrorCode.EPERM))]) total 8 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 20 07:24 . drwxrwxr-x 4 ubuntu ubuntu 4096 Jan 20 07:24 .. -rw-r--r-- 1 ubuntu ubuntu 0 Jan 20 07:24 toto ``` Note that the source files have perm 0o644, they are readable by non-root users. Copying them must succeed and the target files must be owned by the user running the swift script. This is the case here, but `FileManager.copyItem()` throws an error **after** the copy operation.

gregcotten · March 6, 2025, 1:14am

@sebsto, I'm encountering this issue with swift build when a package has resources that need to be copied. Do you have any "quick fix" suggestions that don't involve patching Foundation. Some sort of Docker setting?

sebsto · March 6, 2025, 8:22am

@gregcotten Are you using Docker for some part of your build ?
Can you share the steps to reproduce ? I'm not sure how this can happen with a simple swift build where the same user is used consistently.

gregcotten · March 6, 2025, 4:03pm

I’ll see if I can make a reproducer. The docker container is writing to a mounted volume on the host, to complicate things…

The Ubuntu host (as a normal user) runs a docker command that runs a swift build (using a Rocky Linux 8 swift image) on a mounted volume. I get 513 permission errors all over the place regarding the package dependency cache but those are recoverable because the cache is “skipped”. Ultimately the fatal 513 permission error occurs when swift tries to copy some DocC documentation into a package resource bundle that I don’t even want in the first place…

sebsto · March 6, 2025, 4:12pm

I understand. You're correct. It's probably the same issue.