FileManager enumeration problems

I'm not sure if this is the correct forum for this question, but it's a problem with FileManager used in a swift library opened in xcode (by opening the package file itself, rather than as a xcode framework) . I have an absolute path, and although the actual string describing it is irrelevant, the path does exist. If I change directory into the path in a terminal and do a find, I get the following:

% cd path
% find .
.
./Executables
./Executables/catelogdb
./Executables/catelogdb/Package.swift
./Executables/bpformat
./Executables/bpformat/Package.swift
./Libraries
./Libraries/Core
./Libraries/Core/Package.swift
./Libraries/DbCatelog
./Libraries/DbCatelog/Package.swift
./Libraries/Path
./Libraries/Path/Package.swift
./Libraries/Log
./Libraries/Log/Package.swift
./Libraries/Node
./Libraries/Node/Package.swift
./Libraries/Args
./Libraries/Args/Package.swift

However, if I run the following test code as a test for the library, I get an empty response:

    let fm = FileManager.default
    let url = URL(fileURLWithPath: path)
    if let enumerator = fm.enumerator(at: url, includingPropertiesForKeys: nil) {
        while let suburl = enumerator.nextObject() {
            if suburl is URL {
                print((suburl as! URL).path)
            }
        }
    }

No files show up in the enumeration. However, if I put a sleep(4) immediately after the FileManager.default, then all the files and directories show up. It's almost as if the instance of fm returns immediately, but whatever is going on in the background has not finished.

Has anyone seen this problem before? Am I misunderstanding something, or is there a work around? Thanks in advance.

However, if I put a sleep(4) immediately after the
FileManager.default, then all the files and directories show up.

What platform are you running this on?

Share and Enjoy

Quinn “The Eskimo!” @ DTS @ Apple

What platform are you running this on?

It is an iMac running 10.15.2 and xcode is 11.3. The iMac has a 256G SSD with APFS formatting.

A few additional facts:

  1. Rebooted iMac.
  2. Opened library package file in xcode.
  3. Ran test - succeeded
  4. Immediately ran test again - failed

Tried same thing on a laptop running the same software: got same results.

Well, that’s weird. Next questions:

  • Is nothing printed beacuse the if statement fails? Or because the first call to nextObject always returns nil?

  • If you enumerate the top level of the directory using contentsOfDirectory(at:includingPropertiesForKeys:options:), do you get back any results?

Share and Enjoy

Quinn “The Eskimo!” @ DTS @ Apple

I added an "else" to the "if" which printed out a message for the case where the enumerator returns something other than a URL, and nothing is printed, so the enumerator returns null right away. Normally, I write the while statement as "while let suburl = enumerator.nextObject() as? URL", but I wanted to make sure the enumerator didn't return something else for this test.

Then I added a block that does the contentsOfDirectory(at: url, includingPropertiesForKeys: nil). Call the original block the "enumerator block" and this new block the "contents block".

If I put the contents block first, right after the fm = FileManager.default line, it throws an error saying the path couldn't be opened, but the enumerator block succeeds. If I put the contents block second, after the enumerator block, the enumerator block shows nothing and the contents block succeeds. All this is without the sleep of course, but it does seem consistent with that case. Pretty strange.

Is your macOS app sandboxed? That would prevent it from reading outside of its own container by default.

Well, I'm not sure exactly what that means in this case. I've got a swift library package that I had created before xcode enabled opening
the package file as package. It's just a flat library to be used in a command line executable, not in an app. Moreover, all this testing has been done through the xcode test procedures, and they can yield the previously noted random results, something that might not be expected in some kind of sandbox type of situation.

However, you're comment motivated me to do a bit more experimentation, so I went back to the command line and did a "swift test", where things are built in the local .build directory, and lo and behold, all the tests ran normally. This seems to be an xcode problem rather than a swift foundation problem, but it does make it difficult to use xcode in a package development environment.

This seems to be an xcode problem rather than a swift foundation problem

If you want to dig deeper into this, run fs_usage while running the test to see which file system call is actually returning the error. See the fs_usage man page for more on that tool.

Share and Enjoy

Quinn “The Eskimo!” @ DTS @ Apple

[quote="eskimo, post:9, topic:31682, full:true"]

If you want to dig deeper into this, run fs_usage while running the test to see which file system call is actually returning the error.

As suggested, and after a lot of experimentation, I ran

fs_usage -w xctest

There is a lot of output, opening libraries and such, but when it comes down to the lines where the path (as in the original post) is included, it does a stat64 on the path, then two getattrlist's on the path, and then an open, which fails with return code 4. If I coax the test to succeed, for example by sleeping awhile, then the same lines appear and the open succeeds followed by open lines for all the subpaths. Not sure what the return code for the open signifies (perhaps EINTR?).

1 Like

then an open, which fails with return code 4.

Error 4 is indeed EINTR. That would explain the very odd symptoms you’re seeing (the fact that you can avoid the problem by adding a delay, the fact it only shows up under test, and so on).

You should definitely file a bug about this. Either the kernel needs to stop failing an open with EINTR [1] or Foundation needs to handle this error.

Please post your bug number, just for the record.

As far as workarounds are concerned, the obvious one — detecting this error as well as you can and retrying — should be fine IMO.

Share and Enjoy

Quinn “The Eskimo!” @ DTS @ Apple

[1] If I had a time machine, I’d go back in time and kill EINTR’s grandfather.

Hopefully I've done this correctly, not having used the feedback assistant app before. The number they've given back to me is FB7499074.

The number they've given back to me is FB7499074.

Thanks!

Hopefully I've done this correctly

It looks fine.

One thing that’d help is the actual fs_usage log that shows the EINTR error. Please add that as an attachment, and then note the line number of the EINTR in the bug report.

Share and Enjoy

Quinn “The Eskimo!” @ DTS @ Apple