Limit Memory Usage while reading file using fileManager

Using Swift
1

Hello , i am reading content of file using file manager . Code for the same

var localFullFileName : String = "Path to file"
let fileManager = FileManager.default
//Checking whether file exists or not
if fileManager.fileExists(atPath: localFullFileName as String) {
    //Checking Whether file is readable or not
    if fileManager.isReadableFile(atPath: localFullFileName as String) {
       //Fetching file details
        let fileDetails = try fileManager.attributesOfItem(atPath: localFullFileName)
       //Size of file 
       let totalFileSizeInBytes = fileDetails[FileAttributeKey.size] as! Int
      //  
      var totalBytesRead = 0;
        
        let fileHandle = FileHandle(forReadingAtPath: localFullFileName as String)
        var sizeToWrite = 0
        var BufferSize = 100000
        if fileHandle != nil {
            while totalBytesRead<totalFileSizeInBytes{
                if (totalFileSizeInBytes-totalBytesRead > BufferSize) {
                    sizeToWrite = BufferSize
                } else {
                    sizeToWrite = totalFileSizeInBytes-totalBytesRead
                }
                if sizeToWrite == 0 {
                    break
                }
                //Reading content of file       
                var data = fileHandle?.readData(ofLength: sizeToWrite)
                print(data?.count)
                print("Total bytes read from file : ",totalBytesRead)
                print("Total file size to read : ",totalFileSizeInBytes)
                print("Remaining Size to read : ",totalFileSizeInBytes-totalBytesRead)
                totalBytesRead = totalBytesRead+sizeToWrite
                sleep(1)
                data?.removeAll(keepingCapacity: false)
            }
            
        }
    }
}

so when i am reading file memory usage of the test project increases proportional to bytes read from file.
Current usage of memory :

Screenshot 2020-04-24 at 1.41.29 PM
Screenshot 2020-04-24 at 1.41.29 PM1280×800 205 KB

Expected Memory Usage :

Screenshot 2020-04-24 at 1.59.20 PM
Screenshot 2020-04-24 at 1.59.20 PM1280×800 174 KB

Question :
How can i read file so that i can memory usage not increase proportional to bytes read from file ?

2

I suspect that the call fileHandle?.readData(ofLength: sizeToWrite) returns an autoreleased NSData object. Since you're calling this method in a loop, all the NSData objects are being accumulated in the current autorelease pool until your code returns (because the default autorelease pool only gets drained the next time the run loop runs).

You should be able to fix this by creating your own temporary autorelease pool in every loop iteration, like so:

while totalBytesRead<totalFileSizeInBytes{
    ...
    autoreleasepool {
        //Reading content of file       
        var data = fileHandle?.readData(ofLength: sizeToWrite)
        ...
        // This isn't necessary, but you can keep it in if you want
        data?.removeAll(keepingCapacity: false)
    }
}
2 Likes
3

@ole Thanks for fast reply :smiley: :smiley: . i will check and get back to you