file io in Swift 2.2 for Linux (would like to be pointed in the right direction)

I've been having trouble figuring out how to read and write data to a
textfile, and not finding much out there for Swift on Linux. I found code
for reading data from a file on StackOverflow that works for me, but
cludged together my own code for writing to a file. I'm not sure exactly
what I'm doing, but am making some progress. Can anyone point me toward a
good source of help for reading and writing data in Swift on Linux. Thanks!

This is the code that I found that works for reading data:

// Try to read from a file

// 5/16/2016

import Glibc //import a Linux library

let path = "Resources/sampleIn.txt"

let BUFSIZE = 1024

print("attempting to open a file for input")

let fp = fopen(path, "r")

if fp == nil {print("error reading file")}

if fp != nil {

print("reading...")

var buf = [CChar](count:BUFSIZE, repeatedValue:CChar(0))

while fgets(&buf, Int32(BUFSIZE), fp) != nil {

   print(String.fromCString(buf)!, terminator:"")

}

}

This is the code I pieced together for writing data:

import Glibc //import a Linux library
let path = "Resources/sampleOut.txt"let BUFSIZE =
1024print("attempting to open a file for Output")let fp = fopen(path,
"w+")if fp == nil {print("error writing file")}if fp != nil {
  print("Type a few words to be saved to a file")
  var fewWords=readLine()!
  fputs(fewWords,fp)
  print("writing...")
  }
fclose(fp)

Any help would be appreciated.
Thanks!
John

···

--
John Myers
Mathematics and Computer Science Teacher

------------------------------------------------------------------

Your examples show you transferring the data line-by-line. Is that the end goal? Or just fallout from what you happened to get working?

The reason I ask is that most of the time when I mess around with files I transfer the entire file to and from memory, which is something that the Foundation APIs excel at. OTOH, there are times when line-by-line is necessary.

Share and Enjoy

···

On 18 May 2016, at 23:02, John Myers via swift-users <swift-users@swift.org> wrote:

I've been having trouble figuring out how to read and write data to a textfile …

--
Quinn "The Eskimo!" <http://www.apple.com/developer/&gt;
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

Sorry about being complicit in the one-shot vs streaming I/O diversion. It’s true that Swift’s streaming I/O story is in flux and I’m sure that Jens and co. will have fun designing its replacement, but right now I’d kinda like to bring this back to the original question.

Below is a snippet that shows how to process a file line-by-line using Foundation APIs. As you can see, if you’re just trying to runs simple tests on the lines in a text file, doing this with Foundation is much easier than doing it with the current lower-level APIs.

···

On 18 May 2016, at 23:02, John Myers via swift-users <swift-users@swift.org> wrote:

I've been having trouble figuring out how to read and write data to a textfile, and not finding much out there for Swift on Linux.

---------------------------------------------------------------------------
import Foundation

func quoteFile(atPath filePath: String) throws {
    let input = try NSString(contentsOfFile: filePath, encoding: NSUTF8StringEncoding)
    let inputLines = input.components(separatedBy: "\n")
    var outputLines: [String] =
    for line in inputLines {
        let newLine = "> " + line
        outputLines.append(newLine)
    }
    let output = outputLines.joined(separator: "\n")
    try output.write(toFile: filePath, atomically: true, encoding: NSUTF8StringEncoding)
}

try! quoteFile(atPath: "/Users/quinn/victim.txt")
---------------------------------------------------------------------------

I tested this on OS X (don’t have Linux set up, sorry) with the “2016-05-09 (a)” Swift development snapshot.

Share and Enjoy
--
Quinn "The Eskimo!" <http://www.apple.com/developer/&gt;
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

I believe Mac/iOS developers usually transfer the entire file because that’s the particular hammer that Foundation provides. For some reason Foundation has never had good stream APIs (and they used to be worse!), so it’s much more convenient to just use NSData or NSString methods to read and write the entire file, than it is to mess around with NSFileHandle or NSStream.

Conversely, most other platforms (certainly C/POSIX and Java) make it easier to stream the file (which I believe is more efficient; certainly more scalable) than to block-read/write it.

I’m hoping that the Swift translation of Foundation provides greatly-improved Stream classes.

—Jens

···

On May 19, 2016, at 12:58 AM, Quinn The Eskimo! via swift-users <swift-users@swift.org> wrote:

The reason I ask is that most of the time when I mess around with files I transfer the entire file to and from memory, which is something that the Foundation APIs excel at.

Quinn, thanks for the example. I agree that Foundation makes it easier by
loading the data fully into memory. I still don't understand if it's
currently actually *possible* to use non C-based solutions in Swift which
use streaming. I wanted to use NSFileHandle (for a use case that requires
streaming) and realized that this is an API that cannot really be used in
Swift because it's based on Objective-C exceptions. Are there any
alternatives?

Thanks,
  Matthias

*Matthias Zenger* matthias@objecthub.net

···

On Mon, May 23, 2016 at 10:24 AM, Quinn "The Eskimo!" via swift-users < swift-users@swift.org> wrote:

On 18 May 2016, at 23:02, John Myers via swift-users < > swift-users@swift.org> wrote:

> I've been having trouble figuring out how to read and write data to a
textfile, and not finding much out there for Swift on Linux.

Sorry about being complicit in the one-shot vs streaming I/O diversion.
It’s true that Swift’s streaming I/O story is in flux and I’m sure that
Jens and co. will have fun designing its replacement, but right now I’d
kinda like to bring this back to the original question.

Below is a snippet that shows how to process a file line-by-line using
Foundation APIs. As you can see, if you’re just trying to runs simple
tests on the lines in a text file, doing this with Foundation is much
easier than doing it with the current lower-level APIs.

---------------------------------------------------------------------------
import Foundation

func quoteFile(atPath filePath: String) throws {
    let input = try NSString(contentsOfFile: filePath, encoding:
NSUTF8StringEncoding)
    let inputLines = input.components(separatedBy: "\n")
    var outputLines: [String] =
    for line in inputLines {
        let newLine = "> " + line
        outputLines.append(newLine)
    }
    let output = outputLines.joined(separator: "\n")
    try output.write(toFile: filePath, atomically: true, encoding:
NSUTF8StringEncoding)
}

try! quoteFile(atPath: "/Users/quinn/victim.txt")
---------------------------------------------------------------------------

I tested this on OS X (don’t have Linux set up, sorry) with the
“2016-05-09 (a)” Swift development snapshot.

Share and Enjoy
--
Quinn "The Eskimo!" <http://www.apple.com/developer/&gt;
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

I disagree. One-shot file system APIs make sense for a lot of reasons:

* It’s generally easier to use. Consider NSXMLDocument vs NSXMLParser (-:

* It composes well. Consider the NSXMLDocument example again. This high-level one-shot API is layered on top of a low-level one-shot API. Which isn’t to say that you can’t build streaming APIs on top of other streaming APIs, but it’s not exactly easy, and you end up having to build the one-shot ‘convenience’ API anyway.

* It’s atomic on write; other folks looking at the file system never see ‘half’ the file.

* It radically reduces the state space of the API. When you read a file with a one-shot API, you only have to deal with one possible error. When you stream through a file, you can get an error at any point.

One of the nice things about the file system is that it has reasonable performance and error characteristics such that you /can/ deal with it synchronously. Revel in that fact!

I generally find myself streaming through a file only when I’m dealing with some sort of I/O pipeline, which is the exception rather than the rule.

Share and Enjoy

···

On 19 May 2016, at 17:38, Jens Alfke <jens@mooseyard.com> wrote:

I believe Mac/iOS developers usually transfer the entire file because that’s the particular hammer that Foundation provides.

--
Quinn "The Eskimo!" <http://www.apple.com/developer/&gt;
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

I still don't understand if it's currently actually *possible* to use non C-based solutions in Swift which use streaming. I wanted to use NSFileHandle (for a use case that requires streaming) and realized that this is an API that cannot really be used in Swift because it's based on Objective-C exceptions.

Ah, yes, that’s definitely something you should be wary of. I don’t know what the plan is for getting NSFileHandle to play nicely with Swift. Maybe someone else will chime in.

Are there any alternatives?

You could always use NSStream.

However, building Swift code on top of NSStream is about as difficult as building it on top of UNIX system calls (open/close/read/write) or stdio (fopen/fclose/fread/fwrite), so if you need streaming file I/O you might as well do that.

Give it a go and let us know if you get stuck.

Finally, as has been mentioned on this thread already, it’s easy to imagine a Swift streaming file I/O that’s /much/ better than these options, although AFAIK there are no concrete plans for that.

Share and Enjoy

···

On 5 Jun 2016, at 02:25, Matthias Zenger <matthias@objecthub.net> wrote:
--
Quinn "The Eskimo!" <http://www.apple.com/developer/&gt;
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

I wanted to use NSFileHandle (for a use case that requires streaming) and realized that this is an API that cannot really be used in Swift because it's based on Objective-C exceptions.

You’re right, NSFileHandle is a very archaic class (kind of a coelacanth) and its I/O methods signal errors by throwing exceptions. It’s almost unique in that regard; in general Cocoa APIs are only supposed to throw exceptions for programmer errors like assertion failures.

Are there any alternatives?

NSStream.

—Jens

···

On Jun 4, 2016, at 6:25 PM, Matthias Zenger via swift-users <swift-users@swift.org> wrote:

I disagree. One-shot file system APIs make sense for a lot of reasons:

I do most of my programming in .Net, where streams are the primary IO system. Even simple tasks often require multiple objects and loops copying data between them. Limitations in the API and in the implementations layer on complexity.

I relish whenever I have to do IO on the Mac using the one-shot API. A single line of code almost always accomplishes what I want, and the API is perfectly clear on what it's doing.

I generally find myself streaming through a file only when I’m dealing with some sort of I/O pipeline, which is the exception rather than the rule.

Indeed, and in these domain's it's an excellent conceptual platform to work in. But as the *primary* API it's a wonderful example of distributed cost.

···

On May 20, 2016, at 4:16 AM, Quinn The Eskimo! via swift-users <swift-users@swift.org> wrote:

One of the nice things about the file system is that it has reasonable performance and error characteristics such that you /can/ deal with it synchronously. Revel in that fact!

That’s only really true at small scales, for local filesystems, under light I/O loads. This happens to be true most of the time* for typical Mac/iOS app needs, but it’s not universal.

I believe most people in the Swift community today come from a Mac/iOS background, but as Linux support matures, and with IBM’s backing, that’s going to be less and less true in the future. Me, I’m one of the oddball “mobile guys” at a database company [Couchbase] whose customers [eBay, PayPal, Citi, etc.] very commonly have multi-terabyte data sets.

Consider the NSXMLDocument example again. This high-level one-shot API is layered on top of a low-level one-shot API.

The trouble with this is that those one-shot APIs tend to suck when working with data coming from the network, because you have to buffer the entire data set in memory, and you can’t start parsing it until the entire response is finished. If it does finish — there are protocols like XMPP/Jabber that use indefinitely-long streams of structured data like XML and can’t be parsed at all with one-shot APIs. I’ve recently had to replace NSJSONSerialization with a 3rd-party streaming JSON parser library, because the latency of reading large JSON documents over HTTP was getting in the way of performance and memory requirements.

Sorry if this is off-topic! What’s relevant is that I’d hate to see Apple’s historical bias toward one-shot I/O get in the way of Swift’s Foundation framework having a kick-ass stream API.

—Jens

* But not always. I can attest that during OS startup, when dozens of processes are contending for the disk, reading a 10MB file can take an achingly long time by normal standards. This caused lots of “fun” during performance tuning of the PubSub framework. Also, remember the days when devs at Apple still had NFS-based home directories?

···

On May 20, 2016, at 1:16 AM, Quinn The Eskimo! via swift-users <swift-users@swift.org> wrote:

One of the nice things about the file system is that it has reasonable performance and error characteristics such that you /can/ deal with it synchronously. Revel in that fact!

That’s only really true at small scales, for local filesystems, under light I/O loads. This happens to be true most of the time* for typical Mac/iOS app needs, but it’s not universal.

I believe most people in the Swift community today come from a Mac/iOS background, but as Linux support matures, and with IBM’s backing, that’s going to be less and less true in the future. Me, I’m one of the oddball “mobile guys” at a database company [Couchbase] whose customers [eBay, PayPal, Citi, etc.] very commonly have multi-terabyte data sets.

Couchbase looks like a great product. Cool to see someone from Couchbase is participating in Swift evolution!

Consider the NSXMLDocument example again. This high-level one-shot API is layered on top of a low-level one-shot API.

The trouble with this is that those one-shot APIs tend to suck when working with data coming from the network, because you have to buffer the entire data set in memory, and you can’t start parsing it until the entire response is finished. If it does finish — there are protocols like XMPP/Jabber that use indefinitely-long streams of structured data like XML and can’t be parsed at all with one-shot APIs. I’ve recently had to replace NSJSONSerialization with a 3rd-party streaming JSON parser library, because the latency of reading large JSON documents over HTTP was getting in the way of performance and memory requirements.

Sorry if this is off-topic! What’s relevant is that I’d hate to see Apple’s historical bias toward one-shot I/O get in the way of Swift’s Foundation framework having a kick-ass stream API.

+1. There are many reasons to prefer streaming implementations. It is also possible to write high-level APIs with relatively simple interfaces that use streaming behind the scenes. The fact that this is not always done is not a good argument against providing robust streaming APIs. In fact, it should be considered a challenge to improve the state of the art.

For example, I shared an example of a Ruby DSL I wrote that generated streaming XML parsers (in an Objective-C API and some C in the implementation) from the DSL which defined the structure of the document and how it maps to model objects. This was much easier to use than any DOM-based API (whether XML, JSON, or whatever) and was more efficient. It did not require the entire document to be in memory at once and did not construct all kinds of intermediate objects not needed by the model (dictionaries and arrays, etc).

If / when Swift gets macros we should be able to implement something very similar directly in Swift without needing an intermediate code generation step. It may also be possible to do this a better way in Swift.

The main point I am making here is that taking advantage of the benefits of streaming APIs does not need to be difficult at the application level.

···

On May 20, 2016, at 10:47 AM, Jens Alfke via swift-users <swift-users@swift.org> wrote:

On May 20, 2016, at 1:16 AM, Quinn The Eskimo! via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:

—Jens

* But not always. I can attest that during OS startup, when dozens of processes are contending for the disk, reading a 10MB file can take an achingly long time by normal standards. This caused lots of “fun” during performance tuning of the PubSub framework. Also, remember the days when devs at Apple still had NFS-based home directories?
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Hello

Interesting question

Maybe something worth the reading

Techniques for Reading and Writing Files Without File Coordinators

Gérard

···

From my point of view, I want to parse a huge file without loading it in memory...

Le 20 mai 2016 à 13:59, Maury Markowitz via swift-users <swift-users@swift.org> a écrit :

On May 20, 2016, at 4:16 AM, Quinn The Eskimo! via swift-users <swift-users@swift.org> wrote:
I disagree. One-shot file system APIs make sense for a lot of reasons:

I do most of my programming in .Net, where streams are the primary IO system. Even simple tasks often require multiple objects and loops copying data between them. Limitations in the API and in the implementations layer on complexity.

I relish whenever I have to do IO on the Mac using the one-shot API. A single line of code almost always accomplishes what I want, and the API is perfectly clear on what it's doing.

I generally find myself streaming through a file only when I’m dealing with some sort of I/O pipeline, which is the exception rather than the rule.

Indeed, and in these domain's it's an excellent conceptual platform to work in. But as the *primary* API it's a wonderful example of distributed cost.

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

Quinn, Jens, thanks for pointing me at the NSStream classes. For some
reason, I discarded NSStream quickly after seeing the first paragraph
of Apple's
documentation on output streams
<https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/Streams/Articles/WritingOutputStreams.html&gt;
listing 5 relatively complex steps — without realizing that this API also
has a synchronous interface.

Obviously, the NSStream API is too low level and thus difficult to use.
Therefore, I would also like to see Apple build a great IO streaming API
for Swift. I've implemented Quinn's example using NSInputStream and Swift's
Generator design pattern. Except for mapping NSInputStream to a generator —
which is really messy — everything else is straightforward and much more
extensible and scalable than the standard Foundation approach. The code is
below.

== Matthias

See TextStreams.swift · GitHub

class ByteGenerator: GeneratorType {

  typealias Element = UInt8

  let input: NSInputStream

  var buffer: [UInt8]

  var index: Int = 0

  var eof: Bool = true

  init?(path: String, capacity: Int = 1024) {

    guard let input = NSInputStream(fileAtPath: path) else {

      return nil

    }

    self.buffer = [UInt8](count: capacity, repeatedValue: 0)

    input.open()

    if input.hasBytesAvailable {

      self.eof = input.read(&self.buffer, maxLength: self.buffer.count *
sizeof(UInt8)) <= 0

    }

    self.input = input

  }

  deinit {

    input.close()

  }

  func next() -> UInt8? {

    guard !self.eof else {

      return nil

    }

    if self.index >= self.buffer.count {

      self.index = 0

      self.eof = !input.hasBytesAvailable ||

                 input.read(&self.buffer, maxLength: self.buffer.count *
sizeof(UInt8)) <= 0

      guard !self.eof else {

        return nil

      }

    }

    self.index += 1

    return self.buffer[self.index - 1]

  }

}

struct CharacterGenerator<G: GeneratorType, U: UnicodeCodecType where
G.Element == U.CodeUnit>: GeneratorType {

  typealias Element = Character

  var source: G

  var decoder: U

  mutating func next() -> Character? {

    guard case .Result(let scalar) = self.decoder.decode(&self.source) else
{

      return nil

    }

    return Character(scalar)

  }

}

struct LineGenerator<G: GeneratorType where G.Element == Character>:
GeneratorType {

  typealias Element = String

  var source: G

  mutating func next() -> String? {

    guard let fst = source.next() else {

      return nil

    }

    guard fst != "\n" else {

      return ""

    }

    var line = String(fst)

    while let ch = source.next() {

      if (ch == "\n") {

        return line

      }

      line.append(ch)

    }

    return line

  }

}

if let input = ByteGenerator(path: "/Users/username/filename.txt") {

  var generator = LineGenerator(source: CharacterGenerator(source: input,
decoder: UTF8()))

  var i = 0

  while let line = generator.next() {

    print("\(i): \(line)")

    i += 1

  }

}

*Matthias Zenger* matthias@objecthub.net

···

On Sun, Jun 5, 2016 at 8:36 PM, Jens Alfke <jens@mooseyard.com> wrote:

On Jun 4, 2016, at 6:25 PM, Matthias Zenger via swift-users < > swift-users@swift.org> wrote:

I wanted to use NSFileHandle (for a use case that requires streaming) and
realized that this is an API that cannot really be used in Swift because
it's based on Objective-C exceptions.

You’re right, NSFileHandle is a very archaic class (kind of a coelacanth)
and its I/O methods signal errors by throwing exceptions. It’s almost
unique in that regard; in general Cocoa APIs are only supposed to throw
exceptions for programmer errors like assertion failures.

Are there any alternatives?

NSStream.

—Jens

Almost unique is right. The only other offender I can think of is Distributed Objects, which is even less fun because any method call (including property accesses) on any proxied object can throw language exceptions.

Share and Enjoy

···

On 5 Jun 2016, at 11:36, Jens Alfke <jens@mooseyard.com> wrote:

It’s almost unique in that regard; in general Cocoa APIs are only supposed to throw exceptions for programmer errors like assertion failures.

--
Quinn "The Eskimo!" <http://www.apple.com/developer/&gt;
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

+1. There are many reasons to prefer streaming implementations. It is also possible to write high-level APIs with relatively simple interfaces that use streaming behind the scenes. The fact that this is not always done is not a good argument against providing robust streaming APIs. In fact, it should be considered a challenge to improve the state of the art.

What advantage? Streaming IO is a pain in the ****. I never got what people like at this stuff - Even in my windows days I always used the much better memory mapped file IO. Is streaming IO some kind of C/C++ sickness like macros and templates?

greets, Jan

Looks good, but you'll want some error handling. There's no clean way to integrate it with Generator because the protocol doesn't allow the implementation to throw, unfortunately. (I've run into the same problem building a Generator around a database query.) I think the best solution is to add a property that returns the NSStream.streamError, or a checkError() method that throws the current error if any, and have the caller use those at the end of the iteration.

--Jens

Looks good, but you'll want some error handling. There's no clean way to integrate it with Generator because the protocol doesn't allow the implementation to throw, unfortunately. (I've run into the same problem building a Generator around a database query.) I think the best solution is to add a property that returns the NSStream.streamError, or a checkError() method that throws the current error if any, and have the caller use those at the end of the iteration.

You can do better.

  extension NSInputStream {
    // I will assume you already have a byteGenerator method (which you can use with a for loop)
    // with a checkError() method (which throws if the generation terminated due to an error). However,
    // you've made these private, and will use them to implement this safe public API.
    public func withByteGenerator<R>(blockSize: Int = 1024, iterate: @noescape (NSInputStream.ByteGenerator) throws -> R) throws -> R {
      let generator = byteGenerator(blockSize: blockSize)
      let result = iterate(generator)
      try generator.checkError()
      return result
    }
  }

Now you write:

  guard let stream = NSInputStream(fileAtPath: path) else {
    …
  }
  try stream.withByteGenerator {
    // XXX This would all be more complicated if CharacterGenerator can throw, too.
    for (i, line) in LineGenerator(source: CharacterGenerator(source: $0, decoder: UTF8())).enumerate() {
      print("\(i+1): \(line)")
    }
  }

(I'm assuming that these generators take their almost-automatic Sequence conformance, of course.)

···

--
Brent Royal-Gordon
Architechies

Yes, error reporting is missing (errors are treated like EOF), but your
proposal breaks encapsulation. By not making `NSInputStream` a hidden
implementation detail, you make it possible for clients to interfere with
the caching logic (e.g. by directly invoking `read`, or prematurely closing
the steam).

== Matthias

*Matthias Zenger* matthias@objecthub.net

···

On Mon, Jun 6, 2016 at 1:49 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

> Looks good, but you'll want some error handling. There's no clean way to
integrate it with Generator because the protocol doesn't allow the
implementation to throw, unfortunately. (I've run into the same problem
building a Generator around a database query.) I think the best solution is
to add a property that returns the NSStream.streamError, or a checkError()
method that throws the current error if any, and have the caller use those
at the end of the iteration.

You can do better.

        extension NSInputStream {
                // I will assume you already have a byteGenerator method
(which you can use with a for loop)
                // with a checkError() method (which throws if the
generation terminated due to an error). However,
                // you've made these private, and will use them to
implement this safe public API.
                public func withByteGenerator<R>(blockSize: Int = 1024,
iterate: @noescape (NSInputStream.ByteGenerator) throws -> R) throws -> R {
                        let generator = byteGenerator(blockSize: blockSize)
                        let result = iterate(generator)
                        try generator.checkError()
                        return result
                }
        }

Now you write:

        guard let stream = NSInputStream(fileAtPath: path) else {
                …
        }
        try stream.withByteGenerator {
                // XXX This would all be more complicated if
CharacterGenerator can throw, too.
                for (i, line) in LineGenerator(source:
CharacterGenerator(source: $0, decoder: UTF8())).enumerate() {
                        print("\(i+1): \(line)")
                }
        }

(I'm assuming that these generators take their almost-automatic Sequence
conformance, of course.)

--
Brent Royal-Gordon
Architechies

It’s almost unique in that regard; in general Cocoa APIs are only supposed to throw exceptions for programmer errors like assertion failures.

Almost unique is right. The only other offender I can think of is Distributed Objects, which is even less fun because any method call (including property accesses) on any proxied object can throw language exceptions.

The docs only touch on it, but `-[NSTask launch]` throws an Objective-C exception "if it fails to create a process", which is something which can be very difficult to prevent through ahead-of-time checks.

···

--
Brent Royal-Gordon
Architechies

What advantage? Streaming IO is a pain in the ****. I never got what people like at this stuff

Well for one, it abstracts away the source of the data. It could be a file, a TCP socket, an HTTP body, the output of another algorithm like a GZip codec, the output of another process. One of the big innovations of Unix was the ability to build small tools and chain them together using pipes.

Even in my windows days I always used the much better memory mapped file IO.

Memory mapping is great, but it has limitations and pitfalls.
* You can only use it on filesystems that can’t be unmounted, otherwise if the filesystem disconnects, accessing the memory will crash.
* It ties up a file descriptor for as long as you need to use the data. File descriptors are a limited resource.
* It requires having contiguous free address space equal to the size of the file. I have seen this cause real-world failures in 32-bit processes.
* I’m told that iOS limits the amount of memory-mapped address space an app can allocate, even if it’s a 64-bit process. (I have been meaning to test whether this is actually true.)

Is streaming IO some kind of C/C++ sickness like macros and templates?

This isn’t a Playstation-vs-Xbox debate. The mature developer understands that everything has pros and cons.

—Jens

···

On May 20, 2016, at 9:44 AM, Jan Neumüller via swift-users <swift-users@swift.org> wrote: