Terminal I/O on Windows

Development Windows
1

Hi everyone,

I'm trying to teach basic Swift programming on Windows, with simple command line apps/games, but I'm having some issues getting I/O to work properly.

Specifically:

  • When I run a Swift package with swift run, there can be shell prompts mixed in with the output, the input is sometimes parsed by the shell and not sent to the application, and the cursor can jump to different positions on the screen. Even a simple print("Hello, world!") can be affected by this. This also affects CLion on Windows. Building and running separately does work fine.
  • Complex characters (curly quotes, emoji, ...) don't render properly in Windows terminals. At first, I tried executing chcp 65001 prior to running a program, but on Windows 11, that doesn't help me anymore. What does help is adding the following to my code:
    import WinSDK
_ = SetConsoleOutputCP(UINT(CP_UTF8))
    That does render most emojis (except combined ones) but it's quite cumbersome to require students to add this to all of their exercises.

Has anyone succesfully built any interactive command line apps on Windows? I had hoped to see Unicode output improve by upgrading to PowerShell 7 and the new Windows Terminal, but unfortunately, that didn't fix anything.

2

@compnerd Maybe we should add this line somewhere in stdlib, until code page support has eventually landed.

3

This is very tempting. I don't see any technical reasons for the standard library to not switch the console to and from UTF-8. I suspect that this is something worth considering on all platforms (I could be setting my console to a different encoding even on Linux). I think that @Mike_Ash and @Joe_Groff may have thoughts on this, and I would be very interested in their opinion on this.

I'm sorry, I don't follow this. Could you please help me understand what support this is referring to?

4

I'm not very well-versed in Windows consoles. In Unix land I'm not a fan of tools that change the TTY status since it persists after the process exits, and there isn't a good way to ensure that the status is set back if the process dies abnormally. Would setting the console to UTF-8 on Windows persist if a Swift process crashes? How bad is it in practice if it does?

5

Nope. Code page status is tied to the process (and passed to its subprocesses). Changing the code page will not affect any other existing process.

I wonder if we should also call the input counterpart SetConsoleCP(UINT(CP_UINT8)).

1 Like
6

Iā€™m doubting its necessity, but we can assume that someone may want to use Swift to encode to & decode from different code pages, and currently such support is broken for String.

This would be, however, really useful when it comes to GUI, because some fonts can only work with specific code page instead of Unicode. Parsing existing text files in native code page is also difficult for now.

7

That's great, then it seems reasonable to have Swift processes adopt the UTF-8 code page when console output begins. If it's inherited by subprocesses, are there circumstances where high-level APIs for spawning subprocesses, like Foundation's Process, would need the option to switch the code page back?

8

The only reasons I come up with immediate is:

  • on windows, the spawned process is a non-Swift console application that expects the console to accept a specific, e.g. CP932 (Shift JIS)
  • on Linux, the spawned process consults /etc and finds that the system default configuration is a specific value and the terminal currently is set to something else.

I think that if those cases actually arise, we can deal with them subsequently. The default encoding is UTF-8 for Swift, and print definitely assumes as much. Generally speaking, applications expecting a certain encoding should be setting the process environment to that via SetConsoleCP or setlocale.

Yes, we should set both sides of the IO to UTF8 IMO.