Concise Magic File Names

Yes. You’re right. And I was part of that thread. I guess my memory span is shorter that I thought.

1 Like

You can’t change #file, why break (sure, tests mostly) code?

My personal preference would be a new #something, with paths relative to the source root, which swiftPM could easily pass to swiftc, but otherwise raw instantiations of swiftc would be fullpath or relative to CWD unless the flag that controls this is passed.

Would also be nice to simultaneously go in and remove the “same filename is illegal” restriction for modules, since this is an artificial limitation as confirmed by Rose in previous conversations. The limitation is to identify source units and could again be fixed if the module compilation knew the module-root and thus the relative path of the file. Though this bit is more complex for sure.

If that isn’t going to be fixed (because I can see why nobody could be bothered), then well, this proposal depends on this behavior, which is not intentional, just coincidental. That can be fixed though, the filename portion of the static string could be module root relative.

Finally, #filePath is a little ugly, #filename perhaps?

4 Likes

Just another option.

#file == #file(path)
#file(name) // new
4 Likes

I'm with @aciid and @Max_Howell1: I don't think this pitch has come anywhere close to justifying why #file should be changed. I'm absolutely happy to introduce new variants, and if you want to force users to think about their use of #file I'm ok with seeing it deprecated as well, but changing the behaviour is pretty brutal.

Incidentally, incautiously deprecating a # option is almost always brutal because it's not very easy to write multi-Swift-version libraries that can shim their way out of the problem. The result of that is that multi-Swift-version libraries are forced to disable warnings-as-errors, which tends to let bugs slip by. This is acceptable if the deprecation warning is emitted based on the tools version used to compile the source, instead of the compiler version.

5 Likes

-1 for the reasons exposed by @Max_Howell1.

I personally find the unique filename rule really inconvenient and I'm hoping it might lifted someday.

A collection of responses to various aspects of this objection:

This is a very fair question, and changing #file instead of deprecating and replacing it is certainly the most aggressive part of the proposal. Here are the major benefits of changing it, as I see them:

  1. The benchmarking indicates that, if most code adopts the new behavior, we'll get substantial code size and performance gains at an unbelievably low engineering cost. It's hard to find bang-for-your-buck like this.

  2. The information leaks caused by this are really pretty bad. Most likely, every app I ever shipped as an independent developer had my username embedded in the binary. People don't realize that this is happening and there's no sign of it in their source code. There's a fundamental issue of consent here that needs to be addressed.

  3. Experience with __FILE_NAME__ in clang indicates that merely providing an alternative won't really make a difference—people are unlikely to adopt it in the places they ought to. To actually see these benefits in practice, we would at least need to deprecate #file and migrate people to one of two alternatives. But that would create a lot more friction (and a lot more backwards-source-compatibility issues) than changing #file would.

And here are the reasons I think the source break is acceptable:

  1. Very little code will be broken by this change. The vast majority of uses (my estimate was 94%) merely display the string; they really don't care about its exact contents.

  2. The code that will be broken by it is already very fragile. It would be broken by switching to a different build system, by your existing build system generating compiler invocations slightly differently, by compiling from a temporary copy of the file, by using #sourceLocation(file:), etc. Swift did not, and frankly could not, promise that these uses of #file wouldn't break, because so much of what made them work is outside of its control.

  3. Broken code is very likely to be discovered immediately because these patterns were only ever usable in tests and local scripts. (I would hope tests would break when they can't load fixtures, anyway...) Code that would be broken by this could never have been deployed in binary form because it would not have worked without the project's source code installed at the right path.

  4. Once you have discovered that you need to fix the code, the fix is trivial (change #file to #filePath).

No promises, but I'm hoping that we can get the new identifier(s) supported in Swift 5.2 but not actually deprecate/change the behavior of #file until the version after. That would at least ease the source compatibility problem.

I don't think we can design the language around warnings-as-errors. Warnings exist so that the compiler can point out issues that aren't worth breaking the build; warnings-as-errors overrides that judgment and breaks the build anyway. If you think that some of the warnings the compiler emitted for your code shouldn't be errors, the right answer is to turn off warnings-as-errors for your project, not to stop diagnosing the issue for everyone.

If this feature happened to be released in a version with a new language mode, I think that would be a reasonable solution, but I don't know if that will happen and I don't think it's worth delaying the change to wait for a new language mode.

We have promised to preserve source compatibility, but that promise has always had its limits. For example, there is always potential for us to break some code, somewhere, with any new overload, but that doesn't stop us from adding new overloads unless we see that the break is pretty common (e.g. count(where:)). I think this very fragile, poorly-supported pattern is another candidate for that treatment.


I ultimately think that it's reasonable to decide that changing #file's behavior is a bridge too far, but I think it's best to make the case for changing #file and then let the core team decide if the argument is strong enough. That's why I included a fully-formed alternative that doesn't break existing uses of #file in "alternatives considered"—it's basically ready to go if the core team wants it.

25 Likes

Responses to other comments:

Thank you—I've rephrased parts of the proposal in light of these comments.

I've also called out the fact that, if we wanted to make #filePath absolute, we'd need to make it honor -debug-prefix-map. (Is there any reason why that wouldn't be a good approach?)

The string isn't intended to be parsed, although in practice, nothing will stop you from throwing a regular expression at it.

I don't want to specify too much here so that we can preserve some flexibility, allowing us to, for instance…

Change the format of #file if we drop the unique filename rule without needing an evolution proposal.

A naïve implementation of #context would end up increasing, not reducing, code size. For instance, it would probably contain the #function string, which today is rarely used. It would also contain both #file and #filePath even though only one would likely be used. You would have to count on the optimizer to delete the unused values.

Personally, I would like to see us change the way #file and friends are treated when within default arguments so that they are always generated at the ultimate call site. That would allow anyone to create their own type which automatically captured the contextual information they cared about, and nothing more:

struct Context: Hashable {
  let file: StaticString
  let line: UInt

  init(file: StaticString = #file, line: UInt = #line) {
    self.file = file
    self.line = line
  }
}

func foo(context: Context = Context()) {
  assert(context != Context(line: #line - 1),
         "captured info about foo()'s call site, not our default argument")
}

But that is a different (and separately controversial) proposal.

#filename sounds like it might be the opposite of what it is, i.e. the name of the file by itself without a path. #filePath makes it clear that the path to the file is included.

As a personal rule, I try to make #foo(...) and @foo(...) syntax match normal argument list syntax as much as possible, with the hope that we can eventually make these features more extensible without having to special-case a lot of existing syntax. This would violate that (admittedly non-critical) rule.

4 Likes

Thanks!

I think that would be reasonable, if it was decided to make the path always absolute. One might argue that -debug-prefix-map (and the Clang flag it was based on) only affects the emitted debug info and that it's odd to overload the flag in a way that also affects the runtime behavior of the program, but if we're comfortable saying that #filePath should only be used for debugging, then maybe that's fine.

Digging into the current state of Clang a bit more, it looks like someone (only a few days ago!) just landed a couple new flags around this, based on flags already implemented in gcc: Rather than change the existing behavior of -fdebug-prefix-map, they added -fmacro-prefix-map to remap path prefixes in __FILE__ and -ffile-prefix-map to apply a remapping simultaneously using both -fmacro-prefix-map and -fdebug-prefix-map.

But that feels like a lot of unnecessary complexity. Again, if it was decided that #filePath should always be absolute, I think -debug-prefix-map could also serve that purpose here, unless someone has a compelling reason to want remapped paths in their DI but not elsewhere in their binary?

But looping back around, I think keeping the path in the form originally passed to the compiler is an even better solution because it avoids all these questions.

1 Like

I’m very happy to see movement on this, and especially happy to see a proposal that dares to fix the bad default behaviour rather than assume legacy must always be preserved.

13 Likes

It might be necessary complexity for C—I was terrified to discover recently that #include __FILE__ is a thing that exists—but I agree that it probably wouldn't be necessary for Swift.

1 Like

"Source compatible" is not the same as "behavior compatible". This change does not fall under Swift's source compatibility guarantees. Which is not to say the change should not be given strict scrutiny.

I have nothing more to add over Brent's comments on just how big the code size wins are, and how bad the current information leakage is. These factors are real and considerable. This is going to come down to subjective preferences but these points are so significant IMO as to vastly outweigh concerns about breaking workflows that exploit the presence of the path for testing purposes (if someone finds a compelling example of how this may break code in production rather than tests or build systems, that might be different – but I would still find it a tough sell).

16 Likes

Was thinking it might be useful to separate the pieces of info returned. #file would return just file name, #path would just return path, and #module would return module. Each part could be used to construct what you needed.

2 Likes

I strongly agree on the proposal of fixing this, but I'm a tentative -1 on the justifications for changing the behavior of #file:

However, a great deal of code already uses #file and would in practice probably never be converted to #fileName .

I agree with this, but I'm curious what proportion of projects actually use #file explicitly? I would guess that the vast majority of apps only use it indirectly by calling assert() or some other error handling or logging function that captures it as a default argument.

This magic behavior of assertions in the standard lib is also the reason why developers are inadvertently including path info in their binaries - because they aren't calling #file explicitly they don't realize it's being called at all. Those that are using it explicitly are much more likely to realize what it does and what the implications are.

So it seems that much of the benefit of changing #file would also be achieved by adding a #filename alternative and then replacing all uses of #file with #filename within the stdlib. (You could also reach out to popular 3rd party frameworks that use #file and make sure they are aware they should switch).

The "other alternatives" section also doesn't seem very exhaustive. For example here is another option that seems viable:

Deprecating (not removing) #file and then replacing it with #filename and #filepath seems like a good solution because then everyone using #file would get a warning in their project and would be encouraged to make an explicit choice to switch to the behavior they actually want, but it would not be a source-breaking change.

1 Like

That’s actually the main alternative I discuss, and I agree that it’s the next-best answer.

I believe this to be a decisive point in favor of changing the behavior. The change is really on the level of a bugfix as far as I'm concerned. The existing behavior is wrong. (It's always bothered me.)

At the very least, if we decide it's absolutely necessary for #file to continue to evaluate to a path, it should be a relative path from the project root. But given Brent's description of how they are generated, it's not clear that is feasible.

3 Likes

@brentdax I think this would throw the whole proposal under the bus, but have you considered leaving #file as is and also not introducing any new variant similar to it but rather tackling a more commonly needed #context where we actually can implement the file differently as we would like?!

Edit: Ah sorry the was already mentioned upthread. Concise Magic File Names

Ah sorry, I didn't see it under "Other alternatives" and somehow missed the main alternatives section above. In that case I withdraw my criticism, but I do think that this alternative is the better choice overall.

I'd also add that this option doesn't preclude eventually changing the behavior of #file, or renaming #fileName to #file after a sufficient deprecation period.

On the other hand:

Literal: #file. Type: String. Value: The name of the file in which it appears.

So this seems to be on the same level as changing the description or debugDescription of a type, or the unspecified order in which results in a sequence are produced, or other similar implementation details. These can all be source-breaking but only for undocumented/incorrect usage.

4 Likes

I understand this change might not come under the official "source compatibility" story but the reality is that it will break significant amount of existing code. Sure it would be easy to fix or migrate that code but it's still a lot of churn. IMO it'll be better if users can opt-into the new behavior using a new language version or some other mechanism.

Can you provide any concrete examples? Others have posited that it may break testing code, but no examples have been provided.

Terms of Service

Privacy Policy

Cookie Policy