SR-3423 Starter questions

Thanks Brent for the tips, I think those a well-documented starter guide.
Two thing I couldn’t understand, and you may help.

I dug into the codebase as stated in the steps you mentioned, and tried to change one of the errors to see if it really change when I use Swift REPL, but nothing changed. Is it possible that I change something and directly affect the Swift compiler? (maybe I need to do a build first? Or maybe related to xcode-select?)
Each type of diagnostics has 4 parts, ERROR(ID,Options,Text,Signature). Diagnostics use the first 3 parts and pass parentheses to the signature. What is meant by the signature and why it’s used?

Thanks again for the help.

···

Message: 18
Date: Thu, 9 Nov 2017 08:24:37 -0800
From: Brent Royal-Gordon <brent@architechies.com <mailto:brent@architechies.com>>
To: Mohammed Ennabah <ennabahm9@gmail.com <mailto:ennabahm9@gmail.com>>
Cc: swift-dev@swift.org <mailto:swift-dev@swift.org>
Subject: Re: [swift-dev] SR-3423 Starter questions
Message-ID: <1C627652-F5B7-42F3-8039-1AE7BA5F7E1E@architechies.com <mailto:1C627652-F5B7-42F3-8039-1AE7BA5F7E1E@architechies.com>>
Content-Type: text/plain; charset=utf-8

On Nov 7, 2017, at 4:18 AM, Mohammed Ennabah via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

Hi all,

This is the first time I work on that deep of the compiler (in SR-3423 Enum with tuple raw value not allowed) and I’m not sure where to start and what do I need to do and what files this issue is related to. It would be great if anyone guided me. Thanks.

This is *just* the very first step, but here's what I do when I want to debug something that involves a compiler error message:

  1. Write some code that triggers the bug and compile it.

  For instance, that bug includes a Stack Overflow link—copy that code and paste it into the Swift REPL.

  2. Pick some chunk of the error message that doesn't include any identifiers and search the Swift source code for it.

  For instance, one of the errors is "raw value for enum case must be a literal", so search for that.

  3. Most of the hits will be tests, but one will be in a .def file; that's where the error is defined. Open the file, go to that line, and get the error identifier from it.

  In this case, include/swift/AST/DiagnosticsParse.swift has this line in it:

    ERROR(nonliteral_enum_case_raw_value,PointsToFirstBadToken, "raw value for enum case must be a literal", ())
  
  The identifier I'm talking about is the first argument to the ERROR macro—here, it's "nonliteral_enum_case_raw_value".

  (Before you move on to the next step, look around in this file a little bit and try to get a sense for how error message strings are formatted. It may help you with future error searches.)

  4. Search the Swift source code for that error identifier. Each hit (besides the one in the .def file) is one of the places in the compiler that can generate that error message. Read the code around each of those hits (including the name of the functions, types, files, and folders it's in) and try to figure out if it handles the specific case you want to solve, and if so, how you might be able to modify it to suit your needs.

  In this example, there is only one hit—in a .cpp file in the parser—and, a couple lines above it, you can see a dyn_cast call that checks if the expression is a literal. (`dyn_cast` is equivalent to an `as?` cast in Swift—it returns `null` if the value doesn't belong to the indicated type.) This is where the compiler is testing whether the raw value assigned to a case is a literal. You'll need to modify the code here so that it also accepts tuples of literals (and presumably, tuples of tuples of literals, and so on). There will be more to do, but that's the first step.

When you're reading code, don't be discouraged if you don't understand it immediately. The Swift compiler is a large, complicated system, and there's lots of complicated stuff going on. Be patient, don't be afraid to jump over to another function to see what it does, and if you get stuck trying to figure out one part, move on to something else.

I'm still very much a beginner, but this has helped me get started on a few different bugs. I hope it can help you too.

--
Brent Royal-Gordon
Architechies

------------------------------

Message: 19
Date: Thu, 09 Nov 2017 08:48:33 -0800
From: Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>>
To: Johannes Weiss <johannesweiss@apple.com <mailto:johannesweiss@apple.com>>
Cc: swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>>
Subject: Re: [swift-dev] Zero-cost 'Service Provider
  Interface'/Signature Packages
Message-ID: <E2C40795-B1CD-43FE-856A-F4C34532F53A@apple.com <mailto:E2C40795-B1CD-43FE-856A-F4C34532F53A@apple.com>>
Content-Type: text/plain; charset="utf-8"

On Nov 8, 2017, at 5:27 PM, Johannes Weiß <johannesweiss@apple.com <mailto:johannesweiss@apple.com>> wrote:

Hi Daniel,

On 2 Nov 2017, at 8:15 pm, Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>> wrote:

My personal preference is to:
1. Do nothing for now, but encourage publishing standardized protocols to solve this need.
2. Hope for a future with WMO+LTO magic which recovers the performance, for the case where the entire application ends up using one implementation.

Hmm, but that'll only work if we get 'whole product optimisation', right? If we still compile one module at the time I don't think the compiler will be able to figure out that there's just one implementation of that protocol in the whole program. In fact it can't as that module might be linked into different programs and one of those programs might have a second implementation of that protocol. This is extremely likely as the 'test program' might have a mock/fake or some special implementation. Did I misunderstand you here?

That’s correct, that is what I meant by magic WMO+LTO future.

You can manage some of the “dependency injection” part by making the package which exports the common protocol also export a global variable for the concrete implementation in use (with setters). That could just be a “pattern” people follow. This wouldn’t be particularly pretty, but it would mean that intermediate packages could avoid declaring a concrete dependency on any one implementation, and leave it up to clients to pick.

hmm, two questions:
- what would the type of that global variable be? All libraries in the program will need to know and agree on that type

It would be the type of the abstract protocol.

- sounds like ThreadSanitizer would trap here unless we synchronise it which would make it slow again

Depends on how the protocol is phrased. The global instance could be a type which is instantiated and then doesn’t require locking.

An example of what I had in mind (Package.swifts left to the reader):

$ find . -name \*.swift -exec printf "**** %s ****\n" {} \; -exec cat {} \;
**** ./Client/Sources/Client/main.swift ****
import Log
import BadLogger

Log.registerLogger(BadLogger.self)
let logger = Log.createLogger()!
logger.log("how quaint")

**** ./BadLogger/Sources/BadLogger/BadLogger.swift ****
import Log

public struct BadLogger: Logger {
   public init() {}
   public func log(_ message: String) {
       fatalError("logging considered harmful: \(message)")
   }
}

**** ./Log/Sources/Log/Log.swift ****
import Dispatch

public protocol Logger {
   // MARK: Logger API

   init()
   func log(_ message: String)
}

// MARK: Global Registration

private let queue = DispatchQueue(label: "org.awesome.log")
private var theLoggerType: Logger.Type? = nil

public func registerLogger(_ type: Logger.Type) {
   queue.sync {
       theLoggerType = type
   }
}

public func createLogger() -> Logger? {
   return queue.sync{ theLoggerType }?.init()
}

I’m not saying this is the best solution in the world, but it does work currently without requiring new features. I agree there is a (small) performance cost, but for most use cases I doubt that is the most important consideration.

- Daniel

-- Johannes

- Daniel

On Nov 2, 2017, at 5:57 PM, Johannes Weiß via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

Hi swift-dev,

I talked to a few people about this problem and we agreed that it is a problem and that it needs to be discussed. I didn't quite know where it would fit best but let's go with swift-dev, please feel free to tell to post it elsewhere if necessary. And apologies for the long mail, couldn't come up with a sensible tl;dr...

Let me briefly introduce the problem what for the lack of a better name I call 'signature package' or 'Service Provider Interface' (SPI) as some people from the Java community seem to be calling it (https://en.wikipedia.org/wiki/Service_provider_interface\). For the rest of this email I'll use the term SPI.

In a large ecosystem there is a few pieces that many libraries will depend on and yet it seems pretty much impossible to standardise exactly one implementation. Logging is a very good example as many people have different ideas about how logging should and should not work. At the moment I guess your best bet is to use your preferred logging API and hope that all your other dependencies use the same one. If not you'll likely run into annoying problems (different sub-systems logging to different places or worse).

Also, in a world where some dependencies might be closed source this is an even bigger problem as clearly no open-source framework will depend on something that's not open-source.

In Java the way seems to be to standardise on some logging interface (read `protocol`) with different implementations. For logging that'd probably be SLF4J [4]. In Swift:

let logger: LoggerProtocol = MyFavouriteLoggingFramework(configuration)

where `LoggerProtocol` comes from some SPI package and `MyFavouriteLoggingFramework` is basically what the name says. And as a general practise, everybody would only use `LoggerProtocol`. Then tomorrow when I'll change my mind replacing `MyFavouriteLoggingFramework` by `BetterFasterLoggingFramework` does the job. With 'dependency injection' this 'logger' is handed through the whole program and there's a good chance of it all working out. The benefits are that everybody just needs to agree on a `protocol` instead of an implementation. :+1:

In Swift the downside is that this means we're now getting a virtual dispatch and the existential everywhere (which in Java will be optimised away by the JIT). That might not be a huge problem but it might undermine 'CrazyFastLoggingFramework's adoption as we always pay overhead.

I don't think this problem can be elegantly solved today. What I could make work today (and maybe we could add language/SwiftPM support to facilitate it) is this (:warning:, it's ugly)

- one SwiftPM package defines the SPI only, the only thing it exports is a `public protocol` called say `_spi_Logger`, no implementation
- every implementation of that SPI defines a `public struct Logger: _spi_Logger` (yes, they all share the _same_ name)
- every package that wants to log contains

if USE_FOO_LOGGER
   import FooLogger
#elif USE_BAR_LOGGER
   import BarLogger
#else
   import BuzLogger
#endif

where 'BuzLogger' is the preferred logging system of this package but if either `USE_FOO_LOGGER` or `USE_BAR_LOGGER` was defined this package is happy to use those as well.
- `Logger` is always used as the type, it might be provided by different packages though
- in Package.swift of said package we'll need to define something like this:

func loggingDependency() -> Package.Dependency {
if USE_FOO_LOGGER
    return .package(url: "github.com/...../foo.git <http://github.com/...../foo.git&gt;&quot;, ...)
#elif USE_BAR_LOGGER
    return ...
#else
    return .package(url: "github.com/...../buz.git <http://github.com/...../buz.git&gt;&quot;, ...)
#endif
}

func loggingDependencyTarget() -> Target.Dependency {
if USE_FOO_LOGGER
    return "foo"
#elif USE_BAR_LOGGER
    return "bar"
#else
    return "buz"
#endif
}
- in the dependencies array of Package.swift we'll then use `loggingDependency()` and in the target we use `loggingDependencyTarget()` instead of the concrete one

Yes, it's awful but even in a world with different opinions about the implementation of a logger, we can make the program work.
In the happy case where application and all dependency agree that 'AwesomeLogging' is the best framework we can just type `swift build` and everything works. In the case where some dependencies think 'AwesomeLogging' is the best but others prefer 'BestEverLogging' we can force the whole application into one using `swift build -Xswiftc -DUSE_AWESOME_LOGGING` or `swift build -Xswiftc -DUSE_BEST_EVER_LOGGING`.

Wrapping up, I can see a few different options:

1) do nothing and live with the situation (no Swift/SwiftPM changes required)
2) advertise something similar to what I propose above (no Swift/SwiftPM changes required)
3) do what Java does but optimise the existential away at compile time (if the compiler can prove there's actually only one type that implements that protocol)
4) teach SwiftPM about those SPI packages and make everything work, maybe by textually replacing the import statements in the source?
5) do what Haskell did and retrofit a module system that can support this
6) have 'special' `specialized protocol` for which a concrete implementation needs to be selected by the primary source
7) something I haven't thought of

Btw, both Haskell (with the new 'backpack' [1, 2]) and ML have 'signatures' to solve this problem. A signature is basically an SPI. For an example see the backpack-str [3] module in Haskell which defines the signature (str-sig) and a bunch of different implementations for that signature (str-bytestring, str-string, str-foundation, str-text, ...).

Let me know what you think!

[1]: Backpack: Retrofitting Haskell with Interfaces
[2]: backpack · Wiki · Glasgow Haskell Compiler / GHC · GitLab
[3]: GitHub - haskell-backpack/backpack-str: The Str signature and implementations
[4]: https://www.slf4j.org <https://www.slf4j.org/&gt;

-- Johannes
PS: I attached a tar ball which contains the following 6 SwiftPM packages that are created like I describe above:

- app, the main application, prefers the 'foo' logging library
- somelibA, some library which logs and prefers the 'foo' logging library
- somelibB, some other library which prefers the 'bar' logging library
- foo, the 'foo' logging library
- bar, the 'bar' logging library
- spi, the logging SPI

The dependency default graph looks like this:
+- somelibA ---+ foo
/ / \
app +--------------/ +-- spi
\ /
+- somelibB ---- bar

that looks all good, except that 'foo' and 'bar' are two logging libraries :see_no_evil:. In other words, we're in the unhappy case, therefore just typing `swift build` gives this:

--- SNIP ---
-1- johannes:~/devel/swift-spi-demo/app
$ swift build
Compile Swift Module 'app' (1 sources)
/Users/johannes/devel/swift-spi-demo/app/Sources/app/main.swift:14:23: error: cannot convert value of type 'Logger' to expected argument type 'Logger'
somelibB_func(logger: logger)
                 ^~~~~~
error: terminated(1): /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-build-tool -f /Users/johannes/devel/swift-spi-demo/app/.build/debug.yaml main
--- SNAP ---

because there's two `Logger` types. But selecting `foo` gives (note that all lines start with 'Foo:'):

--- SNIP ---
$ swift build -Xswiftc -DUSE_FOO
Compile Swift Module 'spi' (1 sources)
Compile Swift Module 'foo' (1 sources)
Compile Swift Module 'bar' (1 sources)
Compile Swift Module 'somelibB' (1 sources)
Compile Swift Module 'somelibA' (1 sources)
Compile Swift Module 'app' (1 sources)
Linking ./.build/x86_64-apple-macosx10.10/debug/app
$ ./.build/x86_64-apple-macosx10.10/debug/app
Foo: info: hello from the app
Foo: info: hello from somelibA
Foo: info: hello from somelibB
Foo: info: hello from somelibA
Foo: info: hello from somelibB
--- SNAP ---

and for 'bar' (note that all lines start with 'Bar:')

--- SNIP ---
$ swift build -Xswiftc -DUSE_BAR
Compile Swift Module 'spi' (1 sources)
Compile Swift Module 'foo' (1 sources)
Compile Swift Module 'bar' (1 sources)
Compile Swift Module 'somelibA' (1 sources)
Compile Swift Module 'somelibB' (1 sources)
Compile Swift Module 'app' (1 sources)
Linking ./.build/x86_64-apple-macosx10.10/debug/app
$ ./.build/x86_64-apple-macosx10.10/debug/app
Bar: info: hello from the app
Bar: info: hello from somelibA
Bar: info: hello from somelibB
Bar: info: hello from somelibA
Bar: info: hello from somelibB
--- SNAP ---

<swift-spi-demo.tar.gz>

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20171109/bfd886cf/attachment-0001.html&gt;

------------------------------

Message: 20
Date: Thu, 09 Nov 2017 09:12:27 -0800
From: Breckin Loggins <breckin@apple.com <mailto:breckin@apple.com>>
To: Brent Royal-Gordon <brent@architechies.com <mailto:brent@architechies.com>>
Cc: swift-dev@swift.org <mailto:swift-dev@swift.org>, Mohammed Ennabah <ennabahm9@gmail.com <mailto:ennabahm9@gmail.com>>
Subject: Re: [swift-dev] SR-3423 Starter questions
Message-ID: <76A3C32F-9F9F-4DB7-960F-DE4CBC4E21CB@apple.com <mailto:76A3C32F-9F9F-4DB7-960F-DE4CBC4E21CB@apple.com>>
Content-Type: text/plain; charset=utf-8

This would make a great contribution to the "how to contribute" documentation; it's excellent advice.

On Nov 9, 2017, at 8:24 AM, Brent Royal-Gordon via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

On Nov 7, 2017, at 4:18 AM, Mohammed Ennabah via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

Hi all,

This is the first time I work on that deep of the compiler (in SR-3423 Enum with tuple raw value not allowed) and I’m not sure where to start and what do I need to do and what files this issue is related to. It would be great if anyone guided me. Thanks.

This is *just* the very first step, but here's what I do when I want to debug something that involves a compiler error message:

  1. Write some code that triggers the bug and compile it.

  For instance, that bug includes a Stack Overflow link—copy that code and paste it into the Swift REPL.

  2. Pick some chunk of the error message that doesn't include any identifiers and search the Swift source code for it.

  For instance, one of the errors is "raw value for enum case must be a literal", so search for that.

  3. Most of the hits will be tests, but one will be in a .def file; that's where the error is defined. Open the file, go to that line, and get the error identifier from it.

  In this case, include/swift/AST/DiagnosticsParse.swift has this line in it:

    ERROR(nonliteral_enum_case_raw_value,PointsToFirstBadToken, "raw value for enum case must be a literal", ())
  
  The identifier I'm talking about is the first argument to the ERROR macro—here, it's "nonliteral_enum_case_raw_value".

  (Before you move on to the next step, look around in this file a little bit and try to get a sense for how error message strings are formatted. It may help you with future error searches.)

  4. Search the Swift source code for that error identifier. Each hit (besides the one in the .def file) is one of the places in the compiler that can generate that error message. Read the code around each of those hits (including the name of the functions, types, files, and folders it's in) and try to figure out if it handles the specific case you want to solve, and if so, how you might be able to modify it to suit your needs.

  In this example, there is only one hit—in a .cpp file in the parser—and, a couple lines above it, you can see a dyn_cast call that checks if the expression is a literal. (`dyn_cast` is equivalent to an `as?` cast in Swift—it returns `null` if the value doesn't belong to the indicated type.) This is where the compiler is testing whether the raw value assigned to a case is a literal. You'll need to modify the code here so that it also accepts tuples of literals (and presumably, tuples of tuples of literals, and so on). There will be more to do, but that's the first step.

When you're reading code, don't be discouraged if you don't understand it immediately. The Swift compiler is a large, complicated system, and there's lots of complicated stuff going on. Be patient, don't be afraid to jump over to another function to see what it does, and if you get stuck trying to figure out one part, move on to something else.

Another thing I've learned: don't be afraid to break it. On purpose.

Think of the Swift repo on your machine as YOUR Swift compiler. You can make it do anything you want! Wanna make Swift look like Pascal just to figure out how the lexer and parser work? Have at it! Just because you never intend to commit a change doesn't mean it isn't valuable.

When I don't know where to start with something in a complicated codebase, I always start by breaking something on purpose and putting a "DO NOT COMMIT" comment or warning where I do that. Now that I know I won't inadvertently check in a silly change and embarrass myself, I feel much better about exploring the code with abandon and learning about how things work by studying what happens when they don't... and that's where the learning happens for me.

I'm still very much a beginner, but this has helped me get started on a few different bugs. I hope it can help you too.

If it helps, I still do this on codebases I've been familiar with for years when I'm changing something in an area I don't know fluently. :)

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

------------------------------

_______________________________________________
swift-dev mailing list
swift-dev@swift.org <mailto:swift-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-dev

End of swift-dev Digest, Vol 24, Issue 10
*****************************************

I dug into the codebase as stated in the steps you mentioned, and tried to change one of the errors to see if it really change when I use Swift REPL, but nothing changed. Is it possible that I change something and directly affect the Swift compiler? (maybe I need to do a build first? Or maybe related to xcode-select?)

If you just type "swift" at the command line, you'll run the version of Swift built into Xcode. You will need to build Swift first ("utils/build-script" is the easiest way, but not the fastest), and you will also need to run the version of the Swift interpreter you just built. You can do that with a command like:

  ../build/Ninja-DebugAssert/swift-macosx-x86_64/bin/swift

(The path will be a little different if you use the "-x" flag to build-script, which generates an Xcode project, albeit one that's a pain to use.)

Each type of diagnostics has 4 parts, ERROR(ID,Options,Text,Signature). Diagnostics use the first 3 parts and pass parentheses to the signature. What is meant by the signature and why it’s used?

Error messages actually use a sort of format string language similar to printf(); for those messages which take parameters, the signature gives their types. For example, DiagnosticsParse.def includes this error:

  ERROR(expected_identifier_in_decl,none,
        "expected identifier in %0 declaration", (StringRef))

So code which emits that error must include a string to put in place of the "%0" token:

  // When we write "import" followed by a keyword that isn't "class", "struct", "protocol", etc.:
  diagnose(Tok, diag::expected_identifier_in_decl, "import");

  // When we try to parse an identifier after e.g. "typealias" and don't find one:
  P.diagnose(P.Tok, diag::expected_identifier_in_decl, DeclKindName);

  // When we try to parse an identifier after "case" and find some punctuation or a literal:
  diagnose(CaseLoc, diag::expected_identifier_in_decl, "enum 'case'");

Strings are the most common kind of parameter, but you can also pass other types and choose between different wordings based on them. For instance, here's a fix-it note I added for multiline string literals which takes a boolean to indicate singular or plural:

  NOTE(lex_multiline_string_indent_change_line,none,
        "change indentation of %select{this line|these lines}0 to match closing delimiter", (bool))

This error, which is shown when there's a non-digit in an integer literal, takes both a string containing the bad character, and a number (0 to 3) indicating which literal format it was parsing:

  ERROR(lex_invalid_digit_in_int_literal,none,
        "'%0' is not a valid %select{binary digit (0 or 1)|octal digit (0-7)|"
        "digit|hexadecimal digit (0-9, A-F)}1 in integer literal",
        (StringRef, unsigned))

If you look around that file and the other Diagnostics*.def files, you'll see some other interesting examples. Have fun poking around!

···

On Nov 11, 2017, at 2:59 PM, Mohammed Ennabah via swift-dev <swift-dev@swift.org> wrote:

--
Brent Royal-Gordon
Architechies

If you're in Xcode, you can also put code that will trigger whatever you're working on in one or more files and pass them to the compiler via the "Arguments Passed On Launch" setting in the "Arguments" tab of "Run" section of the relevant scheme ("swift", in this case), by clicking the scheme menu (in the top of the project window, just to the right of the button with the stop icon) and selecting "Edit Scheme...". That all sounds confusing but isn't that bad... if we could embed images, it'd take all of 5 seconds to see what I'm talking about (pun intended, naturally).

For example, I've been working on SR-4102 lately, so I've got the scheme set to pass "/path/to/my/swift/files/TestProjects/SR-4102/SR-4102/main.swift".

- Dave Sweeris

···

On Nov 13, 2017, at 6:00 AM, Brent Royal-Gordon via swift-dev <swift-dev@swift.org> wrote:

On Nov 11, 2017, at 2:59 PM, Mohammed Ennabah via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

I dug into the codebase as stated in the steps you mentioned, and tried to change one of the errors to see if it really change when I use Swift REPL, but nothing changed. Is it possible that I change something and directly affect the Swift compiler? (maybe I need to do a build first? Or maybe related to xcode-select?)

If you just type "swift" at the command line, you'll run the version of Swift built into Xcode. You will need to build Swift first ("utils/build-script" is the easiest way, but not the fastest), and you will also need to run the version of the Swift interpreter you just built. You can do that with a command like:

  ../build/Ninja-DebugAssert/swift-macosx-x86_64/bin/swift

(The path will be a little different if you use the "-x" flag to build-script, which generates an Xcode project, albeit one that's a pain to use.)

I dug into the codebase as stated in the steps you mentioned, and tried to change one of the errors to see if it really change when I use Swift REPL, but nothing changed. Is it possible that I change something and directly affect the Swift compiler? (maybe I need to do a build first? Or maybe related to xcode-select?)

If you just type "swift" at the command line, you'll run the version of Swift built into Xcode. You will need to build Swift first ("utils/build-script" is the easiest way, but not the fastest), and you will also need to run the version of the Swift interpreter you just built. You can do that with a command like:

  ../build/Ninja-DebugAssert/swift-macosx-x86_64/bin/swift

(The path will be a little different if you use the "-x" flag to build-script, which generates an Xcode project, albeit one that's a pain to use.)

Each type of diagnostics has 4 parts, ERROR(ID,Options,Text,Signature). Diagnostics use the first 3 parts and pass parentheses to the signature. What is meant by the signature and why it’s used?

Error messages actually use a sort of format string language similar to printf(); for those messages which take parameters, the signature gives their types. For example, DiagnosticsParse.def includes this error:

  ERROR(expected_identifier_in_decl,none,
        "expected identifier in %0 declaration", (StringRef))

So code which emits that error must include a string to put in place of the "%0" token:

  // When we write "import" followed by a keyword that isn't "class", "struct", "protocol", etc.:
  diagnose(Tok, diag::expected_identifier_in_decl, "import");

  // When we try to parse an identifier after e.g. "typealias" and don't find one:
  P.diagnose(P.Tok, diag::expected_identifier_in_decl, DeclKindName);

  // When we try to parse an identifier after "case" and find some punctuation or a literal:
  diagnose(CaseLoc, diag::expected_identifier_in_decl, "enum 'case'");

Strings are the most common kind of parameter, but you can also pass other types and choose between different wordings based on them. For instance, here's a fix-it note I added for multiline string literals which takes a boolean to indicate singular or plural:

  NOTE(lex_multiline_string_indent_change_line,none,
        "change indentation of %select{this line|these lines}0 to match closing delimiter", (bool))

This error, which is shown when there's a non-digit in an integer literal, takes both a string containing the bad character, and a number (0 to 3) indicating which literal format it was parsing:

  ERROR(lex_invalid_digit_in_int_literal,none,
        "'%0' is not a valid %select{binary digit (0 or 1)|octal digit (0-7)|"
        "digit|hexadecimal digit (0-9, A-F)}1 in integer literal",
        (StringRef, unsigned))

If you look around that file and the other Diagnostics*.def files, you'll see some other interesting examples. Have fun poking around!

Good work. I believe that these are pretty good examples to understand how the Swift diagnosis work in practice.

···

On Nov 13, 2017, at 4:00 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Nov 11, 2017, at 2:59 PM, Mohammed Ennabah via swift-dev <swift-dev@swift.org <mailto:swift-dev@swift.org>> wrote:

--
Brent Royal-Gordon
Architechies

I’m not sure what I’m missing when I build "utils/build-scrip”, but I keep getting
"can't find source directory for cmark (tried /Users/Mohammed/swift-source/cmark)”

Just to know: I mkdir “swift-source” and then cloned the project in it, I tried to git pull origin master, but I’m still the same error. Not sure if this is related, but I did also "brew install cmake ninja".

···

On Nov 13, 2017, at 4:00 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Nov 11, 2017, at 2:59 PM, Mohammed Ennabah via swift-dev <swift-dev@swift.org> wrote:

  • I dug into the codebase as stated in the steps you mentioned, and tried to change one of the errors to see if it really change when I use Swift REPL, but nothing changed. Is it possible that I change something and directly affect the Swift compiler? (maybe I need to do a build first? Or maybe related to xcode-select?)

If you just type "swift" at the command line, you'll run the version of Swift built into Xcode. You will need to build Swift first ("utils/build-script" is the easiest way, but not the fastest), and you will also need to run the version of the Swift interpreter you just built. You can do that with a command like:

  ../build/Ninja-DebugAssert/swift-macosx-x86_64/bin/swift

(The path will be a little different if you use the "-x" flag to build-script, which generates an Xcode project, albeit one that's a pain to use.)

It sounds like you haven't run the 'utils/update-checkout --clone' script.
That pulls down the cmark repository, and so should take care of that error.

The Swift README has step-by-step instructions, which I think mention the
'update-checkout' script. You may want to try following those from top to
bottom. Keep emailing if you have issues!

- Brian Gesiak

···

On Mon, Nov 13, 2017 at 12:52 PM, Mohammed Ennabah via swift-dev < swift-dev@swift.org> wrote:

> On Nov 13, 2017, at 4:00 PM, Brent Royal-Gordon <brent@architechies.com> > wrote:
>
>> On Nov 11, 2017, at 2:59 PM, Mohammed Ennabah via swift-dev < > swift-dev@swift.org> wrote:
>>
>> • I dug into the codebase as stated in the steps you mentioned,
and tried to change one of the errors to see if it really change when I use
Swift REPL, but nothing changed. Is it possible that I change something and
directly affect the Swift compiler? (maybe I need to do a build first? Or
maybe related to xcode-select?)
>
> If you just type "swift" at the command line, you'll run the version of
Swift built into Xcode. You will need to build Swift first
("utils/build-script" is the easiest way, but not the fastest), and you
will also need to run the version of the Swift interpreter you just built.
You can do that with a command like:
>
> ../build/Ninja-DebugAssert/swift-macosx-x86_64/bin/swift
>
> (The path will be a little different if you use the "-x" flag to
build-script, which generates an Xcode project, albeit one that's a pain to
use.)

I’m not sure what I’m missing when I build "utils/build-scrip”, but I keep
getting
"can't find source directory for cmark (tried /Users/Mohammed/swift-source/
cmark)”

Just to know: I mkdir “swift-source” and then cloned the project in it, I
tried to git pull origin master, but I’m still the same error. Not sure if
this is related, but I did also "brew install cmake ninja".
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

3 Likes

Swift depends on the source code of several other projects, like LLVM (for general compiler tools) and CommonMark (for parsing documentation), so you need to pull several other repos into specific folders next to the Swift source code's folder to build it. This is automated by the "utils/update-checkout" script. Instructions for this are in the readme's "Getting Sources for Swift and Related Projects" section: <GitHub - apple/swift: The Swift Programming Language. Once you have, you should be able to use the various commands described in the "Building Swift" section below it.

Hope this helps,

···

On Nov 13, 2017, at 9:52 AM, Mohammed Ennabah <ennabahm9@gmail.com> wrote:

I’m not sure what I’m missing when I build "utils/build-scrip”, but I keep getting
"can't find source directory for cmark (tried /Users/Mohammed/swift-source/cmark)”

Just to know: I mkdir “swift-source” and then cloned the project in it, I tried to git pull origin master, but I’m still the same error. Not sure if this is related, but I did also "brew install cmake ninja".

--
Brent Royal-Gordon
Architechies