You just reminded me that we should consider mutation testing in this conversation. The mull project is bringing that to LLVM-compiled languages.
I dunno how the changes proposed here would affect a workflow that uses MT, but it should consider it if it does. MT is very useful as a check on the quality of one's tests.
Something like this would be great. The ability to declare these tests inline solves the access control issue without creating a new bunch of strange and difficult to remember exceptions, and puts the test code in the context where it makes the most sense. I am very resistant to the suggestions of separate testing files that have yet another set of access control semantics, because this would just make the access control story more complicated than it already is.
I also find this focus on the number of lines in a file quite dogmatic and hard to understand. Surely having short files is just correlated with complexity of code, not inherently good itself. Much like doc comments, code marked @test can't contribute to the complexity of the implementation and can be easily folded away in your editor of choice.
That's not how I understood it here. It makes more sense to allow both, in-line tests and tests separated into files. The latter does not need any extra access control as long as the test file is mirroring an other concrete swift file, because then it will have all the visibility like the developer who reads that swift file.
Such an approach won't slice the community in half and you would be able to chose the most comfortable approach for you.
I think that jaw broken was simply speaking against a new testing paradigm that had separate files with more visibility into the code under test.
I think that was my interpretation as well, but to me this is just another exception to the access control rules that makes them more semantically complicated (e.g. internal except when someone imports your module as @testable, fileprivate except when there's a special testing twin file, private except ??). I find access control messy enough without these exceptions, and would prefer not to add more unless there's a good justification and, as I said above, I don't understand the file length justification.
Well maybe you do don't understand but other people including myself care about the file length. For instance I have a fixed line width of 80 characters max which sometimes leads to more lines than I want to. Forcing the Swift community to write unit tests in the same file will potentially, exponentially explode the file lengths and make it unreadable and hard to maintain. I agree with you that the current access control in Swift is far from being optimal nor is it flexible enough for my liking - and no submodules won't solve that problem either. I don't think it would be a fair argument to say that submodules and hidden box types is a better way to build things than a flexible access control system. I'm also not proposing to add any new access control (even if I would wish that), but I think it is an ideal solution of both worlds, now that we don't have flexible access control, to allow users that do care about file length to separate unit tests into other files. Those unit test files won't gain any more or new access control since such unit tests files will be somehow linked to a specific swift file that you wish to test. @testable as described before and in the other thread I linked above is just a quick and dirty hack which we wish to fix in this thread to potentially deprecate it. If you prefer to write in-line tests, it's completely up to you and do as you seem fit for your codebase but I think it won't lead this discussion anywhere if the half of the community would be against allowing the other half to test their code as they would like to and vice versa.
If I still misunderstood your point, please let me know.
I find this to be a big step back for the language. Testing private methods is not something that should be done, and creating a way to do that will negatively impact the software architecture that people create.
Engineers should be testing the exposed interface. Not the internals.
I believe we should wait on improving our testing libraries until there is more reflection and dynamism built into the language.
What makes internal functions less worthy of tests? There should most definitely be tests for the public API but that does not mean we can't test the semantics of our internal implementation details as well.
How will testing private methods affect the architecture negatively? I don't understand
This is a big argument in testing. IMO Swift should allow the capability to test private members, and leave it up to the respective projects to enforce a testing discipline. Now how that ability to test members is done I think could be up for discussion.
One thing about an in-line test design is that, by locating private-implementation tests next to the implementation, whether the private implementation details have tests or not is effectively an implementation detail. It shouldn't necessarily have any negative impact on your overall architecture since you can still do your interface-level testing in the usual way. If anything, it allows for improvements to your overall architecture, since there's less temptation to unnecessarily weaken access control to private things solely for the purpose of exposing it to be testable as if it were an interface.
I really want to stress this point. This change would allow us, if we desired, to adopt the stance that all tests outside the file are tests of public API. It could be completely clear that exposed API is actually supposed to be public.
It seems to me like that goes too far. It's quite conceivable that there might be tests outside the file (including the "paired" test file) that are still testing internal APIs.
I said "allow" for a reason though. I am not sure that everyone will want this to be the case. Some people will want to place more outside of the file with an eye toward file length or some other concern that I am not considering. My real point isn't that everyone will do this but that we cannot, at present, do this.
Yes, it is a big argument in testing, but I find there to be a lot more credence with the "don't test private methods" camp. Swift is, in my eyes and from its history, an opinionated language and it should try to enforce the better solution for developers. The language should not cater to poor practices. More on why I find this to be a poor practice below.
I believe that was an overexaggeration on my part; I mean more of a code smell or bad practice, my bad. I jumped to architecture initially because testing private methods is generally duplication and causes more of a maintenance overhead. That sort of issue makes me jump to architecture issues. See below.
Testing private methods is a waste of time and effort. If your interface isn't hitting all parts of the classes internals, those internals can just be removed. Thorough testing of the interface should hit all aspects of the private methods. I've seen a lot of people compare cars and other real-world objects to try to rationalize testing private methods, but this is software, and I don't believe those arguments have any merit.
I can understand people wanting to test more individual components while building out the class, but in those cases, making those methods testable has no harm, and as the developer refactors the implementation they can hide away those details and move the test assertions to act on the interface method(s) that hits those internals while making them private.
The purpose of the committed tests is to validate behavior and give confidence that changes will not alter the intended behavior when developers make changes in the future, allowing for faster delivery of value to the customer. The best path to this is writing as little test code as necessary to validate the intended behavior. There is less overhead when initially writing the test, and when maintaining it, if we're only testing the interfaces of a class.
@Joe_Groff brings up a good concern about reducing temptation to lower access levels. However, I think its better for developers to think about their application design instead of just testing the internals. I feel like they could improve the design to some degree instead. I'm unconvinced if there's really a win with keeping private untestable vs opening them up in this regard.
For me, testing non-public methods (like @testable) simply makes it easier to set up the tests, not having to go through the public interface that usually constrains the usage. I can understand your position, though.
I think the ability to test internal methods is also okay for app targets that donât need to be decorated with public or open access modifiers 
Currenly there is no real meaning to open or public in app targets which makes internal the default for everything that is supposed to be public API. This exact situation makes it impossible to encapsulate code as intended and forces one to fallback to (file)private, which is exactly the case why we want to be able to test private members. Furthermore I don't think it's bad nor it's a waste of time and effort to allow someone who wants to test the private API to do so.
Exposing private API or refactoring it in way that can be tested by for instance creating some internal functions that can be tested is solely meant for reusability. When there is nothing to reuse, it doesn't have to be exposed nor moved somewhere else, which circles us back to the point that we don't have a good way to test that part of our code yet. You may want to test private procotol extensions, but you cannot even if you extend a Mock type because you don't have any access to that code.
The desire to test internals is not, intrinsically, a reflection of how well you have crafted your public API. "I have a small piece of code which is crucial and I need to capture what it's behavior should be in certain representative or specific cases" is entirely reasonable even if the piece of code in question is not exposed directly to the outside.
Stated as simply as I can manage:
Black box testing (for me) is meant to capture the promises that I want to make to external clients.
Glass box testing (for me) is meant to capture the promises that I want to make to myself.
Every time I write tests Iâm wishing Swift had this. Itâs an absolutely awesome way to test code. Itâs super convenient and helps document the code base at the same time.
+100 from me.
Regarding access control, you can still put these tests in different files and modules so I donât see what the big deal is there.