For discussion: Including source code alongside binaries

Hello everyone,

I would like to start a discussion on the possibility of enabling
compelling Swift binaries that embed the source code from which they've
been compiled.

The value of having source code embedded alongside the binary is debugging
aid - A developer will be able to download 3rd party framework, include it
into his Xcode project and then drill down in a debug session into the
framework code, having Xcode showing him the source code from which the
current breakpoint instruction was built.

This feature exists in the java world where you can compile a Jar file,
having each .class contain it's .java counter part. I //think// it's stored
in this property
Chapter 4. The class File Format.
This allows opening the Jar of a target library in eclipse and instantly
seeing what the code is doing.

The alternatives today are either decompilers, or .dSYM both are good but
not great as each requires manually steps in either management or enforcing
corrects of the results.

Would the community be interested in such a "plug and play" approach for
attaching source code to build frameworks / applications / stand alone
binaries ?

Is it feasible to expect such a thing from native apps? if not, can it be
an Xcode level feature when only .packages (Framework/Application) will be
added support for shipping the .dSYM and Xcode will know to do the lifting
to make it just work for the developer?

tnx,
-m

I don’t understand that — I would think it would be up to the development (IDE) environment. Swift code is open-source and can be installed, without bundling (assuming the IDE can handle it). Just have to make sure that there is a source package that corresponds to a given release that could be downloaded.

For the most part it would be a function of the development environment, not of the Swift language.

···

On 2016-02-13, at 5:35:21, Maxim Veksler via swift-evolution <swift-evolution@swift.org> wrote:

Hello everyone,

I would like to start a discussion on the possibility of enabling compelling Swift binaries that embed the source code from which they've been compiled.

The value of having source code embedded alongside the binary is debugging aid - A developer will be able to download 3rd party framework, include it into his Xcode project and then drill down in a debug session into the framework code, having Xcode showing him the source code from which the current breakpoint instruction was built.

This feature exists in the java world where you can compile a Jar file, having each .class contain it's .java counter part. I //think// it's stored in this property Chapter 4. The class File Format. This allows opening the Jar of a target library in eclipse and instantly seeing what the code is doing.

The alternatives today are either decompilers, or .dSYM both are good but not great as each requires manually steps in either management or enforcing corrects of the results.

Would the community be interested in such a "plug and play" approach for attaching source code to build frameworks / applications / stand alone binaries ?

Is it feasible to expect such a thing from native apps? if not, can it be an Xcode level feature when only .packages (Framework/Application) will be added support for shipping the .dSYM and Xcode will know to do the lifting to make it just work for the developer?

tnx,
-m
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Why not use _just_ source code and let Xcode / Package Manager to do the job of building?

Or you to want have binary distribution and source code for debug?

- Alexey Demedetskiy

···

Hello everyone,

I would like to start a discussion on the possibility of enabling compelling Swift binaries that embed the source code from which they've been compiled.

The value of having source code embedded alongside the binary isdebugging aid - A developer will be able to download 3rd party framework, include it into his Xcode project and then drill down in a debug session into the framework code, having Xcode showing him the source code from which the current breakpoint instruction was built.

This feature exists in the java world where you can compile a Jar file, having each .class contain it's .java counter part. I //think// it's stored in this propertyhttps://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html#jvms-4.7.11. This allows opening the Jar of a target library in eclipse and instantly seeing what the code is doing.

The alternatives today are either decompilers, or.dSYM both are good but not great as each requires manually steps in either management or enforcing corrects of the results.

Would the community be interested in such a "plug and play" approach for attaching source code to build frameworks / applications / stand alone binaries ?

Is it feasible to expect such a thing from native apps? if not, can it be an Xcode level feature when only .packages (Framework/Application) will be added support for shipping the .dSYM and Xcode will know to do the lifting to make it just work for the developer?

tnx,
-m

Hello everyone,

I would like to start a discussion on the possibility of enabling compelling Swift binaries that embed the source code from which they've been compiled.

The value of having source code embedded alongside the binary is debugging aid - A developer will be able to download 3rd party framework, include it into his Xcode project and then drill down in a debug session into the framework code, having Xcode showing him the source code from which the current breakpoint instruction was built.

This feature exists in the java world where you can compile a Jar file, having each .class contain it's .java counter part. I //think// it's stored in this property Chapter 4. The class File Format. This allows opening the Jar of a target library in eclipse and instantly seeing what the code is doing.

The alternatives today are either decompilers, or .dSYM both are good but not great as each requires manually steps in either management or enforcing corrects of the results.

dsym is not an alternative and don’t provide the sources. It is just a bundle of debug symbols, which are information required to map instruction addresses to function name and source code line. Having the sources without dsym would be mostly useless. You won’t be able to put symbolic breakpoint or source location breakpoint, and won’t be able to get anything useful from the debugger but assembly.

···

Le 12 févr. 2016 à 23:35, Maxim Veksler via swift-evolution <swift-evolution@swift.org> a écrit :

Would the community be interested in such a "plug and play" approach for attaching source code to build frameworks / applications / stand alone binaries ?

Is it feasible to expect such a thing from native apps? if not, can it be an Xcode level feature when only .packages (Framework/Application) will be added support for shipping the .dSYM and Xcode will know to do the lifting to make it just work for the developer?

tnx,
-m
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

That's a good point, we are post distributed version control era.

If I understand your suggestion this means that each binary / framework
should have in it's meta data a path leading to a git (or other) source
repository and a pointer to the exact hash / tag of the commit that
produced the given binary. This would allow IDE automation to clone the
given repo automatically, and display the source during the debugging
session.

I think that this introduces a problem for developers working in
environments that have no direct internet access (gov, army, labs) but work
around could be implemented.

···

On Sat, Feb 13, 2016 at 12:39 AM, Craig Cruden <ccruden@novafore.com> wrote:

don’t understand that — I would think it would be up to the development
(IDE) environment. Swift code is open-source and can be installed, without
bundling (assuming the IDE can handle it). Just have to make sure that
there is a source package that corresponds to a given release that could be
downloaded

--
puıɯ ʎɯ ɯoɹɟ ʇuǝs

Hey Alexey,

Imagine being able to add RxSwift into your project (either via manually,
cocoa pods, carthage, swift package manager or otherwise) and having the
seamless experiences of being able to drill down in a debugging session
into the code of the framework, looking for why it does something you did
not expect.

This experience is very valuable time saver, because today should you wish
to reach this point of being able to drill down you need to:

1. Find the source code repository
2. clone it locally
3. checkout to the correct version / patch
4. add the framework source code into your app
5. fix any build problems
6. remove the binary framework you already have included.
7. run the debugging session and hope to be able to reproduce your bug
7.1 let Xcode compile the framework from source -- time consuming, without
value task.
8. Now you need to choose if you leave the framework in source form or
starting changing your app back into binary form.

So I think a complete automation is possible, but it does require many
moving parts which is why I think Swift poses a great opportunity of making
this leap.

Would be glad to hear your feedback.

···

On Sat, Feb 13, 2016 at 10:20 AM, Alexey Demedetskiy <dalog@me.com> wrote:

Why not use _just_ source code and let Xcode / Package Manager to do the
job of building?

Or you to want have binary distribution and source code for debug?

- Alexey Demedetskiy

> Hello everyone,
>
> I would like to start a discussion on the possibility of enabling
compelling Swift binaries that embed the source code from which they've
been compiled.
>
> The value of having source code embedded alongside the binary
isdebugging aid - A developer will be able to download 3rd party framework,
include it into his Xcode project and then drill down in a debug session
into the framework code, having Xcode showing him the source code from
which the current breakpoint instruction was built.
>
> This feature exists in the java world where you can compile a Jar file,
having each .class contain it's .java counter part. I //think// it's stored
in this propertyhttps://
docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html#jvms-4.7.11. This
allows opening the Jar of a target library in eclipse and instantly seeing
what the code is doing.
>
> The alternatives today are either decompilers, or.dSYM both are good but
not great as each requires manually steps in either management or enforcing
corrects of the results.
>
> Would the community be interested in such a "plug and play" approach for
attaching source code to build frameworks / applications / stand alone
binaries ?
>
> Is it feasible to expect such a thing from native apps? if not, can it
be an Xcode level feature when only .packages (Framework/Application) will
be added support for shipping the .dSYM and Xcode will know to do the
lifting to make it just work for the developer?
>
>
> tnx,
> -m
>
>
>
>

--
puıɯ ʎɯ ɯoɹɟ ʇuǝs

Hey Maxim,

I understand your point. This kind of behavior is ideal development workflow, and we should move our package management systems in this direction.

Including source code into binary will do the job, however, we should consider different approaches.

For example:

1) Inside your project you specify RxSwift dependency.
2) During build phase, IDE / compiler will download sources.
3) Sources are stored in system cache, independent from project tree. Sources are read only.
4) RxSwift sources compilation.
5) Resulting RXSwift framework is stored in system cache.
6) Project is build using resulting framework from cache.
7) During debug, exact sources can be displayed to developer.

Advantages of this approach:
1) No need to keep sources or binaries in project repo.
2) IDE / compiler can reuse resulting binaries.
3) Full debug support.

To be honest, this is exactly what you are suppose to do. But on infrastructure level instead of implementation level.
I believe we can achieve this behavior using combination of small improvements into existing infrastructure,
instead of adding more complexity into compiler.

···

14 февр. 2016 г., в 12:12, Maxim Veksler <maxim@vekslers.org> написал(а):

Hey Alexey,

Imagine being able to add RxSwift into your project (either via manually, cocoa pods, carthage, swift package manager or otherwise) and having the seamless experiences of being able to drill down in a debugging session into the code of the framework, looking for why it does something you did not expect.

This experience is very valuable time saver, because today should you wish to reach this point of being able to drill down you need to:

1. Find the source code repository
2. clone it locally
3. checkout to the correct version / patch
4. add the framework source code into your app
5. fix any build problems
6. remove the binary framework you already have included.
7. run the debugging session and hope to be able to reproduce your bug
7.1 let Xcode compile the framework from source -- time consuming, without value task.
8. Now you need to choose if you leave the framework in source form or starting changing your app back into binary form.

So I think a complete automation is possible, but it does require many moving parts which is why I think Swift poses a great opportunity of making this leap.

Would be glad to hear your feedback.

On Sat, Feb 13, 2016 at 10:20 AM, Alexey Demedetskiy <dalog@me.com <mailto:dalog@me.com>> wrote:
Why not use _just_ source code and let Xcode / Package Manager to do the job of building?

Or you to want have binary distribution and source code for debug?

- Alexey Demedetskiy

> Hello everyone,
>
> I would like to start a discussion on the possibility of enabling compelling Swift binaries that embed the source code from which they've been compiled.
>
> The value of having source code embedded alongside the binary isdebugging aid - A developer will be able to download 3rd party framework, include it into his Xcode project and then drill down in a debug session into the framework code, having Xcode showing him the source code from which the current breakpoint instruction was built.
>
> This feature exists in the java world where you can compile a Jar file, having each .class contain it's .java counter part. I //think// it's stored in this propertyhttps://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html#jvms-4.7.11 <Chapter 4. The class File Format. This allows opening the Jar of a target library in eclipse and instantly seeing what the code is doing.
>
> The alternatives today are either decompilers, or.dSYM both are good but not great as each requires manually steps in either management or enforcing corrects of the results.
>
> Would the community be interested in such a "plug and play" approach for attaching source code to build frameworks / applications / stand alone binaries ?
>
> Is it feasible to expect such a thing from native apps? if not, can it be an Xcode level feature when only .packages (Framework/Application) will be added support for shipping the .dSYM and Xcode will know to do the lifting to make it just work for the developer?
>
>
> tnx,
> -m
>
>
>
>

--
puıɯ ʎɯ ɯoɹɟ ʇuǝs