Communicating with dynamically loaded swift library

Hello folks,

I have been toying with dynamic libraries, trying to implement plugin
functionality. I was able to get to the point where I can call simple
function in loaded library, but I am having troubles starting more
sophisticated communication channel.

There are 3 projects
- PluginConsumer is an app that loads plugin libraries
- MyPlugin is a plugin implementation, output is dynamic library that
PluginConsumer loads
- PluginInterface is common interface that both MyPlugin and PluginConsumer
use, so that they know how to communicate

My first idea was to have PluginInterface be a simple SPM project with
single file where the bare-bones PluginInterface class would be:

open class PluginInterface {

    open func sayHi()

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginInterface",

    products: [ .library(name: "PluginInterface", type: .dynamic, targets: [
"PluginInterface"]) ],

    targets: [ .target(name: "PluginInterface") ]

)

UserPlugin is also very simple project containing only one file:

public func getPlugin() -> AnyObject {

    return MyPlugin()

}

class MyPlugin: PluginInterface {

    override func sayHi() {

        print("Hi from my plugin")

    }

}

Package.swift:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "MyPlugin",

    products: [ .library(name: "MyPlugin", type: .dynamic, targets: [
"MyPlugin"]) ],

    dependencies: [ .package(url: "url_to_PluginInterface", from: "0.0.0"),
],

    targets: [

        .target(name: "PluginInterface", dependencies: ["PluginInterface"]),

        .target(name: "MyPlugin", dependencies: ["PluginInterface"]),

    ]

)

The PluginConsumer is bit more complicated, but here is relevant part (lib
loading and function calling):

typealias InitFunction = @convention(c) () -> AnyObject

let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL)

if openRes != nil {

    defer {

        dlclose(openRes)

    }

    let symbolName = "mangled_symbol_name"

    let sym = dlsym(openRes, symbolName)

    if sym != nil {

        let f: InitFunction = unsafeBitCast(sym, to: InitFunction.self)

        let plugin = f() as? PluginInterface

    }

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginConsumer",

    dependencies: [ .package(url: "path_to_plugin_interface", from: "0.0.0"
) ],

    targets: [ .target(name: "PluginConsumer", dependencies: [
"PluginConsumer"]) ]

)

This all compiles nicely, MyPlugin project creates dylib file that
executable created by PluginConsumer can load, but the problem is with
following line:

let plugin = f() as? PluginInterface

Type of the plugin is MyPlugin, but from the consumer's view, it doesn't
inherit from PluginInterface so I can't call sayHi() method. I assume this
is because there is no relation between PluginInterface class that compiler
uses for MyPlugin project one that it uses for PluginConsumer project.
After library is loaded, they are two completely different classes that
happen to share same name. Is my assumption correct and how do I go about
fixing it?

I had an idea I could make PluginInterface emit dynamic library that would
be dynamically linked by both MyPlugin and PluginConsumer, thus making them
share same PluginInterface class, but I can't figure out how to do that (or
if it's right way of doing this).

Any help appreciated :)

Lope

The way that I have done this in the past is pass a protocol as an unsafe pointer to an exposed entry point:

            let entryPoint = dlsym(handle, “initializePlugin”)
            guard entryPoint != nil else {
                fatalError("missing plugin entry point: \(pluginPath)")
            }
            typealias PluginInitializationFunc = @convention(c) (UnsafeRawPointer) -> ()
            let f = unsafeBitCast(entryPoint, to: PluginInitializationFunc.self)
            f(Unmanaged.passUnretained(self).toOpaque())

and then in the plugin convert back to the appropriate type:

@_cdecl("initializePlugin")
public func initializePlugin(_ ptr: UnsafeRawPointer) {
    let manager = Unmanaged<PluginManager>.fromOpaque(ptr).takeUnretainedValue()

HTH,
- Daniel

···

On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift-users <swift-users@swift.org> wrote:

Hello folks,

I have been toying with dynamic libraries, trying to implement plugin functionality. I was able to get to the point where I can call simple function in loaded library, but I am having troubles starting more sophisticated communication channel.

There are 3 projects
- PluginConsumer is an app that loads plugin libraries
- MyPlugin is a plugin implementation, output is dynamic library that PluginConsumer loads
- PluginInterface is common interface that both MyPlugin and PluginConsumer use, so that they know how to communicate

My first idea was to have PluginInterface be a simple SPM project with single file where the bare-bones PluginInterface class would be:

open class PluginInterface {

    open func sayHi()

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginInterface",

    products: [ .library(name: "PluginInterface", type: .dynamic, targets: ["PluginInterface"]) ],

    targets: [ .target(name: "PluginInterface") ]

)

UserPlugin is also very simple project containing only one file:

public func getPlugin() -> AnyObject {

    return MyPlugin()

}

class MyPlugin: PluginInterface {

    override func sayHi() {

        print("Hi from my plugin")

    }

}

Package.swift:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "MyPlugin",

    products: [ .library(name: "MyPlugin", type: .dynamic, targets: ["MyPlugin"]) ],

    dependencies: [ .package(url: "url_to_PluginInterface", from: "0.0.0"), ],

    targets: [

        .target(name: "PluginInterface", dependencies: ["PluginInterface"]),

        .target(name: "MyPlugin", dependencies: ["PluginInterface"]),

    ]

)

The PluginConsumer is bit more complicated, but here is relevant part (lib loading and function calling):

typealias InitFunction = @convention(c) () -> AnyObject

let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL)

if openRes != nil {

    defer {

        dlclose(openRes)

    }

    let symbolName = "mangled_symbol_name"

    let sym = dlsym(openRes, symbolName)

    if sym != nil {

        let f: InitFunction = unsafeBitCast(sym, to: InitFunction.self)

        let plugin = f() as? PluginInterface

    }

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginConsumer",

    dependencies: [ .package(url: "path_to_plugin_interface", from: "0.0.0") ],

    targets: [ .target(name: "PluginConsumer", dependencies: ["PluginConsumer"]) ]

)

This all compiles nicely, MyPlugin project creates dylib file that executable created by PluginConsumer can load, but the problem is with following line:

let plugin = f() as? PluginInterface

Type of the plugin is MyPlugin, but from the consumer's view, it doesn't inherit from PluginInterface so I can't call sayHi() method. I assume this is because there is no relation between PluginInterface class that compiler uses for MyPlugin project one that it uses for PluginConsumer project. After library is loaded, they are two completely different classes that happen to share same name. Is my assumption correct and how do I go about fixing it?

I had an idea I could make PluginInterface emit dynamic library that would be dynamically linked by both MyPlugin and PluginConsumer, thus making them share same PluginInterface class, but I can't figure out how to do that (or if it's right way of doing this).

Any help appreciated :)

Lope

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

It sounds like that may be the case. Class names are namespaced to their module. If you're literally including the same PluginInterface.swift file in both of your modules, then you're going to end up with a ModuleA.PluginInterface and ModuleB.PluginInterface class. If you built PluginInterface as its own module, and imported the same module from your plugin and host projects, then you should end up with one common class.

-Joe

···

On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift-users <swift-users@swift.org> wrote:

Hello folks,

I have been toying with dynamic libraries, trying to implement plugin functionality. I was able to get to the point where I can call simple function in loaded library, but I am having troubles starting more sophisticated communication channel.

There are 3 projects
- PluginConsumer is an app that loads plugin libraries
- MyPlugin is a plugin implementation, output is dynamic library that PluginConsumer loads
- PluginInterface is common interface that both MyPlugin and PluginConsumer use, so that they know how to communicate

My first idea was to have PluginInterface be a simple SPM project with single file where the bare-bones PluginInterface class would be:

open class PluginInterface {

    open func sayHi()

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginInterface",

    products: [ .library(name: "PluginInterface", type: .dynamic, targets: ["PluginInterface"]) ],

    targets: [ .target(name: "PluginInterface") ]

)

UserPlugin is also very simple project containing only one file:

public func getPlugin() -> AnyObject {

    return MyPlugin()

}

class MyPlugin: PluginInterface {

    override func sayHi() {

        print("Hi from my plugin")

    }

}

Package.swift:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "MyPlugin",

    products: [ .library(name: "MyPlugin", type: .dynamic, targets: ["MyPlugin"]) ],

    dependencies: [ .package(url: "url_to_PluginInterface", from: "0.0.0"), ],

    targets: [

        .target(name: "PluginInterface", dependencies: ["PluginInterface"]),

        .target(name: "MyPlugin", dependencies: ["PluginInterface"]),

    ]

)

The PluginConsumer is bit more complicated, but here is relevant part (lib loading and function calling):

typealias InitFunction = @convention(c) () -> AnyObject

let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL)

if openRes != nil {

    defer {

        dlclose(openRes)

    }

    let symbolName = "mangled_symbol_name"

    let sym = dlsym(openRes, symbolName)

    if sym != nil {

        let f: InitFunction = unsafeBitCast(sym, to: InitFunction.self)

        let plugin = f() as? PluginInterface

    }

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginConsumer",

    dependencies: [ .package(url: "path_to_plugin_interface", from: "0.0.0") ],

    targets: [ .target(name: "PluginConsumer", dependencies: ["PluginConsumer"]) ]

)

This all compiles nicely, MyPlugin project creates dylib file that executable created by PluginConsumer can load, but the problem is with following line:

let plugin = f() as? PluginInterface

Type of the plugin is MyPlugin, but from the consumer's view, it doesn't inherit from PluginInterface so I can't call sayHi() method. I assume this is because there is no relation between PluginInterface class that compiler uses for MyPlugin project one that it uses for PluginConsumer project. After library is loaded, they are two completely different classes that happen to share same name. Is my assumption correct and how do I go about fixing it?

It worked! Took me a while to iron out details, but it is working now. Huge
thanks sir, I will name my firstborn after you.
Thanks for the @_cdecl("initializePlugin") tip as well, I didn't know about
it and it will be very useful.

I am having slightly related problem now (it was there before, but I
ignored it for the time being), not sure if I should start a new thread?

The PluginInterface module has one external dependency on module A,
PluginConsumer has the dependency on module B which has dependency on same
module A that the PluginInterface uses. When I load the plugin library, I
get bunch of errors like:

Class A.XYZ is implemented in both libPluginInterface.dylib and
libMyPlugin.dylib

I know why it is there, but I don't know how to get rid of it. I can't just
remove dependency from PluginConsumer and use the one from PluginInterface
(if that would even work?) because PluginConsumer does not depend on it
directly, but it is going through module B first

Cheers,
Lope

···

On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

The way that I have done this in the past is pass a protocol as an unsafe
pointer to an exposed entry point:

            let entryPoint = dlsym(handle, “initializePlugin”)
            guard entryPoint != nil else {
                fatalError("missing plugin entry point: \(pluginPath)")
            }
            typealias PluginInitializationFunc = @convention(c)
(UnsafeRawPointer) -> ()
            let f = unsafeBitCast(entryPoint, to:
PluginInitializationFunc.self)
            f(Unmanaged.passUnretained(self).toOpaque())

and then in the plugin convert back to the appropriate type:

@_cdecl("initializePlugin")
public func initializePlugin(_ ptr: UnsafeRawPointer) {
    let manager = Unmanaged<PluginManager>.fromOpaque(ptr).
takeUnretainedValue()

HTH,
- Daniel

On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift-users < > swift-users@swift.org> wrote:

Hello folks,

I have been toying with dynamic libraries, trying to implement plugin
functionality. I was able to get to the point where I can call simple
function in loaded library, but I am having troubles starting more
sophisticated communication channel.

There are 3 projects
- PluginConsumer is an app that loads plugin libraries
- MyPlugin is a plugin implementation, output is dynamic library that
PluginConsumer loads
- PluginInterface is common interface that both MyPlugin and
PluginConsumer use, so that they know how to communicate

My first idea was to have PluginInterface be a simple SPM project with
single file where the bare-bones PluginInterface class would be:

open class PluginInterface {

    open func sayHi()

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginInterface",

    products: [ .library(name: "PluginInterface", type: .dynamic,
targets: ["PluginInterface"]) ],

    targets: [ .target(name: "PluginInterface") ]

)

UserPlugin is also very simple project containing only one file:

public func getPlugin() -> AnyObject {

    return MyPlugin()

}

class MyPlugin: PluginInterface {

    override func sayHi() {

        print("Hi from my plugin")

    }

}

Package.swift:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "MyPlugin",

    products: [ .library(name: "MyPlugin", type: .dynamic, targets: [
"MyPlugin"]) ],

    dependencies: [ .package(url: "url_to_PluginInterface", from: "0.0.0"),
],

    targets: [

        .target(name: "PluginInterface", dependencies: ["PluginInterface"
]),

        .target(name: "MyPlugin", dependencies: ["PluginInterface"]),

    ]

)

The PluginConsumer is bit more complicated, but here is relevant part (lib
loading and function calling):

typealias InitFunction = @convention(c) () -> AnyObject

let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL)

if openRes != nil {

    defer {

        dlclose(openRes)

    }

    let symbolName = "mangled_symbol_name"

    let sym = dlsym(openRes, symbolName)

    if sym != nil {

        let f: InitFunction = unsafeBitCast(sym, to: InitFunction.self)

        let plugin = f() as? PluginInterface

    }

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginConsumer",

    dependencies: [ .package(url: "path_to_plugin_interface", from:
"0.0.0") ],

    targets: [ .target(name: "PluginConsumer", dependencies: [
"PluginConsumer"]) ]

)

This all compiles nicely, MyPlugin project creates dylib file that
executable created by PluginConsumer can load, but the problem is with
following line:

let plugin = f() as? PluginInterface

Type of the plugin is MyPlugin, but from the consumer's view, it doesn't
inherit from PluginInterface so I can't call sayHi() method. I assume this
is because there is no relation between PluginInterface class that compiler
uses for MyPlugin project one that it uses for PluginConsumer project.
After library is loaded, they are two completely different classes that
happen to share same name. Is my assumption correct and how do I go about
fixing it?

I had an idea I could make PluginInterface emit dynamic library that would
be dynamically linked by both MyPlugin and PluginConsumer, thus making them
share same PluginInterface class, but I can't figure out how to do that (or
if it's right way of doing this).

Any help appreciated :)

Lope
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

I think SwiftPM is (incorrectly) compiling A.XYZ into each of the modules
that depend on it, as well as into your intended libA.so file.

Try to ensure the plugin provider module (libA) is (only) being compiled
into its standalone shared library file. Try cleaning the swiftpm build for
one (swift package clean) and ensure the Package.swift files are correctly
set up to output the shared library.

Sorry I can’t be more specific, I’ve had these same kinds of issues before
but I’m not 100% what they were.

Geordie

···

Ján Kosa via swift-users <swift-users@swift.org> schrieb am Fr. 6. Okt. 2017 um 14:41:

It worked! Took me a while to iron out details, but it is working now.
Huge thanks sir, I will name my firstborn after you.
Thanks for the @_cdecl("initializePlugin") tip as well, I didn't know
about it and it will be very useful.

I am having slightly related problem now (it was there before, but I
ignored it for the time being), not sure if I should start a new thread?

The PluginInterface module has one external dependency on module A,
PluginConsumer has the dependency on module B which has dependency on same
module A that the PluginInterface uses. When I load the plugin library, I
get bunch of errors like:

Class A.XYZ is implemented in both libPluginInterface.dylib and
libMyPlugin.dylib

I know why it is there, but I don't know how to get rid of it. I can't
just remove dependency from PluginConsumer and use the one from
PluginInterface (if that would even work?) because PluginConsumer does not
depend on it directly, but it is going through module B first

Cheers,
Lope

On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

The way that I have done this in the past is pass a protocol as an unsafe
pointer to an exposed entry point:

            let entryPoint = dlsym(handle, “initializePlugin”)
            guard entryPoint != nil else {
                fatalError("missing plugin entry point: \(pluginPath)")
            }
            typealias PluginInitializationFunc = @convention(c)
(UnsafeRawPointer) -> ()
            let f = unsafeBitCast(entryPoint, to:
PluginInitializationFunc.self)
            f(Unmanaged.passUnretained(self).toOpaque())

and then in the plugin convert back to the appropriate type:

@_cdecl("initializePlugin")
public func initializePlugin(_ ptr: UnsafeRawPointer) {
    let manager =
Unmanaged<PluginManager>.fromOpaque(ptr).takeUnretainedValue()

HTH,
- Daniel

On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift-users < >> swift-users@swift.org> wrote:

Hello folks,

I have been toying with dynamic libraries, trying to implement plugin
functionality. I was able to get to the point where I can call simple
function in loaded library, but I am having troubles starting more
sophisticated communication channel.

There are 3 projects
- PluginConsumer is an app that loads plugin libraries
- MyPlugin is a plugin implementation, output is dynamic library that
PluginConsumer loads
- PluginInterface is common interface that both MyPlugin and
PluginConsumer use, so that they know how to communicate

My first idea was to have PluginInterface be a simple SPM project with
single file where the bare-bones PluginInterface class would be:

open class PluginInterface {

    open func sayHi()

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginInterface",

    products: [ .library(name: "PluginInterface", type: .dynamic,
targets: ["PluginInterface"]) ],

    targets: [ .target(name: "PluginInterface") ]

)

UserPlugin is also very simple project containing only one file:

public func getPlugin() -> AnyObject {

    return MyPlugin()

}

class MyPlugin: PluginInterface {

    override func sayHi() {

        print("Hi from my plugin")

    }

}

Package.swift:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "MyPlugin",

    products: [ .library(name: "MyPlugin", type: .dynamic, targets: [
"MyPlugin"]) ],

    dependencies: [ .package(url: "url_to_PluginInterface", from: "0.0.0"),
],

    targets: [

        .target(name: "PluginInterface", dependencies: ["PluginInterface"
]),

        .target(name: "MyPlugin", dependencies: ["PluginInterface"]),

    ]

)

The PluginConsumer is bit more complicated, but here is relevant part
(lib loading and function calling):

typealias InitFunction = @convention(c) () -> AnyObject

let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL)

if openRes != nil {

    defer {

        dlclose(openRes)

    }

    let symbolName = "mangled_symbol_name"

    let sym = dlsym(openRes, symbolName)

    if sym != nil {

        let f: InitFunction = unsafeBitCast(sym, to: InitFunction.self)

        let plugin = f() as? PluginInterface

    }

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginConsumer",

    dependencies: [ .package(url: "path_to_plugin_interface", from:
"0.0.0") ],

    targets: [ .target(name: "PluginConsumer", dependencies: [
"PluginConsumer"]) ]

)

This all compiles nicely, MyPlugin project creates dylib file that
executable created by PluginConsumer can load, but the problem is with
following line:

let plugin = f() as? PluginInterface

Type of the plugin is MyPlugin, but from the consumer's view, it doesn't
inherit from PluginInterface so I can't call sayHi() method. I assume this
is because there is no relation between PluginInterface class that compiler
uses for MyPlugin project one that it uses for PluginConsumer project.
After library is loaded, they are two completely different classes that
happen to share same name. Is my assumption correct and how do I go about
fixing it?

I had an idea I could make PluginInterface emit dynamic library that
would be dynamically linked by both MyPlugin and PluginConsumer, thus
making them share same PluginInterface class, but I can't figure out how to
do that (or if it's right way of doing this).

Any help appreciated :)

Lope
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

I tried swift package clean, but it didn't help

"Try to ensure the plugin provider module (libA) is (only) being compiled
into its standalone shared library file."
How do I go about this? It is 3rd party module, it doesn't define any
products (https://github.com/apple/swift-protobuf.git\). Is there something
I can do in my Package files to make sure it is loaded dynamically?

···

On 6 October 2017 at 22:52, Geordie Jay <geojay@gmail.com> wrote:

I think SwiftPM is (incorrectly) compiling A.XYZ into each of the modules
that depend on it, as well as into your intended libA.so file.

Try to ensure the plugin provider module (libA) is (only) being compiled
into its standalone shared library file. Try cleaning the swiftpm build for
one (swift package clean) and ensure the Package.swift files are correctly
set up to output the shared library.

Sorry I can’t be more specific, I’ve had these same kinds of issues before
but I’m not 100% what they were.

Geordie

Ján Kosa via swift-users <swift-users@swift.org> schrieb am Fr. 6. Okt. > 2017 um 14:41:

It worked! Took me a while to iron out details, but it is working now.
Huge thanks sir, I will name my firstborn after you.
Thanks for the @_cdecl("initializePlugin") tip as well, I didn't know
about it and it will be very useful.

I am having slightly related problem now (it was there before, but I
ignored it for the time being), not sure if I should start a new thread?

The PluginInterface module has one external dependency on module A,
PluginConsumer has the dependency on module B which has dependency on same
module A that the PluginInterface uses. When I load the plugin library, I
get bunch of errors like:

Class A.XYZ is implemented in both libPluginInterface.dylib and
libMyPlugin.dylib

I know why it is there, but I don't know how to get rid of it. I can't
just remove dependency from PluginConsumer and use the one from
PluginInterface (if that would even work?) because PluginConsumer does not
depend on it directly, but it is going through module B first

Cheers,
Lope

On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dunbar@apple.com> >> wrote:

The way that I have done this in the past is pass a protocol as an
unsafe pointer to an exposed entry point:

            let entryPoint = dlsym(handle, “initializePlugin”)
            guard entryPoint != nil else {
                fatalError("missing plugin entry point: \(pluginPath)")
            }
            typealias PluginInitializationFunc = @convention(c)
(UnsafeRawPointer) -> ()
            let f = unsafeBitCast(entryPoint, to:
PluginInitializationFunc.self)
            f(Unmanaged.passUnretained(self).toOpaque())

and then in the plugin convert back to the appropriate type:

@_cdecl("initializePlugin")
public func initializePlugin(_ ptr: UnsafeRawPointer) {
    let manager = Unmanaged<PluginManager>.fromOpaque(ptr).
takeUnretainedValue()

HTH,
- Daniel

On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift-users < >>> swift-users@swift.org> wrote:

Hello folks,

I have been toying with dynamic libraries, trying to implement plugin
functionality. I was able to get to the point where I can call simple
function in loaded library, but I am having troubles starting more
sophisticated communication channel.

There are 3 projects
- PluginConsumer is an app that loads plugin libraries
- MyPlugin is a plugin implementation, output is dynamic library that
PluginConsumer loads
- PluginInterface is common interface that both MyPlugin and
PluginConsumer use, so that they know how to communicate

My first idea was to have PluginInterface be a simple SPM project with
single file where the bare-bones PluginInterface class would be:

open class PluginInterface {

    open func sayHi()

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginInterface",

    products: [ .library(name: "PluginInterface", type: .dynamic,
targets: ["PluginInterface"]) ],

    targets: [ .target(name: "PluginInterface") ]

)

UserPlugin is also very simple project containing only one file:

public func getPlugin() -> AnyObject {

    return MyPlugin()

}

class MyPlugin: PluginInterface {

    override func sayHi() {

        print("Hi from my plugin")

    }

}

Package.swift:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "MyPlugin",

    products: [ .library(name: "MyPlugin", type: .dynamic, targets: [
"MyPlugin"]) ],

    dependencies: [ .package(url: "url_to_PluginInterface", from:
"0.0.0"), ],

    targets: [

        .target(name: "PluginInterface", dependencies: [
"PluginInterface"]),

        .target(name: "MyPlugin", dependencies: ["PluginInterface"]),

    ]

)

The PluginConsumer is bit more complicated, but here is relevant part
(lib loading and function calling):

typealias InitFunction = @convention(c) () -> AnyObject

let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL)

if openRes != nil {

    defer {

        dlclose(openRes)

    }

    let symbolName = "mangled_symbol_name"

    let sym = dlsym(openRes, symbolName)

    if sym != nil {

        let f: InitFunction = unsafeBitCast(sym, to: InitFunction.self)

        let plugin = f() as? PluginInterface

    }

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginConsumer",

    dependencies: [ .package(url: "path_to_plugin_interface", from:
"0.0.0") ],

    targets: [ .target(name: "PluginConsumer", dependencies: [
"PluginConsumer"]) ]

)

This all compiles nicely, MyPlugin project creates dylib file that
executable created by PluginConsumer can load, but the problem is with
following line:

let plugin = f() as? PluginInterface

Type of the plugin is MyPlugin, but from the consumer's view, it doesn't
inherit from PluginInterface so I can't call sayHi() method. I assume this
is because there is no relation between PluginInterface class that compiler
uses for MyPlugin project one that it uses for PluginConsumer project.
After library is loaded, they are two completely different classes that
happen to share same name. Is my assumption correct and how do I go about
fixing it?

I had an idea I could make PluginInterface emit dynamic library that
would be dynamically linked by both MyPlugin and PluginConsumer, thus
making them share same PluginInterface class, but I can't figure out how to
do that (or if it's right way of doing this).

Any help appreciated :)

Lope
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

I tried swift package clean, but it didn't help

"Try to ensure the plugin provider module (libA) is (only) being compiled
into its standalone shared library file."
How do I go about this? It is 3rd party module, it doesn't define any
products (https://github.com/apple/swift-protobuf.git\). Is there
something I can do in my Package files to make sure it is loaded
dynamically?

When you compile a package depending on protobuf, all the relevant symbols
end up in your package’s library file. So here’s something you might try:

import protobuf into your own “PluginProvider” module (package), which has
a shared library product like this: ‘@_exported import Protobuf’ in some
compiled swift file. Then from the other dependent modules “import
PluginProvider” - the protobuf symbols should be available, and all from
one (nonconflicting) source.

Geordie

···

Ján Kosa <lopenka@gmail.com> schrieb am Sa. 7. Okt. 2017 um 13:34:

On 6 October 2017 at 22:52, Geordie Jay <geojay@gmail.com> wrote:

I think SwiftPM is (incorrectly) compiling A.XYZ into each of the
modules that depend on it, as well as into your intended libA.so file.

Try to ensure the plugin provider module (libA) is (only) being compiled
into its standalone shared library file. Try cleaning the swiftpm build for
one (swift package clean) and ensure the Package.swift files are correctly
set up to output the shared library.

Sorry I can’t be more specific, I’ve had these same kinds of issues
before but I’m not 100% what they were.

Geordie

Ján Kosa via swift-users <swift-users@swift.org> schrieb am Fr. 6. Okt. >> 2017 um 14:41:

It worked! Took me a while to iron out details, but it is working now.
Huge thanks sir, I will name my firstborn after you.
Thanks for the @_cdecl("initializePlugin") tip as well, I didn't know
about it and it will be very useful.

I am having slightly related problem now (it was there before, but I
ignored it for the time being), not sure if I should start a new thread?

The PluginInterface module has one external dependency on module A,
PluginConsumer has the dependency on module B which has dependency on same
module A that the PluginInterface uses. When I load the plugin library, I
get bunch of errors like:

Class A.XYZ is implemented in both libPluginInterface.dylib and
libMyPlugin.dylib

I know why it is there, but I don't know how to get rid of it. I can't
just remove dependency from PluginConsumer and use the one from
PluginInterface (if that would even work?) because PluginConsumer does not
depend on it directly, but it is going through module B first

Cheers,
Lope

On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dunbar@apple.com> >>> wrote:

The way that I have done this in the past is pass a protocol as an
unsafe pointer to an exposed entry point:

            let entryPoint = dlsym(handle, “initializePlugin”)
            guard entryPoint != nil else {
                fatalError("missing plugin entry point: \(pluginPath)")
            }
            typealias PluginInitializationFunc = @convention(c)
(UnsafeRawPointer) -> ()
            let f = unsafeBitCast(entryPoint, to:
PluginInitializationFunc.self)
            f(Unmanaged.passUnretained(self).toOpaque())

and then in the plugin convert back to the appropriate type:

@_cdecl("initializePlugin")
public func initializePlugin(_ ptr: UnsafeRawPointer) {
    let manager =
Unmanaged<PluginManager>.fromOpaque(ptr).takeUnretainedValue()

HTH,
- Daniel

On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift-users < >>>> swift-users@swift.org> wrote:

Hello folks,

I have been toying with dynamic libraries, trying to implement plugin
functionality. I was able to get to the point where I can call simple
function in loaded library, but I am having troubles starting more
sophisticated communication channel.

There are 3 projects
- PluginConsumer is an app that loads plugin libraries
- MyPlugin is a plugin implementation, output is dynamic library that
PluginConsumer loads
- PluginInterface is common interface that both MyPlugin and
PluginConsumer use, so that they know how to communicate

My first idea was to have PluginInterface be a simple SPM project with
single file where the bare-bones PluginInterface class would be:

open class PluginInterface {

    open func sayHi()

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginInterface",

    products: [ .library(name: "PluginInterface", type: .dynamic,
targets: ["PluginInterface"]) ],

    targets: [ .target(name: "PluginInterface") ]

)

UserPlugin is also very simple project containing only one file:

public func getPlugin() -> AnyObject {

    return MyPlugin()

}

class MyPlugin: PluginInterface {

    override func sayHi() {

        print("Hi from my plugin")

    }

}

Package.swift:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "MyPlugin",

    products: [ .library(name: "MyPlugin", type: .dynamic, targets: [
"MyPlugin"]) ],

    dependencies: [ .package(url: "url_to_PluginInterface", from:
"0.0.0"), ],

    targets: [

        .target(name: "PluginInterface", dependencies: [
"PluginInterface"]),

        .target(name: "MyPlugin", dependencies: ["PluginInterface"]),

    ]

)

The PluginConsumer is bit more complicated, but here is relevant part
(lib loading and function calling):

typealias InitFunction = @convention(c) () -> AnyObject

let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL)

if openRes != nil {

    defer {

        dlclose(openRes)

    }

    let symbolName = "mangled_symbol_name"

    let sym = dlsym(openRes, symbolName)

    if sym != nil {

        let f: InitFunction = unsafeBitCast(sym, to: InitFunction.self)

        let plugin = f() as? PluginInterface

    }

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginConsumer",

    dependencies: [ .package(url: "path_to_plugin_interface", from:
"0.0.0") ],

    targets: [ .target(name: "PluginConsumer", dependencies: [
"PluginConsumer"]) ]

)

This all compiles nicely, MyPlugin project creates dylib file that
executable created by PluginConsumer can load, but the problem is with
following line:

let plugin = f() as? PluginInterface

Type of the plugin is MyPlugin, but from the consumer's view, it
doesn't inherit from PluginInterface so I can't call sayHi() method. I
assume this is because there is no relation between PluginInterface class
that compiler uses for MyPlugin project one that it uses for PluginConsumer
project. After library is loaded, they are two completely different classes
that happen to share same name. Is my assumption correct and how do I go
about fixing it?

I had an idea I could make PluginInterface emit dynamic library that
would be dynamically linked by both MyPlugin and PluginConsumer, thus
making them share same PluginInterface class, but I can't figure out how to
do that (or if it's right way of doing this).

Any help appreciated :)

Lope
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

I tried to use @_exported and it helped somewhat. While I still have same
warnings, size of the PluginInterface library went down by 6mb (to 120kb)
so it looks like Protobuf is no longer statically linked to it. However,
size of PluginConsumer executable went up by same 6mb, it looks like it is
linked there twice now.
I also noticed interesting thing. If I build executable using `swift build`
the size is around 17mb, when I generate xcode project and build it using
that, size is around 200kb, but I get same warnings using both approaches

···

On 7 October 2017 at 15:44, Geordie Jay <geojay@gmail.com> wrote:

Ján Kosa <lopenka@gmail.com> schrieb am Sa. 7. Okt. 2017 um 13:34:

I tried swift package clean, but it didn't help

"Try to ensure the plugin provider module (libA) is (only) being
compiled into its standalone shared library file."
How do I go about this? It is 3rd party module, it doesn't define any
products (https://github.com/apple/swift-protobuf.git\). Is there
something I can do in my Package files to make sure it is loaded
dynamically?

When you compile a package depending on protobuf, all the relevant symbols
end up in your package’s library file. So here’s something you might try:

import protobuf into your own “PluginProvider” module (package), which has
a shared library product like this: ‘@_exported import Protobuf’ in some
compiled swift file. Then from the other dependent modules “import
PluginProvider” - the protobuf symbols should be available, and all from
one (nonconflicting) source.

Geordie

On 6 October 2017 at 22:52, Geordie Jay <geojay@gmail.com> wrote:

I think SwiftPM is (incorrectly) compiling A.XYZ into each of the
modules that depend on it, as well as into your intended libA.so file.

Try to ensure the plugin provider module (libA) is (only) being compiled
into its standalone shared library file. Try cleaning the swiftpm build for
one (swift package clean) and ensure the Package.swift files are correctly
set up to output the shared library.

Sorry I can’t be more specific, I’ve had these same kinds of issues
before but I’m not 100% what they were.

Geordie

Ján Kosa via swift-users <swift-users@swift.org> schrieb am Fr. 6. Okt. >>> 2017 um 14:41:

It worked! Took me a while to iron out details, but it is working now.
Huge thanks sir, I will name my firstborn after you.
Thanks for the @_cdecl("initializePlugin") tip as well, I didn't know
about it and it will be very useful.

I am having slightly related problem now (it was there before, but I
ignored it for the time being), not sure if I should start a new thread?

The PluginInterface module has one external dependency on module A,
PluginConsumer has the dependency on module B which has dependency on same
module A that the PluginInterface uses. When I load the plugin library, I
get bunch of errors like:

Class A.XYZ is implemented in both libPluginInterface.dylib and
libMyPlugin.dylib

I know why it is there, but I don't know how to get rid of it. I can't
just remove dependency from PluginConsumer and use the one from
PluginInterface (if that would even work?) because PluginConsumer does not
depend on it directly, but it is going through module B first

Cheers,
Lope

On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dunbar@apple.com> >>>> wrote:

The way that I have done this in the past is pass a protocol as an
unsafe pointer to an exposed entry point:

            let entryPoint = dlsym(handle, “initializePlugin”)
            guard entryPoint != nil else {
                fatalError("missing plugin entry point: \(pluginPath)")
            }
            typealias PluginInitializationFunc = @convention(c)
(UnsafeRawPointer) -> ()
            let f = unsafeBitCast(entryPoint, to:
PluginInitializationFunc.self)
            f(Unmanaged.passUnretained(self).toOpaque())

and then in the plugin convert back to the appropriate type:

@_cdecl("initializePlugin")
public func initializePlugin(_ ptr: UnsafeRawPointer) {
    let manager = Unmanaged<PluginManager>.fromOpaque(ptr).
takeUnretainedValue()

HTH,
- Daniel

On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift-users < >>>>> swift-users@swift.org> wrote:

Hello folks,

I have been toying with dynamic libraries, trying to implement plugin
functionality. I was able to get to the point where I can call simple
function in loaded library, but I am having troubles starting more
sophisticated communication channel.

There are 3 projects
- PluginConsumer is an app that loads plugin libraries
- MyPlugin is a plugin implementation, output is dynamic library that
PluginConsumer loads
- PluginInterface is common interface that both MyPlugin and
PluginConsumer use, so that they know how to communicate

My first idea was to have PluginInterface be a simple SPM project with
single file where the bare-bones PluginInterface class would be:

open class PluginInterface {

    open func sayHi()

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginInterface",

    products: [ .library(name: "PluginInterface", type: .dynamic,
targets: ["PluginInterface"]) ],

    targets: [ .target(name: "PluginInterface") ]

)

UserPlugin is also very simple project containing only one file:

public func getPlugin() -> AnyObject {

    return MyPlugin()

}

class MyPlugin: PluginInterface {

    override func sayHi() {

        print("Hi from my plugin")

    }

}

Package.swift:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "MyPlugin",

    products: [ .library(name: "MyPlugin", type: .dynamic, targets: [
"MyPlugin"]) ],

    dependencies: [ .package(url: "url_to_PluginInterface", from:
"0.0.0"), ],

    targets: [

        .target(name: "PluginInterface", dependencies: [
"PluginInterface"]),

        .target(name: "MyPlugin", dependencies: ["PluginInterface"]),

    ]

)

The PluginConsumer is bit more complicated, but here is relevant part
(lib loading and function calling):

typealias InitFunction = @convention(c) () -> AnyObject

let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL)

if openRes != nil {

    defer {

        dlclose(openRes)

    }

    let symbolName = "mangled_symbol_name"

    let sym = dlsym(openRes, symbolName)

    if sym != nil {

        let f: InitFunction = unsafeBitCast(sym, to: InitFunction.self
)

        let plugin = f() as? PluginInterface

    }

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginConsumer",

    dependencies: [ .package(url: "path_to_plugin_interface", from:
"0.0.0") ],

    targets: [ .target(name: "PluginConsumer", dependencies: [
"PluginConsumer"]) ]

)

This all compiles nicely, MyPlugin project creates dylib file that
executable created by PluginConsumer can load, but the problem is with
following line:

let plugin = f() as? PluginInterface

Type of the plugin is MyPlugin, but from the consumer's view, it
doesn't inherit from PluginInterface so I can't call sayHi() method. I
assume this is because there is no relation between PluginInterface class
that compiler uses for MyPlugin project one that it uses for PluginConsumer
project. After library is loaded, they are two completely different classes
that happen to share same name. Is my assumption correct and how do I go
about fixing it?

I had an idea I could make PluginInterface emit dynamic library that
would be dynamically linked by both MyPlugin and PluginConsumer, thus
making them share same PluginInterface class, but I can't figure out how to
do that (or if it's right way of doing this).

Any help appreciated :)

Lope
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

I tried to use @_exported and it helped somewhat. While I still have same
warnings, size of the PluginInterface library went down by 6mb (to 120kb)
so it looks like Protobuf is no longer statically linked to it. However,
size of PluginConsumer executable went up by same 6mb, it looks like it is
linked there twice now.

To be clear: take protobuf out of the PluginConsumer dependencies.
Actually, I’m not sure which is which, but protobuf should only be listed
as a dependency of one package, where it is imported as @_exported. After
that, your other modules depend on the package that imports / exports
protobuf.

I also noticed interesting thing. If I build executable using `swift build`

···

Ján Kosa <lopenka@gmail.com> schrieb am Sa. 7. Okt. 2017 um 15:27:

the size is around 17mb, when I generate xcode project and build it using
that, size is around 200kb, but I get same warnings using both approaches

On 7 October 2017 at 15:44, Geordie Jay <geojay@gmail.com> wrote:

Ján Kosa <lopenka@gmail.com> schrieb am Sa. 7. Okt. 2017 um 13:34:

I tried swift package clean, but it didn't help

"Try to ensure the plugin provider module (libA) is (only) being
compiled into its standalone shared library file."
How do I go about this? It is 3rd party module, it doesn't define any
products (https://github.com/apple/swift-protobuf.git\). Is there
something I can do in my Package files to make sure it is loaded
dynamically?

When you compile a package depending on protobuf, all the relevant
symbols end up in your package’s library file. So here’s something you
might try:

import protobuf into your own “PluginProvider” module (package), which
has a shared library product like this: ‘@_exported import Protobuf’ in
some compiled swift file. Then from the other dependent modules “import
PluginProvider” - the protobuf symbols should be available, and all from
one (nonconflicting) source.

Geordie

On 6 October 2017 at 22:52, Geordie Jay <geojay@gmail.com> wrote:

I think SwiftPM is (incorrectly) compiling A.XYZ into each of the
modules that depend on it, as well as into your intended libA.so file.

Try to ensure the plugin provider module (libA) is (only) being
compiled into its standalone shared library file. Try cleaning the swiftpm
build for one (swift package clean) and ensure the Package.swift files are
correctly set up to output the shared library.

Sorry I can’t be more specific, I’ve had these same kinds of issues
before but I’m not 100% what they were.

Geordie

Ján Kosa via swift-users <swift-users@swift.org> schrieb am Fr. 6. >>>> Okt. 2017 um 14:41:

It worked! Took me a while to iron out details, but it is working now.
Huge thanks sir, I will name my firstborn after you.
Thanks for the @_cdecl("initializePlugin") tip as well, I didn't know
about it and it will be very useful.

I am having slightly related problem now (it was there before, but I
ignored it for the time being), not sure if I should start a new thread?

The PluginInterface module has one external dependency on module A,
PluginConsumer has the dependency on module B which has dependency on same
module A that the PluginInterface uses. When I load the plugin library, I
get bunch of errors like:

Class A.XYZ is implemented in both libPluginInterface.dylib and
libMyPlugin.dylib

I know why it is there, but I don't know how to get rid of it. I can't
just remove dependency from PluginConsumer and use the one from
PluginInterface (if that would even work?) because PluginConsumer does not
depend on it directly, but it is going through module B first

Cheers,
Lope

On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dunbar@apple.com> >>>>> wrote:

The way that I have done this in the past is pass a protocol as an
unsafe pointer to an exposed entry point:

            let entryPoint = dlsym(handle, “initializePlugin”)
            guard entryPoint != nil else {
                fatalError("missing plugin entry point:
\(pluginPath)")
            }
            typealias PluginInitializationFunc = @convention(c)
(UnsafeRawPointer) -> ()
            let f = unsafeBitCast(entryPoint, to:
PluginInitializationFunc.self)
            f(Unmanaged.passUnretained(self).toOpaque())

and then in the plugin convert back to the appropriate type:

@_cdecl("initializePlugin")
public func initializePlugin(_ ptr: UnsafeRawPointer) {
    let manager =
Unmanaged<PluginManager>.fromOpaque(ptr).takeUnretainedValue()

HTH,
- Daniel

On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift-users < >>>>>> swift-users@swift.org> wrote:

Hello folks,

I have been toying with dynamic libraries, trying to implement plugin
functionality. I was able to get to the point where I can call simple
function in loaded library, but I am having troubles starting more
sophisticated communication channel.

There are 3 projects
- PluginConsumer is an app that loads plugin libraries
- MyPlugin is a plugin implementation, output is dynamic library that
PluginConsumer loads
- PluginInterface is common interface that both MyPlugin and
PluginConsumer use, so that they know how to communicate

My first idea was to have PluginInterface be a simple SPM project
with single file where the bare-bones PluginInterface class would be:

open class PluginInterface {

    open func sayHi()

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginInterface",

    products: [ .library(name: "PluginInterface", type: .dynamic,
targets: ["PluginInterface"]) ],

    targets: [ .target(name: "PluginInterface") ]

)

UserPlugin is also very simple project containing only one file:

public func getPlugin() -> AnyObject {

    return MyPlugin()

}

class MyPlugin: PluginInterface {

    override func sayHi() {

        print("Hi from my plugin")

    }

}

Package.swift:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "MyPlugin",

    products: [ .library(name: "MyPlugin", type: .dynamic, targets: [
"MyPlugin"]) ],

    dependencies: [ .package(url: "url_to_PluginInterface", from:
"0.0.0"), ],

    targets: [

        .target(name: "PluginInterface", dependencies: [
"PluginInterface"]),

        .target(name: "MyPlugin", dependencies: ["PluginInterface"]),

    ]

)

The PluginConsumer is bit more complicated, but here is relevant part
(lib loading and function calling):

typealias InitFunction = @convention(c) () -> AnyObject

let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL)

if openRes != nil {

    defer {

        dlclose(openRes)

    }

    let symbolName = "mangled_symbol_name"

    let sym = dlsym(openRes, symbolName)

    if sym != nil {

        let f: InitFunction = unsafeBitCast(sym, to: InitFunction.
self)

        let plugin = f() as? PluginInterface

    }

}

Package.swift file:

// swift-tools-version:4.0

import PackageDescription

let package = Package(

    name: "PluginConsumer",

    dependencies: [ .package(url: "path_to_plugin_interface", from:
"0.0.0") ],

    targets: [ .target(name: "PluginConsumer", dependencies: [
"PluginConsumer"]) ]

)

This all compiles nicely, MyPlugin project creates dylib file that
executable created by PluginConsumer can load, but the problem is with
following line:

let plugin = f() as? PluginInterface

Type of the plugin is MyPlugin, but from the consumer's view, it
doesn't inherit from PluginInterface so I can't call sayHi() method. I
assume this is because there is no relation between PluginInterface class
that compiler uses for MyPlugin project one that it uses for PluginConsumer
project. After library is loaded, they are two completely different classes
that happen to share same name. Is my assumption correct and how do I go
about fixing it?

I had an idea I could make PluginInterface emit dynamic library that
would be dynamically linked by both MyPlugin and PluginConsumer, thus
making them share same PluginInterface class, but I can't figure out how to
do that (or if it's right way of doing this).

Any help appreciated :)

Lope
_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

That is exactly what I did. The only package that depends on the protobuf
is the PluginInterface. Both MyPlugin and and PluginConsumer depend on the
PluginInterface and not on the protobuf itself. I had to shuffle around my
dependencies a bit, which resulted in smaller number of dependencies but
they don't make much sense now (as in, some target had to depend on
PluginInterface even if they don't need to, just to get access to
protobuf). I could live with that if it solved the issue, but it didn't.

I am adding my Package.swift files in case I missed something:

PluginInterface:

```swift
let package = Package(

name: "PluginInterface",

products: [ .library(name: "PluginInterface", type: .dynamic, targets: [
"PluginInterface"]) ],

dependencies: [ .package(url: "https://github.com/apple/swift-protobuf.git&quot;,

···

from: "0.0.0") ],

targets: [ .target(name: "PluginInterface", dependencies: ["SwiftProtobuf"])
]

)```

MyPlugin:


let package = Package(

name: "MyPlugin",

products: [ .library(name: "MyPlugin", type: .dynamic, targets: [
"PluginImpl"]) ],

dependencies: [

.package(url: "path/to/PluginInterface.git", from: "0.0.0"),

],

targets: [

.target(name: "PluginImpl", dependencies: ["ProtoBufMessages"]),

.target(name: "ProtoBufMessages", dependencies: ["PluginInterface"])

]

)```

PluginConsumer:

```swift

let package = Package\(

name: &quot;PluginConsumer&quot;,

dependencies: \[

\.package\(url: &quot;https://github.com/PerfectlySoft/Perfect-WebSockets.git&quot;,
from: &quot;3\.0\.0&quot;\),

\.package\(url: &quot;https://github.com/PerfectlySoft/Perfect-HTTPServer.git&quot;,
from: &quot;3\.0\.0&quot;\),

\.package\(url: &quot;path/to/PluginInterface&quot;, from: &quot;0\.0\.0&quot;\),

\.package\(url: &quot;https://github.com/krzyzanowskim/CryptoSwift.git&quot;, from:
&quot;0\.0\.0&quot;\)

\],

targets: \[

\.target\(name: &quot;AppMaster&quot;, dependencies: \[&quot;Shared&quot;, &quot;CryptoSwift&quot;\]\),

\.target\(name: &quot;PluginConsumer&quot;, dependencies: \[&quot;Shared&quot;, &quot;CryptoSwift&quot;\]\),

\.target\(name: &quot;Shared&quot;, dependencies: \[&quot;ProtoBufMessages&quot;,
&quot;PerfectHTTPServer&quot;, &quot;PerfectWebSockets&quot;\]\),

\.target\(name: &quot;ProtoBufMessages&quot;, dependencies: \[&quot;PluginInterface&quot;\]\)

\]

\)\`\`\`

App master is separate executable that shares some functionality with
PluginConsumer, but it doesn&#39;t link against it in any way\. I guess it could
be omitted, but I wanted to give you whole thing as it is

On 7 October 2017 at 18:33, Geordie Jay &lt;geojay@gmail\.com&gt; wrote:

> Ján Kosa &lt;lopenka@gmail\.com&gt; schrieb am Sa\. 7\. Okt\. 2017 um 15:27:
>
>> I tried to use @\_exported and it helped somewhat\. While I still have
>> same warnings, size of the PluginInterface library went down by 6mb \(to
>> 120kb\) so it looks like Protobuf is no longer statically linked to it\.
>> However, size of PluginConsumer executable went up by same 6mb, it looks
>> like it is linked there twice now\.
>>
> To be clear: take protobuf out of the PluginConsumer dependencies\.
> Actually, I’m not sure which is which, but protobuf should only be listed
> as a dependency of one package, where it is imported as @\_exported\. After
> that, your other modules depend on the package that imports / exports
> protobuf\.
>
> I also noticed interesting thing\. If I build executable using \`swift
>> build\` the size is around 17mb, when I generate xcode project and build it
>> using that, size is around 200kb, but I get same warnings using both
>> approaches
>>
>> On 7 October 2017 at 15:44, Geordie Jay &lt;geojay@gmail\.com&gt; wrote:
>>
>>> Ján Kosa &lt;lopenka@gmail\.com&gt; schrieb am Sa\. 7\. Okt\. 2017 um 13:34:
>>>
>>>> I tried swift package clean, but it didn&#39;t help
>>>>
>>>> &quot;Try to ensure the plugin provider module \(libA\) is \(only\) being
>>>> compiled into its standalone shared library file\.&quot;
>>>> How do I go about this? It is 3rd party module, it doesn&#39;t define any
>>>> products \(https://github.com/apple/swift-protobuf.git\)\. Is there
>>>> something I can do in my Package files to make sure it is loaded
>>>> dynamically?
>>>>
>>> When you compile a package depending on protobuf, all the relevant
>>> symbols end up in your package’s library file\. So here’s something you
>>> might try:
>>>
>>> import protobuf into your own “PluginProvider” module \(package\), which
>>> has a shared library product like this: ‘@\_exported import Protobuf’ in
>>> some compiled swift file\. Then from the other dependent modules “import
>>> PluginProvider” \- the protobuf symbols should be available, and all from
>>> one \(nonconflicting\) source\.
>>>
>>> Geordie
>>>
>>>> On 6 October 2017 at 22:52, Geordie Jay &lt;geojay@gmail\.com&gt; wrote:
>>>>
>>>>> I think SwiftPM is \(incorrectly\) compiling A\.XYZ into each of the
>>>>> modules that depend on it, as well as into your intended libA\.so file\.
>>>>>
>>>>> Try to ensure the plugin provider module \(libA\) is \(only\) being
>>>>> compiled into its standalone shared library file\. Try cleaning the swiftpm
>>>>> build for one \(swift package clean\) and ensure the Package\.swift files are
>>>>> correctly set up to output the shared library\.
>>>>>
>>>>> Sorry I can’t be more specific, I’ve had these same kinds of issues
>>>>> before but I’m not 100% what they were\.
>>>>>
>>>>> Geordie
>>>>>
>>>>> Ján Kosa via swift\-users &lt;swift\-users@swift\.org&gt; schrieb am Fr\. 6\. &gt;&gt;&gt;&gt;&gt; Okt\. 2017 um 14:41:
>>>>>
>>>>>> It worked\! Took me a while to iron out details, but it is working
>>>>>> now\. Huge thanks sir, I will name my firstborn after you\.
>>>>>> Thanks for the @\_cdecl\(&quot;initializePlugin&quot;\) tip as well, I didn&#39;t
>>>>>> know about it and it will be very useful\.
>>>>>>
>>>>>> I am having slightly related problem now \(it was there before, but I
>>>>>> ignored it for the time being\), not sure if I should start a new thread?
>>>>>>
>>>>>> The PluginInterface module has one external dependency on module A,
>>>>>> PluginConsumer has the dependency on module B which has dependency on same
>>>>>> module A that the PluginInterface uses\. When I load the plugin library, I
>>>>>> get bunch of errors like:
>>>>>>
>>>>>> Class A\.XYZ is implemented in both libPluginInterface\.dylib and
>>>>>> libMyPlugin\.dylib
>>>>>>
>>>>>> I know why it is there, but I don&#39;t know how to get rid of it\. I
>>>>>> can&#39;t just remove dependency from PluginConsumer and use the one from
>>>>>> PluginInterface \(if that would even work?\) because PluginConsumer does not
>>>>>> depend on it directly, but it is going through module B first
>>>>>>
>>>>>> Cheers,
>>>>>> Lope
>>>>>>
>>>>>> On 4 October 2017 at 22:17, Daniel Dunbar &lt;daniel\_dunbar@apple\.com&gt; &gt;&gt;&gt;&gt;&gt;&gt; wrote:
>>>>>>
>>>>>>> The way that I have done this in the past is pass a protocol as an
>>>>>>> unsafe pointer to an exposed entry point:
>>>>>>> ```swift
>>>>>>>             let entryPoint = dlsym(handle, “initializePlugin”)
>>>>>>>             guard entryPoint != nil else {
>>>>>>>                 fatalError("missing plugin entry point:
>>>>>>> \(pluginPath)")
>>>>>>>             }
>>>>>>>             typealias PluginInitializationFunc = @convention(c)
>>>>>>> (UnsafeRawPointer) -> ()
>>>>>>>             let f = unsafeBitCast(entryPoint, to:
>>>>>>> PluginInitializationFunc.self)
>>>>>>>             f(Unmanaged.passUnretained(self).toOpaque())
>>>>>>> ```
>>>>>>>
>>>>>>> and then in the plugin convert back to the appropriate type:
>>>>>>>
>>>>>>> ```
>>>>>>> @_cdecl("initializePlugin")
>>>>>>> public func initializePlugin(_ ptr: UnsafeRawPointer) {
>>>>>>>     let manager = Unmanaged<PluginManager>.fromOpaque(ptr).
>>>>>>> takeUnretainedValue()
>>>>>>> ```
>>>>>>>
>>>>>>> HTH,
>>>>>>>  \- Daniel
>>>>>>>
>>>>>>> On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift\-users &lt; &gt;&gt;&gt;&gt;&gt;&gt;&gt; swift\-users@swift\.org&gt; wrote:
>>>>>>>
>>>>>>> Hello folks,
>>>>>>>
>>>>>>> I have been toying with dynamic libraries, trying to implement
>>>>>>> plugin functionality\. I was able to get to the point where I can call
>>>>>>> simple function in loaded library, but I am having troubles starting more
>>>>>>> sophisticated communication channel\.
>>>>>>>
>>>>>>> There are 3 projects
>>>>>>> \- PluginConsumer is an app that loads plugin libraries
>>>>>>> \- MyPlugin is a plugin implementation, output is dynamic library
>>>>>>> that PluginConsumer loads
>>>>>>> \- PluginInterface is common interface that both MyPlugin and
>>>>>>> PluginConsumer use, so that they know how to communicate
>>>>>>>
>>>>>>> My first idea was to have PluginInterface be a simple SPM project
>>>>>>> with single file where the bare\-bones PluginInterface class would be:
>>>>>>>
>>>>>>> open class PluginInterface \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;open func sayHi\(\)
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift file:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;PluginInterface&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;products: \[ \.library\(name: &quot;PluginInterface&quot;, type: \.dynamic,
>>>>>>> targets: \[&quot;PluginInterface&quot;\]\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[ \.target\(name: &quot;PluginInterface&quot;\) \]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> UserPlugin is also very simple project containing only one file:
>>>>>>>
>>>>>>> public func getPlugin\(\) \-&gt; AnyObject \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;return MyPlugin\(\)
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> class MyPlugin: PluginInterface \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;override func sayHi\(\) \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;print\(&quot;Hi from my plugin&quot;\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;MyPlugin&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;products: \[ \.library\(name: &quot;MyPlugin&quot;, type: \.dynamic, targets:
>>>>>>> \[&quot;MyPlugin&quot;\]\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;dependencies: \[ \.package\(url: &quot;url\_to\_PluginInterface&quot;, from:
>>>>>>> &quot;0\.0\.0&quot;\), \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\.target\(name: &quot;PluginInterface&quot;, dependencies: \[
>>>>>>> &quot;PluginInterface&quot;\]\),
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\.target\(name: &quot;MyPlugin&quot;, dependencies: \[&quot;PluginInterface&quot;
>>>>>>> \]\),
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> The PluginConsumer is bit more complicated, but here is relevant
>>>>>>> part \(lib loading and function calling\):
>>>>>>>
>>>>>>> typealias InitFunction = @convention\(c\) \(\) \-&gt; AnyObject
>>>>>>>
>>>>>>> let openRes = dlopen\(pathToLib, RTLD\_NOW|RTLD\_LOCAL\)
>>>>>>>
>>>>>>> if openRes \!= nil \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;defer \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dlclose\(openRes\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;let symbolName = &quot;mangled\_symbol\_name&quot;
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;let sym = dlsym\(openRes, symbolName\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;if sym \!= nil \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let f: InitFunction = unsafeBitCast\(sym, to: InitFunction\.
>>>>>>> self\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let plugin = f\(\) as? PluginInterface
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift file:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;PluginConsumer&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;dependencies: \[ \.package\(url: &quot;path\_to\_plugin\_interface&quot;, from:
>>>>>>> &quot;0\.0\.0&quot;\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[ \.target\(name: &quot;PluginConsumer&quot;, dependencies: \[
>>>>>>> &quot;PluginConsumer&quot;\]\) \]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> This all compiles nicely, MyPlugin project creates dylib file that
>>>>>>> executable created by PluginConsumer can load, but the problem is with
>>>>>>> following line:
>>>>>>>
>>>>>>> let plugin = f\(\) as? PluginInterface
>>>>>>>
>>>>>>> Type of the plugin is MyPlugin, but from the consumer&#39;s view, it
>>>>>>> doesn&#39;t inherit from PluginInterface so I can&#39;t call sayHi\(\) method\. I
>>>>>>> assume this is because there is no relation between PluginInterface class
>>>>>>> that compiler uses for MyPlugin project one that it uses for PluginConsumer
>>>>>>> project\. After library is loaded, they are two completely different classes
>>>>>>> that happen to share same name\. Is my assumption correct and how do I go
>>>>>>> about fixing it?
>>>>>>>
>>>>>>> I had an idea I could make PluginInterface emit dynamic library that
>>>>>>> would be dynamically linked by both MyPlugin and PluginConsumer, thus
>>>>>>> making them share same PluginInterface class, but I can&#39;t figure out how to
>>>>>>> do that \(or if it&#39;s right way of doing this\)\.
>>>>>>>
>>>>>>> Any help appreciated :\)
>>>>>>>
>>>>>>> Lope
>>>>>>> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>>>>>>> swift\-users mailing list
>>>>>>> swift\-users@swift\.org
>>>>>>> https://lists.swift.org/mailman/listinfo/swift-users
>>>>>>>
>>>>>>
>>>>>> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>>>>>> swift\-users mailing list
>>>>>> swift\-users@swift\.org
>>>>>> https://lists.swift.org/mailman/listinfo/swift-users
>>>>>>

</details>

Is it possible for you to make a small test package that shows the problem, and file a bug in bugs.swift.org? There may be something we need to fix in SwiftPM before this can work (because of our linking model).

- Daniel

···

On Oct 7, 2017, at 10:42 PM, Ján Kosa via swift-users <swift-users@swift.org> wrote:

That is exactly what I did. The only package that depends on the protobuf is the PluginInterface. Both MyPlugin and and PluginConsumer depend on the PluginInterface and not on the protobuf itself. I had to shuffle around my dependencies a bit, which resulted in smaller number of dependencies but they don't make much sense now (as in, some target had to depend on PluginInterface even if they don't need to, just to get access to protobuf). I could live with that if it solved the issue, but it didn't.

I am adding my Package.swift files in case I missed something:

PluginInterface:

let package = Package(
name: "PluginInterface",

products: [ .library(name: "PluginInterface", type: .dynamic, targets: ["PluginInterface"]) ],

dependencies: [ .package(url: "https://github.com/apple/swift-protobuf.git <https://github.com/apple/swift-protobuf.git>", from: "0.0.0") ],

targets: [ .target(name: "PluginInterface", dependencies: ["SwiftProtobuf"]) ]

)```

MyPlugin:

```swift

let package = Package\(

name: &quot;MyPlugin&quot;,

products: \[ \.library\(name: &quot;MyPlugin&quot;, type: \.dynamic, targets: \[&quot;PluginImpl&quot;\]\) \],

dependencies: \[

\.package\(url: &quot;path/to/PluginInterface\.git&quot;, from: &quot;0\.0\.0&quot;\),

\],

targets: \[

\.target\(name: &quot;PluginImpl&quot;, dependencies: \[&quot;ProtoBufMessages&quot;\]\),

\.target\(name: &quot;ProtoBufMessages&quot;, dependencies: \[&quot;PluginInterface&quot;\]\)

\]

\)\`\`\`

PluginConsumer:

```swift

let package = Package(

name: "PluginConsumer",

dependencies: [

.package(url: "https://github.com/PerfectlySoft/Perfect-WebSockets.git <https://github.com/PerfectlySoft/Perfect-WebSockets.git>", from: "3.0.0"),

.package(url: "https://github.com/PerfectlySoft/Perfect-HTTPServer.git <https://github.com/PerfectlySoft/Perfect-HTTPServer.git>", from: "3.0.0"),

.package(url: "path/to/PluginInterface", from: "0.0.0"),

.package(url: "https://github.com/krzyzanowskim/CryptoSwift.git <https://github.com/krzyzanowskim/CryptoSwift.git>", from: "0.0.0")

],

targets: [

.target(name: "AppMaster", dependencies: ["Shared", "CryptoSwift"]),

.target(name: "PluginConsumer", dependencies: ["Shared", "CryptoSwift"]),

.target(name: "Shared", dependencies: ["ProtoBufMessages", "PerfectHTTPServer", "PerfectWebSockets"]),

.target(name: "ProtoBufMessages", dependencies: ["PluginInterface"])

]

)```

App master is separate executable that shares some functionality with PluginConsumer, but it doesn't link against it in any way. I guess it could be omitted, but I wanted to give you whole thing as it is

On 7 October 2017 at 18:33, Geordie Jay <geojay@gmail.com <mailto:geojay@gmail.com>> wrote:

Ján Kosa <lopenka@gmail.com <mailto:lopenka@gmail.com>> schrieb am Sa. 7. Okt. 2017 um 15:27: > I tried to use @_exported and it helped somewhat. While I still have same warnings, size of the PluginInterface library went down by 6mb (to 120kb) so it looks like Protobuf is no longer statically linked to it. However, size of PluginConsumer executable went up by same 6mb, it looks like it is linked there twice now.  >  > To be clear: take protobuf out of the PluginConsumer dependencies. Actually, I’m not sure which is which, but protobuf should only be listed as a dependency of one package, where it is imported as @_exported. After that, your other modules depend on the package that imports / exports protobuf.

I also noticed interesting thing. If I build executable using `swift build` the size is around 17mb, when I generate xcode project and build it using that, size is around 200kb, but I get same warnings using both approaches

On 7 October 2017 at 15:44, Geordie Jay <geojay@gmail.com <mailto:geojay@gmail.com>> wrote:

Ján Kosa <lopenka@gmail.com <mailto:lopenka@gmail.com>> schrieb am Sa. 7. Okt. 2017 um 13:34: > I tried swift package clean, but it didn't help >  > "Try to ensure the plugin provider module (libA) is (only) being compiled into its standalone shared library file."  > How do I go about this? It is 3rd party module, it doesn't define any products (https://github.com/apple/swift-protobuf.git <https://github.com/apple/swift-protobuf.git>). Is there something I can do in my Package files to make sure it is loaded dynamically?

When you compile a package depending on protobuf, all the relevant symbols end up in your package’s library file. So here’s something you might try:

import protobuf into your own “PluginProvider” module (package), which has a shared library product like this: ‘@_exported import Protobuf’ in some compiled swift file. Then from the other dependent modules “import PluginProvider” - the protobuf symbols should be available, and all from one (nonconflicting) source.

Geordie

On 6 October 2017 at 22:52, Geordie Jay <geojay@gmail.com <mailto:geojay@gmail.com>> wrote:
I think SwiftPM is (incorrectly) compiling A.XYZ <http://a.xyz/> into each of the modules that depend on it, as well as into your intended libA.so file.

Try to ensure the plugin provider module (libA) is (only) being compiled into its standalone shared library file. Try cleaning the swiftpm build for one (swift package clean) and ensure the Package.swift files are correctly set up to output the shared library.

Sorry I can’t be more specific, I’ve had these same kinds of issues before but I’m not 100% what they were.

Geordie

Ján Kosa via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> schrieb am Fr. 6. Okt. 2017 um 14:41:
It worked! Took me a while to iron out details, but it is working now. Huge thanks sir, I will name my firstborn after you.
Thanks for the @_cdecl("initializePlugin") tip as well, I didn't know about it and it will be very useful.

I am having slightly related problem now (it was there before, but I ignored it for the time being), not sure if I should start a new thread?

The PluginInterface module has one external dependency on module A, PluginConsumer has the dependency on module B which has dependency on same module A that the PluginInterface uses. When I load the plugin library, I get bunch of errors like:

Class A.XYZ <http://a.xyz/> is implemented in both libPluginInterface.dylib and libMyPlugin.dylib

I know why it is there, but I don't know how to get rid of it. I can't just remove dependency from PluginConsumer and use the one from PluginInterface (if that would even work?) because PluginConsumer does not depend on it directly, but it is going through module B first

Cheers,
Lope

On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>> wrote:
The way that I have done this in the past is pass a protocol as an unsafe pointer to an exposed entry point:
```swift
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let entryPoint = dlsym\(handle, “initializePlugin”\)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;guard entryPoint \!= nil else \{
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;fatalError\(&quot;missing plugin entry point: \\\(pluginPath\)&quot;\)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\}
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;typealias PluginInitializationFunc = @convention\(c\) \(UnsafeRawPointer\) \-&gt; \(\)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let f = unsafeBitCast\(entryPoint, to: PluginInitializationFunc\.self\)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;f\(Unmanaged\.passUnretained\(self\)\.toOpaque\(\)\)

and then in the plugin convert back to the appropriate type:

@\_cdecl\(&quot;initializePlugin&quot;\)
public func initializePlugin\(\_ ptr: UnsafeRawPointer\) \{
&nbsp;&nbsp;&nbsp;&nbsp;let manager = Unmanaged&lt;PluginManager&gt;\.fromOpaque\(ptr\)\.takeUnretainedValue\(\)
\`\`\`

HTH,
 \- Daniel

> On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift\-users &lt;swift\-users@swift\.org &lt;mailto:swift\-users@swift\.org&gt;&gt; wrote:
>
> Hello folks,
>
> I have been toying with dynamic libraries, trying to implement plugin functionality\. I was able to get to the point where I can call simple function in loaded library, but I am having troubles starting more sophisticated communication channel\.
>
> There are 3 projects
> \- PluginConsumer is an app that loads plugin libraries 
> \- MyPlugin is a plugin implementation, output is dynamic library that PluginConsumer loads
> \- PluginInterface is common interface that both MyPlugin and PluginConsumer use, so that they know how to communicate
>
> My first idea was to have PluginInterface be a simple SPM project with single file where the bare\-bones PluginInterface class would be:
>
> open class PluginInterface \{
>
> &nbsp;&nbsp;&nbsp;&nbsp;open func sayHi\(\)
>
> \}
>
> Package\.swift file:
>
> // swift\-tools\-version:4\.0
>
> import PackageDescription
>
> let package = Package\(
>
> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;PluginInterface&quot;,
>
> &nbsp;&nbsp;&nbsp;&nbsp;products: \[ \.library\(name: &quot;PluginInterface&quot;, type: \.dynamic, targets: \[&quot;PluginInterface&quot;\]\) \],
>
> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[ \.target\(name: &quot;PluginInterface&quot;\) \]
>
> \)
>
> UserPlugin is also very simple project containing only one file:
>
> public func getPlugin\(\) \-&gt; AnyObject \{
>
> &nbsp;&nbsp;&nbsp;&nbsp;return MyPlugin\(\)
>
> \}
>
> class MyPlugin: PluginInterface \{
>
> &nbsp;&nbsp;&nbsp;&nbsp;override func sayHi\(\) \{
>
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;print\(&quot;Hi from my plugin&quot;\)
>
> &nbsp;&nbsp;&nbsp;&nbsp;\}
>
> \}
>
> Package\.swift:
>
> // swift\-tools\-version:4\.0
>
> import PackageDescription
>
> let package = Package\(
>
> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;MyPlugin&quot;,
>
> &nbsp;&nbsp;&nbsp;&nbsp;products: \[ \.library\(name: &quot;MyPlugin&quot;, type: \.dynamic, targets: \[&quot;MyPlugin&quot;\]\) \],
>
> &nbsp;&nbsp;&nbsp;&nbsp;dependencies: \[ \.package\(url: &quot;url\_to\_PluginInterface&quot;, from: &quot;0\.0\.0&quot;\), \],
>
> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[
>
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\.target\(name: &quot;PluginInterface&quot;, dependencies: \[&quot;PluginInterface&quot;\]\),
>
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\.target\(name: &quot;MyPlugin&quot;, dependencies: \[&quot;PluginInterface&quot;\]\),
>
> &nbsp;&nbsp;&nbsp;&nbsp;\]
>
> \)
>
> The PluginConsumer is bit more complicated, but here is relevant part \(lib loading and function calling\):
>
> typealias InitFunction = @convention\(c\) \(\) \-&gt; AnyObject
>
> let openRes = dlopen\(pathToLib, RTLD\_NOW|RTLD\_LOCAL\)
>
> if openRes \!= nil \{
>
> &nbsp;&nbsp;&nbsp;&nbsp;defer \{
>
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dlclose\(openRes\)
>
> &nbsp;&nbsp;&nbsp;&nbsp;\}
>
> &nbsp;&nbsp;&nbsp;&nbsp;let symbolName = &quot;mangled\_symbol\_name&quot;
>
> &nbsp;&nbsp;&nbsp;&nbsp;let sym = dlsym\(openRes, symbolName\)
>
> &nbsp;&nbsp;&nbsp;&nbsp;if sym \!= nil \{
>
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let f: InitFunction = unsafeBitCast\(sym, to: InitFunction\.self\)
>
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let plugin = f\(\) as? PluginInterface
>
> &nbsp;&nbsp;&nbsp;&nbsp;\}
>
> \}
>
> Package\.swift file:
>
> // swift\-tools\-version:4\.0
>
> import PackageDescription
>
> let package = Package\(
>
> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;PluginConsumer&quot;,
>
> &nbsp;&nbsp;&nbsp;&nbsp;dependencies: \[ \.package\(url: &quot;path\_to\_plugin\_interface&quot;, from: &quot;0\.0\.0&quot;\) \],
>
> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[ \.target\(name: &quot;PluginConsumer&quot;, dependencies: \[&quot;PluginConsumer&quot;\]\) \]
>
> \)
>
> This all compiles nicely, MyPlugin project creates dylib file that executable created by PluginConsumer can load, but the problem is with following line:
>
> let plugin = f\(\) as? PluginInterface
>
> Type of the plugin is MyPlugin, but from the consumer&#39;s view, it doesn&#39;t inherit from PluginInterface so I can&#39;t call sayHi\(\) method\. I assume this is because there is no relation between PluginInterface class that compiler uses for MyPlugin project one that it uses for PluginConsumer project\. After library is loaded, they are two completely different classes that happen to share same name\. Is my assumption correct and how do I go about fixing it?
>
> I had an idea I could make PluginInterface emit dynamic library that would be dynamically linked by both MyPlugin and PluginConsumer, thus making them share same PluginInterface class, but I can&#39;t figure out how to do that \(or if it&#39;s right way of doing this\)\.
>
> Any help appreciated :\)
>
> Lope
>
> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
> swift\-users mailing list
> swift\-users@swift\.org &lt;mailto:swift\-users@swift\.org&gt;
> https://lists.swift.org/mailman/listinfo/swift-users

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
swift\-users mailing list
swift\-users@swift\.org &lt;mailto:swift\-users@swift\.org&gt;
https://lists.swift.org/mailman/listinfo/swift-users

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
swift\-users mailing list
swift\-users@swift\.org
https://lists.swift.org/mailman/listinfo/swift-users

I was afraid it will come to that :) I will try to make something tonight,
either it will help you fix it, or I will find out what I did wrong

···

On 8 October 2017 at 09:49, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

Is it possible for you to make a small test package that shows the
problem, and file a bug in bugs.swift.org? There may be something we need
to fix in SwiftPM before this can work (because of our linking model).

- Daniel

On Oct 7, 2017, at 10:42 PM, Ján Kosa via swift-users < > swift-users@swift.org> wrote:

That is exactly what I did. The only package that depends on the protobuf
is the PluginInterface. Both MyPlugin and and PluginConsumer depend on the
PluginInterface and not on the protobuf itself. I had to shuffle around my
dependencies a bit, which resulted in smaller number of dependencies but
they don't make much sense now (as in, some target had to depend on
PluginInterface even if they don't need to, just to get access to
protobuf). I could live with that if it solved the issue, but it didn't.

I am adding my Package.swift files in case I missed something:

PluginInterface:

let package = Package(

name: "PluginInterface",

products: [ .library(name: "PluginInterface", type: .dynamic, targets: [
"PluginInterface"]) ],

dependencies: [ .package(url: "https://github.com/apple/swift-protobuf.git
", from: "0.0.0") ],

targets: [ .target(name: "PluginInterface", dependencies: ["SwiftProtobuf"])
]

)```

MyPlugin:

```swift

let package = Package\(

name: &quot;MyPlugin&quot;,

products: \[ \.library\(name: &quot;MyPlugin&quot;, type: \.dynamic, targets: \[
&quot;PluginImpl&quot;\]\) \],

dependencies: \[

\.package\(url: &quot;path/to/PluginInterface\.git&quot;, from: &quot;0\.0\.0&quot;\),

\],

targets: \[

\.target\(name: &quot;PluginImpl&quot;, dependencies: \[&quot;ProtoBufMessages&quot;\]\),

\.target\(name: &quot;ProtoBufMessages&quot;, dependencies: \[&quot;PluginInterface&quot;\]\)

\]

\)\`\`\`

PluginConsumer:

```swift

let package = Package(

name: "PluginConsumer",

dependencies: [

.package(url: "https://github.com/PerfectlySoft/Perfect-WebSockets.git",
from: "3.0.0"),

.package(url: "https://github.com/PerfectlySoft/Perfect-HTTPServer.git",
from: "3.0.0"),

.package(url: "path/to/PluginInterface", from: "0.0.0"),

.package(url: "https://github.com/krzyzanowskim/CryptoSwift.git", from:
"0.0.0")

],

targets: [

.target(name: "AppMaster", dependencies: ["Shared", "CryptoSwift"]),

.target(name: "PluginConsumer", dependencies: ["Shared", "CryptoSwift"]),

.target(name: "Shared", dependencies: ["ProtoBufMessages",
"PerfectHTTPServer", "PerfectWebSockets"]),

.target(name: "ProtoBufMessages", dependencies: ["PluginInterface"])

]

)```

App master is separate executable that shares some functionality with
PluginConsumer, but it doesn't link against it in any way. I guess it could
be omitted, but I wanted to give you whole thing as it is

On 7 October 2017 at 18:33, Geordie Jay <geojay@gmail.com> wrote:

> Ján Kosa <lopenka@gmail.com> schrieb am Sa. 7. Okt. 2017 um 15:27:
>
>> I tried to use @_exported and it helped somewhat. While I still have
>> same warnings, size of the PluginInterface library went down by 6mb (to
>> 120kb) so it looks like Protobuf is no longer statically linked to it.
>> However, size of PluginConsumer executable went up by same 6mb, it looks
>> like it is linked there twice now.
>>
> To be clear: take protobuf out of the PluginConsumer dependencies.
> Actually, I’m not sure which is which, but protobuf should only be listed
> as a dependency of one package, where it is imported as @_exported. After
> that, your other modules depend on the package that imports / exports
> protobuf.
>
> I also noticed interesting thing. If I build executable using `swift
>> build` the size is around 17mb, when I generate xcode project and build it
>> using that, size is around 200kb, but I get same warnings using both
>> approaches
>>
>> On 7 October 2017 at 15:44, Geordie Jay <geojay@gmail.com> wrote:
>>
>>> Ján Kosa <lopenka@gmail.com> schrieb am Sa. 7. Okt. 2017 um 13:34:
>>>
>>>> I tried swift package clean, but it didn't help
>>>>
>>>> "Try to ensure the plugin provider module (libA) is (only) being
>>>> compiled into its standalone shared library file."
>>>> How do I go about this? It is 3rd party module, it doesn't define any
>>>> products (https://github.com/apple/swift-protobuf.git). Is there
>>>> something I can do in my Package files to make sure it is loaded
>>>> dynamically?
>>>>
>>> When you compile a package depending on protobuf, all the relevant
>>> symbols end up in your package’s library file. So here’s something you
>>> might try:
>>>
>>> import protobuf into your own “PluginProvider” module (package), which
>>> has a shared library product like this: ‘@_exported import Protobuf’ in
>>> some compiled swift file. Then from the other dependent modules “import
>>> PluginProvider” - the protobuf symbols should be available, and all from
>>> one (nonconflicting) source.
>>>
>>> Geordie
>>>
>>>> On 6 October 2017 at 22:52, Geordie Jay <geojay@gmail.com> wrote:
>>>>
>>>>> I think SwiftPM is (incorrectly) compiling A.XYZ <http://a.xyz/>
>>>>> into each of the modules that depend on it, as well as into your intended
>>>>> libA.so file.
>>>>>
>>>>> Try to ensure the plugin provider module (libA) is (only) being
>>>>> compiled into its standalone shared library file. Try cleaning the swiftpm
>>>>> build for one (swift package clean) and ensure the Package.swift files are
>>>>> correctly set up to output the shared library.
>>>>>
>>>>> Sorry I can’t be more specific, I’ve had these same kinds of issues
>>>>> before but I’m not 100% what they were.
>>>>>
>>>>> Geordie
>>>>>
>>>>> Ján Kosa via swift-users <swift-users@swift.org> schrieb am Fr. 6. >>>>>> Okt. 2017 um 14:41:
>>>>>
>>>>>> It worked! Took me a while to iron out details, but it is working
>>>>>> now. Huge thanks sir, I will name my firstborn after you.
>>>>>> Thanks for the @_cdecl("initializePlugin") tip as well, I didn't
>>>>>> know about it and it will be very useful.
>>>>>>
>>>>>> I am having slightly related problem now (it was there before, but I
>>>>>> ignored it for the time being), not sure if I should start a new thread?
>>>>>>
>>>>>> The PluginInterface module has one external dependency on module A,
>>>>>> PluginConsumer has the dependency on module B which has dependency on same
>>>>>> module A that the PluginInterface uses. When I load the plugin library, I
>>>>>> get bunch of errors like:
>>>>>>
>>>>>> Class A.XYZ <http://a.xyz/> is implemented in
>>>>>> both libPluginInterface.dylib and libMyPlugin.dylib
>>>>>>
>>>>>> I know why it is there, but I don't know how to get rid of it. I
>>>>>> can't just remove dependency from PluginConsumer and use the one from
>>>>>> PluginInterface (if that would even work?) because PluginConsumer does not
>>>>>> depend on it directly, but it is going through module B first
>>>>>>
>>>>>> Cheers,
>>>>>> Lope
>>>>>>
>>>>>> On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dunbar@apple.com> >>>>>>> wrote:
>>>>>>
>>>>>>> The way that I have done this in the past is pass a protocol as an
>>>>>>> unsafe pointer to an exposed entry point:
>>>>>>> ```swift
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let entryPoint = dlsym\(handle, “initializePlugin”\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;guard entryPoint \!= nil else \{
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;fatalError\(&quot;missing plugin entry point:
>>>>>>> \\\(pluginPath\)&quot;\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;typealias PluginInitializationFunc = @convention\(c\)
>>>>>>> \(UnsafeRawPointer\) \-&gt; \(\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let f = unsafeBitCast\(entryPoint, to:
>>>>>>> PluginInitializationFunc\.self\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;f\(Unmanaged\.passUnretained\(self\)\.toOpaque\(\)\)
>>>>>>> ```
>>>>>>>
>>>>>>> and then in the plugin convert back to the appropriate type:
>>>>>>>
>>>>>>> ```
>>>>>>> @\_cdecl\(&quot;initializePlugin&quot;\)
>>>>>>> public func initializePlugin\(\_ ptr: UnsafeRawPointer\) \{
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;let manager = Unmanaged&lt;PluginManager&gt;\.fromO
>>>>>>> paque\(ptr\)\.takeUnretainedValue\(\)
>>>>>>> \`\`\`
>>>>>>>
>>>>>>> HTH,
>>>>>>>  \- Daniel
>>>>>>>
>>>>>>> On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift\-users &lt; &gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt; swift\-users@swift\.org&gt; wrote:
>>>>>>>
>>>>>>> Hello folks,
>>>>>>>
>>>>>>> I have been toying with dynamic libraries, trying to implement
>>>>>>> plugin functionality\. I was able to get to the point where I can call
>>>>>>> simple function in loaded library, but I am having troubles starting more
>>>>>>> sophisticated communication channel\.
>>>>>>>
>>>>>>> There are 3 projects
>>>>>>> \- PluginConsumer is an app that loads plugin libraries
>>>>>>> \- MyPlugin is a plugin implementation, output is dynamic library
>>>>>>> that PluginConsumer loads
>>>>>>> \- PluginInterface is common interface that both MyPlugin and
>>>>>>> PluginConsumer use, so that they know how to communicate
>>>>>>>
>>>>>>> My first idea was to have PluginInterface be a simple SPM project
>>>>>>> with single file where the bare\-bones PluginInterface class would be:
>>>>>>>
>>>>>>> open class PluginInterface \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;open func sayHi\(\)
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift file:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;PluginInterface&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;products: \[ \.library\(name: &quot;PluginInterface&quot;, type: \.dynamic,
>>>>>>> targets: \[&quot;PluginInterface&quot;\]\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[ \.target\(name: &quot;PluginInterface&quot;\) \]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> UserPlugin is also very simple project containing only one file:
>>>>>>>
>>>>>>> public func getPlugin\(\) \-&gt; AnyObject \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;return MyPlugin\(\)
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> class MyPlugin: PluginInterface \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;override func sayHi\(\) \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;print\(&quot;Hi from my plugin&quot;\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;MyPlugin&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;products: \[ \.library\(name: &quot;MyPlugin&quot;, type: \.dynamic,
>>>>>>> targets: \[&quot;MyPlugin&quot;\]\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;dependencies: \[ \.package\(url: &quot;url\_to\_PluginInterface&quot;, from:
>>>>>>> &quot;0\.0\.0&quot;\), \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\.target\(name: &quot;PluginInterface&quot;, dependencies: \[
>>>>>>> &quot;PluginInterface&quot;\]\),
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\.target\(name: &quot;MyPlugin&quot;, dependencies: \[&quot;PluginInterface&quot;
>>>>>>> \]\),
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> The PluginConsumer is bit more complicated, but here is relevant
>>>>>>> part \(lib loading and function calling\):
>>>>>>>
>>>>>>> typealias InitFunction = @convention\(c\) \(\) \-&gt; AnyObject
>>>>>>>
>>>>>>> let openRes = dlopen\(pathToLib, RTLD\_NOW|RTLD\_LOCAL\)
>>>>>>>
>>>>>>> if openRes \!= nil \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;defer \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dlclose\(openRes\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;let symbolName = &quot;mangled\_symbol\_name&quot;
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;let sym = dlsym\(openRes, symbolName\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;if sym \!= nil \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let f: InitFunction = unsafeBitCast\(sym, to: InitFunction\.
>>>>>>> self\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let plugin = f\(\) as? PluginInterface
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift file:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;PluginConsumer&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;dependencies: \[ \.package\(url: &quot;path\_to\_plugin\_interface&quot;,
>>>>>>> from: &quot;0\.0\.0&quot;\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[ \.target\(name: &quot;PluginConsumer&quot;, dependencies: \[
>>>>>>> &quot;PluginConsumer&quot;\]\) \]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> This all compiles nicely, MyPlugin project creates dylib file that
>>>>>>> executable created by PluginConsumer can load, but the problem is with
>>>>>>> following line:
>>>>>>>
>>>>>>> let plugin = f\(\) as? PluginInterface
>>>>>>>
>>>>>>> Type of the plugin is MyPlugin, but from the consumer&#39;s view, it
>>>>>>> doesn&#39;t inherit from PluginInterface so I can&#39;t call sayHi\(\) method\. I
>>>>>>> assume this is because there is no relation between PluginInterface class
>>>>>>> that compiler uses for MyPlugin project one that it uses for PluginConsumer
>>>>>>> project\. After library is loaded, they are two completely different classes
>>>>>>> that happen to share same name\. Is my assumption correct and how do I go
>>>>>>> about fixing it?
>>>>>>>
>>>>>>> I had an idea I could make PluginInterface emit dynamic library
>>>>>>> that would be dynamically linked by both MyPlugin and PluginConsumer, thus
>>>>>>> making them share same PluginInterface class, but I can&#39;t figure out how to
>>>>>>> do that \(or if it&#39;s right way of doing this\)\.
>>>>>>>
>>>>>>> Any help appreciated :\)
>>>>>>>
>>>>>>> Lope
>>>>>>> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>>>>>>> swift\-users mailing list
>>>>>>> swift\-users@swift\.org
>>>>>>> https://lists.swift.org/mailman/listinfo/swift-users
>>>>>>>
>>>>>>
>>>>>> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>>>>>> swift\-users mailing list
>>>>>> swift\-users@swift\.org
>>>>>> https://lists.swift.org/mailman/listinfo/swift-users
>>>>>>

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
swift\-users mailing list
swift\-users@swift\.org
https://lists.swift.org/mailman/listinfo/swift-users

Hey guys,

I have been able to create simplest possible setup that exhibits this
problem, the protobuf module wasn't necessary. All I import is
PluginInterface and I get the error:

objc[66212]: Class _TtC15PluginInterface15PluginInterface is implemented in
both
/Users/Lope/Dev/swift/SwiftPlugins/PluginConsumer/.build/x86_64-apple-macosx10.10/debug/libPluginInterface.dylib
(0x10fc7b668) and
/Users/Lope/Dev/swift/SwiftPlugins/PluginImplementation/.build/x86_64-apple-macosx10.10/debug/libPluginInterface.dylib
(0x1107dd668). One of the two will be used. Which one is undefined.

You can find modules on my github:

Could you have a quick look if I didn't mess up or missed something and I
will create bug report. If you want to run the code for yourself, you will
have to change the path to dylib in the PluginConsumer (I don't know how to
use relative path and didn't have time to find out yet)

···

On 8 October 2017 at 09:51, Ján Kosa via swift-users <swift-users@swift.org> wrote:

I was afraid it will come to that :) I will try to make something tonight,
either it will help you fix it, or I will find out what I did wrong

On 8 October 2017 at 09:49, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

Is it possible for you to make a small test package that shows the
problem, and file a bug in bugs.swift.org? There may be something we
need to fix in SwiftPM before this can work (because of our linking model).

- Daniel

On Oct 7, 2017, at 10:42 PM, Ján Kosa via swift-users < >> swift-users@swift.org> wrote:

That is exactly what I did. The only package that depends on the protobuf
is the PluginInterface. Both MyPlugin and and PluginConsumer depend on the
PluginInterface and not on the protobuf itself. I had to shuffle around my
dependencies a bit, which resulted in smaller number of dependencies but
they don't make much sense now (as in, some target had to depend on
PluginInterface even if they don't need to, just to get access to
protobuf). I could live with that if it solved the issue, but it didn't.

I am adding my Package.swift files in case I missed something:

PluginInterface:

let package = Package(

name: "PluginInterface",

products: [ .library(name: "PluginInterface", type: .dynamic, targets: [
"PluginInterface"]) ],

dependencies: [ .package(url: "https://github.com/apple/swif
t-protobuf.git", from: "0.0.0") ],

targets: [ .target(name: "PluginInterface", dependencies: [
"SwiftProtobuf"]) ]

)```

MyPlugin:

```swift

let package = Package\(

name: &quot;MyPlugin&quot;,

products: \[ \.library\(name: &quot;MyPlugin&quot;, type: \.dynamic, targets: \[
&quot;PluginImpl&quot;\]\) \],

dependencies: \[

\.package\(url: &quot;path/to/PluginInterface\.git&quot;, from: &quot;0\.0\.0&quot;\),

\],

targets: \[

\.target\(name: &quot;PluginImpl&quot;, dependencies: \[&quot;ProtoBufMessages&quot;\]\),

\.target\(name: &quot;ProtoBufMessages&quot;, dependencies: \[&quot;PluginInterface&quot;\]\)

\]

\)\`\`\`

PluginConsumer:

```swift

let package = Package(

name: "PluginConsumer",

dependencies: [

.package(url: "https://github.com/PerfectlySoft/Perfect-WebSockets.git",
from: "3.0.0"),

.package(url: "https://github.com/PerfectlySoft/Perfect-HTTPServer.git",
from: "3.0.0"),

.package(url: "path/to/PluginInterface", from: "0.0.0"),

.package(url: "https://github.com/krzyzanowskim/CryptoSwift.git", from:
"0.0.0")

],

targets: [

.target(name: "AppMaster", dependencies: ["Shared", "CryptoSwift"]),

.target(name: "PluginConsumer", dependencies: ["Shared", "CryptoSwift"]),

.target(name: "Shared", dependencies: ["ProtoBufMessages",
"PerfectHTTPServer", "PerfectWebSockets"]),

.target(name: "ProtoBufMessages", dependencies: ["PluginInterface"])

]

)```

App master is separate executable that shares some functionality with
PluginConsumer, but it doesn't link against it in any way. I guess it could
be omitted, but I wanted to give you whole thing as it is

On 7 October 2017 at 18:33, Geordie Jay <geojay@gmail.com> wrote:

> Ján Kosa <lopenka@gmail.com> schrieb am Sa. 7. Okt. 2017 um 15:27:
>
>> I tried to use @_exported and it helped somewhat. While I still have
>> same warnings, size of the PluginInterface library went down by 6mb (to
>> 120kb) so it looks like Protobuf is no longer statically linked to it.
>> However, size of PluginConsumer executable went up by same 6mb, it looks
>> like it is linked there twice now.
>>
> To be clear: take protobuf out of the PluginConsumer dependencies.
> Actually, I’m not sure which is which, but protobuf should only be listed
> as a dependency of one package, where it is imported as @_exported. After
> that, your other modules depend on the package that imports / exports
> protobuf.
>
> I also noticed interesting thing. If I build executable using `swift
>> build` the size is around 17mb, when I generate xcode project and build it
>> using that, size is around 200kb, but I get same warnings using both
>> approaches
>>
>> On 7 October 2017 at 15:44, Geordie Jay <geojay@gmail.com> wrote:
>>
>>> Ján Kosa <lopenka@gmail.com> schrieb am Sa. 7. Okt. 2017 um 13:34:
>>>
>>>> I tried swift package clean, but it didn't help
>>>>
>>>> "Try to ensure the plugin provider module (libA) is (only) being
>>>> compiled into its standalone shared library file."
>>>> How do I go about this? It is 3rd party module, it doesn't define any
>>>> products (https://github.com/apple/swift-protobuf.git). Is there
>>>> something I can do in my Package files to make sure it is loaded
>>>> dynamically?
>>>>
>>> When you compile a package depending on protobuf, all the relevant
>>> symbols end up in your package’s library file. So here’s something you
>>> might try:
>>>
>>> import protobuf into your own “PluginProvider” module (package), which
>>> has a shared library product like this: ‘@_exported import Protobuf’ in
>>> some compiled swift file. Then from the other dependent modules “import
>>> PluginProvider” - the protobuf symbols should be available, and all from
>>> one (nonconflicting) source.
>>>
>>> Geordie
>>>
>>>> On 6 October 2017 at 22:52, Geordie Jay <geojay@gmail.com> wrote:
>>>>
>>>>> I think SwiftPM is (incorrectly) compiling A.XYZ <http://a.xyz/>
>>>>> into each of the modules that depend on it, as well as into your intended
>>>>> libA.so file.
>>>>>
>>>>> Try to ensure the plugin provider module (libA) is (only) being
>>>>> compiled into its standalone shared library file. Try cleaning the swiftpm
>>>>> build for one (swift package clean) and ensure the Package.swift files are
>>>>> correctly set up to output the shared library.
>>>>>
>>>>> Sorry I can’t be more specific, I’ve had these same kinds of issues
>>>>> before but I’m not 100% what they were.
>>>>>
>>>>> Geordie
>>>>>
>>>>> Ján Kosa via swift-users <swift-users@swift.org> schrieb am Fr. 6. >>>>>>> Okt. 2017 um 14:41:
>>>>>
>>>>>> It worked! Took me a while to iron out details, but it is working
>>>>>> now. Huge thanks sir, I will name my firstborn after you.
>>>>>> Thanks for the @_cdecl("initializePlugin") tip as well, I didn't
>>>>>> know about it and it will be very useful.
>>>>>>
>>>>>> I am having slightly related problem now (it was there before, but
>>>>>> I ignored it for the time being), not sure if I should start a new thread?
>>>>>>
>>>>>> The PluginInterface module has one external dependency on module A,
>>>>>> PluginConsumer has the dependency on module B which has dependency on same
>>>>>> module A that the PluginInterface uses. When I load the plugin library, I
>>>>>> get bunch of errors like:
>>>>>>
>>>>>> Class A.XYZ <http://a.xyz/> is implemented in
>>>>>> both libPluginInterface.dylib and libMyPlugin.dylib
>>>>>>
>>>>>> I know why it is there, but I don't know how to get rid of it. I
>>>>>> can't just remove dependency from PluginConsumer and use the one from
>>>>>> PluginInterface (if that would even work?) because PluginConsumer does not
>>>>>> depend on it directly, but it is going through module B first
>>>>>>
>>>>>> Cheers,
>>>>>> Lope
>>>>>>
>>>>>> On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dunbar@apple.com> >>>>>>>> wrote:
>>>>>>
>>>>>>> The way that I have done this in the past is pass a protocol as an
>>>>>>> unsafe pointer to an exposed entry point:
>>>>>>> ```swift
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let entryPoint = dlsym\(handle, “initializePlugin”\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;guard entryPoint \!= nil else \{
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;fatalError\(&quot;missing plugin entry point:
>>>>>>> \\\(pluginPath\)&quot;\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;typealias PluginInitializationFunc = @convention\(c\)
>>>>>>> \(UnsafeRawPointer\) \-&gt; \(\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let f = unsafeBitCast\(entryPoint, to:
>>>>>>> PluginInitializationFunc\.self\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;f\(Unmanaged\.passUnretained\(self\)\.toOpaque\(\)\)
>>>>>>> ```
>>>>>>>
>>>>>>> and then in the plugin convert back to the appropriate type:
>>>>>>>
>>>>>>> ```
>>>>>>> @\_cdecl\(&quot;initializePlugin&quot;\)
>>>>>>> public func initializePlugin\(\_ ptr: UnsafeRawPointer\) \{
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;let manager = Unmanaged&lt;PluginManager&gt;\.fromO
>>>>>>> paque\(ptr\)\.takeUnretainedValue\(\)
>>>>>>> \`\`\`
>>>>>>>
>>>>>>> HTH,
>>>>>>>  \- Daniel
>>>>>>>
>>>>>>> On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift\-users &lt; &gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt; swift\-users@swift\.org&gt; wrote:
>>>>>>>
>>>>>>> Hello folks,
>>>>>>>
>>>>>>> I have been toying with dynamic libraries, trying to implement
>>>>>>> plugin functionality\. I was able to get to the point where I can call
>>>>>>> simple function in loaded library, but I am having troubles starting more
>>>>>>> sophisticated communication channel\.
>>>>>>>
>>>>>>> There are 3 projects
>>>>>>> \- PluginConsumer is an app that loads plugin libraries
>>>>>>> \- MyPlugin is a plugin implementation, output is dynamic library
>>>>>>> that PluginConsumer loads
>>>>>>> \- PluginInterface is common interface that both MyPlugin and
>>>>>>> PluginConsumer use, so that they know how to communicate
>>>>>>>
>>>>>>> My first idea was to have PluginInterface be a simple SPM project
>>>>>>> with single file where the bare\-bones PluginInterface class would be:
>>>>>>>
>>>>>>> open class PluginInterface \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;open func sayHi\(\)
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift file:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;PluginInterface&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;products: \[ \.library\(name: &quot;PluginInterface&quot;, type: \.dynamic,
>>>>>>> targets: \[&quot;PluginInterface&quot;\]\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[ \.target\(name: &quot;PluginInterface&quot;\) \]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> UserPlugin is also very simple project containing only one file:
>>>>>>>
>>>>>>> public func getPlugin\(\) \-&gt; AnyObject \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;return MyPlugin\(\)
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> class MyPlugin: PluginInterface \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;override func sayHi\(\) \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;print\(&quot;Hi from my plugin&quot;\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;MyPlugin&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;products: \[ \.library\(name: &quot;MyPlugin&quot;, type: \.dynamic,
>>>>>>> targets: \[&quot;MyPlugin&quot;\]\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;dependencies: \[ \.package\(url: &quot;url\_to\_PluginInterface&quot;, from:
>>>>>>> &quot;0\.0\.0&quot;\), \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\.target\(name: &quot;PluginInterface&quot;, dependencies: \[
>>>>>>> &quot;PluginInterface&quot;\]\),
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\.target\(name: &quot;MyPlugin&quot;, dependencies: \[&quot;PluginInterface&quot;
>>>>>>> \]\),
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> The PluginConsumer is bit more complicated, but here is relevant
>>>>>>> part \(lib loading and function calling\):
>>>>>>>
>>>>>>> typealias InitFunction = @convention\(c\) \(\) \-&gt; AnyObject
>>>>>>>
>>>>>>> let openRes = dlopen\(pathToLib, RTLD\_NOW|RTLD\_LOCAL\)
>>>>>>>
>>>>>>> if openRes \!= nil \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;defer \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dlclose\(openRes\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;let symbolName = &quot;mangled\_symbol\_name&quot;
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;let sym = dlsym\(openRes, symbolName\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;if sym \!= nil \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let f: InitFunction = unsafeBitCast\(sym, to: InitFunction\.
>>>>>>> self\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let plugin = f\(\) as? PluginInterface
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift file:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;PluginConsumer&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;dependencies: \[ \.package\(url: &quot;path\_to\_plugin\_interface&quot;,
>>>>>>> from: &quot;0\.0\.0&quot;\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[ \.target\(name: &quot;PluginConsumer&quot;, dependencies: \[
>>>>>>> &quot;PluginConsumer&quot;\]\) \]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> This all compiles nicely, MyPlugin project creates dylib file that
>>>>>>> executable created by PluginConsumer can load, but the problem is with
>>>>>>> following line:
>>>>>>>
>>>>>>> let plugin = f\(\) as? PluginInterface
>>>>>>>
>>>>>>> Type of the plugin is MyPlugin, but from the consumer&#39;s view, it
>>>>>>> doesn&#39;t inherit from PluginInterface so I can&#39;t call sayHi\(\) method\. I
>>>>>>> assume this is because there is no relation between PluginInterface class
>>>>>>> that compiler uses for MyPlugin project one that it uses for PluginConsumer
>>>>>>> project\. After library is loaded, they are two completely different classes
>>>>>>> that happen to share same name\. Is my assumption correct and how do I go
>>>>>>> about fixing it?
>>>>>>>
>>>>>>> I had an idea I could make PluginInterface emit dynamic library
>>>>>>> that would be dynamically linked by both MyPlugin and PluginConsumer, thus
>>>>>>> making them share same PluginInterface class, but I can&#39;t figure out how to
>>>>>>> do that \(or if it&#39;s right way of doing this\)\.
>>>>>>>
>>>>>>> Any help appreciated :\)
>>>>>>>
>>>>>>> Lope
>>>>>>> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>>>>>>> swift\-users mailing list
>>>>>>> swift\-users@swift\.org
>>>>>>> https://lists.swift.org/mailman/listinfo/swift-users
>>>>>>>
>>>>>>
>>>>>> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>>>>>> swift\-users mailing list
>>>>>> swift\-users@swift\.org
>>>>>> https://lists.swift.org/mailman/listinfo/swift-users
>>>>>>

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
swift\-users mailing list
swift\-users@swift\.org
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users

I filed bug report: [SR-6091] Module linked multiple times if dylib is importing same module that main executable does · Issue #4939 · apple/swift-package-manager · GitHub

···

On 8 October 2017 at 20:31, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

Can you file this on bugs.swift.org? We are likely to lose track of it if
it just an email thread

- Daniel

On Oct 8, 2017, at 8:27 AM, Ján Kosa <lopenka@gmail.com> wrote:

Hey guys,

I have been able to create simplest possible setup that exhibits this
problem, the protobuf module wasn't necessary. All I import is
PluginInterface and I get the error:

objc[66212]: Class _TtC15PluginInterface15PluginInterface is implemented
in both /Users/Lope/Dev/swift/SwiftPlugins/PluginConsumer/.
build/x86_64-apple-macosx10.10/debug/libPluginInterface.dylib
(0x10fc7b668) and /Users/Lope/Dev/swift/SwiftPlugins/
PluginImplementation/.build/x86_64-apple-macosx10.10/debug/libPluginInterface.dylib
(0x1107dd668). One of the two will be used. Which one is undefined.

You can find modules on my github:

GitHub - Lopdo/SwiftPlugins-PluginInterface

GitHub - Lopdo/SwiftPlugins-PluginImplementation

GitHub - Lopdo/SwiftPlugins-PluginConsumer

Could you have a quick look if I didn't mess up or missed something and I
will create bug report. If you want to run the code for yourself, you will
have to change the path to dylib in the PluginConsumer (I don't know how to
use relative path and didn't have time to find out yet)

On 8 October 2017 at 09:51, Ján Kosa via swift-users < > swift-users@swift.org> wrote:

I was afraid it will come to that :) I will try to make something
tonight, either it will help you fix it, or I will find out what I did wrong

On 8 October 2017 at 09:49, Daniel Dunbar <daniel_dunbar@apple.com> >> wrote:

Is it possible for you to make a small test package that shows the
problem, and file a bug in bugs.swift.org? There may be something we
need to fix in SwiftPM before this can work (because of our linking model).

- Daniel

On Oct 7, 2017, at 10:42 PM, Ján Kosa via swift-users < >>> swift-users@swift.org> wrote:

That is exactly what I did. The only package that depends on the
protobuf is the PluginInterface. Both MyPlugin and and PluginConsumer
depend on the PluginInterface and not on the protobuf itself. I had to
shuffle around my dependencies a bit, which resulted in smaller number of
dependencies but they don't make much sense now (as in, some target had to
depend on PluginInterface even if they don't need to, just to get access to
protobuf). I could live with that if it solved the issue, but it didn't.

I am adding my Package.swift files in case I missed something:

PluginInterface:

let package = Package(

name: "PluginInterface",

products: [ .library(name: "PluginInterface", type: .dynamic, targets: [
"PluginInterface"]) ],

dependencies: [ .package(url: "https://github.com/apple/swif
t-protobuf.git", from: "0.0.0") ],

targets: [ .target(name: "PluginInterface", dependencies: [
"SwiftProtobuf"]) ]

)```

MyPlugin:

```swift

let package = Package\(

name: &quot;MyPlugin&quot;,

products: \[ \.library\(name: &quot;MyPlugin&quot;, type: \.dynamic, targets: \[
&quot;PluginImpl&quot;\]\) \],

dependencies: \[

\.package\(url: &quot;path/to/PluginInterface\.git&quot;, from: &quot;0\.0\.0&quot;\),

\],

targets: \[

\.target\(name: &quot;PluginImpl&quot;, dependencies: \[&quot;ProtoBufMessages&quot;\]\),

\.target\(name: &quot;ProtoBufMessages&quot;, dependencies: \[&quot;PluginInterface&quot;\]\)

\]

\)\`\`\`

PluginConsumer:

```swift

let package = Package(

name: "PluginConsumer",

dependencies: [

.package(url: "https://github.com/PerfectlySoft/Perfect-WebSockets.git",
from: "3.0.0"),

.package(url: "https://github.com/PerfectlySoft/Perfect-HTTPServer.git",
from: "3.0.0"),

.package(url: "path/to/PluginInterface", from: "0.0.0"),

.package(url: "https://github.com/krzyzanowskim/CryptoSwift.git", from:
"0.0.0")

],

targets: [

.target(name: "AppMaster", dependencies: ["Shared", "CryptoSwift"]),

.target(name: "PluginConsumer", dependencies: ["Shared", "CryptoSwift"
]),

.target(name: "Shared", dependencies: ["ProtoBufMessages",
"PerfectHTTPServer", "PerfectWebSockets"]),

.target(name: "ProtoBufMessages", dependencies: ["PluginInterface"])

]

)```

App master is separate executable that shares some functionality with
PluginConsumer, but it doesn't link against it in any way. I guess it could
be omitted, but I wanted to give you whole thing as it is

On 7 October 2017 at 18:33, Geordie Jay <geojay@gmail.com> wrote:

> Ján Kosa <lopenka@gmail.com> schrieb am Sa. 7. Okt. 2017 um 15:27:
>
>> I tried to use @_exported and it helped somewhat. While I still have
>> same warnings, size of the PluginInterface library went down by 6mb (to
>> 120kb) so it looks like Protobuf is no longer statically linked to it.
>> However, size of PluginConsumer executable went up by same 6mb, it looks
>> like it is linked there twice now.
>>
> To be clear: take protobuf out of the PluginConsumer dependencies.
> Actually, I’m not sure which is which, but protobuf should only be listed
> as a dependency of one package, where it is imported as @_exported. After
> that, your other modules depend on the package that imports / exports
> protobuf.
>
> I also noticed interesting thing. If I build executable using `swift
>> build` the size is around 17mb, when I generate xcode project and build it
>> using that, size is around 200kb, but I get same warnings using both
>> approaches
>>
>> On 7 October 2017 at 15:44, Geordie Jay <geojay@gmail.com> wrote:
>>
>>> Ján Kosa <lopenka@gmail.com> schrieb am Sa. 7. Okt. 2017 um 13:34:
>>>
>>>> I tried swift package clean, but it didn't help
>>>>
>>>> "Try to ensure the plugin provider module (libA) is (only) being
>>>> compiled into its standalone shared library file."
>>>> How do I go about this? It is 3rd party module, it doesn't define
>>>> any products (https://github.com/apple/swift-protobuf.git). Is
>>>> there something I can do in my Package files to make sure it is loaded
>>>> dynamically?
>>>>
>>> When you compile a package depending on protobuf, all the relevant
>>> symbols end up in your package’s library file. So here’s something you
>>> might try:
>>>
>>> import protobuf into your own “PluginProvider” module (package),
>>> which has a shared library product like this: ‘@_exported import Protobuf’
>>> in some compiled swift file. Then from the other dependent modules “import
>>> PluginProvider” - the protobuf symbols should be available, and all from
>>> one (nonconflicting) source.
>>>
>>> Geordie
>>>
>>>> On 6 October 2017 at 22:52, Geordie Jay <geojay@gmail.com> wrote:
>>>>
>>>>> I think SwiftPM is (incorrectly) compiling A.XYZ <http://a.xyz/>
>>>>> into each of the modules that depend on it, as well as into your intended
>>>>> libA.so file.
>>>>>
>>>>> Try to ensure the plugin provider module (libA) is (only) being
>>>>> compiled into its standalone shared library file. Try cleaning the swiftpm
>>>>> build for one (swift package clean) and ensure the Package.swift files are
>>>>> correctly set up to output the shared library.
>>>>>
>>>>> Sorry I can’t be more specific, I’ve had these same kinds of issues
>>>>> before but I’m not 100% what they were.
>>>>>
>>>>> Geordie
>>>>>
>>>>> Ján Kosa via swift-users <swift-users@swift.org> schrieb am Fr. 6. >>>>>>>> Okt. 2017 um 14:41:
>>>>>
>>>>>> It worked! Took me a while to iron out details, but it is working
>>>>>> now. Huge thanks sir, I will name my firstborn after you.
>>>>>> Thanks for the @_cdecl("initializePlugin") tip as well, I didn't
>>>>>> know about it and it will be very useful.
>>>>>>
>>>>>> I am having slightly related problem now (it was there before, but
>>>>>> I ignored it for the time being), not sure if I should start a new thread?
>>>>>>
>>>>>> The PluginInterface module has one external dependency on module
>>>>>> A, PluginConsumer has the dependency on module B which has dependency on
>>>>>> same module A that the PluginInterface uses. When I load the plugin
>>>>>> library, I get bunch of errors like:
>>>>>>
>>>>>> Class A.XYZ <http://a.xyz/> is implemented in
>>>>>> both libPluginInterface.dylib and libMyPlugin.dylib
>>>>>>
>>>>>> I know why it is there, but I don't know how to get rid of it. I
>>>>>> can't just remove dependency from PluginConsumer and use the one from
>>>>>> PluginInterface (if that would even work?) because PluginConsumer does not
>>>>>> depend on it directly, but it is going through module B first
>>>>>>
>>>>>> Cheers,
>>>>>> Lope
>>>>>>
>>>>>> On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dunbar@apple.com >>>>>>>>> > wrote:
>>>>>>
>>>>>>> The way that I have done this in the past is pass a protocol as
>>>>>>> an unsafe pointer to an exposed entry point:
>>>>>>> ```swift
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let entryPoint = dlsym\(handle, “initializePlugin”\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;guard entryPoint \!= nil else \{
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;fatalError\(&quot;missing plugin entry point:
>>>>>>> \\\(pluginPath\)&quot;\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;typealias PluginInitializationFunc = @convention\(c\)
>>>>>>> \(UnsafeRawPointer\) \-&gt; \(\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let f = unsafeBitCast\(entryPoint, to:
>>>>>>> PluginInitializationFunc\.self\)
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;f\(Unmanaged\.passUnretained\(self\)\.toOpaque\(\)\)
>>>>>>> ```
>>>>>>>
>>>>>>> and then in the plugin convert back to the appropriate type:
>>>>>>>
>>>>>>> ```
>>>>>>> @\_cdecl\(&quot;initializePlugin&quot;\)
>>>>>>> public func initializePlugin\(\_ ptr: UnsafeRawPointer\) \{
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;let manager = Unmanaged&lt;PluginManager&gt;\.fromO
>>>>>>> paque\(ptr\)\.takeUnretainedValue\(\)
>>>>>>> \`\`\`
>>>>>>>
>>>>>>> HTH,
>>>>>>>  \- Daniel
>>>>>>>
>>>>>>> On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift\-users &lt; &gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt; swift\-users@swift\.org&gt; wrote:
>>>>>>>
>>>>>>> Hello folks,
>>>>>>>
>>>>>>> I have been toying with dynamic libraries, trying to implement
>>>>>>> plugin functionality\. I was able to get to the point where I can call
>>>>>>> simple function in loaded library, but I am having troubles starting more
>>>>>>> sophisticated communication channel\.
>>>>>>>
>>>>>>> There are 3 projects
>>>>>>> \- PluginConsumer is an app that loads plugin libraries
>>>>>>> \- MyPlugin is a plugin implementation, output is dynamic library
>>>>>>> that PluginConsumer loads
>>>>>>> \- PluginInterface is common interface that both MyPlugin and
>>>>>>> PluginConsumer use, so that they know how to communicate
>>>>>>>
>>>>>>> My first idea was to have PluginInterface be a simple SPM project
>>>>>>> with single file where the bare\-bones PluginInterface class would be:
>>>>>>>
>>>>>>> open class PluginInterface \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;open func sayHi\(\)
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift file:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;PluginInterface&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;products: \[ \.library\(name: &quot;PluginInterface&quot;, type:
>>>>>>> \.dynamic, targets: \[&quot;PluginInterface&quot;\]\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[ \.target\(name: &quot;PluginInterface&quot;\) \]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> UserPlugin is also very simple project containing only one file:
>>>>>>>
>>>>>>> public func getPlugin\(\) \-&gt; AnyObject \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;return MyPlugin\(\)
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> class MyPlugin: PluginInterface \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;override func sayHi\(\) \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;print\(&quot;Hi from my plugin&quot;\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;MyPlugin&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;products: \[ \.library\(name: &quot;MyPlugin&quot;, type: \.dynamic,
>>>>>>> targets: \[&quot;MyPlugin&quot;\]\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;dependencies: \[ \.package\(url: &quot;url\_to\_PluginInterface&quot;,
>>>>>>> from: &quot;0\.0\.0&quot;\), \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\.target\(name: &quot;PluginInterface&quot;, dependencies: \[
>>>>>>> &quot;PluginInterface&quot;\]\),
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\.target\(name: &quot;MyPlugin&quot;, dependencies: \[
>>>>>>> &quot;PluginInterface&quot;\]\),
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> The PluginConsumer is bit more complicated, but here is relevant
>>>>>>> part \(lib loading and function calling\):
>>>>>>>
>>>>>>> typealias InitFunction = @convention\(c\) \(\) \-&gt; AnyObject
>>>>>>>
>>>>>>> let openRes = dlopen\(pathToLib, RTLD\_NOW|RTLD\_LOCAL\)
>>>>>>>
>>>>>>> if openRes \!= nil \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;defer \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dlclose\(openRes\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;let symbolName = &quot;mangled\_symbol\_name&quot;
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;let sym = dlsym\(openRes, symbolName\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;if sym \!= nil \{
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let f: InitFunction = unsafeBitCast\(sym, to: InitFunction
>>>>>>> \.self\)
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let plugin = f\(\) as? PluginInterface
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;\}
>>>>>>>
>>>>>>> \}
>>>>>>>
>>>>>>> Package\.swift file:
>>>>>>>
>>>>>>> // swift\-tools\-version:4\.0
>>>>>>>
>>>>>>> import PackageDescription
>>>>>>>
>>>>>>> let package = Package\(
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;name: &quot;PluginConsumer&quot;,
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;dependencies: \[ \.package\(url: &quot;path\_to\_plugin\_interface&quot;,
>>>>>>> from: &quot;0\.0\.0&quot;\) \],
>>>>>>>
>>>>>>> &nbsp;&nbsp;&nbsp;&nbsp;targets: \[ \.target\(name: &quot;PluginConsumer&quot;, dependencies: \[
>>>>>>> &quot;PluginConsumer&quot;\]\) \]
>>>>>>>
>>>>>>> \)
>>>>>>>
>>>>>>> This all compiles nicely, MyPlugin project creates dylib file
>>>>>>> that executable created by PluginConsumer can load, but the problem is with
>>>>>>> following line:
>>>>>>>
>>>>>>> let plugin = f\(\) as? PluginInterface
>>>>>>>
>>>>>>> Type of the plugin is MyPlugin, but from the consumer&#39;s view, it
>>>>>>> doesn&#39;t inherit from PluginInterface so I can&#39;t call sayHi\(\) method\. I
>>>>>>> assume this is because there is no relation between PluginInterface class
>>>>>>> that compiler uses for MyPlugin project one that it uses for PluginConsumer
>>>>>>> project\. After library is loaded, they are two completely different classes
>>>>>>> that happen to share same name\. Is my assumption correct and how do I go
>>>>>>> about fixing it?
>>>>>>>
>>>>>>> I had an idea I could make PluginInterface emit dynamic library
>>>>>>> that would be dynamically linked by both MyPlugin and PluginConsumer, thus
>>>>>>> making them share same PluginInterface class, but I can&#39;t figure out how to
>>>>>>> do that \(or if it&#39;s right way of doing this\)\.
>>>>>>>
>>>>>>> Any help appreciated :\)
>>>>>>>
>>>>>>> Lope
>>>>>>> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>>>>>>> swift\-users mailing list
>>>>>>> swift\-users@swift\.org
>>>>>>> https://lists.swift.org/mailman/listinfo/swift-users
>>>>>>>
>>>>>>
>>>>>> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>>>>>> swift\-users mailing list
>>>>>> swift\-users@swift\.org
>>>>>> https://lists.swift.org/mailman/listinfo/swift-users
>>>>>>

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
swift\-users mailing list
swift\-users@swift\.org
https://lists.swift.org/mailman/listinfo/swift-users

_______________________________________________
swift-users mailing list
swift-users@swift.org
https://lists.swift.org/mailman/listinfo/swift-users