Dynamic method replacement


(Arnold Schwaighofer) #22

This is a good question. Incomplete knowledge.

For example, developers could anticipate the need for change in security critical code - say a function that enforces a security policy on a server. They could mark the function as dynamic ahead of time.
Later, when they realize their caution was justified they could ship a module only containing the code for the fixed function. The module containing the fixed function can be loaded, while the server application using it continues to run, replacing the original faulty version.


(Dave Abrahams) #23

I understand that, but as I've said, the motivation cited in the pitch (“a developer foresees the necessity of future extension/replacement of a method”) seems to be addressed by resilience. What kind of foresight would lead one to make a method dynamically replaceable? “You can do this in Objective-C” isn't a good enough reason to add it to Swift. We need examples that show how/why the capability is needed.


(Jeffrey Macko) #24

Hi, @Arnold thanks for making this proposal.

Full disclosure I have worked some time ago on a proposal that aims to add features that are similar to this one, but I think in a more secure and controlled way.

FWIW I'm firmly against this approach because most of the time method swizzling is abused mainly by third-party and it makes you lost control of how your code or the code you are working with operate. There are issues from prioritizing witch implementation win to how to protect my code against this approach. I have lost so much time debugging/patching issues introduced by another library swizzled code that thinking about adding this kind of feature in Swift scares me a bit.

On the bright side, I think we can have the same feature by using an Aspect Oriented Programming approach the wrap behavior is the one that allows you to manipulate the output of a function.

@dabrahams can you please explain what resilience is?

@Rod_Brown I have developed libraries for a long time, and I'm against doing magic tricks behind the developer's back I instead prefer always to give them all the tools they might need to work with and if there is a bug I prefer to fix it myself and re-ship the lib.

@Karl we can install hooks with an Aspect Oriented Programming approach


(Chris Lattner) #25

Hi Arnold,

I'm +1 on adding this to Swift, it fills in a hole that is still sadly @objc-only but that should be available to all Swift types everywhere. I am very glad that dynamic isn't the default, and that API authors are in control.

Question on your design though: why is the argument to "for:" a string? It seems that it should be a declname (not in string quotes) so the compiler can type check it, overload resolve it, etc. You should be able to a likewise specify which member of an overload set is important (e.g. through type matching with your implementation) and should fully support Swift's keyword argument setup). It's not clear if your approach already does this - the double quotes make it seem like it isn't intended, but maybe I'm misunderstanding it.

-Chris


(Jean-Daniel) #26

This is the exact same use case that was presented against making class and methods "final" by default (not open). And at that time, it was simply rejected as not meaningful.


(Joe Groff) #27

There were many tradeoffs involved in that discussion, and I think it's a misrepresentation to say that those concerns were "rejected as not meaningful". Swift tries to make API promises as explicit as possible, and that was the guiding principle behind that discussion. Furthermore, classes are not "final" by default but not publicly subclassable by default; a library can still provide its own subclasses, and choose to make a class open in a future version, whereas final is a promise to never have subclasses.

Being able to opt in to dynamic replacement with a proposal like this is more or exactly in line with the philosophy behind the discussion around the default behavior of public classes. In the same way that a library can choose to make its classes open, with this proposal, it can choose to make its definitions replaceable by clients.


(Jean-Daniel) #28

Method swizzling does not look like a good candidate for that. When you swizzle a method, you do it for all instances. Actually, KVO works by creating a subclass, and use isa swizzling, so only instances that are observed have to pay the cost of KVO.

I imagine, it is possible to do something similar in the Swift runtime with something like vtable swizzling (which would even remove the need for the method to be dynamic)


(Jean-Daniel) #29

That would mean that the proposition should instead introduce a new @replaceable attribute. A developer may need to use the dynamic keyword without choosing to make the method replaceable.


(Joe Groff) #30

That is what dynamic already means in Swift, though, that the method implementation is replaceable at runtime.

To Jordan's point, the @dynamicReplacement(for:) replacement mechanism Arnold proposes here would not be sufficient to implement something like KVO by itself, but the same runtime and compile-time mechanisms dynamic introduce could most likely apply to a dynamic subclassing mechanism as well. We'd still need dynamic as a way to tell the compiler that the method implementation may change at runtime from what the compiler might otherwise assume it is.


(Arnold Schwaighofer) #32

I gave an example further up in the conversation:


For example, developers could anticipate the need for change in security critical code - say a function that enforces a security policy on a server. They could mark the function as dynamic ahead of time.
Later, when they realize their caution was justified they could ship a module only containing the code for the fixed function. The module containing the fixed function can be loaded, while the server application using it continues to run, replacing the original faulty version.


(Arnold Schwaighofer) #33

Question on your design though: why is the argument to "for:" a string? It seems that it should be a declname (not in string quotes) so the compiler can type check it, overload resolve it, etc. You should be able to a likewise specify which member of an overload set is important (e.g. through type matching with your implementation) and should fully support Swift's keyword argument setup). It's not clear if your approach already does this - the double quotes make it seem like it isn't intended, but maybe I'm misunderstanding it.

My approach treats this string as a declname.

The following should type check successfully if Something.init(x:) exists and has the same type signature as init(y: Int) (ignoring parameter labels) or otherwise fail to type check with an error.

extension Something {
   @dynamicReplacement(for: "init(x:)")
   init(y: Int) {
     self.init(x: y + 1)
   }
}

#34

I'm not sure how I feel about this proposal, but either way I do think it would be good to find some solid use cases to support it. I don't think somehow anticipating the need for security patches is a convincing example, but perhaps a somewhat related use case is supporting live reloading/hot reloading of code. This is something that people find useful in other languages, particular in some domains (e.g. developing web services or video games), and I believe something like this proposal would be one way to implement that.


(Alexey Kravchenko) #35

May be I’m wrong but now I see this proposal as «We should have the ability to change behaviour of entity via alternative controlled way without Delegate and Inheritance». it is promising.

Now we (as framework developers) can:

  • add Delegate pattern for client code
  • add property as closure
  • recommend to implement subclass with documented primitive methods and subclassing rules
  • write code via Generics

In some case it is not so straightforward approaches for client code. I think dynamic method replacement can change and improve our approaches a lot (as Protocol Oriented Programming already done).


(Karl) #36

Am I correct in thinking that open methods (belonging to open classes) are not implicitly dynamic? That is, every dynamic method must be explicitly annotated as such?


(Arnold Schwaighofer) #37

Yes it is intended that entities have to opt-in by adding the dynamic attribute.


(Arnold Schwaighofer) #38

To summarize up motivating use cases brought up on this thread so far:

Motivation

dynamic can be used in situations when it is desired to change behavior of an application/framework after its initial compilation.

It allows for replacement of a method during load/runtime.

Use cases are:

  • Customizing functions: Additional code to be executed before, after, or instead the initial implementation, e.g logging, mock testing, patching

  • Type safe way of importing new functionality at runtime: web service, video games

In contrast to other abstraction features which change code in a way that is determined apriori code was compiled.

Example:

The DatabaseAuthenticator module can be loaded at runtime to change the method of authentication while the web application is running.

// Module BasicWebApp

protocol Authenticator {
  func authenticate(user: User) -> Credentials 
}


struct DefaultAuthenticator : Authenticator {
    func authenticate(user: User) -> Credentials  { … }
}


struct WebApplication {
  static dynamic func getAuthenticator() -> Authenticator { 
    return DefaultAuthenticator() 
  }

  func run() {
     request.authenticate(getAuthenticator())
   }
}
// Module DatabaseAuthenticator
struct DatabaseAuthenticator : Authenticator {
    func authenticate(user: User) -> Credentials  { … }
}

extension WebApplication {
  @dynamicReplacement(for: “getAuthenticator()”)
  static dynamic func getDatabaseAuthenticator() -> Authenticator { 
    return DabaseAuthenticator() 
  }
}

(Joe Groff) #39

Yes. Even if a class is open, it could be apparent at compile time when a class or instance has a specific static type, and the compiler could devirtualize those calls if they weren't also marked dynamic:

// module A

open class AClass {
  open class func f() { ... }
  open func g() { ... }
  init()
}

// module B

AClass.f() // Obviously calling AClass.f(), can be devirtualized
let x = AClass() // Calling a designated initializer, so x must be of dynamic type AClass
x.g() // so this must call AClass.g()

Furthermore, in general, the compiler can choose to cache method lookups and hoist them out of loops, which are optimizations dynamic would also want to suppress in order to allow the method implementation to be freely replaced.


(Brent Royal-Gordon) #40

Looking over this thread, my general feeling is that this is a solution in search of a problem. I haven't yet seen a use mentioned for a @dynamicReplacement that only worked with explicitly dynamic methods which didn't seem somewhat contrived. If you anticipate that a particular method will be replaced under blanket conditions in the future, it would probably be better to build some kind of registration system where it would call a runtime-replaceable closure, rather than replacing the method at runtime. If you don't anticipate the replacement…well, you probably wouldn't mark it dynamic in the first place, unless your house style was to mark everything dynamic, in which case you would really want to mark whole types with it instead of individual methods.

I think the more useful use cases will come when we can create and modify types at runtime—something more like the Objective-C runtime than what we're considering here. At that point, it will make sense to tell the compiler that it should disregard its static knowledge of the type when calling a member you anticipate replacing occasionally (although to be honest, I'm a little iffy even on this).


(Joe Groff) #41

For the people asking "what is the use case", I'd recommend reading Jordan's reply. The "tightly coupled but still separately-usable libraries" use case he raises is probably the most important one.


(Brent Royal-Gordon) #42

Sure, but even Jordan says "It's probably not the long-term solution we want for registration". Wouldn't it make more sense for Base's implementation to look for Advanced and call a function in it if it's available? That would either put under user control or outright define away a lot of issues with @dynamicReplacement.