Upstreaming '@differentiable' attribute and 'Differentiable' protocol

As we get closer to being ready to upstream the differentiable programming feature to the master branch, I have a question about how to gate the feature.

There are 4 major user-visible components:

  1. @differentiable declaration attribute
  2. @differentiable function type attribute
  3. Differentiable protocol
  4. Differential operators (top-level functions in stdlib)

Option 1: Underscore all of these things.

  • Pros: Easy to do. Consistent with other experimental features.
  • Cons: We'll need to drop the underscore every time we build a Swift for TensorFlow toolchain.

Option 2: Add a frontend flag --enable-experimental-differentiable-programming. When disabled, all above features are unavailable.

  • Pros: More principled.
  • Cons: A slight bit more engineering. Idk if there's a precedent.

@tkremenek and the core team, what would you recommend?