Ongoing work on differentiable Swift

[Brad_Larson]

I'd like to highlight some of the patches that have gone in over the last year, in a few different categories. Hopefully this will illustrate the work being put in to refine and extend differentiable Swift. I apologize upfront if my categorization of the patches is imperfect, this is just a quick grouping of them.

To start with, these are some of the patches that have corrected numerical inaccuracies in gradients:

• [AutoDiff] [SR-14218] Correctly propagate tangent vectors of inout parameters from functions with multiple basic blocks
• [AutoDiff] Ensure adjoint buffer of unchecked_take_enum_data_addr is zeroed after value is accumulated
• [AutoDiff] Fix adjoint propagation for active bb address arguments
• [AutoDiff] Properly collect inout parameter adjoints

Slava Pestov's incredible work on the Requirement Machine exposed some existing issues in the differentiable Swift implementation and pointed the way to solving them:

• [AutoDiff] Fix two derivative type calculation bugs caught by RequirementMachine
• AutoDiff: Explicitly filter out invalid requirements
• AutoDiff: Workaround for performing generic signature queries on the wrong signature

Differentiable Swift interacted with debug info in some unexpected ways, and much of that has been fixed over the last year:

• [AutoDiff] Use correct debug scope for pullback trampoline block.
• [AutoDiff] Use auto-generated locations for temporary allocations.
• Propagate location from destination alloca in salvageDebugInfo().
• [AutoDiff][DebugInfo] Properly transfer location info from debug_value into alloc_stack in pullback cloner
• [DebugInfo] Ignore noescape bit for all @convention(c) pointers

There have been overall improvements to design:

• [AutoDiff] Conform Array's and Optional's TangentVector to CustomReflectable
• [AutoDiff] Plumb witness derivative generic signatures through SILGen.
• [AutoDiff] Compute derivative types using requirements from archetypes.
• [AutoDiff] Implement cross-file lookup of derivatives
• [AutoDiff] Add missing constraint to Array.DifferentiableView declaration
• [AutoDiff] Allow tgmath differentiation of Double
• [ABI] [AutoDiff] Fix function differentiability kind metadata and mangling.

And finally, general stability improvements:

• [SR-13929][AutoDiff]: Enable [ossa] for all differentiation-generated thunks
• [AutoDiff] Fix several issues related to captured arguments
• [AutoDiff] [SR-14240] Align name mangling for Clang-imported declarations
• [AutoDiff] [SR-14625] Fix for subset parameters thunk over-consume
• Don't Unnecessarily Invalidate Unreferenced Generic Parameters
• [AutoDiff] Fix crasher when type-checking mismatched derivative.
• [AutoDiff] [AST] Handle null decl in 'printDifferentiableAttrArguments'.

There are plenty more pull requests than those I've mentioned here, but I wanted to at least start with an overview of the work that has been happening lately. Again, thanks to everyone who has been contributing to the ongoing refinement of differentiable Swift.