@inlinable only allows the compiler to inline the function; it does not guarantee it. However, doing so also provides the visibility through the module boundary that the compiler needs in order to do specialization of generics and similar optimizations.
When something turns out to be unexpectedly slow, I usually start by perusing the stack looking for generic functions not marked “specialized” (the way Array.subscript.modify is in your screenshot), and make them @inlinable. I am assuming that as a package, your module is only vended as source, in which case the implications for ABI stability are irrelevant.
There is also -cross-module-optimization, which basically erases the module boundary altogether. If you control all the clients, it might be easier to just change the build invocations. But it is usually not feasible to train external clients to do that. There was talk of automating this flag within in SwiftPM, but I don’t think it has been done yet. See this older thread for more details.
swift build -Xswiftc -cross-module-optimization