SE-0354: Regex Literals

[Michael_Ilseman]

@rvsrvs

In another sense, I am strongly drawn to the extensibility of the #regex( ... ) syntax as it makes any similar future extensions to the language purely additive.

I addressed this thoroughly in the pitch thread. Copying some of that content here for easy viewing:

@Michael_Ilseman

That being said, regex literals are unique in their prevalence as fragments passed directly to API or as components of a result builder. I don't think that, e.g., SQL fragments would be used in this fashion.

Nothing in this proposal precludes a scalable approach to foreign language literals using #lang(...) or lang'...' . If that happened, regex could clearly participate as well. Even in an alternate reality where we had a formal concept of foreign language literals with a convention, there's still value to a dedicated regex literal. The alternate reality would just shift priorities around (which is inherent to alternate reality scenarios).

Note there is no work being done, AFAICT, on SQL literals or these other theoretical use cases. I'm personally interested, but they're not plan of record.

@Michael_Ilseman

Note that nothing proposed precludes a general solution, so we're not missing anything. The work involved in making regex work is necessary to... making regex work, so it's not wasted even if we had a general language escape hatch to leverage.

This is formally outside the scope of the pitch review, but we're careful to integrate the regex parser with the compiler in a library-driven fashion that makes it easier to migrate more compiler code into modular libraries. This also helps carve a path for more general purpose foreign language support in the future. My totally-speculative and not even close to a formalized plan dream would be to open up foreign language snippet support to 3rd party libraries, similarly to how we open up literals, custom string interpolation, property wrappers, and result builders to libraries. That requires the compiler integration mechanisms we're developing with regex, and regex can piggy back on whatever escape hatch emerges in the future. They wouldn't declare their own single-character delimiters, they'd use the more general escape hatch.

Also, it seems clear that foreign source fragment might have their own needs above and beyond regex's, so we certainly wouldn't want to limit them from regex-based assumptions. I think it's better to design the general facility in light of general usage, which is multiple releases out as it involves further evolving the compiler-library interaction story. The regex work actually advances it behind the scenes, but it needs more examples than just regex to help complete it.

@Michael_Ilseman

Better library extension is how we achieve this. We have library-extensible literals, string interpolations, property wrappers, and result builders. I hope library-extensible parsing of clearly and unambiguously delineated foreign language snippets happens one day.

I'm happy to engage further, though it's getting pretty far afield of this pitch.