Pitch: New Leaf Body Syntax

pitch

(Tanner) #1

Pitch: New Leaf Body Syntax

Leaf is a templating language built for Swift. This pitch proposes introducing a new syntax for declaring tag bodies that would simplify the parser and make using Leaf easier.

Current Syntax

Leaf has supported the following syntax for declaring tag bodies since version 1.0.

<h1>Users</html
<ul>
#for(user in users) {
    <li>#(user.name)</li>
}
</ul>

This syntax, which we'll refer to as curly-brace bodies, was inspired by Swift's closure syntax. Much of Leaf's original design was copied directly from Swift in an effort to make the new templating language easy to learn.

While this syntax is easy to learn and great for example code, it suffers from some issues in real world use cases--especially when Leaf is used alongside JavaScript.

Problems with Curly-Brace Bodies

The main problem with curly-brace bodies is the prevelance of } in normal template code. Because } denotes the end of a body in this syntax, any appearances of } in the template's text must be escaped.

let xs = [-5, 3, 42, -3.14]
#if(count(xs) != 0) {
	|x| = { x, if x >= 0; -x, otherwise; }

	Solve equations:
	#for(x in xs) {
		|#(x)| = ?
	}
}

In the above example, Leaf would (perhaps unexpectedly) assume the } in the definition of |x| signals the end of the #if tag's body.

The solution to this problem is to escape any } that do not actually intend to close a tag body using \.

#if(count(xs) != 0) {
	|x| = { x, if x >= 0; -x, otherwise; \}

	Solve equations:
	#for(x in xs) {
		|#(x)| = ?
	}
}

While escaping is fairly trivial for this example, one can imagine that it might get out of hand when dealing with large amounts of embedded JavaScript.

Proposed Solution: Colon Syntax

This pitch proposes introducing the following syntax to make parsing tag bodies more predictable and resilient:

  • Tags intending to include a body are immediately trailed by a :
  • #:<tagname> token is used to close a body.

Take a look at the following example:

#if(count(xs) != 0):
	|x| = { x, if x >= 0; -x, otherwise; }

	Solve equations:
	#for(x in xs):
		|#(x)| = ?
	#:for
#:if

This new syntax has a number of key benefits over the existing syntax.

  • Tags that do not declare a body are easier to parse (no need to check for unescaped {)
  • Body closing syntax is less likely to appear in template text (#:for is less common than })
  • Body closing tag matches opening tag (#:for ends #for)
    • This makes it easier for humans to read templates
    • It also allows the Leaf compiler to better catch copy/paste errors

Edge Cases

Escaping Trailing Colon

It's likely that a colon could appear directly after a tag in cases when the developer is not declaring a body.

#for(user in users):
    - #(user.name): #(user.email)
#:for

In these cases, the trailing colon can be escaped or added via string concatenation.

#(user.name)\: #(user.email)
#(user.name + ":") #(user.email)

Escaping Closing Tag

Although unlikely, it's possible that a closing body tag such as #:if could appear in the template text. In this case, the closing tag could be escaped like any normal tag using \ before #.

#if(showLink):
    The link is: http://foo.co/index.html\#:ifasdf
#:if

Single Line Bodies

One of the nice advantages of the curly-brace body syntax is that declaring single line bodies is concise. Take the following example of setting a page title.

#set("title") { Home }

This is considerably less concise with the proposed colon syntax.

#set("title"): Home #:set

One solution that is already available for some Leaf tags is passing a string as the second parameter. This could be expanded to all tags that accept bodies as a general pattern in Leaf.

#set("title", "Home")

This has the added advantage of not introducing extraneous white space around the content.

Deprecated Support for Curly-Brace Syntax

It should be possible to continue supporting the existing curly-brace syntax for declaring tag bodies alongside the new colon syntax. This usage would be considered deprecated and would emit a warning in debug build modes. Support for the syntax would likely be removed in the next version of Leaf.

It's not possible to 100% guarantee support for both syntaxes will be reasonable to implement in the parser, but if it is, it would likely be greatly appreciated by developers with large, existing codebases using Leaf.

Alternatives Considered

End-Syntax

If Leaf had a way to statically determine whether or not a given tag usage required a body, it could be possible to omit requirement of the trailing :. This could allow for the following syntax:

#for(user in users)
   Hello #(user.name)!
#endfor

This syntax does not suffer from the Escaping Trailing Colon issue. However, statically determining whether a tag requires a body could prove difficult to implement and likely restrict custom Leaf tags.

One difficult case is the #set tag.

#set("title", "Home")
#set("content")
    <sidebar>...</sidebar>
#endset

This use-case needs support for dynamically requiring a tag body based on the number of arguments supplied.

Curly-Brace Counting

vapor/leaf#119 proposed counting opening curly braces to more intelligently detect closing braces. While this works well for detecting curly-braces in typical JavaScript usage, it fails to account for some seemingly feasible cases, such as using this smiley face.

#for(user in users) {
	Hello #(user.name)! :}
}

#2

Would a stack-based algorithm work to solve this problem, or would that suffer the same problems as the counting solution?


(Caleb Kleveter) #3

What do you mean by a 'stack-based algorithm'?


(Tanner) #4

I think that would have the same issue as the counting solution. i.e., templates like the following would continue to cause problems:

#if(foo) {
   Hello :}
}
#if(foo) {
   This is an example of broken javascript:
   var x = { }}
}

#5

@calebkleveter it's a way to parse things like HTML

http://hz2.org/blog/parsing_html.html

Every time you see an opening tag, you push that tag onto a stack and start tracking the contents of a body. Every time you see a closing tag, you pop the uppermost tag off the stack and can accept that the last body you were tracking is now complete assuming the closing tag matches the uppermost (opening) tag on the stack.

There are some quirks in HTML that this doesn't work perfectly for, but it does work for simple snippets.

@tanner0101 I see what you're going for. How using common characters as tags causes problems. Thanks!


(Steven Van Impe) #6

Have you given any thought to just deprecating Leaf and going with Stencil instead?

I've never found or heard any arguments as to why Vapor needed a new template language in the first place:

  • Are there fundamental differences between Leaf and Stencil?
  • Are they trying to solve a different problem?
  • Are they trying to solve a problem in a fundamentally different way?

As far as I can tell, the answer to these all seems to be "no".

Our community is way too small to be wasting development efforts. It's great that we have multiple frameworks exploring different avenues, but templating languages aren't exactly a new problem that needs exploration. Swift already has a templating language (Stencil) that works fine, has a familiar syntax, isn't tied to a web framework but is used by other projects, such as Sourcery.

As a user, I'd rather see development time spent on better tooling support, improved code quality and documentation, ... than multiple languages that differ only in syntax.


(Tim) #7

Leafs handling of futures is pretty unique (and a very nice feature!). So I for one would be against loosing that.

Unless that’s all handled by TemplateKit of course and Stencil can sit on top of that.


(Tanner) #8

If this pitch were proposing a new templating language, I would definitely agree with your reasoning. But Leaf is over 2 years old now and a lot of people that use it and like it. I think it would be unfair to those people to deprecate Leaf.

I also disagree that Leaf does not have any differences to Stencil. Leaf is more similar to Stencil than something like Mustache, sure, but they are not identical. Leaf's syntax is arguably more concise. Leaf's extensible tag mechanism allows for more than one argument, default arguments, etc. Leaf is also built directly on and for NIO and Codable, which makes it a performant choice given Vapor 3's architecture.

That said, I think Vapor should have good implementations of existing standards. One of my goals for Vapor 4 is to introduce a pure NIO / Codable based implementation of Mustache. Given Mustache's dead-simple syntax this should prove to be an even more performant alternative to Leaf and could possibly end up being Vapor's recommended package for API development (simple views, email views, etc).

In regards to Stencil, I think given its popularity in the Swift community, it won't be long until there is a NIO friendly implementation with good performance characteristics. When that becomes available (hopefully through the SSWG process) I would be more than happy to support an official Vapor integration.


(Steven Van Impe) #9

Perhaps proposing a standard through the SSWG (whether that's an existing language or a new one) is the right way forward for the long term? Multiple languages don't just lead to duplicated development efforts, there's also the issue of keeping documentation and education materials up to date. For us in education, choice is actually a bad thing. It makes it that much harder for students to find the right information.

Anyway, apologies for going off-topic. This is just one of those issues that I probably care more about than others.

I'll keep an eye on the SSWG. If a standard were to emerge, I'll adopt it right away. Sure, migration always hurts, but it will hurt even more when our community (and codebases) grows :slight_smile:


(Robert Muckle-Jones) #10

Have you taken a thorough look through SE-0200: “Raw” mode string literals? I think that might give you some ideas for solving this problem - ways that would allow you to keep the existing brace block delimiters.