Now that the dust has settled I’m happy to point out another place Spark is being used in the field - on the front page of of the MarketWatch financial news site. I do work as an architect at the company which provides this site, but in this case I specifically stepped back from making the decision to use Spark. A while ago I did offer a proof of concept to the team and lead developers responsible for the decision, and went through a few deep-dive sessions with the tech leads.

In the end it was a safe decision to make. Because view engine compiles down to a class which simply blasts output to the text writer there aren’t really any moving parts which can cause problems. If your view works once it’ll work a million times concurrently.

That said there was one change made near the end of the project based on some valid comments about memory pressure from Brad Wilson, and some numbers which were verified by the JetBrains dotTrace profiler when your view content is very large and there aren’t really any wait states in the controller. The change knocked a few percentage off the cpu for the cost of stringwriter growth and amortized cost of memory pressure.

The change, in a nutshell, makes it so whenever you’re rendering to a view or to a named piece of content nothing’s being directed to a StringWriter or other full-copy spooler. Instead there’s a linked-list-of-string-array named SpoolWriter holding all of the intended output by reference - that’s safe and very efficient because string are immutable. A very minor efficiency is also gained by pooling the string-array-pages, so the new string[500] memory won’t contribute to the volume of bytes that need to be recollected per request.

Holding the strings in this buffer adds nothing to total memory allocated in flight, after all the string already existed as model data or literal strings in the view. Even if they’re dynamic they would have had to become a string instance at some point on their way to the response textwriter.

So! Even though Spark has a multi-pass rendering strategy, it’ll activebuffer[index++] = text what you’re writing during the view pass, and blast it right to the network HttpWriter when the layout is being rendered. Very fast and efficient, and the site in question does take a lot of traffic. A heck of a lot! Especially recently, for obvious reasons.