既存のテンプレートエンジンの問題点と再設計

2015-03-02

<style type="text/css">
.red {
color: red;
}
</style>
<h1 id="既存のテンプレートエンジンの問題点と再設計あるいはarrowsについて">既存のテンプレートエンジンの問題点と再設計、あるいはArrowsについて</h1>
<hr>
<p>Clack Meet Up #1<br>
2015-03-05 @サムライト</p>

<p>===</p>
<h1 id="about-me">About Me</h1>
<hr>
<p><img src="https://KeenS.github.io/images/icon.png" alt="κeenのアイコン"> </p>
<ul>
<li>κeen</li>
<li><a href="https://twitter.com/blackenedgold">@blackenedgold</a></li>
<li>Github: <a href="https://github.com/KeenS">KeenS</a></li>
<li>野生のLisper</li>
<li>Lisp, Ruby, OCaml, Shell Scriptあたりを書きます
===</li>
</ul>
<h1 id="template-engines">Template Engines</h1>

<p>===</p>
<h1 id="existing-architeture">Existing Architeture</h1>
<hr>
<ol start="0">
<li>(リクエスト)</li>
<li>アプリ</li>
<li>引数を計算</li>
<li>テンプレートに引数を渡す</li>
<li>レスポンスをレンダリング </li>
<li>レスポンスをサーバに渡す</li>
<li>(レスポンス)
===</li>
</ol>
<h1 id="rendering">Rendering?</h1>
<hr>
<ul>
<li>結合した文字列はクライアントに返ったらその後はゴミ</li>
</ul>
<ul>
<li>GCへ負荷がかかる (後述)</li>
</ul>
<ul>
<li>クライアントにとって1つの文字列である必要はない</li>
<li>むしろ返せる部分だけ先に返した方が得（後述）
===</li>
</ul>
<h1 id="gc-pressure-sbcl">GC Pressure (SBCL)</h1>
<hr>
<ul>
<li>世代別Copy GC</li>
<li>結合した文字列は比較的大きい</li>
<li><code>alloc_space</code>に入らない大きさならアロケートが遅い</li>
<li>GCを頻繁に起動してしまう</li>
<li>16KBを越えると特別扱いされて遅い/メモリを喰う</li>
<li>LispのWebアプリはレスポンスタイムの分散が大きい（要出展）</li>
</ul>
<h1 id="参考-sbcl-gencgc--x86-linuxhttpcl-wwwmsicojpreportssbcl-gc-memohtml">参考: <a href="http://cl-www.msi.co.jp/reports/sbcl-gc-memo.html">SBCL GENCGC @ x86 Linux</a></h1>
<h1 id="split-response">Split Response</h1>
<hr>
<p>例えば、こんなの</p>
<pre><code class="html"><!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en-us">
    <head>
        <link rel="stylesheet" href="http://localhost:1313//reveal.js/lib/css/xcode.css">
        <script src="//ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js"></script>
        ....
        </script>
    </head>
    <body class="li-body">
    <header>{{ header }}</header>
    {{ body }}
    ...
</code></pre>
<p>===</p>
<h1 id="split-response-1">Split Response</h1>
<hr>
<ol>
<li><code>{{ header }}</code>の前に先頭から<code><header></code>までを返す</li>
<li><code>{{ header }}</code>を返す</li>
<li><code></header></code>を返す</li>
<li><code>{{ body }}</code>を返す</li>
</ol>
<p>…</p>
<p>===</p>
<h1 id="pros-of-split-response">Pros of Split Response</h1>
<hr>
<ul>
<li><code>header</code>を計算してる間にクライアントにhead部分が渡る
<ul>
<li>先に<code><link></code>や<code><script></code>を要求出来る</li>
<li>サーバ側のスループットやレスポンスタイムは変わらないが
クライアントのレンダリング完了までの時間は大幅に短縮出来る</li>
</ul>
</li>
<li>文字列を結合する必要がない</li>
<li>定数部分については長さが判ってるので最適化し易い
===</li>
</ul>
<h1 id="how-tes-work">How TEs Work</h1>
<hr>
<h2 id="compilation">Compilation</h2>
<ol>
<li>テンプレート文字列</li>
</ol>
<ul>
<li>パーサ </li>
</ul>
<ol start="2">
<li>抽象構文木</li>
</ol>
<ul>
<li>コードエミット </li>
</ul>
<ol start="3">
<li>レンダリング関数
===</li>
</ol>
<h1 id="how-tes-work-1">How TEs Work</h1>
<hr>
<h2 id="rendering-1">Rendering</h2>
<ol>
<li>レンダリング関数</li>
</ol>
<ul>
<li>引数 </li>
</ul>
<ol start="2">
<li>文字列
===</li>
</ol>
<h1 id="misc-problems">Misc Problems</h1>
<hr>
<ul>
<li>サーバに渡すのは文字列なのにソケットに書き込む時はオクテット？
<ul>
<li>文字列で返す？オクテットで返す？（デバッグがー）</li>
<li>オクテットの変換はいつ？</li>
</ul>
</li>
<li>ストリームが遅い？
<ul>
<li>POSIX APIが使える"なら"fdの方が速い？</li>
</ul>
</li>
<li>ユーザが用意したバッファに書き出したい？</li>
<li>テンプレートに渡す引数が定数文字列なら畳み込める筈？</li>
<li>リクエストの度にテンプレートパースするのは筋悪だけど開発中は毎回コンパイルするのは面倒？
===</li>
</ul>

<h1 id="arrows">Arrows</h1>
<p>Template flies like an arrow</p>

<p>===</p>

<h1 id="arrows-1">Arrows</h1>
<hr>
<ul>
<li><a href="https://github.com/KeenS/arrows">KeenS/arrows</a></li>
<li>現在開発中のテンプレートエンジン</li>
<li>複数のテンプレートが選べる（予定）</li>
<li>複数のバックエンドが選べる</li>
<li>バックエンドに依ってはnon-consing
===</li>
</ul>

<h1 id="how-arrows-works">How Arrows Works</h1>
<hr>
<h2 id="compilation-1">Compilation</h2>
<ol>
<li>テンプレート文字列<span class="red"> + 定数引数</span></li>
</ol>
<ul>
<li>パーサ<span class="red">(default, cl-emd互換…)</span> </li>
</ul>
<ol start="2">
<li>抽象構文木</li>
</ol>
<ul>
<li><span class="red">最適化(const folding, concat, convert…)</span> </li>
<li>コードエミット </li>
</ul>
<ol start="3">
<li>レンダリング関数<span class="red">(string, octets, stream, fast-io …)</span>
===</li>
</ol>

<h1 id="how-arrows-works-1">How Arrows Works</h1>
<hr>
<h2 id="rendering-2">Rendering</h2>
<ol>
<li>レンダリング関数</li>
</ol>
<ul>
<li>引数 </li>
</ul>
<ol start="2">
<li>文字列<span class="red">、オクテット列、ストリーム書き出し、fast-io…</span>
===</li>
</ol>

<h1 id="how-compiled">How compiled</h1>
<hr>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-html" data-lang="html"><span class="line"><span class="cl"><span class="p"><</span><span class="nt">h1</span><span class="p">></span>Hi {{var name}}!<span class="p"></</span><span class="nt">h1</span><span class="p">></span>
</span></span></code></pre></div><p>を</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-lisp" data-lang="lisp"><span class="line"><span class="cl"><span class="p">(</span><span class="nv">compile-template-string</span> <span class="ss">:xxx</span>
</span></span><span class="line"><span class="cl">  <span class="s">"<h1>Hi {{var name}}!</h1>"</span> <span class="p">())</span>
</span></span></code></pre></div><h1 id="とコンパイル">とコンパイル</h1>
<h1 id="stream-backend">Stream backend</h1>
<hr>

<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-lisp" data-lang="lisp"><span class="line"><span class="cl"><span class="p">(</span><span class="nb">lambda</span> <span class="p">(</span><span class="nc">stream</span> <span class="k">&key</span> <span class="nv">name</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"<h1>Hi "</span> <span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="nv">encode-for-tt</span> <span class="p">(</span><span class="nf">princ-to-string</span> <span class="nv">name</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">    <span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"!</h1>"</span> <span class="nc">stream</span><span class="p">))</span>
</span></span></code></pre></div><ul>
<li>ほとんどアロケートしない
===</li>
</ul>

<h1 id="octet-backend">Octet backend</h1>
<hr>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-lisp" data-lang="lisp"><span class="line"><span class="cl"><span class="p">(</span><span class="nb">lambda</span> <span class="p">(</span><span class="k">&key</span> <span class="nv">name</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nv">with-fast-output</span> <span class="p">(</span><span class="nv">buffer</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="nv">fast-write-sequence</span>
</span></span><span class="line"><span class="cl">      <span class="o">#.</span><span class="p">(</span><span class="nv">string-to-octets</span> <span class="s">"<h1>Hi "</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">       <span class="nv">buffer</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="nv">fast-write-sequence</span>
</span></span><span class="line"><span class="cl">      <span class="p">(</span><span class="nv">string-to-octets</span>
</span></span><span class="line"><span class="cl">        <span class="p">(</span><span class="nv">encode-for-tt</span> <span class="p">(</span><span class="nf">princ-to-string</span> <span class="nv">name</span><span class="p">)))</span>
</span></span><span class="line"><span class="cl">                       <span class="nv">buffer</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="nv">fast-write-sequence</span>
</span></span><span class="line"><span class="cl">      <span class="o">#.</span><span class="p">(</span><span class="nv">string-to-octets</span> <span class="s">"!</h1>"</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">      <span class="nv">buffer</span><span class="p">)))</span>
</span></span></code></pre></div><p>===</p>

<h1 id="how-optimized">How optimized</h1>
<hr>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-html" data-lang="html"><span class="line"><span class="cl"><span class="p"><</span><span class="nt">h1</span><span class="p">></span>Hi {{var name}}!<span class="p"></</span><span class="nt">h1</span><span class="p">></span>
</span></span></code></pre></div><p>を</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-lisp" data-lang="lisp"><span class="line"><span class="cl"><span class="p">(</span><span class="nv">compile-template-string</span> <span class="ss">:stream</span>
</span></span><span class="line"><span class="cl">   <span class="s">"<h1>Hi {{var name}}!</h1>"</span>
</span></span><span class="line"><span class="cl">   <span class="o">'</span><span class="p">(</span><span class="ss">:known-args</span> <span class="p">(</span><span class="ss">:name</span> <span class="s">"<κeen>"</span><span class="p">)))</span>
</span></span></code></pre></div><h1 id="とコンパイル-1">とコンパイル</h1>

<h1 id="how-optimized-1">How optimized</h1>
<hr>
<h2 id="variable-folding">variable folding</h2>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-lisp" data-lang="lisp"><span class="line"><span class="cl"><span class="p">(</span><span class="nb">lambda</span> <span class="p">(</span><span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"<h1>Hi "</span> <span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="p">(</span><span class="nv">encode-for-tt</span> <span class="s">"<κeen>"</span><span class="p">)</span> <span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"!</h1>"</span> <span class="nc">stream</span><span class="p">))</span>
</span></span></code></pre></div><ul>
<li>既知の引数は畳み込む</li>
<li>文字列なら<code>princ-to-string</code>しない
===</li>
</ul>

<h1 id="how-optimized-2">How optimized</h1>
<hr>
<h2 id="const-folding">const folding</h2>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-lisp" data-lang="lisp"><span class="line"><span class="cl"><span class="p">(</span><span class="nb">lambda</span> <span class="p">(</span><span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"<h1>Hi "</span> <span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"&lt;κeen&gt;"</span> <span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"!</h1>"</span> <span class="nc">stream</span><span class="p">))</span>
</span></span></code></pre></div><ul>
<li>定数のエスケープはコンパイル時に済ませる
===</li>
</ul>

<h1 id="how-optimized-3">How optimized</h1>
<hr>
<h2 id="append-sequence">append sequence</h2>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-lisp" data-lang="lisp"><span class="line"><span class="cl"><span class="p">(</span><span class="nb">lambda</span> <span class="p">(</span><span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"<h1>Hi &lt;κeen&gt;!</h1>"</span>
</span></span><span class="line"><span class="cl">                <span class="nc">stream</span><span class="p">))</span>
</span></span></code></pre></div><ul>
<li>複数シーケンスの書き出しは1つにまとめる
===</li>
</ul>

<h1 id="further-ideas">Further Ideas</h1>
<hr>
<ul>
<li>引数計算の遅延</li>
<li>引数計算の並列化</li>
<li>非同期化</li>
<li>HTML compction
===</li>
</ul>

<h1 id="further-ideas-1">Further Ideas</h1>
<hr>
<h2 id="引数計算の遅延">引数計算の遅延</h2>
<ul>
<li><code>name</code>の計算が重いときに先に<code>"<h1>Hi "</code>を返す。</li>
<li><code>name</code>は必要になったら値を計算する(Promise パターン)</li>
</ul>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-lisp" data-lang="lisp"><span class="line"><span class="cl"><span class="p">(</span><span class="nb">lambda</span> <span class="p">(</span><span class="nc">stream</span> <span class="k">&key</span> <span class="nv">name</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"<h1>Hi "</span> <span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="nv">encode-for-tt</span> <span class="p">(</span><span class="nf">princ-to-string</span> <span class="nv">name</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">    <span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"!</h1>"</span> <span class="nc">stream</span><span class="p">))</span>
</span></span></code></pre></div><p>===</p>

<h1 id="further-ideas-2">Further Ideas</h1>
<hr>
<h2 id="引数計算の並列化">引数計算の並列化</h2>
<ul>
<li><code>name</code>の計算が重いときに先に<code>"<h1>Hi "</code>を返す。</li>
<li><code>name</code>は並列に計算して必要になったら値を要求する(Futureパターン)</li>
</ul>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-lisp" data-lang="lisp"><span class="line"><span class="cl"><span class="p">(</span><span class="nb">lambda</span> <span class="p">(</span><span class="nc">stream</span> <span class="k">&key</span> <span class="nv">name</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"<h1>Hi "</span> <span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span>
</span></span><span class="line"><span class="cl">    <span class="p">(</span><span class="nv">encode-for-tt</span> <span class="p">(</span><span class="nf">princ-to-string</span> <span class="nv">name</span><span class="p">))</span>
</span></span><span class="line"><span class="cl">    <span class="nc">stream</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">(</span><span class="nf">write-string</span> <span class="s">"!</h1>"</span> <span class="nc">stream</span><span class="p">))</span>
</span></span></code></pre></div><p>===</p>

<h1 id="further-ideas-3">Further Ideas</h1>
<hr>
<h2 id="非同期化">非同期化</h2>
<ul>
<li>単純にwriteを非同期にする</li>
<li>他にFutureもブロックするので非同期Futureを使う
===</li>
</ul>

<h1 id="further-ideas-4">Further Ideas</h1>
<hr>
<h2 id="html-compction">HTML compction</h2>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-html" data-lang="html"><span class="line"><span class="cl"><span class="p"><</span><span class="nt">ol</span><span class="p">></span>
</span></span><span class="line"><span class="cl">  <span class="p"><</span><span class="nt">li</span><span class="p">></span> item 1 <span class="p"></</span><span class="nt">li</span><span class="p">></span>
</span></span><span class="line"><span class="cl">  <span class="p"><</span><span class="nt">li</span><span class="p">></span> item 2 <span class="p"></</span><span class="nt">li</span><span class="p">></span>
</span></span><span class="line"><span class="cl">  <span class="p"><</span><span class="nt">li</span><span class="p">></span> item 3 <span class="p"></</span><span class="nt">li</span><span class="p">></span>
</span></span><span class="line"><span class="cl"><span class="p"></</span><span class="nt">ol</span><span class="p">></span>
</span></span></code></pre></div><p>を</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-html" data-lang="html"><span class="line"><span class="cl"><span class="p"><</span><span class="nt">ol</span><span class="p">><</span><span class="nt">li</span><span class="p">></span>item 1<span class="p"></</span><span class="nt">li</span><span class="p">><</span><span class="nt">li</span><span class="p">></span>item 2<span class="p"></</span><span class="nt">li</span><span class="p">><</span><span class="nt">li</span><span class="p">></span>item 3<span class="p"></</span><span class="nt">li</span><span class="p">></</span><span class="nt">ol</span><span class="p">></span>
</span></span></code></pre></div><ul>
<li>DOM構造が変わってしまう</li>
</ul>
<p>===</p>

<h1 id="todos">TODOs</h1>
<hr>
<ul>
<li>設計上複数シンタックスをサポート可能だがまだしてない</li>
<li>既存のテンプレートエンジンとの比較ベンチマーク</li>
<li>高速化</li>
<li>多機能化
<ul>
<li>テンプレート</li>
<li>最適化</li>
<li>バックエンド</li>
</ul>
</li>
<li>clackとの連携
<ul>
<li>clackのAPIはメモリアロケーションが多めに必要になる
===</li>
</ul>
</li>
</ul>
<h1 id="summary">Summary</h1>
<hr>
<ul>
<li>既存のテンプレートエンジンは非効率
<ul>
<li>メモリを無駄遣いしていた</li>
<li>ユーザーのことを考えてなかった</li>
</ul>
</li>
<li>新しいテンプレートエンジンを設計した
<ul>
<li>メモリアロケーションをあまりしない</li>
<li>ユーザー側の速度まで考慮した</li>
<li>柔軟