PLDI 2016 Artifact Evaluation


Programmatic and Direct Manipulation, Together at Last
Paper and Supplementary Appendix, November 2015

To help set the context for evaluating our artifacts, we quote some descriptions from our submitted paper:

  • Our implementation, examples, user study videos, and survey instrument (as well as additional tutorial documentation) are publicly available and ready for Artifact Evaluation. (p.2)
  • Our goal with live synchronization [...] is to immediately apply a program update during the user's actions. (p.4)
  • The [examples] resemble typical programs in other functional languages, but for the domain of SVG. (p.7)
  • Sketch-n-Sketch is not a full-featured direct manipulation editor, nor is it a polished language implementation with rich libraries and tools. (p.9)

Quick Start

You can try Sketch-n-Sketch right away by selecting an example using the dropdown menu and then directly manipulating the visual output. To edit the program directly, click Edit Code, type some changes, and then click Run Code to re-evaluate the program. (Removing the need to click Edit Code is a feature that has long been on our TODO list.)

Examples from Paper

The examples depicted in Figure 3 are named Wave Boxes, Logo, Botanic Garden Logo, Active Trans Logo, Lillicon P, Chicago Flag, and Hilbert Curve Animation. The examples depicted in Figure 3 are named Widgets, Rounded Rect, and Tile Pattern. The three tasks depicted in the user study videos are named Ferris Task (Before/After), Keyboard Task (Before/After), and Tessellation Task (Before/After). The Survey Results example implements the histograms in Figure 4. The examples depicted in the Appendix are named 3 Boxes, Elm Logo, and Ferris Wheel.

Documentation and Videos

To do more than directly manipulate examples that have already been written, you will need to learn a bit about the syntax of Little and the features and options in the Sketch-n-Sketch editor. We provide the following resources, along with rough estimates of how long you may decide to spend with them.

user study videos (0 – 45 minutes)

Parts 1, 2, 3, and 4 (approximately 20 minutes in total) provide a high-level overview of Little and Sketch-n-Sketch.

Part 5 shows the tasks we developed for our user study; the survey questions can be found in the Appendix to our paper.

syntax guide (0 – 1 hour)

The syntax guide defines the grammar for Little as well as the encoding of SVG values. To see a variety of Little programs, you may view the source code for prelude.little (automatically included in every program) and the examples.

tutorial (1 – 5 hours)

This multi-part tutorial constitutes the most thorough introduction to Sketch-n-Sketch as well as some more advanced uses.

Additional Features

Our implementation provides two kinds of zones beyond the basic SVG zones described in our paper, namely, color zones and rotation zones. A simple example that uses both:

(def [x y w h] [76 118 127 131])
(def [cx cy] [(+ x (/ w 2!)) (+ y (/ h 2!))])
(def color 216)
(def rotation 45)
(def myShape (rect color x y w h))
(svg [(rotate myShape rotation cx cy)])

When toggling the Zones button to display Extra zones, a slider will be displayed to manipulate the color and a circle will be displayed to manipulate the rotation angle.

Known Issues

Error Messages

Our tool currently does not report helpful error messages. When there is a parse error, the caption "PARSE ERROR!" is displayed under the output canvas without any additional information.

When there is a run-time error, usually the output canvas usually displays the string "[Little Error]" (sometimes with the line and column numbers of the offending expression) but sometimes a JavaScript exception is raised (visible in the brower's Web Console). In the latter case, reload the application.

Performance

There are several aspects of our implementation that would benefit from optimizations: the evaluation of Little programs, re-evaluation of Little programs during live mode, and the data structures computed and maintained by the editor to make heuristic choices and compute triggers. Our current approach for each of these is naive, leading to some known performance issues:

  • Sketch-n-Sketch may fail to load in Firefox on Linux, crashing with a "too much recursion" error. (However, this issue appears to be mitigated in Sketch-n-Sketch v0.4.1, which is the first version we have built with Elm 0.16, the most recent version of the Elm compiler). We believe this is due to the default stack size in this Firefox configuation. We have run Sketch-n-Sketch in Firefox and Chrome on a number of different operating systems without this error. If you have trouble, please try running Sketch-n-Sketch in several different browser and operating system configurations, and let us know if you run into issues.
  • Preparing for live updates — (a) using heuristics to compute zone assignments at the end of one user action in anticipation of the next, and (b) computing a mouse trigger when the user makes an initial click on a zone — sometimes leads to a temporary lack of responsiveness, due to current deficiencies in the data structures we maintain and how we traverse them. We typically notice this slowdown only when there are a large number of shapes. For example, the Keyboard Task Before example exhibits noticeable delays user actions have finished when running in Fair Heuristics mode.

Parser

When a "(def x e1) e2" expression appears anywhere but the top-level scope of the program, the unparser currently inserts extra, unwanted whitespace. The workaround is to use the equivalent let-expression form "(let x e1 e2)", which can appear anywhere without sufferring this issue.

Resizing Panes

The pane separating the code and canvas can be resized. If you stop dragging (by releasing the mouse) before the position of the pane has caught up to the mouse position, the editor may get stuck in a state where it resizes the panes upon each successive click. In such situations, reload the application.

Source Code

Our implementation is open source, but we have not yet made an effort to organize and document it in a way that would make it easy for others to extend. If one wants to poke around anyway, here are a few pointers:

  • Lang.elm defines the types of expressions and values in Little, without any SVG-specific encodings.
  • SVG encodings are defined in LangSvg.elm.
  • The solve function in Sync.elm is the entry point for our two simple solvers, solveTopDown (called SolveB in the paper) and simpleSolve (called SolveA).
  • The assignTriggers function in Sync.elm is the entry point for assigning triggers using heuristics for disambiguation.