Programming + Direct Manipulation + HTML/SVG
Direct manipulation interfaces are useful in many domains, but the lack of programmability in a high-level language makes it difficult to develop complex and reusable content. We envision direct manipulation programming systems that allow users the benefits of general-purpose programming languages but with less keyboard-based text editing and more mouse-based direct manipulation. Sketch-n-Sketch is currently a direct manipulation programming system for creating HTML and SVG documents. We imagine similar possibilities for many other programming and design settings as well. So far in our work, three themes have emerged:
Output-Directed Programming The ability to design and shape the desired output by example, adding new values and adding new intended relationships among them. Then Sketch-n-Sketch infers "large" program transformations to match, for example, by introducing variables, functions, and arithmetic relationships into the program.
Bidirectional Programming Once a high-level program is written — through a combination of output-directed transformations and conventional text-edits — users can directly change the program output. Then Sketch-n-Sketch infers "small" program changes to match, by running the program "in reverse."
Structured Text Editing In additional to unrestricted text-edits, the Sketch-n-Sketch code editor supports structural selection and manipulation of the underlying abstract syntax tree. Based on structural actions, Sketch-n-Sketch proposes context-sensitive refactorings and other structured program transformations.
