MAKING INTERACTIVE PROTOTYPES

There are still a few seats left in this weekend's Making Interactive Prototypes workshop being hosted by Mode Lab and LIFT architects in New York City.  Interactive Prototyping is a process which strives to empower the designer – giving them the ability to build and prototype ideas in a more fluid and cost effective manner.  To find out more, visit: http://lab.modecollective.nu/lab/making-interactive-prototypes-w-an...

About this Workshop:

It is without a doubt that the design process encompasses many things. From ideation, 3D modeling, programming, and manufacturing to material testing, marketing, life cycle assessment, and cost analysis; the designer is often confronted with many challenges. Prototypes give the designer the ability to test or simulate how a given set of parameters will affect a particular design. Prototyping is inherently iterative in nature and we’re constantly searching for faster more powerful ways to build better prototypes. Interactive Prototyping is a process which strives to empower the designer – giving them the ability to build and prototype ideas in a more fluid and cost effective manner.

This workshop will focus on hardware and software prototyping techniques; primarily focusing on a wide range of sensing and actuation modalities and the creation of custom built tools in order to create novel interactive prototypes. Using remote sensors, microcontrollers (Arduino), and actuators, we will build virtual and physical prototypes that can communicate with humans and the world around them. There will be particular emphasis on the concept of prototyping – both digitally and physically – as a means to explore intelligent control strategies, material affects, and the parameters which effect dynamic systems. This fast-paced two day workshop will introduce a number of different topics relevant to the domain of prototyping including: introduction to physical computing and electronics, Arduino essentials, circuit design with Fritzing, interactive prototyping with Firefly, computer vision, soldering, and much more. 

OUT OF HAND / HANDS ON

Out of Hand: Materializing the Postdigital  at the Museum of Arts and Design (MAD) is the first in-depth museum survey dedicated to exploring the impact of computer-assisted methods of production on contemporary art, architecture, and design. Shapeways is partnering with MAD for the exhibition and will host 'OUT OF HAND/HANDS ON,' an educational and interactive space on the Museum's second floor where visitors will be able to experience 3D design and printing firsthand.

Opening in conjunction with the exhibition curated by Ron Labaco on October 16th and running through April 2014, visitors will have the opportunity to explore the 3D printing process and the digital modeling process.  Several interactive 3D applications will be on display including work by LIFT Architects, Nervous System, Rhino 3D, and Uformia.  

Visitors will also have the opportunity to be 3D scanned with help from Occiptal Labs and 3D printed into mini figurines by an in-house high resolution Formlabs 3D printer or in any of the 30+ materials currently available on Shapeways. 

NEW PUBLICATIONS

If you haven't already done so, make sure to check out this months issue (April 2013) of Architectural Design (AD) titled Computation Works: The Building of Algorithmic Thought.  Edited by Xavier De Kestelier and Brady Peters, this issue focuses on emerging themes in computational design practices, showcasing built and soon-to-be-built projects and provides a state of the art look at current computational design techniques.

In addition to some amazing articles written by Daniel Davis, David Rutten, Daniel Piker, Giulio Piacentino, Arthur van der Harten, Thomas Grabner and Ursula Frick, and many more... it also features an article that I co-authored with Jason K. Johnson titled Firefly: Interactive Prototypes for Architectural Design.  

In addition, make sure you also take a look at the book Prototype!  edited by Julian Adenauer and Jorg Petruschat which was published by Form+Zweck last summer (2012).  Written by leading individuals at world renown design labs and research centers, this book offers a unique compilation of articles centered around the topic of advanced forms of prototyping.  In my article, IDE vs. IPE: Toward and Interactive Prototyping Environment I discuss the need to shift toward a more visually oriented Interactive Prototyping Environment (IPE) which addresses the limitations found in the existing IDE paradigm and opens up creative new opportunties for artists and designers.

FACADES + PERFORMANCE WORKSHOP

I am excited to be teaching a one-day Interactive Surfaces workshop for the upcoming Facades+ Conference being held in New York City on April 11th-12th.  The event has an amazing line up of speakers and workshops which are being taught by some of the industries leaders including: Robert Aish (Autodesk), Nathan Miller (Case), Gil Akos & Ronnie Parsons (Studio Mode), Neil Meredith (Gehry Tech), and  John Sargent (SOM).

The Interactive Surfaces workshop will concentrate on producing facade prototypes that are configurable, sensate, and active.  The facade of a building is the liminal surface across which information and environmental performance is frequently negotiated.  Given dynamic context of our built environment; the facade must be capable of intelligent adaptation over time.

In this workshop, we'll be focusing on new hardware and software prototyping techniques; primarily focusing on a wide range of sensing and actuation modalities in order to build novel interactive devices. Using remote sensors, microcontrollers (Arduino), and actuators, we will build virtual and physical prototypes that can communicate with humans and the world around them.  Using both Grasshopper and the Firefly plug-in, you will learn how to create intelligent control strategies for interactive or responsive facades.

Click here to sign up!

The participants who sign up for this workshop will also be the first to get their hands on the new Firefly Interactive Prototyping Shield which I have been developing. This shield provides access to a number of built-in, ready-to-use sensors and actuators including: 3 linear sliders (potentiometers), a light sensor, a two-axis joystick, 3 push buttons, a red LED, a yellow LED, a Green LED, and a Tri-color LED, 2 servo connections, and a high-voltage MOSFET circuit capable of driving lights, valves, DC motors, etc.  Each participant will not only walk away with a kick ass new hardware kit, but valuable knowledge in how to create new types of interactive prototypes!

5-AXIS ROBOTIC MOTION CONTROLLER

I'm excited to release more information about a paper I wrote titled A Five Axis Robotic Motion Controller for Designers which was selected for publication and presentation for this year's ACADIA 2011 Conference: Integration Through Computation held in Banff, Canada from October 13th-16th. Click here to download the full paper.

This project aims to bring physical input and output closer together through the design purpose-built tools for fabrication, which hopefully leads to many new creative opportunities for designers.  Working from observations about the way architects design, this project explores the development of a novel 3D drawing tool or customized 5-axis digitizing arm that takes real-time input and translates movement patterns directly into machine code for robotic fabrication.  An improved workflow for robotic simulation was also developed as part of this project; using design tools that are already familiar to architects and designers, such as Rhino and Grasshopper. The purpose of this project was not to suggest that this new workflow is a ready-made solution to replace the existing robotic fabrication process; rather I hope that this work is seen as a proof of concept that could enable wider use of digital fabrication tools by architects and designers.

The existing design-to-fabrication workflow for industrial robots (seen above) has traditionally been a slow and cumber-some process, especially for designers. Machine tooling, kinematic simulations, and robotic movement programming often require intimate knowledge of scripting and manufacturing processes, all of which limit the utilization of such tools by the 'typical' architect/designer.  

In the traditional robotic fabrication workflow, there is often a discrepancy between the original design intent and the final output, primarily because there is an intermediate step where the designer has to hand off a digital model to a fabrication consultant who has more intimate knowledge of the specific robotic CAM software and the fabrication process in general. Typically, this consultant will use programs such as Robot Studio or Master CAM to create the necessary tool paths for the design, however this process can often take a great deal of time.  And, if during this process, modeling irregularities are found or fabrication problems arise due to reachability or collision detection issues, then the entire process must start anew.

 Conceptually, this project started very simply.  I began by looking at the joint and axis configurations of the ABB-IRB 140 robot, one of the six axis robots available in the Harvard robotics lab.  The design challenge then, was to design a tangible controller around these constraints.  By using the same joint and axis configurations, the digitizing arm has a one to one relationship with the larger industrial robot.  It is very intuitive.  A user immediately grasps the idea that when they move the digitizing arm in a certain way, the robot will respond in kind.

Outside of the development of a new robotic workflow, one of the primary goals of the project was to minimize costs.  Given that all of the parts for this project were paid for out of pocket (a student's pocket), creating a low-cost solution was of utmost importance.  But, beyond my own personal economic restrictions, I wanted this project to be seen as a do-it-yourself solution - something that could be built in any garage or workbench using easily purchased hardware parts and sensors and a few custom fabricated pieces.  The entire controller, shown here, was built for less than $200 dollars. The aluminum body was water jet cut and all of the hardware were pieces that could purchased from local hardware stores or online retailers. All of the sensors, including the five high-precision potentiometers (shown here as the small blue knobs sticking off of the aluminum body) and the two digital sensors on the tool tip were also purchased from online retailers and were chosen because of their affordability.

To create a real-time robotic simulation, data from each of the embedded sensors on the tangible controller are streamed into the computer using a plug-in for Grasshopper that I have also been developing called Firefly.  Among other things, Firefly enables the use of real-world data, acquired from various types of sensors or other input devices to explicitly define parametric relationships within a Grasshopper model.  In this project, sensor information is used to create a forward kinematic robotic simulation. Forward kinematics describes one type of solution for determining robotic positioning.  If given all of the relative angles of each joint and the lengths of each leg; the tool tip (also known as the end effector) can be found by performing a series of matrix transformations on each body in the robotic mechanism.  In this case, each of the potentiometers will return a 10-bit number between 0 and 1023.  These particular potentiometers were able to rotate up to 340º, so the angle between each joint can be found by simply multiplying the current sensor value by the sensor step size.  These angle values are used to perform a matrix transformation on each of the robotic legs, ultimately giving you the precise position of  the tool center point.  And, once you know the location of the end effector, you can record this data over time to create real-time robotic tool paths.

In addition to the five high-precision potentiometers, the digitizing arm is equipped with a tool tip circuit board with two push button controls.  These allow the user to easily record or reset the digitized information on the fly.  I also designed and built a customized circuit board (on the left) which processes all of the sensor information and sends a formatted string of information over the serial port to the virtual parametric interface.

 The Grasshopper definition is relatively straight forward.  The Firefly Read component parses of the sensor information being sent directly from the microcontroller circuit board.  There is a custom component written in VB.NET (seen in item number 2),  which creates the necessary tool data.  The data from both of these components are fed into another custom component which calculates the forward kinematic solution and outputs the position of each leg, creating a real-time preview of the robot moving in the Rhino viewport.  In addition, the robotic simulator also returns all of the RAPID code, or the robotic programming language used by all of the ABB robots, to move the actual robot in the same manner as the forward kinematic preview.

The custom robotic simulation component written inside of Grasshopper outputs all of the necessary RAPID code to control the actual robot.  There are two methods by which this can be done.  First, all of the digitizing information is recorded and formatted into a composite data type called a robtarget.  Each robtarget is defined by its name, absolute position as XYZ coordinates, rotation and orientation of the robot as four quaternion values, and its joint configurations.  Each robtarget is given a unique identification each time the solution is recomputed.  Then the movement commands are created to tell the robot specifically how to get to each robtarget.  Once the program has been written, it can then be saved to a file on disk and uploaded to the robotic controller to be played back.  An alternative method is to stream the angle information from the digitizing arm directly to the robot through a network cable.  In this method, a program is uploaded to the robot which tells it to sit and wait for any information being sent directly from the Grasshopper definition (which can be seen in the video above).

As of today, there have only been a limited number of test runs using the five-axis robotic controller, however, the initial tests suggest that the proposed direct-to-fabrication process could prove to be a viable alternative to existing robotic workflows.  One of the first tests I tried was attaching a custom designed pen tool to the robot to see if the drawing movements of the digitizing arm would match those of the robot.  And while spelling your name isn't the most exciting demo, it did show some of the potential available with this process.  Because virtually any end effector can be attached to the end of the robot, the design opportunities are endless.  And because the tangible controller has a one-to-one relationship with the larger industrial robot, designers immediately understand that their drawing motions will be converted directly into robotic movements, creating a very intuitive interface.

Although there has been considerable progress made in the digital tools used to control robots, I'd like to close by reiterating the fact that there is an identifiable problem in the existing design-to-fabrication process. I would like to propose an improved workflow for robotic fabrication.  It is the hope of this project that the physical articulation of embodied input and output through purpose-built tools for fabrication can allow for wider adoption by and new creative opportunities for architects and designers.  In turn, I hope this will help re-establish the relationship between designers and the physical fabrication process.

Acknowledgments:

I would like to thank Harvard professors Martin Bechthold and Pangiotis Michalotos as well as Neil Gershenfeld from MIT's Center for Bits and Atoms for their support during the development of this project.

FIREFLY & KINECT

Like many people, I've been anxiously awaiting the official release of the Microsoft SDK for the Kinect.  Now, that its officially out, I spent some time over the last two weeks working on a set of Kinect related components that I hope to include in the next release of Firefly (1.007).  The first component I tried to implement was the Skeleton Tracker... and I have to admit that the result are quite promising.  It's surprisingly fast and as long as you stay within the specified range of the sensor, the results are quite good.  Using this component I put together two very quick demo videos.

There has been a big push over the last decade to develop novel 3D technology for multimedia displays (whether its new ways for stereoscopic projection, refractive lens, etc.) One of the most successful implementations and inventive (in my opinion) was Johnny Chung Lee's reverse engineering of the Wii sensor bar.  Another recent example (and equally impressive) is this hack using the Kinect sensor and head tracking. 

The video above is my first attempt to create a real-time 3D display system within Grasshopper using Firefly's newSkeleton Tracker component and some simple camera manipulation. The Skeleton Tracker component outputs a list of points (click here for further explanation).  From there, I simply use the Horster Camera Control component (another 3rd party plugin for Grasshopper) to position the camera at the viewers head and the camera target at a point in space locating the Kinect sensor.  It really is that easy.  Turn on some real-time shadows and you've got a real-time 3D display.  It still needs some tweaking but it's pretty fun to play with. 

This next demo shows how easy it is to turn gestural movements into physical actuation using an Arduino.  The setup is very simple.  My z-value of my right hand (basically the height of my hand) controls the brightness value (or Pulse Width Modulation - PWM) of the LED.  My left hand controls the servo.  When my hand is by my side, the servo goes to position 0 and if I raise my hand above my head the servo moves to position 180.  So simple.  Of course, this could be expanded to control all sorts of things... perhaps that is next.