TML 2020 #07: Completing the Prototype and Jamming Sessions

This journal is an entry in The Maker’s Lab journal series. Click here to read the previous entries.

Daniel: 
In December, 6 months into the process, we worked towards a series of jamming sessions where we invited our puppeteers – Tan Beng Tian, Darren Guo and Sindhura Kalidas to experiment with our puppet prototype. The sessions were led by Myra Loke and Ellison Tan. Most productions typically have a script as a starting point, but our creative process began from the puppet prototype. Using the visuals, sounds and behaviours that emerged from the jamming sessions, Ellison would write a script and Myra would direct the performance.

In this reflection, Xin Feng will share about her process leading up to the jamming sessions – developing the chest mechanism, integrating the eye mechanism into the head, building the limbs, assembling the puppet and finally programming the puppet. 

Xin Feng: 
In November and December, I worked on refining and completing the prototype for the jamming sessions. Due to the large amount of work and the tight timeline, we decided to skip journaling in December and consolidate the work into this entry. 

Modification of Chest Mechanism 

I found that the use of compression springs did not produce as much movement in the chest as I had hoped. Thus, I replaced the compression springs with extension springs (see Figure 1 below). Changing the springs meant that the chest structure had to be almost entirely redesigned as the manner in which the springs are used will change drastically. For the compression springs, I could place them inside the metal rods whereas for the extension springs, I had to build them externally and attach them to the moving plywood piece. 

The movement of the EVA foam increased a lot more with the use of the extension spring. I made a simple low fidelity prototype using kapaline board to test out the new chest size before building it with plywood.  

Along the way, a problem I faced was attempting to find a way to cover the top part of the chest mechanism. In the low fidelity prototype, the EVA foam only covers up to the second platform (as seen in Video 2). The up-down movement of the platform will be a distraction in the performance if I choose not to cover it. As such, I decided to explore using 2 layers of EVA foam. The inner EVA foam will translate the up-down movement of the platform into expansion and contraction. The outer EVA foam will act as the “flesh and skin” of the body that will hide the mechanism. 

For the design of the EVA foam, I was inspired by the Japanese paper-cutting techniques in Kirigami and used a similar method to change both the visual and structure of the EVA foam.  

I was always intrigued by how product designers change the characteristics of a material using simple methods such as cutting or joining. An example is Sekita Koji’s paper chair titled “Watching You” (http://www.dramatic-re.com/Watching%20You.html). I saw his chair design at a furniture fair back in 2013 and was inspired by the way paper was used in his design. I have often seen chairs built from rigid materials such as wood, plastic or metal, but Sekita’s use of paper offered a fascinating alternative material.  

The outer pattern on the chest mechanism made the material more flexible. Also, I can now join the EVA foam to the top of the platform and hide the up-down movement of the chest mechanism. I spent quite a lot of time trying to get the EVA foam to work with the mechanism and even considered replacing it with other materials like stretchable cloth such as lycra. I was glad I did not and instead, started to view the same material from different perspectives. How would the characteristic or behaviour of a material change when combined with another material or by changing its structure?  

Besides working on the structure and EVA foam, another challenge I faced was incorporating the electronics into the puppet. I realised quite late that I had to power the servo motors separately from the Arduino board and had to find space to install the additional battery pack.  Another major issue I faced was wiring up the Arduino (that was on the wearable) to the chest servo motors. The wires had to pass through the head of the puppet and run downwards. However, at the point of wiring, I had not completed the build of the head and so I had to run the wires behind the puppet instead of through the head. This was not ideal as the wires were outside of the puppet as it will interfere with the puppet manipulation process and cause wires to come loose but I had to go ahead with it due to time constraints. I will explain more in the next section.  

Head and Eye Mechanism 

For the puppet’s head, I had some difficulty shaping it as I had no prior puppet-making experience. I decided to build a low poly head structure to determine how much space I could work with for the eye mechanism. The puppet head was first rendered in a 3D software and exported to a flat polygon template using a software called Pepakura (https://tamasoft.co.jp/pepakura-en/). For anyone interested in converting the 3D rendered objects into paper craft, you can try using this software (available on Windows only). The free version allows you to export the template in a PDF format. There are other softwares / extensions available from AutoDesk but I found Pepakura to be the most user-friendly for me.  

The final face should look like the rendered object on the left. Unfortunately, the head was extremely flimsy due to the small head size and large number of joints. It fell apart shortly after it was put together. I did, however, confirm the size of the head and went on to work on shaping the head on a styrofoam block.  

I first positioned the sketch in 9 squares and sketched out on a styrofoam block. These squares helped me align the head shape onto the Styrofoam block. 

I slowly shaved off the front and sides of the styrofoam block to achieve the shape I wanted. 

The shape of the head did not turn out the way I had expected it to. It was a lot more chunky and the nose was a lot sharper than intended. I also overestimated the size needed for the head structure. I worked on a 180mm x 180mm x 180mm styrofoam block and realised I could have further reduced the size of the styrofoam block. This increase in the head size meant that I had to proportionately increase the other aspects of the puppet. Unfortunately, I did not have time to carve a new head and it was easier to make adjustments to the other parts of the body. 

In order to insert the eye mechanism, I had to hollow the back of the head and leave a Styrofoam shell to house the mechanism. As I was figuring out how much space was needed for the eye mechanism, I concurrently started to work on the papier mache on the front of the face. 

Figure 15 shows the attempt to place the eye at the back of the head after shelling the internal head structure. 

It was a challenge getting the eye mechanism to fit into the Styrofoam as there was little room to manoeuvre within the head to efficiently cut the Styrofoam out. I ended up using an old soldering iron to melt away the hard-to-reach corners in the eye socket. (Do remember to wear a respirator mask and work in a well-ventilated room if you do this as there will be toxic fumes.) 

As the distance between the two eyes in my latest eye prototype was too small and could not match the eye holes on the Styrofoam head, I also had to enlarge the base plate of the eye mechanism. 

Figure 17 above shows the new size of the base plate. As for the structure to house the eye mechanism and electronics, I tried two prototypes. The first was a two-layer plywood structure to house the eye mechanism and the wirings. The upper plywood piece will house the eye mechanism and the lower plywood piece will house the electronics. However, I decided to do away with the lower plywood piece as I felt it unnecessarily took up space and added weight. Instead, I worked on the structure as seen in Figure 18. The main platform that holds the eye mechanism sits inside a wood holder where the head rod is attached to. The Arduino Nano and electronics are mounted onto the wood holder. 

The plywood platform is also attached to a perpendicular block of wood (a T-shaped holder) that anchors the neck joint. The eye mechanism is then attached to this T-shaped holder. I used contact glue to attach the eye mechanism. However, after putting everything together, I found that the eye mechanism came loose easily during movement. Furthermore, even with multiple adjustments, I was still unable to fit the eyes in perfectly for this prototype. Moving ahead, I will have to use a different method of mounting the eye mechanism to the head structure and also rethink the housing structure.   

There were a few issues with this prototype. The styrofoam took up a lot of space at the bottom of the head structure, which was originally intended to house the wires from the Arduino Nano to the servo motors. As such, I could barely use the bottom of the head to house the wire. I was also unable to put the wires from the chest mechanism through the head structure as seen in Figure 19. It was supposed to run through the round hole at the base of the head structure, but I did not give enough space to fit the wires in. 

Furthermore, the limited space at the back of the head made it difficult to troubleshoot problems faced when operating the eye mechanism. 

It was quite a nightmare trying to adjust the eye mechanism as I underestimated the amount of work and time to be put into building and assembling the head structure. There were many issues that I did not foresee coming my way and had to employ the help of others to make the prototype work. Eventually, we even had to take a drastic measure of cutting open the entire top part of the Styrofoam head to access the eye mechanism and make necessary adjustments. 

Building the Arms and Legs 

The design of the limbs drew inspiration from the structure of the chest mechanism. Each section of the limbs was made using a rod and plywood, and later wrapped with EVA foam. 

Myra suggested using extension springs for the arms to match the design of the chest mechanism. The final look for the arms can be seen in in Figure 23 below. 

The extension springs are used to hold the arms and allow the arms to “bounce” when left static. This is something I learnt from Kian Sin’s sharing in the third session of The Maker’s Assembly. I found out that makers use elastic bands and strings to hold joints together. The limbs felt more organic and alive with the use of elastic bands. 

I liked the joints learnt during the first session of The Maker’s Assembly conducted by Daniel. He taught us a joint method using wood and a single aluminium strip. Since I was using mainly wood and metal for the puppet, I decided to go with that design. However, the end result did not really fit the overall appearance of the puppet. Also, as there were no weights on the feet, the ankle joint flipped up too easily and looked strange.  

After attaching the legs to the body, we realised that the position of the legs made it difficult for the puppeteers’ fingers to reach and manipulate the legs. As seen in Figure 26, we had to add extra wood to help the puppeteers to reach the legs. 

The little adjustments made to each part of the puppet added to the weight of the puppet and caused the overall weight to be heavier than expected. I will write more about it in the later part of the entry.  

Programming the Accelerometer and Chest Mechanism 

Besides the building of the puppet, I was also taking some programming lessons with Lin Xiang. Lin Xiang, who is currently pursuing his PhD in Electrical and Computer Engineering, was asked to join us briefly as our Arduino Programming Consultant. The learning curve was steep but made the process of coding a lot smoother. I learnt the use of Finite-State Machine (FSM) in coding to chart the possible states I want for the chest mechanism (see Figure 27 below).  

Finite State Machine allows the programmer to visualise the flow of the mode of inputs, triggers, signals sent to a processor and the output they would like seen in the programming. The FSM is modelled into a diagram (Figure 27) and explains to the viewer the transitions the machine undergoes with each input from the sensor. From the diagram, the viewer can identify the initial state, transition states, final states, actions performed and events/ types of activities happening in the processor. It helps in troubleshooting and sometimes, selecting the most efficient way of coding the programme. The FSM also helps the programmer to consider all possible conditions related to a state, whether it continues to stay in a particular state or moves from one state to another.  

Charting out the inputs and outputs of the mechanism helped me to visualise the entire process better. I was introduced to programming through Java in school. However, the coding language used for Arduino is closer to C++. The crash course with Lin Xiang helped put many things into perspective and helped to me to recall and link together coding terms learnt in school. Prior to the programming sessions, I was consulting online web forums, looking into open source code and YouTube videos to learn about how to troubleshoot problems faced in my code, which was time-consuming.  

Once I understood how the programme and sensor worked, I found more potential with the accelerometer. I was also able to better explain how the accelerometer worked and how it receives data and translates that into outputs. The accelerometer works by calculating the rate of acceleration in relation to gravity. Learning the basics allows you to expand on that and change the code to behave in the way you want it to.  

Puppet Assembly 

Putting together the entire puppet was another obstacle I had to overcome. I thought putting together the puppet would take less than half a day but I was wrong and that ended up with Myra, Daniel and Ellison having to come in to work on the puppet as well. I will not forget the day that the three of them stayed past midnight to help with the puppet. I felt extremely guilty for my lack of foresight and underestimating the time needed to build, assemble and troubleshoot the whole puppet.  

There was a lot of “planning” put into it but I was left with little time to build. I think planning is good, but it is also equally important to leave ample time to build and troubleshoot unforeseen circumstances, especially if one is building something from scratch. There will always be human error in building and each difference in millimetre contributes to a build up of adjustments needing to be made in the larger scheme of things, as I had experienced with installing the eye mechanism. Sometimes, it is better to think less and head straight into making the prototype. Often times, I realised I spent too much time planning and thinking about how to avoid mistakes. In making prototypes, the process teaches you to embrace making mistakes and learning from it, instead of shrouding in fear. 

Jamming Sessions and Working with the Creative and Production Teams 

We had four jamming sessions with the puppeteers using the prototype made. This was also another steep learning curve as I learnt to handle questions posed from various stakeholders of the production, such as the director as well as the lighting, sound and set designers.  

Over the course of the sessions, I discovered certain issues with my puppet prototype to continue working on. The puppet was a lot heavier than I had imagined it to be and that gave the puppeteers additional strain. The puppeteers also had problems handling the weight due to the positioning of the joystick on the head rod. The wires outside the head structure also made the handling a lot more difficult as they could not hold it closer to the pivot point (see Figure 28).  

Prior to the jamming session, I was worried about how the puppeteers would react to the puppet. The puppeteers were more experienced in puppetry than I was and I continue to struggle with the purpose of the use of technology in puppetry. If things are fine the way they are with traditional methods of puppeteering, why complicate things? However, after the first session, I had a realisation that technology used in this experiment was not to replace any part of the puppeteer or make the lives of the puppeteers easier. The chest mechanism was intended to allow puppeteers to explore passive manipulation in puppetry. I wanted to challenge the way puppeteers understand puppetry. The accelerometer on the wearable reads the constant movement of the puppeteer and the chest mechanism reacts according to the data received. A puppeteer is used to having full control of the puppet. What happens when the control is lost? How does the puppeteer react to that? In this case, technology allows the puppet to become a “co-actor” on stage. The puppeteer now has to respond to the puppet’s “breath” and react to its output. This becomes an “infinite feedback loop” and raises the question of whether the puppeteer is controlling the puppet, or whether the puppet is controlling the puppeteer?

I was given feedback that the eye mechanism on the puppet allows the puppet to express more emotions and achieve more things that traditional mechanical puppet eyes could not. This was comforting to hear, as I was a little skeptical about the idea prior to the jamming sessions. Both the eye and chest mechanism become a new puppeteering language for the puppeteer to negotiate. 

I also learnt that design decisions affect other stakeholders in the team. For example, the chest mechanism was too loud on the puppet, which would affect the sound designer. The shape of the facial features (with the forehead too protruding) also affects the way the lighting designer positions the lights on stage. Currently, the prototype puppet has deep-set eyebrows and bones. With overhead lighting, this would cast a shadow on the face and the eyes, which takes away the attention on the eyes of the puppet. The size of the puppet will also determine the height of the platform, so that the puppeteer can manipulate the puppet comfortably. This was another steep learning curve for me, as someone who has only had experience executing the design of the project.  

The process reminded me of what I had learnt in my user experience class in school. The final design has to take into consideration the various stakeholders of the project. How do you make the product comfortable for the puppeteers? How can you help other designers of the project work with your design easily? How do you design an ergonomic puppet without causing unnecessary harm to the puppeteers’ body? What are some common user behaviours that the designer can make use of in the design of the puppet?  

I took part in the puppet-making masterclass conducted by Kian Sin at the end of December 2020. That experience also helped me better understand the process of making puppets. I also learnt to use materials in a more versatile manner and how to work fast within the constraints of time. I learnt to view mistakes as a type of beauty in the process of making. In always constantly trying to plan and perfect my work, I often overlook the bigger picture. One other thing I learnt was finding the right help and tools for your work. Being independent does not mean having to rely on yourself to complete your job. I found myself relying on my colleagues more than I should when I tried to work by myself, putting them through unnecessary sleepless nights, which I am still guilty about till today.  

Surprisingly, I learnt more about life lessons through making that I ever would with a self-help book.

This article is a monthly reflection by Sim Xin Feng, the maker of our inaugural The Maker’s Lab as part of an ongoing 9-month experimental laboratory. The Maker’s Lab is curated and managed by Daniel Sim, a core team member of TFP. The ideas and reflections within the article are drawn from Xin Feng’s observations and discoveries as a maker, designer and researcher. Instead of being taken as conclusive, we hope that they serve to be a starting point for thought-provoking conversations and perhaps even debates. We would love to hear from you and can be reached at tfpmakerslab@gmail.com.

If you wish to follow Xin Feng on her journey with The Maker’s Lab, click on the ‘Subscribe’ link below now!