Tangible Music Notation Learning Environment
1. Description
The contemporary music notation system is a system that graphically represents aurally perceived music through written symbols. It is the written language of what is called music -or the science of sounds - that has its own syntax rules. Children normally learn the music notation system watching their teacher drawing symbols on the blackboard and explaining their interrelations and interconnections according to the rules.
Learning the music notation system through a tangible interface inside an ''interactive room'', is my proposal, that tries to link constructionism and the immersive potentials of architecture and new media design.
The idea is influenced by both Papert's constructionism and also the fusion of new media and architecture design,which is oriented towards a procedure of producing open projects rather than finished works. On the one hand, as Ackerman stresses: ''Papert is interested in how learners engage in a conversation with (their own or other people’s) artifacts, and how these conversations boost self-directed learning, and ultimately facilitate the construction of new knowledge. He stresses the importance of tools, media, and context in human development''. On the other hand, new media designers and architects produce projects that are actually interfaces where the three crucial factors, space, humans and objects, are active and indeterminable part of the interface in which they coexist and interact, contributing mutually to a continuous feedback loop. These interfaces are open enough to enable interactions that are subjective, relational and dynamically concrete.
1. Description
The contemporary music notation system is a system that graphically represents aurally perceived music through written symbols. It is the written language of what is called music -or the science of sounds - that has its own syntax rules. Children normally learn the music notation system watching their teacher drawing symbols on the blackboard and explaining their interrelations and interconnections according to the rules.
Learning the music notation system through a tangible interface inside an ''interactive room'', is my proposal, that tries to link constructionism and the immersive potentials of architecture and new media design.
The idea is influenced by both Papert's constructionism and also the fusion of new media and architecture design,which is oriented towards a procedure of producing open projects rather than finished works. On the one hand, as Ackerman stresses: ''Papert is interested in how learners engage in a conversation with (their own or other people’s) artifacts, and how these conversations boost self-directed learning, and ultimately facilitate the construction of new knowledge. He stresses the importance of tools, media, and context in human development''. On the other hand, new media designers and architects produce projects that are actually interfaces where the three crucial factors, space, humans and objects, are active and indeterminable part of the interface in which they coexist and interact, contributing mutually to a continuous feedback loop. These interfaces are open enough to enable interactions that are subjective, relational and dynamically concrete.
The target group of this learning environment is children near the age of 10 that have already obtained some
knowledge on mathematics and the very basics of music theory.
Children who enter in this ''interactive room'' can find different objects that are simulating musical symbols,
which they can use in order to construct -or compose- music and at the same time listen to their creations.
Manipulating these objects will result in audio and visual feedback that will help children understand the music notation system and the relations between the different symbols. Children can work individually or in teams.
Most of the interactions are happening on the floor which is capable of recognizing the position, movement and orientation of the objects. A complementary tracking system (through cameras) in the room can always inform about childrens' position inside the room and their transactions with the objects. The walls that are near a source of melody, can change colors via illumination, according to the pitch of the melodies produced (brightest colors when pitch is higher).
These objects are:
• Thin pieces of malleable rods that can be placed on the active floor one parallel to the other in order to
form a staff. As soon as a staff is created the color of its rods is changing to indicate that it is ready to
work with. Adding more rods one next to the other gives the potential to expand the length of the staff
and create larger (in time duration) melodies.
• Clefs can be clef-shaped objects which are squeezable in one point and at the same time they act as
speakers that produce the sound of the staff they are placed on. Putting some pressure on this point
reproduces the melody that respects the sequence of notes on that staff. The volume of the melody is
proportionally changing, depending on the number of children that are working on a staff.
• Round objects that represent the notes can be placed on the staff. As soon as a note is on the staff, a
visual feedback depicted on the upper side of the object is informing the user about the name of this note
and its tone length. Rotating the note clockwise increases its tone length. Rotating the note anticlockwise
decreases its tone length. The default tone length one is a quarter. An audio feedback
respective to the changes always appears through the note which can act as a speaker also.
• Different shapes can be used to represent sharps, flats, rests or dots.
• A clock like object can be used to represent the time signature. The rotation of its winder will change the
time signature value which is depicted on its top. Is not mandatory to use this object if children just want
to improvise with melodies without restrictions.
• Smaller rods can be used as bars lines to separate the different measures . Once a bar line is placed on
the staff the system is checking if this is ''musically correct'' according to the time signature. If not all the
notes are becoming light red which means that there is a mistake. There are three options for the user to
overcome this mistake. Changing the tone length of one or more notes, changing the position of the bar
line and changing the time signature. A combination of them can also be used in this case.
which they can use in order to construct -or compose- music and at the same time listen to their creations.
Manipulating these objects will result in audio and visual feedback that will help children understand the music notation system and the relations between the different symbols. Children can work individually or in teams.
Most of the interactions are happening on the floor which is capable of recognizing the position, movement and orientation of the objects. A complementary tracking system (through cameras) in the room can always inform about childrens' position inside the room and their transactions with the objects. The walls that are near a source of melody, can change colors via illumination, according to the pitch of the melodies produced (brightest colors when pitch is higher).
These objects are:
• Thin pieces of malleable rods that can be placed on the active floor one parallel to the other in order to
form a staff. As soon as a staff is created the color of its rods is changing to indicate that it is ready to
work with. Adding more rods one next to the other gives the potential to expand the length of the staff
and create larger (in time duration) melodies.
• Clefs can be clef-shaped objects which are squeezable in one point and at the same time they act as
speakers that produce the sound of the staff they are placed on. Putting some pressure on this point
reproduces the melody that respects the sequence of notes on that staff. The volume of the melody is
proportionally changing, depending on the number of children that are working on a staff.
• Round objects that represent the notes can be placed on the staff. As soon as a note is on the staff, a
visual feedback depicted on the upper side of the object is informing the user about the name of this note
and its tone length. Rotating the note clockwise increases its tone length. Rotating the note anticlockwise
decreases its tone length. The default tone length one is a quarter. An audio feedback
respective to the changes always appears through the note which can act as a speaker also.
• Different shapes can be used to represent sharps, flats, rests or dots.
• A clock like object can be used to represent the time signature. The rotation of its winder will change the
time signature value which is depicted on its top. Is not mandatory to use this object if children just want
to improvise with melodies without restrictions.
• Smaller rods can be used as bars lines to separate the different measures . Once a bar line is placed on
the staff the system is checking if this is ''musically correct'' according to the time signature. If not all the
notes are becoming light red which means that there is a mistake. There are three options for the user to
overcome this mistake. Changing the tone length of one or more notes, changing the position of the bar
line and changing the time signature. A combination of them can also be used in this case.
The note object:
The sharp-flat object:
The time signature object:
2. Example scenario
- Ten students aged 9-11 enter in the room and find the objects that they can use to compose music melodies. They have already been provided with informations about these objects and their affordances.
- Three of them start to work individually forming their own staffs. The rest are forming groups of three and four
in order to work with bigger (in length) staffs and work as teams.
- In a while sounds are starting to appear because the children started to place clefs and notes on the staffs. They squeeze the clefs in order to reproduce the melody of their improvisation and start to understand that the more people working on one particular staff the louder the melody is being reproduced. As a consequence, those of who are working individually are entering the preexisting teams in order to produce louder melodies. Finally, there are two teams of 5 children each, whose melodies are making the walls of the room react to the pitch of the melodies being reproduced. The walls close to the first team are now illuminated with dark colors, whereas the walls close to the second team are illuminated with brighter colors.
-Both teams are experimenting with different position of the notes on the staff, in order to change the
illumination of the walls. Later on the two teams are collaborating in order to achieve similar illuminations.
- As soon as they decide to place the time signature object their creations are 'broken' because the notes on the staff are not corresponding to the rules. So they start placing the bar line objects and they also change the tone length of some notes by rotating in order to keep up with the corresponding time signature.
-In the end the children are forming just one long staff on which they work all together to achieve the loudest
possible sound. Sometimes improvising without the time signature constraints and some other times using the time signature and bar line objects to form 'proper' melodies.
- Ten students aged 9-11 enter in the room and find the objects that they can use to compose music melodies. They have already been provided with informations about these objects and their affordances.
- Three of them start to work individually forming their own staffs. The rest are forming groups of three and four
in order to work with bigger (in length) staffs and work as teams.
- In a while sounds are starting to appear because the children started to place clefs and notes on the staffs. They squeeze the clefs in order to reproduce the melody of their improvisation and start to understand that the more people working on one particular staff the louder the melody is being reproduced. As a consequence, those of who are working individually are entering the preexisting teams in order to produce louder melodies. Finally, there are two teams of 5 children each, whose melodies are making the walls of the room react to the pitch of the melodies being reproduced. The walls close to the first team are now illuminated with dark colors, whereas the walls close to the second team are illuminated with brighter colors.
-Both teams are experimenting with different position of the notes on the staff, in order to change the
illumination of the walls. Later on the two teams are collaborating in order to achieve similar illuminations.
- As soon as they decide to place the time signature object their creations are 'broken' because the notes on the staff are not corresponding to the rules. So they start placing the bar line objects and they also change the tone length of some notes by rotating in order to keep up with the corresponding time signature.
-In the end the children are forming just one long staff on which they work all together to achieve the loudest
possible sound. Sometimes improvising without the time signature constraints and some other times using the time signature and bar line objects to form 'proper' melodies.
3. Designing guidelines
a. Construction kit
In order to meet my aforementioned view and belief that interaction designing is constituted by three crucial
factors: space and objects and human beings, I cannot resist of considering children as part of the construction kit.
Therefore the construction kit for this project is: The interactive room as an active space where all the
interactions are happening; the manipulative objects whose affordances can cover a wide range of the musical notation; and the children who construct music inside the room manipulating these objects.
The construction process is: The students that manipulate this set of objects should start by forming a staff on the floor. That is, they construct the platform on which they will start interacting. Afterwards, they start placing note objects on the staff. Each placement of such an object as well as each rotation gives an audio and visual
feedback, respective to the position of the note on the staff and its rotation. Placing the clef symbol gives them
the opportunity to reproduce the melody of the notes that are already placed on the staff. Thus, they are able to
listen to their creation and form melodies improvising as much as they desire. Each time, each different
configuration results to the invention of a new melody. Placing the time signature as well as the bar line objects will help them disambiguate the interrelations of all these symbols and hopefully start constructing melodies that are ''musically correct''.
b. Low floor/ High ceiling / Wide walls
Low floor: This project is a learning environment for novices that will expand their knowledge on basics of
music notation and mathematics. So, people who have absolutely no idea of the music notation system will not find it so pleasant although they can experiment. Hence, the low floor expectation is partly met.
High ceiling: The educational objective of this project is that users will understand the music notation system.
The more experienced a user is, the more fine the melodies that are produced. But this doesn't mean that there is a potential for more sophisticated projects than this, so the high ceiling expectation is not met.
Wide walls: I decided to put more emphasis on the wide walls expectation in order to support the different music explorations that one can experience: Improvising or respecting the time signature; alone or in teams; loud or not; long or short melodies. All these different types of exploring result in different wall illuminations that are coupling to the corresponding melodies, enhancing the visual aspect of the whole experience.
c. Froebel vs Montessori
I believe that this project has a key point that draws a line between Froebel and Montessori approach. This point is the time that people are starting to use the time signature object. Until this point the Froebel approach is present. People are free to experiment and improvise with the objects, forming configurations in an open ended way. The design of the music structures (melodies) and the visual and audio feedback that are accompanying these structures, are encouraging the recognition of the music notation patterns and how these patterns are somehow ''inherent or embodied'' in the corresponding sounds.
As soon as people start using the time signature object, there is a shift to the Montessori approach which is
orienting the users to successful configurations. The creation of music is following a puzzle-like path were the
explicit constraints are there to indicate some goal. This goal is the respect to the music theory rules. The
configuration of the music structures is not any more a result of improvisation. Rather, it is a more specific
configuration which will be characterized as a successful one or not. The attention from this point and on,
polarizes to the interrelations between the objects when building music structures. Hence, it promotes the
conceptual manipulation related to the abstract concepts of music theory.
d. Social learning
The project promotes the creation, sharing and remixing of knowledge that one can obtain in an environment
like this. Children that enter in this room are motivated to collaborate because this results to louder melodies.
Thus, they start to work together, composing music. Music stands as the new language in which they
communicate and therefore, the socialization is not depended on the existing social engagements but on this
new knowledge about this language. The learning activity is collaborative and is taking place in a bottom up
way where no interference of teachers is present. Hence, children are continuously conveying their
understandings to their peers, testing for their validity without any type of reinforcement. Their mistakes are justforcing them to more efficient interactions in order to form proper melodies.
a. Construction kit
In order to meet my aforementioned view and belief that interaction designing is constituted by three crucial
factors: space and objects and human beings, I cannot resist of considering children as part of the construction kit.
Therefore the construction kit for this project is: The interactive room as an active space where all the
interactions are happening; the manipulative objects whose affordances can cover a wide range of the musical notation; and the children who construct music inside the room manipulating these objects.
The construction process is: The students that manipulate this set of objects should start by forming a staff on the floor. That is, they construct the platform on which they will start interacting. Afterwards, they start placing note objects on the staff. Each placement of such an object as well as each rotation gives an audio and visual
feedback, respective to the position of the note on the staff and its rotation. Placing the clef symbol gives them
the opportunity to reproduce the melody of the notes that are already placed on the staff. Thus, they are able to
listen to their creation and form melodies improvising as much as they desire. Each time, each different
configuration results to the invention of a new melody. Placing the time signature as well as the bar line objects will help them disambiguate the interrelations of all these symbols and hopefully start constructing melodies that are ''musically correct''.
b. Low floor/ High ceiling / Wide walls
Low floor: This project is a learning environment for novices that will expand their knowledge on basics of
music notation and mathematics. So, people who have absolutely no idea of the music notation system will not find it so pleasant although they can experiment. Hence, the low floor expectation is partly met.
High ceiling: The educational objective of this project is that users will understand the music notation system.
The more experienced a user is, the more fine the melodies that are produced. But this doesn't mean that there is a potential for more sophisticated projects than this, so the high ceiling expectation is not met.
Wide walls: I decided to put more emphasis on the wide walls expectation in order to support the different music explorations that one can experience: Improvising or respecting the time signature; alone or in teams; loud or not; long or short melodies. All these different types of exploring result in different wall illuminations that are coupling to the corresponding melodies, enhancing the visual aspect of the whole experience.
c. Froebel vs Montessori
I believe that this project has a key point that draws a line between Froebel and Montessori approach. This point is the time that people are starting to use the time signature object. Until this point the Froebel approach is present. People are free to experiment and improvise with the objects, forming configurations in an open ended way. The design of the music structures (melodies) and the visual and audio feedback that are accompanying these structures, are encouraging the recognition of the music notation patterns and how these patterns are somehow ''inherent or embodied'' in the corresponding sounds.
As soon as people start using the time signature object, there is a shift to the Montessori approach which is
orienting the users to successful configurations. The creation of music is following a puzzle-like path were the
explicit constraints are there to indicate some goal. This goal is the respect to the music theory rules. The
configuration of the music structures is not any more a result of improvisation. Rather, it is a more specific
configuration which will be characterized as a successful one or not. The attention from this point and on,
polarizes to the interrelations between the objects when building music structures. Hence, it promotes the
conceptual manipulation related to the abstract concepts of music theory.
d. Social learning
The project promotes the creation, sharing and remixing of knowledge that one can obtain in an environment
like this. Children that enter in this room are motivated to collaborate because this results to louder melodies.
Thus, they start to work together, composing music. Music stands as the new language in which they
communicate and therefore, the socialization is not depended on the existing social engagements but on this
new knowledge about this language. The learning activity is collaborative and is taking place in a bottom up
way where no interference of teachers is present. Hence, children are continuously conveying their
understandings to their peers, testing for their validity without any type of reinforcement. Their mistakes are justforcing them to more efficient interactions in order to form proper melodies.
