Theme 6: Qualitative and case study research

For the qualitative research I chose the paper “Movement Perception in Music Performance – A Mixed Methods Investigation”. This study is trying to investigate the influence that musicians’ movements have in the experiencing music performance. In order to have a deeper level of knowledge they used mixed qualitative and quantitative methods. 

The one of the qualitative methods was a survey with multiple choice and comments. The participants had to categorize the segments via multiple choice, but for each segment they had also to make a comment about their choice. The point of that was a further analysis of the individual interpretation. However, the survey was not easy accessible and was not in everyday language for non musician participants. The pure qualitative method that was used was the interviews. Interview data were collected from the performers and the composers. The data were textually analyzed and synchronized with other research tracks. That gave a different approach to the concepts and insights.

From the paper I have learned how difficult it is sometimes to get qualitative data. The quantitative data were easy to be gathered in this research, but the gathering of qualitative data was challenging. Neither the form nor the language of this study subject was accessible and established by convention. In order to have a valid selection of terms, they based the concept of the qualitative research in literature.

The participants had to evaluate the perceived effort of a music performance. The segments that they had to evaluate contained only auditory or visual information of the performance. The results were quite the same in the evaluation of the auditory and the video performance, because the participants (except one) were musicians and they were able to recognize the effort of the artist only by hearing to his performance. They qualitative method has to be developed in an everyday language, so more participants, and non musician can take part to the study.  

Case study involves an in depth and detailed examination of a particular situation. This method of study is very useful when you want to try a theoretical model in real world situation. In particular, case study research design is useful for testing whether scientific theories and models actually work in the real world. You can see if your model is realistic only by trying it out in a real situation. Also case study is used in social science, psychology, anthropology and ecology, when they come to understand a more specific case this method is regarded as a valid one.

I chose the paper “A case study of Augmented Reality simulation system application in a chemistry course” from Computers in Human Behavior journal. Aim of the study is to develop a set of inquiry-based Augmented Reality learning tools for a specific lesson of chemistry in junior high school. They want to examine its effect on students’ cognitive performance, compare its effects on high-achieving and low-achieving students and investigate students’ attitudes toward the software. They have also reviewed relative computer-assistant tools in chemistry, and they end up that the AR it would be the most suitable since micro world cannot be observed in reality. Also, before the design and the development of the AR tool, researchers interview the teachers about previous experiences in computer assistant tools in chemistry. In my point of view the aim and the prior constructs were well defined.

Three hypotheses have been made for the study. Each of them has a different case concept. By introducing the AR tool in the lesson they are try to improve students’ learning motivation instead of improving the grades. They had based this case in previous relative studies.

Data gathered by quantitative and qualitative methods. Students took a Pre test and after the revision lesson with the AR tool the hade the same test again. They also answered a Likert scale questionnaire after the post test, about their experience with the tool. The questionnaire consists of four constructors in order to cover every need of the study. According to Cronbach’s Alpha coefficient grade, questionnaire and all the constructors are considered as reliable. Researchers choose randomly 5 of the 29 student and interview them to further explore student learning experience. However, in my opinion they should have interview every student in order to have better results. Also they video-recorded the entire class took some pictures during the process and kept literal notes of exceptional student performances.

All these information of the case study were analyzed after the end of the experiment. In order to analyze the quantitative data they conducted a paired t-test on the pre-test and post-test scores. They calculated statistics for the questionnaires. The results were represented but the researchers didn’t really explain “why”. However, they lead them to think of a lot improvements and future applications. 

A lot of research has been done in computer-assistant tools, AR tools and Virtual environments in education. The case study and the results were based in relative previous studies. There are not any conflictions with previous studies represented and it could be interesting to have a different approach by comparing the results to conflicting studies.

Theme 4: Reflections after the Lecture and Seminar

This week’s theme was basically about the quantitative methods and questionnaires. For the theme preparation we had to choose a paper that is using quantitative methods and discuss about them. Also, we read IEEE VR 2012 - Drumming in Immersive Virtual Reality by Ilias Bergström and reflect on the key points. I found this theme different and easier than the previous themes, since it is more practical and straight forward and it does not include any philosophical texts. 

Quantitative methods depend on numerical measurements. These data can be gathered by asking participants (questionnaires), by electronic equipment measuring or by observation under uncontrolled conditions.  Hypothesis testing experiment is about formulating a hypothesis, choose the part of population that you want to test it, design the experiment, analyze and get results. However, it is not sure that the results you will get are the results you wanted. In a controlled experiment there are two types of variables, the independent variables, which often answer the question “how many…” and the dependant variables.  An experiment will be more complex as the number of the variables increase, and we can formulate an argument after a series of tests.

Questionnaires are not used in the same way in all disciplines. For example psychology questionnaires could take years to be considered valid and a lot of revaluations are done during the design process. Some tips on how to create proper questionnaires are: we should not include knowledge questions, each question have to ask about one variable only, the order of the questions is crucial. Every questionnaire has to be tested and validate in order to be sure that people understand the questions and that we have include all the important aspects.

The study of Ilias is an example of a controlled experiment. The main question of the paper is if people behave differently under a body ownership illusion and if the body influences our attitudes and behaviors? The hypothesis was that the most appropriate culturally avatar will lead participants to perform better.  Demographic, personal characteristics and music skills data were collected by pre test questions. The experience of the experiment was recorded by post test questions. It is important to mention that data and graphs are not enough to represent the results. An argument has to be formulated to explain the data.

In the seminar we focused on quantitative and qualitative methods. Quantitative method is not always the best method. The method that we choose based on the field of the research. It depends on the context and the questions that we want to answer with our research. There are questions with qualitative and questions with quantitative need of answer. A combination of methods is better when we have both o types of questions. An interesting example is about a research for the political view of heavy metal music. If we choose to make a survey, the context will be reduced and we will lose the main reason of the study.

We also discussed about the objectivity of the data. Since we are making the surveys and we choose which data we want to collect, those data cannot be subjective. The answer to our questions was that objectivity is not the goal of the method. We care about the before and the after of the experiments. We want to know how people behave without be influenced and which are their subconscious reactions and what is their opinion. For instance, in Ilias study the participants didn’t know anything about the existence of a second body type, so that their performance will not be influenced. In other words the point is to try not to influence the participants before the experiment and as a result to not influence the data that we got.

Theme 3: My comments

1.http://gamlagreker.blogspot.se

2.http://alexisdm2572.blogspot.se

3. http://ixxzw.blogspot.se

4. http://cloudsong223.blogspot.se

5. http://denise-theoryandmethod.blogspot.se

6.http://mediatechnologybycorinna.blogspot.se

7. http://thetheoryandmethodofeverything.blogspot.se

8. http://thewind-egg.blogspot.se  

9.http://meglia.blogspot.se

10. http://aris-totall-loss.blogspot.se

Theme 5: Design research

How can media technologies be evaluated?

It is not easy to define an appropriate method of evaluation, since media technology is a broad field and its context varies.  In the paper of Haibo Li, the system evaluation consists of three parts. First one is the experimental platform, where the platform is designed according to the needs of the research. The next part is the usability evaluation. Usability is defined as the effectiveness, efficiency and satisfaction with which the user can achieve goals in the environment. Trainability is also important in the evaluation of the system, since the user should not need a lot of time to feel comfortable and familiar with the environment in order to interact effectively. As a result, users’ opinions are essential during the design of a product, in order the final product to meet the users’ needs. A prototype could be also a tool of evaluation method since it is a first sample of the final product. People can use it and give us feedback so we redesign our final product, improve it and make it better.

Evaluation of media technology should not only exist at the development of the product but also when we use them. Today’s information and entertainment technologies communicate to us through a powerful combination of words, images, and sounds. We have to enhance our skill to make critical questions about the impact of media technologies, analyze and evaluate them.

What are characteristics and limitations of prototypes?

Prototype is a sample of the final project that the researchers and user can test it and try it in order to evaluate the final product before produce it. In this way, researchers can solve any problems and improve the final product in advance. The materials of the prototype are different from the materials of the final product so the cost of it will be different. Prototype testing reduces the risks that the final product may not perform as indented. However, there are times that the prototype doesn’t work and the final design works under the same conditions, and the opposite.  Also, building the full design and then realize the problems of it, it’s a more expensive and time consuming process since you have to built the product again and again until it meets the specific needs.

What is the 'empirical data' in these two papers?

Empirical data is the information and the knowledge we acquire from experimentation and observation. In other words, empirical data could be the posterior knowledge that Kant has described. This kind of data in the research can be gained with qualitative methods.

At the first paper the empirical data retrieved through the observation of the kids when the used the prototype. That is to say they record and observe the interaction of the kids with the tangible programming system.

In the second paper the authors use several methods for retrieving empirical data. They drove most of the electric cars in the market to understand how driving range is presented to the drivers. They look up available online information about that. To further understand the opinion about guess-o-meter of the drivers of the electric cars they analyze the information in the first 50 relative forum threads on Google and also conduct some interviews. 

Can practical design work in itself be considered a 'knowledge contribution'?

I could be considered as knowledge contribution since a design research could be the starting point of another research. However, the design research is based on the observation of the users, but the results of that observation may also lead to some kind of knowledge.

Are there any differences in design intentions within a research project, compared to design in general?

Design has different intentions in different fields. Within a research project there are specific questions that have to be answered, so the design plan focuses on these. In contrast, design in general focus more on aesthetics and functionality of a product.

 Is research in tech domains such as these ever replicable? How may we account for aspects such as time/historical setting, skills of the designers, available tools, etc? 

Yes, research based on technology field should be replicable. Due to the constant development of the technology, results of the same research could vary if you repeat the same research after a period of time. Since the skills of the designers and the available tools are getting more, a research of the past could be redesigned and perform better giving out different results.

Are there any important differences with design driven research compared to other research practices?

The main difference is that design driven research is that is based in the observation of the user. In this kind of research we need empirical data that we retrieve through qualitative methods like observation and interviews.