Adjusting my data analysis strategy

Posted on by
Reply

I have been reviewing and rethinking my data analysis approach. I have been coding in NVivo directly to the audio track. This has been revealing in terms of coding and memo writing but is extremely time consuming. After thinking through the process from my original strategy of using Convergent Interviewing and keeping short notes rather than transcribing the whole interview, I realised that I need to revisit this and code against my typed notes rather than the whole audio track.

According to Dick the original Convergent Interviewing process relied on note taking without transcription where the notes are linked to the under pinning theoretical framework.

I am now going to:

  1. code in NVivo against the notes I took during and after the interviews
  2. review the contract summary forms to determine themes
  3. query the codes and memos from the text coding and memo writing
  4. combine with coding from the interview recordings

This will return me to my original plan and speed up the data analysis process.

 

Data analysis – is hard

Posted on by
Reply

I’ve had to stop writing my thesis for the time being, I’ve come to the end of the first draft of the literature review but I have been told by my supervisor to concentrate on the data analysis otherwise I’m going to run out of time. I was going to start with In Vivo coding but this felt too loose and unstructured. I prefer to use more structured methods. So, I’ve gone back to basics and reviewed some of the literature on qualitative data analysis.

I’ve been reading The Coding Manual for Qualitative Researchers by Jonny Saldana and Qualitative Data Analysis by Miles and Huberman and have decided to avoid In Vivo coding and start again by creating a conceptual framework based on research questions, hypothesis, problem areas, and / or key variables that the researcher brings to the study

I’m going to create a set of codes based on:

Bogden and Biklen’s (1992) coding accounting scheme.

  1. Settings/Context: general information on surroundings that allows you to put the study in a larger context.
  2. Definition of the situation: how people understand, define, or perceive the setting or the topics on which the study bears.
  3. Perspectives: ways of thinking about their setting shared by informants (“how things are done here”).
  4. Ways of thinking about people and objects: understandings of each other, of outsiders, of objects in their world (more detailed than above).
  5. Process: regularly occurring kinds of behaviour.
  6. Activities: regularly occurring kinds of behaviour.
  7. Events: specific activities, especially ones occurring infrequently.
  8. Strategies: ways of accomplishing things; people’s tactics, methods, techniques for meeting their needs.
  9. Relationships and social structure: unofficially defined patterns such as cliques, coalitions, romances, friendships and betrayals.
  10. Methods: problems, joys, dilemmas of the research process – often in relation to comments by observers.

Libraries and customer service

Posted on by
Reply

It has become clear after extensively reviewing the literature on customer services in the Higher Education sector that student services have been slow to the party when it comes to changing processes and services to reflect changing student expectations.

I have found that the university library sector has been ahead of the game for years and there is a significant amount of published research, case studies and articles outlining what has been achieved over the past twenty to twenty five years.

Libraries have really followed on from IT Help Desk systems where staff in organizations became connected to networked Help systems. These allowed staff to place a ‘ticket’ that specified an issue or system failure. The response would likely be someone visiting the staff or a fix via network tools.

Over time the IT Help Desk was reconfigured into a Service Desk where service staff were able to provide more customer focused services not just system fixes. Service Desk staff were eventually able to provide training and other support.

These Service Desk and Help Desk systems have now been introduced into university student service desks and allow students to access 24 hour self service and receive streamlined services.

The issue for where I am carrying out my research project is the SID system (Student Information Desk) combined with a new physical student HUB has significantly changed the working practices of the Faculty support teams. The main impact has been a reduction in student foot-fall and fewer students coming in person.

This has had considerable impact on the Faculty teams in terms of how they now perceive their roles, the SID system has in fact imposed an alternative identity on teams and individuals. The system has demanded that the staff undertake additional training and changes to work process flows. Students are now more likely to go to the student HUB and issues are posted on the SID system where Faculty interventions are needed.

The system has changed the relationship between the staff and students and what were front line staff have now become back-office staff. These issues have generated a mass of data for my research project.

Reflections on the interviews

Posted on by
Reply

Convergent interviewing as a process is very effective for getting participants to provide a lot of detailed information in a reasonably short period of time. In my experience the participants found the process interesting because the majority of them thought that they would be asked a series of structured questions and when I told them that I only had one main question they found this slightly perplexing.

For the majority of cases the participants did not find it difficult staying on  topic or talking for long periods of time without prompting. There was no issue with any participant about the recording of the interviews. Everyone was happy to sign off on the confidentiality form.

All of the interviews lasted a minimum of 50 minutes and it would have been possible to have continued with the interviews for longer in some cases.

The interviews needed a lot of pre-planning, listening to the last recording, designing the next question, preparing the location, contacting the next candidate to confirm that they are happy to participate. I arranged for each participant to select the location, the date and time.

The snowballing technique worked well. Every participant was able to provide the details of someone who would be useful for me to speak to next or at some point in future. I also put together a list of possible participants.

At the start of the process I intended sticking as closely as possible to the way that Dick had described but I found out quite quickly that I had to compromise the approach. I had intended putting together a reference group, as described by Dick and did recruit a small representative group who agreed to work with me. Very quickly I encountered a problem, the reference group could not agree when to meet as a group mainly due to time constraints and work responsibilities.

Eventually I realised that I needed to get on with the interviews as time was running out. I spoke to each of the people in the representative group and asked them to recommend a group of participants individually rather than meeting as a group. In the end the compromise worked well.

As the interview process has carried on I have not referred to the reference group. I have interviewed three members of the reference group because they line manage several of the subsequent interviewees.

The main thing that has worked is the open-ended unstructured interview and the note taking during the interviews. I have not compromised the process that much only in so far as it makes practical sense. So far the experience of convergent interviewing has confirmed to me the usefulness of semi-or unstructured questions for eliciting answers that are broad based but also contain enough specificity to get enough data to make the outputs useful.

Thesis first draft continued

Posted on by
Reply

I have now started to draft my thesis and have been using the examples mentioned in the previous post. I have organized my literature using Mendeley  by making sure that all the references included are complete. I created a watch alert in Google Scholar so that I now receive an email whenever a new paper is published in a journal that mentions Actor-Network Theory. The alert only refers to papers published in 2016 and 2017 so all the papers I am accessing are up to date. Some are pre-publications (in press) and some are not accessible to the LSBU library but the majority are accessible and very good sources. I have noticed that many recent papers are of extremely good quality in terms of the writing and also that the disciplinary areas that ANT is being applied to are very broad and very interesting.

These papers are an extremely good source of material for my literature review. I have started writing about the history of organizational change and have come across a number of very good references to old sources going back to the mid-1940’s and 1950’s. These seminal sources are in some cases still relevant to organizational change and many such as Kurt Lewin’s unfreeze, move, re-freeze are still being used and adapted by subsequent research.

I will be working on the following sections shortly, personal motivation, the research site and outline of the thesis. I am going to start the literature review with a review of the student help desk and the innovations that libraries have brought to the development of wider student services in universities.

First draft

Posted on by
Reply

I have decided to look at some published / online PhD thesis papers in order to see how they are structured. I have downloaded a few Thesis that use Actor-Network Theory.

I have been reading the structure of the thesis entitled Auditing in Electronic Environments from an Actor-Network Theory Perspective: Case of Egypt by Manal Nour El Din El Safty of the  Helwan University, Egypt. The thesis provides a good structure model for my thesis and provides a reasonable guide on a structure for writing about ANT.

The thesis also gives  a good structure for the literature review and how to organize the research  strategy, design, philosophical underpinning and the research methodology.

Another paper The use of Actor-Network Theory and a Practice-Based Approach to understand online community participation by Gibrán Rivera González of the University of Sheffield is another good example of the structure of an ANT thesis.

I have created a draft template for my thesis and am continuing to read for the literature review.

 

Starting to draft my thesis

Posted on by
Reply

It’s time to start drafting my thesis. I have decided to start with the literature review and to focus on developing a review of the history of change theories  in organisations. The reason for this is that my research is focused on the process of change and the impact of change on the identity of teams. It is important to understand where and why change theories came about and how these relate to the research component of the project. Also Actor-Network Theory is centred on change in terms of networks and their associations and how these come into being, stabilise, become durable or destabilise and fall apart.

The main aspect of the project is centred on the restructuring of the faculty administration teams and this is based in some core change theory.

Reflections on Convergent Interviewing

Posted on by
Reply

Using convergent interviewing was an interesting experience. I had not used the technique before except as part of a short pilot of the project. The technique was selected because it was determined to be sympathetic to the principle of Actor-Network Theory whereby the researcher should have no a priori views of the situation in question, the technique starts with one very open ended question with following questions being directly derived from the answers that the participants provide. There is very little in the way of interviention on the part of the researcher.

The other reason is the technique was sympathetic to the organisation that the research was carried out in. Staff were selected on the basis of their role with subsequent participants selected by other participants which was seen as a democratic way of selecting participants.

The process as described by Dick recommends that Convergent Interviewing is carried out by two viewers, one who asks the question and the other takes notes. Obviously I was not able to work in collaboration so had to ask the question and take notes myself.

Dick also recommends taking hand written notes and no more than a side of paper. Subsequent researchers have used voice recording though and to be thorough I decide do to use voice recording and then transcribe the interviews. I did take notes during the interviews.

At the beginning of the interview process I found it quite difficult to decide when to change the initial question because it seemed to generate a lot of interesting data.

It was pretty difficult to decide when to change the question but I managed to do so. After this is was easier to decide when to change the question. Basically, I changed the question whenever there was a significant change in the type of role a participant was in.

Participants found Convergent Interviewing interesting. Most people we a little confused by there being only one main question and were skeptical when I told them that they should speak of up to an hour without any other major questions or prompts.

What was interesting was that once the participants got going they all found it easy to keep talking without the need for further questions and only minimal prompts.

For the interviews I had typed and printed the main question on a piece of paper and included four or five issues that the participant might  want to talk about although they did not have to and some didn’t.

For the majority of participants I found that I did not need to provide any prompts because after some initial thoughts people found it easy to talk about their job role and the history of the organisation as far as they knew it.

I asked each participant whether they agreed to have the interview recorded and they all did.

After each interview I reviewed my notes and the questions and made some decisions about the next question. I asked each participant to recommend the next participant, so the process was built around snowballing in terms of participant selection.

The main issue with Convergent Interviewing seems to be the amount of time that it takes to analyse the data and the open ended nature of the questions can lead to a lot of divergent data.

I would like to use the technique again and with more inteviewers, as described by Dick. The technique seems to be often used where there is feeling that something interesting is happening but there is no clarity. The process can be used to clarify issues and help focus attention on critical issues for later follow up.

 

Additional interview

Posted on by
Reply

After completing the interviews I decided to review the outputs to date and decided that it is necessary to add an additional interview but with much more detailed questions. The reason for this is that there is a need for some more in depth data regarding the use of the SID system by a key user and manager of the system.

I decided to ask the assistant Registry manager a series of carefully refined questions. The questions stemmed directly from previous participant answers and followed the Convergent Interviewing practice of reviewing the output of a series of questions and answers and then generating additional questions.

From the first question set I drafted a further five questions each with a few supplemental questions. The final question set contained 29 questions so the process moved from being semi-structured to structured.

The aim of the final question set was to allow for the participant to provide some very specific answers regarding the system in use.

 

 

Last interview

Posted on by
Reply

I carried out my last interview on the 3 July. The interviewee was the Academic Services manager who talked about the SID system and the new student HUB that will be opening in September.

The interview was interesting because it was a summary of the situation now and acts as an end point to the start of the interview process that looked back to times before the current organisation structure, over a period of about thirteen years.

The early interviews described long periods of pre-tchnology use where staff self identified as ‘paper pushers’ up to the present day where staff now self identify as having expertise and knowledge that is valuable to academic staff and students.

The next steps will be to tie all the interview strands together and create a chronology of actions or a set of chronotopes.

 

 

Considering Actor-Network Theory

Posted on by
Reply

I am using Actor-Network Theory (ANT) as the lens through which to analyse my findings. ANT can be highly confusing. Reading ANT texts can be extremely obscure and contradictory. ANT can be thought of as a theory, a methodology or  a philosophical / sociological / ontological perspective.

One of the most often asked questions about ANT is how is ANT applied in practice? Actor-Network Theory has several definitions and each definition seems to have its adherents. ANT is highly adaptable to different disciplines, after starting as a method developed for Science Studies, to allow the researcher to investigate how science is done and how scientific innovatons develop but has since been used as a method and theory in subjects as diverse as, information technology studies, archaeology, marine archaeology, paeolentology, architecture, medicine.

Continuing with the interviews

Posted on by
Reply

I have been working on completing the research interviews. My supervisor is concerned that I do not carry on interviewing too many people due to the time that it will take to do the analaysis. The problem is the more people I interview the more interesting the project becomes and the more I want to carry on with the interviews.

In some respects this issue is at the core of the research process, how much data is enough. With the interviews I am getting to the point where saturation has been reached.

In order to provide as comprehensive overview as possible I am planning to carry out a couple of observations of meetings and some observations of staff using the SID system. I have also managed to gather a large number of documents that describe the SID development process, implementation and a series of emails regarding the use of the system by staff.

Towards the end of the interview process

Posted on by
Reply

I have to date interviewed 12 participants. I have decided to carry out five more interviews. These will be:

  1. The Director of Student Services
  2. The Registrar
  3. A user from the Distance Learning team
  4. A project manager who carried out a quantitative survey gathering data and feedback on the SID system post implementation
  5. The Director of the School

Completing these interviews will combined with the already completed interviews, provide me with a good cross section of staff and a rich base of data covering organizational and technological change.

So far I have carried out some initial data analysis using In Vivo and Descriptive coding. I will be revisiting all the coding again. I have been reading Johnny Saldana’s book The Coding Manual for Qualitative Researchers and Kathy Charmaz book Constructing Grounded Theory.

I have so far found the coding process very difficult. The volume of data can be overwhelming but the main problems I’m facing are thinking through the codes so that they make sense in terms of the research questions, having the confidence to code and to know that the coding is adequate.

In Vivo coding seems time consuming and using the words of the participants sometimes feels as though the coding is insubstantial, the descriptive coding also seems too abstract and not representative of what is being said or analytical enough.

Dealing with the number of codes used feels overwhelming and difficult to organise. I have been using post-it notes for the descriptive coding so far to avoid leaping to the use of computer data analysis first off Once I have had a couple of rounds of hand coding I will look at the data using NVivo.

Hopefully the NVivo coding after hand coding will be faster and will get to a set of conclusions quickly.

 

 

 

Considering the thesis

Posted on by
Reply

I have been considering the drafting of the thesis. Once the interviews have been concluded I will start to consider the data analysis. The process of data analysis will need to be carfully planned prior to actually carrying out the analysis. The analysis will need to be aligned with the research plan and  strategy. Using convergent interviewing lends itself to the use of Nvivo for coding.

Before the analysis or at least simultaneously I will be starting to draft the thesis. I am aiming to start with the literature review which will be a significant historical overview of organisational change management and Actor-Network Theory.

I will return to these topics in later posts.

The fifth interview

Posted on by
Reply

The fifth interview was very interesting. The participant was a team member with a very different perspective on the question.

The participant currently works in a customer facing role and has had a lot of student interaction. This has changed though over the past three to four years as a result of the introduction of technologies particularly web based services that allow students to upload their work electronically which previously had to be submitted on paper. The paper had to be processed, catalogued passed on to markers, returned to the administration team and returned to students.

This process was very labour intensive and made the participant feel underappreciated especially when it came to how they felt they were viewed by other staff – particuallary academic staff.

The impact of technology on the team has been to free up time for the participant to spend more time with students and they felt that this allowed them to provide a much better and more personalised service than before,

The number of students coming in person has significantly reduced and this has meant that those coming in have more complex issues to deal with. This has allowed the participant to develop better customer service skills. The participant mentioned that they felt that the team had developed a better skill set and was more appreciated by academic staff as a group that had real expertise – rather than just being paper pushers.

The participant said that the SID system is a success but it is early days and there is a need to constantly reinforce the message – with staff and students – that it should be used.

This was a very interesting interview because the participant is a direct user of the SID system and had some useful insights. Once again I felt the interview went well and there were some very useful convergent points raised.

The fourth interview

Posted on by
Reply

The fourth interview was with a person was quite different from the three earlier ones. The main difference was the three earlier interviews were with managers the fourth was with a team member who had quite a different perspective on the question.

The interviewee was much more focused on operational issues and talked about team structure issues. The issue of staff changes was raised and how these have had an impact on the team and its work. The participant mentioned that over the years the institution has discussed many organizational level changes but many have not been implemented and many others have been only partially implemented.

On one occasion a merger between two teams was stopped due to teams objecting to job changes and the need to develop common practices. This resulted in some conflicts developing due to a lack of openness to new ways of working.

Over time jobs have changed though mainly due to the impact of new technologies being introduced. Technologies that have reduced the need to handle paper. The technologies have increased the skill level of the team and this has in turn built the confidence level of team members who now have more time to provide better customer support.

The use of the student information desk (networked communication system) was raised and the difficulty of building a supportive network of users and how difficult it is to get user buy in. It is felt by the participant that the SID system creates an impersonal service – unlike direct email communication – even though the SID system is seen as a better system than email.

I found that this interview worked well probably because I’ve done a few now and am getting more confident about the process. The data collected is becoming very relevant to the project and ties in very well with relevant theories.

The third interview

Posted on by
Reply

I carried out the third interview yesterday afternoon. The participant talked about changes to the team that they manage and who the changed have occurred over the past five years. Changes include changes to the structure of the team, staff leaving and being replaced and the introduction of technology.

The participant was not sure at the beginning of the interview where to start. The participant decided to start with the question “Who is my team?”. This was a great starting point for an answer about identity and multiple identities.

The participant mentioned that they are a member of four teams each of which has a different identity. The issue of working practices and the variety of technologies entering the team were mentioned.

Issues of structure change at team, department and institutional levels were mentioned. Work complexity was mentioned as an influence on the team and how the team functions.

The introduction of the student information desk was raised as an issue for team members and the different ways that staff members have responded to the system i.e. some consider it to be the enemy and others a member of the team. The view that technology has increased workloads rather than reducing it.

The participant mentioned that staff tended to be more open to adopting new technology and innovations if the new technology replaced a manual system or was clearly an improvement otherwise staff would be inclined to see the technology as a problem or be seen by those introducing the technology as blockers or resistors.

The participant mentioned that there needs to be a fit between the technology, the team and the organisation in order for it to be accepted. The need for a technology champion was mentioned as a way of helping the team adopt the technology.

After the interview I reaslied that there is a need to expand the interview scope from the four originally selected teams to include the IT manager who was responsible for the implementation of the student information desk.

Looking at the project through an ANT lens I need to start ‘following the Actors’ a bit more closely. I will be speaking to the IT team manager in the next few weeks, tracing the documentation around the initial idea for the information desk and how the system was translated from the initial idea into the actual application.

I want to investigate how the decision making process and internal relationships influenced the procurement, the project design, planning, training and implementation. I also want to investigate whether the information desk was seen as being aligned with the interests of team members who would be using the system.

The second interview

Posted on by
Reply

The second went a little better than the first. The main difference between the first and the second is I was a better prepared. I amended some of the documentation after comments from the first interview. For example the first participant didn’t want to sign the Participant Consent form because there was a section for a witness to sign. The participant commented that the interview couldn’t be confidential if the form was witnessed.

I have now removed the witness signature. The second participant was happy to sign the form and the interview started. I repeated the same procedure as the first interview, leaving the question on the table in front of the participant.

Another change I made was I made use of the Sonocent text blocking function. This worked really well in allowing me to write text notes against the recorded voice.

The interview lasted about 50 minutes. The participant was happy to speak openly about the team and covered all the points I needed. I have found interviewing to be quite tiring. The need to listen carefully and manage the recordings and type notes is quite strenuous.

I have found that the participants enjoy the experience of being interviewed and it’s interesting how much information people will provide when asking just one question. This seems to create a space where people feel they can open up freely.