Presentation is loading. Please wait.

Presentation is loading. Please wait.

An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University.

Similar presentations


Presentation on theme: "An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University."— Presentation transcript:

1 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera Jose.abdelnour-nocera@tvu.ac.uk Lynne Dunckley Lynne Dunckley Lynne.dunckley@tvu.ac.uk Institute for Information Technology Thames Valley University England

2 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Relevance: Rapid prototyping allows designers to obtain quick feedback with a minimum investment in the early stages of design; however, no valuable information regarding its usefulness can be collected until a broader adoption of the technology. (De Paula, 2003)

3 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Key objectives: 1.The use of concept of technological frames to explore how systems acquire their usefulness once deployed in their actual contexts of use. 2.Bringing the sociology of technology closer to HCI, CSCW and IS research.

4 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Key characteristics: An alternative epistemology towards the study of technology and the groups related to it that questions: The idea of systems having a fixed useful character. The boundaries between producers and users of IT/ between settings of design and use.

5 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology How many bicycles can you see? A Unsafe bicycle A Macho bicycle

6 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology The working and nonworking of an artifact are socially constructed assessments, rather than intrinsic properties of the artifact. One artifact (in the old sense) comprises different socially constructed artefacts, some of which may be working while others are nonworking. (Bijker, 1995)

7 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology The Metaphor of Technology as Text... just as the meaning of a written text is not a property of the text, so too the character of technology is not determined by its technical structure (Mackay,2000) Interpretative flexibility of Technology:

8 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Soc. of Tech. offers an epistemology congruent with the study of usefulness of IT as social proxies Traditional HCITechnology as Social Proxies Prototype based Subject Screening Lab Settings Extrinsic Motivation Task Orientation Reliable Product Critical Mass Real World Settings Intrinsic Motivation Real Problems and Needs

9 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Previous work on studying the usefulness of systems: Information Systems: Technology Acceptance Model (TAM) Davis (1989) -Perceived Usefulness : reduced to idea of performance -Perceived Ease of Use Limitations of Quantitative Methods Relation to developers responsiveness Gefen, D., & Keil, M. (1998) Relation to User Interface Characteristics Calisir, F., & Calisir, F. (2004)

10 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology HCI: Nielsens view on usefulness Sociology of Technology

11 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Focused on Meaning (Constructionist Paradigm) instead of Perceived Performance (Positivistic Paradigm) Interpretative flexibility of Technology (Sociology of Technology) Usefulness as a social construct:

12 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology The technological usefulness of software artefacts remains unknown, or known only indirectly, and relies upon improvisations that remain invisible to professional design (Suchman, 2002). Users construct technology; they do this both symbolically, in their reading of artefacts, and literally, in the articulation work that is essential before a generic system-product can be used as an artefact supporting day-to-day practices. (Hales, 1993) Usefulness as a social construct:

13 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology The core set of assumptions, expectations and knowledge of technology collectively held by a group or group. (Orlikowski and Gash, 1994, p.199) A technological frame comprises all elements that influence the interactions within relevant social groups and lead to the attribution of meaning to technical artifacts- and thus to constituting technology. (Bijker, 1995, p. 123) Technological Frame:

14 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology This approach integrates Technological Frames with the concept of Breakdowns: A breakdown is not a negative situation to be avoided, but a situation of non obviousness, in which the recognition that something is missing leads to revealing (generating through our declarations) some aspect of the network of tools that we are engaged in using. Winograd and Flores (1986, p.165-166) (Similar ideas in Situated Action –Suchmann, AT, Hermeneutics)

15 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Goals Societal and practical short and long term Problems (breakdowns) Elements of interpretation Assumptions, knowledge (e.g. problems solving strategies, praxis) and expectations: reflected by people or in discourse Elements of practice Any action, practice leading to or responding to the attribution of meaning to technology Revised Conception of Technological Frames:

16 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology EMPIRICAL WORK ELSOFT: European based software vendor of standard ERP software (BIZWARE) for small and medium sized organisations, sold around the world ETNOGRAPHY and QUALITATIVE CASE STUDIES: –RESEARCHER ADOPTED THE ROLE OF USABILITY CONSULTANT Informed Consent issues Conflicts of roles and going native. –ANALYSIS OF THE PROCESS OF USERS ROLE IN DEVELOPMENT –APPROPIATON OF BIZWARE IN CUSTOMER SITES IN INDONESIA, HONG KONG, SPAIN AND UNITED KINGDOM. DATA : –INTERVIEWS, –INTRANET DOCUMENTS, –E-MAILS AND FAXES –PARTICIPANT OBSERVATION

17 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Why Enterprise Resource Planning Software for SMEs? Used to support tangible work activities Standard design used in different cultures Producers presented it as embodying best business practice Access

18 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Natural? Usefulness Utility Usability Stakeholders Technological Frames Software Practical? breakdown Coping Strategies

19 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Usefulness of BIZWARE defined with meanings (themes) related to… Security Efficiency Customer Centredness Situated Usability Openness Completeness Quality of Automation

20 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Technological Frames Identified in the Social Construction of the usefulness of Bizware Sales & marketingDevelopersUsers SupportersVarious User Groups Goal Promote and Sell Bizware.Create a fast and efficient business data processing technology. Solve all the queries and local needs of users Support and automate their business activities wit the least effort. Problems Persuade a very diverse market with a standard system. Keep their software and database bug free. Support different user needs with a standard system and a limited power to change it. Having an unsuitable tool to support their work. Elements of Interpretation Universality of Technology Technical Centredness Universality of Technology Technical Centredness User awareness Previous Experience with Technology Understanding of Information System Genres Perception of Foreign Origin Organizational Models and Culture. Elements of Practice Sales & marketing StrategiesTechnological Inventory and workspace Development practices Contact with Users Coping Strategies: Problem Locus Construction Technological Inventory and workspace Contact with Users Coping Strategies: Suggestions of Workarounds Problem Locus Construction Local Working Practices Coping Strategies: Workarounds Problem Locus Construction Subscription to Technology

21 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology CASE 1: INDONESIA Technological Frames ThemesUser Community IND Developers at ELSOFT Security Users interpretation of the security he Bizware offers for their activities In the current functionality of [Bizware] the cost price of the good can be seen by the user who runs this function. (…) we do not want warehouse people to know the values of the goods for security reasons. (App. 5.2.1) During the evaluation of your product neither was it brought to your attention and thus it was overlooked by assuming that it is similar to our former software..(…) I know I may not be a big customer to your company but I really need the attention on this BUG issue. (App. 5.2.2) Developer: Unfortunately this is not possible. When somebody has the rights to enter receipts, he also can have a look at the purchase order and the prices of the purchase order. (App. 5.2.6) Elements of Interpretation Previous Experience with technology Customers based their definition of the problem in relation to their previous experience with similar software Organisational Model and Culture The breakdown raised by IND reflects an aspect of organisational culture that requires a more refined handling of users rights to information than offered by the Bizware. However, developers at Elsoft do not see this as a critical limitation.

22 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology CASE 1: INDONESIA Elements of practice User Community IND WorkaroundsIND altered some of their procedures to cope with what they construe as a security vulnerability: To survive of these problem, the customer is having to have somebody else (eg. finance people assigned at sales order entry, or somebody who the management can trust better) to do the job for the time being. (App. 5.2.4) These are actions created by users with the intention of solving a breakdown in the use of the software. Problem Locus Construction IND users claimed that the absence of a more refined control of information in the warehouse operation is a limitation of the Bizware. For them this should be a natural condition of the software. This breakdown is located by them in the software and this confirmed when they refer to it as a bug. Developers claim the opposite and indicate that this feature is an option and a special requirement of IND working practices. Thus, the problem is located in located by them in the users. The exchange of letters between IND managers and staff of Elsoft in Indonesia and Europe reflects this difference.

23 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology CASE 2: SPAIN Technological Frames ThemesUser Community ELCA ELSOFT Openness How flexible the software is in allowing users to configure it to suit their activities, i.e., multiple early payment discounts. In this case closed means that the Bizware is not able to handle multiple early payment discounts in invoices: Traject (one of the ERP competitors) was eliminated because we saw it very limited and closed, and afterwards [after the Bizware was implemented] we noticed that also with this software. (Ramon, interview excerpt, 27/07/2003) The main problem is that this is a closed program. It will never be enough for an enterprise. (Ricardo, interview excerpt 27/07/2003) Im telling you this is not because is an intrinsic problem of this company all the food companies at least here in Spain work in that way [with multiple early discount]. (Ricardo, interview excerpt 27/07/2003) Elsoft depicts it software as appropriate for ELCA needs: Sales Manager for Spain: In the same way we consider that the Bizware is a program perfectly appropriate for your needs, because it is precisely a live, modern, and Standard product with continuous functional improvements. (App. 5.3.2) Development manager for Elsoft in Asia: This method is not supported and not planned as a functional change either. (…) We have to be very critical in what we accept as a functional change or not. (App.5.3.4) Elements of Interpretation Software GenresUsers at ELCA compare the Bizware with other software of the same genre (ERP). Based on this comparison they interpret the software as closed. Elsoft presents its software as appropriate, line and modern. Organisational Models and Models One of ELCAs managers emphasises the idea that the Bizware does no support their commercial management by limiting their ability to handle multiple level of discounts. Developers at Elsoft does not see as critical and affirm that the software is appropriate their needs. This is an incongruence between the business model inscribed in the software and that of ELCA.

24 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology CASE 2 SPAIN: Elements of practice WorkaroundsUser Community ELCAELSOFT These are actions created by users with the intention of solving a breakdown in the use of the software. In this case workarounds were characterised by: -Extra Work -Creative use of Interface Elements and Data fields Users had increase manual operations to partially address the discount problem. In ELCA multiple discounts are included before the invoice is created: In order to solve some of the issues we have to perform extra work (…)About 20% of our client has Prepayment discount. For them the final amount is not Correct and that is why they do not pay us.. At the end of the month we make a global discount. (App.5.3.3) When errors have appeared a thing that was usually done in 30 minutes is completed in three hours. This has created a very stressful situation among our staff. Is this going to work? (…).This is not normal. (Ramon, interview excerpt 27/07/2003) To solve this we have had to assign different codes to the same client, that is, creating multiple identical clients on the system but with different discount levels. This is not practical. (Ricardo, interview excerpt, 27/07/2003) Alternative use of Internet Elements: Since we started working with the Bizware we are able to see the expiry date of the product because this can be seen in the sales order reference number. In this way you could see if the customer could stop an order because an expiry date issue. (Marta, interview excerpt, 26/07/2003) In the fields tagged as extra, I create each article (item) card and I use a lot of the extra fields to include cost information per boxes, per pallet, net weight, etc. (Marta, interview excerpt, 26/07/2003) In some order lines some clarifying notes or in your reference field we put some references to our order, I dont need that field so I use it to know, for instance, whats inside that order, e.g., marcador butter. (Ale, Interview Excerpt, 29/07/2003) Development staff at Elsoft suggested a workaround using phantom items in the invoice. However, the local consultant in Spain was not too convinced since it was a partial solution: Please do not reject after all. In job 07.440.257 some improvements have been made. However, the customers still need to be able to record somewhere in the master data that a debtor gets an invoice discount. And this info needs to automatically be used when entering an order/invoice. (App.5.3.4)

25 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology CASE 2 SPAIN: Elements of practice Problem Locus Construction User Community INDELSOFT The General Manager for Elsoft in Spain writes about the discount problem as a fundamental limitation of the Bizware: This is a legal requirement, this is business crucial (App.5.3.4) The development manager in Elsofts headquarters questions the necessity for that feature in the software. This is evident when he says that a functional change is not planned. This means that this feature is not considered as fundamental for the software: This method is not supported and not planned as a functional change either. (…) We have to be very critical in what we accept as a functional change or not. (App.5.3.4)

26 An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University - Institute for Information Technology Conclusions: The definitions of how useful a system is and of the problems it is deemed to solve are not static but negotiable, fluid and subject to the sociocultural perspectives and practices of both producers and users. Contributions and Further Work: TF offer qualitative sociological framework that explains how the usefulness of systems is socially constructed. Analysis framework that can be integrated with novel UCD context-based techniques such as Bill Gavers cultural probes. Reflective approach that can equally consider producers and users perspectives for the identification of conflicts, e.g. elements of interpretation and practice in RAD methods such as XP and DSDM.


Download ppt "An Approach to the Evaluation of Usefulness as a Social Construct using Technological Frames Jose Abdelnour Nocera and Lynne Dunckley Thames Valley University."

Similar presentations


Ads by Google