Presentation on theme: "Workplace Digital Essential Skills in Rural Small Businesses Pilot Project SDRC – Digital Essential Skills Assessment Orientation Session, October 30,"— Presentation transcript:
Workplace Digital Essential Skills in Rural Small Businesses Pilot Project SDRC – Digital Essential Skills Assessment Orientation Session, October 30, 2013, Campbellton, New Brunswick
Identifying the occupations where participants would be trained in Identifying the digital tasks participants would be trained to perform
Identified occupations where there were would be large numbers of lower literacy workers who use Information and Communications Technologies Based to some extent on the National Occupational Classification system and the OLES website – identifying required skills relating to Information and Communication Technologies Confirmed in the pre-ONA conducted in the summer that these were the “right” occupations 3
Administrative (all sectors): general office, administrative, accounting, purchasing, and production clerks Production (manufacturing sector): machine operators, assemblers, inspectors, material handlers, labourers, cleaners Operations (service sector): cleaners, housekeepers, cooks, material handlers, labourers, patient service associates Client service (all sectors): customer service clerks, receptionists, client service assistance, sales associates 4
Examined past research – external validity Advised by Satya Brink, an expert in the field Derived a framework: a continuum of tasks covering the range of digital activities workers perform in their job Tasks are common across, but contextualized to, occupational groups Confirmed in the pre-ONA with participating organizations 5
Use 1.Consult/read documents to the Web or other digital media 2.Use documents/databases on the web or other digital media 3.Complete forms on the Web or other digital media Understand 4.Perform online commercial transactions 5.Search databases on the Web or other digital media 6.Seek, find choose information on the Web Communicate/Create/Do 7.Use a calendar on the Web or other digital media 8.Communicate electronically with co-workers, suppliers, clients to coordinate workplace activities 6
Administrative purpose (pre-training assessment): to determine which digital tasks an employee lacks the skills to perform and therefore needs digital skills training for Employer indicates what tasks the employee does and thus needs to be assessed in Results contribute to “skills map” of the organization shown to employer Research purpose (with post-training assessment): to measure the effectiveness of training in increasing digital skills levels by comparing pre- to post-training levels 7
Pre & post training assessments should be very similar Reason: to ensure measured gains in skills between pre- and post-training are due to the training and not the assessment instrument (easier or harder) Risks: 1. potential for participant to remember the pre-test 2. can’t be totally certain skills gains are due to only the training (may have been other causes) Common to all training evaluations where there is no control group (individuals who were not trained) Should not “teach to the test” - assessment should not have exactly same content as the training otherwise would be testing memory, rather than ability 8
Creating an assessment for each of the 8 digital tasks, with some tasks having up to 3 sub-tasks This means a lot of assessments: for each occupational group each with 4 or 5 component detailed occupations, there are up to 80 cells for which assessments must be produced However, once a “generic” assessment question set is produced for a digital task, it can fairly easily be contextualized across detailed occupations By adapting positions and documents applicable to the particular occupation 9
Digital TasksAdministrationProduction/OperationsClient Service 1. Refer to documents on the Internet or by using other digital media notes, letters, procedures manuals, schedules notes, letters, service or user manuals notes, customer info sheets, delivery schedules 2. Use documents and databases on the Internet or by other digital media address books, inventories, suppliers lists lists, parts catalogues, user manuals client lists, price lists, competitor catalogues 3. Complete forms on the Internet or by other digital media invoices, purchase orders, shipping slips (or waybills), time sheets production reports, purchase orders, time sheets client complaints forms, sales reports 4. Perform online commercial transactions order, pay 5. Search databases (…) on the Internet or other digital media to find useful information (…) supplier lists, supplies lists prices, quantities, contact info suppliers’ part/product lists. dimensions, quantities, contact info client and competitor lists addresses, telephone numbers, email addresses 6. Seek, find and choose information on the Internet prices, government policies, regulations equipment operation or repair specifications competitor prices and products 7. Use a calendar to schedule … on the Web or other digital media activities, meetings, ordersproduction, delivery schedulessales, delivery 8. Communicate electronically with co-workers, suppliers and clients to coordinate workplace activities nothing specific to this occupation
To measure skills gains, need to measure gradations of skill not just whether or not one has a skill Skills gains = difference between pre and post-training skills scores Skill score depends on 3 dimensions of complexity: closed/open and well/un-defined, and complexity of sub-tasks required for each task
Closed/open: ◦ Closed problem has limited solutions: “Identify the number of employees that left the company in April” ◦ Open problem has unlimited solutions: “Improve the system for waste disposal that meets municipal green policies” Well/not (or un)-defined: ◦ Well-defined problem is directive and clearly tells employee what they are supposed to do: “Find and download government form x”. ◦ Undefined problem is less clear about what to do but more clear about the situation and what the employer needs: “Schedule the delivery of products to Calgary within the next 48 hours.”
Complexity of sub-tasks in different domains required to successfully carry out the task: ◦ Prescriptiveness of instructions ◦ Clarity of expected outcomes ◦ Sophistication of technological applications ◦ Complexity of digital info and needed info processing skills ◦ Preciseness of task setting Still working on details to formulate a skill score, but an example follows
14 123 1. USE Task: Use Digital Information for consultation purposes only simple instructions, repetitive actions a few simple identified technological applications within a precise task setting Task: Use Digital Information to perform a simple task more complex instructions but fixed actions using several identified simple technological applications within a defined task setting Task: Use Digital Information to perform multiple, undetermined and complex tasks several complex instructions or no instructions and unidentified actions using unlimited and undetermined complex technological applications (selection) within a familiar task setting 1. UNDERSTAND (ORGANIZE) Task: Understand Digital Information for one specific, simple purpose clear instructions and requirements a few simple well-known technological applications within a less precise task setting : Task: Understand and Organize Digital Information for simple tasks more complex instructions and requirements using several potential technological applications (selection) within a partially defined task setting Task: Understand and Organize Digital Information to perform multiple, undetermined and complex tasks several complex instructions or no instructions and requirements using unlimited and undetermined technological application (selection) within an less determined task setting 1. CREATE, COMMUNICATE AND DO Task: Create Digital Information for one set or fixed outcomes simple instructions and a simple process using a few simple technological applications (selection) within an imprecise task setting : Task: Create and Analyse Digital Information for several expected outcomes with complex instructions and multiple yet simple processes (selection) multiple identified potential technological applications (selection) within undefined task setting: Task: Create, analyse and communicate Digital Information for several self-selected potential outcomes several complex or no instructions using multiple undetermined and unlimited processes and technological applications within an open undefined task setting (selection) CLOSED OPEN NOT DEFINED DEFINED 1 23 2 3 4 3 4 5
Question 1: You received an email from the bookkeeper asking about the arrival of some merchandise. Forward the email to your colleague Michel Laurent to ask him if he received the merchandise. Insert a subject and copy (CC) the bookkeeper on the email. The message should say: Hello Please let me know if you remember receiving the merchandise. (your name) Question 2: You received an email from the production logistics coordinator asking you to forward an attached notice about overtime policy change to two colleagues, Eric Bernard and Gabriel Robert, and to notify them about the upcoming meeting (to discuss the new policy). You need to CC the coordinator on the email, and BCC the director general, Charles Dubois. Insert a subject and write this message: Hello Please read the attached policy on overtime hours. Note that there is a meeting about the policy on Thursday the 14 th at 12:30PM. (your name)
Question 3: You received an email from the production logistics coordinator asking you to send your colleagues, Simon Fremont and Luc Thompson, the attached notice about the water service interruption. Forward the notice as an attachment to your colleagues. Then CC the sender, and BCC the director general, Charles Dubois. Indicate that the email is urgent. Insert a subject and write this email: Hello Please read the attached notice about the water service interruption. Make arrangements to avoid a delay in production. (your name) Archive both the received and forwarded emails in the Management folder of your email application. Question 4: You received an email from the production logistics coordinator asking you to send an attached pamphlet about a new piece of production equipment. You must forward the attachment to your colleagues, Jean-Claude Petit and Yves Lefebvre, and notify them there is an information session about the machine on Wednesday at noon. Make sure you CC the email sender and BCC the director general Charles Dubois. Also, indicate the message is urgent, and archive both emails in the Management folder.
Question 1 Complexity Score: 3 (Closed/Open: 1, Defined/Not defined: 3) Task description (domains): Simple and precise instructions (1), defined outcome (1), defined technological application (1) and information (1), and defined task setting (1), routine task with definite steps. Question 2 Complexity Score: 4 (Closed/Open: 2; Defined/Not defined axis: 3) Task description (domains): Simple task with more complex instructions (2), defined outcome (1), moderately complex technological applications (2), defined pieces of digital information (2), and defined task setting (1), routine task with definite steps.
Question 3 Complexity Score: 4 (Closed/Open: 2, Defined/Not defined: 3) Task description (domains): Simple task with more complex instructions (2), less precise outcome (2), moderately complex technological applications (2), a few defined and more complex pieces of digital information (2), and defined task setting (1), less routine task with less definite steps Question 4 Complexity Score: 4 (Closed/Open: 2, Defined/Not defined axis: 3) Task description (domains): Simple task with more complex instructions (2), less precise outcome (2), moderately complex technological applications (2), a few defined and more complex pieces of digital information (2), and less defined task setting (2), less routine task with less definite steps.
Norm Leckie: firstname.lastname@example.org@srdc.org Social Research and Demonstration Corporation: www.srdc.orgwww.srdc.org