Scope
- Assessments development will also include the initial stages of design and development for the Trainee Portal
- 'Assessments' refers to the progression reviews carried out at least annually, but may be more often, and not to the workplace-based assessments that are managed on the various ePortfolios
- The assessments function will support reviews across medicine, dentistry, pharmacy and healthcare sciences
Objectives
To understand from users' perspective:
- the assessments process and how it differs between professional groups, localities, training stages and specialties
- who does what and the detail of the tasks they perform (what, why, where, when and how)
- their experience of the process - what's good and what's not
- what do they need from a system to support the process
6 local teams:
- East Midlands
- Kent, Surrey & Sussex
- North East
- South West
- Thames Valley
- Yorkshire and the Humber
(Others to be substituted if no availability with initial list - to total of 6 for Alpha)
Objectives
Methods
Contextual Enquiry
- Sitting with an administrator and observing what they do
- Discussing observations, issues and ideas as they arise
- Duration as long as the task takes but minimise disruption of work
- Feedback anonymised – issues will be aggregated and generalised in project outputs e.g. user journeys and personas
Interviews
- Structured but informal interviews to talk through processes (e.g. revalidation) and discuss any issues and ideas
- May be 1-2-1 or include more than one person working within a team (not more than 2 preferred, max 3)
- Usually last about 1 hour
- Feedback anonymised – issues will be aggregated and generalised in project outputs e.g. user journeys and personas
Workshops
- Group of colleagues – either working in similar roles or representing users from end to end of the process
- Work together to create a common view of process pain points and opportunities
- Feedback can be attributed to whole workshop and not to individuals
Model research schedule - 1-2 days with local team, depending on numbers, availability and location
Can be flexible depending on availability, but should generally start with an observation session with a validation officer/manager who actually carries out the tasks involved in administering the process. Whether we use 1-2-1 or workshops depends to a certain extent on the numbers of people available/willing to participate, but I'd suggest we stick to 1-2-1 (or 1-2-2) for the first couple of local teams at least, and move on to workshops if it feels like we're getting a consensus that can be played back to bigger groups (and as we develop assets that can be used as workshop stimulus).
| What | Who do we need at this session | Notes |
|---|---|---|
Contextual Enquiry (c. 2hrs?) | Validation officer/manager who actually undertakes the tasks required for revalidation (RV1-11) in summary:
| If these tasks are carried out by different roles, observation to be with each of these roles individually e.g. if a validation officer is responsible for preliminary checks but a separate person/team responsible for gathering and considering evidence on concerns, then we'd like to go through the part of the process with both of these people idividually. |
Depth interviews (3 x 1hr) | 2 or 3 1-2-1 interviews with other people who perform the role as above. 1-2-1 Interview with a validation manager if they are not covered in the Contextual Enquiry. | As above, need to cover different roles where these are involved (e.g. 1 from Validation team, 1 from Specialty team) |
Interview (ideally 1hr but we'll take what we can get! | 1-2-1 with Responsible Officer | This will be to understand his/her experience of the process, including audit/reporting requirements, plus wider insight into working practices (e.g. self service or supported), digital take-up etc NB - we may not be able to speak to RO in every local team, but will be happy to talk to as many as we can |
Depth interviews 1hr | 1-2-1 with trainee | Number of trainees depends on schedule - if visit is spread across 2 days this can be increased |
0 Comments