Versions Compared

Version Old Version 9 New Version Current
Changes made by

Reuben Roberts

Reuben Roberts

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Using the two-step matching process described in more detail below, 82% of FormRs can be linked with no obvious errors. This leaves ~46 000 FormRs that cannot be linked automatically.

Additional steps to link ~95% of FormRs can be added (leaving ~12 000 unlinked) with error rates estimated to be 0.02% (projected to 50 mis-linked forms).

Certain explicitly linked forms were tagged as dubious and ignored for the purposes of estimating error. These were forms linked to programme memberships that had finished more than 18 months ago, had started more than 18 months ago but were linked in a non-ARCP capacity, where the programme membership no longer exists, where (for PartA) the form start and end dates and specialty match a different programme membership to the one linked to, etc. These attempts to identify and mitigate against user-error would need to be reviewed, as would the set of linking rules applied.

ARCP / non-ARCP metrics have not yet been assessed: given our sample data excludes the peak period for ARCPs, this might need to be more strongly rule-based as opposed to modelled off existing form linkages. While all trainee-linked forms for ARCP were submitted on or after the start of the programme, so were a significant percentage of the non-ARCP forms.

Approach

FormR and TIS data as of early Oct 2024 were retrieved and used to evaluate potential methods of linking FormRs and programme memberships. Both submitted and draft forms were assessed.

...

Trainee has only one current programme membership in the time-period window of interest (from 12 months before form submission date to 4 months after form submission), with matching local office and programme details. SQL.

Matches:

FormR PartA: 62934 (52.8% of test dataset); 1788 manually linked forms

FormR PartB: 76637 (54.2% of test dataset); 2542 manually linked forms

Errors:

None detected.

C. One matching programme membership for submission local office and specialty active within 1 year before and 4 month after submission date

Rule:

Trainee has only one current programme membership with matched local office and specialty in the time-period window of interest (from 12 months before form submission date to 4 months after form submission), with matching local office and programme details. SQL.

Matches:

FormR PartA: 17156 (14.6% of test dataset); 629 manually linked forms

FormR PartB: 17886 (15.3% of test dataset); 659 manually linked forms

Errors:

27, of which 4 were not captured as trainee errors. However, inspecting these errors, at least half appear to trainee error after all:

  1. FormR B (ambiguous)

    1. ID 94f9360d-39f3-47d6-a4cf-ffdda6daf570

    2. trainee 151236

    3. submitted 27-09-2024

    4. LO North Central and East London

    5. programme specialty: ACCS Emergency Medicine

    6. linked to programme membership 8cda9360-2b22-46d9-839e-6a9df2904100 specialty Emergency Medicine.

    7. autolinked to e9829933-d83d-11ec-9eb2-0638a616fc76 specialty ACCS.

  2. FormR B (trainee linked to PM with wrong LO)

    1. ID 3439addb-29f2-46d9-933b-babb9b7f7fe1

    2. trainee 264960

    3. submitted 01-09-2024

    4. LO Health Education England North East

    5. programme specialty: Paediatrics

    6. linked to programme membership 8d815dfa-84e1-4515-a7a9-89958758b33b specialty Paediatrics but LO Yorkshire and Humber

    7. autolinked to dffa0afc-7fc3-4c8d-b4a9-25145e5a90d1 specialty Paediatrics, LO North East

  3. FormR a

    1. ID b0e02e9f-f096-4908-a12a-a9cc0c8d1dd9

    2. trainee 287773

    3. submitted 16-08-2024

    4. LO Health Education England East of England

    5. programme specialty: Internal Medicine Training Stage One (cct1: Renal medicine)

    6. linked to programme membership e26a2708-e323-4c71-9d03-7342059a592a specialty Renal medicine, LO East of England

    7. autolinked to eb31441e-d83d-11ec-9eb2-0638a616fc76 specialty Internal Medicine Training Stage 1, LO East of England

  4. FormR a (trainee linked to PM with wrong LO)

    1. ID e29c8357-b4c7-483d-af9b-5edbbad6ca35

    2. trainee 292919

    3. submitted 08-09-2024

    4. LO Yorkshire and the Humber

    5. programme specialty: Geriatric Medicine

    6. linked to programme membership cadfa7f6-79f0-41d5-8508-ffdb104e0e9e specialty Geriatric medicine, LO East of England

    7. autolinked to f11e2738-2aa3-4525-9863-f7f9d32ae898 specialty Geriatric Medicine, LO Yorkshire and the Humber