This listing comprises a breakdown of the type of TCS errors appearing in the Slack sentry channel 21 Jun 2023 - 21 Jul 2023.
The intention would be to identify which are ‘real’ errors and which can safely be ignored, so that we can update the Sentry message filters to be less noisy.
Group | Source | Title | Description | Example link | Ignore/Attend? | Comment | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | No reference found for id 0 | TCS | Exception during find grade id in for ids [0], returning empty list. Here's the error message 404 |
| Attended | 68K events They’re consuming up our quota. | ||||||||||||||||||||||||
2 | TCS | Exception during find sites id in for ids [0], returning empty list. Here's the error message 404 |
| Attended | 68K events They’re consuming up our quota. | |||||||||||||||||||||||||
3 | Person Update bulk upload time outs | TCS | HttpRequestMethodNotSupportedException | Request method 'GET' not supported url: http://apps.tis.nhs.uk/tcs/api/documentsIOException | PATCH /api/bulk-people/ Connection reset by peer | https://health-education-england-9v.sentry.io/issues/43129142834325776792/?alert_rule_id=1448641613350188&alert_type=issue&project=5752964&referrer=slack | Ignored | 14 events | 4 | TCS | NullPointerException | GET /api/placements/overlapping | https://health-education-england-9v.sentry.io/issues/4312566059/?alert | Attend | Only 1 event on 19th July 11:11:41. It’s very likely to be linked with the stalled Person Update file issue (Andrew Goodhand uploaded a file at 11:03;41)
| |||||||||||||||
4 | get draft placement list by id from PlacementLog table | TCS | IOException | Broken pipe url: http://stage-apps.tis.nhs.uk/tcs/api/placements/draftList/818 | https://health-education-england-9v.sentry.io/issues/4355344968/?alert_rule_id=1448641613350188&alert_type=issue&project=5752964&referrer=slack | Attend | dateFrom or dateTo of the placement to be validated is null. | 5 | TCS | NullPointerException | DELETE /api/placements/{id} null value in entry: delete=nullWe have to take a look this endpoint for programmeId 818.
| |||||||||||||||||||
5 | /api/placements/overlapping | TCS | NullPointerException | GET /api/placements/overlapping | https://health-education-england-9v.sentry.io/issues/43146961554312566059/?alert_rule_id=14486416&alert_type=issue&project=5752964&referrer=slack | Ignored | Attend | dateFrom or dateTo of the placement to be validated is null.
| ||||||||||||||||||||||
6 | TCS | HystrixRuntimeExceptionMethodArgumentTypeMismatchException | GET _USER_PROFILE short-circuited and no fallback availableUT005023: Exception handling request to /tcs/api/findByNationalPostNumber | Ignored | This error has been in TCS for a long time (more than 2 years) | 7 | TCS | HystrixRuntimeException GET_USER_PROFILE could not be queued for execution and no fallback available UT005023: Exception handling request to /tcs/api/placements/2409866/esr-exportedplacements/overlapping
| https://health-education-england-9v.sentry.io/issues/42654642184312901248/?alert_rule_id=14486416&alert_type=issue&project=5752964&referrer=slack | Ignored | has been marked as Resoved before. | 8 | TCS | HystrixRuntimeException | GET_USER_PROFILE timed-out and no fallback available. UT005023: Exception handling request to /tcs/api/posts | Ignored | 9 | TCS | DELETE /api/programme-memberships/{id} Failed to convert value of type 'java.lang.String' to required type 'java.lang.Long'; nested exception is java.lang.NumberFormatException: For input string: "undefined" | https:input data format can not be converted to LocalDate
| ||||||||||
7 | TCS | You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1 |
| https://health-education-england-9v.sentry.io/issues/43146898394314780639/?alert_rule_id=14486416&alert_type=issue&project=5752964&referrer=slack | Ignored | UI passes underfined as ID to the backend.Attend | 29 events | 10 | TCS | MethodArgumentTypeMismatchException | GET /api/people/v2/{id} Failed to convert value of type 'java.lang.String' to required type 'java.lang.Long'; nested exception is java.lang.NumberFormatException: For input string: "undefined"The user didn’t select an item in the npn dropdown list, so the npn sent to TCS is undefined.
| |||||||||||||||||||
8 | TCS | InvalidDataAccessResourceUsageException | could not extract ResultSet; SQL [n/a]; nested exception is org.hibernate.exception.SQLGrammarException: could not extract ResultSet | https://health-education-england-9v.sentry.io/issues/43256500604314780692/?alert_rule_id=14486416&alert_type=issue&project=5752964&referrer=slack | Ignored | 19 events | 11related to the above error | |||||||||||||||||||||||
9 | GET_USER_PROFILE hystrix runtime exception | TCS | MethodArgumentTypeMismatchExceptionHystrixRuntimeException | GET_USER_PROFILE short-circuited and no fallback available UT005023: Exception handling request to /tcs/api/findByNationalPostNumber | https:/placements/overlapping
| Ignored | input data format can not be converted to LocalDate | 12 | TCS | ObjectOptimisticLockingFailureException PUT /api/programme-memberships optimistic locking failed; nested exception is org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect)/health-education-england-9v.sentry.io/issues/4265137394/?alert_rule_id=14486416&alert_type=issue&project=5752964&referrer=slack | Ignored | This error has been in TCS for a long time (more than 2 years) | ||||||||||||||||||
10 | TCS | HystrixRuntimeException | GET_USER_PROFILE could not be queued for execution and no fallback available UT005023: Exception handling request to /tcs/api/placements/2409866/esr-exported | Ignored | has been marked as Resoved before. | |||||||||||||||||||||||||
11 | TCS | HystrixRuntimeException | GET_USER_PROFILE timed-out and no fallback available. UT005023: Exception handling request to /tcs/api/posts | Ignored | ||||||||||||||||||||||||||
12 | MethodArgumentTypeMismatchException Related to UI | TCS | MethodArgumentTypeMismatchException | DELETE /api/programme-memberships/{id} Failed to convert value of type 'java.lang.String' to required type 'java.lang.Long'; nested exception is java.lang.NumberFormatException: For input string: "undefined" | https://health-education-england-9v.sentry.io/issues/43204177694314689839/?alert_rule_id=14486416&alert_type=issue&project=5752964&referrer=slack | 5 events | 13 | TCS | ObjectOptimisticLockingFailureException | PUT /api/personal-details Object of class [com.transformuk.hee.tis.tcs.service.model.PersonalDetails] with identifier [314741]: optimistic locking failed; nested exception is org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect)Ignored | UI passes underfined as id to the backend. 29 events | |||||||||||||||||||
13 | TCS | MethodArgumentTypeMismatchException | GET /api/people/v2/{id} Failed to convert value of type 'java.lang.String' to required type 'java.lang.Long'; nested exception is java.lang.NumberFormatException: For input string: "undefined" | https://health-education-england-9v.sentry.io/issues/43205130674325650060/?alert_rule_id=14486416&alert_type=issue&project=5752964&referrer=slack | Ignored | 19 events | ||||||||||||||||||||||||
14 | ObjectOptimisticLockingFailureException | TCS | ObjectOptimisticLockingFailureException | PUT /api/contactprogramme-detailsObject of class [com.transformuk.hee.tis.tcs.service.model.ContactDetails] with identifier [301675]: optimistic locking memberships optimistic locking failed; nested exception is org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.transformuk.hee.tis.tcs.service.model.ContactDetails#301675] | https://health-education-england-9v.sentry.io/issues/43228617494320417769/?alert_rule_id=1335018814486416&alert_type=issue&project=5752964&referrer=slack | 15Ignored | TCS | Column 'positionNumber' cannot be null | Column 'positionNumber' cannot be null | Attend | positionNumber is defined not null in the database, but it seems ESR sends records with null position number to TIS. | 16 | TCS | DataIntegrityViolationException | could not execute statement; SQL [n/a]; constraint [null]14 events The current update is on a stale eneity. | |||||||||||||||
15 | TCS | ObjectOptimisticLockingFailureException | PUT /api/personal-details Object of class [com.transformuk.hee.tis.tcs.service.model.PersonalDetails] with identifier [314741]: optimistic locking failed; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statementurl: /api/posts/219720/esr-changedStaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) | https://health-education-england-9v.sentry.io/issues/43128195784320513067/?alert_rule_id=14486416&alert_type=issue&project=5752964&referrer=slack | Ignored | seems to be related to the above error | 17 | TCS | HHH000346: Error during managed flush [org.hibernate.exception.LockAcquisitionException: could not execute statement] | transaction: 5 events. The current update is on a stale eneity. | ||||||||||||||||||||
16 | TCS | ObjectOptimisticLockingFailureException | PUT /api/programmecontact-memberships | Ignored | Only 1 event | 18 | TCS | AccessDeniedException | DELETE /api/placements/{id} Access is denieddetails Object of class [com.transformuk.hee.tis.tcs.service.model.ContactDetails] with identifier [301675]: optimistic locking failed; nested exception is org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.transformuk.hee.tis.tcs.service.model.ContactDetails#301675] | Ignored | 1 event | |||||||||||||||||||
17 | DB not null constraints Related to ESR | TCS | Column 'positionNumber' cannot be null | Column 'positionNumber' cannot be null | Attend | positionNumber is defined not null in the database, but it seems ESR sends records with null position number to TIS. | ||||||||||||||||||||||||
18 | TCS | DataIntegrityViolationException | could not execute statement; SQL [n/a]; constraint [null]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement url: /api/posts/219720/esr-changed | Ignored | seems to be related to the above error | |||||||||||||||||||||||||
19 | Data issue - new/old post references the current post itself Related with ESR | TCS | HttpMessageConversionException | Type definition error: [simple type, class com.transformuk.hee.tis.tcs.service.model.Post]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct self-reference leading to cycle (through reference chain: com.transformuk.hee.tis.tcs.service.model.PlacementEsrEvent["placement"]->com.transformuk.hee.tis.tcs.service.model.Placement["post"]->com.transformuk.hee.tis.tcs.service.model.Post["newPost"]) | https://health-education-england-9v.sentry.io/issues/43151222954312522176/?alert_rule_id=14486416&alert_type=issue&project=5752964&referrer=slack | Ignored | 8 events Seems to be permission control. | 19 | TCS | Deadlock found when trying to get lock; try restarting transaction | Deadlock found when trying to get lock; try restarting transaction | Attend |
| |||||||||||||||||
20 | Deadlock when “PUT /api/programme-memberships” | TCS | HHH000346: Error during managed flush [org.hibernate.exception.LockAcquisitionException: could not execute statement] | transaction | https://health-education-england-9v.sentry.io/issues/43286507504328652261/?alert_rule_id=13350188&alert_type=issue&project=5752964&referrer=slack | Ignored | Only 1 event | |||||||||||||||||||||||
2021 | TCS | CannotAcquireLockException | could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.LockAcquisitionException: could not execute statementDeadlock found when trying to get lock; try restarting transaction | Deadlock found when trying to get lock; try restarting transaction PUT /api/programme-memberships | https://health-education-england-9v.sentry.io/issues/43286507524328650750/?alert_rule_id=13350188&alert_type=issue&project=5752964&referrer=slack | Ignored | Only 1 event seems to be related to the above error | |||||||||||||||||||||||
2122 | TCS | You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1 |
| could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.LockAcquisitionException: could not execute statement PUT /api/programme-memberships | https://health-education-england-9v.sentry.io/issues/43147806394328650752/?alert_rule_id=1448641613350188&alert_type=issue&project=5752964&referrer=slack | 13 events | 22 | TCS | InvalidDataAccessResourceUsageException | could not extract ResultSet; SQL [n/a]; nested exception is org.hibernate.exception.SQLGrammarException: could not extract ResultSetIgnored | Only 1 event seems to be related to the above error | |||||||||||||||||||
23 | Elasticsearch Exception about person list | TCS | UncategorizedElasticsearchException | GET /api/people Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]; nested exception is ElasticsearchStatusException[Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]] | https://health-education-england-9v.sentry.io/issues/43147806924325386720/?alert_rule_id=1448641613350188&alert_type=issue&project=5752964&referrer=slack | related to the above error | ||||||||||||||||||||||||
23 | TCS | IOException | POST /api/documents File is empty | Ignored | 3 events | |||||||||||||||||||||||||
24 | TCS | IOException | PATCH /api/bulk-people/ Connection reset by peer | Attend | Only 1 event on 19th July 11:11:41. It’s very likely to be linked with the stalled Person Update file issue (Andrew Goodhand uploaded a file at 11:03;41) | |||||||||||||||||||||||||
25 | TCS | IllegalArgumentException | GET /api/people/v2/{id} Source must not be null | Ignored | It’s not very clear why the shards health check failed, but we don’t always get this error, so it should be okay to ignore it. | |||||||||||||||||||||||||
24 | TCS | UncategorizedElasticsearchException | GET /api/people Unable to parse response body; nested exception is ElasticsearchStatusException[Unable to parse response body]; nested: ResponseException[method [POST], host [https://vpc-es7-4prod-qyke6r2vwvp2fno6j5iod6ffma.eu-west-2.es.amazonaws.com], URI [/persons/_search?pre_filter_shard_size=128&typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=dfs_query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 504 Gateway Time-out] | https://health-education-england-9v.sentry.io/issues/43151434394320912970/?alert_rule_id=14486416&alert_type=issue&project=5752964&referrer=slackIgnored | Attend | 2 events It seems the id user requrested doesn’t exist147 events We might want to take a look what happened. | ||||||||||||||||||||||||
25 | TCS | UncategorizedElasticsearchException | GET /api/people Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failedDataAccessResourceFailureException | 30,000 milliseconds timeout on connection http-outgoing-281 [ACTIVE]; nested exception is ElasticsearchStatusException[Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]java.lang.RuntimeException: 30,000 milliseconds timeout on connection http-outgoing-281 [ACTIVE] | https://health-education-england-9v.sentry.io/issues/43253867204325361391/?alert_rule_id=13350188&alert_type=issue&project=5752964&referrer=slack | 26 | TCS | UncategorizedElasticsearchException | GET /api/people Unable to parse response body; nested exception is ElasticsearchStatusException[Unable to parse response body]; nested: ResponseException[method [POST], host [https://vpc-es7-4prod-qyke6r2vwvp2fno6j5iod6ffma.eu-west-2.es.amazonaws.com], URI [/persons/_search?pre_filter_shard_size=128&typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=dfs_query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 504 Gateway Time-out] | Ignored | Might be hit the windows of snapshot. Only 1 event | |||||||||||||||||||
26 | /api/documents | TCS | HttpRequestMethodNotSupportedException | Request method 'GET' not supported | Ignored Won’t handle soon. | 14 events | ||||||||||||||||||||||||
27 | TCS | IOException | POST /api/documents File is empty | https://health-education-england-9v.sentry.io/issues/43209129704325883577/?alert_rule_id=1448641613350188&alert_type=issue&project=5752964&referrer=slack27 | Ignored Won’t handle soon. | 3 events | ||||||||||||||||||||||||
28 | TCS | DataAccessResourceFailureException | 30,000 milliseconds timeout on connection http-outgoing-281 [ACTIVE]; nested exception is java.lang.RuntimeException: 30,000 milliseconds timeout on connection http-outgoing-281 [ACTIVE]MultipartException | Failed to parse multipart servlet request; nested exception is java.lang.RuntimeException: java.io.IOException: UT000128: Remote peer closed connection before all data could be read | https://health-education-england-9v.sentry.io/issues/43253613914323013492/?alert_rule_id=13350188&alert_type=issue&project=5752964&referrer=slack | Ignored Might be hit the windows of snapshotWon’t handle soon. | Only 1 event28 | |||||||||||||||||||||||
29 | Date deserialization error. Related to UI. | TCS | HttpMessageNotReadableException | JSON parse error: Cannot deserialize value of type `java.time.LocalDate` from String "24-11-04": Failed to deserialize java.time.LocalDate | Ignored | Programme membership detail page only colours the incorrectly formatted date red, but still allow them to be sent to the backend. | ||||||||||||||||||||||||
3029 | Access denied | TCS | HttpMessageConversionException | Type definition error: [simple type, class com.transformuk.hee.tis.tcs.service.model.Post]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct self-reference leading to cycle (through reference chain: com.transformuk.hee.tis.tcs.service.model.PlacementEsrEvent["placement"]->com.transformuk.hee.tis.tcs.service.model.Placement["post"]->com | AccessDeniedException | DELETE /api/placements/{id} Access is denied | Ignored | 8 events Seems to be permission control. | ||||||||||||||||||||||
31 | Entity is not found | TCS | EntityNotFoundException | POST /api/programme-memberships/delete/ Unable to find com.transformuk.hee.tis.tcs.service.model.Post["newPost"])CurriculumMembership with id 404778 | https://health-education-england-9v.sentry.io/issues/43125221764323172335/?alert_rule_id=1448641613350188&alert_type=issue&project=5752964&referrer=slack | AttendIgnoredjira | Only 1 event | |||||||||||||||||||||||
| ||||||||||||||||||||||||||||||
30 | 32 | TCS | EntityNotFoundException | POST DELETE /api/programme-memberships/delete/placements/{id} | https://health-education-england-9v.sentry.io/issues/43231723354363619973/?alert_rule_id=13350188&alert_type=issue&project=5752964&referrer=slack | Ignored Won’t handle soon. | Only 1 event | TCS | MultipartException | Failed to parse multipart servlet request; nested exception is java.lang.RuntimeException: java.io.IOException: UT000128: Remote peer closed connection before all data could be read url: http://apps.tis.nhs.uk/tcs/api/documents | ||||||||||||||||||||
33 | TCS | IllegalArgumentException | GET /api/people/v2/{id} Source must not be null | https://health-education-england-9v.sentry.io/issues/43230134924315143439/?alert_rule_id=1335018814486416&alert_type=issue&project=5752964&referrer=slack | Ignored | Only 1 event | TCS | IOException | Broken pipe url: http://stage-apps.tis.nhs.uk/tcsWon’t handle soon. | 2 events It seems the id user requested doesn’t exist. | ||||||||||||||||||||
34 | TCS | NullPointerException | DELETE /api/placements/draftList/818{id} null value in entry: delete=null | https://health-education-england-9v.sentry.io/issues/43553449684314696155/?alert_rule_id=1335018814486416&alert_type=issue&project=5752964&referrer=slack | Attend | We have to take a look this endpoint for programmeId 818Ignored | 3 events, and all related placements were deleted. The exception is thrown in the |