Evaluation of assessment of tools to enable remote usability testing for the TIS project. Following options are all Cloud-based SaaS (Software as a Service)
NB. Features analysis based on published information
| Software | Description | Minimum system requirements (for desktop testing) | Type | Licence | Indicative cost | Government Digital Marketplace | Devices | Formats | Remote access | Moderated testing? (Live interaction with tester) | Records screen | Records user - video | Records user - audio | Written survey responses | Activity tracking | Reporting | Comments |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
UserZoom | Specialist cloud-based software that enables remote user testing in a 'virtual lab'
| IE, Chrome, Firefox (versions not specified) Requires users (research participants) to install browser add-on Microphone (essential) and webcam (desirable) required on user's PC | Enterprise SaaS | Standard public sector:
| £24,140 pa (1 year minimum contract) | Yes https://www.digitalmarketplace.service.gov.uk/g-cloud/services/763266489702926
| Desktop, tablet, mobile | Websites, mobile websites, apps, prototypes | Yes | Yes | Yes | Yes | Yes | Heat maps & clickstreams | Automated dashboards Data export into Office Suite | Most widely used Only one I can find on G-Cloud (Govt digital marketplace) | |
Validately | User testing and research software platform - partnered with JustInMind prototyping tool which is being used in TIS UI design
| Google Chrome v51+ Windows 7 and above Uses a Chrome app, which automatically downloads and installs at the beginning of a test
Microphone (essential) and webcam (desirable) required on user's PC
| SaaS | Professional:
| $199 / month
(Basic licence available for $79/month but this does not induce moderated testing) | No | Desktop, tablet, mobile | Live websites, mock HTML, images, prototypes (Invision, Azure, Market, JustInMind | Yes | Yes | Yes | Yes | Yes | Yes | Path reports (which pages users viewed) Average time on page Page exits | Automated reporting Charts, filtering CSV download | |
| TryMyUI | Remote usability testing service | Browser requirements not specified on website (and couldn't fine via Google search either) Users must download a recording app to enable them to feedback and a test file for each new test they take part in Microphone (essential) and webcam (desirable) required on user's PC | SaaS or Enterprise SaaS | Team: 10 tests a month 10 test results "from your own users" (i.e. not ones they've recruited) Multi user login for sharing research 10-min highlight video reels of results Agency/Enterprise 100 tests a month Unlimited test results "from your own users" Multi user login for sharing research 20-min highlight video reels of results One-click report generation with integrated video playback | Team $299/month Agency/Enterprise price on application
| No | Desktop, tablet, mobile | Yes | No | Yes | Yes | Yes | Yes - up to 4 questions
Built-in questionnaire - System Usability Scale (SUS) and Single Ease Question) incorporated | Screen capture of user interactions - clicks, taps, swipes
| Automated reporting Task duration and completion rate info (But clicks etc only visible on screen recording?)
| Seems to be more focussed on running tests on clients behalf (incl. recruiting panels) than a truly DIY tool.
Appears to be quite structured and incorporates standard questions - allowing us to ask 4 additional questions. Doesn't appear to be 'self-service' - they are running the tests and editing the feedback? So doesn't seem as flexible as other tools? | |
| Userlytics | Remote usability testing service | IE 8 +, Mozilla Firefox 4.x +, Apple Safari 4.x +, Google Chrome 4.x + Users must download an app which checks they have the necessary equipment (e.g. webcam) and takes them through the test tasks Microphone (essential) and webcam (desirable) required on user's PC | Pro Plan
| $13,900 pa Estimate based on 24 tests (2 per month) with 20 participants each | No | Desktop, tablet, mobile | Websites, prototypes (incl. wireframes, static images), videos, mobile apps, display ads, search and social behaviour | No | Yes | Yes | Yes | Yes - up to 100 questions | Captures screen interactions (computer and mobile devices) | Not automatic - get video and annotations but need to request analysis and executive summary of the results | As above, seems to be more focussed on running tests on clients behalf (incl. recruiting panels) than a truly DIY tool. |
Add Comment