Home
Digital Outcomes opportunities
Procurement details

Procurement details: RM1043.8-1-NHS ENGLAND - C124067 - AAC Tableau dashboard operational layer user requirements and metric development

16

Applications submitted

£140000

Indicative Budget

Outside scope

Off-payroll status

03 May 2023

Application Closing Date

1. Context and requirements

Pre-market engagement: n/a
Work done so far: User research and metrics have been developed for a strategic layer of a new dashboard and a new dashboard is being developed in Tableau. We have explored how a web wrapper for this dashboard might improve the user experience.
Which phase the project is in: Not applicable
Existing team: The performance and evaluation team. The team's experience spans healthcare analytics, statistics, data extraction, dashboard design and Tableau development
Address where the work will be done: Wellington House, 133-135 Waterloo Road, London, SE1 8UG and remotely; occasionally user research activities may take place in Leeds.
Working arrangements: We’d like this to be a call-off approach, we will work with the successful contractor to agree
Provide more information about your security requirements: Baseline Personnel Security Standard (BPSS)
Provide more information about your security requirements (optional): BPSS 1B
Latest start date: 2023-07-01
Enter the expected contract length: 6 months
Extension period: 3 months
Special terms and conditions: None
Write the term or acronym: P&E
Write the term or acronym: AAC
Write the term or acronym: AHSN
Explain the term or acronym: Performance and Evaluation team
Explain the term or acronym: Accelerated Access Collaborative
Explain the term or acronym: Academic Health Science Network
Are you prepared to show your budget details?: Yes
Indicative maximum: 140000
Indicative minimum
Provide further information: FY 23/24 £140,000 ex VAT
Confirm if you require a contracted out service or supply of resource: Contracted out service: the off-payroll rules do not apply
Summary of work: The work is being done to create an updated dashboard to better meet user needs by providing metrics to support decision-making. Although the Performance and Evaluation (P&E) team can deliver Tableau dashboards, they still need to gain all the specialist skills, knowledge, and expertise to create everything required. Therefore, the supplier will be expected to work with P&E to build these skills internally over the contract period. Phase 1: a comprehensive review of existing user engagement and research. Delivery of user research to understand requirements for an operational dashboard. This research phase must explore how users would like the dashboard to work and operate in the future and understand their needs and requirements. The research must also seek to understand user issues, concerns, problems and difficulties with the present dashboard and P&E reporting mechanisms. Phase 2: establishing and refining metrics for operational reporting, including working with stakeholders to identify data sources and plan delivery Phase 3: Working with P&E to deliver and test iterations of a Tableau dashboard and to refine and deliver updated dashboards. Identify other tools that better meet user needs.
Where the supplied staff will work: Yorkshire and the Humber
Where the supplied staff will work: London
Why the work is being done: The work is being done to create an updated dashboard to better meet user needs by providing metrics to support decision-making. Although the Performance and Evaluation (P&E) team can deliver Tableau dashboards, they still need to gain all the specialist skills, knowledge, and expertise to create everything required. Therefore, the supplier will be expected to work with P&E to build these skills internally over the contract period. The dashboard will be used to provide strategic and operational metrics to support the AAC.
The business problem you need to solve: Objectives: • Understand how users are using the dashboard at present and identify how they are working with it and any issues or concerns they have. • Understand dashboard users by what they are trying to do and their roles and responsibilities. • Develop and refine metrics to support what users are trying to do. • Deliver a new dashboard through a series of iterations. • To build the skills, knowledge, and expertise in the P&E team to deliver similar work in the future. The provider should deliver/provide the following: • Review existing user engagement and research undertaken and make recommendations for further user research • Develop a research plan for user research, and outline all elements, including user research methodology, recruitment and results analysis and evaluation. • Lead the delivery of user research working with the P&E team and supplying resources as required to support and complement existing resources. • Lead the delivery of a review of existing metrics and make recommendations for new/refined metrics to meet the user needs identified by the research. Supplying resources as required to support and complement existing resources. • Lead the delivery of dashboard development using an agile approach, supplying resources as required to support and complement existing resources. • Review other tooling options and provide an options appraisal • Named leads for the following roles: Tableau developer/developer, agile coach/scrum master, and UX designer.
First user type: 200 users of the AAC scorecard
Enter more details about this user type: There are currently 200 users of the AAC scorecard working across the AAC collaboration. This includes colleagues from NHS England, Office for Life Sciences, Academic Health Science Network, and industry partners. The dashboard is used to demonstrate the performance of the AACs implementation activity and to drive the uptake of proven innovations in the NHS. As an AAC Board member/OLS Policy Lead, I need to understand what the AAC has delivered so that I can assess its impact and value for money. As an industry partner, I need to understand how the implementation of my technology is progressing at a national level so that I can assess impact and performance. As an AAC deputy director, I need to see what my programme of work has delivered at a strategic level so that I can assess its impact. As an AAC programme lead, I need to see what performance looks like at the AHSN level so that I can manage performance. As an AHSN programme lead, I need to see granular performance data so that I can manage performance at a local level. We require the supplier to include in their response how they will deliver the following: Phase 1: • Review user engagement and research undertaken so far • Recommendations for further user research • Plan, manage, deliver, and conduct the user research recommended in phase 1 Phase 2: • Review existing metrics and the development of new/refined metrics that meet the user needs identified in the research. • Recommendations for new/refined metrics • Plan, manage, deliver, and conduct the metrics review Phase 3: • Delivery of a new operational dashboard • Support the team to move towards agile working • Plan, manage and deliver dashboards Provide CVs for the individuals who will be the named leads for the following roles: • User researcher • Tableau developer/developer • Agile coach/scrum master • UX designer • Two sides of A4 per CV, font size 12 • Details of the team and structure and operating model and working practices

2. Assessment Criteria
Select your pricing model: Time and materials
Additional assessment methods: Presentation
Additional assessment methods: Reference
Additional assessment methods: Work history
Describe the question and answer session: None, all Q&A's to be managed through the portal
Create your scoring criteria: 3
How many suppliers to evaluate: 3
Technical competence: 60
Cultural fit: 10
Social value: 10
Price: 20
Essential skills and experience: 40
Nice-to-have skills and experience (optional): 10
Technical questions: 50
Describe the essential skill or experience:: Detail your experience of delivering user research against the Government Service Standard
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Detail your experience of delivering strong quantitative and qualitative user research
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Detail your experience in developing a variety of user journeys
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Detail your experience of leading projects to develop the vision, scope, and roadmap
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Detail your experience of delivering strong stakeholder management skills
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Detail your experience of participant recruitment and carrying out contextual research
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Detail your experience of data and analytics in the NHS or healthcare
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Detail your experience of developing metrics using NHS data to meet stakeholder needs
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Detail your experience of developing dashboards in healthcare
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Detail you experience of developing logic models and benefits management
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the nice-to-have skill or experience:: Detail your experience of working collaboratively, sharing knowledge and experience
Enter a weighting for this skill or experience in whole numbers, for example 30: 20
Describe the nice-to-have skill or experience:: Provide details of your experience of leading in project governance and delivery
Enter a weighting for this skill or experience in whole numbers, for example 30: 40
Describe the nice-to-have skill or experience:: Provide details of your experience of engaging and working with a range of staff groups
Enter a weighting for this skill or experience in whole numbers, for example 30: 40
Explain the technical question:: Detail an appropriate discovery approach and methodology for this project
Enter a weighting for this technical question in whole numbers, for example 30: 12
Explain the technical question:: Detail your metric development approach and methodology for this project
Enter a weighting for this technical question in whole numbers, for example 30: 12
Explain the technical question:: Detail how you will approach dashboard development
Enter a weighting for this technical question in whole numbers, for example 30: 12
Explain the technical question:: Detail how these approaches would meet our requirements
Enter a weighting for this technical question in whole numbers, for example 30: 12
Explain the technical question:: Please provide estimated timeframes, including milestones, for this work
Enter a weighting for this technical question in whole numbers, for example 30: 12
Explain the technical question:: Detail how you will identify risks and dependencies, and establish mitigations
Enter a weighting for this technical question in whole numbers, for example 30: 12
Explain the technical question:: Provide a clear research plan of how you will deliver each phase of this work
Enter a weighting for this technical question in whole numbers, for example 30: 12
Explain the technical question:: Provide the team structure, and details of the personnel to be involved, including their role for this project and their relevant experience
Enter a weighting for this technical question in whole numbers, for example 30: 16
Your question: Detail and provide evidence of how you will show data evidence for insights and recommendations
Enter a weighting for this question: 20
Your question: Detail and provide evidence your ability to share knowledge and experience and communicate recommendations with non-technical groups
Enter a weighting for this question: 20
Your question: Detail your experience of advocating for agile practices, user centred and inclusive design
Enter a weighting for this question: 16
Your question: Detail how you will provide transparency and collaboration in working practices
Enter a weighting for this question: 16
Your question: Detail how you will work as a team with our organisation
Enter a weighting for this question: 6
Your question: Detail how you will deliver a no-blame culture and encourage people to learn from their mistakes
Enter a weighting for this question: 6
Your question: Detail how you will work with clients with low technical expertise
Enter a weighting for this question: 16
Explain the social value question: Detail what measures you have to support-in-work progression to help people in your workforce move into higher-paid work by developing new skills relevant to the contract.
Enter a weighting for this question: 50
Explain the social value question: Detail how you plan to reconnect people with the environment and increase awareness of ways to protect and enhance it.
Enter a weighting for this question: 50

3. Timeline
Publication of stage 1: April 14, 2023, 4:16 PM
Clarification period closes: April 21, 2023, 12:00 PM
Deadline for suppliers to submit their stage 1 responses: May 3, 2023, 12:00 PM

Clarification questions and responses

There are 43 questions and clarifications relating to this opportunity.

1. It stated the place of work will be London and remote, is there the expectation that we will place people physically in a London office or will the work be predominantly remote with the occasion to travel to their London or Leeds office if required? It would be useful to understand what proportion of the work will require a physical presence at their offices.

The majority of work will be remote, but some user research might benefit from face-to-face meetings or workshops.

Last Updated: 21 April 2023, 14:56

2. Technical Does the data for all the dashboards sit in the same server/database?

Processed data sits on one server, but the data are collected from a wide variety of sources and require varying levels of processing.

Last Updated: 21 April 2023, 14:55

3. Technical As an improved of series of dashboards are expected, do you see any opportunity to merge any of them?

This depends on user needs

Last Updated: 21 April 2023, 14:53

4. Technical Do you expect the complete process to be automated or manual intervention is permitted?

The ETL process has been developed with the aim of limiting manual intervention.

Last Updated: 21 April 2023, 14:52

5. Resourcing To be able to provide you with the right team and resourcing we typically look through the breadth of our network for the right people - are you open to nearshore or offshore resourincing to support this work?

We have previously worked with suppliers based in different time zones. Our core working hours are Monday to Friday, 9am to 5pm.

Last Updated: 21 April 2023, 14:52

6. Platform Should the improved and evolved dasboards be compatibale with mobile/other types of devices other than desktop?

Most users accessed the previous dashboard on a laptop, but how users would like to access the new dashboard could be explored.

Last Updated: 21 April 2023, 14:51

7. Technical Are there technical documents on the existing platform, which will be helful to understand the architecture and come up with a TO-BE state?

The performance and evaluation will walk the supplier through the current ETL process. The SOP is being refined at the moment.

Last Updated: 21 April 2023, 14:50

8. Technical Does the data sit on premise or on the cloud?

Held on an NHS England server but moving to the cloud.

Last Updated: 21 April 2023, 14:49

9. Technical Are there any known performance issues that also need to be improved or considered as part of the dashbaord development?

The dashboard has not yet been built.

Last Updated: 21 April 2023, 14:48

10. Research When it comes to user research and testing of the dashboard we would typically propose a sampling or super user subset - does this appraoch allign with what you've done to date?

This is like the approach taken for the strategic layer.

Last Updated: 21 April 2023, 14:47

11. Technical What is the approximate size of data and the tableau file?

The operational layer has not been built.

Last Updated: 21 April 2023, 14:46

12. Is it a reasonable assumption that vendor is not required to work on any ETL activities and a data model with logical and physical tables exists for the purposes of dashboard creation?

The ETL model has been built.

Last Updated: 21 April 2023, 14:44

13. For the development of new metrics green field, is all of the data available. Or is the sourcing and quality review of this included in the scope?

Not all the data is available, and a gap analysis will be needed. This would include identifying potential data sources or suggesting data that could be collected by programmes.

Last Updated: 21 April 2023, 14:42

14. Are you looking for formal training collateral or just side by side / on the job capability build?

One-the-job training with side-by-side sessions. The team have recently completed training on Tableau and dashboard development and design.

Last Updated: 21 April 2023, 14:41

15. Can you confirm the types of skills you are looking to build capability in and how do you plan to measure this?

Dashboard development and design, user research and using an agile approach. Feedback from the team.

Last Updated: 21 April 2023, 14:35

16. The proposal is based on Tableau dashboards. However there is a line item ‘Review other tooling options and provide an options appraisal’ please could you provide more information about what you are expecting from this?

Dashboard software that might better meet user needs. For example, to create an analytics portal.

Last Updated: 21 April 2023, 14:34

17. Does the £140,000 cover research expenses?

The budget covers all suggested phases of the work.

Last Updated: 21 April 2023, 14:33

18. As per the clarification questions we understand the £140,000 is to cover X resource over a 6 month period. If a 3 month extension is needed is there additional budget for that?

The 3-month extension is to allow for slippage in delivery. The budget is fixed.

Last Updated: 21 April 2023, 14:31

19. Location states Yorkshire and Humber – can we qualify requirement and expectation to travel/work on site in these locations?

Much of the work could be completed remotely, but some user research might benefit from face to face meetings and this might involve travel to NHS England offices in Leeds, Quarry House LS2 7PD.

Last Updated: 21 April 2023, 14:30

20. Is it possible to clarify the various user categories who will be using the finished dashboards as well as the total counts of each?

AAC Board members, Deputy Directors, Programme Leads, AHSN programme leads, other arm’s length bodies and industry partners. There are around 200 members with access to the current dashboard.

Last Updated: 21 April 2023, 14:29

21. For the following question: Detail you experience of developing logic models and benefits management Is the buyer referring to technical logic models underpinning the application or higher level 'Project logic models'

Logic models to identify outcomes and measurable benefits from programmes of work.

Last Updated: 21 April 2023, 14:28

22. Please could you confirm that a performant tableau server is already in place, and will include the provision of all tableau licences required by the delivery team?

All Tableau licences are in place, and NHS England as a Tableau server.

Last Updated: 20 April 2023, 13:24

23. Please could you confirm whether or not you'd be open to a proposal with compressed delivery timelines (compared to those initially envisioned)?

Yes, but not ideal since NHS England is going through a change programme, and this could limit the availability of internal resources.

Last Updated: 20 April 2023, 13:23

24. Will any additional detail be shared through the course of the procurement exercise?

No.

Last Updated: 20 April 2023, 13:18

25. Is the required governance in place to enable rapid review and feedback of metric suggestions and refinements?

The P&E team will review the metrics with the supplier. A working group should review metrics with deputy directors and/or operational leads.

Last Updated: 20 April 2023, 13:16

26. How much work has been completed to date for the to be defined metrics?

The strategic layer of the dashboard has been built, and a layered SQL database has been developed to process data for reporting. User research was completed to develop the strategic layer.

Last Updated: 20 April 2023, 13:14

27. Please could you confirm whether the granular performance data is anticipated to either include or require processing of any PID or Patient Level Data?

The performance and evaluation team will process all data. Any data supplied will be aggregated, and with no patient identifiers.

Last Updated: 20 April 2023, 13:13

28. Are there areas of the required technical skillsets where you may require comparatively more support due to the maturity of your internal capability?

Dashboard development and user research experience design.

Last Updated: 20 April 2023, 13:11

29. The documentation states that the supplier will be expected to work with P&E to develop the capabilities required for dashboard development internally over the contract period. What level of resource would the P&E team commit to deliver this project, over what period?

The team has been fully trained in Tableau and dashboard development and will need some support to apply their knowledge. The team need more knowledge of user experience design.

Last Updated: 20 April 2023, 13:10

30. It specifies on the advert that the engagement is 6 months on one line and 3 months on the other, just checking whether the £140,000 budget is for 9 months for 3 people? If this is not the case, please can you clarify the budget?

The total budget is for the whole project with the 3 month extension to allow for slippage.

Last Updated: 20 April 2023, 13:08

31. We are not able to find this opp on portal. Can you please assist us on this please?

Please contact Crown Commercial Services Customer Services for assistance on using the portal. Email: info@crowncommercial.gov.uk Telephone: 0345 410 2222

Last Updated: 19 April 2023, 09:00

32. Does the dashboard need to be Tableau or are NHS England open minded about exploring alternative technologies that might better fulfil the requirements?

This is the preferred tool since the team is trained in Tableau.

Last Updated: 19 April 2023, 08:58

33. How was the budget determined for this opportunity? Our interpretation from the requirements documents is that you are seeking a supplier to run the three listed phases over a period of 6 months. Our initial appraisal suggested that the budget might be feasible to run stage 1 and perhaps also incorporate some of stage 2, but scaling the team up to deliver the new operational dashboard would we believe require significantly more budget.

The performance and evaluation team will be available to support delivery. The budget is based on the delivery of the strategic layer of the dashboard. In addition, the project will benefit from work already completed during the development of the strategic layer of the dashboard. Tableau dashboard development and design should be led by the supplier, but the Performance and Evaluation team will develop the dashboards.

Last Updated: 19 April 2023, 08:57

34. Are there any specific expectations or requirements for the Agile working model, UX design, and Tableau development?

Only to the degree that the team's skills, knowledge and experience should be developed to help them deliver similar work in the future. The team are currently using DevOps, has some experience with agile methods, has been trained in dashboard development, and all trained to an advanced level on Tableau.

Last Updated: 19 April 2023, 08:56

35. What is the intended scalability of the dashboard, and are there any plans for potential expansion or changes in the future?

Live dashboards will be supported by the performance and evaluation team, and refinement will be driven by programme needs. However, it’s envisaged this will not be more frequent than quarterly. This will flex, however, if programme priorities change. It is expected that around 5 to 10 sub-metric dashboards would be required. Further dashboards will only be entertained once new programmes of work have been properly defined and data flows established and tested.

Last Updated: 19 April 2023, 08:55

36. What is the expected timeline for each phase of the project, and are there any milestones or deadlines that must be met?

User research completed in the first month to six weeks (including a report), metric and MVP development over the first 2 months, and full prototyping to a live dashboard development over the remaining term of the contract.

Last Updated: 19 April 2023, 08:54

37. Are there any existing methodologies, frameworks, or tools that must be used for the user research and engagement process?

Generally, we follow the Government Service Manual for user research. For our design thinking, we use the understand, define, develop, refine and support as a framework for dashboard development.

Last Updated: 19 April 2023, 08:53

38. What are the user roles, permissions, and accessibility requirements for the dashboard?

This will need to be defined as part of the user research

Last Updated: 19 April 2023, 08:52

39. What are the data sources that the new dashboard will pull from, and are there any specific data integration or security requirements?

Routinely collected healthcare data, national statistics, sales data and data collected as part of delivery that are consolidated into a single SQL database structure for AAC reporting. This includes quantitative and qualitative data. Data processing is handled by the operational team and a separate project is being developed to create an innovation data repository. A further project is considering data harmonisation and the interoperability of AHSN systems.

Last Updated: 19 April 2023, 08:17

40. Are there any non-negotiable features/functionalities that need to be incorporated into the updated dashboard?

Access to the dashboard must be tailored to different stakeholder groups, including outside of NHS England in some cases. Commercially sensitive information must be restricted.

Last Updated: 19 April 2023, 08:16

41. Who are the key stakeholders and decision-makers involved in this project, and what are their expectations from the new dashboard?

Members of the AAC Board, director, deputy directors, programme leads, and AHSN delivery leads. Key decision makers are the deputy director for the AAC and the Director. High-level expectations are to have easily accessible data to inform the monitoring of performance and to meet reporting needs. The dashboard must be easy to use, visually appealing, and the content easily transferred to PowerPoint slides.

Last Updated: 19 April 2023, 08:15

42. Are there any specific project pain points or limitations with the current dashboard that the updated dashboard must address?

The current dashboard only provides strategic information. Data for reporting is provided using Excel workbooks as operational dashboards do not currently exist for all core programmes. Current issues include inconsistent reporting standards, manual reporting, and the inability for programmes to drill down through the data without data analyst support.

Last Updated: 19 April 2023, 08:14

43. What are the specific project goals and objectives of the updated dashboard in comparison to the existing one, and how will its success be measured?

Authority Response: To create a series of consistent, programme-specific operational dashboards that meet user needs, enable operational teams to track the performance of their programmes, identify immediate areas of need and improvement, monitor progress towards measurable benefits, and provide data for reporting to programme boards. The current version of the dashboard only gives strategic information to the Accelerated Access Collaborative’s sponsoring group. Success will be measured based on user feedback/ satisfaction, a reduction in ad-hoc requests for available data, and the number of times the dashboard is accessed by different user groups. The overall benefit will be a reduction in the time needed to generate regular reporting or the provision of data for the AAC Board.

Last Updated: 19 April 2023, 08:12

Help

You can contact us by email, phone or using the enquiry form (opens in a new tab) .

Email:

info@crowncommercial.gov.uk

Telephone:

0345 410 2222

CCS customer services team is available Monday to Friday, 9am to 5pm.

Cookies on Contract Award Service (CAS)