Home
Digital Outcomes opportunities
Procurement details

Procurement details: RM1043.8-1-Defence AI Centre Agile Delivery Partner

13

Applications submitted

£25000000

Indicative Budget

Outside scope

Off-payroll status

06 April 2023

Application Closing Date

1. Context and requirements

About adding context and requirements: test
Work done so far: DAIC-X was established in 2021, and in partnership with the successful bidder on another DOS contract (https://www.digitalmarketplace.service.gov.uk/digital-outcomes-and-specialists/opportunities/15410) has delivered multiple AI and AS concepts. This has established ways of working, infrastructure and processes which will be used and built on for this opportunity. IP of previous work is owned by MOD so can be re-used for future concepts.
Which phase the project is in: Not applicable
Existing team: Throughout concept delivery, the supplier will be working with a mixed team of MOD stakeholders, Dstl specialists and other contracted partners delivering DAIC-X. The principal engagement on this task will be with: DAIC-X lead for AI Concept Development & Experimentation DAIC-Ops leads for product development Concept product owners DAIC-X concept technical partners There will also be significant engagement with the wider DAIC team in Dstl, Defence Digital and other parts of MOD.
Address where the work will be done: Dstl Porton Down (SP4 0JQ), Dstl Portsdown West (PO17 6AD), with regular travel to stakeholders around the UK, and remote working opportunities. Some international travel may be required.
Working arrangements: We would like the supplier to become a partner integrated into DAIC. This will requires some presence at MOD sites, though remote working is encouraged where feasible. Working at MOD/Dstl sites will be required for classified elements of the work, and there will be a need to travel to conduct research and development activities at the user's locations. This may include international travel to support International Research Collaboration meetings and exercises. Reasonable expenses may be claimed in line with Dstl's expenses policy and will require prior approval.
Provide more information about your security requirements: Security Check (SC)
Provide more information about your security requirements: Developed Vetting (DV)
Provide more information about your security requirements (optional): Security Check level of clearance is required for majority of team. Some personnel with DV clearance would be highly advantageous. Some elements of the work will require UK nationals only (no dual nationals or non-UK nationals). The supplier is expected to process and hold their own clearances. Clearances can be managed by 3rd parties. The authority will not hold or sponsor clearances.
Latest start date: 2023-08-01
Enter the expected contract length: 2 years
Extension period: 1 year
Special terms and conditions: DEFCON 703 shall apply throughout.
Are you prepared to show your budget details?: Yes
Indicative maximum: 25000000
Indicative minimum: 1200000
Provide further information: Total Limit of Liability is £25M (ex VAT), evenly spread over FY23/24 and FY24/25. This contract provides Dstl the ability to place up to £25M worth of work with the chosen supplier. This amount however is not guaranteed to be spent. Within this overarching contract, Dstl wishes to be able to agree individual tasks with the supplier, which will set out the specific deliverables, team composition and expected costs. The initial task to be let under this contract is for the first concept development task in FY23/24 and has a budget of £1.2M. We will be evaluating the price based on this first task, with supplier responses to this forming the rate card for the duration of the contract so the hourly/day rates will remain firm for the duration of the contract. There could be an option to increase in value to a maximum Limit of Liability of £45M total. This could be in conjunction to an extension to the duration of the contract.
Confirm if you require a contracted out service or supply of resource: Contracted out service: the off-payroll rules do not apply
Summary of work: The Defence AI Centre (DAIC) requires an Agile Delivery Partner to provide multi-disciplinary teams at all stages of the agile life-cycle (https://www.gov.uk/service-manual/agile-delivery) to develop novel AI concepts to meet the needs of our users.
Where the supplied staff will work: South East England
Where the supplied staff will work: No specific location (for example they can work remotely)
Why the work is being done: AI can be defined as a family of general-purpose technologies, any of which may enable machines to perform tasks normally requiring human or biological intelligence, especially when the machines learn from data how to do those tasks. As the National AI Strategy notes, these technologies have huge potential to rewrite the rules of whole industries, drive substantial economic growth and transform all areas of life. Harnessing the potential of AI will be essential to Defence modernisation, enhancing capability across the whole organisation and enabling us to maintain strategic advantage. MOD has established the Defence AI Centre (DAIC) to catalyse the integration and adoption of these critical and ubiquitous capabilities at near-development speed for enduring technological superiority. This opportunity is for two DAIC components: the Defence AI Centre Experimentation hub (DAIC-X) and Defence AI Centre Operations (DAIC Ops). Led by Dstl, DAIC-X accelerates the adoption of the next generation of AI and AS technologies for Defence, including through the conduct of rapid AI and AS concept development and experimentation. DAIC Ops, led by Defence Digital and DE&S, integrates and scales AI and AS systems into operational use.
The business problem you need to solve: DAIC requires an Agile Delivery Partner to provide multi-disciplinary teams at all stages of the agile life-cycle (https://www.gov.uk/service-manual/agile-delivery) to develop novel AI concepts to meet the needs of our users. These teams will have experience and expertise in developing AI components of digital and physical systems, and deploying these onto platforms e.g. as a cloud-hosted application, or onto a physical system. We would be very keen to receive bids from partnered suppliers who can collectively provide the required capabilities, including non-traditional defence suppliers and small and medium-sized enterprises.
First user type: military user
First user type: back-office user
First user type: any user of AI in Defence
Enter more details about this user type: As a military user, I need to be able to develop and exploit AI to augment my ability to analyse and gain insight from large volumes of diverse data, so I can improve situational awareness and enhance decision making across a wide variety of military tasks. As a military user, I need to be able to exploit the benefits of increasingly autonomous systems, so that I can deliver the operational outputs required of me in a range of military tasks.
Enter more details about this user type: As a back-office user, I need to be able to exploit the benefits of AI so I can deliver a wide range of business tasks more efficiently and effectively.
Enter more details about this user type: As any user of AI in Defence, I need to be confident that AI-enabled capability is delivered and used in a safe and responsible manner, as per to MOD's approach to the delivery of AI-enabled capability in Defence (https://www.gov.uk/government/publications/ambitious-safe-responsible-our-approach-to-the-delivery-of-ai-enabled-capability-in-defence)

2. Assessment Criteria
Select your pricing model: Capped time and materials
Additional assessment methods: Presentation
Create your scoring criteria: 3
How many suppliers to evaluate: 5
Technical competence: 60
Cultural fit: 10
Social value: 10
Price: 20
Essential skills and experience: 10
Nice-to-have skills and experience (optional): 5
Technical questions: 85
Describe the essential skill or experience:: Demonstrable experience of delivering digital services through discovery, alpha and beta stages, in an agile, user centred way, utilising GDS standards.
Enter a weighting for this skill or experience in whole numbers, for example 30: 15
Describe the essential skill or experience:: Demonstrable experience in the past 2 years of successfully applying AI techniques to heterogeneous data to deliver benefits to users.
Enter a weighting for this skill or experience in whole numbers, for example 30: 15
Describe the essential skill or experience:: Demonstrable experience in the past 2 years of the use of open source data science and machine learning technologies.
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Demonstrable experience in the past 2 years of machine learning operations (MLOps) or similar approaches to deliver live AI-enabled services.
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Demonstrable experience of providing quality assurance of AI products (verification and validation, test and evaluation).
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Demonstrable experience of delivering AI services in accordance with MOD's ethical principles for AI in Defence (https://www.gov.uk/government/publications/ambitious-safe-responsible-our-approach-to-the-delivery-of-ai-enabled-capability-in-defence), or a strategy to do so.
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Demonstrable experience of advising and supporting development of AI development infrastructure and architectures.
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Demonstrable experience of applying good practice and standards to generate reproducible work, and effectively recording project artefacts, including: Discovery outputs, Alpha developments, and handovers of User Centred Design activity across evolving delivery teams.
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the essential skill or experience:: Demonstrable experience of managing multiple multidisciplinary teams.
Enter a weighting for this skill or experience in whole numbers, for example 30: 10
Describe the nice-to-have skill or experience:: Demonstrable experience of delivering AI components of Robotic and Autonomous (physical) systems.
Enter a weighting for this skill or experience in whole numbers, for example 30: 20
Describe the nice-to-have skill or experience:: Demonstrable experience of delivering AI components of Intelligent Digital (virtual) Systems.
Enter a weighting for this skill or experience in whole numbers, for example 30: 20
Describe the nice-to-have skill or experience:: Demonstrable experience of military environments and working with uniformed personnel.
Enter a weighting for this skill or experience in whole numbers, for example 30: 15
Describe the nice-to-have skill or experience:: Demonstrable access to military experience to support concept development.
Enter a weighting for this skill or experience in whole numbers, for example 30: 15
Describe the nice-to-have skill or experience:: Demonstrable of knowledge and experience of MOD accreditation processes.
Enter a weighting for this skill or experience in whole numbers, for example 30: 15
Describe the nice-to-have skill or experience:: Demonstrable experience of supporting customer organisations enable the widespread adoption of AI.
Enter a weighting for this skill or experience in whole numbers, for example 30: 15
Explain the technical question:: How the supplier's approach or solution meets our goal.
Enter a weighting for this technical question in whole numbers, for example 30: 30
Explain the technical question:: Evidence AI-enabled services that have been developed and deployed to end-users, the approaches adopted and the issues encountered and resolved.
Enter a weighting for this technical question in whole numbers, for example 30: 25
Explain the technical question:: Evidence on the diversity of AI-enabled services delivered, supporting Intelligent Digital Systems (virtual) and Robotic and Autonomous Systems (physical) solutions.
Enter a weighting for this technical question in whole numbers, for example 30: 25
Explain the technical question:: How the supplier has identified risks and dependencies and offered approaches to manage them.
Enter a weighting for this technical question in whole numbers, for example 30: 10
Explain the technical question:: Team structure and resourcing models, including experience of working alongside clients in co-located, integrated teams.
Enter a weighting for this technical question in whole numbers, for example 30: 10
Your question: Demonstrable experience of focussing on delivery and making a difference for users.
Enter a weighting for this question: 20
Your question: Demonstrable experience of creating effective supporting governance to nurture agile team working and collaboration and learning across multiple teams.
Enter a weighting for this question: 20
Your question: Demonstrable experience of working collaboratively as 'one team' with client and other suppliers, or strategies that will be used.
Enter a weighting for this question: 15
Your question: Demonstrable experience of creating a no-blame culture with encouragement for people to learn from their mistakes.
Enter a weighting for this question: 15
Your question: Demonstrable experience of taking responsibility and embracing accountability for their work, and respond to feedback.
Enter a weighting for this question: 15
Your question: Demonstrable experience of sharing knowledge and experience with other team members.
Enter a weighting for this question: 15
Explain the social value question: Social value theme - Tackling economic inequality. Describe the commitment your organisation will make to ensure that opportunities under the contract demonstrate collaboration throughout the supply chain, and a fair and responsible approach to working with supply chain partners in delivery of the contract. These include but are not limited to; understanding of opportunities to drive greater collaboration in the supply chain and measures to ensure supply chain relationships relating to the contract will be collaborative, fair and responsible.
Enter a weighting for this question: 50
Explain the social value question: Social value theme - Equal opportunity. Describe the commitment your organisation will make to ensure that opportunities under the contract demonstrate action to identify and tackle inequality in employment, skills and pay in the contract workforce and support in-work progression to help people, including those from disadvantaged or minority groups, to move into higher paid work by developing new skills relevant to the contract.
Enter a weighting for this question: 50

3. Timeline
Publication of stage 1: February 27, 2023, 12:00 AM
Clarification period closes: March 3, 2023, 5:00 PM
Deadline for suppliers to submit their stage 1 responses: March 13, 2023, 5:00 PM

Clarification questions and responses

There are 13 questions and clarifications relating to this opportunity.

1. With reference to Nice-to-Have Skills and Experience question 2, please may the Authority confirm the meaning of intelligent Digital (virtual) Systems, as a combination of 1) intelligent, virtual assistants (i.e. Siri), 2) synthetic environments/digital twins, 3) augmented reality (i.e. Metaverse), or is there a specific area/focus that you had in mind?

Intelligent Digital Systems refers to those systems which utilise AI in a virtual i.e. non-physical setting.

Last Updated: 7 March 2023, 16:33

2. 1. Do you want rate cards for different AI / software development roles? 2. is there an incumbent on this tender?

1. Yes, this will be required as part of the Stage 2 process. It is not required for this current Stage 1. 2. There is currently a supplier providing services in this role. This procurement is for continuation of these services in a similar role once that contract comes to a close.

Last Updated: 7 March 2023, 16:31

3. Is that correct the answer for each question need to be be max of 750 characters?

Each response should not exceed 750 characters in length (the equivalent of about 150 words), this includes spaces. You will not be excluded if you go over this count, but we will only evaluate the first 750 characters of the response.

Last Updated: 7 March 2023, 16:29

4. Can the Authority please confirm that incumbent suppliers, and those associated with the delivery of the current contract, are appropriately constrained under a ‘Conflict of Interest’ agreement such that they cannot gain competitive advantage before and during this procurement, thus ensuring a level playing field for the competition?

The set procurement process is being followed so as to ensure fair and open competition. No one supplier is being given any unfair advantage over any other supplier and all are being treated as on an open playing field. All bids received via this competitive process will be assessed using the technical and commercial criteria as stated in the requirement. We confirm that the incumbent supplier has not been involved or consulted in formulating this requirement.

Last Updated: 7 March 2023, 16:25

5. 1. From the brief is not entirely clear whether the existing concepts are to be developed further or new concepts are required to be discovered. Could the authority clarify whether this assignment requires resources across Government Service Standard delivery lifecycle from discoveries to live or specifically more towards discovery projects which are then delivered by the DAIC operational teams? 2. Will the supplier have access to the initial discovery material / output?

1. This requirement covers the full GSS delivery life cycle from discovery to live. It will include both developing existing concepts further, and conducting discovery and possible delivery to live of new potential concepts. 2. Yes, the supplier will have access to any relevant initial discovery material for the concepts.

Last Updated: 7 March 2023, 16:21

6. How will the data be delivered for the training of the ML models?

Data requirements will be identified as part of discovery activities.

Last Updated: 7 March 2023, 16:19

7. 1. Is MLOps - the deployment of the ML models - in scope of the contract or is there already a dedicated MLOps team that perform deployment? 2. Will the team be working with data classified mostly at O-S and is there any requirement for S? 3. What is the estimated percentage of onsite/remote working?

1. MLOps is in scope of the contract. 2. There will be a requirement for working at S. This will be concept dependent, i.e. not all concepts will require work at S. 3. Estimated 20% on site working, though this will vary depending on the concept.

Last Updated: 7 March 2023, 16:16

8. 1. Would it be possible to confirm who is evaluating first and second stage DOS responses? 2. Can you define what is meant by Physical vs Virtual systems in Nice to have questions 1 & 2? (and whether the meaning of Digital or Virtual in this question means something more specific than in the other questions? And whether there is a meaningful difference between AI service and AI product which seem to be used interchangeably in the questions) 3. Can you confirm that the 750 character limit on text responses includes spaces? 4. Could you explain what the potential uplift from £25m to £45m is envisaged to be used for. Would this most likely be to increase the volume, increase the duration or expand the scope of services provided? 5. Considering this is a new DOS response format, can Dstl confirm how many examples they are expecting in each response? 6. Can you confirm the Procurement Lead/organisation responsible for managing the procurement for this tender, on behalf of the Buyer? 7. Q4 – could you clarify what is meant by “Demonstrable access to military experience”

1. Technical and project management representatives will be evaluating Stage 1, and in addition a commercial representative will be included to evaluate Stage 2. All representatives will be from Dstl/MOD. 2. Physical systems = Robotic and Autonomous Systems, i.e. systems which perform physical actions. Virtual systems = Digital systems, i.e. systems that do not perform physical actions. 3. The 750 character limit includes spaces. 4. This would be to expand volume / potentially extend time but not to increase scope. 5. Please use your best judgement – as many or few as you see fit. 6. Dstl are the buyer for this requirement, and will also be managing the procurement for this tender. 7. This refers to the ability to bring personnel with military experience into concept teams, to provide context and expertise of the military environment.

Last Updated: 7 March 2023, 16:14

9. 1. Do DAIC intend that the supplier would deliver or support existing DAIC-X and DAIC Ops in-service products or would they continue to be developed/supported by the original supplier? If support for existing in-development/service products is required, could you list the existing products and respective current suppliers? 2. Is Equality, Diversity, Inclusion relating to AI development and agile delivery teams a factor in your cultural fit questions and scoring? 3. Who will be marking the stage 1 and stage 2 responses? 4. Do DAIC-X use the same open source AI, ML, MLOps technologies as DAIC Ops? If not, are there any technologies that are particularly relevant to DAIC-X delivery? 5. For questions that ask for 'Demonstrable Experience', do you want to see a specific case study for this evidence and are you happy with evidence gained from more than one case study?

1. The supplier may be asked to support further development and support of current DAIC concepts. 2. AI development and agile delivery teams do not have to factor in all your answers under the Cultural Fit questions, although agile team working is specifically mentioned under question #2 so your response should take this into account. The Cultural Fit responses will be required for Stage 2 evaluation. 3. Technical and project management representatives will be evaluating Stage 1, and in addition a commercial representative will be included to evaluate Stage 2. All representatives will be from Dstl/MOD. 4. DAIC-X and DAIC Ops use a blend of open source and commercial technologies, according to best fit for the specific need. 5. Yes, and yes.

Last Updated: 7 March 2023, 16:11

10. 1. What is the expected level of on-site presence versus remote working? 2. International travel requirement is quoted - What level of international travel is expected? 3. What is the current team sizes of the DAIC-X and DAIC Ops? 4. What is the expected growth into those functions from this contract? 5. What is the expected split of resources required to the DAIC-X and DAIC Ops functions 6. Are there any specific roles that the DAIC requires. 7. Are there any particular areas of focus from the AI family of general purpose technologies? 8. Is the focus on professional services or are there any technology supplier or product licensing requirements across DAIC-X or DAIC Ops? 9. Is there an expected ration of cloud versus on-premise solutions? 10. What are the current tech stacks and infrastructure deployed in the DAIC-X, established via a previous DOS in 2021. 11. Will the successful bidder be working alongside this partner in the DAIC-X function?

1. Estimated 20% on site working, though this will vary depending on the concept. 2. It is expected there will be a need to travel for approx. 4-8 weeks per year. This will be a limited number of personnel for a specific concept. 3. DAIC-Ops has one MLOps Team of 6 pax and currently an additional 6 pax. DAIC-X has 3 teams of 3 pax. 4. Expecting to grow to c2-3 additional product teams in DAIC-Ops but would be keen to scale further if we can. 5. We envisage a 50/50 split. 6. We will be asking suppliers to propose team structures and role in the second phase of this opportunity. No specific roles will be mandated, we will be looking for suppliers to propose how best to deliver the requirement. 7. No, concepts are expected to cover a range of AI technologies so the successful bidder will need to demonstrate broad expertise. 8. No product licensing requirements. 9. No expected ration, deployment needs will be concept dependent. 10. Current tech stack is a blend of commercial and open source tooling. 11. There will be a handover period from the previous Agile Delivery Partner to the successful bidder of this opportunity, but aside from that the successful bidder will not work with the partner from the previous DOS in 2021. The successful bidder will be expected to work alongside other teams and partners in DAIC, as per the “Existing team” section of the requirement.

Last Updated: 7 March 2023, 16:06

11. In the evaluation weighting, the technical competence is listed as: 10% essential skills, 5% nice to have, and 85% technical. For clarity of understanding, please could the Authority confirm that: a. For Phase 1, and the Authority’s downselection of bidders, only essential and nice to have skills and experience are assessed b. For Phase 1, that all ‘skills and experience’ criteria have equal weighting and use the same evaluation scoring scheme (i.e. 0 to 3) c. Essential and Nice to Have skills and experience scores are only used for the Phase 1 evaluation 2. Please could the Authority confirm that a level playing field has been ensured between the incumbent Agile Delivery Partner and new bidders such that: (a) evidence used from the current Agile Delivery Partner work is not favoured over other evidence in the evaluation, (b) staff associated with the incumbent or other bidders are not involved in the evaluation, and (c) the Authority benefits from experience and expertise from both inside and outside Defence. 3. Please could the Authority confirm that the price evaluation in Phase 2 will allow bidders to design and propose the mix of roles and effort to deliver the “first concept development task” in a way they consider would be most efficient and effective to deliver the Authority’s requirement? 4. Concerning essential skill and experience #6, for clarity, please could the Authority confirm (as appears the intent), that demonstrating experience of delivering AI services in accordance with similar ethical principles in other organisations will be assessed equally to experience of delivering to MOD’s ethical principles. Given the relatively recent publication of these (July 2022) and therefore limited opportunity for suppliers to demonstrate delivery to them, this would help to maintain a level playing field between the incumbent and new suppliers.

1a. The evaluation weighting for Technical competence listed as 10% essential skills, 5% nice to have, and 85% technical is to be used for Stage 2 evaluation. In Stage 1 (this stage) we will be evaluating responses under the Technical competence; Essential skills and experience (total weighting of 100%) and Nice-to-have skills and experience (total weighting 100%). Both of these will be taken through to Stage 2 where they will equate to the 10% essential skills and 5% nice to have. Only essential skills and experience and nice to have skills and experience are assessed in Stage 1. 1b. Not all skills and experience criteria have equal weighting, some score 20%, some 15% and some 10%; please refer to the tables in Attachment 3. We will use the same evaluation scoring scheme throughout. 1c. Essential and nice to have skills are the only assessment being used for Stage 1, but their scores will be taken through to Stage 2 and form part of the 15% Technical competence evaluation weighting. 2a. The set procurement process is being followed so as to ensure fair and open competition. No one supplier is being given any unfair advantage over any other supplier and all are being treated as on an open playing field. All bids received via this competitive process will be assessed using the technical and commercial criteria as stated in the requirement. We confirm that the incumbent supplier has not been involved or consulted in formulating this requirement. 2b. Only Dstl/MOD staff are involved in the evaluation process. 2c. Dstl is happy to embrace suppliers from all backgrounds and industries and encourages diverse supply chains and partnering. 3. We would be content for any mix of roles and effort as you see fit to meet the requirement. We would require that each role is clearly defined with a corresponding grade and hourly rate. 4. Your assumption is correct.

Last Updated: 7 March 2023, 15:44

12. Could you please confirm that spaces are not included within the word limit.

Last Updated: 7 March 2023, 15:37

13. Could you clarify the format in which we should respond to this opportunity please? Attachment 3 does say "Responses to Essential and Nice-to-have skills and experience" but there is no space available to insert the responses. Should we carriage return to insert a box under each question or provide responses on a separate document? Just to note that most other DOS 6 opportunities we`ve seen have included a forth attachment to respond with.

Unfortunately, during the publishing process there was no preview of what the bidders pack would look like, therefore we could not anticipate what documents would form the bidders pack. We would be happy with either responses inline on Attachment 3 by creating your own space or a separate document. If DOS allows a response via the portal then that would also be fine; again we are unable to see from the Buyer side if that is an option available to you as suppliers.

Last Updated: 7 March 2023, 15:34

Help

You can contact us by email, phone or using the enquiry form (opens in a new tab) .

Email:

info@crowncommercial.gov.uk

Telephone:

0345 410 2222

CCS customer services team is available Monday to Friday, 9am to 5pm.

Cookies on Contract Award Service (CAS)