Validate AI Conference - 5th November at the Royal Society

The conference was inspired by the participation of the co-proposers to discussions at the Royal Society in 2018 about the development of data science skills nationally, promoting greater collaborative work between public, private and academic sectors.  These discussions were recently shared in a Royal Society publication titled Dynamics of Data Science Skills.

The importance of the Validate AI theme was also highlighted by a specialist Masterclass learning program led by Shakeel Khan at HMRC and Dr Tony Bellotti of Imperial College London to promote machine learning maintenance best practice that is commonly found in the financial services sector that is very relevant to tax authorities.

Algorithms touch every aspect of our modern lives, helping us make millions of decisions every day. The additional monetary benefit of Artificial Intelligence (AI) to UK GDP has been estimated to be up to £232bn by 2030. The use of algorithms is also growing exponentially and is well established across a broad spectrum of sectors, including banking, health, taxation, environment, agriculture, security and defence. Examples of the impact on individuals are numerous and include supporting the diagnosis and prognosis of life-threatening illnesses such as heart disease and cancer, self-driving cars, identifying high risk taxpayers, crop management, identifying banking fraud cases, customer experience enhancement and disease containment.

There is also a major debate ongoing about public acceptance of AI and concerns that such technology may compromise fairness and quality of decisions when replacing the many roles previously undertaken by humans. Governments, and indeed the private sector, recognise the need to demonstrate that AI is both ethically sound and robust for the task it was developed for, in order to maintain public trust. Democratisation of algorithm development and the recent relative ease with which such solutions are being created and deployed provides great opportunities for more widespread economic and societal benefits. With this also come risks to AI validity and ongoing maintenance. There is an immense responsibility to ensure models are fit for purpose and fair both in the public and private sector, and importantly for this to be demonstrated openly in society.

The assessment of quality and methods for AI maintenance may differ dramatically depending on the nature of the problem and sector to which the AI is being applied. There may also be legislative constraints that impact on the particular algorithm, requiring greater transparency, for example, in how a specific outcome or decision is reached.

Whilst it is now becoming easier to produce AI solutions at lower cost, we need a more rigorous debate to raise the importance of measuring whether the solutions are fit for purpose, safe, reliable, timely and trustworthy. This conference explores how such systems can depart from this ideal, examining tools and methods for ensuring sound and appropriate behaviour in a variety of different application domains, and looking at open challenges. Issues explored will include accurate and unbiased performance and its evaluation, model testing and formal verification, ensuring resilience against adversarial attacks, and the effective maintenance of systems as their working environment evolves. This last may include changing populations, increasing data loads, new and unforeseen kinds of data, and policy and other changes.

A critical aspect of this conference is to bring together representatives from public, private and academic sectors to promote greater collaboration of these issues, sharing experiences, challenges, and solutions. There are for example sectors where AI reliability metrics have a reasonable level of maturity, such as for customer lending in financial services, and lessons need to be learnt from such practices to promote awareness and adoption across sectors where appropriate.

We anticipate up to three hundred delegates at the conference which will be held at the Royal Society on Tuesday 5th November 2019. 

A broad spectrum of sectors and problem areas will be represented. This is a single stream conference, with speakers invited to showcase innovative applications of validation, maintenance, ethics and transparency in their sectors and specialist.

Presentations will be from the UKs national centre for data science and AI institute as well as for banking, tax compliance, autonomous vehicles, safety critical systems and health to name but a few. We would expect this to promote further research of in AI maintenance. We are very pleased to have leading academics and practitioners including from the Turing Institute, Imperial College London, Oxford University, HMRC, Google Deepmind and, ONS Data Science Campus  

The target audience is policy makers and AI practitioners in industry, academia, and government.

We plan panel discussions for the morning and afternoon sessions, where examples of questions may include:

• How do we incentivise collaboration in AI maintenance between public, private and academic sectors as theorised by the triple-helix model?
• How do we raise awareness of maintenance best practice and research collaboration across sectors?
• What are the implications for measurement of quality when developing big data ML?
• How can system reliability be ensured in AI solutions to meet GDPR regulations?
• How do we identify discriminatory AI resulting potentially from a lack of diversity in those developing solutions and/or from the underlying data being biased?
• How do we facilitate capability building and collaboration in AI maintenance post conference?

We are particularly grateful for the immense contribution of the Imperial College London – Data Science Institute to the conference in terms of sponsorship, programme participation and organisation of the event. We look forward to collaborating with them further to shape how best to proceed post conference. We are also very pleased the Validate AI conference is running concurrently with the Imperial College London – DSI’s 5th Anniversary to celebrate their world class contribution to data science.