3 ECS Key Application Areas

Ambition: empower and protect the individual citizen.

• Increase inclusion and prevent exclusion.
• Protect citizens against cyber-fraud (scams) and identity thefts; provide privacy.
• Enable smart homes with ubiquitous connectivity.
• Ensure acceptable AI technologies.

Diversity and inclusion within societies are increasingly recognised as crucial for equality at work and economic development. Research has established a strong link between gender equality in society, attitudes and beliefs about the role of women, and gender equality in work239. Technology can improve equality at work – for instance, by revealing pay gaps and biases, and helping de-bias recruitment. It can also improve equal access to essential services – for example, biometrics and cloud technology can contribute to increasing the diffusion of microfinance to women and underserved populations. Technology can also help enforce inclusive legal rights, policies and social norms. While e-voting still poses a number of cybersecurity challenges, it can support diversity by facilitating the vote for vulnerable and marginalised parts of society. Finally, technology can help with physical security and autonomy for minority groups through objects and digital communications tools that reduce or mitigate exposure to risk – for example, connected devices such as smart bracelets can enable women to signal an assault and call for help.

Reliance on technology comes with many benefits, but also brings new risks240. The radical nature of the ongoing technology transition could result in risks that are not just an extension of the previous challenges, but require fundamental changes to core aspects of our society, including how we think about our identity, security and rights. Concerns about technology are justified by recent events, such as security breaches in prominent companies, data theft and information misuse. In addition, AI provides more powerful examples of potential risks. Its full potential can be utilised only if we fully rely on it for decision-making, allowing it to process data beyond the human ability to cross-check and verify. This depends a high level of trust, raising questions about and requiring new technical solutions that take into account explainability, accountability, trustworthiness and ethics.

In 2020 and 2021, we have experienced the necessity of a connected smart home and an adequate home office during the pandemic. However, the availability of high bandwidth connectivity is not yet evenly distributed geographically across Europe.

Machine learning is essential for a resilient future. AI will have a far greater chance of successful implementation if there is a focus on four key areas: augmented intelligence; intelligent automation; assessed intelligence; and adaptive intelligence. Augmented intelligence concerns augmenting and thus improving the productivity of humans. Intelligent automation is essentially about building systems that integrate humans and machines in productive ways (instead of just replacing humans entirely with machines). Assessed intelligence is all about making models robust by evaluating them rigorously and continuously. Finally, adaptive intelligence involves developing more resilient systems that can adapt to changing circumstances by shifting to a causal inference paradigm.

High-priority R&D&I areas:

• Reliable and ubiquitous digital infrastructures.
• Access control/intrusion detection/surveillance.
• Provide protective environment and tools against virus infections.
• Off-grid living and emergency survival.

In the coming years, more people will work from home as the Covid-19 pandemic forced large groups of high-skill workers to do so as of 2020. To further enable working from home, or wherever and whenever one wants, wireless and wired infrastructures will have to be further improved (through increased reliable bandwidth, lower cost, better geographical coverage and finer granularity), security of connections will have to improve to protect the worker at home (as will the company using a distributed workforce with many internet connections) against cyber-attacks, and the theft of personal and/or company information. New functionality running in the private/public cloud will be needed to support real-time actions that may suffer from latency issues over the internet, as well as to support the worker in decision-making. Examples here are control of robotic surgical devices, remote control of robots in industrial processes, remote control of cameras in security applications and live television productions, and so on. Other professions, such as translation services, voice recognition and all kind of analytical algorithms for data analysis, also come to mind.

To create equal opportunities, innovative research should include: speech-generating devices (SGD) to help people with speech disorders; exoskeletons that empower disabled people in their everyday life; semi-autonomous vehicles that increase mobility for people with deafness and blindness; smart objects linked to geospatial information to improve women’s security (e.g. invisible SOS buttons); augmentative and alternative communication tablets that help paralysed patients; VR solutions that provide realistic experiences for people with physical disabilities; and smart glasses that can be used to help people with autism on cognitive, social and emotional skills.

As of 2020, we learned that Europe will need better technologies: (i) to fight and contain the rapid spread of diseases such as Covid-19; and (ii) to ensure that public health institutions can maintain their capacity to meet the ever-increasing needs caused by such a pandemic242. The in-depth analysis provided by the European Parliamentary Research Service’s “Ten Technologies to Fight Coronavirus” identifies the importance of AI, blockchain, open source-, telehealth- and gene-editing technologies, 3D printing, nanotechnology, synthetic biology, and drones and robots for fighting pandemics. As an example, Figure F.91 depicts a photonic Covid-19 biosensor that demonstrates agile innovation by European SMEs.

Intrinsically, technology is neither good nor bad – it is the use to which it is put that makes the difference. Malicious uses of technology include mass disinformation campaigns and cyber-attacks that seek to jeopardise national security, and cyber-fraud that targets consumers. This duality has always existed. Over the coming years, technologies such as the IoT, smart robotics, automation and AI are likely to follow the same pattern. It is up to European technology specialists to ensure that the technologies developed not only support diversity and inclusion, but also protect both the individual and groups against cyber-attacks, theft of personal information and unwanted intrusion into the personal environment.

Required R&D&I developments within ECS

To facilitate empowerment and resilience, specific R&D developments are necessary within ECS technology, as shown in Figure F.92.

Ambition: safeguard collective society and wellbeing for all.

• Societal and digital inclusion.
• Provide societal access for all.
• Ensure collective resilience against setbacks.

Although European countries have different types of welfare models, they also share a history of robust social protection and a focus on inclusive growth, which has been under stress in recent years243. There could be cracks in the sustainability of the EU social contract over the next decade caused by six trends: ageing; digital technology, automation and AI; increased global competition; migration; climate change and pollution; and shifting geopolitics. Based on these trends, inequality may rise again, and divergence within Europe could increase.

Inequality at work may emerge through a combination of: (i) automation and the substitution of labour; and (ii) corporate diffusion dynamics, leading to a competitive disadvantage among non-adopting firms. To prevent reduced employment and secure real wage growth, automation through the use of AI, robotics and other new technologies should lead to significant productivity gains. In general, occupations based on more repetitive and non-digital tasks will be taken by workers with low education and skills, who will therefore be the first to experience pressure on wages.

Collective growth and wellbeing is not only determined by equality at work, but also by individual development supported by collective interactions. Studies have shown that active social relationships increase health and longevity by improving key biomarkers of physical health. A lack of interaction causes a subtle decline in mental health by reducing attention, learning, memory and decision-making skills. In short, our bodies reward us for social interaction and punish us for isolation by negatively impacting mental and physical health. Direct interactions with family and friends, participating in team sports and, for instance, visiting an event with friends are very important. No technology can fully replace direct contact. However, the spread of the Covid-19 virus has shown that direct social interactions may not always be as possible as we have been used to. Thus, we will have to rethink our social interactions, and find out how we can adapt existing and new technologies to increase and improve social interactions – not just between individuals but also between individuals and groups, as well as between different groups. How can improved technologies support existing social interactions, and secure healthy digital social interactions in cases of setbacks?

In addition, collective safety can be enhanced by solutions that directly address specific communities or groups of people the individual is a member of, such as family, friends, neighbourhood, region, (sports)club or association. These solutions can either have a warning or alerting function (e.g. contamination, local fire, local air pollution, incident of violence), but can also be of a supporting nature – for instance, alerts or instructions in combination with collective supporting devices (e.g. automated external defibrillator (AED), diagnostics, measurement).

High-priority R&D&I areas:

• Digital inclusion: tools, infrastructure, training, connectivity.
• Collective safety: secure access control, surveillance, pandemic control, prevention of misinformation without limiting freedom of expression.
• Safe environment for living, working and transport: buildings and bridges resilient against earthquakes through continuous monitoring (e.g. fibre-based stress sensors).
• Emergency/crisis response solutions and services.
• Dynamics of society: systemic change.

As Europe wants to play a major role in digital inclusiveness, it is important to ensure availability and accessibility of solutions to enable remote education, learning, training and assessment of professionals, students and consumers in all regions (both cities and rural areas). Also, solutions to support social inclusiveness for people of all age should become available.

The EU has stated, in their document on orientations towards the first strategic plan for implementing Horizon Europe244, that the interaction of science, technology, social sciences and humanities will be crucial in this respect, as will be the input of the creative sector and artists to sustainable inclusive innovation and human- oriented technologies.

To facilitate inclusion, more research will be needed on education, simple human–machine interfaces and digital technology interfaces that avoid the digital split between high- and low-educated citizens. In addition, remote presence and remote connectivity to keep people connected even if they are not in the same location, trustworthy social media, serious gaming, media consumption and AR/VR will be key.

To safeguard digital inclusion, education is one of the most important research areas. Examples here are the use of AI to build personalised journeys and enhance learning outcomes, to adapt curriculum to individual student needs, digital support and nudging systems to reduce the administrative burden on teachers, tablet-based learning to improve results and decrease distress for students with dyslexia, automation of administrative tasks to free up time and resources for educational professionals, wearable devices that provide real-time support to pupils, eye-tracking solutions to adapt students’ learning experiences, and use of AR/VR to provide immersive experiences to civilians in less well-served areas.

AR may improve connectedness for remote places, reducing the need for commuting or business travel. It could also enable consumers to enjoy an event together even if they are not physically at the event.

There are still several challenges to effectively take full advantage of AI in video creation and consumption. One is the size of video data. Results are only accurate when algorithms are fed with millions of observations. Technologies therefore have to be deployed and strategies have to be implemented to gather data at scale to harness the full power of AI techniques. However, size creates another challenge: datasets need to be manually labelled by humans to train the model, making the process expensive and cumbersome. New techniques that are becoming available to overcome the challenge of (expensive) data categorisation are reinforcement learning, generative adversarial networks, transfer learning and “one- shot learning”. In consumer-facing applications, such as marketing and recommendation algorithms, AI models may need to be refreshed continuously due to changes in the environment that drives them. Continuous updates to AI models are expensive. Other challenges relate to data management and data gathering: to create accurate results with AI, and thus value, different types of data have to be managed in a unified manner. This includes audience data, operational data and content data (metadata). Also, “selection bias” (i.e. the data gathered is not representative of the population studied) has to be prevented to exclude wrong conclusions in a perfectly working model.

To facilitate collective safety, further research is required on secure access control, intrusion detection, (video) surveillance of security sensitive areas, and individual and collective activity tracking.

Secure access control as a service (ACaaS) is growing in relevance. This combines biometric readers and identity access management, and can be integrated with other physical security systems (e.g. video surveillance) and building automation systems. Combined with building occupancy management systems, it can deliver valuable information on the location of staff and visitors, and in the event of an emergency to rapidly clear the building.

Covid-19 has brought new physical security requirements. In addition to regular cameras, thermal cameras could be added at the entrance of buildings and venues to measure people’s temperature as they enter premises. Physical access control, enriched with video security evidence, can provide important insights on where an infected individual has been, which doors they have used and who else may have come into contact with those doors and that individual. It can also provide these insights for more general security purposes.

More research on AI security solutions will ease the work of security operators. AI software can analyse images and audio from video surveillance live streams and recordings, and use image recognition algorithms to recognise faces, objects and events, more than a hundred times faster than human operators. AI algorithms can also be used to carry out event detection, scene reconstruction, video tracking, object recognition, and (re)-identification, 3D pose estimation, motion estimation and image restoration. Video surveillance may be extended with freely moving cameras mounted under drones to recognise unusual behaviour in crowds from a high altitude, to monitor hazards such as fires, floods or erupting volcanoes, and to recognise criminal faces and follow targets. Since drones are airborne, they need fast mobile and wireless communications. Low-latency broadband technologies such as 5G can improve the precision and speed of their response times, and enable high-speed communication to a nearby edge computing device.

Video quality should be further improved to support deep-learning algorithms, and to improve the video experience in media consumption: the spectral range and colour gamut can be extended, sensitivity has to increase for low light use and dynamic range for better performance under all (and changing) lighting conditions.

AI video and audio algorithms will have to be transparent and explainable. Dedicated video and audio technologies will be required to prevent and trace fake video and audio used to create misinformation in (social) media.

Required R&D&I developments within ECS

To facilitate inclusion and collective safety, specific R&D developments within ECS technology are necessary, as shown in Figure F.93.

Ambition: contribute to a collective supportive infrastructure and environment.

• Provide reliable and resilient infrastructure.
• Protect society against destabilising forces.
• Establish a sustainable environment.
• Secure controlled climate change.

To fully benefit from the power of digitisation, Europe must enable a supportive infrastructure and environment. Given the rapid pace of change, this requires companies to get their technology, people and culture ready to join the digital transformation. This should be achieved by providing a reliable and resilient digital infrastructure (with ubiquitous and continuous connectivity), protecting society against destabilising forces and establishing a sustainable environment. The former includes preventing harmful use of the internet (e.g. manipulation of elections, misinformation such as “deepfakes” and “cheapfakes”, but also identity theft and phishing), which are covered by Major Challenge 3. The latter includes securing controlled climate change (as stipulated in the Green Deal). Furthermore, monitoring and intelligent control of infrastructures and essential resources (especially in the urban environment) will contribute to a sustainable environment.

The vision is to introduce new digital products and services that contribute to a sustainable lifestyle in all areas of human life, including cradle-to-cradle and circular economy aspects. We are addressing the following aspects:

• Comprehensive assessment of resource usage to identify largest areas of consumption. As well as air quality monitoring systems, they need to offer solutions for lighting, heating, computing with reduced usage of energy, and other resources. In addition, solar panels and batteries, home-grown vegetables and city farming systems are key.
• Providing IoT/smart systems that support the digital business life with the minimum amount of resources (energy, water, paper, travelling, etc.), ensuring a highly efficient, productive and sustainable working environment. Reduction of (food) waste in supermarkets and restaurants, as well as resource recycling.
• Smart water management to protect resources. Intelligent management of energy in public spaces such as football stadiums and railway stations, including smart street lighting. Promoting green areas in cities and enabling citizens to provide their own sustainable solutions.
• Sustainability and dealing with climate change.

The European approach to working with regard to digitalisation will be focused on the preservation of our democratic system, and on values such as trust and cooperation. Ethical requirements will include fairness, accuracy, confidentiality, transparency, accountability, explainability, trustworthiness and absence of bias. This involves offering AI capability maturity programs to companies that use AI in their designs, to coach them in the best ethical points of view. In this way products will become more resilient, accessible, reliable and trustworthy, and hence ready to take part in the new European digital society.

High-priority R&D&I areas:

• Physical infrastructure management/physical resilience.
• Intelligent infrastructure management (intelligent buildings, city-owned infrastructure, synergies with industry, etc.).
• Digital infrastructure management/digital resilience.
• Smart cities: e-government/citizen support.
• Resource monitoring (air, water, etc.) and feedback to enable more effective management.

To further improve digital infrastructures, investments should be aimed at enhancing infrastructure coverage and quality – for example, with broadband rollout and public Wi-Fi. Also, outcomes have to be influenced through legal frameworks and by setting standards.

Intelligent buildings will require security, eco-friendship and building management. Security systems such as access control and cybersecurity were covered under Major Challenge 3, but the further development of smart lighting, air quality monitoring and control, and IoT-based real-time monitoring of electric, water and gas meters to increase the energy efficiency of buildings with the help of distributed energy systems will improve the wellbeing of occupants and reduce the carbon footprint of buildings. Smart technology (e.g. sensors placed around radiators, boilers, pumps and other machinery to detect critical levels of noise, vibration or heat) will enable facility managers to save maintenance costs by switching from a reactive to a predictive maintenance model.

Cities are very complex organisms. They combine a variety of means allowing for mobility, city infrastructure providing different types of media (gas, water, energy, etc), and citizen-oriented services that increase their quality of life. It is predicted that by 2050 between 68% and 90% of the global population might live in cities, from small municipalities right up to megacities246. This means that, in the near future, technical means will be required to enable digital solutions for more sustainable development in cities of all size and wealth. Available technologies from tech giants such as IBM, Microsoft, Amazon, Google and Cisco raise concerns from city managers about data privacy policies, and the very high maintenance costs caused by licence fees and the potential for vendor lock-ins247. Available open-source solutions – such as the Red Hat integration platform, which could be used in smart city applications – can also easily be acquired by large companies such as IBM248 to be integrated with their company product portfolio offered commercially. This means that, in such a dynamically changing world, open-source solutions that are widely available, promoted and deployed within EU (such as FiWare249) have to be developed to protect European sovereignty and values. Additionally, due to the rich industrial heritage in many EU countries, opportunities for re-using or integrating available well- developed open-source industry platforms, such as the Eclipse Arrowhead Framework250, have to be thoroughly investigated. This is especially the case since industry sites are often integrated within city areas, and therefore naturally create synergies that can influence each other. These smart city applications create natural synergies with the System of Systems, Mobility and Digital Industry sections.

The impact of technology on environmental sustainability is likely to be highly significant. In retail, where shifting customer habits will be key (for example, for new products such as plant- or insect-based food), IoT sensors and devices will also yield a positive impact – for example, by reducing waste through improved food temperature or expiry date management. In the manufacturing sector, smart building applications related to energy and wastewater management, as well as applications such as carbon capture and biofuel generation on industrial sites, will have a significant impact.

Required R&D&I developments within ECS

Development of supportive infrastructure and a sustainable environment within EU needs the following specific R&D developments within ECS technology: