3 ECS Key Application Areas

Ambition: empower and protect the individual citizen.

• Increase inclusion and prevent exclusion.

• Protect citizens against cyber-fraud (scams) and identity thefts; provide privacy.

• Enable safe smart homes with ubiquitous connectivity.

• Ensure acceptable AI technologies.

Diversity and inclusion within societies are increasingly recognised as crucial for equality at work and economic development. Research has established a strong link between gender equality in society, attitudes, and beliefs about the role of women, and gender equality in work⁵. Technology can improve equality at work – for instance, by revealing pay gaps and biases, and helping de-bias recruitment. It can also improve equal access to essential services – for example, biometrics and cloud technology can contribute to increasing the diffusion of microfinance to women and underserved populations. Technology can also help enforce inclusive legal rights, policies, and social norms. While e-voting still poses a number of cybersecurity challenges, it can support diversity by facilitating the vote for vulnerable and marginalised parts of society. Finally, technology can help with physical security and autonomy for minority groups through objects and digital communications tools that reduce or mitigate exposure to risk – for example, connected devices such as smart bracelets can enable women to signal an assault and call for help.

Reliance on technology comes with many benefits, but also brings new risks⁶. The radical nature of the ongoing technology transition could result in risks that are not just an extension of the previous challenges but require fundamental changes to core aspects of our society, including how we think about our identity, security, and rights. Concerns about technology are justified by recent events, such as security breaches in prominent companies, data theft and information misuse. In addition, AI provides more powerful examples of potential risks. Its full potential can be used only if we fully rely on it for decision-making, allowing it to process data beyond the human ability to cross-check and verify. This depends on a high level of trust, raising questions about, and requiring, new technical solutions that take into account explainability, accountability, trustworthiness and ethics.

In the early ‘20’s we have experienced the necessity of a connected smart home and an adequate home office during the pandemic. However, the availability of high bandwidth connectivity is not evenly distributed geographically across Europe.

Machine learning is essential for a resilient future. AI will have a far greater chance of successful implementation if there is a focus on four key areas: augmented intelligence; intelligent automation; assessed intelligence; and adaptive intelligence. Augmented intelligence concerns augmenting and thus improving the productivity of humans. Intelligent automation is about building systems that integrate humans and machines in productive ways (instead of just replacing humans entirely with machines). Assessed intelligence is all about making models robust by evaluating them rigorously and continuously. Finally, adaptive intelligence involves developing more resilient systems that can adapt to changing circumstances by shifting to a causal inference paradigm.

Ambition: safeguard collective society and well-being for all.

• Societal and digital inclusion.

• Provide societal access for all.

• Ensure collective resilience against setbacks.

Although European countries have diverse types of welfare models, they also share a history of robust social protection and a focus on inclusive growth, which has been under stress in recent years⁸. There could be cracks in the sustainability of the EU social contract over the next decade caused by six trends: ageing; digital technology, automation, and AI; increased global competition; migration; climate change and pollution; and shifting geopolitics. Based on these trends, inequality may rise again, and divergence within Europe could increase.

Inequality at work may emerge through a combination of: (i) automation and the substitution of labour; and (ii) corporate diffusion dynamics, leading to a competitive disadvantage among non-adopting firms. To prevent reduced employment and secure real wage growth, automation using AI, robotics and other innovative technologies should lead to significant productivity gains. In general, occupations based on more repetitive and non-digital tasks will be taken by workers with low education and skills, who will therefore be the first to experience pressure on wages.

Collective growth and well-being are not only determined by equality at work, but also by individual development supported by collective interactions. Studies have shown that active social relationships increase health and longevity by improving key biomarkers of physical health. A lack of interaction causes a subtle decline in mental health by reducing attention, learning, memory, and decision-making skills. In short, our bodies reward us for social interaction and punish us for isolation by negatively impacting mental and physical health. Direct interactions with family and friends, participating in team sports and, for instance, visiting an event with friends are particularly important. No technology can fully replace direct contact. However, during the former Covid pandemic, we had to rethink our social interactions, and to adapt technologies to increase and improve social interactions – not just between individuals, but also between individuals and groups, as well as between different groups. How can improved technologies support existing social interactions, and secure healthy digital social interactions in cases of setbacks?

It may seem obvious that the Electronic Components and Systems used are trustable devices and technology, but this is an important boundary condition in Europe.

In addition, collective safety can be enhanced by solutions that directly address specific communities or groups of people the individual is a member of, such as family, friends, neighbourhood, region, (sports)club or association. These solutions can either have a warning or alerting function (e.g. contamination, local fire, local air pollution, incident of violence), but can also be of a supporting nature – for instance, alerts or instructions in combination with collective supporting devices (e.g. automated external defibrillator (AED), diagnostics, measurement).

High-priority R&D&I areas:

• Digital inclusion: tools, infrastructure, training, connectivity.

• Collective safety: secure access control, surveillance, pandemic control, prevention of misinformation without limiting freedom of expression.

• Safe environment for living, working and transport: buildings and bridges resilient against earthquakes through continuous monitoring (e.g. fibre-based stress sensors).

• Emergency/crisis response solutions and services.

• Dynamics of society: systemic change.

As Europe wants to play a key role in digital inclusiveness, it is important to ensure availability and accessibility of solutions to enable remote education, learning, training and assessment of professionals, students, and consumers in all regions (both cities and rural areas). Also, solutions to support social inclusiveness for people of all age should become available.

The EU has stated, in their document on orientations towards the first strategic plan for implementing Horizon Europe⁹, that the interaction of science, technology, social sciences and humanities will be crucial in this respect, as will be the input of the creative sector and artists to sustainable inclusive innovation and human- oriented technologies.

To facilitate inclusion, more research will be needed on education, simple human–machine interfaces and digital technology interfaces that avoid the digital split between high- and low-educated citizens. In addition, remote presence, and remote connectivity to keep people connected even if they are not in the same location, trustworthy social media, serious gaming, media consumption and AR/VR will be key.

To safeguard digital inclusion, education is one of the most important research areas. Examples here are the use of AI to build personalised journeys and enhance learning outcomes, to adapt curriculum to individual student needs, digital support and nudging systems to reduce the administrative burden on teachers, tablet-based learning to improve results and decrease distress for students with dyslexia, automation of administrative tasks to free up time and resources for educational professionals, wearable devices that provide real-time support to pupils, eye-tracking solutions to adapt students’ learning experiences, and use of AR/VR to provide immersive experiences to civilians in less well-served areas.

AR may improve connectedness for remote places, reducing the need for commuting or business travel. It could also enable consumers to enjoy an event together even if they are not physically at the event.

There are still several challenges to effectively take full advantage of AI in video creation and consumption. One is the size of video data. Results are only accurate when algorithms are fed with millions of observations. Technologies therefore have to be deployed, and strategies have to be implemented to gather data at scale to harness the full power of AI techniques. However, size creates another challenge: datasets need to be manually labelled by humans to train the model, making the process expensive and cumbersome. Many new techniques are becoming available to overcome the challenge of data categorisation, such as reinforcement learning, generative adversarial networks, transfer learning and “one- shot learning” and large-language-models. In consumer-facing applications, such as marketing and recommendation algorithms, AI models may need to be refreshed continuously due to changes in the environment that drives them. Continuous updates to AI models are expensive. Other challenges relate to data management and data gathering: to create accurate results with AI, and thus value, diverse types of data have to be managed in a unified manner. This includes audience data, operational data, and content data (metadata). Also, “selection bias” (i.e. the data gathered is not representative of the population studied) has to be prevented to exclude wrong conclusions in a perfectly working model.

To facilitate collective safety, further research is required on secure access control, intrusion detection, (video) surveillance of security sensitive areas, and individual and collective activity tracking.

Secure access control as a service (ACaaS) is growing in relevance. This combines biometric readers and identity access management and can be integrated with other physical security systems (e.g. video surveillance) and building automation systems. Combined with building occupancy management systems, it can deliver valuable information on the location of staff and visitors, and in the event of an emergency to rapidly clear the building.

Both the former Covid pandemic and the military conflicts on our continent has brought new physical security requirements. In addition to regular cameras, thermal cameras as well as different sensing devices could be added at the entrance of buildings and venues to measure people’s state (e.g. temperature) and investigate if they do not carry dangerous materials as they enter premises. Physical access control, enriched with video security evidence, can provide important insights on where an infected individual has been, which doors they have used and who else may have come into contact with those doors and that individual. It can also provide these insights for more general security purposes.

More research on AI security solutions will ease the work of security operators. AI software can analyse images and audio from video surveillance live streams and recordings, and use image recognition algorithms to recognise faces, objects, and events, more than a hundred times faster than human operators. AI algorithms can also be used to carry out event detection, scene reconstruction, video tracking, object recognition, and (re)-identification, 3D pose estimation, motion estimation and image restoration. Video surveillance may be extended with freely moving cameras mounted under drones to recognise unusual behaviour in crowds from a high altitude, to monitor hazards such as fires, floods, or erupting volcanoes, and to recognise criminal faces and follow targets. Since drones are airborne, they need fast mobile and wireless communications. Low-latency broadband technologies such as 5G/6G can improve the precision and speed of their response times and enable high-speed communication to a nearby edge computing device.

Video quality should be further improved to support deep-learning algorithms, and to improve the video experience in media consumption: the spectral range and colour gamut can be extended, sensitivity must increase for low light use and especially dynamic range for better performance under all (and changing) lighting conditions as this has the greatest impact on the perception.

AI video and audio algorithms will have to be transparent and explainable. Dedicated video and audio technologies will be required to prevent and trace fake video and audio used to create misinformation in (social) media. Audio and video equipment used to create content must be able to watermark content streams to prove authenticity, and metadata should be added to the streams to prove what processing activities have taken place from image capture until display at home. Next to that equipment used for content creation based on open public infrastructures (such as internet and 5G) and with processing or control activities in the cloud must be better secured to prevent unwanted take-over by aggressors.

Required R&D&I developments within ECS

To facilitate inclusion and collective safety, specific R&D developments within ECS technology are necessary, as shown in Figure 3.6..

Figure 3.6.5 - Required R&D&I developments within ECS – Major Challenge 3

Ambition: contribute to a collective supportive infrastructure and environment.

• Provide trustworthy, reliable, and resilient infrastructure.

• Establish a sustainable and secure environment.

• Provide means for controlled climate change.

To fully benefit from the power of digitisation, Europe must enable a supportive infrastructure and environment. Given the rapid pace of change, this requires companies to get their technology, people, and culture ready to join the digital transformation. This should be achieved by providing a reliable and resilient digital infrastructure (with ubiquitous and continuous connectivity), protecting society against destabilising forces and establishing a secure and sustainable environment. The former includes preventing harmful use of the internet (e.g. manipulation of elections, misinformation such as “deepfakes” and “cheapfakes”, but also identity theft and phishing), which are covered by Major Challenge 3. The latter includes securing controlled climate change (as stipulated in the Green Deal) by providing adequate means that can be utilized by citizens. Furthermore, monitoring, and intelligent control of infrastructures, essential resources, and their recycle process (especially in the urban environment) will contribute to a sustainable environment.

The vision is to introduce new digital products and services that contribute to a sustainable lifestyle in all areas of human life, including cradle-to-cradle and circular economy aspects. We are addressing the following aspects:

• Comprehensive assessment of resource usage to identify largest areas of energy and materials consumption and CO₂ production. As well as air quality monitoring systems, they need to offer solutions for lighting, heating, computing with reduced usage of energy, and other resources. In addition, solar panels and batteries, home-grown vegetables and city farming systems are key.

• Providing smart systems based on IoT and unmanned/robotic platforms that provide secure environments and support digital business life with the minimum amount of resources (energy, water, paper, travelling, etc.), ensuring a highly efficient, productive, and sustainable working environment.

• Smart water management to protect resources. Intelligent management of energy in public spaces such as football stadiums and railway stations, including smart street lighting. Promoting green areas in cities and enabling citizens to provide their own sustainable solutions. Reduction of (food) waste in supermarkets and restaurants, as well as resource recycling.

• Development of new digital solutions that can be used by citizens themselves or be used to stimulate societal attitudes and activities towards sustainability and addressing climate change, e.g. car sharing, use of public transportation, bikes, and e-scooters.

Figure 3.6.6 - Growth of smart cities market (Source: www.marketsandmarkets.com)¹⁰

The European approach to working with regard to digitalisation will be focused on the preservation of our democratic system, and on values such as trust and cooperation. Ethical requirements will include fairness, accuracy, confidentiality, transparency, accountability, explainability, trustworthiness and absence of bias. This involves offering AI capability maturity programs to companies that use AI in their designs, to coach them in the best ethical points of view. In this way, products will become more resilient, accessible, reliable, and trustworthy, and hence ready to take part in the new European digital society (with an effective AI act).

High-priority R&D&I areas:

• Physical infrastructure management/physical resilience and security.

• Intelligent infrastructure management (intelligent buildings, city-owned infrastructure, synergies with industry, etc.).

• Digital solutions and infrastructure for resources management, digital resilience, sustainability, e-government and citizen support.

• Resource and environment monitoring (air, water, etc.) and feedback to enable more effective management.

To further improve digital transformation towards more secure, effective, and sustainable environments, new digital tools employing IoT, unmanned vehicles/robots and AI/ML-enhanced algorithms have to be infrastructures, investments should be aimed at enhancing infrastructure coverage and quality . Also, outcomes have to be influenced through legal frameworks and by setting standards.

Intelligent buildings will require security, eco-friendship and building management. Security systems such as access control and cybersecurity were covered under Major Challenge 3, but the further development of smart lighting, air quality monitoring and control, and IoT-based real-time monitoring of electric, water and gas meters to increase the energy efficiency of buildings with the help of distributed energy systems will improve the well-being of occupants and reduce the carbon footprint of buildings. Smart technology (e.g. sensors placed around radiators, boilers, pumps, and other machinery to detect critical levels of noise, vibration, or heat) will enable facility managers to save maintenance costs by switching from a reactive to a predictive maintenance model.

Cities are very complex organisms. They combine a variety of means allowing for mobility, city infrastructure providing diverse types of media (gas, water, energy, etc.), and citizen-oriented services that increase their quality of life. It is predicted that by 2050 between 68% and 90% of the global population might live in cities, from small municipalities right up to megacities¹¹. This means that, in the near future, technical means will be required to enable digital solutions for more sustainable development in cities of all size and wealth. Available technologies from tech giants such as IBM, Microsoft, Amazon, Google, and Cisco raise concerns from city managers about data privacy policies, and the very high maintenance costs caused by licence fees and the potential for vendor lock-ins¹². Available open-source solutions – such as the Red Hat integration platform, which could be used in smart city applications – can also easily be acquired by large companies such as IBM¹³ to be integrated with their company product portfolio offered commercially. This means that, in such a dynamically changing world, open-source solutions that are widely available, promoted and deployed within EU (such as FiWARE¹⁴) have to be developed to protect European sovereignty and values. Additionally, due to the rich industrial heritage in many EU countries, opportunities for re-using or integrating available well- developed open-source industry platforms, such as the Eclipse Arrowhead Framework¹⁵, can be adapted to smart city needs based on requirements gathered in EU-funded projects. This is especially the case since industry sites are often integrated within city areas, and therefore naturally create synergies that can influence each other. These smart city applications create natural synergies with the System of Systems, Mobility and Digital Industry sections.

The impact of technology on environmental sustainability is likely to be highly significant. In retail, where shifting customer habits will be key (for example, for new products such as plant- or insect-based food), IoT sensors and devices will also yield a positive impact – for example, by reducing waste through improved food temperature or expiry date management. In the manufacturing sector, smart building applications related to energy and wastewater management, as well as applications such as carbon capture and biofuel generation on industrial sites, will have a significant impact.

Required R&D&I developments within ECS

Development of supportive infrastructure and a sustainable environment within EU needs the following specific R&D developments within ECS technology:

Figure 3.6.7 - Required R&D&I developments within ECS – Major Challenge 4