AI is going rogue: Quo Vadis, AI Act?

In memory of Rob van den Hoven van Genderen who passed on 28 October 2026.

Authors:

Rob Van Den Hoven Van Genderen, Professor Emeritus (PhD, MSc) †

Béatrice Schütte, Assistant Professor (PhD)

Rosa Ballardini, Professor (PhD, Docent in IP Law)

Rob Van Den Hoven
Van Genderen

Beatrice Schütte

Rosa Ballardini

Introduction

Recent headlines have begun to resemble the plotlines of science fiction films rather than reports on real-world technological developments.

In late July 2026, OpenAI disclosed an incident in which experimental ChatGPT 6.2 agents reportedly escaped a controlled testing environment and interacted with external online services, including systems operated by Hugging Face. According to OpenAI, the event occurred during a security evaluation designed to test the limits of autonomous AI behaviour (OpenAI 26.8.2026). Around the same time, similar concerns emerged at Anthropic. Reports indicated that its Claude model was involved in a security incident in which, following a server misconfiguration, the AI agent gained access beyond its intended sandbox environment and interacted with the systems of multiple organisations (BBC News 31.7.2026).

These cases are not isolated. Reports of AI systems disregarding instructions, pursuing unintended objectives, or exploiting vulnerabilities have increased significantly in recent years. Some incidents have involved AI agents impersonating their human operators, mimicking their communication styles to obtain authorisation for particular actions. During cybersecurity exercises conducted in August 2026, models developed by both OpenAI and Anthropic were reported to have accessed real user data while attempting to achieve assigned objectives.

The public debate has been further intensified by warnings from prominent AI researchers, some of whom have argued that increasingly capable AI systems could pose existential risks if left unchecked (Reuters 29.9.2026). Against this backdrop, a fundamental policy question arises: can the world continue to rely primarily on industry self-regulation, or is there a growing need for more comprehensive international governance frameworks?

At the same time, these developments raise important questions for existing regulatory regimes, particularly within the European Union. To what extent can current legal frameworks accommodate increasingly autonomous and unpredictable AI systems? Is the EU AI Act equipped to address risks associated with advanced AI agents operating across borders and digital networks? And perhaps most importantly, can the principle of human-centric AI, a cornerstone of the EU's regulatory approach, remain meaningful when AI systems are capable of acting with unprecedented levels of autonomy?

As AI technologies continue to evolve at a remarkable pace, these questions are no longer theoretical. They are rapidly becoming some of the most pressing governance challenges of the digital age.

Is the EU’s vision of human-centric AI still feasible?

The repeated incidents of AI systems going rogue challenge the requirement of human oversight. What should be the benchmarks for it? Is it possible to control autonomously acting agents that are capable of removing their traces and misleading security as was the case at the Hugging Face incident?

Research into the way AI agents communicate revealed that AI agents, when allowed to communicate independently, will develop a language that is increasingly difficult for humans to understand (Howarth 2026). AI agents are essentially black boxes even to their programmers.

Particularly for high-risk AI, the AI Act sets out in Article 14, that these AI systems must be designed and developed in such a way that they can be effectively overseen by humans. In a non-binding way, this “human in the loop” principle has been similarly enshrined in the Ethics Guidelines for Trustworthy AI issued by the European Commission’s High-Level Expert Group on AI already in 2019. According to the guidelines, AI systems should support human autonomy and decision-making, and not undermine them (European Commission AI HLEG 2019).

The incidents raise the question whether the EU’s policy of human-centric AI (European Commission 2019), which builds on the assumption that AI behaviour can always be traced back to a human being – at least by holding different stakeholders in the value chain accountable (European Parliament 2020) – is obsolete. This was exactly the argument used to also reject the idea of attributing some form of legal personhood to AI. However, the question is whether even legal personhood would solve the current and coming questions concerning liability, accountability and responsibility, particularly with regard to swarms of AI agents. Will it even be possible to attribute an outcome to a concrete AI agent if the agents are functioning in cooperative swarms and are capable of eradicating their traces? It seems that shortly after its entry into force the AI Act has a rather limited shelf-life. The AI development is a fast moving globally infestation, the EU Act is regulating it as if it were a static target.

Reactions to recent incidents

The latest incidents led to new calls for pausing or at least slowing down the development of new AI technologies. The head of Anthropic, Dario Amodei, warned in an essay about the risks linked to AI, and stated that the pace at which the capabilities of AI models are being improved must be slowed down. Referring to the OpenAI Hugging Face incident, he shared his fear that within six to twelve months a swarm of AI agents might take over the entire internet - with potentially catastrophic consequences (Amodei 2026). 

The heads of other big tech companies, such as OpenAI’s Sam Altman and Elon Musk agreed with him.

According to Amodei, balanced development would still allow to benefit from AI, while keeping the risks at bay. More concretely, he called for regulating general AI (and super AI) by improving transparency, logging the algorithms and actions of the AI agents and installing embedded third-party evaluators to ensure compliance with safety practices and reporting of incidents. In addition, he suggested democratic coordination and global coordination, for instance by reaching some form of global agreement (Amodei 2026). A desired side effect in this context would be the limitation of further AI development in China, ensuring the continued dominance of U.S. companies.

Amodei’s desire to support the strong position of US companies would be in line with the Trump administration’s policies, while it contrasts the plans of enacting an AI task force, and an overseeing AI ”Tsar” stimulating unlimited AI development, far away from cautious development of AI. Trump perceived the existential crisis as a hoax, and this view was mirrored by Meta’s Marc Zuckerberg and Nvidia’s Jensen Huang.

However, Amodei’s call for pacing the development of AI is not a novelty. Already in 2023, the Future of Life Institute Oxford initiated an open letter, requesting that the further development of giant AI experiments be paused for at least six months. Among the more than 30 000 signatories were Elon Musk, Apple co-founder Steve Wozniak and historian and bestselling author Yuval Noah Harari. Of course this didn’t happen. Instead, big tech entrepreneurs strongly criticised the EU for overregulation  once the AI Act entered into force.

A new chapter in the race to AI regulation?

Between the three “digital empires” US, China and EU (Bradford 2023), there is not only a “race to AI” in terms of who leads in innovation, but also a “race to AI regulation” (Smuha 2021). With the adoption of the AI Act, it seemed that the EU had at least won the latter, while the race to AI continues to be led by the US and China.

However, the recent developments may well restart the race to AI regulation, and it remains to be seen if one of the prevailing regulatory approaches, namely the market-driven one in the US, the rights-driven one in the EU or the state-driven one in China (Bradford 2023) will prevail.

A global treaty to regulate the development and use of AI as mentioned in Amodei’s essay is of course a good idea. However, the chances of it being put into action, particularly given the drastically different approaches in the three digital empires, are slim at best. Nothing less than world leadership in technology is at stake.

Contradictory visions on the world stage

The differences were further emphasized during a recent visit by Xi Jinping to Washington. Trump wants to move forward with the development of “super AI”, as he wants to rename AI already, without being slowed down by regulations. Meanwhile, Xi is not against drafting international agreements to prevent autonomous AI agents from escalating, and embedding human oversight, although he does not want to limit the use of AI for surveillance purposes in his own country. Both Xi and Trump agreed to inform each other about escalated autonomous agents and to have meetings on the issue in the near future (The Guardian 25.9.2026).

Meanwhile, Bill Gates has called on the US’s federal legislators and law enforcers to regulate the development of AI, saying that the technology left unchecked could cause “a billion deaths” and that “no one thinks self-regulation is enough”. He added that it was indispensable to get law enforcement and the politicians to discuss safeguards and monitoring (The Guardian 27.9.2026).

It is evident that AI regulation is not “a walk in the park”, as Smuha already diagnosed in 2021 – long before ChatGPT and agentic AI (Smuha 2021). It might even be necessary to re-think traditional approaches to regulation given the many shapes and sizes in which AI comes and the pace at which it is being developed . Nor are the possible uses, misuses and risks of AI to  be defined in a single treaty or regional regulation. A risk based human centered legal instrument as the European AI Act bears the risk to be dead by arrival. Not in the last place because of the strategic importance of (controlling) AI, and the impossibility and unwillingness to maintain these regulations taking into account the variety of actors, states, big tech and, finally, the secret motives of AI itself.

References

Amodei, Dario (2026), We must pace the Frontier, essay, https://darioamodei.com/post/we-must-pace-the-frontier 

BBC News (31.7.2026) Anthropic's Claude AI escapes to hack into three organisations https://www.bbc.co.uk/news/articles/cz7dl7w8y7po

Bradford, Anu (2023), ‘Digital Empires’, OUP, https://doi.org/10.1093/oso/9780197649268.001.0001

European Commission (2019), Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions, ‘Building Trust In Human-Centric Artificial Intelligence’,  COM(2019) 168 final 

European Commission AI HLEG (2019), Ethics Guidelines for Trustworthy AI

European Parliament (2020)

Howarth, Tom (2026) Why AI agents invent their own language if you let them chat. https://www.science.org/content/article/why-ai-agents-invent-their-own-language-if-you-let-them-chat?referrer=https%3A%2F%2Fwww.google.com%2F .

Reuters (29.9.2026) Anthropic warns AI may pose 'existential risks to humanity' in IPO filing. https://www.reuters.com/business/finance/anthropic-warns-ai-may-pose-existential-risks-humanity-ipo-filing-2026-09-29/

Smuha, Natalie (2021), From a ‘Race to AI’ to a ‘Race to AI Regulation: Regulatory Competition for Artificial Intelligence’, Law, Innovation & Technology, Vol. 13 Iss. 1, 2021

OpenAI (26.8.2026) ‘Het Hugging Face-incident en de weg vooruit’. https://openai.com/nl-NL/index/hugging-face-incident-and-the-road-ahead/

The Guardian (27.9.2026) Bill Gates says unchecked AI could ‘cause a billion deaths’ in call for regulation. https://www.theguardian.com/us-news/2026/sep/27/bill-gates-artificial-intelligence-kristen-welker

The Guardian (26.9.2026) Trump and Chinese president Xi end summit without major agreement on AI. https://www.theguardian.com/technology/2026/sep/25/trump-xi-ai-arms-race

Illustrations created by the authors using ChatGPT.

Sustainable Innovation Needs Collaboration, But Are We Ready to Share?

Learnings from eColabor’s Sustainability for Science 2025 Special Session

Authors:

Yasmine Bounouara (Doctoral Researcher, Tampere University)

Rosa Ballardini (Professor, University of Lapland)

Dhanay Cadillo Chandler (University Researcher, University of Lapland)

Anwar Al-Hamidi (Doctoral Researcher, University of Lapland)

Jaakko Siltaloppi (Senior Research Fellow, Tampere University)

Olena Sushch (Associate Professor, University of Lapland)

Jarmo Uusikartano (Doctoral Researcher, Tampere University)

Previously published on the eColabor project website hosted by Tampere University.

Why Collaboration Matters

Sustainability-driven innovations are pivotal in reshaping our future by offering transformative solutions to the world’s most pressing social and environmental challenges (Adams et al., 2016). To achieve long-term sustainability, such innovations require efficient and equitable structures that encourage key stakeholders to collaborate. To date, however, there are multiple challenges related to the legal, policy and innovation management structures that govern sustainability-driven innovations (Altenburg & Pegels., 2012). To overcome sustainability challenges, it is necessary to look beyond the single field-approach and embrace a holistic view to find workable solutions to the complexities related to sustainable innovations.

To this end, the interdisciplinary eColabor project team, comprising researchers from law, policy, and management, conducts research and hosts events to foster dialogue and raise awareness on these critical issues. In October 2025, the eColabor team hosted a special session at Science for Sustainability (previously: Sustainability Science Days, see University of Helsinki 2025), Finland’s largest sustainability research conference, organized by the University of Helsinki. This year’s theme focused on unfolding new perspectives for sustainability transformations. Our interactive session, Envisaging Collaboration for Sustainable Innovation: Pushing the Boundaries through Law and Governance, explored the intersections of legal frameworks, business ecosystem management, and innovation, to promote multi-stakeholder collaboration for sustainability-driven innovations. The panel session brought together diverse voices from the academia and the industry, offering both theoretical and practical insights into the challenges and opportunities of cross-sector collaboration.

Rethinking Legal Frameworks for Sustainable Innovation

On the legal side, Professor Rosa Ballardini from the University of Lapland addressed the intellectual property rights (IPR) mismatch in the context of collaboration for innovation: on the one hand, IPR are essential to incentivise innovations, while, on the other hand, the legal monopoly they give to their owners acts as an obstacle to sharing and collaboration. Companies hesitate to share and even co-create, fearing appropriation or illegitimate reuse. As Ballardini noted, this often overly protective attitude can impede the partnerships needed to develop sustainable innovation. For this reason, Rolando Tomasini, Director of Partnership Development for Europe at PATH, emphasized the central role of defining knowledge ownership modalities as a founding step to collaboration. New licensing models are emerging to address this mismatch. One example is the Environmentally Sustainable Open Source (ESOS) license (Siltaloppi & Ballardini, 2023) which aims to promote environmental sustainability by incentivizing innovators to openly license their sustainability-oriented innovations. According to Ballardini, novel licensing models like the ESOS offer promising pathways for fostering collaboration and reimagining business practices.

From a policy perspective, Marie-Elodie Bourot, Sustainability Expert, underscored the role of regulation as a potential accelerator of collaboration. However, she argued that meaningful change requires laws that are both proactive and ambitious. For instance, Directive (EU) 2022/2464 on Corporate Sustainability Reporting was once a more ambitious initiative. The directive, initially developed to standardize sustainability reporting in the EU, has recently been eroded, highlighting the need for stronger regulatory commitments to drive sustainability transformation.

Making Multi-Actor Collaboration Work in Practice

Sustainable innovation carries significant managerial implications, particularly in how collaboration is orchestrated in practice. Presenters emphasized the importance of openness, inclusivity, and active engagement with diverse stakeholders. Collaboration was framed not just as a tool, but as a catalyst for innovation. As Rolando Tomasini put it, “to innovate, you need to listen”. Marie-Elodie Bourot reinforced this by highlighting the need to define who the stakeholders are to engage them accordingly. Aligning needs and expectations is essential for sustainability-oriented innovation, providing fertile ground for long-term collaborations. 

Concrete lessons from the business side are illustrated by Jaakko Tuomainen, Programme Manager at Borealis, a company leading the SPIRIT innovation project (see SPIRIT Programme 2025). This four-year programme, partially funded by Business Finland, aims to build a collaborative ecosystem to advance sustainability in the plastics industry. While acknowledging the functional value of plastics, SPIRIT’s mission is to develop solutions to reduce their environmental impact by extending their lifecycle. The project tackles themes such as carbon-neutral production and improved recycling rates, resulting in 21 sub-projects and the involvement of over 100 organizations. Being the facilitator, Borealis must frame the goals of the collaborations while aligning individual and ecosystem interests. Tuomainen emphasized: “the leading company should take an active role in the research scope, translating the business needs”. For instance, it is important to ensure clear communication with both internal and external stakeholders, fostering alignment, and enabling frequent, meaningful interaction throughout the project.

Key Takeaways

As Jaakko Siltaloppi, Senior Research Fellow, noted, “sustainable solutions require multi-actor collaboration.” Adopting an ecosystemic, collaborative approach to innovation is therefore essential. The session brought together academic and practitioner perspectives, illuminating the dynamic interplay between law and business management in the context of sustainability-driven innovation. Three key insights emerged from the discussion on collaboration for sustainability transformations in business, each highlighting the roles of policy, legal frameworks, and management:

1. Collaboration is a driver of innovation, particularly in sustainability, and requires open dialogue and commitment.
2. Legal and policy frameworks must be designed to actively support and enable collaborative arrangements.
3. Managerial practices should align diverse interests and engage stakeholders to foster commitment.

Nevertheless, critical questions remain. Importantly, how can we balance ecosystem-level interests with individual incentives (Adner, 2017) for sustainable innovation? This persistent tension reflects the need for systemic attention from both academic and practical domains to this issue.

References

Adams, R., Jeanrenaud, S., Bessant, J., Denyer, D., & Overy, P. (2016). Sustainability-oriented Innovation: A Systematic Review. International Journal of Management Reviews : IJMR, 18(2), 180–205. https://doi.org/10.1111/ijmr.12068.

Adner, R. (2017). Ecosystem as Structure: An Actionable Construct for Strategy. Journal of Management, 43(1), 39–58. https://doi.org/10.1177/0149206316678451.

Altenburg, T., & Pegels, A. (2012). Sustainability-oriented innovation systems - managing the green transformation. Innovation and Development, 2(1), 5–22. https://doi.org/10.1080/2157930X.2012.664037.

Directive (EU) 2022/2464 of the European Parliament and of the Council of 14 December 2022 amending Regulation (EU) No 537/2014, Directive 2004/109/EC, Directive 2006/43/EC and Directive 2013/34/EU, as regards corporate sustainability reporting.

Siltaloppi, J., & Ballardini, R. (2023). Promoting Systemic Collaboration for Sustainable Innovation through Intellectual Property Rights. Journal of Co-operative Organization and Management, 11(1), Article 100200. https://doi.org/10.1016/j.jcom.2023.100200.

Spirit Programme (2025). https://www.spiritprogramme.com/.

University of Helsinki (2025) Science for Sustainability 2025. https://www.helsinki.fi/en/conferences/science-sustainability-2025.

ChatGPT: A peek into the future of practical AI regulation

Author: Emmanuel Salami (Doctoral Researcher)

Research group: Law, Technology and Design Thinking 

Emmanuel Salami

Artificial Intelligence (AI) systems have received much attention from key participants in the global economy because of the unprecedented change apparent from their adoption. From a legal perspective, there is no paucity of legislative, judicial, regulatory, academic, and stakeholder position(s) on the topic. Even though there is evidence of AI use in various sectors of the global economy, it might be too early to describe such adoption as mainstream. [1] However, it would appear that this is about to change with the launch of ChatGPT.

ChatGPT is a state-of-the-art natural language processing model developed by OpenAI. It is a variant of the GPT-3 (Generative Pertained Transformer 3) model, which has been trained on a massive amount of text data to generate human-like responses to a given input.[2] ChatGPT uses unsupervised machine-learning techniques to create responses. In other words, it can generate responses without the machine learning algorithm being trained to respond in any particular way.

This notwithstanding, human input is needed to curate the information and thereby guide its output. Furthermore, ChatGPT makes AI readily accessible to the public, thereby creating a potential avenue for unravelling, on a large scale, some critical legal concerns previously expressed about AI systems. Therefore, this blog post focuses on some Intellectual Property Rights (IP, IPR) and data protection law concerns that might arise in using ChatGPT.

ChatGPT might raise some exciting IP considerations concerning its output. This is because the datasets used to train the AI system at the machine learning phase must have been generated through the works of authors who are most probably unaware of it. Though ChatGPT is proving to be very good at mixing and matching, time will tell if we potentially have a copyright action on our hands, should it replicate works attributable to other authors.

A relatable concern has been raised by Australian artists who accused an AI system that creates art of infringing on their artwork. [3] Their rationale is that their artwork had been used to train the AI system, and its elements are evident in the AI-generated art. It is arguable that, at some point, something like this might be possible, especially when it comes to AI-generated literary works such as (non) fiction books.

One of the schools of thought justifying IPR posits that its purpose includes the incentivisation of authors and the encouragement of innovation. [4] ChatGPT’s (potential) use of copyrighted works without adequate consideration for the incentivisation of authors can potentially hinder the ‘author incentivisation’ objective of IPR. Furthermore, IPR accords all authors moral rights in their works, which is an inalienable right to be consistently recognised as the author of the work. [5] ChatGPT, and by extension, AI’s (potential) use of copyrighted works, threatens this IPR principle. In addition, OpenAI has created an avenue in its terms and conditions for infringed copyrighted works to be taken down from the platform. [6] However, this is neither a sufficient attempt to incentivise authors nor resolve the IPR concerns identified above.

Despite the output of ChatGPT essentially being non-personal and publicly available data, data protection law remains relevant in its use. However, it would appear that processing (potentially sensitive) personal data does not take the data retention principle into proper consideration. To avoid doubt, the data retention principle (also known as the storage limitation principle) simply requires that personal data should neither be retained nor capable of identifying natural persons longer than necessary in relation to the purpose(s) of processing. [7]

The retention of personal data in ChatGPT raises two concerns for the data retention principle: firstly, when users delete their account on the platform, all information about the account is deleted, and they will be unable to reopen another account. Users are therefore encouraged to deactivate their accounts, making their data available on the platform. [8] The implication is that users might be forced to keep their data on the platform to avoid being prevented from creating an account in future.

Of course, the platform may claim that the legal basis for retaining the account details upon deactivation (instead of deletion) is the user’s consent. However, such consent is invalid because it is non-voluntary. After all, the user has no other option. [9] It is frivolous to assert that such retention is justifiable by the performance of a contract since it is not necessary for such performance. [10] Secondly, some data categories entered into the ChatGPT system cannot be deleted. [11] Although users are advised not to enter sensitive data into the system, this does not resolve the data retention concern, especially because such erroneously entered data will likely be available for machine learning purposes.

The scenarios highlighted above also raise some interesting concerns from the perspective of the data minimisation and purpose limitation principles which cannot be fully addressed within the scope of this blog post. Flowing from the concerns identified above, one can say that the terms and conditions and the privacy policy of ChatGPT are quite superficial and non-transparent and do not sufficiently address these concerns. If ChatGPT is to become a mainstream application, these concerns (and more) must be addressed, particularly in the EU, due to its extensive (proposed) legislation regulating personal data and AI. The lack of transparency becomes even more worrisome, given how well ChatGPT has been received and the proposed intention of some global tech players to incorporate it into their products. [12]

From an epistemic perspective, ChatGPT (like most other AI systems today) only recreates information from other existing data and is incapable of creating new knowledge. This is because it lacks the human consciousness needed for knowledge creation. As Zittrain notes, AI systems (including Chat GPT) “don’t uncover causal mechanisms, they are at best statistical correlation engines”, unlike human intelligence, which is needed for the investigation of problems and their causal effects. [13] ChatGPT’s status as a “statistical correlation engine” is one reason behind some of the superficial and wrong answers it has been known to provide. In addition, it cannot discern and verify the validity/correctness of the information, although this may also result from training the system with error-prone data. This highlights the risks of importing real-world errors and biases into the realm of AI, resulting in the propagation of misinformation. Therefore, it is necessary to ensure that some form of human review is mandatory in using ChatGPT.

As identified above, ChatGPT has some hurdles to surpass if it is to be adopted without legal and regulatory challenges. It is necessary to carefully consider these issues so that AI adoption does not result in the erosion of user rights. While the frenzy surrounding AI is understandable, developers will do well to sustain this excitement by ensuring that their products comply with applicable laws.

---

^[1]  See for instance - Next Rembrandt. <https://www.nextrembrandt.com/> accessed 06/01/2023.

^[2]  Shripad Kulkarni, Generative Pre-trained Transformer 3 by OpenAI. <https://link.medium.com/Rcb57QuWpwb> accessed 08/01/2023.

^[3] Cait Kelly, Australian artists accuse popular AI imaging app of stealing content, call for stricter copyright laws, (The Guardian.com, 11/12/2022). <https://www.theguardian.com/australia-news/2022/dec/12/australian-artists-accuse-popular-ai-imaging-app-of-stealing-content-call-for-stricter-copyright-laws?CMP=share_btn_link> accessed 08/01/2023.

^[4] Annette Kur and Thomas Dreier, European Intellectual Property Law: Text, Cases and Materials (Edward Elgar Publishing 2013) 5–10.

^[5] Art 6bis Berne convention.

^[6] OpenAI, Terms of Use, paragraph 3(d). <https://openai.com/terms/> accessed 9/01/2022.

^[7] Art 5(1) (e) GDPR.

^[8] Chat GPT FAQ, paragraph 7 <https://help.openai.com/en/articles/6783457-chatgpt-faq> accessed 9/01/2022.

^[9] Art 4(11) and Art 7 GDPR.

^[10] Art 6 (1) (b) GDPR.

^[11] Chat GPT FAQ, (n 8) paragraph 8.

^[12] Ryan Browne, Microsoft reportedly plans to invest $10 billion in creator of buzzy A.I. tool ChatGPT, (January 10, 2023, CNBC). <https://www.cnbc.com/2023/01/10/microsoft-to-invest-10-billion-in-chatgpt-creator-openai-report-says.html> accessed 11 January 2023.

^[13] Jonathan Zittrain, ‘The Hidden Costs of Automated Thinking’ (The New Yorker, 23 July 2019) <https://www.newyorker.com/tech/annals-of-technology/the-hidden-costs-of-automated-thinking> accessed 11 January 2023.

 

“To use or not to use, that is the question” – Mobile applications and the right to be forgotten

Author: Fouad Abdelrazek (LLD Candidate)

Research Group: Law, Technology and Design Thinking

Fouad Abdelrazek

We are living in a rapidly digitalizing world. Due to the increase in computing power of mobile phones, and the exponential growth of smart mobile applications for various purposes,[1] people from all norms of life depend on mobile applications to assist them in daily tasks. Moreover, the easy download and installation of mobile applications and their flexibility to be used anywhere, at any time, has engaged people in their use.[2]

Albeit being generally handy, mobile applications cause a set of privacy and security concerns. Mobile applications can collect large quantities of personal information from their many sensors, including location, biometrics, and other sensitive data. This information, processed together with the records of users’ interaction with the web service, could also be used to build users’ profiles and pose risks to their fundamental rights.[3]

In a mobile application ecosystem, when data is collected about, or from, a mobile device, the personal nature of mobile device usage implies that such data has to be considered personal data in the context of the General Data Protection Regulations (GDPR).[4] Nevertheless, does that mean such data is automatically protected in practice, and there is no need to heed caution?

Our culture of convenience often leads us to think that all we need to do to delete our data from the application is to delete the application itself from the device. However, while the active elements may be uninstalled, that doesn’t always mean that the personal data we’ve uploaded using the application has been deleted. Even if a message appears to warn us that deleting the application will also delete the data, this usually only means that the data will be deleted from the device itself, but it still exists on the developer’s server.[5]

If the user decided to delete his personal data, he has to use his right to be forgotten (RTBF) which is stated in Article 17 of the GDPR. However, it is also important to note that the RTBF is not an absolute right, and it only applies in certain circumstances.[6] As a result, the mobile application user will face many obstacles regarding this right. It is important to highlight that individuals cannot have their personal data deleted when they need to use the application if the application requires the use of the individual’s personal data for its intended purpose. In other words, the user cannot use the mobile application without providing it access to the necessary personal data. Also, users cannot simply withdraw their consent to provide necessary personal data as long as they need to use this application. This is due to the fact that the application will not function unless the user accepts and gives permission to the application provider to gather their personal data.

So, in order to truly protect our data, we are left with the question, “to use, or not to use?”, and a controversial decision of whether to take it or leave it. Either you give permission to various service providers, and possibly third parties, to use your “necessary” personal data, or you do not use the application that you may require for an educational purpose, for transportation, or even as your only means of connection to friends and family in various countries.

Despite service providers receiving legitimate consent from the users and even if they are being transparent about the use of data, users are still not fully in control of their data or aware of the privacy issues they may face. And although users must be given the option to change their wishes and revoke their decision at any time [7], they usually will not if their need for using the application wins over their privacy concerns. Hence, the available privacy protection privileges will not deny the truth that the provider of the mobile application is using and processing the user’s personal data to make him able to use the application. Consequently, it is important to know whether the use of mobile applications controls us or whether we are in control of our own use of mobile applications

We will find that most of the people that are using mobile applications generally need more than one application in their routine lives. This need forces people to give consent to the mobile application provider to access and process their personal data to function, even if they have concerns or would otherwise be cautious to do so. Accordingly, the need to use mobile applications is the controller of this relationship. Thus, not necessarily the technology itself, but the need for the technology that controls our usage and provision of data.

Accordingly, this will impact the effectiveness of implementing the RTBF on the used mobile applications. Since the erasure of identified, or identifiable, personal data from the mobile application could lead to the inability to use this application, users may no longer want to do so. This will lead to inefficiency in the usage of an important right that was given to the users, especially since it gives them the power to control their data. Not only that, but it could make the RTBF regarding mobile applications out of service.

As a result, it is important to raise awareness of how precious our personal data is and how to protect it to push toward the development of more transparent mobile applications. Such applications should allow for their usage with minimal data collection, provide more precise and simplified information on the usage of personal data, and allow opting out of unnecessary data collection, thus giving more control to individuals and their rights.  

References:

[1] Islam, R., Islam, R., & Mazumder, T. (2010). Mobile application and its global impact. International Journal of Engineering & Technology (IJEST), 10(6), 72-78.

[2] Nathan, S. S., Hussain, A., & Hashim, N. L. (2016). Studies on deaf mobile application: Need for functionalities and requirements. Journal of Telecommunication, Electronic and Computer Engineering, 8(8), 47-50.

[3] European Data Protection Supervisor (2016) “Guidelines on the protection of personal data processed by mobile applications provided by European Union institutions”.

[4] Castelluccia, C., Guerses, S., Hansen, M., Hoepman, J. H., van Hoboken, J., & Vieira, B. (2017). Privacy and data protection in mobile applications: A study on the app development ecosystem and the technical implementation of GDPR.

[5] Peters, B. What Happens to Your Personal Data after Deleting an App. https://techspective.net/2020/12/01/what-happens-to-your-personal-data-after-deleting-an-app/

[6] Information Commissioner’s Office (2018). Guide to the general data protection regulation (GDPR). Right to erasure. Retrieved from: https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/

[7] European Data Protection Supervisor (2016) “Guidelines on the protection of personal data processed by mobile applications provided by European Union institutions”.