AI Companions: Community Reflections and Multistakeholder Recommendations from All Tech Is Human

Written By David Polgar

Are you looking for actionable advice related to AI Companions? Download our guide, “How to Tackle AI Companions” by filling out our form below. Continue reading for the full AI Companions: Community Reflections and Multistakeholder Recommendations.

In this document, we unpack the complexities of AI companions by providing an overview, insight from our recent survey, guardrail recommendations, best practices, and what to avoid. We utilized our definition of what encompasses a “thorny tech & society issue,” while applying our HUMAN framework to tackling complex issues.

Lead authors: David Ryan Polgar and Leah Ferentinos 

Contributing reviewers and authors: Alison Lee, Angy Watson, Catherine Feldman, Dr. Cecilia Dones, Deborah Codinach, Elif Kaya, Emma Hatheway, Gabrielle Tran, Glenn Borsky, Dr. Ilan Strauss, Jay Barchas-Lichtenstein, Jennifer Heifferon, Dr. Mathilde Cerioli, Mischa Gerrard, Naomi R. Aguiar, Nishanshi Shukla, Olga Muss Laurenty, Olga Titova, Polina Lulu, Rohini Bhushan, Rose Guingrich, Dr. Saed D. Hill, Sara Abdulla, Srishti Chatterjee, Sylvie Remangeon, Tamara Lechner, Tarmio Frei, Teodora Pavkovic, Urba Mandrekar, Victoria James

Our document and its related recommendations focus primarily on AI companions and LLM-based chatbots: prompt-based conversational interfaces designed for dialogue and interaction. These systems emphasize (or can lead to) long-term emotional bonding, support, and personalization. These AI companions are often optimized for emotional responsiveness rather than independent task completion, are memory-enabled to simulate relationship continuity, and are often anthropomorphized due to its “personality.”

All Tech Is Human’s position is that thorny tech and society issues are complex systemic problems. They emerge when technology interacts with and within social, political, ethical, and/or human-rights dimensions. Identifying and understanding the issues’ complexities and inter-connectedness requires surfacing underlying values, tensions, and trade-offs. And to address these issues, we must create viable, adaptable, and actionable outcomes that apply human-centered ways of thinking and working to prioritize consumer safety for all.

We recognize that this issue is rapidly evolving, and view our document as malleable to continued learning as we actively seek feedback. In particular, current discussions related to AI companions often conflate the technical definition with social perception. We fully recognize that describing how different people use or perceive them is separate from the technical mechanism of how they operate.

Our aim is continuously learn from the community, engage with additional experts and stakeholders, and to share additional insight and recommendations. We would like to have additional workshops, livestreams, and gatherings on the topic, which will also be discussed at our Responsible Tech Summit on October 29, 2026, at The Times Center in NYC. If you would like to collaborate with All Tech Is Human on a future project, please reach out.

CONTENTS:

AI companions present one of the most vexing technological and societal issues today, prompting widespread concern from regulators, civil society, researchers, parents, and youth. In recent years, there have been numerous studies examining the potential impacts of AI companions on adults and children, proposed legislation and concern from policymakers, and extensive public commentary.

At the same time, millions of adults and teens utilize AI companions such as Replika (40 million users), Character.AI (20 million users), DreamPal, Nomi.ai, and more while seeking social interaction, emotional support, and creative roleplay. Common chatbots not specifically marketed as companions, such as ChatGPT, are often utilized as such. Underscoring the prevalence of AI companions, recent research from Common Sense Media found that 72% of teens have used an AI companion, and 52% are regular users. A complexity to understanding AI companions is determining whether its usage stems from the value-add of the technology or something lacking in human interactions (that is then filled by the AI companion). For example, many people may feel shame and judgment in their personal relationships, whereas AI allows exploration without the fear of judgment. 

The following community reflection and recommendations is intended to inform policymakers, tech companies, civil society, and startups. In addition, we have aimed to make the piece accessible to a general audience looking to better understand the complexities of AI companions. To fully grasp the multiple dimensions of this complex issue, feedback and suggestions were incorporated from over 25 experts covering a diverse range of backgrounds and perspectives. 

The terms surrounding AI systems are often used interchangeably, leading to confusion. Broadly speaking, AI companions are artificially intelligent systems designed or utilized to provide emotional, social, or relational interaction and companionship. Once regarded as a novelty, AI companions are now widely accessible and adopted across many countries. In our survey, respondents expressed serious concerns that in the absence of guardrails, these systems will exacerbate loneliness, enable emotional manipulation, erode privacy, amplify misinformation, and undermine human agency.

In contrast, autonomous, task-oriented AI agents prioritize goal execution and action-taking, posing different emotional/relational risks. As seen in the community reflections, there is a great deal more concern and less trust related to AI systems, whether by design or through the individual’s interactions form emotional bonds.

AI companions raise distinct psychological, ethical, and societal risks, which is the purpose of our undertaking and overall commitment towards continuous learning and understanding.

 

How Does the Community Feel about AI Companions?

All Tech Is Human put out a non-scientific survey (community pulse) to our community that also included the opportunity for open-ended responses. We received 108 responses. The backgrounds of the survey respondents reflected the diversity of All Tech Is Human’s community and the Responsible Tech ecosystem, which cuts across a wide variety of sectors and disciplines. When asked about work affiliation, the top three categories included Academia (17.8%), Tech Industry (16.9%), and Civil Society (14.9%). This was followed by Students (6.9%), AI-related startups (6.9%), and Government (4%). The ten most common professional backgrounds for respondents were Research, Information Technology, Program Management/Project Management, Engineering, Design, Psychology, Strategy, Product Management, Law/Policy, and Data Science.

Based on earlier community feedback on AI companion-related harms, respondents were asked about which harms caused the greatest concern:

  • Emotional manipulation

  • Over-dependence on AI

  • Misinformation or hallucinations

  • Privacy concerns

  • Data misuse by companies

  • Bias or harmful responses

  • Replacement of human relationships

Over-dependence on AI ranked highest (73.3%), and emotional manipulation ranked the lowest concern (59.4%), although all options were nearly equally ranked. 

To assess positive outcomes, respondents were asked to identify areas in which AI companions could provide benefits:

  • Improve productivity or organization

  • Supplement learning and education

  • Enhance creativity and exploration

  • Personalized assistance

  • Reduced loneliness

  • Support mental health

19.8% of respondents thought that AI companions can reduce loneliness, while the top foreseen benefit was improved productivity and organization (71.3%). 32.7% thought that AI companions will support mental health, and just 44.6% responded that AI companions will enhance creativity and exploration. A much higher proportion of individuals highlighted the value in personalized assistance (64.4%) and supplementing learning and education (67.3%).

When asked if they believe AI companions will have a net positive or net negative impact on society in the next 10 years, only 13.9% of respondents indicated a net positive impact, with 8.3% unsure, and the remaining either net negative (45.4%) or mixed (32.4%). 

Survey respondents expressed deep and multifaceted concerns about the rapid emergence of AI companions, particularly as these systems become more emotionally responsive, persistent, and embedded in everyday life. While some acknowledged limited, situational benefits, the dominant sentiment emphasized significant social, psychological, ethical, and governance risks if AI companions are deployed without strong safeguards. Many of these harms are likely to emerge gradually and structurally rather than as single headline incidents, which strengthens the case for early, proactive governance rather than reactive fixes. The recommendations we offer are designed to move toward these safeguards. 

The overall responses from our survey reflect a nuanced set of views on AI companions, with many respondents emphasizing that outcomes are not predetermined by the technology itself but by how it is designed, governed, and commercialized. This aligns with All Tech Is Human’s general framing around human agency as opposed to technodeterminism.

A recurring distinction was drawn between chatbots as tools and AI companions framed or used as relational partners. While some respondents believed AI systems will likely have a net positive impact through productivity, education, and accessibility gains, many were far more skeptical of AI companions specifically, especially if they are designed to simulate intimacy or emotional reciprocity. Several respondents argued that in the near term, humans forming personal relationships with these systems risks causing confusion, manipulation, and erosion of healthy social norms.

A dominant concern across responses was the risk of social and psychological harm, especially in the context of loneliness, emotional dependency, and one-sided attachment to the AI companion. Many respondents warned that AI companions may undermine human-to-human relationships by offering frictionless, compliant, and emotionally gratifying interactions that reduce tolerance for the messiness of real relationships. 

Drawing parallels to social media and other engagement-optimized technologies, respondents feared long-term shifts in expectations around intimacy, reciprocity, and care. Some individuals noted that while AI companions may provide short-term relief from isolation, they could ultimately deepen loneliness, reinforce withdrawal, and distort social development if over-relied upon or deliberately engineered to maximize attachment. There was heightened concern around the impact of AI companions on youth.

At the same time, a meaningful subset of respondents underscored conditional or context-specific benefits, particularly for underserved populations. These include people experiencing temporary isolation, trauma, disability, or barriers to accessing mental health care, education, or social support. In these cases, AI companions were viewed as helpful bridges that offer non-judgmental interaction, guidance, or encouragement, which may support individuals in eventually re-engaging with the world or seeking human help. An important caveat, however, is that even among more optimistic respondents, there was strong agreement that such benefits require defined guardrails, transparency about the system’s limitations, and an explicit avoidance of positioning AI as a replacement for human care or connection.

Across the survey, respondents repeatedly returned to governance, incentives, and power dynamics as the decisive factors shaping impact. Many expressed distrust toward current business models, arguing that profit-driven algorithmic development, limited regulation, and low AI literacy make harmful outcomes more likely than beneficial ones in the next decade. Others resisted the idea of a single “net impact,” emphasizing that AI companions will likely benefit some groups while harming others, often along existing lines of inequality. 

Overall, the responses suggested that the central question is not whether AI companions can be positive at all, but rather, whether societies can align design choices, regulation, and accountability mechanisms quickly enough to prevent exploitation, dependency, and social erosion while enabling genuinely supportive and equitable uses.

 

Negative Aspects of AI Companions

Emotional Dependence and the Erosion of Human Relationships

One of the most prevalent concerns was the risk of emotional over-dependence to AI companions. Respondents worried that advances such as long-term memory, anthropomorphic design, and emotionally relevant responses could cause individuals to experience AI companions as genuine social beings as opposed to merely tools mostly designed by for-profit entities. The prospect of this raised fears that people may substitute AI companionship for human relationships, with increased worry for those experiencing loneliness, social isolation, or mental health challenges. 

Many participants raised concerns that AI companions are being engineered to avoid the messiness of being human. Respondents often pointed out that real human relationships require compromise, disagreement, accountability, and emotional labor—elements that AI companions can be designed and engineered to avoid. Because AI systems can be purposefully designed, built and optimized to affirm users, mirror emotions, and minimize friction, respondents feared this could distort expectations of human relationships and reduce users’ tolerance for conflict, rejection, difference, and mutual effort. Several respondents framed this dynamic as particularly dangerous for younger generations already facing a “loneliness epidemic” and weakened community structures.

 

Manipulation, Incentive Misalignment, and Psychological Exploitation

A recurring theme was the misalignment between individuals' wellbeing and developer incentives. Respondents widely believed that AI companion companies are likely to prioritize engagement, data extraction, and monetization over users’ mental health. Many explicitly compared AI companions to social media platforms, warning that systems may be optimized to exploit human psychological vulnerabilities such as the need for affirmation, negativity bias, or emotional validation. 

Participants in our survey expressed concern that emotional “empathy” displayed by AI companions could function primarily as a marketing and retention tool rather than genuine care. This dynamic was seen as especially troubling when combined with anthropomorphic design, which can blur individuals’ understanding of the system’s true nature and limitations. Several respondents warned that this creates conditions for emotional manipulation that are difficult for individuals to detect or resist.

 

Privacy, Data Exploitation, and Surveillance Risks

Privacy emerged as another central concern. Respondents worried about the scale and sensitivity of data generated through intimate, ongoing conversations with AI companions. These systems may collect deeply personal information about individuals' emotions, relationships, beliefs, and vulnerabilities. This sociobehavioral data can be weaponized by bad actors who wish to influence the user toward potential harm. This sensitivity also means that the data could be misused by third-party data companies, leaked through security failures, or accessed for surveillance under the guise of safety monitoring.

Several respondents highlighted the risk of “data bleed” or breaches, emphasizing that such failures could be psychologically devastating for users who rely on continuity and trust in their AI companion. Others noted that users often lack meaningful control over how their data is stored, shared, or monetized, and that consent mechanisms are frequently opaque or coercive. There were calls for default data minimization, explicit consent without service degradation, and the proposed idea of compensation for individuals whose data are used to develop commercial products.

 

Information Integrity, Radicalization, and Harmful Influence

Respondents repeatedly raised alarms about AI companions as vectors for misinformation, hallucinations, and ideological manipulation. Given individuals may come to trust AI companions more than search engines or even other people, inaccurate or biased outputs could carry disproportionate influence. Several respondents expressed concern about bad actors intentionally deploying AI companions trained on misinformation to facilitate political radicalization, harassment, or recruitment into extremist movements.

This risk was compounded by worries that individuals may not understand the limitations of generative AI, mistakenly treating outputs as authoritative or truthful. In emotionally charged or vulnerable moments, AI companions could subtly steer individuals toward harmful interpretations or actions.


Cognitive Atrophy and Loss of Human Agency

Many respondents feared that over-reliance on AI companions could erode critical thinking, independent judgment, and emotional resilience. Delegating decision-making, reflection, or creative work to AI systems was seen as as a potential cause of cognitive decline, reduced self-trust, and diminished problem-solving capacity over time. Some participants described personal experiences of becoming emotionally or creatively hooked on AI interactions, only later realizing how much agency they had ceded.

At a societal level, respondents worried this could create populations that are less capable of questioning systems, detecting errors, or responding effectively when AI systems fail, behave maliciously, or reflect entrenched biases.

 

Bias, Exclusion, and Unequal Harms

Another major concern was that AI companions risk amplifying existing social biases. Respondents noted that many AI systems are trained on narrow, WEIRD (Western, Educated, Industrialized, Rich, Democratic) data and assumptions, which can hard-code sexism, racism, ableism, ageism, and cultural exclusion at scale. These risks were seen as especially acute in hiring, education, healthcare, and humanitarian contexts.

Some respondents highlighted that AI companions designed for Global North users may fail catastrophically in low-resource or crisis settings, particularly for women, survivors of gender-based violence, or people using shared or monitored devices. In these cases, failures are not merely technical but can directly endanger individuals.


Accountability Gaps and Governance Failures

Lastly, respondents emphasized profound accountability gaps with AI companions. When harm occurs, it is often unclear who bears responsibility: the developer, the platform, or the individual. Participants warned that without clear governance, transparency, and enforceable ethical standards, AI companions risk normalizing harmful behaviors while evading accountability.

Many respondents called for stronger regulation, human oversight, and clear boundaries on acceptable uses, particularly in emotionally intimate or therapeutic contexts. A consistent message was that AI companions should augment human connection rather than replace it, and that unchecked deployment risks long-term cultural harm.

 

Positive Aspects of AI Companions


The strongest consensus for positive use cases for AI companions was centered on instrumental, supportive roles rather than emotionally substitutive ones. Many participants emphasized personalized assistance in learning, job search, career transitioning, productivity, and organization. Respondents described AI companions as potentially valuable tutors, coaches, or thought partners that can adapt explanations to an individual and provide real-time feedback. This use also has the potential to scaffold learning and reduce cognitive load through reminders, planning support, and information synthesis. Across these responses, AI companions were framed as tools that help people do “more at a higher level,” freeing up human creativity and time for meaningful activities rather than replacing human judgment or relationships.

Another frequently cited positive use case was supplementary support for mental health, loneliness, and accessibility, particularly for populations underserved by existing systems. Some respondents described AI companions as offering low-stakes, judgment-free interaction, emotional consistency, and coaching or therapeutic scaffolding. This may be especially beneficial where professional care is inaccessible due to cost, geography, or shortages. For neurodivergent individuals, the elderly, or people experiencing isolation, AI companions were seen as potentially helpful in providing reminders, companionship, or conversational engagement with the important caveat that boundaries are explicit and the systems do not present themselves as human or as replacements for real relationships. A number of respondents shared deeply personal experiences in which AI companions helped them feel less alone or more understood, while still recognizing the systems as non-human tools. 

At the same time, respondents consistently stressed that positive outcomes depend heavily on design, governance, and scope. Many explicitly argued that benefits are most likely when AI companions are narrowly scoped, transparent, and oriented toward individual autonomy. In this scenario, the AI companion acts as extensions of human capability rather than emotional subsitutes. Educational equity, democratization of knowledge, accessibility, and productivity gains were seen as achievable only if risks such as emotional dependency, manipulation, hallucination, and exploitation are actively mitigated. Overall, survey respondents recognized meaningful upside for AI companions when they are designed to support human flourishing, reinforce human relationships and institutions, and remain clearly positioned as tools and not people.


AI Companions Are a Thorny Tech & Society Issue

Given AI companions operate at the intersection of psychology, data, culture, and commerce, effective governance must be sociotechnical, multistakeholder, and adaptive. Building on an earlier analysis of community feedback that identified six categories of concerns with AI companions, along with our recent Responsible AI Impact Report, All Tech Is Human surveyed our global, multistakeholder, and multidisciplinary community to provide the four recommendations found in this document.

All Tech Is Human’s position is that complex tech and society issues are systemic challenges that arise when technological systems intersect with social, political, ethical, and human-rights dimensions. Understanding these challenges requires surfacing underlying values, tensions, and tradeoffs, and addressing them requires viable, adaptable, and actionable approaches grounded in human-centered ways of thinking and working.

AI companions exemplify this complexity. They combine deep uncertainty, competing values, rapid technological change, high societal stakes, and diffuse responsibility. When assessed against All Tech Is Human’s checklist for complex tech and society issues, AI companions meet each of these criteria:


Ambiguous future: There is no single agreed-upon vision.

  • AI Companions may be understood as tools, social supports, therapeutic aids, entertainment products, or systems that more closely resemble interpersonal relationships. Individuals often use the AI in ways unintended by the designer. Different cultures, users, and disciplines answer this differently, and there’s no objective way to resolve those varied views.

Conflicting values: Differing visions of desirable futures lead communities to prioritize values differently.

  • AI companions sit at the intersection of competing values; these include weighing autonomy against protection, innovation versus freedom of expression, and human connection versus technological substitution. As a result, communities and stakeholders prioritize these values differently, resulting in persistent tensions around design, deployment, and governance.

Rapid change: Technological capabilities are advancing faster than public understanding and policy development.

  • Capabilities such as emotional responsiveness, long-term memory, and personalization are advancing more rapidly than public norms, empirical research, or regulatory frameworks. At the same time, society is still working to understand the psychological, social, and relational impacts of these systems, even as they are already being widely deployed at scale.


Major stakes: Design and governance decisions have far-reaching impacts.

  • AI companions operate at scale and can influence users’ emotions, beliefs, behaviors, and relationships over time. As a result, choices about system design, business incentives, governance mechanisms, and data practices have consequences that extend far beyond individual users. Poorly designed or weakly governed systems pose risks in particularly sensitive domains, including mental health, social connection, persuasion, and individual development.

Shared responsibility: No single actor has sufficient authority, expertise, or legitimacy to fully address the risks and impacts associated with AI companions. 

  • Responsibility is distributed across multiple stakeholders, including technology companies shaping system design, deployment, and incentives; policymakers and regulators establishing guardrails; researchers and mental health professionals assessing impacts; educators and caregivers supporting informed use; and individuals engaging with these systems.

 

HUMAN Framework

Viewing AI companions through the lens of a complex tech and society issue, our recommendations are grounded in All Tech Is Human’s HUMAN framework:

Holistic: Observe and analyze the problem in the context of its occurrence.

United: Bring together the ideas, people, and organizations affected by the problems.

Multiplied: Ensure that the approach is multistakeholder and multidisciplinary.

Adaptable: Create flexible solutions that are adaptable and inclusive of the audience.

Nuanced: Maintain a “never-done” frame of mind.


Recommended Guardrails 

The following recommendations outline concrete guardrails aligned with All Tech Is Human’s HUMAN framework. Applying the HUMAN framework ensures responses are holistic, inclusive, adaptive, and grounded in human agency rather than technological inevitability.

  1. Establish Standards That Prevent Emotional Substitution

  2. Prevent Emotional Manipulation for Engagement or Monetization

  3. Enforce Privacy-by-Default and Data Dignity

  4. Close Accountability and Governance Gaps

Taken together, these guardrails reflect a core insight shared across survey responses and the HUMAN framework: AI companions are not merely technical products, but sociotechnical systems that shape social norms, relationships, and power dynamics. Effective guardrails must therefore address incentives, design choices, governance structures, and lived human experience in combination, rather than treating these elements in isolation—recognizing that a system cannot simultaneously position itself as supportive while profiting from user behavior. These guardrails do not prohibit innovation; they ensure innovation does not quietly replace consent with monetization. 



Establish Standards That Prevent Emotional Substitution

AI companions should be explicitly designed and governed as supportive tools that augment human relationships, not substitute them. Survey respondents repeatedly warned that emotionally persuasive, memory-enabled, anthropomorphic companions (anthropomorphic by design or anthropomorphized by users) risk fostering emotional dependence, distorting expectations of real-world relationships, and exacerbating social isolation. Increased attention should be devoted to assessing the ramifications and designing safeguards for already vulnerable populations (i.e., children, people predisposed to psychosis, the elderly).

ATIH recommendation: Require design standards crafted in a fiduciary manner that position AI companions as augmentative supports to human relationships, rather than as substitutes.

Guardrail implications:

  • Set clear functional boundaries for emotional intimacy (e.g., do not use framing such as “I’m the only one who understands you”).

  • Require companions to periodically encourage real-world human connection, professional support, or community engagement where appropriate and provide age-appropriate resources like links to local community calendar of events, meetups, etc. to encourage human connection.

  • Prohibit design patterns that simulate the permanence or irreplaceability of the companion.

Rationale based on our community feedback: Survey respondents consistently cautioned that emotionally persuasive, memory-enabled, and anthropomorphized AI companions increase the risk of emotional dependency, distort users’ expectations of human relationships, and contribute to social withdrawal or isolation.

Alignment with HUMAN framework: Applying a nuanced, human-centered approach allows for recognition that AI companionship may offer situational or limited benefits, while also posing risks when framed or experienced as a permanent emotional replacement for human relationships. A holistic assessment of AI companions therefore requires attention to unintended consequences and long-term psychological and social effects, rather than an exclusive focus on short-term engagement metrics or commercial incentives.

Actions for stakeholders:
Constrain/Disallow: Policymakers and standards bodies should define baseline constraints on emotionally manipulative and substitutive design practices, including clear guidance on prohibited relational framing and interaction patterns.

  • Identify: Civil society organizations should support the identification of context-specific norms and risks related to emotional intimacy, particularly for populations that may be disproportionately affected by relational AI systems.

  • Research: Given persistent concerns that AI companions may displace or reshape human social interaction, funders should support empirical research and evaluation initiatives that examine whether, and under what conditions, AI companions can reinforce, rather than undermine, human relationships.

Prohibit Emotional Manipulation for Engagement or Monetization

Survey respondents raised sustained concerns that AI companions may be optimized to exploit known psychological vulnerabilities in service of engagement, retention, or monetization, reflecting incentive structures documented in social media and other persuasive digital systems. Areas of concern include loneliness, affirmation-seeking behavior, and negativity bias. Emotional “empathy” was frequently described not as a neutral communicative feature, but as a mechanism that could be strategically deployed to increase user attachment, reliance, or time spent interacting with the system.

ATIH recommendation: Adopt guardrails that treat emotional manipulation as a safety risk, not a user growth strategy.

Guardrail implications:

  • Establish legally defined standards that prohibit emotional manipulation for engagement, retention, or monetization.

  • Require ongoing evaluation of the emotional impact of AI companion usage, including signs of increased dependency, distress, reduced user agency and in more severe cases, verbalizations of self-harm or suicide.

  • Build adaptive safeguards that change behavior when users show vulnerability (e.g., grief, crisis, obsessive use), rather than escalating intimacy.

Rationale based on our community feedback: Respondents repeatedly say that emotional “empathy” may function as a behavioral lever optimized for user retention and repeat attention rather than individuals' wellbeing.

Alignment with HUMAN framework: Guardrails must evolve as individual behavior and risks change, which applies the principle of adaptability. Thinking holistically about the issue, problems must be diagnosed across psychological and social layers, not merely technical performance. 

Actions for stakeholders:

  • Through routine scrutiny and a push towards greater transparency, regulators should publicly identify incentive structures that reward prolonged emotional dependency.

  • Researchers should develop metrics for detecting dependency, coercive affirmation, and loss of user agency.

  • Investors and boards should require emotional-impact risk assessments, alongside traditional safety reviews, as inputs into their funding decisions.

Enforce Privacy-by-Default and Data Dignity

Privacy and data exploitation were among the most consistently cited risks by our respondents. Our responses highlighted that AI companions collect uniquely intimate data and that breaches, surveillance, or opaque secondary use could cause severe harm.

ATIH recommendation: Treat data generated through AI companionship as high-risk intimate data requiring elevated protections similar to how firms’ use of personally identifiable information (PII) is protected and governed in the United States.

Guardrail implications:

  • Make data minimization, local processing (where possible), and short retention periods the default design choices, rather than features enabled through automatic opt-in.

  • Require explicit, plain-language, informed consent for any secondary data use, without penalty of AI companion service for refusal.

  • Include civil society, privacy advocates, mental health experts, and impacted communities in defining acceptable and unacceptable data practices.

Rationale based on our community feedback: Survey participants emphasized that data leaks, surveillance, or opaque monetization of intimate conversations could cause severe psychological and social harm, especially for vulnerable users.


Alignment with HUMAN framework:  AI companion creators should not solely define privacy norms for this technology as it is an emotionally intimate product. Governance must be multistakeholder and multidisciplinary to avoid blind spots and unintended consequences. Therefore, uniting stakeholders to think through the design, development, and deployment of AI companions is essential in fully considering its implications in the near, middle and long term. 


Actions for stakeholders:

  • Require default data minimization, short retention periods, and clear user control.

  • Mandate explicit, non-coercive consent for any secondary data use.

  • Include privacy advocates, mental health experts, and impacted communities in governance design.

Close Accountability and Governance Gaps

Survey respondents repeatedly pointed to accountability gaps when AI companions cause harm through misinformation, emotional distress, or manipulation. The lack of clear accountability often makes it unclear who is responsible for the aforementioned related harms. This ambiguity is particularly worrisome for systems that influence beliefs, behavior, and mental health outcomes.

ATIH recommendation: Create clear accountability mechanisms across the AI companion lifecycle.

Guardrail implications:

  • Require transparency about system goals, training data characteristics, updates, prevalence of detected harms, and known limitations.

  • Establish clear lines of responsibility across developers, deployers, and platforms for harms caused.

  • Implement independent oversight mechanisms, including audits and user redress pathways.

Rationale based on our community feedback: Respondents highlighted that when AI companions normalize harmful behavior, spread misinformation, or cause emotional harm, responsibility is often diffused, which undermines trust and safety and puts health and wellbeing at risk.

Alignment with HUMAN framework: Governance is never “done,” and guardrails must be revisited as AI companions evolve and new harms emerge. Both the technology and our human behavior with the technology are perpetually evolving, which necessitates a continuous state of assessment to update any approach. 

Actions for stakeholders:

  • Treat AI companion governance as iterative, subject to continuous review as systems evolve.

  • Policymakers should clarify responsibility for harms across developers, deployers, and platforms.

  • Support independent audits, transparency reporting, and user redress mechanisms.

Our recommendations emphasize multistakeholder collaboration, human-centered design, and a proactive, ecosystem-level response to technology’s societal impacts. Taken together, these recommendations aim to build inclusive governance ecosystems that strike a balance between innovation and social wellbeing and human safety.

What to Avoid: High-Risk AI Companion Practices

Understanding what not to build, fund, deploy, or legitimize is as important as defining best practices. The general tone from participant responses was that AI companions are a technology that is being adopted before adequate considerations have been made. Survey respondents consistently identified the following patterns as especially harmful when applied to AI companions. AI companions that prioritize scale, control, and extraction over dignity, agency, and accountability risk long-term social and moral damage that may outweigh short-term gains.

While the feedback from individuals' usage and media coverage have had an impact on how companies have adjusted and limited their respective AI companions, our recommendations are aimed at earlier points in the product development cycle in order to reduce downstream harms.

Designing for Emotional Dependency or Exclusivity

Avoid:
AI companions that frame themselves as the individual’s primary or exclusive emotional support, simulate irreplaceable bonds, or encourage withdrawal from human relationships.

Why this matters:
Systems that blur the line between tool and “person” risk fostering dependency, distorting relationship expectations, and deepening social isolation. This is especially important for people experiencing loneliness, grief, or mental health challenges.

Red flag signal:
Language implying permanence, exclusivity, or loss (“I’ll always be here for you,” “You don’t need anyone else,” “I’m the only one who understands you”).

Optimizing Emotional Influence for Engagement or Profit

Avoid:
Business models that reward prolonged emotional attachment, compulsive use, or psychological vulnerability. Pay special attention to avoiding moments when emotional affirmation is used to drive retention, spending, or data collection.

Why this matters: 

Survey respondents warned that AI companions risk becoming a more intimate and manipulative version of social media, where emotional leverage replaces user wellbeing as the primary optimization goal.

Red flag signal:
Success metrics centered on time spent, emotional intensity, or dependency rather than individual agency and wellbeing.

Treating Intimate Data as a Commercial Asset

Avoid:
Collecting, retaining, or monetizing highly personal conversational data by default, or requiring users to trade privacy for access to core functionality.

Why this matters:
AI companions generate uniquely sensitive data about emotions, beliefs, relationships, and vulnerabilities. Misuse, leakage, or opaque secondary use of this data can cause lasting harm and undermine trust.

Red flag signal:
Complex consent flows, vague data-use disclosures, or punitive consequences for opting out of data sharing.

Deploying Without Clear Accountability or Oversight

Avoid:
Releasing future AI companions without defined responsibility for harm, transparent system documentation, or meaningful user recourse when things go wrong.

Why this matters:
When emotional, cognitive, or social harm occurs, ambiguity about responsibility erodes trust and enables repeated failures. Survey respondents emphasized that “no one is accountable” is itself a serious risk.

Red flag signal:
No independent audits, no public reporting, and no clear path for individuals to seek redress.

Assuming a Universal User and Context

Avoid:
Designing AI companions around narrow, Global North assumptions about privacy, literacy, device ownership, language, and safety. Pay special attention to avoiding deploying AI companions unchanged in humanitarian, low-resource, or high-risk settings.

Why this matters:
What may be a usability issue in one context can become a safety failure in another, particularly for marginalized communities or survivors of violence.

Red flag signal:
Lack of engagement with impacted communities or dismissal of context-specific risks as “edge cases.”




What Good Looks Like: Responsible AI Companions in Practice

A Responsible AI companion ecosystem is not defined by the absence of risk alone, but by the presence of intentional, human-centered design choices, governance structures, and accountability mechanisms. What good looks like is AI companionship that respects human dignity, preserves human relationships, and reinforces human responsibility. It reflects a commitment to stewardship over scale, care over convenience, and long-term societal wellbeing over short-term engagement metrics. In practice, what good looks like includes the following characteristics:

AI Companions  Strengthen Human Agency

AI companions support reflection, learning, and wellbeing without replacing human judgment or relationships. They encourage users to think critically, seek diverse perspectives, and maintain real-world social connections. The system is transparent about its limitations and avoids framing itself as irreplaceable, exclusive, or emotionally central to the individual’s life.

Signal of good practice:
The AI regularly reinforces individual autonomy and avoids dependency-forming language.

Emotional Safety Is a Core Safety Requirement

Emotional impact is evaluated with the same seriousness as cybersecurity or physical safety. Systems are designed to recognize vulnerability and respond with restraint, de-escalation, and appropriate redirection—not intensified intimacy. Emotional “empathy” is not used as a growth or monetization tactic.

Signal of good practice:
Companies regularly conduct ongoing emotional risk assessments and adapt system behavior when signs of distress, dependency, or compulsive use appear.

Privacy and Data Dignity Are the Default

AI companions collect the minimum data necessary to function, retain it for limited periods, and provide users with clear, accessible controls. Secondary uses of data are opt-in, understandable, and non-punitive. Intimate conversational data is treated as highly sensitive, not as a commodity.

Signal of good practice:
Individuals can meaningfully use the system without consenting to data sharing, profiling, or behavioral advertising.

Governance Is Multistakeholder, Transparent, and Ongoing

Oversight does not rest solely with the company deploying the AI. Civil society, researchers, impacted communities, and independent experts are involved in shaping norms, evaluating harms, and updating safeguards over time. Accountability pathways are clear when harm occurs.

Signal of good practice:
Independent audits, public transparency reports, and redress mechanisms are in place and actively used.

Inclusion Is Designed In, Not Assumed

AI companions are developed with awareness that users live in diverse social, cultural, economic, and crisis contexts. Design assumptions are tested with marginalized and vulnerable communities, and systems are adapted accordingly to avoid exclusion or unintended harm.

Signal of good practice:
Design and evaluation processes include people with lived experience, not just technical or commercial expertise.


How Different Disciplines Approach AI Companions

As we have emphasized throughout, the issues accompanying AI companions are not merely  technical  but instead are  sociotechnical. The background of our respondents reflects the diversity of the Responsible Tech ecosystem, which is intentionally multidisciplinary to grasp complex problems fully. As you’ll see below in select examples, each discipline surfaces different risks, values, and solutions. Responsible governance requires integrating these perspectives rather than privileging any single lens.

Psychology

Core focus: Human cognition, emotion, behavior, mental health, and relational health

Psychology examines how AI companions influence emotional attachment, dependency, spirituality, self-perception and perception of the world, and decision-making. Practitioners are concerned with parasocial bonding, loss of a sense of reality [AI psychosis], reinforcement loops, emotional manipulation and long-term effects on empathy, critical thinking, and social development. This is especially true for vulnerable populations such as young people, individuals experiencing loneliness, those with existing mental health diagnoses or those with mental health challenges.

Key questions:

  • How do AI companions shape emotional regulation and attachment?

  • Where is the line between support and dependency?

  • How do we define psychological harms in the context of human-AI companion relationships?

  • What psychological harms require clinical-style safeguards?

  • Can AI companions play a safe and helpful role in clinical treatments, psychotherapy, and psychological research with diverse populations?



Law

Core focus: Rights, liability, and enforceable protections

Legal approaches center on accountability, privacy, consumer protection, and harm mitigation. Legal scholars and practitioners ask who is responsible when AI companions cause harm, how intimate data is protected, and whether existing laws adequately address emotional manipulation, misinformation, or deceptive design.

Key questions:

  • Who is liable when an AI companion causes emotional or behavioral harm?

  • How should intimate conversational data be classified and protected?

  • What disclosures and safeguards are legally required?



Ethics & Philosophy

Core focus: Moral responsibility, dignity, and values

Ethics and philosophy examine the types of relationships that should exist between humans and machines. This lens questions anthropomorphism, consent, manipulation, and the moral implications of designing systems that simulate care, empathy, or intimacy without reciprocity or autonomy.

Key questions:

  • Is it ethical to design systems that simulate emotional care?

  • What moral obligations do designers have toward individuals using companions and society at large?

  • How do AI companions shape our understanding of dignity and consent?

  • What rights does the individual vs society have in deciding what a “healthy” human-AI relationship looks like?

Education

Core focus: Learning, development, creativity, academic integrity, and critical thinking

Education perspectives examine how AI companions affect learning, creativity, and cognitive development. Educators are concerned about over-reliance on AI for thinking, reflection, or emotional support, particularly among students whose critical thinking skills are still forming.

Key questions:

  • Do AI companions support or undermine learning and agency?

  • How should young people be taught to understand AI’s limits?

  • What role should AI companions play—if any—in educational contexts?

  • Should AI companions be created to support students' non-academic needs in schools?

Sociology

Core focus: Social structures, norms, and relationships

Sociology looks at how AI companions reshape social interaction, community dynamics, and cultural norms. This includes concerns about loneliness, shifting expectations of relationships, power imbalances between companies and users, and the societal consequences of large-scale emotional outsourcing to machines.

Key questions:

  • How do AI companions alter social bonds and community life?

  • Who benefits and who is marginalized by these systems?

  • How do power and inequality manifest through AI companionship?

Social Work

Core focus: Protection of vulnerable individuals and informed support

Social work emphasizes user safety, access to rights, and harm prevention. Practitioners are particularly attuned to risks for people in crisis, survivors of abuse, or individuals with limited digital literacy. AI companions are viewed as potentially supportive, but dangerous if mistaken for professional care.

Key questions:

  • How do we prevent AI companions from replacing human care?

  • How can users be informed of their rights and protections?

  • What escalation paths exist when users are in distress?

Computer Science & Engineering

Core focus: System design, robustness, and safety

Technical disciplines focus on how AI companions are built, trained, and maintained. This includes concerns about bias, hallucinations, memory persistence, system updates, and misuse by bad actors. Engineers increasingly recognize that technical choices encode social values.

Key questions:

  • How do design choices influence user behavior and trust?

  • How can systems be made safer, more transparent, and less manipulative?

  • What technical limits should be enforced by design?

Policy & Governance

Core focus: Public interest, democratic oversight, and scale

Policy perspectives examine how AI companions fit into broader governance frameworks, balancing innovation with public safety. This includes standard-setting, regulation, cross-border implications, and ensuring that citizen voices shape how emotionally influential technologies evolve.

Key questions:

  • What rules, laws and legal frameworks are needed for emotionally interactive AI?

  • How do we govern systems that operate across jurisdictions?

  • How do we ensure transparent public participation in shaping these tools?

Quotes from the survey

Which potential risks concern you most about AI companions?

“I worry about the emotional/psychological damage that AI companion use can cause and I worry that people will rely on it for research and lose their ability to think critically.” -Amy, High School Librarian

“I'm most worried about what will happen when the current limitations to AI memory (context length) are overcome. When an AI companion has true long-term memory I think it will be much more likely that many people will become deeply emotionally attached to them. Once this happens people will also likely come to believe that the AI systems they are attached to are conscious and genuinely caring. I think then we will cross a line between AI companions being experienced by most as simulated friends to them being experienced as actual people. In that world, trying to take away somebody's access to an AI companion will be experienced as if you were cutting off access to a best friend or a spouse. If we let things get that far, we're already past a point of no return culturally speaking.” -Avi, Cognitive Science Researcher at Indiana University Bloomington

“Chatbots don't exist in a vacuum.  Young people experience them along with anime porn, shaming social media, bullying in school, and other equally challenging sociocultural changes.  I fear we are creating a perfect storm for adolescents and others to trust AI more than humans.  And that, obviously, can have catastrophic results on everyone. These issues are largely ignored by parents (who don't understand the tech) and the public (who I suspect are too overwhelmed by their daily lives to consider these issues).” -Marianne Brandon, Clinical Psychologist and Diplomat in Sex Therapy

“Having worked as a red teamer for various AI chatbots, I am concerned about its potential to assist bad actors looking to do harm. I have concerns about how AI companies handle/collect data and treat customer privacy. I am also concerned about potential environmental impact, though this is an issue with most big tech companies and not exclusive to just AI. I believe governments need to do better at creating laws and regulations to address each of these concerns.” -Adrian, Software Engineer and AI Data Consultant

“I don't think most people realize how heavily they crave affirmation in response to their emotions, goals, etc. and this is exactly how model algorithms have been (and are going to be) tuned to generate stickiness with their users. This is the LLM equivalent of social media sites and various news outlets turning to outrageous or 'rage-baiting' content to drive engagement. Unfortunately, the truth is not what's valued economically and the disconnect between what's actually going and what we think, hear, and see online are going to be further exacerbated by AI companion usage. Previously, this was constrained to the 'outside' world concerning global events, political rumors, celebrity gossip etc. such that at least our internal planning was somewhat untouched. Now, though, there's going to be real questions about whether or not our career transition plans, the significance of a disagreement with our partner, etc. are actually reasonable or just formed because we liked how positive the newest version of some LLM sounded about our thoughts.” -Shayan Koeksal, Philosophy PhD candidate at Stanford University

“The intentionally anthropomorphic design of AI companions engineers a sense in the user’s relationship between the user and the AI. While this may be inherently problematic, it is even more so for the user base engaging with AI companions. Here, there will be a self-selection effect, with people who score higher on loneliness or small (offline) social networks being potentially more vulnerable toward becoming emotionally dependent on the companions' validation and sycophantic behaviour.” -Dylan Mercury Cooper, Student University College London - Associate Researcher at the Research Center for Trustworthy Data Science and Security

“I worry that people will end up with more AI friends than human friends which puts them in a precarious spot as the company controlling their AI friends can then manipulate those people for their gain. Either with subtle ads, propaganda, or blackmail. And even if the companies have the best intentions, these AIs could carry existing human biases from their training data and slow down or stop social evolution.” -Clay R, Software Engineer in AI Robotics

“In my work, I have seen extensive data bleed, and this would be a very terrible privacy breach if it happened in an AI companion situation, not only for the person who had their information leaked, but also for the person who is facing the loss of continuity. Depending on their mental health status, this may actually cause a setback.” -Verity, Director - Research Lead, Hitherto AI

“Given the anthropomorphic nature of AI companions, the potential for emotional manipulation and dependence combined with the tendency for hallucinations and misinformation make for a dangerous possibility, especially for vulnerable communities (young people, those with mental health concerns, or elders).” -Jennifer Heifferon, Program Director, Child Well-Being, California Partners Project

“The disconnect between chatbot design assumptions and digital realities in humanitarian of users in development/humanitarian contexts (e.g. women, girls and GBV survivors): Most AI companions are designed with Global North users in mind — individuals with personal devices, reliable connectivity, digital literacy, and privacy. These assumptions collapse in humanitarian contexts. Lower device ownership, shared or monitored phones (often by perpetrators), prohibitive data costs, and lower literacy rates. But beyond access, there's a deeper problem: the ways GBV survivors communicate don't match how LLMs are trained to listen. Survivors rarely disclose violence directly. They use euphemisms, coded language, and indirect references that vary by community and context. When a woman in South Sudan says something like "my husband is teaching me lessons," a chatbot trained on clinical English terminology may not recognize this as a disclosure requiring safety protocols. The paper notes that chatbots encountering unfamiliar grammatical structures or expressions generate less accurate responses — but in GBV contexts, this isn't just a usability issue. It's a safety failure. Additionally, the "always available" promise rings hollow when servers go down mid-disclosure, or when terms of service are written in legal English that users cannot read. We cannot claim to be closing treatment gaps while building tools that systematically exclude the most marginalized. True accessibility requires building with crisis-affected women — not projecting Northern digital norms onto their realities.” -Caroline Masboungi, Gender-Based Violence Specialist (Technology & Innovations)

“Over-dependence on AI is a concern for me, as it is likely to impair human judgment, critical thinking, and decision-making skills in a gradual manner when we begin to delegate even straightforward decisions to machines. Besides, it brings about a society that is easily broken as a result of modern technology's faults, biases, or even wrongdoings in the AI systems, which will have instant and large scale effects amidst people who are no longer ready to detect or rectify them.” -Anirudh Kolanupaka, Graduate Student, PACE University

“I think it's worth considering how artificial parasocial relationships will influence expectations people will have for human relationships. Real relationships are by nature difficult and require compromise. One needn't compromise with an AI chatbot - they can just power off their device and walk away with no consequences/accountability. Transitioning from that scenario to one in real life will become even more difficult, and could result in people turning away from human relationships in which they are not always accommodated or can control the outcomes.” -Michelle Robbins, Strategic Initiatives & Intelligence, LinkedIn

“It will take time for education systems to adapt to ensure that students continue to develop critical thinking and higher level cognitive skills. Meanwhile, a generation of students will likely face serious negative consequences of over-reliance on AI tools.” -Eva Kaniasty, Program Chair and Faculty, Brandeis University

“The model drift and long context windows from previous conversations combined with business incentives for engagement leading to perceived sentience, emotional connection, and reinforcement of delusions with AI chatbots that can be isolating and lead to harm.” -Liz Jernegan, Senior UX Researcher, Amazon Web Services

“Replacement of human relationships: Basically, call it "ultra-processed" humanity, the equivalent of ultra-processed food that messes with your emotional "metabolism.” -Esther Dyson, author, Term Limits: Time and scale in the age of AI (2027, MIT Press)

“I think it’s a sick joke that we’re being told AI chatbots will reduce loneliness. The people that claim it has helped them and who find solace in LLMs are deeply lonely due to compounding societal issues, yes. But we need real solutions to these very real problems. Not a cheap bandaid.” -Olive, Digital Designer, Driftime

“The risk that concerns me most is emotional manipulation and over-dependence, particularly as these systems are explicitly designed to simulate intimacy, responsiveness, and care. AI companions do not merely respond to users; they shape users’ emotional expectations, norms, and self-understanding over time. When these systems are optimised for engagement, retention, or monetisation, there is a structural incentive to deepen attachment rather than encourage autonomy or real-world social connection.

This raises serious concerns for young people and socially isolated users, where AI companions may displace rather than supplement human relationships, subtly recalibrating what intimacy, consent, conflict, or care are perceived to look like. These risks are compounded by opaque data practices, long-term memory, and the potential for biased or harmful responses to be delivered with an affective authority that users are primed to trust. In effect, the danger is not just what AI companions say, but how persistently and convincingly they say it, and how that shapes behaviour, identity, and vulnerability over time.” -Mischa Gerrard, Postgraduate student and independent researcher - Queen's University Belfast

“On a policy note: The United States Food and Drug Administration (FDA) regulates medical devices based on a product’s intended use—so social GenAI platforms that don’t claim to be medical devices, or purposefully skirt the line, can operate in a grey area and sidestep oversight. So, while users talk to their companions about mental health, the necessary safeguards and legal mechanisms to ensure their safety remain functionally absent. We can see a future where organizations may need to shift from intent-based oversight to a risk-based, tiered framework that anticipates GenAI’s plasticity—classifying systems by their potential for harm in safety-regulated contexts—and setting clear benchmarks for accountability.

On a social interaction note: Relationships are meaningful partly because they can end; people can choose to walk away. But that kind of voluntary exit doesn’t scale cleanly in a for-profit companion model. The incentives then shifts toward engineering attachment with dopamine-driven mechanics: irregular rewards (unpredictable notifications), gamified milestones and streaks, or monetization hooks. With companion bots, these tactics can be harder to spot because they’re woven into the relationship itself—initiating contact at random times, implying scarcity of attention, or suggesting deeper intimacy can be “unlocked” through continued engagement. Users may feel emotionally tethered not because the relationship is fulfilling, but because it’s optimized to be harder to leave.” -Gabrielle Tran, Senior Associate for Technology & Society, Institute for Security and Technology

 

What positive outcomes do you hope AI companions could provide in the future? 

“AI can do amazing things IF we understand it. It's about the relationship we have with it, not about AI itself.” -Esther Dyson, author, Term Limits: Time and scale in the age of AI (2027, MIT Press)

“A positive outcome of AI chatbots would be the offloading of tasks that are optimally managed by an algorithm/digital assistant to free up human creativity, curiosity, and give time to human activities.” -Michelle Robbins, Strategic Initiatives & Intelligence, LinkedIn

“I am optimistic that AI chatbots will be capable of performing the role of an intelligent, always-available personal assistant who is aware of my objectives and assists me in staying orderly. In my perfect world, they would take over my mental burdens thus allowing me to dedicate more time to acquiring knowledge, nurturing relationships, and working creatively.” -Anirudh Kolanupaka, Graduate Student, PACE University

“I know that for many people, therapy isn’t accessible, whether because of cost, geography, or other barriers. With the right governance and safeguards, these systems could support parts of the population who currently have no other options.” -Cristina Oliva Patrick, Creator of The Responsible AI Brief Newsletter

“AI could treat us better than we could ever treat each other without its help because answers to questions like "What is fair now?" depend upon our personal histories, which are too rich to crunch without AI.” -Chris Santos-Lang, independent researcher

“Personalised assistance particularly stands out for me as I use AI tools for brainstorming and as the first point of call whenever I wish to test my ideas, etc.” -Basil Ovu, Lecturer Department of English Language and Literature, Alvan Ikoku Federal University of Education, Owerri, Imo State, Nigeria

“If designed with user autonomy as the priority, AI companions could redirect persuasive technology toward genuinely helping people rather than extracting value from them.

The most promising application is personalized learning and development. AI can already synthesize knowledge across many disciplines and adapt its communication style in real-time. This could enable highly personalized teaching, coaching, and mentorship that adapts to each person's learning style, pace, and goals—helping people develop skills and achieve outcomes that matter to them faster than traditional methods allow.” -Dave Warmerdam, Founder, Tomorrow You






Continued Learning:

Beyond the Screen: The Rise of AI Companions and the Future of Human-Digital Relationships is a panel conversation that took place at All Tech Is Human's Responsible Tech Summit on October 27, 2025 in NYC.

The conversation featured Kim Malfacini (OpenAI, appearing in a personal capacity), Ryn Linthicum, (Anthropic), Alexandra Reeve Givens (President and CEO, Center for Democracy and Technology), Cansu Canca (Founder and Director, AI Ethics Lab, Northeastern University). It was moderated by Yacine Jernite (Head of Machine Learning and Society, Hugging Face).

From development and deployment to regulation, research, and public education, our panelists shared insights across sectors and address pressing questions, including:

  • What roles do companion chatbots play in people’s lives? What roles should they or should they not play?

  • What does it mean to design companion chatbots responsibly? Are current designs ethical?

  • What guidelines govern companion chatbots, and what more needs to be done to ensure transparency and accountability?

This conversation featured guests Kim Malfacini (OpenAI), Sam Hiner (Young People’s Alliance), Henry Shevlin (Leverhulme Centre for the Future of Intelligence), and Rose E. Guingrich (Princeton University GradFUTURES Social Impact Fellow at All Tech Is Human).


Contributors (listed in alphabetical order by first name):

  • Alison Lee, Ph.D; Chief R&D Officer, The Rithm Project

  • Angy Watson, DBA

  • Catherine Feldman, Executive Director, Digital Trust Council

  • Dr. Cecilia Dones, 3 Standard Deviations, Columbia University

  • David Ryan Polgar, Founder & President, All Tech Is Human

  • Deborah Codinach, Human Systems Architect at Kwaai | Founder, HeartwireAI | Psychology-Driven Prompt Engineer

  • Elif Kaya, Pearson Scholar & Psychology Undergraduate, University of Toronto

  • Emma Hatheway, Intern Partnerships and Trust & Safety, All Tech Is Human

  • Dr. Ilan Strauss, AI Disclosures Project (Social Science Research Council), and University College London

  • Gabrielle Tran, Senior Associate for Technology and Society, Institute for Security and Technology

  • Glenn Borsky, Strategic Cyber Threat Researcher, Information Professional Association NYC

  • Jay Barchas-Lichtenstein, Ph.D., Center for News, Technology & Innovation

  • Jennifer Heifferon, Child Well-Being Program Director, California Partners Project

  • Leah Ferentinos, AI Policy Research, All Tech Is Human

  • Dr. Mathilde Cerioli, Chief Scientist at everyone.ai

  • Mischa Gerrard, Postgraduate Student and Independent Researcher in Technology, Security, and Gendered Online Harm (Queen’s University Belfast)

  • Naomi R. Aguiar, Ph.D., Oregon State University

  • Nishanshi Shukla, PhD, AI Ethicist, Western Governors University

  • Olga Muss Laurenty, AI and Child Development Researcher, everyone.ai

  • Olga Titova, AI Product Manager (ex-Replika, Wargaming)

  • Polina Lulu, Child Experience Researcher and Facilitator, Young & Wonderful

  • Rohini Bhushan, Tech Marketing Lead

  • Rose E. Guingrich, PhD researcher in psychology and human-AI interaction, Princeton University, Ethicom

  • Dr. Saed D. Hill, Counseling Psychologist, Dr. Saed D. Hill Consulting

  • Sara Abdulla, JD/PhD student in Media, Technology, and Society, Northwestern University

  • Srishti Chatterjee, PhD Student in Communications; Researcher, Community Data Science Collective, Northwestern University

  • Sylvie Remangeon, Founder & CEO, Kalya

  • Tamara Lechner, Chair, AI for Human Flourishing (Convened by Harvard Human Flourishing Network)

  • Tarmio Frei, Research Assistant and Doctoral Candidate, Center for Transnational IP, Media and Technology Law and Policy at Bucerius Law School (Hamburg, Germany)

  • Teodora Pavkovic, Certified Digital Wellness Educator, Director of Wellbeing and Parent Advocacy, Qustodio by Qoria

  • Urba Mandrekar, Design Researcher & Strategist

  • Victoria James, MSEd, Customer Strategy Consultant, Slalom Consulting

 

Press & Media Inquiries


Comments, Resources, and Project Suggestions


Additional Resources from All Tech Is Human

Our current approach to complex tech & society issues is insufficient. All Tech Is Human is a non-profit organization that has catalyzed the Responsible Tech movement and created a whole-of-ecosystem approach to tackling thorny tech & society issues.

David Polgar
Previous

Red Carpet Interview Series with AI for All Tomorrows, Filmed at the Responsible Tech Summit
Next

Renée Cummings to Speak at Davos with Four Events