Philosophers Develop AI-Based Teaching Tool to Promote Constructive Disagreement (guest post)
One way of thinking about the job of a philosophy instructor is that it’s about teaching students to disagree well. Yet when it comes to some moral, social, and political issues, students may seem reluctant to voice their own views in the classroom, let alone argue about them there.
To help encourage and facilitate constructive disagreement among their students, a pair of philosophers have developed a new teaching tool: an AI-based chat platform that has already shown some promising results, and that they are making available to other teachers for free.
In the following guest post, Simon Cullen and Nicholas DiBella (both at Carnegie Mellon) introduce us to this technology, which they’ve named Sway.
[Laura Owens, Untitled (detail), 2016]
Sway: an AI-Based Teaching Tool to Promote Constructive Disagreement
by Simon Cullen and Nicholas DiBella
Over half of American college students are afraid to discuss the Israel-Palestine conflict on campus, according to the Foundation for Individual Rights and Expression. Other important issues like abortion, gun control, and affirmative action aren’t that far behind. Clearly, campuses need more respectful, scaffolded environments where students can practice constructive disagreement, honing skills like intellectual humility, perspective-taking, and critical thinking.
We created a new kind of chat platform—called Sway—to address this need. Sway connects pairs of students who disagree over topics chosen by their instructor and then uses AI to facilitate more open, reasonable conversations between them.
Sway scaffolds discussions in two main ways:
- Discussion guidance. An AI Guide participates in every chat. We’ve designed Guide to de-escalate tense moments, ensure students aren’t talking past each other, and make sure everyone’s voice gets heard. More importantly, Guide aims to improve student reasoning: it poses challenging questions, prompts students to clarify vague or incomplete arguments, unearths implicit assumptions, detects tensions and inconsistencies, and provides relevant factual information.
- Charitable rephrasing. When a student composes a message that contains unconstructive language, the platform suggests a better way for the student to make their point. This feature aims to preserve the core meaning of the original message while providing immediate feedback to help students develop a habit of clear and respectful communication. Students are free to dismiss suggested rephrasings, but doing so will invoke Guide; this ensures the conversation doesn’t get derailed.
You can see Sway in action in this demo video, in de-identified transcripts from actual student discussions, and (for interested philosophers) this snippet of a metaphysical Sway chat with Alan Hájek. You can also check out student feedback on our website.
Using Sway with students
Instructors can use Sway with their students in several ways. One of the coolest we’ve seen is as preparation for in-person discussion sections. For example, one instructor ran weekly 30-minute “discussion assignments” on Sway that started at the beginning of the week and finished a day before students met for in-person sections. Students came better prepared and their discussions were, we’re told, more thoughtful. It can also be offered as an optional course component—with no deadlines or time limits—for students who might want to continue discussing course topics with each other outside of class.
We’ve built Sway to be easy for instructors to use: the platform automates the administrative work involved in offering a discussion-based course component. This 3-minute explainer video walks through setting up a discussion assignment for students. Instructors can find more info on our website, including language that can be adapted for syllabi and course announcements.
After an instructor inputs potential discussion topics, Sway elicits student opinions and then tries to find every student a chat partner who disagrees with them. When this isn’t possible, it pairs students in other ways. For example, Guide might lead an exploratory discussion between two students who each have “no idea” about a topic. Or, if students who share the same opinion have to be matched, one is randomly selected to play devil’s advocate—and, crucially, is supported in this role by Guide. That said, we’ve found that the vast majority of students do get paired with a chat partner they disagree with. (When a class has an odd number of students, the platform creates one group of three.)
Sway notifies students of new messages, reminds them about upcoming deadlines, administers personalized post-chat quizzes to evaluate how well they understand each other’s arguments, and collects post-discussion feedback (see the figure below for results). To encourage students to talk freely, chats are private and encrypted and their instructors can’t read them. However, the platform collects post-chat student feedback and analyzes anonymized transcripts to provide instructors with a high-level summary of each discussion as well as any common themes that emerged across the class. These findings are then summarized in Instructor Reports.
Background
Grounded in the approach Simon developed for his award-winning CMU course, Dangerous Ideas in Science and Society, Sway builds on our research on improving reasoning and communication. The success of Dangerous Ideas demonstrated that students thrive when they’re given a scaffolded environment that fosters charitable and rigorous discussions of controversial topics. Unfortunately, as with many approaches to promoting better student discussions, Dangerous Ideas required substantial resources: with the help of a 15-person teaching staff that included specially trained TAs, Simon could teach at most a few hundred students per semester. With Sway, we aim to make it easy for instructors everywhere to give students the same kind of challenging and rewarding discussions regardless of what resources are available.
We’re also excited to offer Sway for broader campus activities—including first-year orientation, residential community dialogues, student government, and to enhance events that promote debate and civil discourse. In this last case, events can be followed by Sway chats that let students put ideas into practice. Rather than discouraging controversial student speech, we hope Sway will make it easier for campus leaders to turn disagreements between students into opportunities for learning and community building.
Many educators, ourselves included, worry about AI undermining learning. But we don’t think this is a risk with Sway. To the contrary, we’ve designed Sway to be demanding, to challenge students to engage intelligently with important disagreements—not to outsource their thinking to ChatGPT. This is also why copy-paste is disabled in Sway: students can only type or dictate their messages. We’ve found this all but eliminates AI-based cheating. And to be clear, students can’t complete discussion assignments by chatting only with Guide; they have to talk to each other.
We address other common questions in our FAQ, and we recently discussed Sway’s backstory on the Teaching in Higher Ed podcast.
Early results
Sway’s development has been guided by our empirical studies involving cross-partisan discussions of divisive topics, including January 6 and the 2020 election, Israel’s war against Hamas, American support for Ukraine, and U.S. abortion policy. These studies provided strong initial evidence that AI-scaffolded discussion can be a powerful tool to help people connect more constructively across moral and political divides.
Since then, we’ve also seen an extremely positive response from the first students who have used Sway. For example, around 250 UCLA students enrolled in a sociology of gender class discussed a range of contentious topics, including: the appropriate public response to educational disadvantages faced by boys and men, whether it’s Islamophobic to oppose mandatory hijab laws, and whether Kamala Harris’s defeat is best explained by misogyny. Despite the potentially divisive subject matter, the vast majority of students rated their chat as “Awesome” or “Good,” and virtually all agreed that their partner was respectful (see figure below).
These self-report data are consistent with student behavior: of the tens of thousands of messages students have sent, only 0.3% have been flagged by the platform as potentially unconstructive. Reproducing Simon’s classroom findings, nearly all students felt their partner was respectful and genuinely tried to understand their perspective. This is noteworthy because “gender” is consistently high on lists of difficult-to-discuss topics according to national student surveys.
Students can also volunteer as research participants, allowing us to use their de-identified chat and survey data to improve Sway and rigorously study how to promote great discussions—especially across moral and political divides. We expect this data to be valuable far beyond our own work (e.g., in social psychology, human-computer interaction, conflict studies, and so on), and we invite interested researchers to get in touch with us.
Conclusion
We’re collaborating with faculty, campus leaders, practitioner organizations, and funders to normalize reasoned conversations between students with opposing perspectives. Over 100 faculty members collectively teaching around 9,000 students have registered to use Sway thus far. We expect—and hope to demonstrate empirically—that, like many students who took Simon’s Dangerous Ideas course, students who use Sway regularly will transfer its lessons. They may come to genuinely believe and even act on J.S. Mill’s famous observation: “He who knows only his own side of the case knows little of that.”
For many students, this would be a significant transformation.
If you’re a college instructor or campus leader who would like to use Sway with your students, you can sign up for free here.
Below is a video demonstrating Sway in use.
Many questions.
I believe that to move your pedagogical practice to this sort of model is to collaborate in an ongoing process that is destroying our known form of life and replacing it with a completely unknown and less human one. Alternatively, we as teachers could just host lots of debate/discussion sections where we, and not machines, facilitate difficult conversations. Where we, and not machines, are the solution to the problem of collective silence over difficult topics.
Well said.
“Are we forgetting that the development of online sociability always makes people less able to socialize in-person?”
Source needed.
More generally, why do philosophers loathe technology so much?
Heidegger.
“More generally, why do philosophers loathe technology so much?”
Source needed.
I understand the antipathy for having AI systems do work heretofore done by professional philosophers. I very much like being employed to do that sort of work, and the work is scant enough as it is.
But while AI-based philosophy might make it harder to make a living as a professional philosopher, that’s only insofar as it would make it easier for the public to enjoy the benefits of philosophy. It’s hard for me to dismiss that consideration. If we can make AI systems which can do some sort of work in a way that requires few societal resources, what business do we have demanding society expend significantly greater resources to have us do the work instead?
One can, of course, doubt that the AI systems are really up to the task. Maybe the limitations of the large language models––or of the sterile text-based environment––will mean that AI systems cannot fully replace us. If so, great! Then we can keep our jobs, and with clear consciences.
But even if we’re solidly better than AI systems, AI systems are solidly better than nothing. And nothing is exactly what many people who might be interested in philosophy currently have access to. This project has the potential to substantially expand the public’s access to the sort of measured reasoning we cherish. That would be a paramount good.
You make a really important point here about accessibility. I’d only add that the quality of moderation we see on Sway already often exceeds what I’m capable of providing myself (even if I had limitless energy and patience!)
Thanks for your comments—I’ll take them in turn.
(1) Online, text-based communication plays a huge role in students’ studies, daily lives, and future jobs. In fields like tech, finance, media, etc., work revolves around instant messaging and email. (For example, over a billion messages are transmitted via Slack every day.) So if students only learned how to communicate and reason more effectively when engaging in text-based chats, we think that’d be no small accomplishment. However, as we mentioned, we hope and expect that students will transfer skills that they hone on Sway to other contexts—a hypothesis we plan to test empirically. We also plan to introduce voice chat as more advanced speech-in-speech-out models become more affordable and capable.
(2) It does not “seem obvious” to me that having humans moderate all discussions is better. For one thing, many students comment on the benefits of Sway compared to in-person discussion. A couple of examples from recent anonymous student feedback:
“Sway makes it much easier because it is not face to face and it is easier to express your opinion.”
“I love talking on Sway because I have more time to compose my opinions in an organized and articulate way than if I were talking in class.”
We suspect that many students appreciate the low-stakes chat environment and that it will be especially helpful for preparing reticent students to participate more in classroom discussions.
But putting all that aside, the suggestion that “we as teachers just host lots of debate/discussion sections” is wildly impractical unless you advocate for (a) extremely low wages for the proposed teachers, (b) very large discussion groups for the students, or (c) leaving them more or less to their own devices.
Hiring a 16-person teaching team for a single undergraduate class was basically unprecedented in CMU’s philosophy department, and as you might imagine, it’s bureaucratically and financially nontrivial to do. Wages for TAs and graders came to around $40,000 last time we did it, and my department could only afford that thanks to the financial and administrative support of a generous college dean who also happens to be a philosopher.
Given the cost and scarcity of grad students, I had to rely almost entirely on undergrad TAs and graders. And while undergrads are somewhat easier to find and a bit cheaper to hire, they also tend to be less reliable and to require much more training and oversight than grad students. This is a time commitment that many instructors are unable or unwilling to make. When I was teaching a class of just 200 students, I had little time left to focus on anything else—and that’s with a job that allows me to devote nearly all my time to a single class.
So putting aside your strange remark about “explicitly celebrating the forced redundancy of 15 hypothetical TAs,” the relevant choice is rarely whether to use AI discussion facilitation or hire 15 more TAs. More likely, it’s between keeping the class small or running a large class without regular opportunities for meaningful, scaffolded student discussions.
Finally, even with more TAs than any other class in my department, nearly all student discussions in my class still proceeded partially unsupervised. This is because when you’re teaching controversial material, asking a room full of undergraduates to articulate and defend their sincere views or to try on different views for size will probably result in something close to a replication of Solomon Asch’s conformity studies. Of course, that’s also part of the reason for our focus on student privacy.
If these arrangements are this difficult to organize even with strong administrative support at a wealthy R1 university, I suspect they are effectively impossible elsewhere. And even at CMU, I’ve had to leave hundreds of students stuck on the waitlist—students who will not get to take the class.
(3) I’m aware of methods to bypass copy-paste controls, but I’m pleased to say that we haven’t seen any evidence of students or recruited experimental participants using them. If this changes, we have server-side controls and analytics that can easily detect and report this kind of cheating. But again, we’ve found that a simple tweak—disabling copy-paste—has meant virtually all students are more motivated to chat than to cheat.
“Are we forgetting that the development of online sociability always makes people less able to socialize in-person?”
Apart from the empirical questionability of this claim—and apart from the fact (as Simon noted in his reply) that online communication plays a huge role in students’ actual lives—we should not be denigrating the value of online discussions. For many people, especially disabled people, online communication is often the only form of communication that is possible or practical. It’s exclusionary to regard the ultimate end of discussion as effective in-person discussion. If we care about improving discussion, then we should care about improving discussion of all kinds—online and in-person alike.
Nick’s concerns are valid, but I think this scaffolding tool for constructive conversation can be used in ways that minimize the perceived risks to in-person oral disagreement, instructor &/or TA facilitated oral argument, and modeling human moderation of toxic topics. How the tool is used is up to the teacher. The authors note:
@ Simon Cullen: Do you see any use cases for integrating or supplementing Sway with your previous work on argument mapping, the open source MindMup? A minimal viable product would simply be a download of a Sway transcript, with one or both individual participant’s construction of an argument map, either individually or collaboratively. Thank you.
@paul Wilson, we were hoping someone would ask about this! We’re planning to enhance Sway with automated, real-time visualizations of argumentative chats as they unfold. We think this will be an extremely powerful feature for improving the quality of all kinds of discussions, but perhaps especially for polarized topics and subtle philosophical disagreements! By visualizing the reasoning as it emerges in a discussion, we plan to make it easier for students to clarify, develop, communicate and test their arguments—even if they begin in an inchoate or vague state.
Here’s a related article about a “Habermas Machine” to help resolve disagreements: https://arstechnica.com/ai/2024/10/googles-deepmind-is-building-an-ai-to-keep-us-from-hating-each-other/
Will be interesting to see how effective these AI apps are, which might be tricky to measure without pre-specifying what the outcome ought to be, i.e., without a benchmark or target.
I’m also concerned that there might not be any reasonable common ground for at least some disputes, so looking for compromise there could be inappropriate or unjustified. Can the AI determine that in advance, or does it accept both sides (or all positions) of any given dispute as legitimate?
As attributed to journalist Edward R. Murrow in Good Night and Good Luck, “I simply cannot accept that there are, on every story, two equal and logical sides to an argument.”
That said, politics and civility aren’t about what’s logical; and compromising, even when it’s absurd to do so, may have pragmatic value in a democracy…
How about Sway prompting with a thesis, debate style?
Resolved: x is y.
Flip a coin. Better, put opponents on opposite sides of a resolution – each argue FOR the opposition.
Paraphrase to mutual satisfaction.
A. because a, b, c, granted assumptions, d, e, and f, logical conclusions follow.
B. acknowledged assumptions a and b, but c, not granted, discounted for elementary evidentiary &/or logical reason WHY, so conclusion f fallacious; d and e maybe true and relevant.
Randomize participants – or reverse their predispositions – and prompt to paraphrase to satisfaction of opponent or make strongest case for opposing view?
I think it depends on what x and y are.
For instance, take the thesis “Everyone in a certain group x should be exterminated or allowed to die off” or “Everyone in group x is sub-human.” These are actual thesis-types that are being advanced today.
Why should anyone indulge the deplorable thesis and work toward some mutual understanding on this issue? And can you even imagine the blowback if you assigned a couple students to this thesis, esp. if one of them has a serious problem with it, such as being a member of group x?
Sure, it might be an interesting conversation, esp. if philosophers are clever enough to create defenses for any given position, but is it an appropriate exercise for an academic classroom?
Maybe the solution here is that the thesis must be stipulated by the human instructor who can exercise judgment over which theses are productive to select for the exercise. In other words, human supervision is still critical for AI schemes.
This is exactly how Sway is setup! Check out this short instructor walkthrough video.
@paul, we pair students to prioritize real disagreements, but after that our approach is similar to what you suggest here! Like we say in the post: “When this isn’t possible, Sway pairs students in other ways. For example, Guide might lead an exploratory discussion between two students who each have “no idea” about a topic. Or, if students who share the same opinion have to be matched, one is randomly selected to play devil’s advocate—and, crucially, is supported in this role by Guide.”
Re: the Habermas Machine, it’s definitely interesting but also quite different from Sway in terms of its aims and functionality. For one, the Habermas Machine doesn’t involve real-time discussion facilitation. Rather, it relies on a group of people privately submitting their opinions, which it (iteratively) synthesizes in search of a consensus statement. Our aim, by contrast, is to promote intellectual growth (in the form of improved reasoning, discussion, and mutual understanding). Sway might lead to consensus between students in certain cases, but that’s not at all our aim.
It’s an interesting concept, and a worthy goal to try to get people to disagree more constructively. I share similar concerns to Nick above. It seems an open question how well the ability to do this digitally will translate to the ability to have these difficult discussions in person. But I also think just being able to improve peoples’ capacity for doing this digitally would be a very good thing. It is a bad thing that people are connecting to each other less in person, and that’s a ceterus paribus reason to dislike some of the stuff described here, but you have to weigh that against the real benefits that would come from improving our online discourse.
One thing I’m quite skeptical about is that even if people are able to disagree constructively on difficult issues that doesn’t necessarily mean they will won’t still avoid these discussions. You won’t hear many discussions about the Israel-Palestine conflict going around in the halls of philosophy departments. That’s one reason to curb your expectations for what this kind of project can do.
Edit: I also wonder whether there’s any empirical work on whether philosophers actually are better at disagreeing constructively when things get emotionally touchy. If not, or if the evidence suggests that we aren’t, then that’s a very good reason for skepticism about this project.
We suspect part of the reason you don’t hear more discussions about Israel-Palestine—and a wide range of other controversial topics—is that by now people are aware that holding such discussions in public places like “the halls of philosophy departments” carries certain risks. This sentiment is certainly common when students are surveyed anonymously. Just a few examples from my own surveys at CMU:
These comments are not cherry-picked; rather, they are depressingly common. We also regularly get feedback from students commenting on how much easier it is to chat on Sway (some examples shared in an earlier comment, but you can also check out the feedback and transcripts on our website).
I’m not sure I understand the relation to what I said.
Is it your view that these risks arise because people are unskilled at constructive disagreement? I would have thought you can have a constructive disagreement but then still think poorly of people holding the opposite view, which would give rise to the same social stigmatization effect.
At any rate, by “the halls of philosophy departments” I was referring mostly to grad students and professors. These groups are evidently highly trained and competent with the skills your project is aiming to cultivate. Is there any evidence that they are more likely to engage in these discussions, or are more competent at remaining constructive when things get emotionally charged? I’m not sure how what you said bears on these questions, which seem highly relevant for the prospects of this project.
I’m not sure there’s any reason to think philosophers are especially highly trained and competent with the skills this project is aiming to cultivate. Philosophers are skilled at understanding logical argumentation, but I don’t know that we have any particularly special training at engaging earnestly with people we disagree with on a deeply emotional issue. (Maybe people working in some areas of social philosophy have practiced this skill, but most of the rest of us primarily engage with people we disagree with in somewhat more technical ways.)
But I don’t see that that’s a particular reason for skepticism about this project, as you suggest in the edit on your first comment. We don’t have to think that philosophers in general are good at something to think that some philosophers might have useful ideas for a tool that could help people do that thing.
The way that Sway was described as being useful for “scaffolding discussions” in the main post, within the points Discussion Guidance and Charitable Rephrasing, draws heavily on the skills philosophers are supposed to have cultivated, including “clarifying vague or incomplete arguments”, “unearthing implicit assumptions”, and charitable rephrasing itself.
Now you’re right that philosophers don’t have to be good at X to create a tool that helps other people become good at X. But I would think that this makes it easier. Analogy: you’re more likely to be an effective basketball coach if you yourself are good at basketball.
Perhaps this is all besides the point, however, as Simon seems to indicate in his below post that the primary purpose of Sway is not develop constructive disagreement skills in students, but to give them a confidential space where they can have these discussions facilitated. So there is no real lofty goal to improve online discourse, but just to carve out a small digital space where these discussions may occur.
When students write things like “I will never, ever, under any circumstances let anyone except my closest friends know that I’m not so sure about p” (which they do), where p is one of many campus orthodoxies, it’s usually not because they’re worried that their argumentation skills aren’t up to task. Constructive disagreement skills are important but they also aren’t at the top of parents’ minds when advising their kids to “stay silent about controversial topics” on campus… When you actually ask students in anonymous surveys, you find that they worry about a variety of things: from getting bad grades for holding unpopular views to serious harms like being cast out by their friendship groups or having their career prospects damaged by a poorly phrased comment coming to the attention of a DEI or Title IX officer—perhaps via a fully anonymous report someone made to their college’s “bias response team”.
The detention and possible deportation of Mahmoud Khalil suggests the situation is set to worsen, at least on the topic of Israel-Palestine. One of the effects of recent federal and state policies and executive orders will probably be more fear surrounding discussions of race and gender (we’ll find out in next year’s FIRE data). In other words, Sway is trying to create private, scaffolded spaces where students can talk openly for a good and well-documented reason.