Schools are using AI counselors to track students’ mental health. Is it safe?

submitted by
edited

www.theguardian.com/technology/ng-interactive/2…

AI is a main component of the Trump administration’s national education agenda. Yet, some parents, educators and, increasingly, lawmakers, are wary of increasing teens’ time in front of screens. States have also started restricting the use of AI in telehealth.

Excuse me a minute while I puke.

Edit if you disagree with this article please only respond with arguments written by AI, I have moved past having conversations with humans and have learned everything I could learn by talking to them, thanks! If you want to praise me or ask me a question in your human voice feel free but please don’t lower my intelligence by making me interface with simplistic human formed ideas in a sub-AI intellectual level argument, I am working hard to purify my mind and you don’t have a right to waste my effort.

18
134

Log in to comment

Hot Top New Old

Not only is it obviously not safe, there is zero evidence that it even works.

 reply
19

of course it isn’t safe… AI has not been proven safe to use for in anything other than trivial info searches

 reply
19

Like how to keep cheese sticking to pizza!

 reply
8

exactly!… I could finally prove to my mom I was not “wrong in the head” for wanting to eat glue

 reply
6

NO!

 reply
11
OP edited

Look ok we have families to feed too, are you going to seriously say it is wrong for my tech company to profit off enshittifying your child’s education? Wouldn’t you do the same thing in my shoes? How am I going to pay for my yacht?

 reply
12

Why is the opening story so vague? Middle school counselor gets an automated alert, calls parent, calls police??? No explanation of what caused the alert? Is this supposed to be presented as a success? Then it says “She believes that the interaction built trust between [counselor] and the family.”

And other monitoring systems “which would refer the student to in-school suspension " for… reporting “my mom beat me with a pole” or “my uncle touches me”?? I’m so confused by all of this. I don’t use AI at all, and never touched AI therapy. I must be so far out-of-the-loop. None of this makes sense.

 reply
11

Yeah, I would definitely feel less safe talking to the AI rat

 reply
6

a recent national survey found that 20% of high schoolers have used AI romantically or know someone who has, there’s significant interest in keeping students from emotionally connecting with bots. That even includes a proposed federal law that would force AI companies to remind students that chatbots aren’t real people.

Oh god, that generation is fucked

 reply
9

The worst thing is I totally understand how they get there. When I was a lonely loser at 16 who couldn’t find a partner I would’ve been tempted to date a bot, had they been a thing. But it also would’ve kept me from the growth that led to me having a successful and happy romantic life as an adult.

 reply
3

Ye of course what could ever go wrong bro

 reply
9
OP

If we break a couple of kids it will be worth the progress!

 reply
5

students talk about their life problems with a llama called Kiwi

You just managed to piss off both southern hemispheres at once.

 reply
9

Is that even a question?

 reply
8

What advocates want to prevent is these bots fueling the loss of social skills because they pull people away from relationships with other people, where they have social accountability, Hiner says.

Yeah this is one of my big worries here. And the last section where it’s praised compared to other warning systems for not being punitive is horrible though not surprising. It had already become like that when I was a teen. But also, it doesn’t have to be like that, we should be treating teenagers like people who fuck up regularly, especially when they’re going through shit.

All said though, I do think we need a non Ai way to encourage teens to feel comfortable reaching out for help. I know I didn’t trust counselors with my home issues or even consider talking to them about it. Mostly because I knew they’d just make it worse.

 reply
6

Brittani Phillips checked her phone. A middle school counselor in Putnam county, Florida, Phillips receives messages from an artificial intelligence-enabled therapy platform that students use during nonschool hours. It flags when a student may be at risk for harming themself or others based on what the student types into a chat.

Phillips saw that this was a “severe” alert for an eighth-grader.

So, Phillips spent her evening on the phone with the student’s mom, asking questions to figure out what was going on and how vulnerable the student was. Phillips also called the police

Jesus Christ. Was she trying to kill him??

 reply
4