This post is the first section of a 3 part series, created by members of the Mayor’s Youth Council featuring discussions and ideas regarding the upcoming technologies of AI used in academic settings.
Introduction: Everyone’s obsessed with what AI can do. However, we’re not talking enough about what it’s quietly getting wrong.
Here’s something I’ve noticed, the loudest voices about AI are almost always either total believers or total skeptics. The believers want you to know it’s going to cure cancer and write your emails. The skeptics want you to know it’s coming for your job and your soul. Both camps are kind of exhausting.
The more interesting conversation is the one I think we actually need, lives in the messy middle. AI is already embedded in how we work, how we learn, how we get information. And it comes with some genuinely serious problems that tend to get glossed over in the hype. So let’s actually talk about them.
It Carries Our Biases
One of the most persistent myths about AI is that it’s neutral because it’s mathematical. It’s not. AI learns from data, enormous amounts of it, and that data was created by humans, in a world full of historical inequalities. So when a hiring algorithm trained on decades of biased hiring decisions continues to favor certain demographics, that’s not a glitch. That’s the system working exactly as designed, just reflecting patterns it was taught.
USC Annenberg has written extensively on this, and the picture isn’t pretty. The good news is that researchers are actively working on it. MIT recently published work on a technique that identifies and removes the specific training data points driving biased predictions, rather than trying to patch over the problem after the fact. But “researchers are working on it” and “the problem is solved” are very different things.
Jobs Aren’t Disappearing Overnight, But They Are Changing
The “AI will take all our jobs” headline gets clicks, but the reality is more gradual and arguably more complicated. What’s actually happening is that certain tasks, especially repetitive, predictable ones in retail, customer service, and logistics are being automated at a pace that workers and institutions aren’t fully prepared for.
This doesn’t mean mass unemployment is inevitable. New technologies have historically created new categories of work. But that transition has never been frictionless, and the people most affected are rarely the ones best positioned to pivot. That’s worth taking seriously, not dismissing.
Your Data Is The Product
To function well, AI systems need data. Lots of it. Your search history, your browsing habits, your voice assistant conversations, all of it feeds the machine. Most people click “agree” on privacy policies without reading them (no judgment, they’re designed to be unreadable), and end up with very little understanding of what they’ve handed over.
This isn’t paranoia, it’s a documented, structural issue with how most consumer AI products are built. If you’re not paying for the product, the product is often your attention and your data. That’s not new, but AI makes the data collection more sophisticated and the stakes higher.
Overreliance Is A Real Thing
This one doesn’t get enough attention. A BYU study found that the reasons people are reluctant to use AI aren’t really about sci-fi fears, they’re grounded concerns about trust, accuracy, and losing the human element in things that matter. And honestly? Those instincts are valid.
There’s solid research showing that people regularly trust AI outputs over their own judgment, even when the AI is wrong. The concern isn’t just that AI makes mistakes, it’s that when people outsource their thinking to it, they often stop noticing when it does. That’s a real risk in high-stakes situations: health decisions, legal questions, anything where being wrong has serious consequences.
As the BYU researchers put it plainly, AI can be a crutch that replaces actual learning rather than supporting it. Using it well means staying in the driver’s seat, not handing over the wheel.
So What Do We Actually Do About It?
I’m not going to end this with “AI is bad, log off forever.” That’s not realistic, and it’s not even the right conclusion. The goal is to use these tools in a way that keeps you thinking, and to push the people building them to do better.
Practical things worth doing:
- Stay skeptical of outputs. Double-check anything important, especially health, legal, or financial information. AI confidently states wrong things.
- Look at privacy policies. At minimum, know what data an AI tool collects and whether it shares it. Avoid putting sensitive personal information into tools you don’t trust.
- Choose transparent tools. Support companies that are open about how their models are trained and what they’re doing to address bias.
- Keep thinking for yourself. Use AI as a starting point or a sounding board, not as the final word. Your judgment still matters!
- Ask questions in your community. When AI is being used in your school, workplace, or local government, it’s reasonable to ask how, and who’s responsible when it goes wrong.
None of this requires being an AI expert. It mostly requires the same critical thinking we should apply to any powerful, widely-adopted technology. The fact that AI feels impressive doesn’t mean it deserves unconditional trust.
The potential here is real. So are the problems! Holding both of those things at once, without collapsing into either hype or panic is probably the most useful place to stand. With these ideas in mind, the next installment, “Ethics of AI Usage” will cover how AI use on small levels can greatly affect society as a whole.
Sources
https://news.mit.edu/2024/researchers-reduce-bias-ai-models-while-preserving-improving-accuracy-1211
https://news.mit.edu/2026/smarter-way-to-debias-ai-vision-models-0429
https://www.weforum.org/reports/the-future-of-jobs-report-2025/
https://www.nu.edu/blog/ai-job-statistics/
https://itif.org/publications/2025/12/18/ais-job-impact-gains-outpace-losses/
https://www.digitalocean.com/resources/articles/ai-and-privacy
https://www.pewresearch.org/global/2025/10/15/how-people-around-the-world-view-ai/