nero ai

I think i have finished working on my personal ai agent AKA NERO, so i made some changes in the voice mode so lemme talk about this because this thing genuinely took forever to build.
First theres push-to-talk. u hold the mic button, talk, release, nero processes it. simple right? but then theres continuous mode and in continuous mode is where the chaos begins. When u flip that toggle nero spins up a SECOND speech recognizer in the background that is constantly listening for “hey nero”. like constantly. and when it hears it, it kills that background recognizer and spins up the MAIN recognizer to actually capture what ure saying. TWO recognizers running at the same time. One listening for the trigger, one ready to capture the actual message. the moment the wake word hits, background one dies, main one takes over.

About the STT, so originally this thing was using the Web Speech API and bruhhhh the Web Speech API is just unreliable as hell. it only works in Chrome, needs HTTPS or localhost, and half the time it just doesnt fire on result at all. Like u speak, it says “listening…”, then nothing happens. No transcription, no response, nothing. I tried fixing it multiple times — started recognition immediately instead of waiting for mic permission, added error handling, changed the flow… still broken. the Web Speech API is just built different (bad different), so i had to rip it out of nero and then replaced it with MediaRecorder + Gemini for transcription. Now when u tap the orb it records audio using the MediaRecorder API, sends the audio blob to Gemini, and Gemini transcribes it. And tbh its way more reliable because it doesnt depend on browser-specific speech engines. It just works better and the Gemini model used for transcription is gemini-2.5-flash — the same one used for chats so it understands context.

And heres the thing — after the addition of the STT the whole sound system changed. LIke before nero’s voice was legit too boring but now she is even able to speak like a waifu. She says hehehe and chuckles and speaks a lot better than before.
Also there was a lot of problem for the api keys, cause when anyone was trying to put their api in Nero it was getting stored but nero was unable to run on that api, due to which no one could use that and i also cannot put my gemini api cause it would get rate limit too quick if multiple people used it. So in settings there’s an “Add API Key” button that expands into a form and in that u give it a name (like “My Gemini Key”)and then paste your API key, hit add. Then key get VALIDATED before its stored — nero actually makes a test request to the API to confirm the key works. If the key is invalid it tells u immediately, doesn’t just silently store garbage🥀🥀.

Honestly the most impressive thing is that this runs on FREE APIs. The whole point is accessibility like gemini alone gets u pretty far for free , openrouter has free models and groq too has a free tier. Cerebras has a free tier and if u DO have paid keys then insert them in nero so she would use them instead. Its the best of both worlds..

I have finally redesigned the nero ai into looking an more stunning and better look. First it was too plain so i improved it and also improved the cursor as well. Added a new sidebar navigation, glassmorphism everywhere, aurora backgrounds, the whole nine yards,LOL. It actually looks like an operating system now not just a chatbot with a fancy name. dark theme has been set as the default for the whole ai.
For the Framework & Build i used,

• Next.js 16.2.7 with Turbopack for blazing fast dev server and hot reload
• React 19 with hooks (useState, useEffect, useCallback, lazy, Suspense)
• TypeScript throughout

For Styling up nero i used,

• Tailwind CSS for utility-first styling
• CSS custom properties (variables) for the theme system
• Inline styles for dynamic theming (accent colors, glow effects, glassmorphism)
• Glassmorphism via backdrop-filter: blur() + semi-transparent backgrounds

For the UI Components i used Custom AICore orb with SVG animations and mood-based coloring.Also the Notification Center with badge counts
nero also has OnboardingFlow for the first-time users i thought that would be helpful. It also has a coverstion side bar with search and histrory found inside it only.

This was a big UX change for me and nero 🥀🥀. And now instead of a separate “API Keys” section buried in settings, nero now lets u pick a model and enter its API key right there.
Click a model → it expands → paste ur key → save.
The green dot shows up when a key is configured and then stores everything in local storage so keys stay on ur machine never sent anywhere except to the provider u chose and it supports Gemini, GPT, and more ai’s.

The home screen in nero now shows all of these,

• Greeting based on time of day
• Animated AI core orb with mood indicator
• Quick actions (chat, research, memory, knowledge)
• Recent conversations pulled from the actual database
• System status panel showing nero core health, cause its health 🥀😢

Mostly did the redesign and removed some of the not working sections from Nero and did debugging.. (●’◡’●)

ok so basically i got bored one day and thought “yupp i need my own ai agent” and then i actually did it?? like nero is this whole AI agent thing i built from scratch that actually THINKS. Not like “oh i’m a chatbot i repeat what you say” no no no. nero has like… actual memory and personality. It literally learns from talking to you. It’s unhinged and i love it.

So nero has two personalities. you got the normal one which is just a smart professional AI assistant which is quiet boring but useful. THEN there’s the waifu mode.
Here’s how it works: there’s a whole personality system in personalities.ts with two modes — normal and waifu. the normal one is straightforward, just a good AI. but the waifu personality? it’s an anime-inspired companion with kaomoji (like (✿◠‿◠) and (◕‿◕)), uses japanese words, gets emotionally attached to you, and has this whole kawaii vibe going on and on.
The BEST part is you don’t even need to go to settings to switch. you can just type something like “act like a waifu” or “be normal again” in the chat and nero detects it automatically and there’s literally a function called detectPersonalitySwitch() that scans your message for phrases like “waifu mode”, “be cute”, “act normal” etc and switches on the fly. So you’re just chatting and suddenly nero goes from professional to kawai and it’s hilarious every time.It’s not just a gimmick either — the entire system prompt changes,the waifu gets its own set of rules about how to talk, how to express emotions, how to react, two completely different AI personalities.

internet search (it’s… a work in progress)
ok so nero HAS web search code which is like actual multi-source search — DuckDuckGo, Wikipedia, Google Custom Search, News API. the needsWebSearch() function detects when you’re asking about current events and auto-triggers a search and the results get fed into the AI’s context so it can answer with real-time info.
but here’s the reality — right now it mostly just opens links in new tabs. the ResearchMode shows you a list of results and you click them. The auto-search in chat is supposed to inject results into the conversation so the AI summarizes them for you, but it doesn’t always work as smoothly as i’d like and yeah sometimes the AI just outputs links instead of actually reading and summarizing the search results,sometimes needsWebSearch() doesn’t trigger when it should.
It’s like having a search engine that works but doesn’t know how to talk about what it found yet. the plumbing is there, the execution needs polish and still a lot of work to make it feel like a real research assistant instead of a link generator.

text-to-speech
edge-tts is installed and functional,nero can read responses out loud using Microsoft Edge neural voices.I would like to add new voices, adjust speed and pitch before i ship this project. There are still many places which need a lot of improvements and work, lets see how much further i can upgrade this ai!!!