Devlog 09

So, I combine two stardance projects (backend, frontend) into one (fullstack), at the time I thought I could ship both together but I had to have both repos connected in one project, I didn’t know at the time and I then abandoned the project and this project was the backend & I added the frontend repo to it so all the fronted devlogs are gone.

In this devlog I will write the changes made to backend after the previous devlog + I will summarised what was done in the fronted so far and after this devlog, I will write the devlog as normal as you guys will catch up to speed.

Backend (Express)

• In zod using z.number() or .date() were resulting in error if passed as a string as data would always get converted to string before reaching backend so i added z.coerce -> z.coerce.number() or .date
• Added github actions workflow to automatically pull changes and build with docker + fixed perms issue in Dockerfile.
• Added redirecting of / to /docs

Frontend (NextJS)

• Auth Pages are done (Login, Signup, Forgot-Password, Reset-Password) with zod and react-hook-form validation
• Pages are made using shadcn ui components
• Home page is done with listing of business that user is a part of
• Sidebar, header, Create-business-form and BusinessCard, invoice table components are also commited

Devlog 12

So I think this one is the last before ship, I promise. 😂🥹

• Fixed some bugs here and there
• Added readme
• Added some bg and showcase photos
• Added a layout for the home pages (lobby, settings, leaderboard, match history) where the top navbar and the friend sidebar is given to all.
• I think that’s it.

I got no friends, or the friends I have aren’t like best of friends So I wasn’t able to test the game on scale. Feel free to open a issue in github or message me on slack

Devlog 11

So I have done lots of stuff and this is the last devlog before the first ship.

Little sleepy rn as its 2:35 am my time, so I will wrap up fast.

• Added pino in Nextjs (Its a logger)

• Added caching wherever possible in nextjs using the cacheLife

• Added auth refresh event so if the name is changed it doesn’t need socket reconnect

• Optimised some code in socket server and also nextjs.

• Games getting saved to DB and there is a career history page now.

• Mostly did fixes of stuff and redesigned some elements in the lobby page.

Like there is nothing else to talk about I’m mostly fixed stuff.
I gotta sleep, have to ship the project too maybe I do it after waking up because readme is not ready yet.
I think I’m gonna do after waking up so tired rn

Devlog - 10

I will keep this devlog sweet and short as I wanna ship this project as soon as possible as It has taken a lot of time now.

• I added some more rate limiting
• remove game Id / party id payload where neccessary from client in socket server as game/party id can be retrieved from player stored in redis.
• I’m directly getting friendships from db instead of relying on nextjs api in the socket server to show the initial friends list with presence (online/offline status).
• Removed TTL from party and game as they may get deleted mid game (prev ttl was 10 mins).
• I have also added token refreshing in socket server as socket server only verifies token once so now when the token refreshes (only for when user details are mutated) socket disconnects and connects, I know this is a bad logic I will change that but 10h 37m will be logged so I don’t wanna waste any more time , so it will be in next devlog

• Using AVIF instead of PNG for Agents Banner & Icon
• Added new fields to user (tagline, banner, usernamedUpdatedAt, nameUpdatedAt)
• Cleaned up many of the files in the frontend party
• Added logic for If user changes their avatar it should delete the old image in cloudinary
• Added Match ending design
• ENV are now accessed from ENV.ts
• Removed console.logs etc from nexjs
• Fixed some bugs.

Devlog - 9

NextJS

• I added upload avatar in the settings page, user uploads an avatar and crops it accordingly , max file size limit is 5mb also the image gets uploaded to cloudinary and the secure url of the image is stored in db
• Changed the lobby UI to make it less cluttered
• Now there are drawers for gamemode settings and party stuff
• Changed the friends sidebar to be attaced to the right wall and closed and when you open it , it comes out of the screen not occupieing any space.
• Made a UI for FREEFORM questionMode
• Added a animated starting screen.

Socket Server

• Added a new questionMode inside the Guess Agent gamemode now you can have freeform questions or preset
• Preset is same as before you will have a preset of questions to choose from and all its all like before
• Freeform is different you will have two buttons instead of questionPanel, you will have make guess and consume no. This gamemode is meant to be played in a vc like discord or any of your choice you can ask questions to the opponent direclty if there is a no you can just consume no and if you wanna guess you can guess. its like a offline game but you can count the nos and guess and have a agent panel to deduce
• Added a game starting phase where some details are created to show to the frontend.
• Added game cleanup functions, not perfect but will do for now
• Added a new event CONSUME_NO to consume no for questionMode
• Made some changes to type to adjust for freeform question mode

EXTRA

• Added rate limiting in some parts
• In socket server rate limiting is token bucket and in nextjs rate limiting is sliding window counter (somewhat strict rate limiting in guest login and magic link)
• The rate limiting is handled by the same redis used for game states and stuff
• Fixed some bugs here and there.

I know this devlog doesn’t feel personal and I only talk about the stuff I did, its because I wanna Ship this project as soon as possible and get done with it and move on to something else, because projects like this never end and you have to decide when to stop.

I think next devlog will be the last devlog before the first ship

Devlog - 8

Big devlog, here we go again.

Lets talk about nextjs first.
So I added magic link to the app as the feedback I received was I don’t wanna use my google account as I don’t the trust the website enough to use it and I also want to use all the features so guest won’t work. This pushed me to add somewhat of a email login. I dont like password login so I wanted to add passwordless login and it wasn’t a debate I just picked one which is magic link.

• I am using resend for sending emails and i already have a domain for the app so using that for send email with auth@mydomain
• I created a template in resend dashboard itself so i can direclty use it in my nextjs app.

Now lets talk about the big thing which made this a long devlog !!! Drum Roll please 🥁🥁🥁 I decided to restructure many parts of my socket server because of scalability as my previous architecture wouldn’t have survived If i added more gamemodes and all.

What were the flaws

• Room was acting like if it was the party and the game
• Current gamemode is available is guess the agent so all the connecting modules were designed based on that which would have made adding other gamemodes impossible.
• There was no generic game module it was just Guess the Agent

What I did

• Changed room to game and separated game ( previously room ) from party and added a separate events and module for party and party gets stored in redis as well.
• Inside the game module added a core folder where core mechanics (listener, types, service) of each game regardless of gamemode lives.
• Each gamemode has its own folder inside game containing the logic specific to that game.
• Redis player now has a partyId element too.
• Improved player presence (connection and disconnect) and using switch statements in places where if the mode is different some logic might be different.

Why did it took so much time and What I learnt.

Because when I started the restructure I didn’t know how my architecture would look like or what would work ( still haven’t tested anything ). So I researched some stuff but some stuff you decide along the way.
I learnt that no matter what the project is and what you decide at the start that you are gonna build there will always be a time where the plans fail and we have to decide should we go through with it or suffer in the future or should we just focus right now and work hard so that in the future it all comes back to us with a lots of happiness!!!

Cheers,
Neerav

Devlog - 7

Another Long Devlog Time

So I was setting up my project to be deployed on the nest container that I have since nest only gives public ipv6 I couldn’t host my nextjs app on vercel because then I wouldn’t have been able to connect to my postgres database which was on nest as vercel dont have ipv6 support so I moved my nextjs to nestjs and It caused me hell lot of problems and bugs.

• First of all had to make sure nextjs, postgres and socket server were running on same docker network.
• My nest container was crashing when building the nextjs for docker.
• Had to move building process to my laptop then push the image to ghcr.io
• And some silly mistakes that took me long time to fix.

Lets talk about the project what I update in that

• So first of all made a presence for managing friends online status.
• Added logic to connection and disconnection events of player socket. As when players connects through socket it should know whom his friends and what friend requets he got and what friends are currently online and also if he is reconnecting (For now 1minute reconnect window) the socketId should be updated in the needed area (room and player redis store).
• When the socket disconnects, set the socketId in player redis store to null . After the reconnect window is over check if he is in a room, if yes remove him from the room and in both cases delete the player record from redis .
• Also lets expand on connection syncFriends the socket requests data from an internal api route only accessible through the socket for what friendship records are there and formats it in

  friends: [], 
  requests: [
    incoming: [], 
    outgoing: []
  ]

Then socket server retrieves all the playes using their playerid from redis via mget and then returns the same object structure but in friends its adds a online: true/false to the player objects

• Also removed polling for friendships as now socket does them all. So there is less db calls now.
• Fixed few bugs too.

Now let me tell you a funny story. When testing stuff in prod, I was confused why stuff wasn’t working because it was working fine during development then i remembered I didn’t update the .env file in prod. It may have cut some years from my lifetime.

Devlog - 6

This is gonna be a long devlog.

Lets start with the frotend

• I committed the prev work from the previous devlog as mentioned there.
• Added welcome, settings page, Added friends sidebar with realtime functionalities.
• Added a friends store to store incoming friends invites.
• Fixed some file structure to have protected routes inside (protected).
• And mostly this covers it for frotend

Now lets talk about backend that made this devlog of 10 hours

Backend

• I first connected the socket with the player from the db via jwt tokens.
• Stored some player data like id, username inside socket.data.
• Then I changed my room mapper implementation to use player Id instead of socket Id and messed up in some places where socket Id was still getting passed to room mapper which made me spent so much time looking for what is the issue why isn’t my room details getting sent to me and I was looking in nextjs frontend for the issue. I spent so much time finding this bug
• I added some more events for player invites and added a new redis key:value pair -> player:playerid to store player details like id, username, socket id, roomId, lastHeartBeatAt also removed lastHeartBeatAt and reconnectToken from room player.
• Added events for invite, server event of invite:sync and made realtime invite work which took me forever.
• Made some changes in connection event so it handles reconnection

Finally let me tell you something when visting the website if the thing looks small the implementation of that thing takes so much time.

Bro Literally the friends sidebar took so much code to make in the frontend and in the backend and in the socket server as well. like that small piece of the page took me so much time to make which also made this devlog So long.

Incountered so many bugs along the way. They were all silly mistakes some were logic error but most were silly mistakes as there are so many files in the project now its hard to find some things.

Devlog 5

• So I am now using drizzle with postgres and I also took some time to tinker with it and understand it a little better.

• I am also using better auth with google signIn and anonymous login.

• In the user i have username and name , username is for adding friends or locating a user, name is to be displayed , username is unique, name isn’t.

• Created the db model with friends support and matches history.

• We can add friends in the game also added user settings to customise stuff.

• Project is starting to look like something. All the drizzle + backend part that i have done in the period is committed and pushed but the frontend part still remains to be committed.

Devlog 4

• So first of all I switched from using npm to pnpm and updated the docker file accordingly for the socket server.

• Then I started working on adding users / db functionality in the nextjs app.

• I created login and register page with two options for signIn -> google or guest.

• I started tinkering with prisma and better auth and the prisma8 has just come so I started setting it up and had made the schema and had written up some services too but then when I tried to logIn i got few problems which I researched and searched and found out that betterAuth hasn’t adjusted to prism8.

• So I did what I should have done before, I switched to drizzle. Around 3-4 or more hours were wasted!.
  Now I’m using drizzle and have setup the schema and tables and also better auth in it and the signIn are working.

• Now I have to create routes or server Actions to retrieve data or send data. Now the games will be saved and stored once finished and user will able to see it in match history.

Devlog 3

Backend Socket Server

• Made few changes to setting type and added a questionPool in gameState
• Settings now have a option to change totalQuestions and a randomly generated questionPool from the totalQuestions
• Removed spectator logic as not planning on pursuing it right now
• Also added a heartbeat event which supplies heartBeat every 5 seconds or so and if there is no heartBeat provided by the player for over 30 seconds the player gets removed from the room

Now Frontend

• Redesigned the lobby page and home page
• If you open the game for the first time it asks for the name which is currently stored in localstorage for a guest user.
• When users are added then people would be able to login using google and save their details in database.
• For now nothing is stored , only gameState are persistent for the duration of the game in redis and after few minutes of inactivity in the room , the room expires via ttl.
• Also added SEO Optimisations.

Future Plan

• I will either work on adding more gamemodes or adding the user authentication stuff using postgresql and prims direclty in nextjs
• After the auth has been added then I will publish the project so that it has atleast some complete features and users can see their stats and prev games results

Devlog 2

Making this devlog after a long time of doing all this work, the frontend part is still uncommited in github though

Lets talk about backend first ->

• I first of all was using playerId for the routes to verify the user, but now I’m using socketId
• And also for reconnection I’m now using a reconnectToken which is created on the socket server and given to the frontend to it can pass it when reconnecting.
• Also I added a ROOM_KICK event.
• And I added docker setup too but its still buggy
• Changed Winning logic so that first correct guess wins

Now Frontend

• I designed the pages using AI because I am not a good designer rather I am a good coder you can say, because if there is a already made design, I can make a similar one using tailwind and shadcn and other UI libraries
• I added socket setup for the frontend which goes as follow
• src/socket -> index.ts, emitter.ts, events.ts
• src/components/SocketProvider.tsx
• Socket Provider has the listeners
• emitter.ts has small functions that can be directly used inside tsx instead of importing socket
• There are 3 pages currently -> /, /lobby, /play

Devlog 1

So I’ve been working on the socket server for this game for the past couple of days. What I’ve learned so far is that game logic is sometimes so frustrating like there can be so many edge cases you have to think about and there can be many nested if else.

So my tech stack for the socket server is:

• Socket.io
• IoRedis (redis hosted using docker)
• Zod
• and node httpServer

Ofcourse everything is written in typescript. I didn’t made separate devlogs because I forgot, sorry for that

Here’s what I did

• I made the folder structure moduler having different modules for different logic : room, game, question
• Shared files are in shared folder
• Each module has its own listener, service, schema with some having util/helper file
• I first worked on room then game then question
• Some things are still left to do, but first I would like to make some of the frontend

Devlog - 08

Since I was building a front end for this project and when designing the front end I didn’t had a route for getting all the businesses the logged in user is part of. So I added that route to show all the businesses on the home page of dashboard

And also I have email, password login so I needed a route to verify the user who has used email password using a verification code sent on their email. So I updated the user model and added that route too.

Next thing was I added those routes to my openApi docs and also I updated my readme.md and other mds in root docs/ folder.

One more thing is left in this project is to have separate analytics routes for separate modules like clients analytics, products analytics then main dashboard analytics.

I will make them once a get a jump on front end since I’m laking behind there

Devlog - 07

Since its a Backend API so I needed to create a api endpoints docs to showcase the project to other people as some UI helps in showcasing

I used scalar and openAPI docs for my project, It took me some time to understand what to do but I was finally able to do it

And then during deployment I made an error of putting some dependencies into dev dependencies as they were prefixed with @.

Finally was able to get my docker deployment working

Devlog -6

Added a analytics endpoint and its time to end the backend project and move on to the frontend of the project.

If issues come up designing the frontend, will have to comeback to fix it.

Till then i have deployed it with docker on railway.com

Devlog - 05

After adding tests, I decided to dockerize the project. But i was using puppeteer and it causes some issues in docker so i changed it to puppeteer-core and add install commands for chromium separately in docker file.

I made everything work after some brutal hours of debugging and fixing and learning as this is my first time working with docker

Devlog 4 - Backend Testing Marathon 🧪

Spent few days building out integration tests for my Express + TypeScript backend.

Added test coverage for:

• User registration & login
• Refresh token flow
• Forgot/reset password
• Clients
• Products (including file uploads)
• Invoices & payments
• Workspace/business authorization

Also set up:

• MongoDB Memory Server for isolated test databases
• Factories for generating users, businesses, invoices, products, etc.
• Mocks for Cloudinary, Resend, and bcrypt

Ran into a pretty annoying EPIPE error while testing multipart file uploads. After a lot of debugging, I tracked it down to file stream handling in tests. Switching upload tests to use Buffers instead of fixture files made the suite stable again.

Current stats:

• 39 tests
• 15 test files
• Full integration test suite passing ✅