It was an impromptu decision. I was watching Juxtaposed’s video about “redesigning Apple Music’s UI” and that got me wondering about how much Apple Music lacks that Spotify already has.

The reason I initially switched was due to two things: The tight integration of Apple Music and the Apple ecosystem in general. Like the fullscreen background on the lockscreen – those used to look beautiful! and it’s the sound quality. Apple Music just provides lossless. Even though I still use bluetooth to this day. And also other features such as the karaoke feature and beautiful lyrics screen.

However, the novelty died off. The ‘tight’ integration just doesn’t impress me anymore. Every single new feature added to Apple Music was just considered nice to have. Not revolutionary.

I kept reading online about how one service is better than the other due to some codecs and streaming compression and Apple’s chokehold of the whole audio pipeline in Apple Music but that could just be confirmation bias (even though it might be factual)

So... For the first few hours of switching back, I had to make sure to squash my confirmation bias by A/B testing between the two services. I’m not so much worried about the low ends because these DO come through on the AirPods Max. I’m more worried about the high end, especially in a busy track. I’ve determined that both of them is indistinguishable to my ears. So yup, audio quality is about the same (on a bluetooth connection anyways)

So what did I miss from Apple Music? The karaoke feature, aptly named ‘Sing’, the iCloud syncing feature which allowed me to sync and listen to my bought albums on my other devices (i.e. Breach: Digital Remains by Twenty One Pilots) and most probably the official sets playlist and videos.

But the bigger question is: what did I miss from Spotify when I was on Apple Music? Music recommendation. I don’t mean it like playlist recommendations from the home page. I mean the autoplay music that comes after a song. Apple Music SUCKED at it. It kept playing music that I would only name ‘safe choices.’ All the artists that play after are your top 100 charts artists. I don’t even think genre was even considered in Apple Music’s autoplay. Secondly is Spotify Connect/Jam – this one’s huge. Being able to control music playing on my computer through my phone must be magical. And Spotify Jam? Makes it easier for me and my friends to hang around and listen to music together. I finally don’t feel like the lone-Apple-Music guy.

Spotify has always been playlist and social-centric while Apple Music is put out there to do one thing: For you to listen to music. Recommendations are not their priority (and screw you people who keep saying to give Apple Music some time to learn about your taste. I have done EXACTLY that – 4 months to be precise). And that’s fine. Maybe that works for some people. I, am however, a gen z, with gen z friends who’s backbone is Spotify and I shall be with my kind.

And that is my take on Spotify vs Apple Music. The intersection of digital music and social features.

Author’s Note

I’ve been gone for long.

I’ve been serving my nation (National Service) and have been using the time to relax, learn different skills and upgrading myself in different ways.

Unexpectedly, I also went full-on into my other hobby… Playing the guitar and the bass.

Hence, catch me playing bass and guitar on:

• TikTok

• YouTube

“AI IS SO COOL!”

Not long ago, I was hellbent on learning AI.

Well, to be fair, I caved in with the whole marketing jigamajig.

Other than the whole “AI bubble” thing, I’ve always wanted to learn AI. Intelligence in computers has always sparked something in me.

Moreover, I was convinced (at the time I graduated Poly) that to succeed in the IT sector space, one has to either choose between Cybersecurity or AI. Being a Software Developer in the big ‘23 was outdated. Like everybody can build a node.js server and set up some SaaS and just piece it together…

After I graduated, I took this unofficial course on YouTube that kick-started my ML learning journey.

It was a “Learn PyTorch in 24 hours” where I only could make it halfway, 12 hours before giving up because of how technical it was for me.

Time skip to 2026, I decided to jump on it again.

I’ve always been inspired by how people use AI to do insane things but I never understood how.

So I did some research and came across a new term: “AI Engineer”.

My initial thoughts was “What is that?” and I thought an AI Engineer is the same as ML Engineers except AI Engineers build LLMs.

But that was when I discovered that the meaning of “AI Engineer” keeps changing and the current one is just a Software Engineer who uses the capabilities of LLMs and other technologies to streamline mundane tasks and processes.

During my research about AI Engineers, I come across this course on Datacamp titled “Associate AI Engineers for Developers” and thought “Great! I’m a Developer, and this course teaches me about being an associate AI engineer! At least the courses treats me like a developer so I don’t need to learn everything from scratch”

So I took the course (well, it was a free 3 month trial)

Time jump to 3 weeks later, after learning majority of the except for the LangChain stuff, I decided to use my skills.

So far, here's the summarised version of what I've learned that I think is important:

• Prompt engineering

• RAG

• Vector databases

• Embeddings

• LLMOps (testing, model selection, RAG vs fine-tuning, model costs and etc.)

Applying my skills

Let me introduce to you the problem: I'm serving NS and I take care of the Ops Room.

That's my vocation: Ops Room.

Our job, among many things, is to take care of the keys to the whole of the training center. We issue alot of keys daily.

The problem lies in the loads of important key information that is somehow left out and hard to keep track.

Here’s what I mean by that: throughout the months that I've been serving, keys and information tend to change. Sometimes they're no longer stored in the ops room. Sometimes keys go by different names. Sometimes you need to ask permission to issue out a certain key. And these information is relayed once and sometimes gets lost in the sauce.

Sure, a seasoned experienced person like me would know all these information and can blurt out information at lightning speed but what about the newcomers? (I mean, to be fair, they can just store these knowledge in their notes)

Hence the solution, building an RAG system using the information of keys and other important contextual information.

To be fair, the problem isn’t even a real issue to begin with but you know me… I like to hone my skills by solving a non-existent problem.

It’s like the saying goes, “I take 10 hours to build a solution that takes me 2 minutes by hand every day.”

Disclaimer: Privacy

Before working on this project, I understand the importance of privacy. Especially the information that I’ve been working with. Which is why I made an effort to:

• Scramble data when interacting with online services

• Censor sensitive information in this blog post (i.e. make it as vague as possible)

• I made sure all of this is ran locally on my machine!

Data processing

I sat down in my chair. Back arched into my desk. Furiously typing and looking for relevant files to feed the vector database while thinking about processing the data for the embeddings.

Yes, there's the problem of what MEANINGFUL data to embed but before that I was already stuck about converting the master-list excel file into a structured data

FYI: All record of keys and what rooms they're for is kept in an excel master-list file that has multiple sheets, one for each section of the training center. It is not as simple as your typical CSV file – some pages have a subheading that merged the cells and whatnot. It's not a simple "read excel file from python" problem.

Introducing... ChatGPT! Yep – I used the skills I learned in Prompt Engineering to format my excel file into a JSON format I wanted.

In the end, I wrote a quick process_keys.py file that helped me take the multiple json files and combined them into one json filed called merged_keys and looks somewhat like this:

[
    ...,
    {
        "bunch_no": "930D",
        "no_of_keys": 8,
        "location": "DORM 93",
        "remarks": "",
        "key_section": "'D' - DORMS",
        "building": "DORM BLOCK",
        "id": "D_93"
    },
    ...
]

Note that the information on top has been scrambled with false information. The structure remains true.

Embedding data

With the help of the lectures from the Datacamp course, I used ChromaDB as my Vector Database, ChatGPT-ed and asked for the best embedding model. It's my choice because there is a persistent store and useful for quick prototypes. Don’t need to run a docker container and whatnot.

At first, when embedding data, I embedded all the information I got from the keys.

The original JSON structure looked like this:

{
    "bunch_no": "930D",
    "no_of_keys": 8,
    "location": "DORM 93",
    "remarks": "",
    "key_section": "'D' - DORMS",
    "building": "DORM BLOCK",
    "id": "D_93"
}

Each key was down to a string that looked like this:

Bunch No: 930D
Number of keys: 8 key(s)
Location: DORM 93
Remarks: 
Key Section: 'D' - DORMS
Building: DORM BLOCK
...

I was ready to test out my big masterpiece next-gen AI app that will crown me the best tech adopter in the whole of Singapore!!!

When life give you lemons...

You squeeze it into your eyes! Just kidding.

Did you really thought my first test would pass?

It failed miserably. Queries were returning unexpected documents.

For example, when asked "What is the keys to bunk 930D"

It would return god-knows-what key that has nothing to do with bunk 930D

I was “debugging” furiously.

To me, this whole “embedding model” thing is a black box. You can't really debug the semantics that the embedding model captured.

Sure, you can visualise the embeddings on a graph but whatfor? (correct me if I'm mistaken)

I went back to ChatGPT and asked "What is the best embedding model that can be ran locally?"

I was so sure that it was the embedding model that didn't capture my search query semantics – maybe it was dumb enough to not understand "bunk 930D" as "dormitory 930D" but well, I'm not any smarter either.

I did a quick switch of embedding models and ran it again, and…

Do you not understand?

It worked! I’m kidding!

Nope, it didn’t.

Yeah… I was baffled.

I thought a better embedding model would understand the nuances of the word “dorm” being interchangeable with "dormitory" or "bunk".

I went back to the drawing board.

I thought hard about how do I make these models understand these nuances.

Do I have to train my own model to do so? Will that take so much of my time?

I was convinced it was somehow the embedding model UNTIL I asked ChatGPT during my research process.

I asked whether my semantics were captured, why it did not return what I expected and if the issue is with the dimensionality of the embeddings.

ChatGPT, being the supportive friend it is (/s), told me that there wasn't ever an issue about embeddings but could be a myriad of other issues:

• Metric for search (dot product vs cosine similarity)

• Vector normalisation

• Embedded text data

• ... and many more

Embed or not to embed?

Okay, now I’m convinced it’s the data that I’m embedding

I went back to my code and went back to the embed_keys.py file.

I looked at the initial data that was sent for embedding – ChatGPT told me to embed information that’s concise but store all the other important information in the metadata of the document.

So I did just that.

From each key being given a embedding data of:

Bunch No: 930D
Number of keys: 8 key(s)
Location: DORM 93
Remarks: 
Key Section: 'D' - DORMS
Building: DORM BLOCK
...

I reduced it to:

DORM 93. Key number/bunch number: 930D. 8 key(s). DORM BLOCK Building. Remarks:  'D' - DORMS.

✅ Concise and understandable.

Maybe that's what the embedding model wants: A concise sentence that captures the information!

I excitedly re-embedded all the documents and tested it out to query for keys for bunk 930D...

It didn't work again.

I went through all that nail-biting, toe-curling research using ChatGPT and Google just to find out that the changes I made didn't work.

Don't even get me started about how I hard I had to research on how to delete a ChromaDB collection.

You might think what in the world is going on...

Here's the rundown

• First, I wanted to create an RAG to practice the skills I've learned

• So I decided to solve a (non-existent) issue – the nuances and context with keys in my building. How easy it would be to just ask the Chatbot "What keys are bunk 930?" and it'll return the relevant information.

• Hence, I built RAG system using ChromaDB and a local embedding model

• After embedding, I tried asking the vector DB "What keys are bunk 930" but it didn't work

• I was baffled because I thought embedding models are supposed to understand the nuances of ‘bunk, dorm, dormitory’ but it didn't

• I went through a nail-biting research session using ChatGPT and Google because embedding models are a black box and these type of issues are not easily debuggable.

• ChatGPT suggested (among other potential issues) that maybe the data I'm embedding for each key might be too verbose.

• I went back to reduce the embedding data for each key making it concise. Concise one sentence like "DORM 930. KEY/BUNCH NUMBER 930D. DORM BLOCK"

• I was excited to test again by asking "What keys are bunk 930?"

• Did NOT work

Yeah, that sums it up.

I was close to giving up. Maybe it was built to answer YouTube questions (it’s a joke pointing to how one of the lectures required us to code out a RAG system querying from a YouTube video, description, title and links dataset) and not answer questions about the keys.

So, I finally took a break. I went about my day. Every single thing I do, the problem haunts me, nudging me every second.

I caved in to the hauntings.

I thought about it...

How do I make them understand that when I ask "What keys are bunk 930?", I actually want the program to understand that I'm asking for Dorm 930 keys.

And something clicked!

“If RAG was a system to retrieve relevant documents (all it returns are just documents and context for the LLM to understand), then this means I can add my own context to be embedded!”

I opened up my laptop.

I created another file called info.json that contains all important context for keys.

[
    ...,
    {
        "remarks": "dorm 930 is also known as dormitory 930 or bunk 930 which is bunch 930d at dorm block",
        "id": "dorm_930_info"
    },
    ...
]

and finally...

The Win

I cleared all the database items and re-embedded all the information and asked the vector database "What keys are bunk 930?" and LO AND BEHOLD! It returned the information correctly!

So, what's next?

Well, now that I got the prototype RAG built.

I will

• build a more robust system using other vector DBs like Pinecone.

• using LangChain to use LLMs so answers are nicely structured when asked (obviously, there's more to make out of it and it can expand to other domains – not just asking about keys in the place)

Yeah, that’s all I could think about as of writing.

Well. That's about it for now. Catch y'all soon!

I want to extend my special thanks to the team members that I’ve met, Russell, Bryan, Dinie and other team members who helped out too! Along with the wonderful and friendly Republic Poly lecturers that I’ve met (especially Nona, Firman and my favorite photographer – RJ).

Dinie: https://www.linkedin.com/in/muhammad-dinie-bin-baharudin-7a1680213/

Russell: https://www.linkedin.com/in/russell-low-b94b44210/

Bryan: https://www.linkedin.com/in/bryan-chan-146a1820b/

Preface

We have reached the end of the life of our game. Our game life has extended beyond just the open house in January 24'.

Recently, on May 20th, 2024, we concluded the final session for secondary school students to introduce them to the Diploma in Cybersecurity and Digital Forensics (DCDF), formerly known as the Diploma in Information Security (DISM).

This blog post recalls the timeline from when I started entering the project until the end. I am not from Republic Polytechnic, and my involvement in this project is not on record (i.e., I don’t receive CCA points for this; I’m doing it of my own will).

Introduction

Back then, I was in the middle of my internship, towards the final stages where we were supposed to finalize the last features of our mobile app, get it built, shipped, packaged, and sent to the app stores. I knew I had some free time, so I said yes.

I said yes for two reasons.

First, Dinie was admirably convincing, and the idea he pitched was awesome. Second, I had the opportunity to learn about Lua, Roblox programming, the Roblox game infrastructure, and game development design practices. Game development had scared me away for the longest time, but I figured that it's just Roblox—what harm could it bring?

Before that, a quote from the OGs

I joined the project at a later stage (in November), let's hear it from Dinie, Russell and Bryan who has been there since day one (and are in the same school)

Back in the first half of year two, Dinie, Bryan, and I toyed with the idea of creating a Roblox game for our final year project (FYP). While the concept was exciting, I secretly doubted that our school would ever allow us to make a Roblox game, thinking it was too far-fetched.

As time passed without much discussion between us, I joined another FYP group, believing that the Roblox game idea was just a pipe dream.

Soon after, at the end of the half of year three, and after our FYP projects were done for, Dinie invited me to work on his (separate) game for the RP Open House 2024. This time, I didn't hesitate. I gratefully accepted the offer to be part of this exciting project.

– Russell Low, Team Member

Back in April 2023 when this project began, some people doubted our ability to create an interactive 3D cybersecurity capture the flag game, claiming it had never been done before. Despite these doubts, we not only made it happen but also demonstrated our capability to bring this CTF game to life, proving them that nothing is impossible.

– Bryan Chan, Team Member

So, what can I bring to the team?

I asked Dinie, "What I could bring to the team?"

He answered, stating that he was the sole developer for the game and wanted me on board to create a really cool functionality for the game.

The functionality is the ability to 'trip' the room of another player, causing their room to lose power. This would halt any progress they were making in attacking other players. The affected players would then have to either shoot the person who tripped their power or solve a puzzle on the laptop in their hotel room.

As he explained each detail, my brain couldn't help but create a mental diagram of how this would work. I knew I would need several things: a 'lever' for the player to pull and a state system for the rooms to keep track of which rooms are 'tripped.'

But then I stopped myself. I had obviously forgotten that this was my first experience in game development and that I needed to learn.

Learning Lua Game development

Yes, I would say that my fundamentals in programming are strong – not just concepts such as conditionals and loops, but also design patterns. Learning Lua was not that hard. Sure, the syntax is much different than what I am used to – but isn't that what Google is for? However, the one thing that caught me was the Roblox ecosystem.

What do you mean by Roblox ecosystem?

It's the idea of data storage and persistence via DataStores, server scripts that run on the server, local scripts that run on the client, properties and attributes of items, modifying the properties and attributes programmatically, and many more. These ideas are hard to grasp, so I took it slowly, asking Dinie for a little leniency as I explored and learned about Roblox.

The first thing that came to mind as I searched for a solution was the fact that I wanted to establish data persistence. That's when I asked the group chat for assistance. I drew what I wanted in Excalidraw and messaged the group chat asking for the best way to store this data.

Upon further research, I found that using a DataStore is the best method for my use case. Between starting the feature and finishing it, I hopped on calls with Dinie to clarify and gain his thoughts, because I've got to respect him as a Roblox game developer. These back-and-forth ‘meetings’ are what I call productive meetings because each one has a goal. We lay down our thoughts, mix both of our thoughts, and the cycle continues.

Showcasing to the School of Infocomm, Republic Polytechnic

At this point in time, we have reached a milestone; most of the features are done, and the game is at its most playable state. We need to attend a showcase for the Operations/Business side of the school. Dinie asked if I could come along, and I agreed. I wanted to meet the lecturers who put in their time outside of work to mentor the team. But uniquely, I have to play a guise. I can’t let them know that I am a student from Singapore Polytechnic.

That day, I met the cutest lecturer, Nona. She reminds me of my grandma. The way she was enthusiastic about her ‘escape room’ and showing where she hid the ‘scroll’ was just way too cute. Her enthusiasm, chuckle, and her pause, waiting for our reactions to her genius-ness, was cute!

I went with Dinie to the showcase, and along with us were the projects from other diplomas, such as a VR simulator for a server room and a blockchain exchange website by the Diploma in Financial Technology. These projects really opened up my eyes to the people with all their eagerness to showcase their projects!

At the showcase, we were given some feedback regarding the playability of the game and how players might find it confusing – especially when we need to stick with 20-minute sessions on the day of the Open House itself. We took this feedback personally but constructively too. Dinie and I brainstormed ideas and went back to the drawing board.

We relayed the feedback and the ideas to the team, where the team gladly gave their input. The power of teamwork, am I right?

In a future iteration, Dinie added an introduction cutscene to the game, which is mind-blowing because it has cool camera animations (that even measly Nabil couldn’t do).

Refactoring, leading to bugs, leading to refactoring and leading to bugs.

Have you ever encountered a scenario where you have to refactor code, but after refactoring, new bugs are introduced, creating a vicious cycle? It sure as hell happened in this project.

For instance, we had a play-test between the 4 team members over Discord, where we actually put the actual gameplay to use. A short session of 1 hour harvested so many gameplay issues and bugs that I needed to fix (because it was my fault).

Mind you, this was way past the time of me coding the project, so I had to open up the project and look at the mess of the code I had written. This is where the refactoring part comes in. I refactored most of the code to use RemoteEvents – remote events are self-explanatory; they are events that can be called from server to server and vice versa. There is also another concept called BindableEvents where it can only be called client to server and vice versa.

This cycle continued until I got it to a stable state. Other than that, we just have to lay low and wait for the day of the open house

The day of the open house

The day has come; all the hard work our team put in has led to this moment. For three full days, we showcase our project at our course booth, promoting the DISM (now known as DCDF) booth.

At first, I found it strange. I am not a student from Republic Polytechnic. Heck, I’m not even from the same course. I am from the Diploma in Information Technology. I build apps and websites for a living – sure, I do have some experience in cybersecurity, but not much. I was nervous.

But my nervousness quickly died as I reached our booth, just in time for the briefing. Everybody had one job: to keep the booth alive. Our Roblox game was held in sessions in the lab room, but meanwhile, we took care of our mini booth dedicated to showcasing the game.

I took on a different role in the open house; I helped out in the escape room booth that Nona made.

It was fun. I learned about steganography and discovered that you can hide text in images. Having to put on a facade to the prospective students and parents of the students, saying that there is an ongoing attack in RP and that we need to find the date and time of the attack, was both tiring and really fun. I engaged with a lot of prospective students, so much so that I lost myself in the sauce; I literally told them to join RP DISM. What was I doing? Is this the purest form of betrayal? Shouldn’t I be promoting my school instead?

This goes on for all three days of the open house.

The sessions in the lab

As I said, we held game sessions in the lab. Like the saying goes, 'anything that can go wrong, will go wrong' – Murphy's Law. That is exactly what happened. For three days, six (or more) sessions were filled with the game breaking. Dinie and I fixed the game in 30 minutes during our lunch break. It was intense, more intense than the time we spent building the game. This time we were constrained by time: 30 minutes to fix the game before the session started for the day. I am, however, pleasantly surprised by Dinie’s quick thinking ability. He converted what was already a broken game into an engaging one by changing the rules of the game to be based on the maximum number of points instead of attacking/defending.

My friends, Russell and Bryan.

These people are the friendliest I've met in RP (not that I have many friends from RP anyways). We slacked when nothing was going on and talked about the most random stuff. They were an absolute delight to be around.

Debrief

After three tiresome days, we ended with a debrief led by Dinie, where we discussed the future of the game. I also extended my thanks to the team members for forming a connection, a friendship that I’ll remember despite meeting only for these three short days. The immense amount of pressure, time, and all the fun things we did in these three days!

The future of the game

The open house ended on January 6th, 2024. Between then and around April 2024, we didn’t touch the game at all. We led our lives until recently. We somewhat went back to the drawing board and fixed what needed to be fixed. To my surprise, the game is being used outside of the open house. Our game, which shamelessly was broken, is being used to promote the diploma. I never got that opportunity in my own school. Being able to do that in another school felt both wrong and empowering.

In a recent Discord call, we gathered all four team members: me, Dinie, Bryan, and Russell, and we discussed and talked for a bit. The main point of the call was to discuss what needed to be done in preparation for May 20th, when a group of secondary school students will come to RP for a tour.

Conclusion

This is too much to conclude. Six months, to be precise. But it opened my eyes. I learned a lot from kids on YouTube, teaching me Roblox Studio to the team members who are determined, to learning about soft skills from Dinie and the other team members – and lastly, to forming a friendship between me, Bryan, and Russell. This is one of the projects that I will never forget. From being a total noob in Roblox and game development to coming out confident about the ins and outs of Roblox game development.

Let's hear it from the OGs again for their conclusion of the project

I am grateful to be given this opportunity to develop an interactive 3D cybersecurity capture the flag game together with my teammates Dinie, Joharie, Nabil and Russell.

Developing a game from scratch presented numerous challenges, including defining game concepts, gameplay mechanics, target audience, development timelines, and marketing strategies.

Throughout the development process, challenges were encountered where things didn't always go as planned. Despite these obstacles, we persevered, ultimately bringing the game to life.

Our effort, dedication and hard work paid off when our game was selected to showcase at GovWare 2023, Republic Polytechnic open house and secondary school students.

This game wouldn’t be possible without the collective effort from the team. I would like to extend my heartfelt thanks to everyone for their contribution in making this game project possible.

– Bryan Chan, Team Member

Working on the game has been a wonderful experience. I had the privilege of collaborating with amazing people, including Nabil from Singapore Polytechnic, an extremely skilled programmer who significantly contributed to the game's success and was just in general an incredibly fun dude to be around.

As the person spearheading the project, I am extremely impressed by Dinie's ability to lead the team in achieving his vision for the project, think on the fly, and keep a level head in fixing issues despite the immense time constraints and pressure.

Bryan, who designed the 3D assets for the game, made the game look unbelievably stunning with his meticulous attention to detail and artistic flair. The vibrant and immersive lighting in the environments he created added a level of polish that truly brought our game to a new level.

Outside of our team, the open house also afforded me the opportunity to connect with many other people outside of my cybersecurity discipline who shared a common interest in application development. It was inspiring to see the diverse projects and innovations from students across different fields, and it gave me a broader perspective on the possibilities within tech development.

Looking back, I'm glad I seized this second chance. Being part of this Roblox Capture the Flag game has not only been fulfilling but also a testament to following one's passion and approaching new opportunities with an open mind.

– Russell Low, Team Member

Preface

I have been a React developer for the majority of my years.

I've witnessed React evolve from Class Components to Stateless Functional Components, and then to what we have now: Functional Components with Hooks.

While this article might seem biased, I assure you that I strive to be reasonable, logical, and objective in any judgments and conclusions I make. I pour my heart and mind into this endeavor.

What you are about to read is just my two cents and should not be taken to heart.

You can view the repository for Thoughtful here.

With that, the story begins prior to WWDC 2024.

About a week before WWDC24...

It's about a week away from WWDC 2024. News outlets and YouTube channels are scrambling to be the first to upload reported leaks about the new iOS 18. Features such as customization of the control center, which surfaced last year, are making a comeback.

Amidst all this chaos, Apple Developer's YouTube channel started bulk uploading WWDC 2023 sessions. At that time, it was my first time ever watching a WWDC session, so I clicked on the most interesting video and gave it a watch.

I was pleasantly surprised. Even though I don't really understand Swift, I could infer the language syntax. What caught my eye was how seamlessly Apple's libraries and frameworks work together. It's like butter. Everything just works! Need a map? Import MapKit. Need tips to appear in your app? TipKit!

While watching the session video, I remembered that I had bookmarked an official tutorial guide by Apple, "Develop in Swift," which I had been longing to follow.

Deep down, I had bookmarked the tutorial just in case I got bored and needed something to do. Additionally, I had been eager to learn Swift and iOS app development after following a Udemy course but gave up halfway through.

Since WWDC 2024 is about a week away, I decided to take my chances and follow the tutorial. Maybe I could keep track of WWDC 2024's new Platform State of the Union changes?

Is "Develop in Swift" any good?

I don't know who is the target audience for the guide but all I knew was that it wasn't targeted towards individuals who have never programmed in their life.

Even I had trouble wrapping my head around some of Swift's concepts, such as closures and trailing closures. It's really a syntax issue that I need to get used to. Other than that, it is strongly-typed and comes with neat features like computed properties.

The tutorial guide covers the basics of Swift, building an app in SwiftUI, integrating data persistence using SwiftData, and more. One thing worth highlighting is that the analogies used in the tutorial guide are easy to grasp and really good. They make difficult concepts digestible!

I, shamelessly, haven't completed the tutorial guide in full and stopped after the part on SwiftData as I tried to integrate these concepts into a project I started long ago, but this time, on iOS.

Learning Swift through building an old project I started

For those that didn't know, in February/March this year, I decided to impart some of the skills and knowledge I learned throughout my Mobile App Dev internship by building an app called "Thoughtful." Thoughtful is a daily gratitude journal app that prompts the user to journal about their life.

After a few days of designing in Figma and perfecting every user interaction, I built the app in Flutter. I stored the data in SQLite, on-device, and used Flutter BloC for state management. While it was a huge learning curve to understand BloC (I still detest BloC to this day), my states are kept in sync, and that is what is important.

Now that you know about Thoughtful in Flutter, I tried to build Thoughtful for iOS. It was a complete redesign (though sticking to some elements) and adhered to Apple's Design Guidelines.

Yes, I know that Flutter is cross-platform. I built Thoughtful for iOS as a way to cement my knowledge from the tutorial guide.

What I loved about Swift, SwiftUI and SwiftData (as compared to React)

Declarative approach to building User Interfaces

As I was developing using Swift and SwiftUI, I couldn't help but notice how similar it is to Flutter. Both SwiftUI and Flutter have a declarative approach to laying out a UI.

Declarative, in this context, means laying out a UI involves me "telling" SwiftUI what I want instead of "making" the element in SwiftUI into what I want.

In this case, declarative would be Button().background{RoundedRectangle(cornerRadius: 24)}.

Imperative would be React-like where we have to declare a Component ("Button") and use styling to style the Button – i.e. I make the Button in React into what I want

<Button style={styles.buttonStyles} />

This declarative approach makes it easy to prototype UI but at the same time can make the codebase messy as styles might be everywhere (instead of having one source of truth, like a stylesheet file).

However, it is nice to know that colors (such as background or foreground) are propagated throughout the children when applied at a top-level view.

Packages tend to work, or at least the ones I used do

A big gripe I will forever have with React Native (and Expo) is that packages would break after upgrading either React Native's or Expo's version.

With every Expo SDK upgrade, I would spend more time modifying components, installing compatible packages, or replacing packages.

There was a time when I had to debug for 1.5 hours because the react-svg-loader package broke, and the error message didn't have anything to do with the package.

On the other hand, almost everything is built into SwiftUI. Tooltip? Tab Navigation? Stack Navigation? It's built-in. No need to install packages that might break one's workflow.

SwiftData is a Lifesaver

SwiftData is a way for developers to persist data in their apps. The underlying technology is SQLite (the same as in Thoughtful Flutter), but this "wrapper" allows us to read, insert, update, and delete as if we're interacting with objects and classes rather than doing what we're used to as developers (ID lookups and whatnot). It's really easy!

// Read "Item" from storage and sort it by the date_created
@Query(sort: \Item.date_created) var items: [Item] = []

// Insert
modelContext.insert(Item(...))

// Update – Notice how I didn't even have to touch the modelContext as the Item struct has all the bindings with SwiftData. Editing the object directly would update the Item in the database
Item.date_created = Date.now

// Delete
modelContext.delete(instanceOfItem)

iOS Previews and View Previews

The iOS Previews and View Previews allow us to quickly preview views in a side panel (called "Canvas"). I found it intuitive and fast. It hot-reloads too!

Quality (and quantity) of documentation and support

The quality and quantity of support available for Swift is mind-blowing. Apple's official documentation includes comprehensive guides that have been invaluable.

YouTube channels like Sean Allen's have provided me with more than enough knowledge to build this project.

Where React shines

Cross Platform

That's the biggest driving factor behind basing an entire project on React Native. While React Native may not be as performant (due to having a layer of JavaScript engine running in the application), for small teams, it makes sense to sacrifice some performance for a better developer experience (DX). Moreover, we can build for iOS and Android using a single codebase.

Maturity

Yep, I said it. React Native is more mature than SwiftUI and Swift. This maturity also means better packages (if needed) and better documentation for those packages (for instance, Spotify's iOS API documentation is in Objective-C).

It's easy for developers who's familiar with React

Yeah, it's true. React developers would likely find the switch to React Native much smoother compared to switching to Swift. Even switching to Flutter and Dart would be smoother than switching to Swift.

My biggest gripe with iOS App Development – The App ID Limit

The signing limit really frustrates me, especially as a developer who's not enrolled in the Apple Developer Program (it costs $99 USD/year, but I'm just a student with no budget).

This issue arises when I try to run the build on my own device to test out the widgets I made. Unfortunately, I can't, and I have to wait 7 days.

You might ask, "How did you use up all 10 App IDs already?" My answer would be that I followed the "Develop in Swift" tutorial, which guides you through building multiple apps.

Conclusion

Apple's primary message and agenda in recent years have been to make their tools more accessible so that anybody can code. However, Swift and SwiftUI introduce many abstractions that novice developers might initially overlook.

I want to clarify that I'm not gatekeeping. I would be proud to see a novice developer build an app from scratch, but these abstractions could pose challenges in later stages of development for beginners. For instance, they might wonder, "What if I want to build a Kotlin Composable version of this app?"

This is my opinion, but I'm confident that determined novice developers would find ways to overcome these challenges.

Swift offers numerous language features that simplify development, and Xcode's deep integration with Simulator and macOS (enabling apps to run directly on your Mac) makes it compelling for developers to create apps across multiple Apple platforms (iOS, iPadOS, macOS, watchOS, tvOS, and VisionOS).

SwiftUI has emerged as a leader with its declarative approach to UI development and simplified state management. Gone are the days of setState() and React.createContext(). States and contexts are now more intuitive, not to mention the straightforward approach to data persistence with SwiftData.

In conclusion, Swift is easy to learn and encourages developers to create better mobile apps through tightly integrated APIs like TipKit, SwiftData, WalletKit, WidgetKit, and more. These APIs enable developers to think less and accomplish more – which should be at the heart of every app.

For React Native developers, adopting Swift can enhance app personalization for their Android counterparts. While React offers native APIs that interface with the mentioned APIs, having Swift in your skillset offers unique advantages.

Flutter developers transitioning to Swift will find the switch relatively straightforward. Concepts like states transfer smoothly from Flutter to Swift (and vice versa with React to Flutter), and mastering Swift within a day or two is achievable.

My main mission is to not spend a single dime and rely on ways to make this application free. The last thing I want to do is spend money for graduation.

The Site (though you can't do much because you don't have an access code)

In this blog post, I'll run you through my thought process, splitting into ideas and then bringing everything together with conclusions and what I've actually learned.

~ Starting off with how I would like to 'design' the access code for each individual.

Ideas for an access code

The access code plays an important part. Each person will receive a unique access code. Initially, I came up with the idea to use pass-phrases such as shuffle-playing-encircle-thickness. But then, I want the access codes to be personalized for each person.

And that's where the idea came in! I can just use their name followed by adjectives. Such as jane-the-broccoli or jack-the-OG.

After all, personalization is really my thing ✨

Okay, where do I store these images and videos?

I sat down and thought of ways to host images and videos for free. Each person averages 3 pictures, showing my fondest memories with them throughout my 4 years in poly.

First, I thought of using AWS S3 Free-Tier. It was generous and also I have used it during my internship. But then, I don't want to spend time learning it again – I totally forgot how to upload assets to S3 programatically. Moreover, I want the site to be up and running. Graduation is coming in two weeks!

I sat down and realized I can use Imgur and unlistied YouTube videos!

I got the assets sorted... What framework should I use for the site?

Okay, now that assets will be handled by Imgur and YouTube. I need a framework that can help me get the site running in no time. Something that, perhaps, have simple hosting solutions (that are also free!) Sounds familiar? – was it Next.js?

Apart from Next.js being the fastest framework I can use to get the site up and running, other considerations such as easiness of spinning up an API route and Server-Side Rendered (SSR) pages are why I chose Next.js

Anyways, in a later section, SSR will play a huge role. Keep this in mind!

Designing the data structure

Again, I want it to be fast, not in terms of speed of the site but the development time. By now, graduation is about 10 days away. The first thing that came to my mind is JSON! Why not, they're easy to access anyways. That is when I designed the following data structure:

Let's say John is a really great friend of mine, here's how his data will look like:

"john-the-og": {
  "name": "John",
  "content": "Thanks John!",
  "images": [
    {
        "url": "https://image.com/img.jpg",
          "desc": "Us in class"
    }
  ],
  "videos": [
    {
        "yt_embed": "9D0Sjd99J",
          "desc": "A song that reminds me of you"
    }
  ],
}

Ah, that makes it simple. Each person will have it's access code as the key and the data as the value of the object.

Now... where do I store all these?

Storing the data

Okay, call me wild, but for the sake of development speed, I initially wanted to just have a data.json file containing all the data and access code in the structure shown above. But after some thinking, I didn't want the codebase to have any traces of anyone's data – not that it contains personal data. It's just that I want to make it as 'modular' as possible.

I guess deadlines do make one think of crazy things.

If you're thinking "Why not just spin up an AWS RDS instance or a Supabase project?", you'd be half-right. While they're fast to spin up, I'll have to bloat the site (which should be simple) with Supabase client library or even worse... ORMs.

That's when I remembered in an earlier project, plsgrade.me, I used Vercel KV to store some user-generated data. Moreover, it checked a lot of boxes. First being that it is fast to spin up, second is seamless integration with Next.js.

So that is exactly what I did. I spun up a Vercel KV storage (which then spins up an upstash instance – Redis) for me.

The only downside of this approach is that every time I need to write data, I'll have to store it in Redis in String and when fetched from the storage, I'll have to parse the JSON. Moreover, due to the nature of how I chose to create data entries, I would need to omit any apostrophes from the stringified JSON (i.e. "you're" -> "youre")

I wrote some test data, manually, to test if the application really shows up.

The first test

Image of the home page

What it looks like when accessing the page using the correct access code

Time to generate data for the rest of the people...

By now, graduation is about 1 week away, I was thinking of ways to improve the application. The first test data works... It shows up, I'm ecstatic.

Now, I'm writing each person's data into Vercel KV CLI.

But this task is proven to be a chore... For every person, I need to type out the data structure, change the name, content, url and then copy the key for the access code and paste in the value in the CLI format: SET <key> <value>.

That's when I made my own script to help me generate data in a type-safe way and pretty-print, so that I can just copy and paste into the CLI: generateData.ts.

And running the file would output these into the console, making it easier for me to bulk insert data into the CLI:

A – A ready-to-copy-and-paste text to paste into Vercel CLI

B – The URL for the QR Code

The essence of anticipation!

With about less than 7 days, I prepared everything – the goodie bags and the site – cleaning up messes, smoothing out rough edges.

One thing that was missing is the essence of anticipation – taking cues from other sites, some sites incite anticipation via countdowns. And that is what I did!

I created an if-check to check if the user's current date is before 6th of May, 6 p.m. (Graduation date)

Big Issue!

After deploying the test version, I realized that users are able to change their device's date and time to skip the countdown and display the hidden content.

So, how did I fix this? I knew that the problem lies in that the check for the time checks it against the user's date and time. I researched on ways to solve this and came down to the solution – instead of relying on the user's date and time, rely on the server's date and time.

I created an api at /api/time

Remember when I said SSR played a part? It does because when the component is rendered and sent to the client, no other code is being sent to the client. If you view the page source, you'll only see the code for the countdown but not the hidden content. SSR – 1, Hackers – 0.

Final touches

Okay, everything is done. From the access code, to the note, videos, images and the anticipation element. One final thing, which was REALLY not needed but somehow fun to do is the idea of access logging.

I would love to know when someone entered their access code and viewed their note.

I leveraged Redis, again, to store the data. Instead of using a simple GET and SET, I used LPUSH (List Push) which allows me to store items as part of an array. Retrieving it is as simple as using LRANGE.

Tada, it works!

Now, all I got to do is send to my friends the website and wait for the day to come.

Conclusion

The day came, and I think the site is well received:

What did I exactly learn?

Yeah, this seems like a simple job. There is no out-of-the-world technologies being implemented. But hey, every project has a lesson.

Countdown and it's nuances

Since it is my first time working with time, the issue of relying on user's date and time versus relying on the server's date and time is new to me. And I'm pretty sure I'll keep this in mind for future projects.

Using Redis

While learning how to use Redis, I also learned that Redis can be used for plethora of other things such as rate limiting. Isn't that interesting? This is something new to me, and I'll explore using Redis for my other projects. I know that you can use Redis as a complete data store but not recommended to. Maybe I'll explore data caching using Redis?

What was done right?

I feel that I am proud of how fast I got the project up-and-running. Considering some security measures implemented, which is overkill, would benefit other projects BUT there is no harm to me using what I've learned thus far into this small site.

What could've been done better?

Let's take a step back. If there was something that can be done better, it is to create a dedicated page for data creation rather than having to manually create data form the Vercel KV CLI.

Maybe this new page can interact with Imgur's and YouTube's API to handle uploading of assets rather than me having to manually upload the assets and copy the URL and ID.

It was a pretty old project, with any major change being about 2 years ago. The most significant thing after was moving to Chakra UI as the UI library.

The Next.js version of the project was 12, still using the page router. While it was sufficient for that time, I came back to work on the project to work on a nifty new feature that I have been planning for days...

So here I go, into the codebase...

The messy code

Coming back, I wasn’t really given a nice greeting with how messy the code is. Messy, as in, redundant functions, unneeded abstractions and the lack of code comments.

The lack of code comments made it hard for me to navigate and change the code.

Recently, I’ve watched advancements in Next.js. Introducing App Router and all the cool features but it was a bit of a jump for me, which is why I didn’t bother upgrading the Next.js of the project to version 13 at that point in time… Until recently…

Preparing myself for migration

I read the migration guide from Vercel themselves

https://nextjs.org/docs/app/building-your-application/upgrading/app-router-migration

And then I realised, why not make the switch and along the way, clean up the mess I’ve wrote?

I prepared myself for the hell to come, knowing that a lot of functionality will break. I prepared myself by creating a new branch (feat/migrate-nextjs) and got my hands dirty.

Initial migration steps

According to the guide, we just have to install the latest Next.js, React, React DOM and the ESLint configuration. And then, we can incrementally adopt pages from the pages directory to the app directory.

Remember how I mentioned that I’ve watched advancements in Next.js and didn’t bother to upgrade? One of the main reasons is the seemingly different file structure. For example, why is there page.tsx and why can’t we name the routes as is such as signup.tsx anymore.

But after doing some reading and trying it out for myself, I actually thought the new way is genius! The page.tsx serves as the main page, the content of the page while other tsx files such as loading.tsx serves as the component to show when the page is being loaded from the server (if it is an RSC) and error.tsx serves as the component to show when there is an error.

The migration process was made simpler as the old pages from the pages directory are still working, thank you Vercel for allowing me to incrementally adopt the app router.

A taste of React Server Components

At that point in time, the biggest feature that came with future version of Next.js was the use of React Server Components. Watching their Next.js 13 video, I am quickly amazed by how you can just call database functions from your React Components. It’s like magic! In the past, I’d have to expose an API route and make a HTTP request from the client. But now, I can just call database functions directly? You can imagine how mind-blown I was!

This, in result, cuts down the codebase by a lot. In the past, Musicn had a lot of API routes exposed, each of them has a lot of boilerplate code because I needed to make sure it was as typesafe as possible. But with this new addition, types are automatically inferred from our database calls (types inferred here refers to types from the Prisma client).

Moreover, React Suspense allows me to show a loading indicator as the page loads, this is a massive bump for user experience. In the past, because of getServerSideProps , there wouldn’t be any loading indicator that the page I clicked is loading, all I got was a page that was frozen.

The trouble of migration

In the process of migration, there were a ton of hiccups. So much so that I devised a plan:

1. Migrate key app features first (exclude styling, and minor features)

2. Either prepend old API/page files with ‘_’ (i.e. ‘signup.tsx’ → ‘_signup.tsx’) OR place conflicting old API files in the ‘.old.api’ folder in the root of the project.

3. Implement proper access control in the now-migrated features.

Using this plan, I was able to migrate key features (except login/signup) within the first day of migration. And finished up the migration of other features the very next day.

But realistically, this is how it went:

1. Migrate features

2. Clean-up code

3. Test deploy

4. Build fails

5. Fix issue

6. Test deploy again

Repeating steps 3-6 for about 5 times.

If every failed build lowers my sanity by 10 points, I'd be dead by now.

Use client? Use server? What exactly am I supposed to use?

Yep, trouble arises! With the new 'use client' and 'use server' directive, it is now simpler to dictate whether a component should be a normal React component or a Server Component.

The impressive mind-blowing feature I mentioned earlier about calling database calls directly in your components? It is only available in Server Components, understandably. This is because the component is being processed by the server before sending it to the client.

Only pages that require SEO content is marked as Server Components. This includes: the users page, and the user profile page. Other than that, it is client components.

Does authentication have to be confusing? – Reviewing and tackling authentication (again)

In the past, Musicn’s authentication flow has been:

1. Log In/Sign Up with email and password

2. Link Spotify account to your Musicn account

3. Done

This simple authentication flow proved problematic as we cross the roads of Email activation, Password reset, Password changing and etc. Which is why I wanted to offload the authentication providers to services like Auth0.

But after meticulous planning, I found out that is it possible to just let users Sign In with their Spotify account via OAuth2.0 and let Musicn handle the creation of their Musicn account based on their Spotify profile.

So that’s exactly what I did!

Now, Musicn’s authentication flow is:

1. Log In using Spotify Account

2. If user doesn’t have a Musicn account, create one automatically based on their Spotify profile

3. Done

And also did I tell you that this process is further simplified using a library I found called Lucia?https://lucia-auth.com/

Lucia has helped immensely by abstracting the confusing parts of authentication and session management and I’ll forever be thankful for this library.

The main reason I used Lucia is because it is both customizable and un-opinionated. Allowing me to store what I want in each session (that, btw, gets stored in my database via the Prisma Adapter for Lucia). And customizable in a sense that I am allowed to pass in the scopes needed to retrieve what I need from the access token.

Conclusion

From migration to code clean-up, this experience has been humbling for me. This is what sets apart hobbyists and realists. As hobbyists, it would’ve been my instinct to immediately create a new repo and start from scratch using Next.js 13 but I avoided doing that because in a real work scenario (as backed up by my internship), we tend to upgrade versions and not rebuild and I just have to have a taste of what it feels like to migrate to a major version of a framework.

With the new features added in Next.js 13, I am just glad that the codebase got much more readable and cleaner. And I am really hoping to see what the future of Next.js brings with React Developer Team’s advancements in React – such as introducing form actions.

The Problem

Now here comes a problem that occurs a lot. We receive notice that one of our modules are broken (Payment Module). This would require the whole backend to have a downtime while we fix/patch and re-deploy. Other modules (e.g. Search) would be affected too because of the downtime. In this modern world where there are millions, if not billions of users of the internet, is this really the way to go?

Introducing Micro-services

Micro-services solves the problem by lifting different modules of our application in a separate container. This allows for the easy plug-in-plug-out scenario. Let's say here is the proposed Micro-services architecture of our current app (not comprehensive):

So how does Micro-services solve the problem?

Simple, we do not need to bring down the whole backend to fix one module. We can instead plug-out, fix and plug it back in, redeploy the affected service without affecting other services (that are not reliant)

In the old architecture, the Backend is affected (as a whole)

In the new architecture, only the Payment service is affected

Companies that adopt Micro-services

While discovering this new technology, I am sure that I am not the only one who sees Micro-services the same way. Companies such as Shopee, Google and Netflix are adopting and using Micro-services in their products to serve large scale software.

Testing and Running Micro-services locally

You can run Micro-services locally using Docker. Writing backend services that adopts the Micro-services pattern is available using Nest.js

You have to simplify the way you are explaining things.

First, you grow up in college (polytechnic) with the fact that the friends or classmates you have are well-versed in your course (or study). Everyone around you know what you mean by "debugging" and "fixing that one bug". And it was somewhat of a culture shock to me that in the real world, not everyone understands the "under-the-hood" processes that happen. This is highly emphasized when my supervisor (or boss) asks for the timeline and I have to explain that there will be some setbacks to the initial timeline because of technical limitations (in which I go in-depth) until the confusion sinks in their face and I realized that they do not understand a single bit of what I have just explained.

I find it easier to use analogies in real life or maybe a run-down version of what is happening.

You are expected to work overtime.

Sure, we get pushed around as interns. But I do have to remind everybody that the position I signed up for requires us to have experience in React, Typescript, Nest.js and other frameworks. The lie in "no prior experience needed" is a huge lie and should be the biggest sin in this world.

Obviously, I'm joking. But the real fact is that we are expected to work overtime even though not explicitly said. I feel pressured because I am the project manager and am expected to be the man between my team and the boss.

That also means that I need to work overtime to compensate for the things that we didn't do in the office, such as creating and reviewing Pull Requests, merging them and ensuring compatibility.

And I'm not being paid for overtime (but who am I to complain, After all, I am an intern).

However small your company is, team structure is important.

Small companies tend to not have the budget for their developers. And that is factual. When I entered the company, there was no structure. Everything was here and there and very disorganized.

This makes collaborating hard. Which is why I introduced structure within the software team. Some of them include using some elements of the Scrum methodology such as having a backlog, reviewing code, code review and retrospective. I also instilled the importance of VCS, Pull requests, branch naming conventions, conventional commits, atomic commits and many more. As a result of the policies I introduced, the software team became easier to manage. Especially for the current project manager who will be accepting PRs and merging PRs.

Being a project manager is fun, you get to go on power trips.

I am the project manager for this current sprint, which is the last leg before the release of the first version of the application. That also means that there is a strict timeline involved during this sprint.

First, this is hard. On top of the overtime that I have been doing, I also have to do more work past my working hours.

Second, managing the team must be stellar. In this last leg, it is obvious that there can't be any mess-ups and if there are any, it needs to be rectified immediately. I also had to delegate tasks to team members who specialize in them. This helps the team to make faster progress over time.

I believe that being a Project Manager allows me to on a power trip but I also realize to do it cautiously. Sure, you feel like you're managing the team but it is also important to consider the interest of the team and the goal at the end. Being on a power trip is beneficial if it benefits the team, boosts team morale while working towards our goal and should never be used to self-indulge in power (I learned this the hard way).

Technical debt comes to bite you in the ass again.

I have seen this first hand (and I am sure other people too!). You create a feature, you create the most bare-bones thing to get it working and by the time you want to clean up, you realise that you've used all of your time and have to push to the development branch.

Now there are two possible outcomes: Stay around to see the technical debt bite you in the ass or quit your job, find a new job and let the new people handle the technical debt.

Surely, being an intern for a year, we're on the former outcome. Yaouch! The technical debt in question is the "chat" feature that is enabled by Ably that is poorly implemented that bites us in the ass which caused slowdowns in the team's progress and productivity.

Remember, always write clean code and ensure you have time to clean up the code you wrote. If you do not have time, remember that you can always revisit it later, but make it your priority to revisit.

Be ready to switch up your processes.

In a small company, things tend to get disorganized, with no structure and that is the new norm. You just have to deal with it. There may be days where the reviewer would give a green light on the feature and the next day might request additional features (that does seem easy to them) but between you and me, we all know it's gonna take time. Other processes might also not be done properly such as UI design review. For instance, the current project we are working on is based on the web version which means that the components on the mobile version of the app are copied from the web (with some pizzazz – such that it works on mobile) and we are working without wireframes (well, mainly because our company has no designers) and these UI decisions are instead made by the boss. The best way I could say it is this: "Some UI decisions are questionable".

So just keep in mind, that some processes that are taught to you in school may be switched up in the real world and these may be due to lack of manpower, costs or even time.

At the end of the day, we must remind ourselves that the projects we built are going to be used by our clients and not you, hence we don't need some of those trendy 3D things that are trending in UI these days.

Conclusion

I've never been at the other end of the spectrum, working in a big company with a relatively big software team. I'm pretty sure big companies' software team have more structure in them and does code review for real (rather than our role-play positions). But at the same time, these past months have been a great experience to learn and do (and also manage a team). Being at a small company with a half-assed software team has never been easy and will never be but at least we got some perks.

As a web developer, I have always been fascinated by the possibilities of WebSockets, and I was eager to put my skills to the test by creating a web-based game that uses this technology.

Introducing Troof!

Troof!, the online-based truth or dare game that brings people together through the power of honesty and adventure. In Troof!, players can chat and react to each other's truths and dares, creating a fun and interactive experience that fosters connection and bonding.

You can play Troof! here*:* https://troof.nabilridhwan.com

The inspiration for Troof came about when my friend and I were playing truth or dare over a voice call. One annoying thing was that I had to repeat the question because they were in the middle of doing something else.

This is where the idea for Troof was born! - a truth or dare game that always displays the question to all players in the room, with the added functionality of chat. With Troof, you can play without the need for a voice call and never miss a question again.

The Planning Stage

Scalability?

Initially, I tried putting both my socket.io server together with next.js, but I quickly realized that this would not be scalable.

In order to improve performance and ensure that my application could handle a high volume of users, I decided to move all backend services to their own separate servers.

This allowed me to better manage and optimize my application, and ultimately provide a better user experience.

So Troof now has a frontend hosted on Vercel and a separate backend handling all the little socket.io stuff.

Usage of TypeScript in this Project

I knew from the start that I wanted to use TypeScript for both the front end and back end of my application.

The reason for this is that TypeScript is strongly typed, which means it is less prone to stupid mistakes.

By using TypeScript, I was able to catch errors early on and write cleaner, more reliable code.

Overall, I found that TypeScript greatly improved the development process and helped me create a more stable application.

The Development Stage

Socket.io and TypeScript?

As I continued to develop my application, I realized that socket.io can be used with TypeScript to define how server-to-client and client-to-server events would look like. (Reference: https://socket.io/docs/v4/typescript/).

An example of it would look like this:

export interface ServerToClientEvents {
    [EVENTS.PLAYERS_UPDATE]: (players: Player[]) => void;
    [EVENTS.GAME_UPDATE]: (room: Room) => void;
    [EVENTS.LEFT_GAME]: (playerRemoved: Player) => void;

    [TRUTH_OR_DARE_GAME.INCOMING_DATA]: (log: Log, player: Player) => void;

    [TRUTH_OR_DARE_GAME.LEAVE_GAME]: (room: Room) => void;
    [TRUTH_OR_DARE_GAME.SELECT_TRUTH]: (room: Room) => void;
    [TRUTH_OR_DARE_GAME.SELECT_DARE]: (room: Room) => void;
    [TRUTH_OR_DARE_GAME.CONTINUE]: (log: Log, player: Player) => void;
    [TRUTH_OR_DARE_GAME.JOINED]: (log: Log, player: Player) => void;

    // Messages
    [MESSAGE_EVENTS.MESSAGE_NEW]: (message: MessageUpdate) => void;
    [MESSAGE_EVENTS.MESSAGE_ANSWER]: (message: MessageUpdate) => void;
    [MESSAGE_EVENTS.MESSAGE_REACTION]: (message: MessageUpdate) => void;
    [MESSAGE_EVENTS.MESSAGE_SYSTEM]: (message: SystemMessage) => void;
}

This interface defines how Server to Client events are made. So every io.emit or socket.to().emit() follows the interface structure above.

This interface is then used in the initialization of the socket, as shown below:

const io = new Server<ServerToClientEvents>(server);

This allows TypeScript to scream at us if we give an unexpected payload for an event and also gives us nice auto-completion.

This interface ServerToClientEvents and ClientToServerEvents are the same on both the backend and front end so it gives us really strong code maintainability.

By using socket.io and TypeScript together, I was able to create a more robust and efficient communication system between the client and server.

This helped me avoid common pitfalls and write better, more reliable code.

Clean architecture for Socket.io

I learned to make my code cleaner by following best practices

io.on("connection", (socket) => {
    console.log("Current active sockets: ", io.engine.clientsCount);
    console.log(`A user connected (${socket.id})`);

    roomHandler(io, socket);
    gameHandler(io, socket);
    messageHandler(io, socket);

    socket.on("disconnect", () => {
        console.log(`A user disconnected (${socket.id})`);
    });
});

In each of the "handler" files, I pass in io and socket as such:

In roomHandler.ts,

const roomHandler = (
    io: Server<ClientToServerEvents, ServerToClientEvents>,
    socket: Socket<ClientToServerEvents, ServerToClientEvents>
) => {
    console.log("Registered room handler");

    const joinRoomHandler = async (obj: RoomIDObject) => {
        // Server to Client events
        io.emit('...', {});
    };

    // Listener here (Client to Server events)
    socket.on(EVENTS.JOIN_ROOM, joinRoomHandler);
}

Notice that I specifically cast the io and socket to use my generics while also highlighting where the client-to-server and server-to-client events happen.

The troubles with using Socket.io with Next.js

One of the challenges I faced during development was dealing with socket.io and next.js. Every time a user visited a new page, they would use a new connection of socket, which caused performance issues and made it difficult to manage multiple connections.

The useSocket hook

The useSocket hook is meant to be a simple hook that can be used by every file that needs a socket instance. It checks if there is already a socket or if useEffect already ran, in which it will return.

export function useSocket() {
    const [socket, setSocket] = useState<Socket<
        ServerToClientEvents,
        ClientToServerEvents
    > | null>(null);

    const [effectRan, setEffectRan] = useState(false);

    useEffect(() => {
        console.log("USESOCKET: USEEFFECT RUNNING");
        if (effectRan || socket) return;
        clientSocketInitializer();

        return () => {
            if (socket) {
                console.log("USESOCKET: Disconnecting socket");
                (socket as Socket).disconnect();
            }
        };
    }, [socket, effectRan]);

    const clientSocketInitializer = async () => {
        console.log("USESOCKET: Client Socket Initializer Ran");
        const url = process.env.NEXT_PUBLIC_SERVICES_URL!;
        const s = io(url);
        setSocket(s);
        setEffectRan(true);
    };

    return { socket };
}

This hook is only used in the context I created:

export const SocketProvider = ({ children }: SocketProviderProps) => {
    const { socket } = useSocket();
    return (
        <>
            <SocketProviderContext.Provider value={socket}>
                {children}
            </SocketProviderContext.Provider>
        </>
    );
};

However, despite my efforts, I was unable to find a satisfactory workaround for this problem. I am still looking for solutions to improve the performance and reliability of socket.io in next.js.

Not everything can be replicated in development!

One challenge I faced during deployment was the issue of users continuously pressing the "continue" button. This problem was discovered while testing and playing with my friends.

Since the backend was regionally far, it would take a few hundred milliseconds to process each request. This led to the game skipping multiple users' turns.

To address this problem, we implemented a simple solution - a loading state that would be set to true when the user clicks on continue and then set to false when new data is received from the backend socket (in this case, the next player).

This allowed us to prevent the game from skipping turns and provided a smoother, more enjoyable user experience.

It is important to test your application after deployment in order to ensure that it is performing at its best. Testing allows you to measure the speed of the backend response and get a real-world user experience. By regularly testing your application, you can identify and address any performance issues before they become major problems. This will help you provide a better user experience and improve the overall stability of your application.

The Affair of Messages and Real-time communication.

In a recent new feature that is released to Troof! I added a feature where people can reply to other people's messages.

This feature means that the chat table needs to be changed. So I added a new column called reply_to which is an ID of chat.

So if I have a chat ID of 5 saying "hello", and a chat ID that replies to 5 with "hey".

The table would look something like this:

However, it means that for every message we sent, we need to wait for the database to finish writing (which eats time), and then we will know what the ID of the "hello" message is.

The OLD way is shown below

The items in red are what are taking up a lot of time.

So I sat down pondered and wondered to myself "what is a better way for users to have that real-time feeling and also have the items written to the databases at the end of the day?"

Then I had an idea. What if I replaced the chat's ID with UUID and then on the backend, generate a UUID and write it to the database? This way, I do not need to wait for the writing to be complete because I will be the one supplying the ID.

So here it is, the NEW way:

Note that the brown part is what changed and the green part is the background processes.

Using this new method. I can keep that real-time communication by the message with the ID broadcasted back and then have it written to the database in the background. I do not need to wait for the database to write!

Authentication over Socket.io?

To send your JWT Token and as part of every WebSocket request

First, make sure that your initialization of the client Socket.io passed it's token.

io("http://localhost:3030", {
    auth: {
        token,
    },
});

At the backend...

io.use uses express' middleware pattern. This snippet below is of the index.ts server file and it's not part of any event handlers.

io.use((socket, next) => {
    // This token is part of the auth.token sent above
    const { token } = socket.handshake.auth;

    if (!token) {
        return next(new Error("Authentication error"));
    }

    const verifiedData = JWT.verify<PlayerIDObject>(
        token,
        process.env.JWT_SECRET!
    );

    if (!verifiedData) {
        return next(new Error("Authentication error - cannot verify token"));
    }

    socket.data.player_id = verifiedData.player_id;
    next();
});

Accessing the "player_id" in your listeners is as easy as...

const selectTruthHandler = async (obj: RoomIDObject & PlayerIDObject) => {
        // ...
        const current_player_id = getCurrentPlayerID();

        // Note that the data.player_id we did above is part of the socket object now!
        if (socket.data.player_id !== current_player_id) {
            // ...
        }
}

As you can see above, you can use JWT to have authentication as part of your Socket.io application to process if the user can have access to your sockets or not.

This has helped me to

• Disallow unauthorized "continue" turns in the game if the event is not received by the current player or the party leader.

• Disallow getting unauthorized information about a player.

• Disallow random people from joining rooms (someone using a POST request and etc.)

End-to-end encryption of messages

As a programmer, my top priority is ensuring the security of my programs and systems. I understand the potential consequences of security breaches and take the necessary steps to protect against them.

I learn this stuff either as part of my curriculum at school or as self-learning.

Asymmetric Encryption

I learned about the concept of Asymmetric encryption as part of an elective in school called "Cybersecurity Essentials" and how it is an end-to-end encryption system. One particular word that pops up in my mind is "RSA".

What is RSA?

RSA is a widely used encryption and digital signature algorithm. It is named after its creators, Ron Rivest, Adi Shamir, and Leonard Adleman, who developed it in 1977. RSA uses the mathematical properties of prime factorization to encrypt and decrypt messages, making it a popular choice for securing sensitive data. The algorithm has been widely adopted in a range of applications, including securing online transactions, encrypting email messages, and providing authentication for users accessing networked systems.

Why was RSA my choice?

It was my choice because mainly it has a private and public key, and its asymmetric!

On my way to make messages end-to-end encrypted.

I make up ideas on how do I make messages encrypted in-store and in transit and I made the data visualization below.

The main points are

1. All messages in transit from the user to the server are encrypted using the room's public key.

2. All messages in the database is encrypted using the public key. (The backend decrypts the public key encrypted data with the private key)

3. All messages in transit from the server to the user are encrypted using the room's private key.

4. The client (or user) decrypts any new messages from the server using their public key.

Result

All messages are end-to-end encrypted

Debugging Socket.io using Postman?

Postman recently introduced the ability for us to debug Socket.io events just like how we always use Postman to debug our REST API Endpoints.

I am able to send and listen to events just and this helps a lot rather than manually making a request everytime.

Conclusion

In conclusion, I learned a lot while developing my first ever WebSocket-powered game. I learned the importance of thinking about scalability and the benefits of using a separate backend. I also discovered the power of using TypeScript and socket.io together, which greatly improved the maintainability of my frontend and backend code. Finally, I learned the value of testing your application after deployment in order to identify and address any performance issues. Overall, I found that this project taught me many valuable lessons and I am excited to continue exploring the world of web development.

Special thanks

I would like to extend a special thanks to Jaslyn, Xuan Rong, and Windy for playing with me after deployment. Their feedback and testing helped me discover some game-breaking bugs that I was able to fix. Their support and encouragement made this project a success and I am grateful for their contribution. Thank you!

When I first created the other Musicn, I was putting my skills at that time to use. I used my knowledge in API design, React, and the PERN stack to create the application.

As time passes, new technologies emerge, and I want to use them to create the best possible application.

Introducing, The new Musicn

Musicn-next (or the new Musicn) is powered by Next.js and Serverless architecture. The speed is unbelievable as compared to the previous Musicn.

Super-fast Musicn?

The problem with the old Musicn is that it is a slow website. It is really slow. Each API request can take up to 2 seconds, which is such a big no-no, especially for users.

In research by Jakob Nielsen titled "Respone Times: The 3 Important Limits", he said:

• 0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result.
• 1.0 seconds is about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 seconds, but the user does lose the feeling of operating directly on the data.
• 10 seconds is the limit for keeping the user's attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely highly variable since users will then not know what to expect.

It is important to note that the response time is not the same as the time it takes to render the page and the time it takes to send the response to the user. And it is my mission to make the response time as fast as possible.

The Initial Problem

Converting all current existing APIs

The old Musicn has many endpoints. And a bulk of them is focused on authentication, and my goal with the new Musicn is to make it a full-stack application. This new goal came to light so that development will be maintained on ONE codebase.

Solution

However, Next.js allows us to make API routes simply by making a file in the pages/api directory. The mapping of the route will be the filename. So user.ts will map to /api/user in the URL. How cool! Moreover, Next.js API's API and syntax are akin to Express.js, making the switch even easier!

Better solutions to query data from the database

The old Musicn uses pg for ultimate speed. But there is a trade-off regarding readability and how developers can get on board with development. It is hard because you need to type in SQL databases manually, and then you need to know the table names, column names, the datatype, and relations. AH! IT'S A MESS!

To integrate an ORM into the old Musicn will be counter-productive and many of its codes need to change, and that will make everyone go insane.

Solution

This new Musicn has one goal, and that is to use an ORM to get data. And Prisma is the ORM of choice. it is our choice because of how developer-friendly it is, the autogeneration of the schema files, and many other features that make it stand out from other ORMs such as Sequelize and TypeORM.

The shortcomings

No implementation has 0 shortcomings, it's always a compromise. And building the new Musicn, I am humbled by some of the shortcomings.

Next JS middleware

Although Next.js has a similar syntax to Express.js. One of the things I missed in Express.js is the middleware. After one hour of looking for middleware solutions and then looking for more, even trying the new Middleware feature introduced by Vercel. I used an old method:

import APITokenHandler from '@/util/APITokenHandler';
import { NextApiRequest, NextApiResponse } from 'next';

export default function withProtect(
    handler: IHandler,
    rejectWhen: APITokenHandler.REJECT_WHEN = 'none'
) {
    return (req: NextApiRequest, res: NextApiResponse) => {
        console.log(
            `=====CHECKING FOR TOKEN IN ${req.url}, (REJECT TYPE = ${rejectWhen})=====`
        );

        // TODO: Cleanup code
        if (rejectWhen === 'none') {
            if (!APITokenHandler.hasAPIToken(req)) {
                return APITokenHandler.reject(rejectWhen, req, res);
            }
        } else if (rejectWhen === 'has') {
            if (APITokenHandler.hasAPIToken(req)) {
                return APITokenHandler.reject(rejectWhen, req, res);
            }
        }

        return handler(req, res);
    };
}

And then, it is used by wrapping the API handler:

export default withProtect(handler as IHandler);

Deployment

Vercel provides hosting for Next.js sites, and it's a one-click solution. Register for an account, link your GitHub, select your repo and deploy! (and also some configurations of environment variables)

Some tweaks I have done are:

Changing the country of where the serverless function runs

By changing the Serverless Function Region, the speed of API calls increased significantly. (and obviously will introduce some extra time if someone is further away from the Singapore server)

Using Caching headers

Read more: https://nextjs.org/docs/going-to-production#caching

Using caching, I can cache responses for the APIs and pages. This makes it fast! I even made a class to handle caching headers for me:

export default class Cache {
    private static headerName: string = 'Cache-Control';

    /**
     * Revalidates the cache in the edge server while showing the old stale data meanwhile.
     * @param res NextApiResponse
     */
    public static revalidateInBackground(res: NextApiResponse): void {
        res.setHeader(this.headerName, 's-maxage=1, stale-while-revalidate');
    }

    /**
     * Caches the data in the edge server and expires after the given seconds, to which, it'll refetch and cache new data again.
     * @param res NextApiResponse
     * @param expiresAfterSeconds number
     */
    public static inEdgeServer(
        res: NextApiResponse,
        expiresAfterSeconds: number
    ): void {
        res.setHeader(this.headerName, `s-maxage=${expiresAfterSeconds}`);
    }
}

Conclusion

I have learned a lot about the React ecosystem and the ecosystem of Musicn. I have redefined endpoints to be more useful and as simple as possible.

Along the development journey, I learned more about technologies and how they work, and how it is important.

I learned even more stuff from when I built the old Musicn.

• Caching
• Server-side rendering
• Next.js
• Prisma (in production)
• Serverless architecture

Rebuilding applications using new shiny technology is always fun, it allows you to explore and learn more (even refresh your memory on what you thought you knew).

My message to upcoming aspiring developers: Build what you're passionate about, and continue building it. There's never a "complete" software, there's always room for improvement, and you shall pursue it in terms of watching talks of technologies and figuring out a way how to integrate it with your product.

I have already set up a proxy. Please email me at contact@nabilridhwan.com privately!

Read more here: https://signal.org/blog/run-a-proxy/

This article uses DigitalOcean Droplets and a Domain name from Cloudflare to run a Signal Proxy and focuses on being as user-friendly as possible.

What you'll need

• A VPS (DigitalOcean) ($4/month) - Free credits for students under the GitHub Student Developer pack
• A Domain Name (Cloudflare) ($14 per year)

Creating a DigitalOcean Droplet

1. Create a new Droplet in DigitalOcean. The basic $4.00/mo plan will help.
2. Create new SSH Keys by clicking the new SSH Keys. The generation of SSH keys could be found in the original DigitalOcean documentation. But in this case, I used 1Password to generate an SSH key.

Configure domain in Cloudflare Dashboard

1. Copy your Droplet's IP Address
2. Go to your Cloudflare dashboard, log in and configure one of your domains. (We are going to add a subdomain)
3. Configure accordingly by making an A record with the name which points to the IP address you copied.
4. Click save.
5. Wait. (Different timing for different domain name providers but mine took approximately ~10 minutes)

Installing the proxy on your new Droplet

Note, you can follow the instructions written by Signal: https://signal.org/blog/run-a-proxy/

1. After creating the droplet, wait a little while for the droplet to set up, and then right-click on the three dots and click on 'Access Console.'
2. You'll be redirected to a new page, remember to log in as root and click on launch droplet console.
3. From here on, you can follow the article written by Signal: https://signal.org/blog/run-a-proxy/

What I Did

If you didn't follow the article, you could follow what I did.

1. Install docker, docker-compose, and git by running sudo apt update && sudo apt install docker docker-compose git
2. Clone the Signal TLS Proxy repo from GitHub by running git clone https://github.com/signalapp/Signal-TLS-Proxy.git and change directory to the newly cloned repo by running cd Signal-TLS-Proxy.
3. Run the helper script provided by Signal that configures and provisions a TLS certificate from Let's Encrypt by running sudo ./init-certificate.sh. At some point, you'll be asked to enter your domain name. Enter the domain name you configured.
4. Use Docker Compose to launch the proxy by running sudo docker-compose up --build -d

Voila!

You're done! Share your Proxy with the world. A recommendation is not to share your proxy URL in public since they can just add your IP to a blacklist. Instead Signal encourages people to DM each other!

Credits

Signal for having an easy-to-follow article: https://signal.org/blog/run-a-proxy/

Footnote

Sorry if this article is not explained in detail on the background information. It's just a simple article to get users with resources to join and help!

I had this long desire to create a web application that allows people to see what I listen to on Spotify. I have had several attempts at it. Until I made Musicn. Musicn allows a Spotify user to sign up for an account and have their own Spotify profile to share with others. When others click the link, they can see their current listening, top songs of the month, and recently played songs.

Let's talk about Storing data

Relational vs Non-Relational database

I knew I wanted to use a relational database. I was introduced to the concept of a relational database in the previous semester in school, and I wanted to get my feet more wet. Since I'm a student, I wanted to go cheap since I'm broke and have no money 😭.

Choice of database

I settled on Supabase just because it's PostgreSQL, user-friendly because you can insert into the database with just four lines of code without typing any SQL and comes with a table editor in its online dashboard.

I used the official Supabase library for Javascript and realized that database requests take long (around 200+ ms per query). The issue became apparent to me while I was doing Authorization, and I realized that if I have no credentials, it rejects the request in a mere 7ms. However, with credentials, it had to do some queries, resulting in 200+ ms of query time!

I knew something was wrong because the query times for the database should be way shorter than that (based on my school final assignment for the year).

I made a huge realization. The realization is that the Supabase library fetches the Supabase API, which takes around 200+ ms to resolve, and it all started making sense. So instead, I used a library called pg. And my query times are shortened to 90-100+ ms. Sure not a big difference but after several testing, I concluded that the difference it made to the Heroku deployment is 46.8% faster!

Talking about Security?

Security of user data

Security plays a massive role in Musicn since it is hosted publicly for anyone to use. The data that are used and collected are:

Musicn (when Signing up for an account)

• Email
• Username
• Password

Spotify (When linking your Spotify account)

• Email
• Refresh Token
• Display Name
• Profile Picture URL

It's essential to keep these data locked away and secure. In terms of obtaining user data, I took extra measures such as:

• Not exposing the Spotify's Refresh Token online (If it is exposed, it allows the attackers to get the token and make a request to get the user's email. This is dangerous, and hence Musicn acts as a Proxy API)
• Only display crucial information for displaying on the page (This means creating multiple model files to fetch the different sets of data from the database)

How Musicn works in terms of Proxy API to Spotify Web API

Using Supabase Javascript Library

Using pg Library

Note how the fetch request for nabil is way faster (1.18s vs. 558ms) (ignore the top_playing and currently_playing fetches because that uses Musicn as a Proxy API for Spotify Web API)

Security of JWT tokens

Musicn allows users to edit their profile by heading to the Profile page. Users can change their username so they can visit their friend's profile by going to a link such as https://musicnapp.herokuapp.com/user/nabil so hence it is a must to store the JWT token somewhere as part of its front end so that we can send requests to the backend for Authorization to the Profile page.

So I searched online to get some idea on how to store tokens securely because, in my previous assignment, we stored it in LocalStorage, and it just screams SECURITY BREACH to me! For those who don't know, storing JWT or anything remotely "secretive" in the LocalStorage is a big no-no!

After countless hours spent watching excellent talks by developers, I settled on using cookies. At first, it alarmed me because it is stored on the browser, but you can set some parameters to make the cookies way more secure upon further research. You can read this excellent article written by Ryan Glover explaining more about implementing safe cookies.

res.cookie("cookie-name", {
    httpOnly: true,
    secure: true,
    ...options
})

For my explanation, httpOnly cookies do not allow the browser to access the cookie via Javascript code.

Misc Security Stuff

Like what they say, security is obscurity, and that means not letting the attackers know what your application is built on or what database it uses. But Ironically, Musicn is open-sourced, and pretty much the world knows it's built on

• Express
• React
• Tailwind CSS
• Supabase
• pg

But if you're building an actual world application, you can remove the x-powered-by header, which tells the attacker what framework you are using, and using the backend as a proxy to APIs make Musicn somewhat more secure (because no keys are exposed)

Using React as a frontend on the same domain/port as the Express app?

One group of developers builds the backend and the frontend separately, but I was dead on making the backend and frontend run on the same domain.

In the building phase, I used fetch in the React application to fetch API requests. It went something along the lines of:

fetch("http://localhost:4000/api/user")

And then it dawned on me. What if I changed the port number? What if I want to change the host? Well, adding a variable may help. Something like:

const host = "http://localhost:4000"
fetch(host + "/api/user")

But that means I have to write the host variable at every component file I created. Another alternative you could do is to export the host variable out of a single Javascript file but it became troublesome every single time I switch between development and production. Do I have to find my way to change the value of host?

That is when I found "Proxying" in React. Since I know that my React app will run on the same host and domain as my backend, I can proxy every fetch request to this host defined in package.json in the React project folder.

     ...
    "proxy": "http://localhost:4000"
} // EOF

From then on, all my fetch requests can look something like this

fetch("/API/user")

So every fetch request in development will pass it through to http://localhost:4000, and later in production, the proxy value is discarded and will pass through the domain.

You can read more about what I've talked about here

In conclusion

Creating Musicn is a wonderful journey. I researched how to make my application way more secure and pleasant to use. Here are some take-aways:

• Using pg instead of Supabase can give you speed advantages but results may vary (depends on your use case, if you're using Supabase Auth, then yeah, you have to stick with it)
• Security is obscurity (API only shows data that is needed and does not show everything)
• Secure Cookies using httpOnly and secure options.
• Proxying API requests for development in React application.