Spotify Analytics Dashboard

Overview

This project involved analyzing the Most Streamed Spotify Songs of 2023 dataset, published by Nidula Elgiriyewithana on Kaggle. The primary objective was to extract insights such as the highest streamed track, song characteristics, and release patterns over time.

Key Questions Explored

1. How can ChatGPT be leveraged to enrich the dataset and use external tools like the Spotify Developer API?.
2. Which track has the highest number of streams?
3. How do average streams per year compare to the top tracks?
4. What characteristics define popular songs?
5. What is the release pattern of these tracks?

Process & Tools

1. Data Enrichment with ChatGPT

Initially, I attempted to enhance the dataset using ChatGPT to add song or album URLs. However, due to limitations in internet connectivity, I shifted to using the Spotify Developer API. I successfully appended the URL column by executing a custom Python script, providing direct links to the song covers.

2. Handling Missing Data

Upon reviewing the dataset, I identified the absence of a date column. Utilizing the available information (year, month, and day), I created a calculated column with the "DATE" function in Power BI. To enhance time-based analysis, I employed Bravo, an external tool, to generate a calendar table.

Visualizations

• A line chart illustrating the number of tracks released over time, leveraging the calendar table to implement time intelligence.
• Slicers for Date Range, Artist, and Track, enabling users to filter data and derive additional insights.
• Bar charts displaying stream counts for each track and artist, highlighting the highest streamed songs and popular artists.
• KPI cards presenting key data points such as the highest streamed track, total streams, and the number of tracks per artist.
• HTML card to display album artwork from the URL column, enhancing the visual appeal of the dashboard and providing instant context for the tracks and artists, making the experience more engaging for users.

Key Insights

• The track “Shape of You” by Ed Sheeran ranked second in total streams. However, Bad Bunny outperformed with 26 tracks and 15.4 billion streams, compared to Sheeran's 13 tracks and 14.6 billion streams.
• The dashboard offers interactive insights, allowing users to explore data by track, artist, and release date.

Impact and Outcomes:

The interactive dashboard provided valuable insights into streaming trends, allowing stakeholders to gain a deeper understanding of patterns in song popularity and artist performance. By integrating external tools and APIs, the project not only enhanced the dataset’s value but also showcased proficiency in data enrichment and API utilization. This integration helped to significantly improve the usability of the data, making it a more powerful tool for analysis and decision-making.

Homelab

Overview

The lag for smart bulbs to turn on always takes too long. Because most widely available IoT devices take a trip from your house to a data center and then back to execute a command like “set the lights to yellow.” The solution is to do it locally! A simple small server that acts as a central hub running software like Home Assistant can be installed on a Raspberry Pi. So, what a normal person would do, I bought a brand-new second-hand small form factor workstation. An overpowered computer for a small task. But this way could allow us to host other services, like file sharing, ad blocking, media backups, and playback, to name a few, with little help from virtualization. This technology lets you create virtual simulated servers, networks, or another physical machine, essentially cutting up the hardware resources and allocating them to multiple little virtualized computers.

This can be done on a normal computer using Windows. However, I decided to go for a type one hypervisor for the main operating system. This reduces the “noise“ for allocating underlying resources. Additionally, it's safe and ideal for experimenting with new technologies, and you can spin up a VM or lightweight containers (LXC) in seconds. For me, having a playground to tinker makes me less anxious if I mess things up, I won't have to worry about breaking my main gear.

Goals

1. Host my own services (file sharing and media streaming, home automation, private cloud)
2. Testbed for new technologies (isolation)
3. Lower cost of backing up and storing data
4. Own my data and Achieve data privacy
6. Access it from anywhere

Hardware & software

Implementation

I found an HP Z240 workstation, which will be enough to run all of the services I want and some more. I started by gutting the computer and then putting it in a roomy-er tower to make space for the new HDD, which we’ll need to store our data. After installing two additional 8GB RAM sticks (check what the max RAM is for you) for a total of 32GB of RAM, 16GB was more than enough, but more is better. My 2.5GB network card didn't arrive at this time. So I proceeded to install the OS of my choosing from a USB to the SSD. This is what is amazing about having a server you can add this later. After going through the setup wizard for Proxmox and setting the password, region, and IP address (which must be static), you have to remember them for the next step.

The first thing I installed was Pi-Hole in an LXC, an open-source DNS that lets you monitor local internet traffic and, most importantly, block ads on a network gateway level. So you can enjoy an ad-free experience on all the devices. Later, I wanted to fix what started it all. A local IoT automation solution that will eliminate our need to rely on companies' data centers and unify all controls on a single app. When normal consumers have smart home devices, they are highly likely to have more than one app. Each company will try to host its own service, such as Philips TP-link or Tuya. Home Assistant is a home automation application that has an “app store” to download all of your IoT device's apps and control them from one dashboard.

Next, I started tackling NAS (Network-attached storage) for file storage and sharing. There are many options like TrueNAS, Cockpit, or UNRAID. I installed my 3 2TB HDD and Finally decided to go for TrueNAS because it checked all the boxes for me, like ZFS storage, and has an active community. I spun up a VM, installed it, and then created a file share. Now that we have a machine that can be used to create VMs and containers in isolation and saved up in the long term on storage from the cloud and physically own our data and media, we have knocked down most of our goals. But to securely reach the server from outside the home network, you’d have to expose it to the internet. A simple way is to use a VPN. While searching for a solution, I came across Tailscale. Simply it >>>>>>>>>>. So I spun up an Ubuntu LXC and followed their documentation to advertise the network and made it an exit node. This lets you tunnel any internet traffic through the server from anywhere without exposing any ports.

Future Upgrades

next cloud for cloud storage
Setup media storagw and streaming.