ay in heSTAGE-I- PROJECT REPORT
on
"Smart music streaming service"
By
Rohit Ramaswamy (115A1018)
Vinay Ambre (115A1031)
Rishabh Gajra (115A1046)
UNDER THE GUIDANCE OF
Prof. Preeti Godabole
SUBMITTED IN PARTIAL FULFILLMENT FOR THE DEGREE OF BACHELOR OF ENGINEERING
In
Computer Engineering
DEPARTMENT OF COMPUTER ENGINEERING SIES GRADUATE SCHOOL OF TECHNOLOGY NERUL, NAVI MUMBAI – 400706 ACADEMIC YEAR 2018 – 2019
i
CERTIFICATE This is to certify that this is a bonafide record of Project Stage-I of the project titled “ Title of the Project” carried out by the following students of final year in ( Name of the Branch).
Sr. No. Name Roll No. 1. Rohit Ramaswamy 115A1018 2. Vinay Ambre 115A1031 3. Rishabh Gajra 115A1046
The report is submitted in partial fulfillment of the degree course of Bachelor of Engineering in Computer Engineering, of University of Mumbai during the academic year 2018-19.
Internal Guide Head of Department Principal
We have examined this report as per University requirements at SIES Graduate School of Technology, Nerul (E), Navi Mumbai on ____________.
Name of External Examiner: _______________________________
Signature with Date: _____________________________
Name of Internal Examiner: _______________________________
Signature with Date: _____________________________
ii
Abstract
Rhythm engine will curate the music library using machine learning, dynamically and create smart playlists based on a myriad of factors.Rhythm will take away all the distraction and unnecessary elements that complicate users’ listening experience. Rhythm engine will analyze and take into consideration a variety of data, such as: user’s listening behaviour (skip, lower or raise the volume), the music library data, time and anonymous analytics information from other users. This data would help rhythm engine to better recommendations. Rhythmenginecontinuouslyteachesitselfabouttheuser’smusictaste and would help the user to explore new songs that are similar and what he/she might like.
Keywords: Machine Learning, Recommendor system,Collaborative filtering.
CONTENTS
Sr.No. Topic Page No. 1. Introduction 1 2. Literature Survey 4 3. Proposed System 7 4. Database Design 10 5. Methodology 13 6. References 17
i
Chapter 1
Literature survey
Streaming is the future of music listening. Its cheap or even free and is more convenient than any physical format. Sound quality is a lock too: In some cases these services sound indistinguishable from a CD. Streaming music has become easier than ever with the help of a variety of streaming services available in the market. We have compared some of the most popularly used music streaming services. There are many music streaming services such as Apple Music, Spotify, Amazon Prime Music, Tidal, etc. Each of these services have their own advantages that make them one of the best if not the best in that aspect. For our literature survey, we have performed a study on Saavn, SoundCloud and Spotify. Saavn being the most popular hindi and english music streaming service in India, SoundCloud being the most popular amongst teenagers and budding artists. Spotify was chosen for this study as Spotify is the best known and the pioneer in the music streaming space. Some services are known for the size of their online library, some for their exclusivity of albums/artist while some are known for their features and recommendation systems. Thesearesomeoftheparametersthatcanbeusedtodeterminethegoodness/practicality of a service. All the above services have a simple, easy to use User Interface(UI) to their app and website. They do provide an array of features for the user. Some are more of a social network for budding artists, while some enjoy exclusivity of a number of popular record labels. We have compared the three aforementioned music streaming services and have found a few shortcomings in their working. SoundCloud misses out on automatic playlist generation. SoundCloud does allow the user to create his own playlist and share them with fellow users but does not create one for the user. Saavn and Spotify both create playlists for their users. Saavn, however created a "For You" playlist at the time of registration for the user to begin with, but it not being dynamic is a let down. Spotify is the leader in this aspect. Spotify created "Daily Mixes" for all of its users. These automatically generated playlists keep changing everyday as per what the user listens to. If there are many type of songs that the user listens to in a short period of time, Spotify created multiple Daily Mixes, one for similar songs you listen to.
1
SoundCloud has the advantage of being a social network for artists as well as a popular music streaming service. This serves to be a great platform for people who like experimenting with their music. Users have the options of reposting(sharing) songs and commenting on them. Some of the songs can be downloaded for free from the platform. This also has its cons, it is good for discovering remixes of songs made by users just like you, but the original track posted gets lost at times. However, being open is a good thing for new artists to come in the limelight of the music industry. Spotify and Saavn do not have a very good profile management system. The songs one listens to is uploaded by verified artists and not by users. Saavn does not make the playlist created by the user to be made public. One cannot use the platform to search for another user’s playlist, one has to rely on the user sharing the link of his playlist in order to listen to those songs. Spotify allows the user to make their playlist public which helps the user to gain popularity on the platform. After studying the three services, none of the services are perfect but are the best in their own way. SoundCloud being a social network helps the user to discover more variety of artists and songs. Spotify helps you to discover new music and has curated playlists for each mood of yours, whether you are relaxing, working out or partying. Saavn provides songs from top artists in the country, custom radio stations and exclusive releases. Rhythm aims to overcome the shortcomings of the existing services and provide certain features that are absent in the aforementioned services. Rhythm aims to fix certain issues that may not seem big but go a long way in improving the user’s music listening experience. Rhythm wants to improve the listening experience and suprise the user with its recommendations making it adapt to the user’s taste in music.
2
Fig 1.1: Comparison of existing music streaming services
3
Chapter 2
Proposed system
Figure 2.1: Comparison of existing music streaming services
There are a plethora of parameters to be considered before recommending a song to the user. User data, music library, time of the day, volume monitor and data analytics are important sources of data necessary for the algorithms to provide the recommendation. The major sources for generating a favourable song would be the music library and the analytics data of the user collected over a period of time from all the users using or having used the system. Hence, the more the user have used the system, the copious the user’s music library is, the better would be the suggestion of a favourable song or playlist.
4
2.1 User data
The user data is the personal profile of a specific user using the system. The personal information would include the user’s age as a parameter which would be helpful in classifying the songs based on age groups and also recommending songs for the user’s in the same age group. Again, it solely doesn’t rely on just the age parameter but can be considered a significant parameter in suggesting a song. The gender, location are other various parameters which comes under user data that can be user for suggesting the songs. The user data would also be available to the other user’s to identify individual and explore other user’s library or playlist.
2.2 Music Library
Music library is a major data set that comes into consideration while predicting or suggesting a specific song to the user. It actually serves the purpose of understanding the genre, artist, period of the songs the users likes or has listened to earlier and make predictions accordingly. It helps in finding the common set of songs in two or more than two similarindividualsandsuggestingasongthatisnotcommonintheirseti.e. collaborative filtering. The number of times a user has played a song also comes into consideration giving the likeness of an individual towards a specific genre or artist, hence suggesting the individual the songs related to that artist or genre. There are various other parameters from the metadata of the songs that may be helpful in giving an accurate suggestion. The playlist suggestions can be easily done from them. Generating playlist on a specific artist or album for the user which the user likes to listen or the ‘daily mix’ which contains the playlist of songs that the user listens to daily. Overall, collaborative filtering has a major advantage as the intersection of users subsets of various songs would make it very easy in recommending a bunch of songs for any user very easily.
2.3 Data Analytics
Analytics can be considered to be calculation or prediction of certain values or trends in the users of the system. Gathering information is the major work to be done before performing analytics, and based on this data various trends of variation in the users habits can be visualised or somewhere considered in suggesting songs. The more number ofuserswhousethesystemthelargeramountofdatawillbecollectedandusedtopredict the parameters. Regression fitting can done to obtain a slope equation thereby, making it possible to predict unknown parameters accurately. Trends in music selection can be found and recommendation of songs based on weather or time of the day can be given accurately. The data analytics will also provide the daily and weekly charts consisting of top played songs to the user to explore. Variations in songs selection representing the users behaviour can be noted and can be used to suggest songs from a wide domain to the user. The top charts can be analyzed to find the popularity of the songs and their significant variations in plays. This data will be available to the user to let the user know
5
about his or hers preferences and habits. The other users in the system can identify similar music tastes and follow their profile.
2.4 Clock/Time
The type of songs played by the user during different period of the day can be used to suggest similar songs at similar particular time of the day. This can be done by collecting the data about type of songs played by the user and suggesting similar genre songs to the user.
2.5 Volume Monitor
The average volume preference of the user will be collected for every user, thereby, setting the initial volume of the user’s device to that value, making it easier for the user. The average volume of a specific song can also be calculated and added to the parameters of a song for further suggestions and calculations thereby adjusting the song’s volume according to what the other user’s are preferring.
2.6 Rhythm Engine
The Rhythm engine will be the junction where the data from various sources will be extracted and various algorithms will be applied in order to achieve specific output. The variousalgorithmstobeconsideredtobeappliedinrecommendationandpredictionwould depend on the user’s need. The output will also be sent to the Analytics to improve the integrity of the analytics data. A few factors will be selected from all for the prediction, like the suggestion of volume control won’t require data about dat/time preference and user data. The rhythm engine will hence take into consideration all the parameters and provide the prediction as needed. The algorithms used by the Rhythm engine would be from linear regression on the analytics, collaborative filtering on the music libraries of various similar user and clustering on the users of the system to distinguish similar users. The collaborative filtering and clustering would be interdependable as without clustering it would be insensible and tiresome in collaborative filtering to suggest specific songs.
6
Chapter 3
Technologies
3.1 Express
Express js is a nodejs framework for building web applications. With a myriad of HTTP utility methods and middleware at your disposal, creating a robust API is quick and easy. The apis will be called from the mobile app and the express js controllers will respond with the response. The mongoosejs module will be used as a driver to access the data storage of mongodb. Hence, mongoose queries will be used to access the data. Mongoose js module is an ORM provided by Mongo DB for modelling and querying the data stored in any clusters in mongoDB. The working of the backend system can be illustrated as
Figure 3.1: Basic working of Express
7
3.2 Firebase
The Firebase Realtime Database is a cloud-hosted database. Data is stored as JSON and synchronized in realtime to every connected client. Instead of typical HTTP requests, the Firebase Realtime Database uses data synchronization—every time data changes, any connected device receives that update within milliseconds. Firebase is used to keep track of the count of daily number of plays for a song and giving a real time accurate data to all the users of the system.The data is simultaneously synced with the mongoDB using the firebase cloud functions. It is also used in the number of likes and ratings for the song and giving an accurate count to the user.
3.3 Sci Kit
The machine learning library to be used is sci kit, sklearn. It is a basic library used for machine learning applications in python language. Various other libraries like panda, numpy are used along with this as a tool for machine learning operations. Surprise is a Python scikit building and analyzing recommender systems. Surprise provides a bunch of built-in algorithms. All algorithms derive from the AlgoBase base class, where are implemented some key methods (e.g. predict, fit and test). The list and details of the available prediction algorithms can be found in the prediction_algorithms package documentation. Every algorithm is part of the global Surprise namespace, so you only need to import their names from the Surprise package.
3.4 Android Studio
Android Studio is the official IDE for building android apps. To build our Android app – Rhythm, Android Studio would be used. Android Studio is used to develop mobile, tablet as well as apps for wearables, televisions and cars. Designing the User Interface of the app would be done in Android Studio using XML(eXtended Markup Language). There are officially two languages supported by Android Studio that are used to develop apps, they being Java and Kotlin. Rhythm would be developed completely in Java. A plethora of APIs are available as part of the Android SDK(Software Development Kit) that comes bundled with Android Studio. These APIs can be used for building layouts, writing logic for computation, controlling hardware and linking the computation to the views of the app. However, these standard APIs are not always the best option to do the job. Android Studio allows developers to use external libraries into their project. Plenty of open source libraries that can be used in one’s project.
3.5 ExoPlayer
ExoPlayer is an open source project that is not part of the Android framework and is distributed separately from the Android SDK. ExoPlayer’s standard audio and video
8
components are built on Android’s MediaPlayer API, which was released in Android 4.1 (API level 16). Because ExoPlayer is a library, you can easily take advantage of new features as they become available by updating your app. ExoPlayer supports majority of the audio formats. ExoPlayer can be used to play audio and video both locally and over the internet. Advantages of ExoPlayer over MediaPlayer API:
1) The problem with traditional Android MediaPlayer was its inability to support DASH(Dynamic Adaptive Streaming over HTTP) and SmoothStreaming which is the reason why ExoPlayer came in existence. It also provides support for HTTP Live Streaming (HLS). 2) ExoPlayer supports a variety of media formats which are MP4, MP3, WebM, M4A, MPEG-TS and AAC. 3) ItprovidesadvancedHLSfeatures,suchascorrecthandlingof#EXT-X-DISCONTINUITY tags. 4) One more very good feature of ExoPlayer is the ability that we can customize and extend the player as per the requirement allowing many components to be replaced with custom implementations. 5) Updation is not a headache as its a library project included in application apk, you can easily update by managing versions in Build.Gradle file of your project.
9
Chapter 4
Database Design
The database which will be used is MongoDB for nosql database. The advantage of choosing a nosql database is to make it convenient to organize user’s favourite selections and relate them further. For example, the saving of user’s playlist as a list as an array is easy in nosql as compared to relational sql. There will a lot of trivial relational collections present but the core schema will be of most importance. The core database schema will be consisting of
1) User schema 2) Songs Schema 3) Playlists Schema
4.1 User Schema (User Data)
The user schema will be consisting of users profile, favourite songs and his created playlists.The favourite songs and the playlists will be stored as a relation to the other two schemas (Playlist and Songs). All the user information can be extracted in an efficient query from a single database schema. The data from the user schema will be an integral part for the social network features in the system like rating, etc.
10
Figure 4.1: Working of song schema
4.2 Songs Schema
The songs schema is an integral collection that will be used in considering various factors while considering the predictions. The more the elements in the songs schema, the more theaccuratepredictionswillbe. Itwillgivethemetadataofthesongsthatwillbeusedby the algorithms and also for generating automatic playlists. The various attributes of the songs schema will consists of songs name, artist, album, period, genre, etc. Other characteristics of the songs that will be used for giving suggestions will be loudness, acoustic ness, etc. This way extracting a particular information about a song or even extracting songs for a specific album, artist or genre would be efficient. The genre attribute will be used predicting the nature of the people and clustering them in groups to suggest them the songs from similar genre.
4.3 Playlist Schema
The playlist schema will be consisting of the playlists created by all the users of the system. Each document in the collection will be consisting of the name of the playlist, timestamps and the array of lists of songs consisting of the playlist. These songs in the array will be the object id of the songs in the Songs schema. This will make it efficient in extracting playlist information and avoiding the redundancy.
11
Figure 4.2: Working of playlist schema
4.4 Other trivial collections
Aside the core collections as mentioned, various other schema’s will be used wherein the relation between the schemas will be used in order to make predictions and suggestions. Some of the collections are Daily mix collection – consisting of the automatically generated playlist by the system for a specific user based on the user’s plays. Reposts – consisting of the songs reposted by the user for social interaction.
4.5 Real-Time Database
Firebase will be used as an alternative for real time database. This database will be collecting real time data about the number of plays in a day and likes, thereby giving accurate charts about the songs currently consisting in the system. The user can view these charts for his/her exploration. The firebase real time database consists of a single node below which numerous node can be made hierarchically. These nodes would consists of object id’s of the songs and their daily count and likes. These counts will be updated in real time for integrity.
12
Chapter 5
Methodology
5.1 Collaborative filtering
Collaborative filtering models are based on assumption that people like things similar to other things they like, and things that are liked by other people with similar taste. Clustering too comes into picture before performing collaborative filtering on any dataset.These recommendation systems leverage our shopping/ watching/ listening patterns and predict what we could like in future based on our behavior patterns. It is based on the idea that people who agreed in their evaluation of certain items in the past are likely to agree again in the future. A person who wants to see a movie for example, might ask for recommendations from friends. The recommendations of some friends who have similar interests are trusted more than recommendations from others. This information is used in the decision on which movie to see. Example,
User/Item Batman Star Wars Titanic Bill 3 3 Jane 2 4 Tom 5
Table 5.1: collaborative filtering of movied
If we want to compare Bill and Jane, we can mention that Bill hasn’t watched Titanic and Jane hasn’t watched Batman until this moment, so we can measure their similarity only by Star Wars. Collaborative Filtering approaches can be divided into two main sections: user-item filtering and item-item filtering. A user-item filtering takes a particular user, find users that are similar to that user based on similarity of ratings, and recommend items that those similar users liked. In contrast, item-item filtering will take an item, find users who liked that item, and find other items that those users or similar users also liked. It takes items and outputs other items as recommendations.
13
Collaborative filtering systems have many forms, but many common systems can be reduced to two steps:
1) Look for users who share the same rating patterns with the active user (the user whom the prediction is for). 2) Usetheratingsfromthoselike-mindedusersfoundinstep1tocalculateaprediction for the active user.
5.2 Matrix Factorization
Matrixfactorizationordecompositionisoneoftheapproachforperformingrecommendationbycollaborativefiltering.Theideabehindsuchmodelsisthatattitudesorpreferences of a user can be determined by a small number of hidden factors.
Figure 5.1: Matrix Factorization
In above figure, a higher number from dot product of user-X and movie-A matrix means that movie-A is a good recommendation for user-X. This way if a prediction rating is higher than a threshold then it can be used to suggest that song to the specific user. The threshold value needs to be adjusted to a precise value for better suggestion mechanism.
5.3 Content Based recommendation
Content based recommendation will also be used for some of the scenarios where there is a cold start problem for the Collaborative filtering system.It is knowledge based. New items can be recommended quickly in content based recommendation.
14
Figure 5.2: Content based recommendation
Hence, like the collaborative filtering it does not compare users with same taste in the music but compares the similar songs and then recommends them. Content-based recommendation systems share in common a means for describing the items that may be recommended, a means for creating a profile of the user that describes the types of items the user likes, and a means of comparing items to the user profile to determine what to recommend. The profile is often created and updated automatically in response to feedback on the desirability of items that have been presented to the user.
5.4 K-Means Clustering
K-means clustering is one of the better-known non-hierarchical clustering methods that chooses K centres for K clusters and assigns each instance to the closest cluster. It then recomputes the centre of each cluster by taking the average for each variable of all instances that are part of the cluster and repeats the process.
15
Figure 5.3: K-Means Clustering
Clustering will be used to form clusters of similar users and then these clusters will be further used in performing collaborative filtering. Thus improving efficiency. Else, By using K-means clustering algorithm, we can recommend pieces of music which are close to user’s preference even though he likes several genres. The clustering of musical genres canalsobedonebyconsideringafewparameterslikeacousticness, loudness, danceability, etc.
16
References
1) https://expressjs.com 2) https://firebase.google.com 3) https://www.cnet.com/how-to/best-music-streaming-service/ 4) https://medium.com/@sifium/machine-learning-types-of-classification-9497bd4f2e14 5) Paul Covington, Jay Adams, Emre Sargin. Deep Neural Networks for YouTube Recommendations. 2016 6) https://medium.com/recombee-blog/machine-learning-for-recommender-systems-part1-algorithms-evaluation-and-cold-start-6f696683d0ed 7) https://medium.com/recombee-blog/machine-learning-for-recommender-systems-part2-algorithms-evaluation-and-cold-start-6f696683d0ed 8) https://blog.statsbot.co/recommendation-system-algorithms-ba67f39ac9a3 9) Asela Gunawardana,Guy Shani. A Survey of Accuracy Evaluation Metrics of Recommendation Tasks. 2009 10) http://www.tothenew.com/blog/introduction-to-android-exoplayer
17re…