1. AIM
Stock market plays a remarkable part in the economy of a country so it is very vital to identify the factors which has an effect on stock market trends. A talk about a product or a business news have a greater influence on the markets, but it is really unknown how accurately it affects the market movements. This project visualize the relation of news sentiments and stock market trends. R language’s available functions and algorithms helps to facilitate rapid data analysis and visualization and it very easy to interface with new data sources and other computing environment in the financial industry. There are more than 70 packages in R that are presumably developed directly solve the problems in computational finance.
2. INTRODUCTION
People are getting their news from different sources like traditional print, radio, television and social media. People depends on news to know what is happening in the surroundings and even used for to take a smarter choices like when and where to buy stuffs or which people to elect into office. When it comes to business mind, people are always curious about the industry which they are going to invest. Before taking a good decision, they will look for the current industry trends, corporate news, best practice company’s features and other related information about corporate industry. In short, news has a pivotal role in all fields.
Now a days stocks are considered as the pivotal financial equipment and it allows a person to own a share of a public corporation so that he can profit from earning’s growth and dividends. Google Finance and Yahoo Finance are the two most popular site to get current financial news, market data for various companies and their shares, updated news and trends. So it is very easy for everyone to know about stock specific data in every second. It also allows us to download the data in CSV format also.
For this project author have collected news headlines of ‘Apple’ from Yahoo Finance for a month. In the present scenario, succession of good news about a company may lead investors to overreact in market and it also boosts the investor’s self-confidence and reducing risk mentality. Sentiment analysis of such news helps investors more to predict the stock market trends. In definition “Sentiment Analysis is the process of computationally identifying and categorizing opinions expressed in a piece of text, especially in order to determine whether the writer's attitude towards a particular topic, product, etc. is positive, negative, or neutral [1]”.
R software provides good functionality for sentiment analysis and time series plotting. Especially quantmod in R is designed to assist the quantitative trader in the development, testing and statistically based trading models. Charting with quantmod provide a better understanding and visualisation of trade data. This project correlate the news sentiments and trading data with the help of R language functionalities.
3. LITERATURE REVIEW
There are mainly two methods existing to predict stock market. They are fundament analysis and Technical analysis. Fundamental analysis is calculating the intrinsic values of company and Technical analysis is based on evaluating the company’s past market activity (J. G. Agrawal, Dr. V. S. Chourasia, Dr. A. K. Mittra “State-of-the-Art in Stock Prediction Tech-niques” April 2013) This two techniques are not enough to predict the market trends, there are various other economic and political factors are also taken into consideration. The recent and up to date political and economic information can be obtained from social media and from news reporting agencies. Nowadays people are interested to express their idea and atti-tude in Twitter. Anshul Mittal and Arpit Goel (2009) investigated the causative relation be-tween public mood from twitter and DJIA values. But there is no straight association be-tween the people who invest in stocks and who use twitter more frequently. Sang Chung and Sandy Liu (Dec 2011) stated that twitter sentiment score only forecast stock market trend if the sentiment score is moving towards positive but not negative. There is no significant pre-dictive power for twitter sentiment score. The above stated conflicts lead me to find a static solution for stock market trend prediction that is the correlation between News headlines sen-timent score and stock market data. The motivation lead to do this are, everyone are con-scious about the company which they are going to invest and online news sources which are preferably free of charge. Methodology used for the analysis is also simple, with the help of statistical computing software ‘ R’ ,Yahoo finance and some functionalities of Excel.
4. FINANCIAL BACKGROUND
History of Stock market
In the 20th century, stock market investor’s activities were very different from what exists on today, only few people purchase individual stock. On that day, the main source of information was printed on paper. History of stock prices were available in library in the form of big books that were also several months old. If a person interested in some stocks he probably wait for one more month to check how they were going. When he was ready to buy a stock, he immediately call his stockbroker and have live conversation about what he wanted to buy. Depending on what he decided to buy, the stock broker place the order within the hour but might not be executed until the next business day. Stockbroker got paid commission for every stock they sold and very stock they bought.
By the late 1980s, personal computer had become very powerful and a serious tool. As a result people started to use PCs in their homes and the emergence of internet, stockbrokers turned into online brokers and communicated via internet and execute stock transactions. Over the last 200 years stocks are considered as the best return on money than any other kind of investment. In case of stocks historical performance is not a model for anticipating performance and it is very true about the risk associated with stock market. If we identify the risk and have a better knowledge about the company which we are going to invest, then will have an excellent chance of doing well in the future.
Why Stock market is so important?
The primary function of stock market is to support the growth of industry and commerce in the country. Rising stock market is a sign of developing industrial sector and a growing economy of a country .It will also affect the pension funds because most of the private pension funds will invest in the stock market. Movement in the stock market can have an economic impact on everyday people. A collapse in the share prices have widespread economic depression. Stock market crash on 1929 showing as a key factor in causing the great depression of 1930.Stock market makes the stock as a liquid asset unlike real estate investment.
4.1 STATE OF THE ART OF STOCK MARKET TRENDS PREDICTION
Stock Market prediction is the art of estimating the future value of companies’ trade or other financial instrument that traded on exchange. More specifically, stock markets movements are analysed or predicted in order to guide investors the best time to buy or sell the trade.
Fundamental analysis of stock prediction
Fundamental Analysis involves the in-depth analysis of companies past performance, It involves understanding the company apparently in terms of its product sales, man power quality,infrastructure,profitability on investment. It also consider revenues,earnings,future growth, return on equity, profit margins, social acceptance and other data which determines the companies underlying values and potential growth for future. The concept of stock valuation can be understood by knowing some ratios. They are Price to book ratio (P/B), price to earnings ratio (P/E), debt to equity ratio, dividend yield, returns on equity etc. Each of the ratios has their own importance and none of them are inferior or superior to each other. By combining these methods of valuation, we can get a better view of the stocks worth.
Advantages of Fundamental Analysis
• It is a systematic approach and its capability to forecast changes before they published on the chart.
• High level method for long term stability and growth
Disadvantages of Fundamental Analysis
• It is very difficult to implement and all this knowledge for the purpose of automation and interpretation of this information may be subjective.
• It is not possible to time the market effectively using fundamental analysis.
Technical analysis of stock prediction
Technical Analysis is the science or skill of forecasting the future financial stock trends based on an evaluation of past price movements. Like nature events forecasting, technical analysis does not ends in hundred percent prediction result about the future. Instead, technical analysis may help traders to think what is exactly going to happen to prices over time.
Technical analysis looks for pattern and indicators on stock charts that will determine a stock future performance. It looks for peak, bottoms, trends, patterns and other factors affecting a stock’s price movement. Technical analysts are not worried about the companies’ reputation. This analysis believe that market trend moves directed by frequently changing reactions of investors in response to different influences.
Advantages of Technical Analysis
• It is cast off by majority of the leading stock traders
• It is used to assess stocks for shorter period of time
Disadvantages of Technical Analysis
• Regardless its widespread use, it is highly criticized and is subjective also [2].
• Different individuals and predict or interpret the chart differently
From this two analysis methods, we came to a conclusion that existing techniques are not safe for prediction of stock market trends as well as price of different stocks. Still there exist a gap between technologies and user requirement for a safe and accurate stock prediction system.
While collecting News Headlines it is very important to select the trade of a relevant organisation. I have selected the most valuable and profitable company in the world, that is nothing but APPLE. It beat the whole market and running a successful business. Apple is considered as the most valuable in design and uniqueness in their brand. It has greater revenue and rapidly growing net income. Apple’s growth rate is extra ordinary while compare to other brands, in case of buying any Apple gadgets customers only look for the brand name. They are not concerned about other factors and have trust on the company products. It is a greater advantage to become first in the market and beat others. The stock for Apple is always volatile. I have depend on Yahoo Finance to obtain the news feed. It helps to obtain the leading and breaking business news. We can get the RSS feed of Yahoo Finance on a worksheet with the help of Excel functionality.
Steps to obtain Yahoo Finance RSS feed on an Excel worksheet
1. Right click on the RSS icon and copy the link
Create an XML Map in a workbook using the RSS link where we want to create the RSS feed table
2. On the Developer Tab in an Excel sheet click on Source.
3. in the opened XML Task pane, click on XML Maps button
4. In the dialogue box click Add to create a new XML Map
5. Click in the appeared File Name box and copy the RSS link
6. Click Open and click OK, if a message appears
7. In the XML Source Task pane, find the item selection and click on item heading to select it.
8. Drag title from the item section to the worksheet space
9. This create a table on the work sheet with the heading from the item section
10. To see the list items, right click on a cell below headings and click Refresh Data under Developer Tab.
Sentiment Analysis is the scientific study of people’s opinion, attitude and emotions towards an entity. This entity can be product, issues, organisation, topic and so on. Since 2002 researches are taking place actively in sentiment analysis or opinion mining. Sentiment is simply meaning the positive or negative feeling implies in an opinion and some opinion doesn’t contain any impact i.e. a neutral feedback. Suggestions or feedback are so important to business and organisations because they always want to know how their products are liked by the public. In the same way we can predict stock market trends by using news headlines and it is correlated to each other.
R software is a very good software for computing and data analysis, it is free, open source and highly extensible R software provides a wide variety of statistical techniques like linear, non-linear, time series analysis, classification, correlation and graphical visualisation facilities for data analysis. It is developed at Bell Laboratories by John Chambers and his colleagues. The packages in R are helpful for rapid data analysis and visualisation techniques.
Here I am using simple sentiment algorithms used by Jeffrey Breen to determine the scores of News headlines of Apple company (AAPL) from January 1st 2016 to February 12th 2016.Opinion lexicon used by him is based on the research papers of Hu and Liu.This algorithm assigns a score by counting the “positive” and “negative” occurrences in a statement.Hu and Liu published an “opinion lexicon” which almost contain 6800 English words separated as positive and negative and also allowed to download. Main strength of R is the facility to load and scan data. We can load the downloaded positive and negative words and store in the local machine and import using scan function.
Now we will score the News headlines of APPL using Jeffrey Breen sentiment algorithm. This can be available from http://jeffreybreen.wordpress.com/2011/07/04/twitter-text-mining-r-slides/. ‘Plyr’ package in R helps to split data sets apart into smaller subsets and apply methods to parted subsets and combine the results.‘Stringr’ package used for text processing.