Predict The Stock Market With Machine Learning And Python
Video Overview & Insights
In this tutorial, we'll learn how to predict tomorrow's S&P 500 index price using historical data. We'll also learn how to avoid common issues that make most stock price models overfit in the real world.
Hi everyone! You can find the code for this tutorial here - https://github.com/dataquestio/project-walkthroughs/tree/master/sp_500 .
We'll start by downloading S&P 500 prices using a package called yfinance. Then, we'll clean up the data with pandas, and get it ready for machine learning.
We'll train a random forest model and make predictions using backtesting. Then, we'll improve the model by adding predictors. We'll end with next steps you can use to improve the model on your own.
Firstly, on what basis did we arrive to use 2 days close price to predict following day's price trend ? Instead use swing highs and lows in these 2 days and then derive a relationship to predict the trend. That would work better because there is a relationship between highs and lows, be it shorter or longer timeframe.
That apart a good attempt to predict though.
You can find an overview of the project and the code here - https://github.com/dataquestio/project-walkthroughs/tree/master/sp_500 .
If you enjoyed this tutorial, check out this link https://bit.ly/3O8MDef for free courses that will help you master data skills.
That’s awsome. Thank you ❤
Chapters
00:00 - Introduction
this is so amazing
01:28 - Downloading S&P 500 price data
03:30 - Cleaning and visualizing our stock market data
so, a caveat is that this approach is based on freely available day end stock price data -> if you want to see live data, there is often a fee.
04:29 - Setting up our target for machine learning
08:19 - Training an initial machine learning model
i like the trainer very much!!
17:01 - Building a backtesting system
23:05 - Adding additional predictors to our model
Hi Vik! Thanks for sharing, I am wondering if you could share the code again as it expired on github.
28:45 - Improving our model
33:37 - Summary and next steps with the model
BRO YOU DON'T KNOW YOUR IMPACT!!!!!!YOU ARE ABSOLUTELY GOD SEND,THANK YOU VERY MUCH FOR THIS TUTORIALS,
---------------------------------
Join 1M+ Dataquest learners today!
Does this work in general algo trading?
Master data skills and change your life.
Sign up for free: https://bit.ly/3O8MDef
More User Perspectives
Thanks for the video! Is it possible to use weekly data with a monthly target variable or intraday data with a daily target variable, or will this cause data leakage/lookahead bias? Because the target variable essentially looks past the 2nd observed date.
@jacquesmieny1705This video started my journey. Thanks so much for the overview. I have since spent 100s of hours using this same method to develop automated pipelines, create a website, and now finishing up an autotrader application that will take the prediction from the ML pipeline to make a opening and closing position automatically. Turning it on in January! Best model is generating 67% ROI for last 11 months.
@joerice746716:02 mine was 0.0, i don't know where did it go wrong
@VivekChoudhary-e9bVery good!👍I am new the ML. Keep up the good work!
@Everydayhappiness123can i keep this project in resume , is it good or looks like very small project anyone please say
@RajuTatipudi-lk9jydid you try these methods in real life trading, i mean if you have then how much you have made profits, please share some P&L Videos also
@ra9118Time to make billions
@gamer7200Interesting. Why not predicting more days instead of just one? For example 30 days in the future? (More errors yes, but more precise for long-term too)
@EdViajaI learned a lot from this video. My question is what would you change if you were building the model today from what you did three years ago when this video came out. Technology has exploded over the past three years. And probably, Python has evolved as well. Based on these changes, what would you do differently?
@Doug091549Sir do some real world projects
@ankit456-04Wtf bro 😂 this data leakage crazy why not split dataset ?
@GrowthJsonFor a production project, should you be training on real-time historical stock data to update the model?
@jencinas8586Couple of questions:
Firstly, Many thanks for this it is tremendously helpful.
Is there an optimal number of predictors or does it get confused if you add more?
Secondly, If I wanted to use stock data where I only have (say 1yr) could I use multiple stocks for the training then test on different stocks?
Any advice would be really seriously helpfu.
Many thank, Ruth 🤫
Before wasting time predicting the stock market research “complex adaptive systems”
@Rcrsvcan you do a demo of stock predictions with a LSTM RNN?
@jtluns9sir i dont know how but my precision_score is coming out to be 1 which is not possible
@prathamrana8916Thank you so much
@trojean.cedricto remove rows with 0 volume - sp500 = sp500[sp500['Volume'] != 0]
@krrishmeena77If you predict correct 90 times and get 1 dolar on each as average, and if you lose 10 times and lost 10 dolar as average. Totally you lose 10 dolar. So prediction of win loss days is not important. Try to optimize the total profit and always compare the performance results also with sharpe ratio. Because gain is not always enough. Assume you win 1000 dolar today, and lose 2000 tomorrow. Then again win 1000. Even if you win 10 000 at the end. You would not feel comfortable.
@yatrgitsin9067I'm sorry but this video is bullshit. If this guy could predict stock market he would be rich and not making youtube videos.
@MatkoFakaI take it all you folks are now multi millionaires by now ? That would be the true test of the effectiveness of this method.
@ddoc1964Have you tried deep learning such as LSTM algorithm? It's great for time-series data.
@fantom065boo8First, appreciate for the excellent work! Question: at 16:03 , precision_score = 0.423529, random guess accuracy is 50%. So, why use ML prediction to guide the trade?
@rongfan5675why not try using an input proba of 80% in stead of 60% ?
how would it change the result ?
so why aren't you billionaire?
@istaruscanada6572I ALREADY KNOW THIS VIDEO IS BULLSHIT , B/C THIS DOES *NOT* WORK. IF IT DID , A LOT OF PEOPLE WOULD BE RETIRED MILLIONAIRES. SAVED 35 MINS. OF MY LIFE, I HOPE I SAVED SOME TIME FOR SOMEONE WHO WAS SMART ENOUGH TO READ THIS.
@freddurst4420How much accuracy
@hulksmash456Bro......HOW MUCH (crores) you earned so far in this technique ???????
@MrTally000Thx and god bless , regards from Hong Kong 😃
@wuyanchuWhere can I code this on and how do I install the modules, the stock tables dont appear when i code this on vs studio
@evad1564Great video, would you consider doing a follow up on some of the stuff you mentioned that would further enhance it?
@rosscortbTo see similar results as the tutorial, you would want to add the following lines to limit the period max
-->
sp500 = sp500.history(period="max")
end_date = "2022-05-19"
sp500 = sp500[sp500.index <= end_date]
haha good joke
@DEEPAKCHAUDHARI-d9gBefore you start investing, it's crucial to understand the basics of investing, different asset classes (stocks, bonds, real estate, etc.), and the associated risks. are you investing for retirement, buying a home, or building an emergency fund? Your goals will help shape your investment strategy.
@micheal_millsVery nice video and a great explanation . You didn’t mention finally how to get stock price predictions for tomorrow
@sundsrik2154Is he doing classification? (I wonder because most people do Regression) Thank you for your reply.
@rathanonsriwong2462Is is not clear to me why you decided to re-train your model as you are backtesting. Wouldn't it be more correct (and more fair) to re-use the same trained model as you back test? Otherwise you won't even know what model to use at the end. Also I would calculate the precision score across the training data to have insights for under/over fitting.
@soliveirajrFirst of all, this whole economic chaos was powered by optimism that the FED is done with hiking interest rates. Now that interest rate crash is the situation, where do we go from here? How would you advise I safely allocate $250k funds at this point?
@HannahbenowitzI followed the steps religiously but my initial precision showed 0.65 and then at the end it showed 0.53 please help out
@ishitahumnabadkar7855