I Ran ChatGPT on a Raspberry Pi Locally!
Video Overview & Insights
Full Tutorial Instructions Here: https://buildwithparallel.com/products/run-advanced-llms-on-your-raspberry-pi
Full Tutorial Instructions Here: https://buildwithparallel.com/products/run-advanced-llms-on-your-raspberry-pi
Product Links (some are affiliate links)
- Raspberry Pi 5 👉 https://amzn.to/48Qgy4O
Lier. ChatGPT has dedicated servers and specialized equipment that make your “chatgpt on a pi” look like a paper weight.
🧠 Need expert help fast? Book a 1:1 session and get unstuck today 👉 https://bit.ly/42I10y5
🎥 NEW: Unlock members-only videos and behind-the-scenes drops 👉 https://bit.ly/4iyBm4I
🛠️ The exact tools and gear I trust (and actually use) 👉 https://amzn.to/44fKDv4
📚 Step-by-step setup guides, templates, and insider resources 👉 https://bit.ly/4ivZDID
Bro didn't run chatgpt locally, bro run an LLM that's not even fine-tuned yet.
🛒 Grab custom gear and tools designed by me 👉 https://etsy.me/4isKwjb
📩 For sponsorships or business inquiries, reach out: macgyvertechnology@gmail.com
You kinda didn't - lets not bate the clicks.
GitHub Repo: git clone https://github.com/antimatter15/alpaca.cpp.git
Model Weights: https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml/tree/main?clone=true
This is the most braindead comment section ive seen in a while..
As part of Meta’s commitment to open science, today we are publicly releasing LLaMA (Large Language Model Meta AI), a state-of-the-art foundational large language model designed to help researchers advance their work in this subfield of AI. Smaller, more performant models such as LLaMA enable others in the research community who don’t have access to large amounts of infrastructure to study these models, further democratizing access in this important, fast-changing field.
making LLaMA available at several sizes (7B, 13B, 33B, and 65B parameters) and also sharing a LLaMA model card that details how we built the model in keeping with our approach to Responsible AI practices.
So you’re telling me I could make a spiritual AI
Over the last year, large language models — natural language processing (NLP) systems with billions of parameters — have shown new capabilities to generate creative text, solve mathematical theorems, predict protein structures, answer reading comprehension questions, and more. They are one of the clearest cases of the substantial potential benefits AI can offer at scale to billions of people.
Smaller models trained on more tokens — which are pieces of words — are easier to retrain and fine-tune for specific potential product use cases. We trained LLaMA 65B and LLaMA 33B on 1.4 trillion tokens. Our smallest model, LLaMA 7B, is trained on one trillion tokens.
Can we really ask everything to that AI in such a tiny computer ? I still don’t understand how it works. Like where is all the knowledge stored ?sorry for the mistakes in English,it’s not my first language so please forgive me.
Like other large language models, LLaMA works by taking a sequence of words as an input and predicts a next word to recursively generate text. To train our model, we chose text from the 20 languages with the most speakers, focusing on those with Latin and Cyrillic alphabets.
There is still more research that needs to be done to address the risks of bias, toxic comments, and hallucinations in large language models. Like other models, LLaMA shares these challenges. As a foundation model, LLaMA is designed to be versatile and can be applied to many different use cases, versus a fine-tuned model that is designed for a specific task. By sharing the code for LLaMA, other researchers can more easily test new approaches to limiting or eliminating these problems in large language models. We also provide in the paper a set of evaluations on benchmarks evaluating model biases and toxicity to show the model’s limitations and to support further research in this crucial area.
Sorry, cannot follow you, too complicated.
To maintain integrity and prevent misuse, we are releasing our model under a noncommercial license focused on research use cases. Access to the model will be granted on a case-by-case basis to academic researchers; those affiliated with organizations in government, civil society, and academia; and industry research laboratories around the world. People interested in applying for access can find the link to the application in our research paper.
We believe that the entire AI community — academic researchers, civil society, policymakers, and industry — must work together to develop clear guidelines around responsible AI in general and responsible large language models in particular. We look forward to seeing what the community can learn — and eventually build — using LLaMA.
arr is array, but pirate pun would be much better
More User Perspectives
LOL so slow. waste of time
@BlizzardnzMASSIVE LLM lol what a joke. they are tiny llm's
@Blizzardnz😬EHHHHH the I'm Elon joke really not aging well
@marahansen29812:44 dude 13 gigs not that heavy no need to reduce it
@kaledazzahraniYou are an impressive person. It’s unbelievable to see someone as smart as you with the free time to make good YouTube videos.
@CaliTekShameful clickbait. Imagine being either so unscrupulous that you do this on purpose; or you're so incompetent that you can't see the difference between ChatGPT and a Llama.
@Doolboi have a pi 2 (planning to get a pi 5) and i want to run the ai model on the pi while having access to it on a windows machine (control via edge sidebar), is this possible?
@taranagnew436Lowkey clickbait though, you clearly did not run Chatgpt
@pravinvedurla6787I'm black
@TimStarkClickbait mofo
@FruchtKnechtCongrats. Thanks to lying in your video titled, your channel has been marked “Do not recommend channel” in my account. It’s too bad that a monetized video that lies cannot be hit with false advertising claims. If you can’t be truthful about a video titled, well, you’re just not someone who can be trusted with anything else.
@robertallenpayneI don't see the point of a low power AI that's offline whats the point if it Don't know how to answer a question or be able to find the info to help with coding since that's all I ever use AI for
@phillangstrom8693Talking about privacy and using warp terminal… ok
@vorant941:20 AI wrote sentences saying that AI is the future. Not today, Skynet!
@seanwieland9763Clickbait bullshit video. Shame on you
@seansingh4421This is like having the title of your video being "How to install Windows 11 on Raspberry Pi" and then proceeding to explain how to install Raspberry Pi OS
@rheymanda1074Isn't this just olama
@dutArkhaminstall linux in a laptop and run model as a server without internet. Nice video. Thank you.
@braveonderHow come when I run the chat file I get a permission denied. I tried “sudo chmod +x chat” and then ran it but still have the same problem
@wagnergriffin670I don't get claiming to run chatgpt on a pi. You are not, You are running Llama on a pi. Then when you compare a small model like 7b llama's responses to another model you compare it to the production version (gigantic) active web based version of gpt 4. its like saying look I have a Lambo except I don't its a Civic look at the civic... ahh see the civic has this 0 to 60 time but the top of the line Lambo has this 0 to 60 time, Then you just keep comparing the civic to the top of the line Lambo even though they are not the same or close to each other. The premise of the video was cool. Look I am running a llm on a raspberry pi. Then you went on with... I don't know what. I wish people would give llama the credit it deserves, when you compare it to the big version of gpt instead of the live llm version of llama you aren't even comparing the two from the same company. You used gpts name to get people into the video then didn't even really give llama the credit it deserves to letting us have access to this awesome technology that GPT makes everyone pay for at the top tier.
@ShamanETMIm not a hater, actually this is pretty awesome, but you are not running ChatGPT... its a very very very quantized model of a LLM but, its very awesome to run it on a rb pi
@IvanchweloWhat happens when you add a hailo chip will it boost the performance?
@vish2k-o7smain: seed = 17216393XX
llama_model_load: loading model from 'ggml-alpaca-7b-q4.bin' - please wait ...
llama_model_load: ggml ctx size = 6065.34 MB
zsh: segmentation fault ./chat
Why would that scare me? It's a chatbot.
@augustuslxiiiIt's work without internet?
@نفيس_الشيخ😎🤖
@thesimplicitylifestyleI am using big LLMs on a bunch of Tesla p40s but since the cooling options are pretty loud and it consumes a lot of energy I wonder if I get better inference with a coralAI TPU on a raspberry pi than using llms on the pi without anything else. Also, would it make sense to build a pi cluster, each fitted with a coral ai tpu via pcie port?
@OVERLOARD94949450gb vram? I have 60.
@OVERLOARD949494The thing is this guy is running this on raspberry pi 4 and the new raspberry pi 5 is 2.5x times faster just think how fast the ai would be
@thanishursThe single most significant innovation in history is certainly NOT the internet. AI very much aligned with contemporary human overconfidence...
@clipbastlerElon? Elon is a criminal.
@_specialneedsnow my question is can i run it on Asus Tinker board RK3288
@OriginalAceXDGood video, but the title is misleading.
A Llama-1-7b with only just 4 bit is really far from ChatGPT. ChatGPT has 175b parameters compared to 7b parameters in 4 bit. I would say ChatGPT passible outperforms this local Llm by 200-500% in every task.
Could this be used to be trained to search a local pdf library? I have seen people make cyberdecks with Wikipedia and other preparedness related PDF documents. It would be incredible to not have to read a whole document, but rather put a question into a chat box and it search for specific information from said PDF libraries!
@DontTreadOnMyLibertyI couldn’t help it but lol at calling Ubuntu “tried and true” linux distribution. While it has its merits, Ubuntu is a Debian downstream.
@foursixnineClickbait, Chat GPT does not run locally and a Raspberry Pi is not even close to being capable of supporting it if it could.
@corey_derocheHow can I train a chat ai on a specific very large body of text from a person from the past to bring them back to life? What would the possibilities of that chat be like?
@roryleitner1532Hmmm we need a way to cluster this in a parellel processing raspberry pi 4 and 5 cluster.
@armisischatgpt is not open source, you clickbaited the entire industry
@HUEHUEUHEPonyhi i have a error on my PI
main: seed = 1712111644
llama_model_load: loading model from 'ggml-alpaca-7b-q4.bin' - please wait ...
llama_model_load: ggml ctx size = 6065.34 MB
Segmentation fault
can you help
thanks