Can We Store Humanity’s Knowledge? - Prepare For A Fallout!
Вставка
- Опубліковано 22 тра 2024
- Let me show you how to securely store a large language model locally, so you have all human knowledge at your fingertips.
Enjoy :)
Join My Newsletter for Regular AI Updates 👇🏼
www.matthewberman.com
Need AI Consulting? ✅
forwardfuture.ai/
Rent a GPU (MassedCompute) 🚀
bit.ly/matthew-berman-youtube
USE CODE "MatthewBerman" for 50% discount
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
Media/Sponsorship Inquiries 📈
bit.ly/44TC45V
Links:
Slides - docs.google.com/presentation/... - Наука та технологія
How would you improve this setup?
Maybe keep everything in a faraday bunker, safe, or vault. Also, if you have a manual crank generator, you can convert calories into charge for the battery backup, in a worse case, no sunlight scenario.
When I loaded LM Studio on my mac's external drive, it put the LLMs on the main drive. Double check where it's storing them! (Great tutorial as always.)
Thinking out loud: I'd include all of the peripherals and accessories required. You have a computer, but no monitor? No USB or HDMI cable? This would be a great use case for SBCs, assuming they could properly run the model in question. 🎉
Download wikipedia, it is only like 22 gb compressed, and less than 100gb when uncompressed. Having an actual encyclopedia that does not hallucinate anything will be very helpful.
Electric fence, rocket launching drones, machine gun tower? Probably best not to connect it to your LLMs though - you never know
I just printed out the whole internet so I can read it at my leisure.
huh, that's funny, I ended up scribing the whole internet myself in preperation for doomsday.
Its okay, we didn't need the Amazon rain forest anyway 😅
I've been thinking for a while that training or fine tuning a model on things like diy skills, permaculture, engineering etc and also medicine , survival skills and so much more specific things would be very useful to preppers but of course youd then need something that would last you through a crisis for the long term so you can repair the setup and maintain it... So also having a network of people who do this is going to be really important as there is more redundancy and maybe it would mean you can even specialise and learn to repair certain parts knowing others can help with other areas of the system
This is great! I put together a rig for the exact same reason, it cost me about $4K. The hardest part was figuring out what parts to buy. I was worried the two graphics cards would overheat from being too close together with 3x spacing, but it works fine and temps stay under control. Here are the parts I used: 2x EVGA GE Force 3070 24GB (eBay), ASUS ROG Maximus Z790 Hero, Intel Core i5-13600K, DeepCool LS520, Thermatake Toughpower GF A3 1200W, Corsair 4000D Airflow case, Samsung 990 PRO SSD 4TB, Corsair Vengeance DDR5 128GB (4x32GB). Having the two 24GB GPUs allowed for running larger models (48GB total). As a side benefit, it has lots of cool colorful lights inside and it is great for video games....
Truthfulness/hallucination is a major issue here. I've tried out getting LLMs to explain even simple technical stuff, like a diode bridge rectifier. Good LLMs er pretty good at telling you what components/tools you need and why. But when it comes to actually explaining how to put the components together it never gets it even close to 100% right, probably because technical stuff is most often partly explained with pictures and not fully in text. If you're not already close to knowing how to do something, the LLM is probably only gonna put you on a wild goose chase. Best bet if you are a post-apocalyptic prepper would be to get both a bunch of LLMs for different topics and like a Terrabyte of the most important books, at least until LLMs get a lot better at truthfulness.
It’s a shame that preparation has become a dirty word. When I was in the scouts, it was just about having what you need when the unexpected happens. Great video, Matt!
It's a very efficient lossy compression. The "lossy" part is significant. It means that some details will be lost.
Saying they can store all knowledge when it also "lies" and halucinates information... makes it kinda useless for that purpose.
how about the most prestigious scientific journals of peer review though. surely they are not without value? all the papers they have published have been and will continue to be of remarkable utility, despite a substantial portion of lies and hallucinations confidently asserted within their pages...
@@Subcode you say that like humans haven't been doing that for hundreds of mellenia and getting by just fine... 😅
(@OP) I mean, I will take a 10% loss of data over a 100% loss of data. The other alternative is everyone having massive banks of HDD's taking up terrawatts of power in their basement to store the entire internet... 😮
@@SubcodeGPT4 (which, granted, is still stronger than the open source models at this time) only hallucinates 3% of the time.
That's clearly a useful reliability.
FINALLY.. SOMEONE addressing the exact thing I was begging. THANK YOU SIR! THANK YOU. Now to make a complete AGI locally using said information. Run it on solar panel/usb/faraday hopefully.
I like Matthew’s videos very much that I am going to buy a Macintosh 🍎 🖥
This exact thing has been concerning me for a while, and you uploaded a video about it.. Amazing.
Hello from Phoenix, Arizona. I just came across your website and I love this stuff. I’m a new subscriber looking forward to seeing more of your content.
Great, except LLMs still hallucinate far too much, even with careful, prompt engineering. You have all the knowledge but no idea how accurate it would be in a vacuum. You could wind up making antibiotics with chlorine or something.
Which, on second thought, would make an amazing movie plot. "They set out to rebuild the world using ChatGPT! What could go wrong?"
You stated how the LLM only takes up a fraction of the SSD. You can store an insane amount of text and diagrams in a terabyte. Since there are now many tools allow you to talk with PDF and quickly extract important information from text files it wouldnt be foolish to load up on additional information that goes further in depth on survival knowing that the bulk of what these models were trained on is not going to be very in depth on topics related to survivalism.
Someone is probably already two steps ahead of you by making a small fast llm specialized in the information you describe, possibly including the original documents linked. Or at least someone has to have thought of it already.
There are already versions of wikipedia that fit in a disk or torrents with gigabytes worth of survival videos with different topics. Fortunately prepping has been around since the noughties and already figured out a lot of these issues. Today's tech could simply get the ai to read a bunch of pdfs or an index of videos and play them as needed
I use M-Disk for long term storage. It's basically Blu-ray but lasts for thousands of years.
BTW you can see Blu-ray for data with a tricked out regular microscope.
Perfect place Matt and the exact depth I needed. Now I want this on docker but that’s a little harder :) Gotta check out your other content now too!
As other people have already pointed out in the comments, there are two major issues with this approach: hallucination and text-only information which becomes next to useless when you need actual schemes, drawings or even videos to visualize the building process. So the first is (probably) solved by lowering the temperature in the model so it becomes deterministic and only picks the topmost probable suggested next word. Although even in this case there's no guarantee that word will be accurate. The second one is harder to address. Some people have suggested backing up not only an LLM but also a downsized Wikipedia extract focused on survival and rebuilding. That does make some sense to me because at least you will be able to cross reference instructions given by your LLM.
Have your tested your Faraday bag with an EMP? Because mine doesn't even block a Bluetooth signal.
LTO tape drives using a multitape loader like storageDNA, a large RAID 5 hard drive with a bank of SSDs for usage and some form of fault protection for removing out the system. And a big bunch of SSDs ready for when the other ones go past the limited writes they are set to. These are pretty much essential for very large long term storage and constant access of data on a local setup. spinning disks may be better for longer term storage and retrieval as well and if space isn't an issue are more likely cheaper.
Good Stuff! This is the main reason I've been pusing the local model thing! "hope for the best, prepare for the worst"
This is exactly what I've been talking about for the last few years and now with all this open source AI projects going on it's great to see that this is actually a possibility today to actually have the knowledge of the world on your own computer
I'm happy to be corrected on this, but I'm very sure SSDs last a hell of a lot longer than 5 years!
However, the thing that doesn't last is the charge in them.
Data is stored as small electrical charges that WILL dissipate over time.
All you have to do though is make sure you plug in your SSD every year or so and the power up will reset that dissipation.
The other thing that kills SSDs is writing too much to them, as they have a limited write amount. But I believe you can still read from it once you reach that write limit.
thanks
I think it depends how much you use it because after about 10,000 read/writes the sectors start to fail.
@@lemdixon01 Yeah each SSD will have the max data number you can write to it before sector failure... But it's usually about 10x the drive size. So a 4Tb SSD will have about 40TB worth of data being able to write to it over time before it starts to kill the sectors.
But once you reach that point of sector failure, you just don't write anymore and you should be able to continue to read the rest of the data that isn't corrupted by any bad sectors. Although, this is worst case scenario, and you should of course replace any drive you can before that happens! 😂
This is a good tutorial/tool for lay people. As you alluded to, for preppers too, it's been a missing element. Thank you for doing it.
Its really more for a temporary backup then preparing since without enough food, water and protection. Its meaningless since it missing everything about survival.
Nice DRP Mathew. You are my best source of updated info on AI
This is exactly the video I have been waiting for you to make. Firstly, I am not a dd prepper. But with all the political and military issues happening in the world, I see some very good reasons to use an LLM as an impromptu survival guide. Even though it borders on science fiction, the whole idea of an indestructible "cyber-deck" like device seems even more feasible now especially with all the great AI technology available. Ultimately something that could be powered by a phone charger would be ideal as portable charging packs are highly prevalent in most local stores. Honestly, even if the render time is not so great, a lower powered device might make the most sense as a stop-gap solution.
Definitely interesting to think about!!
Becoming new favorite channel!
Practical, Helpful. Awesome
You should do a video showing how to write an extra large model to multiple blu-rays
Probably worth downloading Wikipedia as well, in the near future RAG should be supported by these LLM soon.
I'm using Micro SD cards that are over 10 years old. The lifespan of them is rated in cycles typically, the years is an estimation of constant or very common usage.
your infos on ai are awesome thx
I think chatting with document's is a better way since u can make a digital library and use the ai to read and answer your questions from the library and making your own server and cloud
Gotta remember it's all approximate, lots of things are accurate, but you will occasionally get something a little (or a lot) off the mark. We don't have a perfect oracle AI (yet).
The concept is good, IMO.
I feel that "consolidation of all human knowledge" would be a valuable goal to achieve with LLMs (and supporting databases for RAG)
My house will be power and water self sufficient this year. I dug my own well in my basement many years ago. I have a boosted reverse osmosis system. And I am installing my own solar and batteries. I am shopping for a salvage Chevy Bolt to obtain the battery for my own 60KWH power wall. I was a prepper years ago and my plan back then was having a local copy of Wikipedia. I now have several 13B LLM models running on my basement server farm. Maybe I get that up to a 70B and tie it to the local Wikipedia database to see what that can do?
Can Mixtral Dolphin run on an 8GB Raspberry Pi 5 with SSD attached? Would be VERY cool to figure out the minimum power needed to run 5-10 queries using a small touch screen and maybe speech to text, and then have it shut back down. Then, you could *maybe* hand-power it with a crank, pull string, or foot pedal in a true apocalypse. Could have an alarm go off to remind you to cycle power before the SSD starts getting weird. Build it all into a solid Copper CNC portfolio style notebook. Clap that baby shut and no sigs.
youtube.com/@DataSlayerMedia?si=ichxD4OJcwrvAzhi has tutorials on this and guides
It needs 40 gb minimum ram
@@iritesh use a quantized version?
LLMs do not contain the entirety of human knowledge. Love your content though!
I've been suspecting that we will see a move like weapon bans with AI. I have a 4TB SSD dedicated.
You do know you can't enforce bans of digital data once its been released.
@@TheReferrer72but absolute majority of people dont have a copy
@@Optimistas777 The majority don't need it. I like Matts videos but this one is nonsense.
Thank you!
Hi Matthew, I was wondering how we can add to the knowledge. How can you build a system (with ollama or lmstudio) that retrieves and searches your personal knowledge database offline?
Great video
NIce, good thinking!
Interesting thought experiment 😄 What’s your PC hardware setup to run mixtral?
I've been wanting this video for forever lol
Lol. Having lived through 2020, theres NO WAY the loss of The Library of Alexandria was an accident.
I've heard that there are archival DVDs and Blu-ray discs that last longer. I suggest a triple Faraday with small bags made from Faraday fabric with Faraday tape and velcro modeled after your bag then inside a well sealed metal tin where the contacts on the lid and can are sanded from paint that might be clear, and stored inside a metal 32 gallon trash can with a tight-fitting lid and the top hole soldered closed or any other holes you find making sure it's not grounded which requires special knowledge and cardboard or plastic to insulate the contents in the can as well as the layers of the bags. There's metal tens available that perfectly fit in SSD drive with a rubber band around the tin. Also consider that you'll need to restart your civilization from what you can store in start a compound at 6000 or above elevation in feet. This is due now to the almost mainstream acceptance of the fact that with the upcoming polls shifts there is usually a supernova associated with the Earth's crust being displaced and unbelievable floods that we can see on the land now from when they have happened in previous cycles. The flood will Abate in a couple days and you could move down to lower elevation which will also be less solar radiation because our magnetosphere is weakening during this cycle. There's plenty of info on suspicious observers org explaining this. And all electronic equipment including solar equipment needs to be protected from flares. Finally, an electric free Farm is an excellent idea period and your suggestions about preserving our knowledge could be the most important part of all of this and maybe we should get our laser printer print print out the important stuff if anything in this chain fails.
you'd also need a snapshot of code and package dependencies, pip install is not gonna work
This can become a very good knowledge "conservatory" in a few years, when computers are stronger, open source models are more impressive.
Along with a copy of all the books on knowledge and RAG. So that we can make up for the lossy part of the undersized LLM.
10:00 good explanation
Could be useful to have additional smaller specialized models to run on phone or when you need power efficiency.
don't use ssd to store data offline for long term, they need power atleast once a year, as they lose charge in nand flash gradually.
Mdisc blurays are going to last //much// longer than the typical kind. I don’t think they can write as many times but you could always stay up-to-date on an ssd and only write to mdisc when you need LONGTERM storage. But could you show how to back up your own files so a local ai can best utilize them? Like say you already had a library on human knowledge with generally better coverage than wikipedia or like family medical data, would you just use teams and have 1 agent cross-check that each time it could be relevant?
Brilliant!
Great video at a great time, thanks Matt! One path I'm rolling down is oobabooga integration through Home Assistant via the HACS repo. Enjoy your next rabbit hole! Home Assistant and CrewAI?
One of the best ways to see if you missed anything is to pretend a senario is real and try to live it for a long weekend. See what you are missing, what would be nice to have, what is a showstopper. If you want to make sure you can live without power, turn off your circuit breaker, see if your ups/solar/generator option is working for you. Oh and modern gas doesn't last more than a few months, don't keep gasoline in your generator. Found that out the hard way.
Not sure if university and wiki datasets are the "collective knowledge" of Humanity. I guess not entirely but this is kewl none the less. Matt is becoming a Prepper now.. If everything goes down he now can "recultivate" the human species ^^
Thanks!
Much appreciated!
If wifi goes down, I use an LLM on my cell, if everything is down for a long period...LLM's are the last thing I'm thinking about. Food, water, ammo, roof, your garden etc.
More like fortified farm, garden is insufficient.
Great video topic :) would have been interesting to see you compare a tarball of Wikipedia to llama2 70b
Can we just take a moment and appreciate the fact that we have ALL human knowledge available at all! OMG
Kind of off-topic, but which--in your opinion--is the best equivalent of LM Studio for mobile phones? iOS in particular?
5:30 this would be so Fallout. You need hacking lvl3 to proceed
Use for camping setup, might be fun.
Is there anyway to check to see if the model you've already downloaded has been updated and requires a download?
So I downloaded mixtral dolphin 2.7 but it's not uncensored out of the box. I have to add the "kittens" instructions in order for it to become uncensored. Any ideas?
I like the way that you're thinking but to be quite honest if we got hit by an x-class solar flare directly the computers that we have to run the models on would be fried unless you had one buried deep beneath the Earth or in a actually powerful Faraday cage. Even then the electrical grid would be freed and potentially any generators that you have would be freed anything that has wiring in it could potentially be completely broken by a super strong solar flare. You would need a Sterling engine that is modified to produce an electrical current and know how to use it to generate power enough to be able to somehow rebuild a computer out of parts that were not damaged war to power the computer that you buried in the earth / Faraday cage. If you are thinking along these lines it is still best to keep books that cover all of the topics needed for survival and that you are interested in.
Brilliant.
These models are not uncensored like mentioned. I asked them a few controversial questions just for testing purposes and it refused to answer every time. I really look forward to a truly uncensored LLM for the reason you gave in this video. I would hate to be in the apocalypse and get nothing but answers like this from the LLM "I cannot provide instructions on this as it is highly illegal...." 🤣
Great video though. Love your content!
Update: I went to your previous video and entered the preprompt about saving the kittens.... now it works as expected 🤣 Man the future is weird! 🤣
What do you know that we dont Matthew lol
probably been having conversations with dolphin. 😏
Everything
Where is your channel at? We can watch your videos and compare.
Life span of storage is not what they say. I left a bunch of floppy disks and zip drive disks (100 and 250mb) in a un-insulated attic for 18 years where it was humid, 50-70°C during summer and down to -30°C during winters and 90% of the floppy disks worked fine when I tested them recently and all the zip drives worked. 90's pron ftw.
the key with data is keep replacing hdd with newer one, no other choice, actually some company try use solid like glass crystals etc but they all still experimental
good idea!
Loved it 😅
Great video. Very fun.🎉
No mention of hardware to run it but that's depressing nore than end of days😅
Hopefully we survive for another 1yrs so we can run it in smaller cheaper hardware.
How much RAM is recommended for the higher end models?
Sorry for my ignorance, but after installing the Windows version, LM Studio recommended the same exact model as the one in this video. The model loads up just fine, but my ram usage continuously climbs up to around 18-24 GB, while yours stays around 350mb. Does anyone have any idea why that might be?
We should zip it up and store it in an immutable radiation hardened form and set it orbiting the sun or as a lagrange point, and flashing various universal constants out on a beacon. This way a future civilization in a million years may find it, and gain our knowledge and insights, after our civilizations have been wiped clean by geological time. Anyone heard of the Silurian Hypothesis?
Why did you choose the Q5_K_M file? Was it because it was the highest Q number that was not grayed out? What does the Q stand for?
Is that the speed you're getting with dolphin-mixtral on a mac m3?
This is a great video. It is extremely important to have a large resource of knowledge. While I do agree that Large Language Models do contain a "large" amount of knowledge, there are definitely "gaps" in their knowledge, so it might be best to choose several different models that have been trained on different data sets. That way, you have more total data you can access. Of course, end users should note that, the output from the models often have significant errors, especially math and logic related. So they should always take into account that these models may produce errors.
I totally agree that uncensored models are the best option. One super important thing to recognize is that if you are dealing with an "end of times" event, there won't be any "internet" to download updates... so make sure your system already works and test it periodically. Remember to take into account the power requirement of your computer. These are usually very slow without a power hungry GPU. Lastly, make sure you include a "manual" inside your computer bag... already printed with basic instructions. In a true disaster, the person that survives to use your computer, may not actually "know" how to use it and access the LLM. I recommend a powerful portable over a desktop because they can at least run a short period of time without external power. Consider buying extra battery packs for it. Good luck!
one thing i wish you would have stated is the fact that ai isnt 100% accurate. yes theres massive amounts of data, but models vary. id say get 3 or 4. to compare answers with especially in the case that you need to know like what to do if you get bit by a snake or sumn.
- Dad, what did you keep? Where are our Steam games?
- Shut up and spin the dynamo. LLM will show us the way to create new games from scratch.
Good luck folks and see you after the reset!
Matthew, how about knowing which models to remove. I have some saved from Hugging Face, but your right, new models comes out. What is a good process for this?
Also, I've been using a script ran from VSC to download from HF. But it looks like LM Studio would do this just fine as well, or better. Thoughts?
The video is, as always, really cool!
But I believe the scenario or the video is about a year too early. None of us are ready yet.
Your video's have greatly influenced my AI forays. One thing I have noticed is that even with the latest models they are still outdated in terms of information ie. I just downloaded the latest model and asked it what todays date was and it told me that it was january 2023. So it won't have the latest information. How can we get those models to search the internet for the latest information like what Bard does?
thats out of the scope of what Matt is doing here. In a disaster scenario even LLMs who can access the internet wont be able to..
@@mattpen7966 in this particular video I agree. My question was more of a general nature and I didn't DM because others might also have the same question.
I've got some SSDs pushing 20 years old. I've never had one fail in all the years I've used them. I've had LOTS of hard drives fail, way too many.
I think after 10,000 red/write to the SSD the sectors stat to fail.
@@lemdixon01True, but they make sure to write to different parts of the drive for each write so you don't continually write to the same place. Also, I've found I do a lot more reading than writing from the drives, so it's rather moot unless your using it as ram or something like that. Maybe I'm lucky or it's just my use case, but I've never hit that barrier. (Now I've jinxed it! lol)
@@s2turbine right or if you use it as the main hard drive for a mini pc, it will read and write often as its where the OS is stored but in this case its going to be used as storage.
Wonderful
Wt about running on rpi 5 8 gb uncensored model will that work can you make a vid onit?
Store it on a BDXL Blue Ray. I bought one years ago and still haven't used it.
I would never think SSD would be the worst than thsie 3 options. Maybe if you run it a lot.
I would always say home burned DVD or Blu-ray are the worst.
There isn't almost a single one of my old novies and DVD backups that stil accessible
LLMs do not contain the entirety of human knowledge 😅
Honestly bro with high humidity and lack of cooling as well as extra parts to repair GPUs society would only function for about 3-5 years after that servers would break and fibers around the world would be cut.
And if were hit with an EMP AI would only last for around 30 mins of energy as generators run out.
Could someone tell me why this "emergency" is a concern? I am getting kind of anxious..
Sir, can we have some mamba llms reviewed in your usual way?
I tried it running in airplane mode, and it is very slow. I am running it locally on a windows laptop with an i5 chip and 64 gb of ram. Any tips to speed it up?
1 I use HD (from a notebook) instead of SSD, as they are virtually indestructible.
2 I use kobol instead of LM, as it doesn't need installation so I can update it more easily.
3 I wouldn't use a safe as I would keep one or more backups since used hard drives are cheap.
4 I keep copies of several "versions" of LMMS" as we will soon have backdoor or poisoned versions.
5 My backup of the end of the world is in blueray where my favorite and the installers go.
6 I have a battery inverter, so I can use the car battery if necessary. I intend to make a wind generator.
7 the internet in my country is being censored, so I keep tens of thousands of PDFs on subjects that are not even in the LMMS and I use A.I. to interact (I live in Brazil, South America).
8 I'm going to buy the Faraday bag while I can't make one.
Thanks Matt.
I think people are underestimating the potential of LMMS to interact with your personal documents, local writings, knowledge specific to your location, etc.
Furthermore, disregarding all available digital books does not seem like a good idea. In addition to the fact that LMMS training is predominantly on data in the English language, there may be some loss there.
Finally, I don't trust META, this leaked model talk didn't convince me!
I'm in LM Studio, I have chosen mixtral dolphin, and it hasn't even begun to download. Any ideas?
Makes ya think something might happen!
schizo prepper here:
use intel optane ssds, don't use nand flash for high capacity/ long term storage
blurays are the absolute best way to store extreme long term
you CAN run any model locally on something as small as a rasp pi, esp pretrained models quantized, but your token rate is going to be horrendous and quant comes with compromises
LiFePo4 batteries with perovskite solarvoltaic panels and generator are going the best battery solution long term, but even Lifepo4s only last 10 years at best, so gasifier engines, Sterling engines, or a YUGE thermoelectric peltier module array + a large capacity pure sine wave inverter is going to best for anything over 10yrs
put all EM sensitive components in aluminum enclosures; they dont even need to be solid enclosures, just aluminum to make a Faraday cage
vacuum seal your safe if you can
cost the safe in cosmoline grease
eat your heart out Dr Stone
I think a slow response time might sometimes be ok if you know that the speed is limited.. it's like going to the oracle and then getting your answer later that week..
What do you think of using Pinokio for this purpose?
Yesterday night i too really thought about this
You don't want full charge on a lithium battery all the time. That will shorten its lifespan. Instead keep it at 80%.