Відео

@@abhijeethvijayakumar6513 Hey, one video idea failed a bit, since I grew unhappy with the research papers I was trying to cover. That, plus a new startup I am co-founding, plus the fact I am having a tropical workation for a few months and only have a laptop with me. I should be back on track in early 2025!

I probably did not make it very clear myself, I wanted to clarify. Regarding the question how AI will actually get conscious (or reach AGI, at this point this might be synonymous to some people). I think it's not gonna be LLMs, at least not current gen. Whatever you use them now for, current gen LLMs certainly feel like a tool. As you say, you might lower the temperature to zero - and get the same result very reliably. Then it's certainly just a tool. But the debate is important for whatever the next (or next after next) gen will bring to us. We're just scoping the terrain of the Future, without properly seeing all the details in it. And this debate is going on while new discoveries are made seemingly every month. Which makes it all the more fascinating!

@@FutureIsAmazing569 I Agree BTW. Personally I think we will achieve "consciousness" by whatever definition in a way that's really, really debatable, similar to the debate right now, but, I think that non LLM form will leave the two sides in a more philosophical position. Similar to the 'doesn't have a soul so doesn't count' position some people have, and which you correctly picked up on by targeting the Penrose position. I actually think Penrose BTW is more right than wrong and that at some point we will find something 'intrinsically different' about humans and I have no basis by which I can explain why I believe this in a way that's rationally acceptable to anyone else. However I think that research on human consciousness around the death event' is more likely to help bridge the connection. So, while I fully agree with the 'Meat\silicon' computer argument I also think that science just hasn't found the missing pieces yet. Like with cosmological physics where we invented a model that 90%+ non existent to balance a model that's probably not right I think we'll keep discovering things in the Penrose space.

@@glamdrag well it’s kinda natural and intuitive for anyone ascribing to scientific materialism. I think I first heard it from Sam Harris

Oh surely, Reddit's all fun, until you're the witch

I loved running different simulations, since there's way more of them than I was able to show. Not sure about the animation part :), It's so labor intensive!

@@FutureIsAmazing569 I see! Thank you for your insight!

This is very much I had in my first simulations. Plots were indeed smoother and delayed, but the underlying principle was exactly the same. Bubbles would still form when the threshold was reached, just slower. I opted to go with more extreme example, since these simulations made a video a bit too long. I was affraid I will loose attention of my audience.

@@FutureIsAmazing569 makes sense to not include in this video. Good content!

Oh, that's exactly what I have for one of my next videos. I use ASP to solve word problems. (but it's not very exciting at the moment, not sure it will see the light of day). And yep, word problems ( or something like Sudoku ), is doable by Prolog, but quickly explodes

Hipster strat is absolutely amazing, when you encounter crowd psychosis, classic example is shown in the movie "The Big Short" which portrayed the 2008 crisis. Some traders went hipster enough to bet against the housing market and were considered crazy!

@@FutureIsAmazing569 oh ya..nice example. When u go hipster strat need to know whether fomos are in it to play as well.only then hipster can profit from the strat .

OMG this comment!, Thank you so much, my last video kinda flopped, but after reading this, I feel motivated again!!

@@FutureIsAmazing569 keep doing what you are doing as the habituals did to fomos and hipsters.. 😅😄😄🤣

Well, HAL-9000 could both play chess and explain what's happening on the board at the same time. Not sure at what level was it playing though.. But ability to reason was certainly way better than the current generation of LLMs

@@djan0889 in our brains, abstract structures often emerge without us going back and forth, but rather by multiple neurons entering a synchronicity and firing at once. That is certainly not happening in current architectures we use. So yeah, I would tend to agree with you

​@@FutureIsAmazing569 Yes, I agree, but I didn’t say it should go 'back and forth', it's just one of those primitive version of brains features. Those structures emerged by evolution . Also I think we don't need to copy human brain. Silicon chips may develop brand new cognitive skills using evolutionary approach by simulating a lot of architectures and functions. Currently trend is making search in vector space. -_- And AGI don't need a lot of data to start unlike llm's. We are on wrong path with false promises :/

Thanks for the link, this video goes more in depth than mine for sure. Regarding Othello - I've quickly went through the paper. It does seem they claim that, but I'd like to verify it. And throw other problems at it too. I'd imagine some problems, even if they look 2D, might be conceptualized easier than others.

@@FutureIsAmazing569 Chess must be more complicated, Othello has a single rule that's hard to visualize (the flipping of other pieces) while chess has a variety of pieces and an unintuitive notation system (it's never explicitly stated where a piece comes from). But I'd argue that (in a purely informal sense) they're the same class of problems. If a system can visualize Othello, then a more powerful version can visualize chess. That however doesn't mean it can calculate deeply. PS: I am taking the paper at face value, it's possible they're being dishonest

@@mnm1273 There's also a moment, that the paper is a bit similar to the chess paper from Deepmind in regards of training a smaller, specialized transformer model, not a LLM. In general I can totally accept the idea that even simple perceptrons are able to from 2D abstractions. That's really not a problem. It's when you ask a general purpose LLM to do that.

The channel is still very young, no worries. Thanks for the kind words tho!

This works wonderfully, when you can hold your finger on the pulse, and have clear objectives. Doing this during automation steps is a bit more challenging. So my intuition is to always try to get away with a zero-shot or (if available) one-shot prompting - just so I can automate this easier down the line. But you're right, especially for code generation, that might not be enough and we have to resort to complicated tactics, like the example you've given.

I would not go so far :) I think they are pretty competent at what they do. But if they would benefit from it, I think I would be fine with it. Whatever it takes to advance this amazing new tech!

@@FutureIsAmazing569 Well you said you hoped to make a buck or two with your approach you can't tell where they get their ideas from (from my experience people from high academia background are smart but usally very uncreative).So if you want to make money keep yur important ideas for yourself until they are market ready (you can use local models to help you). Given that S.Altman is their CEO I would be more cautious if you look how he behaved towards very poor people with his crypto currency back then. (If TLDR then the short version: He" buyed" biometric scans of eyeballs of those people without *informed consent* for some cryptocurrency per eyescan until the government Kenyan government halted it. OpenAI also used very low paid Kenyan workers to create trainingdata not long ago.)

Yes, I haven't been diligent enough of thinking about tasks which can still not be solved with o1. I am sure there's plenty though

I did not try Prolog with smaller models, but I would suspect they should be good at it. Great idea to try later, thanks!

gpt-o1 will certainly perform perfectly for this approach. The only problem I had is that o1 chain of thought reasoning was beating the Alice in the Wonderland+, so there was no point of improving it. But you're absolutely right, the approach is still valid. Just as the next paper comes out, which poses a problem o1 can't solve, I will be back at it!

Thanks a lot for sharing your rant :) Blending AI models with fuzzy logic, plus integrating with logical languages is amazing new area to study. Hey, if we sprinkle some Qubits in there, consciousness is guaranteed!

@@FutureIsAmazing569 Never go full Deepak Chopra, my friend! Quantum computing can only (ever?) speed up specific algorithms by a quadratic factor. Quantum processing units (QPUs?) will be like GPUs or TPUs or hardware noise samplers in computers --that is, task specific accelerators. Thanks for your video!

But I'm not going full Deepak Chopra :), Just a bit of Sir Roger Penrose!

I agree - even simple word puzzles are quite difficult to do in Prolog ( Unless you're using a special library, like bibmm.pl). Something like MiniZinc is way better at it. I chose Prolog for this project, since I found GPT-4o is quite good at writing Prolog code. But yep, you're right, o1-preview makes almost every logic enhancement obsolete

This is explored in quite an old (January 2023) Wei et al.'s paper.:"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" arxiv.org/pdf/2201.11903 "That is, chain-of-thought prompting does not positively impact performance for small models, and only yields performance gains when used with models of ∼100B parameters." o1-mini is exactly 100B, while o1-preview is 300B, so you're absolutely right, scaling laws do apply

I'm not in academia right now, my publishing days are long forgotten, but thanks!

Yep, I thought of doing a video specifically for logic puzzles with Z3. But from what I've already tried, LLMs are way worse in generating Z3 (also tried ASP), compared to Prolog. I think that might be due to sheer amount of training data available on the wild, which LLMs were exposed to. Did not try Alloy, maybe I'll try aggregating various reasoning systems in one video. I also have an idea to pick one and fine-tune LLama3 on it to the max.

@@FutureIsAmazing569 they're ok in generating Z3 code if you do it step by step and via a code analysis feedback loop - like you should with any other language.

you’re right, but any non one-shot prompting would have introduced additional complications to an already complicated multi-step process. While Prolog seems quite fine even with one-shot

@@FutureIsAmazing569 I'm mostly using small local models, so many-shot is a default even with Prolog and Datalog. It's not too hard, and having an unlimited feedback loop improves model performance many-fold, so it's a good idea in general to do it with any tool. Another nice thing with local models is that you can do inference harnessing - nudge the model to select only tokens that form correct syntax, and provide a very tight feedback loop for tool usage. Even if you're getting an ok Prolog most of the time with one-shot, it's never guaranteed to be ok for all cases, so a feedback loop is needed even for the very large and powerful models anyway.

@@vitalyl1327 thanks for the insight. The monster PC I use to run local models has been idle lately, since it’s been quite hot lately. I should get back to doing just local models in a week! I agree that feedback loop should be default for such tasks

100% agreed

@@ioannischrysochos7737 it does feel like the future you’re talking about has already arrived in the form of o1. It does feel like this might be already be present in it.

@@FutureIsAmazing569 there is a difference between statistics and formal logic. In some cases you need to be sure that answer is correct, on most of the cases 99% correctness will suffice.

Prolog is more exciting that llms. But nobody knows prolog, or logics. Wrong time. But defenitly a part of agi what ever that is.

@@adrianojordao4634 Prolog works well with LLMs both ways - not just Prolog generated by LLMs, but Prolog execution traces explained to LLMs the way they can understand. There are some potentially interesting "explainable prolog" attempts out there, check out pyexpert Python package for example.

Single chat bot (no matter how much I prompt-engineer it) is clearly failing. But there's plenty of papers where researchers claim they can achieve some results with multiple agents. I did not verify this yet myself, but am planning to do that. Regarding AI training data - yep, I think fine tuning a single chatbot would solve all of these issues. But I don't think this is a piece of the AGI puzzle, we would just be specializing the chatbot to one additional specialization.

I tried making a small group "simulation". With 6 chat bots. It was pretty interesting, how they could talk, make plans and execute on those plans and move around the game world. I really couldn't run it very long though. It ate through my budget in an instant. It was just hammering that API with large requests. I was basically network speed limited. and money limited.

Oh, yes, if you want to do anything automatic with multiple bots, it has to be local models. Which are of course a bit weaker. But in this case, it was done with local Llama3, but pretty much matched GPT-3.5 turbo

Plus, very interesting that you mentioned how we evolved to get ours priors. My guess is that a potential breakthrough will come from "large evolutionary models" where agents will learn cooperate compete and reproduce in environments, solving tasks and learning how to generalize. Btw deepmind has developped algos where agents where learning games in a rich 3d environment with dynamic objective function and they were able to generalize pretty good on unseen games.

@@geogeo14000 Thanks a lot. Arc is really under the radar during the current AI hype

@@geogeo14000 evolutionary algorithms coupled with neural networks is such an interesting idea. But I feel that architecture itself needs to subjected to evolutionary pressure. E.g. we would modify DNA with mutations and crossovers, and that DNA could be used to grow slightly different architectures. Evolving into a recurrent network with memory would be totally viable. But of course resource wise this is a huge task.

@@gawty that shouldn’t be hard, will do, thanks for the advice!

Thanks, the channel is brand new, so no worries, I am just glad you liked it!