Відео

Hahahaa, so true!

@@DevXplaining So I was correct? 😀

Yah, it's a good choice! :)

Thank you! Appreciated!

Thank you! I'll take a look at those, details are important.

Yeah, they tend to have great challenges and competition leagues going on most of the time, some with really great prices including trip to Vegas. I won a smaller qualifier and got some sweet swag for it. As for how to win. I try to give some basics in this video, but the fun thing is: There's no right or wrong answer. I always try to compete against myself, make it faster than before. What I do know is that typically the fastest models are trained using waypoints that are specific to track, and since that requires human to define optimal routes, it's not what I typically like to do myself. I enjoy letting the algorithm figure it out using 100% machine learning. It does not grant the greatest results but I enjoy it. I always start with a theory of how to reward. Let's say I reward based on progress on track, and keeping close to centerline. Then I let the training do its thing. I've surprisingly had great results often with very simple reward function, just a lot of training on proper track. I can drop one hint, there's a great Discord community that I'm also part of, they will happily discuss approaches, tips and tricks, and the community includes some very skilled people who tend to do well in the challenges. Besides, things are often more fun in a community, to share. Here's the link: join.deepracing.io/

@@DevXplaining let me try this. Thanks man

Hahaa thanks! :)

Hi, yes, that's a known issue, it's not perfect. Pdf readers often struggle with table data even in the best RAG solutions, and of course cannot easily understand any information that's not in text format already. It's mostly for poc level example, to make it work better, much more work is needed. What can be done? - check which part is not parsed properly - you can isolate the code that parses pdf and play with it - there are typically adjustments you can make to better meet your needs, but as mentioned, it can mostly handle text chapters and titles, table data and images need some other approach overall - possibility to see the library issues list, see if anyone else is struggling with same issues - possibility to check to see if there's a better library available for your needs - there typically are options. This was just one that was fast to get working. So, pdf documents are not the easiest usecase to feed for RAG, something that would be better structured is typically going to give better results. But it's a nice starting place. I'm a bit concerned myself about losing the graphs, images, and potentially having table data cut in halves etc, those can alter the results vastly. If you can find any better means of converting the pdf to any structured format that would retain what you need, that would improve the results.

Hahaa yeah, I can relate to that :)

Perfect! It's gonna be slowwww... But fully local too :)

Thank you!

Thank you for your feedback! Much appreciated!

Indeed. Seems it's not going to make it into future releases.

Thank you! Much appreciated! More to follow on the topic. Stay tuned. :)

Thank you! Very good ideas!

Well, I use both. VSCode is awesome for dealing with multiple languages and config files, very flexible. Jupyter is great when dealing with mostly Python and data files, soecialized in those. I've got Jupyter plugin in VSCode too, so I use both, depending on what I'm working on.

@@DevXplaining Fair enough... does Lab do anything vsc doesn't?

No, the preview features are always behind the preview flag. Of course the idea is that eventually they get out of preview and become just normal features and thus flag is no longer neede. (Or not, they may never become features, or may change a lot before becoming features). So anything behind the flag is more like a cutting edge sneak peak than something actually released and stable. So good for playing and feedback, but not so reliable for production use - yet. Of course you can hide any flags you need in some kind of startup script etc, so nobody needs to actually see them. Java typically uses all kinds of parameterization anyways.

@@DevXplaining yeah got it thanks. Hope they’ll make those features normal soon ✌🏼

Oh yes, you've hit one of the weaknesses/limitations of the current RAG model in general (and vector databases). They are suprisingly awesome at digesting and using huge amounts of unstructured data. But the simple examples do not even attempt to deal with bits that require specific interpretations. PDF readers are good at grabbing document data, titles and paragraphs, to make sense of it. But they currently already struggle with tabular data, diagrams, charts, and yes, math formulas. Furthermore, those are fed into vector database as text typically, and similarity search also works text-based. And when you retrieve it, you retrieve text. I'm afraid there's not much to do about it, except you can of course create a specific pre-parser for the documents, or tune your pdf processor or write your own, to decide the rules how the pdf or html is interpreted and stored. I don't know your exact use case, but my advice would be: yes, it's not working, because it's not a trivial problem to solve. Yes, you can probably solve it, but you need to rethink it. Math formulas typically benefit from more structured content. You can always handle them outside the system, perhaps store some links or references in the rag solution to find and fetch them etc. I have to say haven't had to solve this issue yet but already the tables are a common challenge and question with current gen RAG solutions. It's a decent start but more is needed.

Oh, forgot one thing: multimodal models that are able to make sense of images might be interesting to explore for formulas, to work around text conversion.

Awesome idea! Working on something else right now, but this sounds like a good plan for a long form video, and a fun one.

Give me your job, Ill do it for easy

Oh yeah, a very good point. Job market is a bit more challenging right now, so anything concrete to be able to show might just give you that edge. In that sense, even the lightweight free certificates you can do within few hours work. Probably not going to be the main decision factor, but just a little bit of extra on top.

Thank you! I did not include frontend because on top of API that is quite straightforward, and tried to keep the video compact. But I've been thinking of doing a bigger video with full stack, because it looks a lot nicer. But not there yet, at least. It would really be a trivial NextJs (or any other framework) web app that you dump into spring boot 'public' folder under sourcecode. So I'll definitely take that under consideration, but probably not going to happen soon.

Interesting idea, haven't done that yet so need to look into it, if I find something cool I'll definitely be back. Not sure how the API works for them yet, but can see how that could be useful.

Haha, should actually be delivered as sealed physical scrolls I think :)

Thank you! Much appreciated!

Hmm, perhaps a topic for another video. Short answer is... depends.

Thank you for the feedback! Much appreciated! Love to learn and share these things, I understand best by building and tinkering. Some more cool RAG stuff coming up soon! :)

Sure, I've done few in the past on my channel, but of course might be nice to redo since both Java and Spring have gone a long way since then. I've mostly been showing Spring Boot + Web module, to create RESTful APIs, and what you want to do behind that is of course up to you, but most often it includes database. But, while waiting for new video, these should still work to give you the big picture: - ua-cam.com/video/xxRw0XIoKvA/v-deo.html - ua-cam.com/video/KsWg6nPVwsg/v-deo.html - ua-cam.com/video/Sg1b0sugbtg/v-deo.html Thanks for tuning to my channel!

Thank you!

Hmm ,interesting question. If I got it right, some customers would get answers based on the document, but others would get answers based on some APIs? If so, in both cases you want to inject some context in system level instruction, either from vector database/document, or an API - so that part remains the same. So when you need to answer an API, you would call the API, get the context for the question, then include that in system direction. No need for vector database, as long as what the API returns a) has all the context info needed and b) is relevant and c) is not too massive to chat in the available tokens. So in both cases, grab the context information, feed in the system level instruction like in my code. Decide on top of that which context is used for which user. One thing I'm not doing in this video, is keeping a conversational memory: I'm just asking a question, and activating context + chatgpt to handle it, but then forgetting all the conversation. So with conversational memory, would need to be a bit more careful, because also the conversational context is typically injected into the calls. Also I think would have to be careful with size, of course depending on your model token limits. And you would need to keep the contexts separate so each user would see their own personal context only (instead of same shared one for everyone). I hope this helps and I got the question right!

Thank you! It seems RAG usecases are in demand right now, and there's also uses for own internal tools. Some of these of course will appear as part of available tooling, but being able to do it yourself is great for like-minded tinkerers that like to understand things by building them, I think.

Noted, need to make a longer video then but would be fun :)

Haha, I actually aporeciate feedback on my pronounciation, lets me work on it. Thank you!

Thank you! Appreciated! :)

Haha, you know, I coded myself an IDE too, when starting to learn Java Swing. It never got full featured enough, but it was fun to build it from scratch and then use it to develop it further. Also made me appreciate people who work on modern IDEs a lot more - there's a boatload of details to solve and take care of to make them work beautifully.

@@DevXplaining i tried to do it with imgui, and failed bc macos is stupid :( and i never even figured out how to make it so JNI executes on the main thread. Altho if java adds a feature like this then ill probably try doing it all over again

Thank you! :)

Yes, a good question. Short answer is that yes, typically that's what happens, and typical software project inside any web framework pretty much ignores you needing to tap into parallelism - you just think about it when accessing background resources. But two reasons to still keep up with Java parallelism features: 1) Well sometimes you do build something from scratch, that would mainly happen if you want to do some intensive calculations without web servers, or if you are building a server. 2) Virtual Threads are currently finding their way into popular servers and frameworks, so mostly you can just enjoy the benefits, but it's at least good to get the idea why they may bring more performance when your favorite framework start using them. So mostly I think this is the message that Java platform is still very much alive and kicking, and still getting better. Spring Framework is already embracing them, for example: spring.io/blog/2022/10/11/embracing-virtual-threads Might be a fun topic for a future video.

okay great. Maybe you might consider making a video in the future for this question or you could please share your ideas here. I'm a bit confused if Virtual Threads are going to replace Reactive Programming. If they would, why? and if they won't, why?@@DevXplaining

Yeah, it's a good question. AI takes many forms, so rule of thumb is: anything you would not like to/be permitted to publish on front page of a popular magazine tomorrow, you should not send to AI - unless you know how the data is handled, stored, etc. And additionally, is your input used to train the models (which could cause it to pop up as an answer for future users). So for example using ChatGPT like I do here, I'd limit it to general advice. But for example Github Copilot Enterprise is my tool for daily work, and there are some rules in place to stop my code from leaking. So either know how your AI handles input data, or be safe and just use it for general questions. But as always, keep your secrets secret! :)

very insightful, Thank you@@DevXplaining

Yes, agreed. I was teaching Java for a decade and that getting started part for new coders is tough with it. Once you push through it does get better. So I'm very happy to see they finally added beginner-friendly features, even though old hands really don't need all that :) But much easier to learn it, teach it, do something small just for fun.

Hi, that's a good question. That's built-in the speech recognition library, so it stops to interpret the speech when there's long enough pause. So I'm kinda giving it the whole text by not pausing before I'm at the end. Similarly, the SR library also starts interpreting speech to text only when some audio begins. For any SR library (I'm using Google API in this example), there are typically settings to adjust how long pause means speech has ended, and it's time to interpret it. Alternatively, there's also possibility to introduce walkie-talkie-like push button to speak mechanism to signal end. Finally, one could also of course just interpret constant length blocks constantly, but then that might not make sense to be sent for AI to interpret.

To give a more concrete answer, here are the parameters that could be passed in my code to adjust how long pause means end of block of speech (in seconds): recognizer = sr.Recognizer(pause_threshold=0.5) # or other value

Hi, you know, that is a possibility I've been thinking about. :) Two things on the way: It takes some effort, so I need to find the time for it. Second, there's been a lot of options available currently, so need to choose which one is good. If I can find the time, what I've been thinking about would be to get something running in Raspberry Pi/other low-end hardware that's easily available. I would probably use a suitable Mistral model right now. But no promises, time is hard to come by these days :)

Thank you! Much appreciated!