Web Scraping with Prolog

Поділитися
Вставка
  • Опубліковано 9 лис 2024

КОМЕНТАРІ • 18

  • @jorgschuster773
    @jorgschuster773 3 роки тому +1

    Thanks for this video. I would be interested in more videos about Scryer Prolog. It would be nice to see some of Scryer Prolog's string parsing capabilities and also how Scryer and Rust interact.

  • @michaelkohlhaas4427
    @michaelkohlhaas4427 4 роки тому +1

    *Beautiful code snippet! Great video! Thanks!*

  • @zacharylarson1245
    @zacharylarson1245 4 роки тому +1

    Awesome, nice work! I've got it working and playing around with it. How would I scrape for other media content and save to disk, for example?

    • @ThePowerOfProlog
      @ThePowerOfProlog  4 роки тому +1

      Thank you very much for your interest, I am glad you find it useful!
      The Scryer functionality for actually fetching web pages is still under heavy development, and will at some point allow to read a web page into a list of characters. At the moment, there are no public interface predicates to do this. However, as an interim solution, you can do it already now with the internal predicate read_to_end/2 from library(http/http_open), which you can invoke with http_open:read_to_end(Stream, Chars).
      Once we have obtained such a list of characters Chars, we can write it to a file for example as follows, using format/3 with a stream as its first argument:
      setup_call_cleanup(open(FileName, write, Stream),
      format(Stream, "~s", Chars),
      close(Stream)).
      To convert between different encodings, a few features are necessary, and I hope they will become available soon. See for example a generalized open/4 to read from streams with flexible encodings:
      github.com/mthom/scryer-prolog/issues/614
      Enjoy!

  • @Moousaka
    @Moousaka Рік тому

    Great video. I'm currently wondering how I can do the same thing but with json instead of html? Is there any helper functions I can use? I think my goal is to produce terms from that json. Cheers!

  • @michaelkohlhaas4427
    @michaelkohlhaas4427 4 роки тому +1

    *How does Scryers fit into the Prolog landscape? You seem to be very inclined to it. Are there any reasons to choose Scryer over eg. Swi-Prolog?*

    • @michaelkohlhaas4427
      @michaelkohlhaas4427 4 роки тому +1

      *Ok. I see. Compact representation of strings is a big plus for some type of applications. Will do some quick and dirty performance comparisons. Had no problems compiling Scryer although I'm a bit sceptical about Rust. It's supposed to be very fast and the results on Techempower speak for themselves. But when you look at some ordinary programs like word frequency, sudoku solver etc. it's rather discouraging. I'll check it out.*

    • @ThePowerOfProlog
      @ThePowerOfProlog  4 роки тому +2

      Yes exactly! The compact representation of strings as lists of characters enables the use case Prolog was designed for: efficient text processing. Scryer Prolog is the first Prolog system that implements this, and best of all, Scryer fully conforms to the Prolog ISO standard, so the programs you write for Scryer will always run with this system, and can be easily ported to and from other Prolog systems.
      That being said, Scryer is still in its earliest stages of development, and I expect its performance to increase significantly in the coming years and decades.

  • @michaelkohlhaas4427
    @michaelkohlhaas4427 4 роки тому +1

    Would it make sense for subscribers of your channel to make propositions of possible next video topics?

    • @ThePowerOfProlog
      @ThePowerOfProlog  4 роки тому

      Absolutely, please let me know any time what you are interested in! There are already a few requests in the comments, so there already is a bit of a backlog. If I know there is specific demand for certain topics, I will surely prioritize them if at all possible. Thank you a lot for your interest!

    • @pricesmith1793
      @pricesmith1793 Рік тому

      @@ThePowerOfProlog nlp, semantic web, reasoning! I'm sure there's more.

  • @SO-kd3co
    @SO-kd3co 3 роки тому

    Hello,can enhello can someone help me, i loaded file and it saves ina temp directory, how can I change the directory?

    • @ThePowerOfProlog
      @ThePowerOfProlog  3 роки тому

      In Scryer Prolog, you can use the predicate working_directory/2 from library(files) to change the current working directory. Does this work for you?

    • @SO-kd3co
      @SO-kd3co 3 роки тому

      @@ThePowerOfProlog I will try it, thank you very much.

    • @SO-kd3co
      @SO-kd3co 3 роки тому

      @@ThePowerOfProlog But my Prolog is Swi ..