Why Maximum Entropy?

Поділитися
Вставка
  • Опубліковано 4 гру 2024

КОМЕНТАРІ • 7

  • @klam77
    @klam77 5 років тому +1

    Indeed: Not just stat mech--->bio physics; but Today, stat mech--->economics (aka "econo-physics"). More fruitful insights than traditional economists have EVER been able to assert about societies of individuals (Macro economics). Fascinating: Boltzmann distribution represents distribution of wealth.

  • @jayakumarsingaram8218
    @jayakumarsingaram8218 11 місяців тому

    Nice

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w 2 роки тому

    Is Maximum Entropy is graduate level subject? Which course subject is it usually taught in if I want to search google or course catalog?

  • @1637shubham
    @1637shubham 9 років тому

    Hi I was reading about independent component analysis (ICA) which is a method for Blind Source Separation problem. The heart of the ICA is finding components whereby maximum entropy is possible. The question is how is the entropy associated with source probability?

    • @spresse1
      @spresse1  9 років тому

      +SHUBHAM GUPTA
      Hi Shubham, I am not sure I completely understand your question but I'll try. A measure of a source distribution's (i.e. probability) information content is related to its entropy. This is true of any distribution. While I am not an expert on ICA, a quick wiki search reveals that the goal of ICA is to identify maximally independent latent variables underlying a distribution. I can see how using a Kullback-Leibler divergence with P(A,B) as p and P(A)P(B) as q (with entropy being -sum p*log p/q) and maximizing over A and B would be a very reasonable thing to do. Essentially, this would give those values of A and B for which the joint distribution is as close to the product of marginals as possible.

  • @jupita3000
    @jupita3000 9 років тому +1

    Could you describe how max entropy is used in classifying pixel values in a digital image given a training sample?

    • @spresse1
      @spresse1  9 років тому +3

      +Devin Johnson
      Hi Devin, suppose each pixel intensity comes with some uncertainty. Then, accounting for all possible pixel values across all pixels, there are a wide range of acceptable images. MaxEnt allows you to converge to the least biased of those intensity assignments across all pixels. The result is the "deconvoluted image". It's a fascinating field and I recommend google Skilling/image processing/MaxEnt for more info. Hope this helps!