Re: adaptive word scoring

[Lars Magne Ingebrigtsen <larsi@ifi.uio.no>]

Sean Lynch <seanl@Internex.NET> writes:

> I remember reading earlier in this thread about the possibility of
> rating words based on interestingness, and I think this is probably
> the way to go.  The fundamental theorem of information theory tells us
> that the value of any piece of information is inversely proportional
> to its probability of occurrence.  Therefore, we should keep some sort
> of history of the number of occurrences of each word in the adaptive
> scoring criteria (i.e. the subject lines) and estimate the probability
> of each word's occurrence, weighting the affect of each word on the
> final score by the inverse of the probability.

Would it suffice to calculate this on the fly (from the articles
currently in the summary buffer), or does this have to be stored in a
database? 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen