Jump to content

Featured Replies

Posted
Word Frequency Distributions

Word Frequency Distributions

by R. Harald Baayen

Publisher Springer Science & Business Media
Published Date 2002-09-30
Page Count 335
Categories Computers / Artificial Intelligence / General, Computers / Artificial Intelligence / Natural Language Processing, Computers / Speech & Audio Processing, Language Arts & Disciplines / Linguistics / General, Language Arts & Disciplines / Linguistics / Historical & Comparative, Language Arts & Disciplines / Linguistics / Psycholinguistics / General, Mathematics / Probability & Statistics / General
Language EN
Average Rating N/A (based on N/A ratings)
Maturity Rating No Mature Content Detected
ISBN 1402009275
Book Cover This book is an introduction to the statistical analysis of word frequency distributions, intended for linguists, psycholinguistics, and researchers work ing in the field of quantitative stylistics and anyone interested in quantitative aspects of lexical structure. Word frequency distributions are characterized by very large numbers of rare words. This property leads to strange statisti cal phenomena such as mean frequencies that systematically keep changing as the number of observations is increased, relative frequencies that even in large samples are not fully reliable estimators ofpopulationprobabilities, and model parameters that emerge as functions of the text size. Special statistical techniques for the analysis of distributions with large numbers of rare events can be found in various technical journals. The aim of this book is to make these techniques more accessible for non-specialists. Chapter 1 introduces some basic concepts and notation. Chapter 2 describes non-parametricmethods for the analysis ofword frequency distributions. The next chapterdescribes in detail three parametricmodels, the lognormal model, the Yule-Simon Zipfian model, and the generalized inverse Gauss-Poisson model. Chapter 4 introduces the concept of mixture distributions. Chapter 5 explores the effectofnon-randomness inword use on the accuracy of the non parametric and parametric models, all of which are based on the assumption that words occur independently and randomly in texts. Chapter 6 presents examples of applications.
More Information
  • Views 20
  • Created
  • Last Reply

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

Important Information

Terms of Use Privacy Policy Guidelines We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.