Caucuses are influenced by a.)who can be present (although as Kevin points out Nevada has modified that), b.) those who are system-savvy, and c.) are open to influence and intimidation that is not present in a secret ballot. The drawback you point out--that of an individual candidate with a modest but devoted following--is much more likely to happen within a caucus than a closed primary election (i.e., an election which is only open to declared party members and does not allow for people to change parties to vote in a different party the day of the election. I do not support open primaries because they are not representative of the party).

For the record, the primary system was developed in order to circumvent backroom manipulation by party leadership within the caucus system. This cut out all but the most privileged. As it is now, caucuses bias toward those who are not working on that day, who have the privilege of time to attend, and who have the ability to be present. IOW, caucuses trend toward the motivated, those who can be recruited/paid to be there, and the upper classes. Primaries are much more representative of ALL the members of a political party than caucuses are.

Also note that my undergraduate degree is in political science with an emphasis on electoral politics. Besides personal experience I have the training to back my informed opinion.