The concept of social dominance is not a myth. A myth is an invented story. The concept of dominance has been, and remains, a very important one that has been misunderstood and misused, often by those who haven't spent much time conducting detailed studies of other animals, including those living in the wild.
Dominance is a fact. Nonhuman (and human) animals dominate one another in a number of ways. Individuals may dominate or control (1) access to various resources including food, potential and actual mates, territory, resting and sleeping areas, and the location in a group that's most protected from predators; (2) the movements of others; or (3) the attention of others, an idea put forth by Michael Chance and Ray Larsen. Even if dominance interactions are rare, they do occur, and that is why it's important to log many hours observing known individuals. As one gets to know individuals in a group he or she also learns more and more about the subtle ways in which a wide variety of social messages are communicated, including those used in interactions in which one individual controls another.
Complicating the picture is the phenomenon of situational dominance. For example, a low ranking individual may be able to keep possession of food even when challenged by another individual who actively dominates him or her in other contexts. I've seen this in wild coyotes, dogs, other mammals, and various birds. In these cases possession is what counts. I've studied social, dare I say dominance relationships, in a wide variety of species, and any introductory textbook on animal behavior contains various definitions of dominance and many examples. Another complicating factor is that there's a lot of variation in the way in which dominance is expressed both within and between species.
What has happened over the past 30 or so years based on extensive comparative behavioral research is the discovery that dominance is not a simple or ubiquitous explanatory concept as some took it to be. For example, for many years it was assumed that dominant animals mated the most and controlled access to various resources. Now we know this isn't necessarily so in all species or even within different groups of the same species. Often, less dominant or subordinate animals are able to mate and can control others in different contexts.
So, is there much new under the umbrella of dominance? Yes and no. In 1981 renowned primatologist Irwin Bernstein published a most important essay on dominance in which he discussed all of the above and more. Bernstein and others since have convincingly argued that we need to be very cautious about throwing out the baby with the bathwater because the concept of dominance is useful despite newly discovered complexities and subtleties.
To be sure, ethologists have not called dominance a myth. Rather, they've noted that a univocal explanation of dominance, one relying on a single unambiguous meaning of what dominance is, is misleading and simplistic. An excellent discussion of dominance in various animals can be found here.
Dominance surely is a slippery concept with respect to how it's expressed and how individual variations in social dominance influence behavior. A narrow definition doesn't necessarily hold across species, within species, or across different contexts. Many discussions in which the broad concept of social dominance is criticized are very informative, but to claim that dominance is a myth flies in the face of what we know about the subtle, fleeting, and complex social relationships and on-going social dynamics of many group living species.
Note: Some of the critics of social dominance include those who study and/or train dogs and it was this essay that got me revisiting the notion of dominance. In this essay the author writes, "Dr. David Mech, the world's leading expert on wolves, says that in 13 years of studying the wolves on Isle Royale in Michigan he never [my emphasis] saw any displays of dominance." When I read this I was (and remain) incredulous. In the limited time I've watched wild wolves in Yellowstone National Park I saw dominance displays on a number of occasions and other researchers also report these sorts of interactions.
Some of the critic's concerns are legitimate because we need to be very careful about generalizing from the behavior of wild and captive wolves (from whom dogs emerged) to the behavior of dogs. It's also important to realize that the misuse of the concept of dominance that results, for example, in a person violently dominating a dog, is not a valid, respectful, or humane way to treat or to train our best friends.
Note 2: David Mech's essay can be found here. It's important to note that he does not reject the notion of dominance (nor does he reject it here). Indeed, he wrote, "Similarly, pups are subordinate to both parents and to older siblings, yet they are fed preferentially by the parents, and even by their older (dominant) siblings (Mech et al. 1999). On the other hand, parents both dominate older offspring and restrict their food intake when food is scarce, feeding pups instead. Thus, the most practical effect of social dominance is to allow the dominant individual the choice of to whom to allot food."
Clearly there's lots of confusion on this matter (as well as the use of the word "alpha") and there seem to be myths about what Mech actually thinks about these matters. He does argue, as do others, that the notion of social dominance is not as ubiquitous as some claim it to be, but doesn't reject it across the board.
Note 3: Another of David Mech's papers titled "Prolonged Intensive Dominance Behavior Between Gray Wolves, Canis lupus" that clearly shows he does not at all reject the notion of dominance can be found here (see also).
In response to my essay, David Mech wrote to me:
"I probably won't have time to read this right now, for I'm preparing for a trip out of the country early next week. However, a quick scan of the Kelley article reveals much misinformation attributed to me. This misinterpretation and total misinformation like Kelley's has plagued me for years now. I do not in any way reject the notion of dominance."