Unlike most pinnipeds, adult harbor seals are usually solitary and rarely interact other than to mate. However, they often haul out in loosely organized groups. These groups may include both sexes and all ages.

Harbor seals generally do not touch each other when hauled out. They maintain a space between them of a meter (several feet) or more. If touched by another harbor seal, they respond with growling, snorting, flipper-waving, head-thrusting, scratching, or biting.

Harbor seals often haul out onto rocks or beaches. Harbor seals can haul out any time of the day or night. Haul out time is often limited by tide height. At many sites harbor seals are more likely to haul out at certain times of the day, such as afternoon and evening hours, although this behavior may vary with the season.

Harbor seals often are found sharing haul-out space with other pinnipeds, such as California sea lions and Northern elephant seals. Harbor seals rarely interact with other species but show aggression if threatened.