Comparing Bird Records While Accounting for Effort and Detection

The short answer

Two sets of bird records can be compared fairly only when they were collected the same way, or when the differences in how they were collected are measured and accounted for. That means checking the protocol, location, date, start time, duration, distance traveled, party size and whether the observer reported every species they detected. A raw count tells you what was recorded during a particular outing. It does not tell you how many birds live in an area, and a change in raw totals between two seasons or years is not, on its own, evidence of a population trend. Population or trend claims need a documented analysis that models effort and detection.

This applies whether you are looking at your own notebooks from the prairie and wetlands around Medicine Lake in northeastern Montana, a set of shared checklists from a birding group, or public eBird data for a county.

Terms that shape every comparison

• Checklist: in eBird, a record of one observation event. It includes the protocol used, the location and time, how long the observation lasted, how far the observer traveled and how many people were in the party.
• Complete checklist: a list on which the observer reported every species they detected and identified. Only on a complete list can a missing species be read as a non-detection.
• Non-detection: a species was not detected on a complete list. This is not proof that the species was absent. It may have been present but silent, hidden or missed.
• Incomplete checklist: a list that reports only some species, such as highlights. It cannot distinguish a species that went undetected from one the observer simply chose not to report.
• Detectability: the probability that a species that is present will be detected and correctly identified. It varies with the species, habitat, season, effort, time of day and conditions such as wind or rain.

Why raw records mislead

Bird records reflect both the birds and the observation process. The eBird Best Practices guide identifies several ways the process distorts what the data seem to show.

• Detection varies. A species that sings loudly at dawn in open grassland may be easy to record in early summer but hard to record on a windy afternoon in late summer, even if the same number of birds are present.
• Observers report some species preferentially. Unusual or exciting birds are more likely to be written down than common ones, especially on incomplete lists.
• Accessible places get sampled more. Roadsides, refuge auto tours and popular hotspots receive far more visits than remote pasture or wet meadow, so the records describe accessible habitat more than the landscape as a whole.
• Participation is uneven across seasons and years. A rise in submissions can mean more people were birding, not that more birds were present. Changes in the number of records often reflect changes in coverage.
• Shared checklists duplicate the same event. When a group birds together and each member submits a copy of the list, the same birds appear several times. These should be collapsed to one event before analysis.
• Locations can be approximate. A traveling checklist or a hotspot coordinate gives one point for an outing that may cover kilometers, so it may not show where each bird was actually seen.

A worksheet for checking comparability

Before comparing two sets of records, fill in the same fields for each. Any field that is missing or mismatched is a reason to separate the records or to treat the comparison as descriptive only.

Comparison question:      ______________________________
Record set (A or B):      ______________________________
Species:                  ______________________________
Route or habitat:         ______________________________
Dates and start times:    ______________________________
Protocol:                 (stationary / traveling / other)
Duration and distance:    ______________________________
Party size:               ______________________________
Conditions:               (wind, precipitation, water levels)
Complete-list status:     (yes / no)
Shared-event ID:          ______________________________
Raw count:                ______________________________
Identification doubts:    ______________________________
Model used, if any:       (quantity estimated and assumptions)

Steps for a fair comparison

• 1. Define the question precisely: which species, which place or habitat, which dates and which kind of comparison, such as this June versus last June on the same route.
• 2. Flag records with missing or mismatched fields from the worksheet.
• 3. Separate records by protocol. Stationary counts and traveling counts should not be pooled as if they were the same.
• 4. Collapse duplicate shared checklists so each observation event counts once.
• 5. Keep complete and incomplete lists apart. Use only complete lists when you need to interpret a missing species as a non-detection.
• 6. Compare matched groups descriptively, for example records per hour on complete traveling lists of similar length started at similar times.
• 7. Name any unresolved differences in coverage or detection before interpreting an apparent change.
• 8. If you need a statement about population size or trend, use a documented analysis suited to that question rather than raw totals.

Example: why effort changes the picture

This is a hypothetical illustration. One checklist records 12 birds of a species over 2 hours. Another records 9 birds of the same species over 1 hour. By raw count the first list looks higher. Divided by time, the first works out to 6 recorded birds per hour and the second to 9 per hour, so the order reverses.

The rate is a fairer descriptive comparison than the raw count, but it is still not an estimate of abundance. It does not correct for differences in distance traveled, party size, time of day, weather or habitat, and it assumes detections accumulate evenly over time, which they often do not.

Descriptive counts versus modeled estimates

Criterion	Descriptive count or rate	Modeled relative abundance
What it reports	Birds recorded on specific outings	Expected count on a standardized checklist
Handles effort	Only if you match or divide by it yourself	Effort included as predictors, then fixed at a standard value
Handles detection	No	Accounts for variation in detection, but does not estimate absolute detection probability
Output	What was recorded	A relative index, not a population total
Suitable for	Describing your own or a group's observations	Comparing patterns across space or time under stated assumptions
Main risk	Mistaking changes in coverage for changes in birds	Comparing outputs built on different species, regions, seasons, effort or model designs

How a modeled estimate is built

The eBird Best Practices guide combines suitable filtering of checklists with predictors that describe the observation process, such as duration, distance, start time and party size. It also uses spatial and temporal subsampling to reduce the effect of some places and periods being far better covered than others. The guide's numerical filtering thresholds depend on the scale of prediction; they are not universal rules for how a survey must be run.

Its relative-abundance example uses two stages. The first predicts a calibrated encounter rate, meaning the chance the species is reported on a checklist. The second models the count given that it is reported. Predictions are checked against data withheld from model training, and effort is then fixed at a standard value so maps compare like with like.

In the guide's worked example, a Wood Thrush model for Georgia is trained on June records from 2014 to 2023. Predictions are illustrated for one observer traveling 2 kilometers in one hour on June 15, 2023, starting around 6:37 a.m. Those settings define that example only. They say nothing about prairie species or trends at Medicine Lake.

Comparing two modeled outputs

When two maps, indexes or trend estimates appear to disagree, check that they share the same species, region, season, standardized effort and model design, and look at how well each model predicted withheld data. If any of these differ, the outputs are not directly comparable, and neither should be preferred on the basis of the numbers alone.

What this means for prairie records

If you compare seasons or years of your own records from grassland and wetland sites, you can describe what you recorded under matched conditions. Saying that a species increased or declined in the area requires a documented analysis that accounts for effort, detection and uneven coverage. Raw checklist totals cannot establish that, however many years they span.