Because more time is spent being overtaken by other vehicles than is spent in overtaking them, that’s what a Nature paper points out.

Traffic jam

Indeed, here is an example. Suppose there are two lanes on the way to work. One lane moves twice faster than the other. Say, the distance to work is 10 km, the speed on the slow lane is 25 km/h, then the speed of the fast lane is 50 km/h. So the travel time to work over the slow lane is 24 min, the travel time over the fast lane is 12 min.

A driver does not know which one is which, so every morning s/he choses a lane at random. Suppose s/he works 20 days per month, out of which in the expectation 10 days s/he would arrive by the fast lane, and 10 days by the slow lane. The total travel time spent while traveling on the fast lane would be 2 hours, and the time on the slow lane would be 4 hours. The driver would be spending 2/3 out of all the travel time on the slow lane and watching cars on the other lane pass by.

Even though the chances of ending up on a slow or fast lane are the same, more time is spent on the slow lane, hence, more time is spent observing that the other lane is going faster. So that is why it seems for the driver that s/he is unlucky when choosing lanes.