There's no guarantee that it's a simple arithmetic mean. They could weight by recency or do any number of more complicated calculations.

At the very least, if you're like most users who rate between 4 and 5, you can most likely tell if the driver rated you a 5 (rating goes up) or below a 5 (rating goes down). It would take a pretty strange algorithm to violate that. Of course, this only helps if your rating changes at all

While this is likely to work much of the time, it wouldn't take a particularly strange algorithm to break it. Consider an average rating that is just the arithmetic mean of your last 10 rides. Your rating going up would just mean your most recent rating was better than the rating 11 rides ago that it displaced. If you got a 2 11 rides ago and a 3 on your last ride, your average rating would go up despite getting a below-average rating.

Sure. But to clarify, it seems like for most people, the ratings they receive consist of mostly 5s, with rare outliers (anecdotally in this thread, it seems most ratings are >= 4.5 and most are ~4.8). In that case, the odds of one outlier pushing another outside the moving average window is pretty low.

If you keep track of the total number of rides.

And you assume that there's no weighting or discarding of ratings that goes into the average.

Not necessarily. It shows you only the first decimal.