The graphs at SMOFInfo.com haven't been updated in five years -- partially because I haven't gotten around to doing it myself -- but I saw in some of the post-Worldcon blogs someone else having done a similar graph. Trying to leave out the significant outliers like the Australian Worldcons, the trend to me appears to be weakly downward. Not hugely so, though; off the top of my head, the 20-year average attendance is around 5500 +/- 1500 for North American and European Worldcons. That's still a lot of uncertainty, and I think 5500 is too low for effective use of resources, for reasons I've discussed excessively before.