Maybe if the database wasn't still holding masses of dead/inactive accounts/devices it would be more efficient?

Would reflect a more realistic userbase figure too.

But then, as one of the biggest projects it has a lotta stats to process I spose.

Stats and other maintenance has in fact been much quicker since the last tweaks **. From what i see the actual lockout part of stats run started like 1.5 hours later.... because we had that onslaught of Work Unit Volume to deal with that knreed wrote about. The second half of Monday processed 126 CPU years to get an idea and for the day we broke thru the record of WU big time by getting 485 thousand tasks in which was 18k over the Friday record.

The 6 or 12 hour interval just deals with personal stats. The daily also handles all the other stats like team and bunch of housekeeping. And what you don't see but will at some time is that for each member WCG is also keeping track of contribution per project. The product of that you already saw in the badges. Wait until you see more appear on your My Grid page for the boast factor. It's always work in progress and always seeking new boundaries.

** The Saturday noon run is scheduled to take over 2 hours and was done in an hour to get the idea!

So far my party line walk .... cruching on

Some of this is a result of the design of the UD process, which takes more and more time to process longer and longer chains.

The remainder is the result of recent workunit splits that gave a lot of short workunits, kicking up the results and the workload.

On the old grid.org, which had been around longer it took the entire evening to process UD, as they only had one stat run per day.

Once UD goes away and becomes a constant, it can be merely added to the conversion of the BOINC result, rather than tying up the works.

I understand that bigger volumes take longer to process but I also thought we had a set up that was supposed to be easily scalable. Guess that doesn't necessarily apply to the stats. When you offer daily stats, regardless of what other projects do, they need to be timely to be useful. On the flip side of the coin, I can understand the limited resources and being hamstrung by UD until it deservedly bites the dust. Not being familiar with the behind the scenes aspect of just what is involved in updating the stats, it's hard to understand these complexities and limitations. Most users won't bother trying to consider such things. They expect timeliness. How about this for an idea - take the current process and have it simply restart itself each time it finishes so that it is running all the time. Each cycle will simply process what has come in during the previous cycle. Rather than have it lock users out of the statistics side of the website, let it update one of two copies of the databases and such. At the times you want to "release" the updated info to the public, you simply "redirect" the website to the current copy of the databases and let that copy "replicate" with the old copy and start the repeating cycles over again. That would eliminate or significantly minimize the current outage time and would hopefully involve a minimal amount of work. Even with the better/faster/fancier stuff coming post-UD, it could be a good approach to handling future growth.