EOC Folding @ Home - Stats News & Updates
Pages (14): « 1 2 3 4
 6 7 8 9 »
... Last »
|Did some testing yesterday...04.02.08, 10:09am CST
Yes that's right, I came into the office on a Sunday afternoon to do some work for you guys. I managed to hack the code into a working state to work with the current DB and make it SMP capable and run some time trials.
Scaling up incrementally from 1 to 4 threads showed almost the same improvement as before, there was a lot more "umph" to begin with since these have been the fastest processors I've used on the stats. The stats even scaled up to 6 cores, but I just wasn't that impressed with the improvement in performance from 4. Beyond 6 there really wasn't much happening, I think it finally hit the bottleneck of the 15K RPM hard drives.
Even though the code is kind of in a rough state for SMP usage, and it's not running the latest version of MySQL, I think a single quad core will suffice and give the best "bang for the buck". That greatly simplifies things and gives more options for hardware to buy.
I also did some testing with the Dell by removing some of the FBDIMM modules to see how power usage changed. Each FBDIMM uses a minimum of 10W idle... Yeah, so you can see how 12 DIMMs x 10W = 120W of power usage right there! The AMB chip in each module is responsible for at least half the power consumption, so using higher density modules could drop usage considerably, but the cost also goes up considerably. But since I don't think I will be going with a dual-socket board, I will not need to use FBDIMMs. I just thought I would post that as some "food for thought".
So today I'm going to call up Dell and return the server. Once they credit me I can order parts for the new (v2) server.
On a different note, I've gotten some complaints from people about a long delay in their points showing up in the stats. This is not a problem on my end, this has something to do with Stanford's back-end. I can only process the numbers which are provided in the files they generate for us. I emailed Vijay this morning and will let you know what he says as soon as I get a reply.
UPDATE: I got a reply from Vijay, "There was an unusual situation this weekend, where 2 servers weren't being credited over the last few days." The issue should be resolved, I'm not sure how long it will take for the points to all show up (if they haven't already).
|Good News... Bad News...03.29.08, 8:58pm CST
Before getting into the good and the bad, I wanted to post a pre-notice about a change to the stats that will affect anyone using sigs or XML. I'm going to be re-numbering all the users since there are a lot of gaps and waste when users were getting added and deleted. Also I must get an email a week from someone who is confused about this number and how it relates to everything. So I think it would just be best to eliminate it from the user side and stick to the same format as the FAH client: User Name / Team ID. Eventually the web pages will all reflect this too, so links will show the user name instead of an arbitrary number. I'm going to update the sig page hopefully tomorrow or Monday to remove the number based id info, but it will still work for a while. Like I said this is just a pre-notice, I don't want to finalize anything until I get the new stats coded.
First for the good news... We finally have our Internet back up and running at the office. It was actually switched on a day early (Wednesday), after two and a half weeks of down-time!
Now for the bad news... The Dell server is just too hideously loud and generates too much heat (even when idle), so it has to be returned. When re-building the office we just didn't plan for *that* much capacity in the closet, otherwise we could of added additional cooling and noise-insulation, heck even made the room bigger. I guess the saying still holds true, hindsight is always 20/20. I called up Dell and there is no problem sending it back (no restock fee or anything). She gave me a couple options and I have a few days to make a final decision. We considered making some modifications to the closet to add a temp-controlled vent fan, but still the high-pitched server fan noise permeates throughout the whole office. It's kind of hard to make major modifications now since all the wiring and equipment and such is finished out on the walls in the closet, so we can't pull the walls down to insulate them better either. I've purchased servers from Dell before, admittedly not this powerful, but none were *that* noisy or space heaters. :(
So, in the mean time since the Internet came back up I've been trying to help someone out with a F@H problem, so the server has been crunching away to gather some useful data for them. I've gotten the server up to 470W under load with FAH, but even that was only really utilizing 75% CPU capacity, they still have some tweaking to do for the SMP cores I guess. I expect the folding stats to peg all processors though, and then some, but we shall see. I also spent part of today prepping the server for the multi-threaded tests, I installed MySQL, compiled PHP, loaded some old data (from Sep I believe, it was the first file I found so I used it). There's no reason I can't see exactly how well it will scale for up to 8 cores before having to ship it back. ;) I found my old multi-threaded code, I just need to bring it up-to-date then I can test it out on Monday. Changing the amount of cores for the code to use is as easy as modifying one variable, so I can also see how MySQL will respond with its thread concurrency setting and find out the best balance. Essentially, instead of one team getting processed at a time, the code basically splits it up and hands out one team to each core to be processed, as the process finishes it gets another team, and so on.
Since I'm sending back the Dell server, I've been doing a lot of research and calculating to what I can do to keep the performance I need, but keep it within a reasonable power budget. Yes, this time it's going to be a home-built system. Another experiment I want to do with the Dell is pull out most of the FB-DIMM modules and see how much power drops. Once I see that first-hand I can better determine if the system will be a single quad-core with regular DDR2, or a dual socket w/quads (like it is now - and I would have to use FB-DIMMs) but utilize higher-density modules and have less RAM too. DDR3 is out of the question, its price is way to high (especially for 8GB) right now.
|The EOC Folding Stats Need You!03.21.08, 9:56am CST
I realize this post might be a little long, but please read all of it so you know a little background and can understand what's at stake.
Lately I've been getting more and more questions and requests from people whom I don't process stats for (because their rank is below the cutoff). It would be nice if I could process everyone's stats but the amount of processing on my server prohibits it. As a matter of fact the current load is getting rather intense and lengthy, and has been putting quite a strain on the main EOC site & forums during updates.
I've been slowly working on the next major overhaul of the stats, most of it is back-end changes, but there are a few visual tweaks to the pages. One thing I decided to do was process the stats on a local development server in my office, which would in turn upload the DB changes to the main site which would only take a couple minutes at most to update. This would alleviate a lot of the load from my web server that has been causing the sites to bog down, and give the ability to speed up processing the FAH stats.
I considered just upgrading my server package at my hosting company and moving all the sites over, but to be honest that is a major pain and a major undertaking. Not to mention the increased monthly costs would double what I'm currently paying, and the additional computing requirement is really only needed in bursts so all that extra power would be sitting idle most of the time. So instead of spending all this extra money each month I figured it would be more cost-effective to buy a new server that can sit here in my office and would just be a one-time expense and a lot cheaper in the long run.
While my development server here is decent (just an A64 3200+ w/2GB RAM), I haven't loaded all the old fah data yet to see how long it will take to process regular updates. But I'm not expecting to see any huge gain and it might even take longer. So before making the official switchover I would like get a halfway decent Intel server with one or preferably two quad-core processors, at least 4GB of RAM, and preferably the fastest hard drives possible (10K or 15K drives in a RAID 1 or 10 configuration). The beauty of the stats is that processing teams can be in parallel, greatly reducing the overall processing time as long as you don't have any big bottlenecks.
I did some quick browsing at the Dell website looking at some of the poweredge servers, and while they have some that cost more than a car, I think a more realistic system can be built for $1,500 - $2,000 USD.
So now if you read this far, here's my plea to you. I would *really* appreciate some donations to help fund this new server. The easiest and preferred way would be to use PayPal (there is a link on the left sidebar).
If nobody bothers to donate then I guess you guys will just have to live with whatever speed the stats process at. I probably won't be able to do any expansion on the teams / individuals being processed, and might even have to reduce the numbers back down to what they were before.
3/5/08 - 2pm Update: Since posting this yesterday afternoon, already several people have sent in a total of $100 USD. Thanks Guys!
3/5/08 - 7pm Update: Wowza, since putting the link at the top on all folding pages I guess a lot more people took notice! I've tallied up a little over $1,000 in donations thus far! A few people emailed me that do not use PayPal and would still like to make a donation. If you would like my address to mail a donation just email me please.
3/6/08 - 10am Update: Eureka! With over 80 people donating anywhere from $5 to $100 we've collected over $2,000! I'm still going to end up spending more than what was donated, in case some extra donations come in, and also so I can say I chipped in my share. ;) I'm going to post a list of the teams / individuals (don't worry I won't put full names) that have donated later today. A HUGE thank you to everyone that has sent in some money, I honestly can't say thank you enough! The only bad news is that the Dell promos with the discounted 2nd proc & free PERC RAID card expired yesterday. I still have to wait for some e-checks to clear, hopefully they will have another promo soon. Either way, by the end of next week I'm going to place the order.
3/7/08 - 11:00am Update: HOLY COW! I just checked my PayPal account and it is almost at $3,100!!! I counted up all the emails and there were about 125 individual donations made (thus far)! I was surprised at how many donations still came in overnight, I'm truly touched by everyones support. It makes me feel all warm and fuzzy inside knowing how much you all are willing to help out, the folding community is really a special group of people.
I do check each of the paypal notifications, thanks to everyone who left some words of encouragement. Also, thank you for all the emails, I'm still trying to reply to everyone!
3/8/08 - 2:00pm Update: Man you guys are keeping me on my toes. Every time I check my email there are always more donations! I just checked and the total is up to almost $3,800!!! I've totally lost count of how many people have donated, lets just go with "A LOT". I thought initially trying to raise just $1,500 was going to be a challenge but you guys just blew me away with your charity and support.
I think most of the inflow has subsided, and the target goal has been surpassed, so I'm going to go ahead and order the server Monday morning. Once I do, I will post all the specs. Obviously when it arrives I'll be sure to post some pictures. I can tell you now that more than likely it will be a Dell PowerEdge 2900 III.
3/10/08 - 7:30am Update: Monday morning has finally rolled around. I updated the list of teams below with even more donations that came in over the weekend! I'm planning on ordering the server a little later this afternoon once my coffee has kicked in. ;)
3/12/08 - 9:30am Update: I finally got the server ordered yesterday. Should be in sometime next week. I'll try to post the specs this evening, but I can tell you the system is super sweet and should last for years to come. I'm in the process of moving my office (same office complex, long story about that) and somehow the phone company has totally screwed up moving the phone and my DSL. The phones were supposed to be moved yesterday, and as of this morning they are still in limbo. I have no service at my old office, and no service in my new office. You call the number and it says 'no longer in service'! So after I do this I'm going to have to call them again and make some poor person cry and hate their job because I am NOT happy. The DSL is *supposed* to be up and running Friday, but I'm not holding my breath. I'm on my laptop using one of the other tenants Internet so I can get a few things done this morning, but I need to get back to work moving stuff and unpacking everything in the new place.
3/20/08 - 8:30am Update: I have good news and bad news. The good news is the server finally arrived Monday or Tuesday I think it was. The bad news is I *still* do not have Internet access at my office. The phone company (AT&T/SBC) keeps giving me the run around saying there is some error with the system and until that gets fixed they can't put the order through for DSL. They can't give me a time frame for WHEN it will be resolved, or even WHO is in charge of fixing the problem. If it's not up and running tomorrow (Friday) I'm calling Comcast Cable and they can have my business, then I'll be calling AT&T and telling them they can stick it where the sun don't shine because this is beyond ridiculous.
I installed CentOS on the server from DVD, I have never seen an install go so fast! But without Internet I'm kind of stuck dead in the water beyond that point. Today I'm going to take some pictures of the server. The rough specs are:
- 2 x Intel Xeon E5440 (2.83 GHz) - I chose these because they were the fastest that were offered in the "E" power range (80W TDP). The next two processors were X models which means 120W TDP and I really wanted to keep this server somewhat energy efficient.
- 24 GB RAM (12 x 2 GB modules) - I was originally going to get 16 GB (Remember there are 8 cores that will be running simulatenously plus regular services needing memory) but figured to fully populate the slots out (they go in sets of 4) since the cost was not that much more and was part of the splurge that I chipped in for. Jumping up to 4 GB modules would of been more energy efficient, but wasn't really cost effective (at this time).
- 4 x 146 GB 15K RPM SAS hard drives - These drives are running from a PERC6/i in a RAID-10 configuration. There wasn't much of a cost difference between the 73 & 146 GB, the 300 GB drives are still very pricy. I chose RAID-10 over RAID-5 or 50 because I'm not using that many drives, and also RAID-10 still gives the best write performance which is key for the stats updates.
- Redudant Power Supplies - a little upgrade, but well spent since the power supply is not your standard ATX form factor. I didn't want to be dead in the water if it kicked the bucket.
- DRAC-5 card - this is a remote access card that essentially is like hooking up a monitor and keyboard directly, but it is done over the network. Another little splurge from my pocket. Since the system will be in the networking closet I didn't want to have to lug in a monitor and keyboard every time I needed direct access, or lug the server out drag it all the way down to my office.
That's about the bulk of the server specs, everything else is just little stuff. Running idle the server consumes a little over 350W of power, I haven't had a chance to stress test it yet but I'm willing to bet that number will jump to at least 500W under stress. The server weighs (according to the shipping tag) 148 lbs, and it sure feels like every last pound is there! I'm glad I got the caster wheels for it, as it would be near impossible to move around on my own. Final cost after tax, shipping, and everything was just a hair under $4,900. There have been in excess of $4,000 in dontaions and I would of felt bad if I didn't put it all towards the server. As you can see from the specs above it is smoking fast. I'm really excited to see what it can do once I get my Internet back up at the office.
Well, time to go to the office, keep your fingers crossed that my Internet is working today.
3/21/08 - 10:00am Update: After getting transferred around a few times this morning with the phone company, a person *finally* thinks they have figured out the problems with my DSL. *Apparently* it is still showing service at my old address, so they have to clear out that from the system before I can re-order new service at my new address. Problem is it will be Tue or Wed before the old info clears out and I can re-order, which then will be another couple days for it to become active. So best estimate (assuming they aren't lying to me again) is next Thu or Fri (27th or 28th) before I have Internet back again. I found a card that has a local manager's phone number when they hooked up the regular phone. I'm about to call him and give him a piece of my mind.
The list below should be current as of 3/21/08, I think I found all the emails that everyone sent since last update. If you made a donation and you don't see your team on the list (and would like to have it included), please send me an email and I will be sure to add it!
- 2CPU.com Folding@Home
- Alliance Francophone
- AMD Zone
- Arch Linux
- Badcaps.net Forum Members
- Beefysworld.com Forums
- Bleeping Computer
- Dutch Power Cows
- EOC Folding@Home Team
- Fighter Ops - Folding@home Team
- Folding For Ancient
- Gentoo Linux Users Everywhere
- HWC (Hardware Central)
- Macrumors.com - Team Folding
- Manic Folders
- Maximum PC Magazine
- MSI HQ Red Rockets
- NCIX.com Forum Folding Team
- NCSU Wolfpack
- Overclock Intelligence Agency (OCIA)
- Overclockers Australia
- Overclockers Club
- PC Games Hardware
- PC Perspective Folding Frogs
- Phoenix Rising
- Pholding Phreaks
- PitCrew @ PCPitStop
- Planet 3DNow!
- Rechenkraft.net - Germany
- SilentPCReview Folding@Home
- Stevens Institute of Technology
- Sudhian Media
- Team 2ch
- Team AnandTech
- Team Cancer Killerz
- Team Engadget
- Team Firefox
- Team Hack-a-Day
- Team Helix
- Team ICT
- Team MacOS X
- Team PimpRig / pcApex.com
- Team Rage3D
- Team Takoda
- Team Ufie
- Team XPS - Folding@Home
- Tech Support Forum
- The Genome Collective
- The Longevity Meme
- the motherboards.org team
- The New Xtreme Resources
- The Outcasts
- The Tech Report
- The University of Texas
- Toms Hardware Guide Community
- Transform The Planet
|Some Updates!12.10.07, 5:01pm CST
Long time since the last update, so hold on to your seats!
First, I am now processing the top 3,000 teams (instead of 2,000). This really has a minimal impact on processing time and hopefully will allow smaller teams to start tracking their progress much sooner.
Second, since changing the processing style over from the indivual files to just the master team / user files I am no longer limited to the top 800 teams for user processing. Users can be added before the 800 mark if their team rank is above 1,200 and the team has moved up at least 5 spots within the past 7 days. This has a slight impact on performance but again this helps the smaller teams out that are really putting forth the effort.
Third, team & user colors! There is a new color, PURPLE which I'm using in the lower part of the scale. The last time the colors were changed was in May, 2004!
The OLD Team scale was:
0 / 1+ / 2.5k+ / 5k+ / 15k+ / 30k+ / 50k+ / 75k+
The NEW Team scale is now:
0 / 1+ / 1k+ / 5k+ / 10k+ / 25k+ / 50k+ / 100k+ / 200k+
Likewise the user scale has been changed.
The OLD User scale was:
0 / 1+ / 100+ / 200+ / 400+ / 800+ / 1.2k+ / 1.6k+
The NEW User scale is now:
0 / 1+ / 100+ / 200+ / 400+ / 800+ / 1.5k+ / 3k+ / 6k+
These numbers might be tweaked some later this week after the numbers settle back down from the hiccup a few days ago. I wanted to go ahead and make at least some change while I was in the process of other things, hopefully it will get some peoples attention and they will check this main page for updates. There were way too many people "red", due to improvements in the FAH client (like PS3, GPU, and SMP support), along with the general increase in processing power. Numbers in the lower parts of the scale didn't change much for users, I didn't want to discourage the people that only fold on one or two machines. Excluding the inactive users (87% of processed users), when you look at the active people, almost 39% have between 1-99 points 24/hr avg, and almost another 20% have between 100-199 for their 24/hr avg. So rather than up the scale across the board and bump more people into a lower group I simply tweaked the higher numbers for the smaller percentage that are super-competitive.
I am sure many people noticed the stats being down the other day. I was doing a copy of the user history table to a separate archive table, it just took WAY longer than I expected (almost 24 hours to process) and really bogged down the server. Along with that I pruned out all historical data older than Jan 1, 2007 to help speed up the stats processing. I'm going to be pruing out some extra non-essential records and adding back in some data but all that will be processed on a different machine and updated accordingly. The stats will probably go down for an update or two when I make that big update.
|Stanford's Network Is Down...09.13.07, 9:43am CST
Just do you guys don't start emailing me, here is some info you should know about:
9/12/2007 Unplanned network outage
There was a network outage today. Everything in FAH is up and running, except for the web site. We are working to get that back up and expect to have it back up on Thursday morning PST. We have been keep a close eye on the stats and all looks fine there. For those who are curious, here's what happened:
At 11:45 a.m. today, a contractor cut a cable to Forsythe Hall resulting in a disruption of university computer service. There was no disruption to 911 service at either the university or Stanford Hospital. Interim recovery procedures were in effect at 1:45 p.m., according to the Forsythe emergency operations center. Faculty and staff using the network may experiences periodic network problems throughout the day as the university works to repair the damage.
I'm going to take the downtime opportunity today to backup the stats, so they will be unavailable for a little while.
|CSV output updated - Colors to change soon05.21.07, 2:33pm CST
First, I updated the code for the CSV output for teams. It now includes the number of active users and total users (for when there is info). So people can get some historical idea of the people on their team.
Second, since I've been flooded with requests, I'll update the colors to better fit with the higher output that most people are capable of with the GPU, SMP, and PS3 clients. Maybe I should just code in Moore's Law to figure out the levels on an ongoing basis and be done with it...
|XML Back Online05.01.07, 11:46am CST
|The XML feeds are back online. Please read the Data Export page for more info. As mentioned before, you will need to contact me via email (Contact Us page is in the upper right corner) to request access, and I will send you a special URL for you to use.
|Some Changes04.24.07, 2:37pm CST
I made some changes to the sig images yesterday, the change should be transparent to everyone. It was just some code modifications to speed up processing image requests.
The XML feeds are offline at the moment while I make some code changes, I'm sorry I did not get a chance to post an announcement about it sooner. My appologies to all the current XML users.
One change will be multiple results with one query. I've noticed there has been a need for people to query a few users and instead of making them separate requests it can be done all at once.
Another change is I am going to make the XML a "by request only" feature, giving each person (or utility) their own unique URL. This way I can better track usage and also prevent abuse (or broken scripts) without interrupting service to everyone. So if you use the XML code, please send me an email and I will get back to you asap with a URL for you to use.
There hasn't been any issues with the CSV feeds, so those will remain unchanged for the time being.
|Fixed Slight Glitch...04.07.07, 4:33pm CST
There was a slight glitch causing the team processing script to terminate early. I meant to fix it earlier today but I was working on my income tax stuff.
Anyhow, it's fixed now *crosses fingers*. If you see something wrong related to the teams then please drop me a line and I'll look into it further.
|Stats Back To Normal... For The Most Part...03.30.07, 9:01pm CST
It's been a couple days and the stats seem to be back to normal, running fully automated. Yippee!
I know they are running a little slow, and I *think* I know the cause, hopefully within the next day or two I can look more into the issue.
Pages (14): « 1 2 3 4
 6 7 8 9 »
... Last »