Re: Scam Or Not?
 

<font size="2" face="Verdana, Helvetica, sans-serif">Perhaps this is the problem. Now surely this shouldn't be hard to avoid.

Re: Scam Or Not?
 

<font size="2" face="Verdana, Helvetica, sans-serif">Granted; MS could have done much better. The analysis only applies when going back to fix it, not when compiling the list originally. It is easier to maintain a no-duplicates list than it is to change a list to the no-duplicates variety.

Re: Scam Or Not?
 

Best would be an algorithm which checks new addresses against the list before even putting them in!
Keep N small in the first place, and there's less trouble later.
Each submitting machine could keep a filter list of the Last couple submissions so as to cut down on the work the main server has to do.

If you had a sorted list, then the duplicate checking would be really easy.
Decent insertion routines would help a lot too.
Bucket sort to servers holding a piece of the list, then insert using your favorite routine.

Get the n^2 work done as it drips in, so you have years to spend on the problem, instead of rushing it just before trying to send emails.

Re: Scam Or Not?
 

Actually, if you maintain it as a sorted, no-duplicates list from the start, the total computational effort is actually more along the lines of n*lg(n), which isn't unreasonable, even with a billion entries.

Come to think of it, one could always dump the current list into such an algorythm, and then only use ~ 30,000,000,000 comparisons - roughly eight hours, using the same numbers as the earlier analysis for a single machine. Nevermind; MS has no real excuse.