[Jack Simth]

Originally posted by Imperator Fyron:
You do not have to load every single address into active memory at once. In fact, with that loop, each address is deleted from active memory (essentially) after it is checked against the one you are comparing it to.

Apparently you missed the numerous disclaimers:

... goes something like:

rather than "is"; my note at the bottom

Granted, there are several ways to shave time off of the above analysis, but that just gives a general idea of what it would take.

and the

at most

to indicate it was a worst-case (for that algorythm, worst-case = no duplicates) analysis.

In the worst case, the Last entry checked must be checked against every other entry, and so all must be available (in memory, or accessed from the disk). The point was to give a general idea of what was required, not the exact algorythm needed. Things would be further complicated by the likelyhood that it isn't a matter of a single database of addresses being worked with. There are a zillion (exaggeration) assumptions in my analysis, and several valid shortcuts that could be built into the algorythm. It's an estimate to support what I said that DavidG had doubts about, not an exact analysis for that particular number set.