IMPORTANT ANNOUNCEMENT REGARDING PMs!

[douglas]

I notice you mentioned in the Dom 3 forum that about 100000 PMs were taking longer than everything else put together. Considering there are over 5 times that many posts alone, plus the user list and various other data on top of that, I can't think of any reasonable explanation for it other than an index or constraint requiring excessive redundant checking.

My first guess would be that the PM content column has a unique constraint but no index, which would force a string comparison of each and every PM with each and every other PM. This isn't that big a deal in normal use when you only have to worry about one new PM at a time, but doing 100k at once takes forever. Removing the constraint in this case would drop the import time immensely. Creating an index for it instead would greatly reduce the number of comparisons required, but the index itself would take some extra resources to maintain. Assuming my guess is correct, I would first try creating an index for the column and see how much that helps. If it still takes too long, then remove the constraint entirely before importing and then see if it can be added back in a reasonable amount of time.

Edit: Then again, I don't know that much about how most databases handle indexes on strings specifically, so it's possible that it does have an index on PM content and that removing that index could help. I doubt this is the case, but it might be possible.

[Mindi]

Like I said, I don't know all the finer points of this sort of thing. I do know what Richard did to speed up the main part of the import was to take off the indexing, but I don't think the PMs work the same way and we still haven't figured out what the total time will be once we put the index back on at the end because we haven't had a complete, successful import to try it on yet since the PM part failed last time.

I don't really like talking about all of this because it's not my area of expertise and I only know a little bit about everything Richard is doing. He's the back end, I'm the one putting in modifications, tweaking settings, importing smilies, etc.

[gregebowman]

For someone who's still has one foot in the 20th century, what the heck is a PM?

[Ed Kolis]

Private message

[Fyron]

"Private Messages" existed for much of the latter decade of the 20th century.

[narf poit chez BOOM]

...So less than 10%?

[Azselendor]

Oh, you are all very very wrong.

Private Messages have existed for as long as there has been children passing private messages in class. This of course was always accompanied by the Department of Homeland Security - formerly known as Teachers - whom would intercept and read them randomly and to your embarrassment.

[Fyron]

Didn't say they didn't exist before the 90s! They were just horribly inefficient, dependent upon the sneakernet.