MBM crashed sending, crashes recovering

[fabprog]

MBM ver. 8.4.6 Mac, OS X 10.9.1
Sending about 46,000 recipients, crashed after about 18.000.
No way to recover: when launched, MBM starts and ask for restoring sending, it seems to load the document correctly, but keeps freezing while trying to restore recipients send statuses from log file.

Update: after tracing myself the send log and restart with remaining recipients, MBM crashed again after about 2h30' of sending (3 concurrent SMTP connections, single, 50 email per group, 2" pause between connections, Amazon SES smtp server).
Analyzing the send log, I discovered that threads [1] and [3] completed much sooner than thread [2], which was the only one taking much longer and still active at the moment MBM crashed.

 

[stanbusk]

MBM ver. 8.4.6 Mac, OS X 10.9.1
Sending about 46,000 recipients, crashed after about 18.000.
No way to recover: when launched, MBM starts and ask for restoring sending, it seems to load the document correctly, but keeps freezing while trying to restore recipients send statuses from log file.

It is not freezing but processing the crash log. That can take some time since that in your case MaxBulk Mailer has to process 828 millions entries. We are working on enhancing that but it will never be as fast as we would like.

Update: after tracing myself the send log and restart with remaining recipients, MBM crashed again after about 2h30' of sending (3 concurrent SMTP connections, single, 50 email per group, 2" pause between connections, Amazon SES smtp server).
Analyzing the send log, I discovered that threads [1] and [3] completed much sooner than thread [2], which was the only one taking much longer and still active at the moment MBM crashed.

Have you deactivated App Nap for MaxBulk Mailer?

 

[fabprog]

It is not freezing but processing the crash log.

I suspected it could be. But after about twenty minute I decided to kill it and relaunch.
I understood (not difficult) how to detect the three last sent email recipients for the three concurrent send threads. May be I lost track of some unsent addresses, but couldn't wait more (also, I was not sure whether MBM was stuck or working...).
I think there are chances you'll be able to widely improve restoring time. Reading log file was straightforward to me and I can't tell why automated restoring takes so long.

MaxBulk Mailer has to process 828 millions entries.

Connection log file was about 14MB in size, 284,369 lines long. Which looks perfect assuming an average of about 15 lines per recipient (about 18,000 sent emails before crash x about 15 lines = 270,000). It's strange, scanning shouldn't be that heavy.

Have you deactivated App Nap for MaxBulk Mailer?

Yes, definitely.

 

[stanbusk]

Next MaxBulk Mailer version will boost crash recovery, I found how to do it much more eficiently

 

[fabprog]

Well since it happened again I had to restore after about 15.000 emails sent on 20.000 total, this time I decided to give it a try and let it go and finally it succeeded: but restoring took about half a hour!! We look further for this improvement...

 

[stanbusk]

With next version I made a test with a 60k file and a 40k recovery file, it now takes less than 1 minute.