Error with import from database

I

ilektro

Member
I have sent a mail to 17000 users yesterday and by coincidence noticed that something weird happened to the import resulting in wrong data for all accounts (email addresses)

When I import, all is ok, so I suppose this happens during sending or when I had to restart because of an internet de-connection.

In the last_name field now appears "Sent" for thousands of addresses (presumably the ones that were restarted), so that what is in the last_name column now is in the "company" column. then a couple of columns are correct and suddenly there is an empty field, shifting up all columns to the right

so I end up with an html email where all those are called "Mr. Sent", and have wrong data, not being able to unsubscribe (as link is wrong) etc :(((

I had downloaded the last (8) version and this is my query

SELECT `mail`, `login`, `sms`, `tel`, `first_name`, `last_name`, `lg`, `gender`, date_format(birthday,'%m-%d-%Y'), `member_fsg`, `cities`, `nbr_rejected`, `sms_rejected`, `id`

FROM `mails`
WHERE `mail` LIKE '%@%'
AND `member`
IN (
'g', 'b', 'p'
)
AND `cities` LIKE '%b%'
AND `nbr_rejected` <6

Unfortunately this makes the program yet another time unusable for me, next to the existing issue with the import of data and the limit of 20 columns. Luckily I used this for a small customer ... I'd have lost my major accounts if I would have sent a newsletter with that much crap data in it.
 
stanbusk

stanbusk

Administrator
Staff member
I don't understand well. Do you mean you saved a list and what you got back was not what you saved????
 
I

ilektro

Member
no, it means that the list changed during the sending and I suppose it happened when there was a deconnection and I had to restart the sending

it is difficult to know when something happens, I only see the end result : emails with wrong data and when I open the saved message I see the wrong data now in the columns. I performed the same import to see if it happens when imported but that is NOT the case so it happened afterwards but not only when opening a message as I didn't close it when sending the mails.
 
stanbusk

stanbusk

Administrator
Staff member
During sending the list is never modified, just the icon is changed. There is no code in MaxBulk Mailer modifying the list during a delivery. That looks like some kind of memory corruption.
 
I

ilektro

Member
I would agree if the word "Send" wasn't added, that is added by the program
It's also hard to believe on a computer that has no issues with any program and is of the top range of Mac (Book Pro)
Has this program been tested with > 15000 mails imported on a database and sent in multiple times?
Is there a way to check that the data keeps consistency?
 
stanbusk

stanbusk

Administrator
Staff member
If you search this forum you will never find any report similar to this (and in any of the 20'000 support ticket I have handled so far). This is the reason why I believe it is some kind of memory corruption. This is extremely rare but the symptom are completely random. As I told you and speaking as the developer of MaxBulk Mailer there is no code in MaxBulk Mailer modifying the list during a delivery except on column 0 to change the icon.
 
I

ilektro

Member
ok, I will keep an eye on it for the next send
in attachment I put a print screen, maybe you see more on that
some have the "sent" icon (from the first delivery, before the internet went down)
some have the "normal" icon but have "Sent" in the last_name
(I blurred the email addresses & phone nbrs)
 
stanbusk

stanbusk

Administrator
Staff member
Ok. I recommend you to download Onyx. It is a tool that can help you fix problems on your system. Go to the 'Automation' panel, select all the options and click on the 'Execute' button. Then restart your computer and check if your problem has been fixed.

If your problem has not been fixed you will have to run the tasks manually. Note that you will be asked to reboot after each task. There is no need to do that, just reboot once you have run all the tasks.

Maintenance panel
- Run the permissions reparation from the first sub-panel by clicking on 'Execute'.
- Select the 'Daily', 'Weekly' and 'Monthly' scripts from the 'Scripts' sub-panel and click on the 'Execute' button.
- Select 'LaunchServices' and 'dyld's from the 'Rebuild' sub-panel and click on the 'Execute' button. (You can also select to rebuild the Spotlight Index).

Cleaning panel
- Check everything from the 'System' sub-panel and click on the 'Execute' button.
- Check everything from the 'User' sub-panel and click on the 'Execute' button.
- Check 'Browser Cache' and 'Download Cache' from the 'Internet' sub-panel and click on the 'Execute' button.
- Check 'System and Users' from the 'Fonts' sub-panel and click on the 'Execute' button.
- Check everything from the 'Logs' sub-panel *BUT* 'Conversations log of instant messaging' and click on the 'Execute' button.
- Check 'Recent Items', 'Obsolete items' and 'Previous 'iTunes Libraries' from the 'Misc' sub-panel and click on the 'Execute' button.

Stan
 
I

ilektro

Member
Thanks, I performed all
As far as I can see, there were no errors but I will monitor and see how it works for a next mail
Hopefully in the meantime the new version will come out with more flexibility to import fields
 
I

ilektro

Member
I again sent out a message to over 10000 and again the data got all mixed up. As a result an entire mail campaign where a particular link contained important info, now contains info from other columns. This time all data looks ok on screen but I have several own addresses in my maillist and see that the links in those mails are wrong. I also get hundreds of error messages that we programmed when someone clicks a wrong link so they all have mails with a wrong link.

this is the result in the mail:
http://www.website.be/attending.php?c=w ... &p=plastic

this is what it should be
http://www.website.be/attending.php?c=w ... 8c11083875

From other info in the mail it is clear that columns have been swapped around so that now I see "Your status : 44362" while this should be "Your status : plastic".

I have a new Mac now. Last time I used another Mac and had the same issue with compiling. I do have quite some internet deconnexions but that should not fail to put the correct info in the email. I'm using version 8-0 US Pro.

This together with the issue I already posted ( viewtopic.php?f=5&t=4630&start=30 ) makes it really unusable for big maillists.

Do you have experience sending > 10000 with internet deconnexions?
Is there a double check when merging the data with the mail?
Did you work on the issue of importing columns from a database?

I hope you can find a solution as I now f*cked up a second mail for a customer, I can't afford a third one.
 
stanbusk

stanbusk

Administrator
Staff member
I have send myself messages to lists up to 100'000 recipients in the last 10 years and I have never seen the columns being mixed as you describe. Are your list files plain text files? What encoding? Not UTF-16 right?

Last month I sent very complex messages to two 50'000 lists with stats activated, not a single problem. Furthermore you can search this forum, nobody has never created a thread on such issue. Are you running an anti-virus on your computer?
 
I

ilektro

Member
I have done another mail with now to "only" 900 subscribers importing from a database and again same issue.
I have an html only message with iso 8859 1 formatting.

On the attachment you can clearly see that the data is all mixed up. This time I had 3 of my own email addresses in the maillist and only 1 shows correct data.

I believe it is not professional to say "never had that". I can say the same: I have worked with Group Mail and Kristianix on Windows and never had it either.

I can imagine that it is very difficult to find and I believe you didn't have it yet. But that doesn't solve the issue. If this is a local issue, then it strangely enough happened on 2 different computers. It is clearly not a problem of amount of mails looking at the last mail. Maybe it is because I use many fields in the mail (18), maybe it is because I upload from a server? In any case, there should be something that detects the integrity of the data as "Sent" seems to be put in the last_name column instead of changing the icon to "Sent" resulting in an email with "Dear David Sent" and as all data shifts up 1 column, all is wrong.
 
I

ilektro

Member
the database :
MyISAM utf8_unicode_ci
8.5 Mb
31711 rows
70 columns
some columns have format latin1_swedish_ci, some utf8_unicode_ci
import 17 with encoding ISO 8859 1
data looks perfect when loaded
only after sending the data shows corrupt

there is ' and " in customers names, hopefully this doesn't cause issues?
 
stanbusk

stanbusk

Administrator
Staff member
Have you tried to download your data with phpmyAdmin and then import it to MaxBulk Mailer?
 
I

ilektro

Member
that would take an extra step which is what I'm trying to avoid as that is an extra risk of loss of data or conversion issues. it could not be managed every time as I need to reload all data at every mail I send as my customers keep the info updated on their servers/databases.

moreover: the problem is that the data looks normal when loaded so I wouldn't be able to see if it works better or not until I send the actual mail out. I'm now hesitating of using it again as I get complaints from my customers (of course).
 
I

ilektro

Member
It's not that I don't want to, it's that I need to work with customer's mails to test it and that I would hope that there is another way eg by checking at every sent line that the date is still correct or by highlighting errors in the data and stopping the sending.

Moreover : I don't know how to export only a part of a database with a query through PhpMyAdmin or which format should I export. PhpMyAdmin also exports wrongly the foreign characters.
 
stanbusk

stanbusk

Administrator
Staff member
I have to investigate the problem but at first glance it looks like memory corruption.

MaxBulk Mailer doesn't use the data from the list directly, it copies it to an array first.
 
You must log in or register to reply here.
Top