EBH & MLM over SSL

[Anton]

I have been successfully using MaxBulk Mailer with remote lists (MLM) for some time, but I've never gotten EBH to work successfully with the remote lists.
EBH is finding the bounces just fine and the bounces are deleted off the mail server. But MLM is not updated (no bounces are transferred).
As far as I can tell, the remote URL is properly configured (it works just fine in MBM). The only thing I think might have a bearing is that the URL begins https:// - somehow I think I recall that this was an issue I had with remote lists on MBM, but you were able to tell me where to tweak the code to make it work.
Up until this point I had been using EBH in demo mode, but we are looking to dramatically expand the size of our mailing list in the coming months, so I'd really like to get this last issue resolved.
I've now bought a license and I'd be extremely grateful of your support.

Many Thanks - Anton

 

[stanbusk]

eMail Bounce Handler support https MLM connection actually. Do you see any error in the connection log panel?

 

[Anton]

Thanks so much for replying.

There is no error in the logs that I can see. When a bounce is found it says, send to MLM that's all I see. No error.

I am able to access the remote lists via MBM without issue and un-subscriptions & click-through tracking is working as it should. Just no bounces.

Any suggestions as to how I might trouble shoot the issue?

Many thanks - Anton

 

[Anton]

Further to my last, in order that I can go digging, what is the location of the log files written by EBH?

I'd like to look at the entries where it's trying to update MLM, but EBH rolls over it's log files:
[Prior log entries has been saved to file named "Connection Log 2016-03-17.log"]

I've hunted everywhere I can think of but I can seem to locate them.

I've tried ~/Library/Logs, /Library/Logs and /var/log but I don't see them anywhere.

Thanks for your help - Anton

 

[Anton]

Hello again,

I managed to find the log files in ~/Library/Application Support/Maxprog/eMail Bounce Handler/Connection Logs.

Here's an extract from the log file where a bounce has been found (with the email address changed for privacy).

3/15 16:11:21 > LIST 48
3/15 16:11:21 +OK 48 23309
3/15 16:11:21 > RETR 48
3/15 16:11:21 +OK 23309 octets
3/15 16:11:21 ~ Mail Subject: 'Undeliverable: Look out, Look out, Herr Knockoff's about!'
3/15 16:11:21 [x] Extracting address from DSN record (#5.7.1)
3/15 16:11:21 ~ Extracting bounce email addresses from "Soft Delivery not authorized, message refused"
3/15 16:11:21 ~ Duplicate Address: <[email protected]>
3/15 16:11:21 ~ Send <[email protected]> to MLM
3/15 16:11:21 ~ Status: Messages downloaded = 48 of 187
3/15 16:11:21 > DELE 48
3/15 16:11:21 +OK Marked to be deleted.
3/15 16:11:22 ~ Status: Messages marked for deletion = 2 of 187

As you can see, there does not appear to be any error. But nothing is being successfully sent to MLM

Any further ideas, help or information would be gratefully received.

Thanks - Anton

 

[stanbusk]

Ok, have you asked your server support? They have access to the web server and to the PHP processor. Maybe the server doesn't accept the MLM command because of its size...

 

[Anton]

Thanks so much for replying.

It's my own server, so I can configure it to suit my needs without issue. May I ask, exactly what is required, server side to make EBH write bounces to the MySQL database?

MaxBulk Mailer works just fine... Click through tracking and un-subscriptions work perfectly, just no joy with bounces.

I guess you wrote this application, so there is probably nobody on the planet who knows it better than you. Do you offer any kind of paid support option to assist in resolving this issue?

Best - Anton

 

[stanbusk]

Yes, I wrote the application. eMail Bounce Handler uses the MLM settings you have entered in the preferences to send commands to your MLM. Those commands are for adding bounces. You should have access to the Apache and PHP logs. Do you see any error?

 

[Anton]

And a darn fine job you did of it too. I think MBM is an awesome program and worth way more than you charge for a license! If I can get EBH working the way it should with MLM, I'm going to be more than happy.

I'm stumped why MBM works with MLM, but EBH does not when it's the exact same scenario in terms of computers and settings. The only thing I can think of (off the top of my head) is that MBM is reading from MLM but EBH is trying to write. I don't know if EBH tries to write directly to the MySQL database, or this is done via a web/php process. Given that our customers can subscribe and unsubscribe without problem and the fact that click-through tracking works the way it should, it would tend to indicate there must be some difference in the way MBM and EBH access the remote lists (database).

So, I'm happy to go trawling through log files, but to make my job easier, it there any way I can simulate writing a bounce to MLM. Do you have a snippet of php code or similar, so I can test more effectively?

Thanks so much for your kind support, I really do appreciate it.

Best regards - Anton

 

[stanbusk]

Bounces are added thru MLM, not directly. Let me explain how it works. When you send a message with MaxBulk Mailer and MLM activated, bounce tracking info is added to all messages thru the 'X-Bounce-Tracking-Info:' header. Example:

X-Bounce-Tracking-Info: <CQkJaGVucmkubGllYnJlZ3RzQGJldG9uc29uLmNvbQlbTmV3c10gTWF4cHJvZyBuZXdzbGV0dGVyIEFwcmlsIDIwMTUJMzI1MgltYXhwcm9nX2FwcmlsXzIwMTUJMTgzOTA4CWJvdW5jZQlubwlubw==>

When eMail Bounce Handler processes a bounce and find that header it extracts the data and send it to MLM with a command like:

../lm.php?tk=CQkJaGVucmkubGllYnJlZ3RzQGJldG9uc29uLmNvbQlbTmV3c10gTWF4cHJvZyBuZXdzbGV0dGVyIEFwcmlsIDIwMTUJMzI1MgltYXhwcm9nX2FwcmlsXzIwMTUJMTgzOTA4CWJvdW5jZQlubwlubw==

This data is base64 encoded, it contains the recipient name, email address, message subject, list name,...

One problem can be the server not accepting the command, because it is too big? because it contains Base64 encoded data? Anyway it doesn't accept it because a security measure is in place.

 

[Anton]

Thanks so much for your continued support.

So I went digging through the log files. The Base64 string is being passed as part of the HTTP request. There is no error being thrown, as far as I can see.

I've enclosed a couple of excerpts from the log file to demonstrate. The first is from click through tracking which works and is reported in the stats:

24.161.90.43 - - [23/Mar/2016:23:29:10 -0400] "GET /email/lm/lm.php?tk=S2F0aGxlZW4JTydCcmllbgkJa2F0aDQ2ZEBnbWFpbC5jb20JSGV5IFtGaXJzdG5hbWVdLCBkaWQgeW91IHdpbiBhIFJhbmRvbSBBY3Qgb2YgS2luZG5lc3M/CTE5MQlMb2NhbCBNYWlsaW5nIExpc3QJNTM1NAljbGljawl5ZXMJbm8=&url=https%3A%2F%2Fyoutu.be%2F4DBLONXmTQ0 HTTP/1.1" 302 - "-" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36"

The next excerpt is a bounce sent from EBH. It appears that the request is formatted correctly and the Base64 string decodes OK, but nothing is being written to MLM

184.152.93.198 - - [15/Mar/2016:16:12:02 -0400] "GET /email/lm/lm.php?tk=SmFuZXQJTGVvbgkJamxlb25AdWljLmVkdQlIZXkgW0ZpcnN0bmFtZV0sIGRpZCB5b3Ugd2luIGEgUmFuZG9tIEFjdCBvZiBLaW5kbmVzcz8JMTkxCVVTQSBNYWlsaW5nIExpc3QJNTM1NAlib3VuY2UJbm8Jbm8= HTTP/1.0" 200 - "-" "-"

As far as I can tell, there is no issue with the server - at least from the log files. Message opening, click-thru tracking and unsubscribe all work as they should, but the bounce http request from EBH seems to fall upon deaf ears.

I really need to get this working, so if you can throw me any ideas, I'd be more than grateful. This is proving to be quite tricky to solve.. but, in the words of Sherlock Holmes, "when you eliminate the impossible, whatever remains (no matter how improbable) must be the truth".

Thanks again for the continued support.

Anton

 

[stanbusk]

Ok. What if you go to /email/lm/lm.php?tk=SmFuZXQJTGVvbgkJamxlb25AdWljLmVkdQlIZXkgW0ZpcnN0bmFtZV0sIGRpZCB5b3Ugd2luIGEgUmFuZG9tIEFjdCBvZiBLaW5kbmVzcz8JMTkxCVVTQSBNYWlsaW5nIExpc3QJNTM1NAlib3VuY2UJbm8Jbm8= with your browser? I can't do it myself because I don't know the full lm.php url on your server. Do you get any error?

 

[Anton]

Hello again.

I tried what you ask. No error reported in the browser. Just white screen (No content) which is as expected. No error thrown in the logs. But no bounce reported either.

Here's what's written in the log file:

184.152.93.198 - - [25/Mar/2016:12:59:40 -0400] "GET /email/lm/lm.php?tk=SmFuZXQJTGVvbgkJamxlb25AdWljLmVkdQlIZXkgW0ZpcnN0bmFtZV0sIGRpZCB5b3Ugd2luIGEgUmFuZG9tIEFjdCBvZiBLaW5kbmVzcz8JMTkxCVVTQSBNYWlsaW5nIExpc3QJNTM1NAlib3VuY2UJbm8Jbm8= HTTP/1.1" 200 - "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/601.5.17 (KHTML, like Gecko) Version/9.1 Safari/601.5.17"

The full URL of the server is https://secure.awarenessshop.com/ (on port 443), I've also temporarily set up insecure access on port 80 for testing. That URL is http://mailroom.awarenessshop.com/

As always, I very much appreciate your help.

Best - Anton

 

[Anton]

Further to my last, I'm still pondering ideas as to how to fix this issue.

Seemingly, there is no issue with the web/php side of things (No errors, no clues in log files).

So, given that my MLM backend is a MySQL database, is there any way I can directly simulate writing a bounce to the db. I've looked at the Schema of the db in MySQL Workbench, it consists of 4 tables (tbl_mlm_sub, tbl_mlm_sub_clk, tbl_mlm_sub_clk_test and tbl_mlm_sub_loc) it does not appear immediately clear where a bounce record would be written.

Is it possible you could give me an example of the MySQL query that is generated by the bounce event sent to MLM by EBH.

Thanks in anticipation - Anton

 

[stanbusk]

You already have those examples, all the code I sent are MLM commands to add bounces:

/email/lm/lm.php?tk=SmFuZXQJTGVvbgkJamxlb25AdWljLmVkdQlIZXkgW0ZpcnN0bmFtZV0sIGRpZCB5b3Ugd2luIGEgUmFuZG9tIEFjdCBvZiBLaW5kbmVzcz8JMTkxCVVTQSBNYWlsaW5nIExpc3QJNTM1NAlib3VuY2UJbm8Jbm8=

See the 'tk=' this is for adding a bounce. MLM decode the BASE64 data and then add the bounce.

Indeed the page should be blank but you can activate MLM debugging from the 'lm_prefs.php' file. Replace:

$mlm_debug           = false;

with

$mlm_debug           = true;

. Then try to add the bounce again with the URL above. Anything?

 

[Anton]

As a matter of fact I see an error...

"Notice: Undefined index: url in /path/to/file/email/lm/lm.php on line 90"

The path to file is actually my server's directory structure, which I don't want to post here

Hopefully this helps?

Best - Anton

 

[Anton]

And looking at line 90 in lm.php, it reads:

        $url   = $_REQUEST['url'];

Hope this helps - Anton

 

[stanbusk]

You could try to change

$url   = $_REQUEST['url'];

with

$url   = ".../lm.php";

where ... is the URL of lm.php on your server.

Does this fix the problem?

 

[Anton]

Hi Stanbusk,

Thanks again for trying to help me. Unfortunately, that didn't fix the problem.

The 'url' undefined error notice went away (just see a blank screen on the browser with debug turned on), but still no bounce is being written to MLM

Trawling through the lm.php code is see that unsubscriptions are handled by 'un=base64string' and click through tracking by 'tk=base64string'. Looking at the base 64 string we're currently using to test, the prefix is tk=. Given that click through tracking is working fine, but bounce handling is not, there has to be a difference in the way MLM (or the php code talking to it) handles writing the record to the MySQL database.

So, what is that difference? Where is the point of failure?

I really need to get this sorted, so anything you can do to help would be gratefully appreciated.

Best - Anton

 

[Anton]

Hi Stanbusk,

You suggestion to change the line in lm.php to the hard-coded URL, actually had a nasty consequence. It completely broke click thru. Instead of a person being directed to the link they expected to view, they were taken to the log in page of MLM. Not good.

Needless to say, I changed it back.

Best - Anton