STDIN File Handle

The dump() and rise() methods of the Sisimai class read bounce email data from standard input and return the decoded results when the first argument is STDIN as the file handle.

% cat ./path/to/email | perl -MSisimai -lE 'print Sisimai->dump(STDIN)' | jq
[
  {
    "listid": "",
    "smtpagent": "Sendmail",
    ...
                

% cat ./path/to/email | perl -MSisimai -lE 'print $_->recipient->address for @{ Sisimai->rise(STDIN) }'
kijitora@example.jp
sironeko@example.org
                

STDIN Object

The Ruby version of Sisimai also works similarly. When the first argument of the dump() and rise() methods of the Sisimai class is the STDIN object, it reads bounce email data from standard input and returns the decoded results.

% cat ./path/to/email | ruby -Ilib -rsisimai -e 'puts Sisimai.dump(STDIN)' | jq
[
  {
    "catch": "",
    "token": "d295b517a3ad6f8748031e1068e07ebd089c0277",
    ...
                

% cat ./path/to/email | ruby -rsisimai -e 'p Sisimai.rise(STDIN)'
[#<Sisimai::Fact:0x007fa1b0993050 @addresser=#<Sisimai::Address:0x007fa1b0990e40 @user="kijitora", ...
                

Succeeded Delivery as a Sample

Emails that were successfully delivered are available as samples in the github.com/sisimai/set-of-emails repository. Specifically, the rfc3464-28.eml can be used as an example. This section provides sample code for decoding this email.

delivered => 1

The second argument of rise() and dump() takes a delivered key-value pair. When set to 1, the decoded results will include emails that were successfully delivered (delivery notifications). If delivered is set to a false value (0, undef, or an empty string), the decoded results will not be generated.

#! /usr/bin/env perl
use Sisimai;
my $f = './set-of-emails/maildir/bsd/rfc3464-28.eml';
my $v = Sisimai->rise($f, 'delivered' => 1);
for my $e ( @$v ) {
    print $e->action;           # "delivered"
    print $e->deliverystatus;   # "2.1.5"
    print $e->reason;           # "delivered"
    print $e->diagnosticcode;   # "250 2.1.5 Ok"
    print $e->hardbounce;       # 0
}
                

delivered: true

Similarly to the Perl version, in the Ruby version of Sisimai, specifying delivered: true as the second argument of rise() and dump() will include emails that were successfully delivered (delivery notifications) in the decoded results. If a false value (nil or false) is specified for delivered, the decoded results will not be generated.

#! /usr/bin/env ruby
require 'sisimai'
f = './set-of-emails/maildir/bsd/rfc3464-28.eml'
v = Sisimai.rise(f, delivered: true)
v.each do |e|
  puts e.action         # "delivered"
  puts e.deliverystatus # "2.1.5"
  puts e.reason         # "delivered"
  puts e.diagnosticcode # "250 2.1.5 Ok"
  puts e.hardbounce     # false
end
                

Arguments passed to c___

The arguments that can be passed to the c___ (c and three underscores, looks like a fishing hook) in the rise() method of the Sisimai class are one array reference for Perl hook methods (subroutines) and one Array object for Ruby hook methods. The structure of the arguments is as follows:

my $code0 = sub {
    my $args = shift;
    my $head = $args->{'headers'}; # Email headers
    my $body = $args->{'message'}; # Message body
};
my $code1 = sub {
    my $args = shift;
    my $kind = $args->{'kind'}; # (String)  Sisimai::Mail->kind
    my $mail = $args->{'mail'}; # (*String) Entire email message
    my $path = $args->{'path'}; # (String)  Sisimai::Mail->path
    my $fact = $args->{'fact'}; # (*Array)  List of Sisimai::Fact
};
my $list = Sisimai->rise('/path/to/mbox', 'c___' => [$code0, $code1]);
                

Arguments inside of c___[0]

The string obtained from the message is the body of the bounce email. It does not include the headers of the bounce email itself, so please refer to the hashed headers.

Arguments inside of c___[1]

The string obtained from the mail is the entire bounce email, including the headers of the bounce email itself.

Email: rhost-google-03.eml

The following code snippet demonstrates how to operate the original email by obtaining the contents listed below from rhost-google-03.eml via a callback.

• Retrieve the value of Return-Path in the bounce email header.
• Retrieve the value of Authentication-Results in the bounce email header.
• Retrieve the value of X-Postfix-Queue-ID in the body.
• Add a custom header to the original email and save it to a different path.

Return-Path: <>
Received: from mail4.example.com (mx4.example.com [192.0.2.24])
	by secure.example.jp (Postfix) with ESMTPS id 4SnNbh0r3mz24sXs
	for ; Thu,  9 Apr 2022 23:34:45 +0900 (JST)
Received: by mail4.example.com (Postfix)
	id t8WLKlmm99zqL71C; Thu,  9 Apr 2022 23:34:45 +0900 (JST)
Authentication-Results: secure.example.jp; spf=fail smtp.helo=mail4.example.com
Date: Thu,  9 Apr 2022 23:34:45 +0900 (JST)
From: MAILER-DAEMON@mail4.example.com (Mail Delivery System)
Subject: Undelivered Mail Returned to Sender
To: nginx@mail4.example.com
...
 (expanded from ): host
    gmail-smtp-in.l.google.com[64.233.188.26] said: 550-5.7.26 The MAIL FROM
    domain [mail4.example.com] has an SPF record with a 550-5.7.26 hard fail
    policy (-all) but it fails to pass SPF checks with the ip: 550-5.7.26
    [192.0.2.24]. To best protect our users from spam and phishing,
    550-5.7.26 the message has been blocked. For instructions on setting up
    550-5.7.26 authentication, go to 550 5.7.26
    https://support.google.com/mail/answer/81126#authentication
    2-222222630d46222222b002b2d2neko2cat2222222cat.22 - gsmtp (in reply to end
    of DATA command)

--Lms5yfCw0Vz5CRp3.1647426971/mail4.example.com
Content-Description: Delivery report
Content-Type: message/delivery-status

Reporting-MTA: dns; mail4.example.com
X-Postfix-Queue-ID: Lms5yfCw0Vz5CRp3
X-Postfix-Sender: rfc822; nginx@mail4.example.com
Arrival-Date: Thu,  9 Apr 2022 23:34:45 +0900 (JST)

Final-Recipient: rfc822; kijitora@google.example.com
Original-Recipient: rfc822;neko@example.co.jp
Action: failed
Status: 5.7.26
Remote-MTA: dns; gmail-smtp-in.l.google.com
Diagnostic-Code: smtp; 550-5.7.26 The MAIL FROM domain [mail4.example.com] has
...
                

Sisimai->rise($f, 'c___' => [$c0, $c1])

You can pass two code references as the second argument c___ to the rise() and dump() methods of the Sisimai class. These code references should contain the code for the subroutine that you want to execute.

#! /usr/bin/env perl
use Sisimai;
my $fn = 'set-of-emails/maildir/bsd/rhost-google-03.eml';
my $c0 = sub {
    my $args = shift;
    my $data = { 'queue-id' => '', 'authentication-results' => '', 'return-path' => '' };

    $data->{'return-path'} = $args->{'headers'}->{'return-path'} || '';
    $data->{'authentication-results'} = $args->{'headers'}->{'authentication-results'} || '';
    if( $args->{'message'} =~ m/^X-Postfix-Queue-ID:\s*(.+)$/m ) {
        $data->{'queue-id'} = $1;
    }
    return $data;
};
my $c1 = sub {
    my $args = shift;
    my $kind = $args->{'kind'}; # (String)  Sisimai::Mail->kind
    my $mail = $args->{'mail'}; # (*String) Entire email message
    my $path = $args->{'path'}; # (String)  Sisimai::Mail->path
    my $fact = $args->{'fact'}; # (*Array)  List of Sisimai::Fact

    for my $e ( @$fact ) {
        # Store custom information in the "catch" accessor.
        $e->{'catch'} ||= {};
        $e->{'catch'}->{'size'} = length $$mail;
        $e->{'catch'}->{'kind'} = ucfirst $kind;

        # Save the original email with an additional "X-Sisimai-Parsed:" header to a different PATH.
        my $a = sprintf("X-Sisimai-Parsed: %d\n", scalar @$fact);
        my $f = IO::File->new(sprintf("/tmp/sisimai-%s.eml", $e->token), 'w');
        my $v = $$mail; $v =~ s/^(From:.+)$/$a$1/m;
        print $f $v; $f->close;
    }

    # Remove the email file in Maildir/ after decoding
    unlink $path if $kind eq 'maildir';
};

my $js = Sisimai->dump($fn, 'c___' => [$c0, $c1]);
my $rv = Sisimai->rise($fn, 'c___' => [$c0, $c1]);
printf("Authentication-Results: %s\n", $rv->[0]->{'catch'}->{'authentication-results'});
printf("X-Postfix-Queue-ID: %s\n", $rv->[0]->{'catch'}->{'queue-id'});
printf("Return-Path: %s\n", $rv->[0]->{'catch'}->{'return-path'});
                

Sisimai.rise(f, c___: [c0, c1])

You can pass a Proc object containing the code for the function that you want to execute to the rise() and dump() methods of the Sisimai class as the second argument c___.

#! /usr/bin/env ruby
require 'sisimai'
fn = 'set-of-emails/maildir/bsd/rhost-google-03.eml'
c0 = lambda do |args|
  data = { 'queue-id' => '', 'authentication-results' => '', 'return-path' => '' }
  data['return-path'] = args['headers']['return-path'] || ''
  data['authentication-results'] = args['headers']['authentication-results'] || ''
  if cv = args['message'].match(/^X-Postfix-Queue-ID:\s*(.+)$/)
    data['queue-id'] = cv[1]
  end
  return data
end

c1 = lambda do |args|
  kind = args['kind'] # (String)  Sisimai::Mail.kind
  mail = args['mail'] # (*String) Entire email message
  path = args['path'] # (String)  Sisimai::Mail.path
  fact = args['fact'] # (*Array)  List of Sisimai::Fact

  fact.each do |e|
    # Store custom information in the "catch" accessor.
    e.catch ||= {}
    e.catch['size'] = mail.size
    e.catch['kind'] = kind

    # Save the original email with an additional "X-Sisimai-Parsed:" header to a different PATH.
    a = sprintf("X-Sisimai-Parsed: %d", fact.size)
    b = mail.sub(/^(To:.+$)/,'\1'+ "\n" + a)
    File.open(sprintf("/tmp/sisimai-%s.eml", e.token), "w") do |f|
      f.puts(b)
    end

    # Remove the email file in Maildir/ after decoding
    File.delete(path) if kind == 'maildir'
  end
end

js = Sisimai.dump(fn, c___: [c0, c1]) # JSON文字列を得る
rv = Sisimai.rise(fn, c___: [c0, c1])
printf("Authentication-Results: %s\n", rv[0].catch['authentication-results'])
printf("X-Postfix-Queue-ID: %s\n", rv[0].catch['queue-id'])
printf("Return-Path: %s\n", rv[0].catch['return-path'])
                

JSON string by dump()

The JSON string obtained by passing a hook method to the dump() method of the Sisimai class will have the following contents:

[
  {
    "addresser": "kijitora@google.example.com",
    "messageid": "vDcsF1hr4zzywp69@mail4.example.com",
    "recipient": "kijitora@google.example.com",
    "diagnosticcode": "host gmail-smtp-in.l.google.com[64.233.188.26] said: The MAIL FROM domain [mail4.example.com] has an SPF record with a hard fail policy (-all) but it fails to pass SPF checks with the ip: [192.0.2.24]. To best protect our users from spam and phishing, the message has been blocked. For instructions on setting up authentication, go to https://support.google.com/mail/answer/81126#authentication 2-222222630d46222222b002b2d2neko2cat2222222cat.22 - gsmtp (in reply to end of DATA command)",
    "smtpagent": "Postfix",
    "destination": "google.example.com",
    "alias": "neko@example.co.jp",
    "reason": "authfailure",
    "lhost": "gmail-smtp-in.l.google.com",
    "senderdomain": "google.example.com",
    "token": "fe75b94dd0489b106fa041c460064dd6afa8196b",
    "subject": "Nyaan",
    "deliverystatus": "5.7.26",
    "timezoneoffset": "+0900",
    "listid": "",
    "origin": "set-of-emails/maildir/bsd/rhost-google-03.eml",
    "action": "failed",
    "timestamp": 1649514885,
    "hardbounce": 0,
    "rhost": "gmail-smtp-in.l.google.com",
    "catch": {
      "size": 4436,
      "kind": "Mailbox",
      "authentication-results": "secure.example.jp; spf=fail smtp.helo=mail4.example.com",
      "return-path": "<>",
      "queue-id": "Lms5yfCw0Vz5CRp3"
    },
    "diagnostictype": "SMTP",
    "replycode": 550,
    "feedbacktype": "",
    "smtpcommand": "DATA"
  }
]