STDIN File Handle

The dump() and rise() methods of the Sisimai class read bounce email data from standard input and return the decoded results when the first argument is STDIN as the file handle.

% cat ./path/to/email | perl -MSisimai -lE 'print Sisimai->dump(STDIN)' | jq
[
  {
    "listid": "",
    "smtpagent": "Sendmail",
    ...
                

% cat ./path/to/email | perl -MSisimai -lE 'print $_->recipient->address for @{ Sisimai->rise(STDIN) }'
kijitora@example.jp
sironeko@example.org
                

STDIN Object

The Ruby version of Sisimai also works similarly. When the first argument of the dump() and rise() methods of the Sisimai class is the STDIN object, it reads bounce email data from standard input and returns the decoded results.

% cat ./path/to/email | ruby -Ilib -rsisimai -e 'puts Sisimai.dump(STDIN)' | jq
[
  {
    "catch": "",
    "token": "d295b517a3ad6f8748031e1068e07ebd089c0277",
    ...
                

% cat ./path/to/email | ruby -rsisimai -e 'p Sisimai.rise(STDIN)'
[#<Sisimai::Fact:0x007fa1b0993050 @addresser=#<Sisimai::Address:0x007fa1b0990e40 @user="kijitora", ...
                

string "STDIN"

The Dump() and Rise() functions of the libsisimai.org/sisimai package read bounce email data from standard input and return the decoded results when the first argument is STDIN as a string.

package main
import "fmt"
import "libsisimai.org/sisimai/v5"

func main() {
    args := sisimai.Args()
    json, _ := sisimai.Dump("STDIN", args)
    if json != nil && *json != "" { fmt.Printf("%s\n", *json) }
}
                

% cat ./path/to/email | go run ./sisid.go | jq
[
  {
    "catch": "",
    "token": "d295b517a3ad6f8748031e1068e07ebd089c0277",
    ...
                

Succeeded Delivery as a Sample

Emails that were successfully delivered are available as samples in the github.com/sisimai/set-of-emails repository. Specifically, the rfc3464-28.eml can be used as an example. This section provides sample code for decoding this email.

delivered => 1

The second argument of rise() and dump() takes a delivered key-value pair. When set to 1, the decoded results will include emails that were successfully delivered (delivery notifications). If delivered is set to a false value (0, undef, or an empty string), the decoded results will not be generated.

#! /usr/bin/env perl
use Sisimai;
my $f = './set-of-emails/maildir/bsd/rfc3464-28.eml';
my $v = Sisimai->rise($f, 'delivered' => 1);
for my $e ( @$v ) {
    print $e->action;           # "delivered"
    print $e->deliverystatus;   # "2.1.5"
    print $e->reason;           # "delivered"
    print $e->diagnosticcode;   # "250 2.1.5 Ok"
    print $e->hardbounce;       # 0
}
                

delivered: true

Similarly to the Perl version, in the Ruby version of Sisimai, specifying delivered: true as the second argument of rise() and dump() will include emails that were successfully delivered (delivery notifications) in the decoded results. If a false value (nil or false) is specified for delivered, the decoded results will not be generated.

#! /usr/bin/env ruby
require 'sisimai'
f = './set-of-emails/maildir/bsd/rfc3464-28.eml'
v = Sisimai.rise(f, delivered: true)
v.each do |e|
  puts e.action         # "delivered"
  puts e.deliverystatus # "2.1.5"
  puts e.reason         # "delivered"
  puts e.diagnosticcode # "250 2.1.5 Ok"
  puts e.hardbounce     # false
end
                

args.Delivered = true

In the Go version of Sisimai, as with the Perl and Ruby versions, if you specify true for Delivered in the sis.DecodingArgs struct, which is specified as the second argument to Rise() and Dump(), the decoded results will include successfully delivered emails (delivery notifications). The initial value of Delivered is false.

Since Sisimai 5.4.0, Rise() function return slices instead of pointers

package main
import "fmt"
import "libsisimai.org/sisimai/v5"

func main() {
    path := "./set-of-emails/maildir/bsd/rfc3464-28.eml"
    args := sisimai.Args()
    args.Delivered = true

    sisi, _ := sisimai.Rise(path, args)
    if len(sisi) > 0 { fmt.Printf("Recipient = %s\n", sisi[0].Recipient.Address }
}
                

Arguments passed to c___ (Perl and Ruby)

The arguments that can be passed to the c___ (c and three underscores, looks like a fishing hook) in the rise() method of the Sisimai class are one array reference for Perl hook methods (subroutines) and one Array object for Ruby hook methods. The structure of the arguments is as follows:

my $code0 = sub {
    my $args = shift;
    my $head = $args->{'headers'}; # Email headers
    my $body = $args->{'message'}; # Message body
};
my $code1 = sub {
    my $args = shift;
    my $kind = $args->{'kind'}; # (String)  Sisimai::Mail->kind
    my $mail = $args->{'mail'}; # (*String) Entire email message
    my $path = $args->{'path'}; # (String)  Sisimai::Mail->path
    my $fact = $args->{'fact'}; # (*Array)  List of Sisimai::Fact
};
my $list = Sisimai->rise('/path/to/mbox', 'c___' => [$code0, $code1]);
                

Arguments inside of c___[0]

The string obtained from the message is the body of the bounce email. It does not include the headers of the bounce email itself, so please refer to the hashed headers.

Arguments inside of c___[1]

The string obtained from the mail is the entire bounce email, including the headers of the bounce email itself.

Arguments for Callback0, Callback1 (Go)

The arguments and return values of Callback0 and Callback1 in the sis.DecodingArgs struct in the libsisimai.org/sisimai package's Rise() function have the following structure:

args := sisimai.Args()
args.Callback0 = func(arg *sisimai.CallbackArg0) (map[string]interface{}, error) {
    fmt.Printf("Headers = %v\n", (*arg).Headers) // All the headers
    fmt.Printf("- From: = %s\n", (*arg).Headers["from"][0])
    fmt.Printf("- Subject: = %s\n", (*arg).Headers["subject"][0])
    fmt.Printf("Body = %s\n", (*arg).Payload)    // Message body
    data := make(map[string]interface{})
    data["from"] = (*arg).Headers["from"][0]
    return data, nil
}
args.Callback1 = func(arg *sisimai.CallbackArg1) (bool, error) {
    fmt.Printf("- Path = %s\n", arg.Path)  // Path to the bounce mail file
    fmt.Printf("- Kind = %s\n", arg.Kind)  // Kind of the email
    fmt.Printf("Body = %s\n", (*arg.Mail)) // Message body
    fmt.Printf("Body = %s\n", (*arg.Fact)) // Decoded results
    return true, nil
}

sisi, nyaan := sisimai.Rise(path, args)
                

Email: rhost-google-03.eml

The following code snippet demonstrates how to operate the original email by obtaining the contents listed below from rhost-google-03.eml via a callback.

• Retrieve the value of Return-Path in the bounce email header.
• Retrieve the value of From in the bounce email header.
• Retrieve the value of X-Postfix-Queue-ID in the body.
• Save the original email to a different path when it input from STDIN.

Return-Path: <>
Received: from mail4.example.com (mx4.example.com [192.0.2.24])
	by secure.example.jp (Postfix) with ESMTPS id 4SnNbh0r3mz24sXs
	for ; Thu,  9 Apr 2022 23:34:45 +0900 (JST)
Received: by mail4.example.com (Postfix)
	id t8WLKlmm99zqL71C; Thu,  9 Apr 2022 23:34:45 +0900 (JST)
Authentication-Results: secure.example.jp; spf=fail smtp.helo=mail4.example.com
Date: Thu,  9 Apr 2022 23:34:45 +0900 (JST)
From: MAILER-DAEMON@mail4.example.com (Mail Delivery System)
Subject: Undelivered Mail Returned to Sender
To: nginx@mail4.example.com
...
 (expanded from ): host
    gmail-smtp-in.l.google.com[64.233.188.26] said: 550-5.7.26 The MAIL FROM
    domain [mail4.example.com] has an SPF record with a 550-5.7.26 hard fail
    policy (-all) but it fails to pass SPF checks with the ip: 550-5.7.26
    [192.0.2.24]. To best protect our users from spam and phishing,
    550-5.7.26 the message has been blocked. For instructions on setting up
    550-5.7.26 authentication, go to 550 5.7.26
    https://support.google.com/mail/answer/81126#authentication
    2-222222630d46222222b002b2d2neko2cat2222222cat.22 - gsmtp (in reply to end
    of DATA command)

--Lms5yfCw0Vz5CRp3.1647426971/mail4.example.com
Content-Description: Delivery report
Content-Type: message/delivery-status

Reporting-MTA: dns; mail4.example.com
X-Postfix-Queue-ID: Lms5yfCw0Vz5CRp3
X-Postfix-Sender: rfc822; nginx@mail4.example.com
Arrival-Date: Thu,  9 Apr 2022 23:34:45 +0900 (JST)

Final-Recipient: rfc822; kijitora@google.example.com
Original-Recipient: rfc822;neko@example.co.jp
Action: failed
Status: 5.7.26
Remote-MTA: dns; gmail-smtp-in.l.google.com
Diagnostic-Code: smtp; 550-5.7.26 The MAIL FROM domain [mail4.example.com] has
...
                

Sisimai->rise($f, 'c___' => [$c0, $c1])

You can pass two code references as the second argument c___ to the rise() and dump() methods of the Sisimai class. These code references should contain the code for the subroutine that you want to execute.

% cat set-of-emails/maildir/bsd/rhost-google-03.eml | perl ./sisid.pl

#! /usr/bin/env perl
# sisid.pl
use Sisimai;
my $c0 = sub {
    my $args = shift;
    my $data = { 'queue-id' => '', 'from' => '', 'return-path' => '' };

    $data->{'return-path'} = $args->{'headers'}->{'return-path'} || '';
    $data->{'from'} = $args->{'headers'}->{'from'} || '';
    if( $args->{'message'} =~ m/^X-Postfix-Queue-ID:\s*(.+)$/m ) {
        $data->{'queue-id'} = $1;
    }
    return $data;
};
my $c1 = sub {
    my $args = shift;
    my $kind = $args->{'kind'}; # (String)  Sisimai::Mail->kind
    my $mail = $args->{'mail'}; # (*String) Entire email message
    my $path = $args->{'path'}; # (String)  Sisimai::Mail->path

    if $kind eq 'stdin' {
        my $f = IO::File->new("/tmp/sisimai-decoded-1.eml", 'w');
        print $f $$mail; $f->close;
    }
};

my $rv = Sisimai->rise(STDIN, 'c___' => [$c0, $c1]);
printf("From: %s\n", $rv->[0]->{'catch'}->{'from'});
printf("Queue-ID: %s\n", $rv->[0]->{'catch'}->{'queue-id'});
printf("Return-Path: %s\n", $rv->[0]->{'catch'}->{'return-path'});
                

Sisimai.rise(f, c___: [c0, c1])

You can pass a Proc object containing the code for the function that you want to execute to the rise() and dump() methods of the Sisimai class as the second argument c___.

% cat set-of-emails/maildir/bsd/rhost-google-03.eml | ruby ./sisid.rb

#! /usr/bin/env ruby
# sisid.rb
require 'sisimai'
c0 = lambda do |args|
  data = { 'queue-id' => '', 'from' => '', 'return-path' => '' }
  data['return-path'] = args['headers']['return-path'] || ''
  data['from'] = args['headers']['from'] || ''
  if cv = args['message'].match(/^X-Postfix-Queue-ID:\s*(.+)$/)
    data['queue-id'] = cv[1]
  end
  return data
end

c1 = lambda do |args|
  kind = args['kind'] # (String)  Sisimai::Mail.kind
  mail = args['mail'] # (*String) Entire email message
  path = args['path'] # (String)  Sisimai::Mail.path

  if kind == 'stdin'
    File.open("/tmp/sisimai-decoded-1.eml", "w") do |f|
      f.puts(mail)
    end
  end
end

rv = Sisimai.rise(STDIN, c___: [c0, c1])
printf("From: %s\n", rv[0].catch['from'])
printf("X-Postfix-Queue-ID: %s\n", rv[0].catch['queue-id'])
printf("Return-Path: %s\n", rv[0].catch['return-path'])
                

sisimai.Rise(path, args)

You can assign and pass functions containing the processing you want to execute to Callback0 and Callback1 of the sis.DecodingArgs struct, which is passed to the Rise() and Dump() functions of the libsisimai.org/sisimai package.

Since Sisimai 5.4.0, Rise() function return slices instead of pointers

% cat set-of-emails/maildir/bsd/rhost-google-03.eml | go run ./sisid.go

package main
// sisid.go
import "fmt"
import "strings"
import "libsisimai.org/sisimai/v5"

func main() {
    args := sisimai.Args() // sis.DecodingArgs{}
    list := []string{"Return-Path", "From"}

    args.Callback0 = func(arg *sisimai.CallbackArg0) (map[string]interface{}, error) {
        data := make(map[string]interface{})
        data["queue-id"] = ""

        for _, e := range list {
            ff := strings.ToLower(e)
            if len((*arg).Headers[ff]) == 0 || (*arg).Headers[ff][0] == "" { continue }
            data[ff] = (*arg).Headers[ff][0]
        }

        if arg.Payload != nil && len(*arg.Payload) > 0 {
            name := "X-Postfix-Queue-ID"
            mesg := *arg.Payload
            p0 := strings.Index(mesg, "\n" + name + ": "); if p0 < 0 { return data, nil }
            p1 := p0 + len(name) + 3
            p2 := strings.Index(mesg[p1:], "\n");          if p2 < 0 { return data, nil }
            data["queue-id"] = mesg[p1:p1 + p2]
        }
        return data, nil
    }
    args.Callback1 = func(arg *sisimai.CallbackArg1) (bool, error) {
        if nyaan := ioutil.WriteFile("/tmp/sisimai-decoded-1.eml", []byte(*arg.Mail), 0600); nyaan != nil {
            return false, nyaan
        }
        return true, nil
    }

    sisi, nyaan := sisimai.Rise("STDIN", args)
    if len(sisi) > 0 {
        list = append(list, "Queue-ID")
        for _, e := range sisi {
            // e is a sis.Fact struct
            fmt.Printf("Recipient = %s\n", e.Recipient.Address)
            re, as := e.Catch.(map[string]interface{})
            if as == false { continue }
            for _, f := range list {
                ff := strings.ToLower(f)
                if ca, ok := re[ff].(string); ok {
                    fmt.Printf("- Catch[%s] = %s\n", f, ca)
                }
            }
        }
    }
    if len(nyaan) > 0 { fmt.Fprintf(os.Stderr, "%v\n", nyaan) }
}
                

JSON string by dump()

The JSON string obtained by passing a hook method to the dump() method of the Sisimai class will have the following contents:

[
  {
    "addresser": "kijitora@google.example.com",
    "messageid": "vDcsF1hr4zzywp69@mail4.example.com",
    "recipient": "kijitora@google.example.com",
    "diagnosticcode": "host gmail-smtp-in.l.google.com[64.233.188.26] said: The MAIL FROM domain [mail4.example.com] has an SPF record with a hard fail policy (-all) but it fails to pass SPF checks with the ip: [192.0.2.24]. To best protect our users from spam and phishing, the message has been blocked. For instructions on setting up authentication, go to https://support.google.com/mail/answer/81126#authentication 2-222222630d46222222b002b2d2neko2cat2222222cat.22 - gsmtp (in reply to end of DATA command)",
    "decodedby": "Postfix",
    "destination": "google.example.com",
    "alias": "neko@example.co.jp",
    "reason": "authfailure",
    "lhost": "gmail-smtp-in.l.google.com",
    "senderdomain": "google.example.com",
    "token": "fe75b94dd0489b106fa041c460064dd6afa8196b",
    "subject": "Nyaan",
    "deliverystatus": "5.7.26",
    "timezoneoffset": "+0900",
    "listid": "",
    "origin": "set-of-emails/maildir/bsd/rhost-google-03.eml",
    "action": "failed",
    "timestamp": 1649514885,
    "hardbounce": 0,
    "rhost": "gmail-smtp-in.l.google.com",
    "catch": {
      "size": 4436,
      "kind": "Mailbox",
      "authentication-results": "secure.example.jp; spf=fail smtp.helo=mail4.example.com",
      "return-path": "<>",
      "queue-id": "Lms5yfCw0Vz5CRp3"
    },
    "diagnostictype": "SMTP",
    "replycode": 550,
    "feedbacktype": "",
    "smtpcommand": "DATA"
  }
]