Sisimai: Mail Analyzing Interface

Get Started

Environment

Sisimai runs on systems capable of executing binaries compiled with Go: go build command, as well as Perl and Ruby scripts (including some versions of JRuby)."

Read

How To Install

Sisimai can be downloaded from CPAN (for the Perl version), RubyGems (for the Ruby version), and GitHub. (for the Perl, Ruby, and Go versions). The Go version can also be installed using go get.

Read

Basic Usage

This article will introduce you to the basics of using Sisimai. We will cover how to decode bounce emails, retrieve structured decoded data, and convert it to JSON.

Read

How To Decode

More advanced topics, such as reading from STDIN, including delivery success in the decoded results, and incorporating custom processing using the callback feature.

Read

System Requirements

Sisimai runs on systems capable of executing binaries compiled with Go: go build command, as well as Perl and Ruby scripts (including some versions of JRuby).

It is important to review the release notes and the differences with Sisimai 4 page before installing or upgrading, as there are several breaking changes in Sisimai 5.0.0 that may affect your use of the software.

Perl 5.26.0 or later

Download from CPAN

Ruby 2.4.0/ JRuby 9.2 or later

Download from RubyGems

Go 1.17 or later

go get

Install Sisimai

Sisimai can be downloaded and installed from GitHub, CPAN, RubyGems.org, or go get command. Additionally, the Perl version of Sisimai can be installed from the FreeBSD ports collection or Debian packages.

It is important to review the release notes and the differences with Sisimai 4 page before installing or upgrading, as there are several breaking changes in Sisimai 5.0.0 that may affect your use of the software.

From GitHub

The latest version and stable version of Sisimai, including development versions, can be downloaded from the 5-stable branch or the 4-stable branch of the Sisimai GitHub repository. Both the Perl and Ruby versions are available in this repository.

On February 2, 2024, the default branch of the repository was changed to 5-stable.

Perl

Perl version of Sisimai(v5)

The repository name for the Perl version of Sisimai is sisimai/p5-sisimai. Sisimai 5.0.0 and later requires Perl 5.26.0 or later, so please check the version of Perl installed on your system before installing. 

% perl -v

This is perl 5, version 30, subversion 0 (v5.30.0) built for darwin-2level

Copyright 1987-2019, Larry Wall
...
                
% git clone https://github.com/sisimai/p5-sisimai.git
% cd ./p5-sisimai
% make install-from-local
./cpanm --sudo . || ( make cpm && ./cpm install --sudo -v . )
--> Working on .
Configuring Sisimai-v5.0.0 ... OK
Building and testing Sisimai-v5.0.0 ... Password: <sudo password here>
OK
Successfully installed Sisimai-v5.0.0
1 distribution installed

% perl -MSisimai -lE 'print Sisimai->version'
5.0.0

A previous version (v4)

As of February 2, 2024, the default branch of the repository has been changed to 5-stable. If you want to use Sisimai, please specify 4-stable as the branch name. 

% git clone -b 4-stable https://github.com/sisimai/p5-sisimai.git
Ruby

Ruby version of Sisimai

The repository name for the Ruby version of Sisimai is sisimai/rb-sisimai. Sisimai 5.0.0 and later requires Ruby 2.4.0 or later, so please check the version of Ruby installed on your system before installing. 

% ruby -v
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin21]
                
% git clone https://github.com/sisimai/rb-sisimai.git
% cd ./rb-sisimai
% sudo make depend install-from-local
gem install bundle rake minitest
...
3 gems installed
if [ -d "/usr/local/jr" ]; then \
    PATH="/usr/local/jr/bin:$PATH" /usr/local/jr/bin/gem install bundle rake minitest; \
fi
...
3 gems installed
/opt/local/bin/rake install
sisimai 5.0.0 built to pkg/sisimai-5.0.0.gem.
sisimai (5.0.0) installed.
if [ -d "/usr/local/jr" ]; then \
    PATH="/usr/local/jr/bin:$PATH" /usr/local/jr/bin/rake install; \
fi
sisimai 5.0.0 built to pkg/sisimai-5.0.0-java.gem.
sisimai (5.0.0) installed.

A previous version (v4)

As of February 2, 2024, the default branch of the repository has been changed to 5-stable. If you want to use Sisimai, please specify 4-stable as the branch name. 

% git clone -b 4-stable https://github.com/sisimai/rb-sisimai.git
go get
Go

Go version of Sisimai

The Go version of Sisimai is installed into $GOMODCACHE directory using the go get command. Since libsisimai.org/sisimai package does not contain a file declaring the main package, you need to create a directory and import it as a dependency, as shown below. 

$ mkdir ./sisimai
$ cd ./sisimai
$ go mod init example.com/sisimaicli
go: creating new go.mod: module example.com/sisimaicli

$  go get -u libsisimai.org/sisimai@latest
go: added golang.org/x/net v0.35.0
go: added golang.org/x/text v0.22.0
go: added libsisimai.org/sisimai v0.0.1

$ cat ./go.mod
module example.com/sisimaicli

go 1.20

require (
	golang.org/x/net v0.35.0 // indirect
	golang.org/x/text v0.22.0 // indirect
	libsisimai.org/sisimai v0.0.3 // indirect
)
CPAN
Perl

From CPAN

The release version of Perl Sisimai can only be downloaded from CPAN. It can also be installed using the cpan command, but using the cpanm or cpm commands makes installing Perl modules much easier. 

% cpanm --sudo Sisimai
--> Working on Sisimai
Fetching http://www.cpan.org/authors/id/A/AK/AKXLIX/Sisimai-5.0.0.tar.gz ... OK
...
1 distribution installed
% perldoc -l Sisimai
/usr/local/lib/perl5/site_perl/5.30.0/Sisimai.pm

If you want to install an older version, please specify the URL as follows: 

% cpanm --sudo https://cpan.metacpan.org/authors/id/A/AK/AKXLIX/Sisimai-v4.25.16.tar.gz
% sudo cpm install https://cpan.metacpan.org/authors/id/A/AK/AKXLIX/Sisimai-v4.25.16.tar.gz
gem
Ruby

From RubyGems.org

The release version of Ruby Sisimai can only be downloaded from RubyGems.org and installed with the gem command. If you are using JRuby, you can also install it with jgem. 

% sudo gem install sisimai
Fetching: sisimai-5.0.0.gem (100%)
Successfully installed sisimai-5.0.0
Parsing documentation for sisimai-5.0.0
Installing ri documentation for sisimai-5.0.0
Done installing documentation for sisimai after 6 seconds
1 gem installed

If you want to install an older version, please specify the desired version with the -v option as follows: 

% sudo gem install sisimai -v 4.25.16
Ports
Perl

From the Ports Collection(FreeBSD)

The Perl version of Sisimai can be installed from FreeBSD Ports Collection (mail/p5-Sisimai). It can be installed with a single make command, but please note that if the timing of the inclusion of Perl version of Sisimai in the Ports collection is not the latest version, you may not be installing the latest version of Sisimai. 

# cd /usr/ports/mail/p5-Sisimai
# make install
===> License BSD2CLAUSE accepted by the user
===>  p5-sisimai-4.25.16 depends on file: /usr/local/sbin/pkg - found
=> Sisimai-v4.25.16.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
...
apt
Perl

From Debian package

The Perl version of Sisimai can be installed from Debian or Ubuntu package (libsisimai-perl). It can be installed with a single apt copmmand, but please note that if the timing of the inclusion Perl version of Sisimai in the Debian/Ubuntu packages is not the latest version, you may not be installing the latest version of Sisimai 

# apt install libsisimai-perl
Reading package lists... Done
Building dependency tree
Reading state information... Done
...
Need to get 366 kB/563 kB of archives.
After this operation, 1892 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://jp.archive.ubuntu.com/ubuntu focal/universe amd64 libclass-accessor-lite-perl all 0.08-1 [8136 B]
Get:2 http://jp.archive.ubuntu.com/ubuntu focal/universe amd64 libsisimai-perl all 4.25.5-1 [358 kB]
Fetched 366 kB in 2s (171 kB/s)
...
Setting up libjson-perl (4.02000-2) ...
Setting up libsisimai-perl (4.25.5-1) ...
Setting up libjson-xs-perl (4.020-1build1) ...
Processing triggers for man-db (2.9.1-1) ...

Basic Usage

rise()

rise() method of Sisimai

The following code is an example of how to use the rise() method of Sisimai to decode bounce emails and obtain data from the decoded results. For more information on the other arguments that can be specified for the rise() method, please see the How To Decode. The old make() method was deprecated in Sisimai 5.0.0.

Perl

Perl version of Sisimai

#! /usr/bin/env perl
use Sisimai;
my $v = Sisimai->rise('/path/to/mbox'); # or Path to Maildir

if( defined $v ) {
    for my $e ( @$v ) {
        print $e->addresser->address;   # "michitsuna@example.org" # From
        print $e->recipient->address;   # "kijitora@example.jp"    # To
        print $e->recipient->host;      # "example.jp"
        print $e->deliverystatus;       # "5.1.1"
        print $e->replycode;            # "550"
        print $e->reason;               # "userunknown"
        print $e->origin;               # "/var/spool/bounce/new/1740074341.eml"
        print $e->hardbounce;           # 1
    }
} else {
    # There is no bounce message in the mailbox
    # or Sisimai could not decode
}
Ruby

Ruby version of Sisimai

#! /usr/bin/env ruby
require 'sisimai'
v = Sisimai.rise('/path/to/mbox') # or Path to Maildir

if v.is_a? Array
  v.each do |e|
    puts e.class                # Sisimai::Fact
    puts e.recipient.class      # Sisimai::Address
    puts e.timestamp.class      # Sisimai::Time

    puts e.addresser.address    # "michitsuna@example.org" # From
    puts e.recipient.address    # "kijitora@example.jp"    # To
    puts e.recipient.host       # "example.jp"
    puts e.deliverystatus       # "5.1.1"
    puts e.replycode            # "550"
    puts e.reason               # "userunknown"
    puts e.origin               # "/var/spool/bounce/Maildir/new/1740074341.eml"
    puts e.hardbounce           # true
  end
else
  # There is no bounce message in the mailbox
  # or Sisimai could not decode
end
Go

Go version of Sisimai

The following code is an example of using Sisimai's Rise() function to decode bounced emails and obtain the decoded data. For other arguments that can be specified for the Rise() function, please refer to How To Decode with Sisimai. 

$ vi ./sisid.go
package main
import "os"
import "fmt"
import "libsisimai.org/sisimai"

func main() {
	path := os.Args[1]     // go run ./sisid /path/to/mailbox or maildir/
	args := sisimai.Args() // sis.DecodingArgs{}

	// sisi is a pointer to []sis.Facti
	sisi, nyaan := sisimai.Rise(path, args)
	if len(*sisi) > 0 {
		for _, e := range *sisi {
			// e is a sis.Fact struct
			fmt.Printf("- Sender is %s\n", e.Addresser.Address)
			fmt.Printf("- Recipient is %s\n", e.Recipient.Address)
			fmt.Printf("- Recipient host is %s\n", e.Recipient.Host)
			fmt.Printf("- Delivery status is %s\n", e.DeliveryStatus)
			fmt.Printf("- SMTP reply code is %s\n", e.ReplyCode)
			fmt.Printf("- Bounced due to %s\n", e.Reason)
			fmt.Printf("- Email file is %s\n", e.Origin)
			fmt.Printf("- Hard bounce is %t\n", e.HardBounce)
		}
	}
	// nyaan is a pointer to []sis.NotDecoded
	if len(*nyaan) > 0 { fmt.Fprintf(os.Stderr, "%v\n", *nyaan) }
}
                
$ CGO_ENABLED=0 go build -o ./sisid ./sisid.go
$ ./sisid /path/to/mailbox

Dump as JSON

dump()

dump() method of Sisimai

The dump() method introduced in Sisimai v4.1.27 is useful for obtaining decoded results in JSON format. As shown below, it can be done with a simple code that only passes the PATH of the email you want to decode.

Perl

Perl version of Sisimai

#! /usr/bin/env perl
use Sisimai;
print Sisimai->dump('/home/postmaster/Maildir/cur');
Ruby

Ruby version of Sisimai

#! /usr/bin/env ruby
require 'sisimai'
puts Sisimai.dump('/home/postmaster/Maildir/cur')
Go

Go version of Sisimai

package main
import "fmt"
import "libsisimai.org/sisimai"

func main() {
	args := sisimai.Args()
	json, _ := sisimai.Dump("/home/postmaster/Maildir/cur", args)
	if json != nil && *json != "" { fmt.Printf("%s\n", *json)   }
}
jq

Filter using "jq" utility

The jq command is useful for visually inspecting the JSON data of the decoded results. As shown below, the output of the dump() method can be displayed in an easy-to-read format.

Perl

Perl version of Sisimai

% perl -MSisimai -lE 'print Sisimai->dump(shift)' path/to/mbox | jq .
Ruby

Ruby version of Sisimai

% ruby -rsisimai -e 'puts Sisimai.dump($*.shift)' path/to/mbox | jq .
Go

Go version of Sisimai

% go build -o sisid ./sisid.go && ./sisid path/to/mbox | jq .
JSON

JSON string filtered by the jq utility

[
  {
    "destination": "google.example.com",
    "lhost": "gmail-smtp-in.l.google.com",
    "hardbounce": 0,
    "reason": "authfailure",
    "catch": null,
    "addresser": "michitsuna@example.jp",
    "alias": "nekochan@example.co.jp",
    "smtpagent": "Postfix",
    "smtpcommand": "DATA",
    "senderdomain": "example.jp",
    "listid": "",
    "action": "failed",
    "feedbacktype": "",
    "messageid": "hwK7pzjzJtz0RF9Y@relay3.example.com",
    "origin": "./gmail-5.7.26.eml",
    "recipient": "kijitora@google.example.com",
    "rhost": "gmail-smtp-in.l.google.com",
    "subject": "Nyaan",
    "timezoneoffset": "+0900",
    "replycode": 550,
    "token": "84656774898baa90660be3e12fe0526e108d4473",
    "diagnostictype": "SMTP",
    "timestamp": 1650119685,
    "diagnosticcode": "host gmail-smtp-in.l.google.com[64.233.187.27] said: This mail has been blocked because the sender is unauthenticated. Gmail requires all senders to authenticate with either SPF or DKIM. Authentication results: DKIM = did not pass SPF [relay3.example.com] with ip: [192.0.2.22] = did not pass For instructions on setting up authentication, go to https://support.google.com/mail/answer/81126#authentication c2-202200202020202020222222cat.127 - gsmtp (in reply to end of DATA command)",
    "deliverystatus": "5.7.26"
  }
]