Bug#868503: exim4-config: update-exim4.conf brokenness

[astian]

Package: exim4-config
Version: 4.89-3
Severity: normal

Dear maintainer,

Brokenness in the debconf part of the package has resulted in all local
email (sent from system daemons to users in the same host) to become
undeliverable and therefore frozen.

Background:

  Exim4 was automatically installed on the affected system, probably due
  to package recommendations.  It had been working "without apparent
  problems" (local email was being successfully delivered) since the
  operating system installation (about a month ago).

  As best as I recall, I was never presented with any debconf prompt of
  any kind for this package, not during the system installation, nor at
  any later time (including package upgrades, of which I think 2 took
  place).

  This week I noticed that no new email was being received.  After
  investigating I discovered that since at least 2 weeks ago all email
  was being frozen.

The reason for this was that Exim was using "(none" (sic) as value for
"qualify_domain" (from macro ETC_MAILNAME), while variable
"local_domains" had been set to "@:localhost:(none)" (from macro
MAIN_LOCAL_DOMAINS).  Therefore incoming messages using unqualified
local usernames as source and/or destination addresses were not
recognised as local:

  2017-07-10 06:26:20 1dUrQy-00032m-1m <= root@(none U=root P=local S=702
  2017-07-10 06:26:20 1dUrQy-00032m-1m ** root@(none R=nonlocal: Mailing to remote domains not supported
  2017-07-10 06:26:20 1dUrQy-00032o-5d Error while reading message with no usable sender address (R=1dUrQy-00032m-1m): at least one malformed recipient address: root@(none - domain missing or malformed
  2017-07-10 06:26:20 1dUrQy-00032m-1m Process failed (1) when writing error message to root@(none (frozen)

Notice that "(none)" is Linux's default hostname.  IIUC, systemd will
call sethostname early during boot to change it to something else (taken
either from /etc/hostname or a build-time fallback), so I don't quite
understand how the debconf scripts in exim4-config got that value from
the "hostname" command (which seems to be the only method used to learn
the hostname).  I believe my /etc/hostname was always properly set up
(and even if it wasn't, the fallback in systemd should have replaced
"(none)"; see function hostname_setup in systemd), so this has me
puzzled.

Possible leads
--------------

IIRC, the hostname of the system at installation time was simply
"localhost".  Some time ago I made two changes that may or may not have
caused the problem to surface:

  1. I changed the hostname using hostnamectl to a single word, say,
     "foohost" (both "transient" and "static").  I also added "foohost" to
     /etc/hosts as an alias.
  2. I changed the name of the administrator user ("oldusername", which
     appears below in some configuration dug up by reportbug).

Brokenness
----------

The patch for bug #215319 (reported 2003, closed 2006) was supposed to
result in just the first word of /etc/mailname being used for macro
ETC_MAILNAME, but what it really did is to erase "non-word" characters
after the first span of (seemingly arbitrarily chosen) "word"
characters: the regex is not anchored; also, it probably wasn't meant to
allow backslashes, but it did.  This explains why "(none)" was turned
into "(none".

In 2007, the badly named filter "check_ascii_pipe" was introduced: it
didn't just "check", it also transformed; it didn't just filter
non-ASCII, it filtered anything outside an apparently arbitrary set of
characters; again it allowed probably unintended characters; missing
shell quoting, it could result in an incorrect error message; that error
message is as wrongly worded as the function name.  During 2007, the
maintainer kept increasing the set of allowed characters in the
mentioned filter, not because they were expected in /etc/mailname but
because the same filter was used in a bunch of places with different
requirements; this made the filter "loose".  Eventually he needed to add
square brackets: continuing to repeat the backslash mistake the regex
was broken, resulting in only filtering characters followed by "]".

(Looking at the code and Git log, it appears that the maintainer
struggled to write simple shell scripts; or RTFM.)

Conclusion
----------

About update-exim4.conf: I was going to send a patch (it's trivial) but
after reading the script, I believe the best course of action here is
not a patch but a flamethrower.

About the weird configuration state observed in the affected system: I
don't understand how it happened; manually running "dpkg-reconfigure
exim4-config" was enough to clean things up, "apt purge exim4" further
cleaned things.

About the frozen messages: In case it could be useful to someone, I'll
mention that Exim seems perfectly capable of delivering messages to
"@(none)" addresses, as weird a domain as that seems.  So after manually
editing the configuration and thawing the frozen messages they will be
correctly delivered.

Thanks to the maintainer for his volunteer work, and apologies for the
acrimony.

Cheers.

-- Package-specific info:
Exim version 4.89 #2 built 15-Jul-2017 10:46:16
Copyright (c) University of Cambridge, 1995 - 2017
(c) The Exim Maintainers and contributors in ACKNOWLEDGMENTS file, 2007 - 2017
Berkeley DB: Berkeley DB 5.3.28: (September  9, 2013)
Support for: crypteq iconv() IPv6 GnuTLS move_frozen_messages DKIM DNSSEC Event OCSP PRDR SOCKS TCP_Fast_Open
Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch cdb dbm dbmjz dbmnz dnsdb dsearch nis nis0 passwd
Authenticators: cram_md5 plaintext
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir/mailstore autoreply lmtp pipe smtp
Fixed never_users: 0
Configure owner: 0:0
Size of off_t: 8
Configuration file is /var/lib/exim4/config.autogenerated
# /etc/exim4/update-exim4.conf.conf
#
# Edit this file and /etc/mailname by hand and execute update-exim4.conf
# yourself or use 'dpkg-reconfigure exim4-config'
#
# Please note that this is _not_ a dpkg-conffile and that automatic changes
# to this file might happen. The code handling this will honor your local
# changes, so this is usually fine, but will break local schemes that mess
# around with multiple versions of the file.
#
# update-exim4.conf uses this file to determine variable values to generate
# exim configuration macros for the configuration file.
#
# Most settings found in here do have corresponding questions in the
# Debconf configuration, but not all of them.
#
# This is a Debian specific file

dc_eximconfig_configtype='local'
dc_other_hostnames='(none)'
dc_local_interfaces='127.0.0.1 ; ::1'
dc_readhost=''
dc_relay_domains=''
dc_minimaldns='false'
dc_relay_nets=''
dc_smarthost=''
CFILEMODE='644'
dc_use_split_config='false'
dc_hide_mailname=''
dc_mailname_in_oh='true'
dc_localdelivery='mail_spool'
mailname:(none)
# /etc/default/exim4
EX4DEF_VERSION=''

# 'combined' -	 one daemon running queue and listening on SMTP port
# 'no'       -	 no daemon running the queue
# 'separate' -	 two separate daemons
# 'ppp'      -   only run queue with /etc/ppp/ip-up.d/exim4.
# 'nodaemon' - no daemon is started at all.
# 'queueonly' - only a queue running daemon is started, no SMTP listener.
# setting this to 'no' will also disable queueruns from /etc/ppp/ip-up.d/exim4
QUEUERUNNER='combined'
# how often should we run the queue
QUEUEINTERVAL='30m'
# options common to quez-runner and listening daemon
COMMONOPTIONS=''
# more options for the daemon/process running the queue (applies to the one
# started in /etc/ppp/ip-up.d/exim4, too.
QUEUERUNNEROPTIONS=''
# special flags given to exim directly after the -q. See exim(8)
QFLAGS=''
# Options for the SMTP listener daemon. By default, it is listening on
# port 25 only. To listen on more ports, it is recommended to use
# -oX 25:587:10025 -oP /var/run/exim4/exim.pid
SMTPLISTENEROPTIONS=''

-- System Information:
Debian Release: buster/sid
  APT prefers unstable-debug
  APT policy: (500, 'unstable-debug'), (500, 'unstable'), (500, 'testing'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.9.0-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages exim4-config depends on:
ii  adduser  3.115
ii  debconf  1.5.63

exim4-config recommends no packages.

exim4-config suggests no packages.

-- Configuration Files:
/etc/exim4/passwd.client [Errno 13] Permission denied: '/etc/exim4/passwd.client'

-- debconf information:
  exim4/dc_postmaster: oldusername
  exim4/dc_smarthost:
  exim4/dc_relay_nets:
  exim4/dc_relay_domains:
  exim4/mailname: (none)
  exim4/dc_eximconfig_configtype: local delivery only; not on a network
  exim4/dc_local_interfaces: 127.0.0.1 ; ::1
  exim4/dc_minimaldns: false
  exim4/dc_readhost:
  exim4/use_split_config: false
  exim4/dc_localdelivery: mbox format in /var/mail/
  exim4/dc_other_hostnames: (none)
  exim4/no_config: true
  exim4/exim4-config-title:
  exim4/hide_mailname: