UPS monitor/shutdown service

stoicmaverick@lemmy.world · 1 day ago

UPS monitor/shutdown service

stoicmaverick@lemmy.world · 20 hours ago

Thanks. Good info. Feel like sharing your conf file to see how it’s formatted in an example that actually works? Might make this my weeks evening project.

tychosmoose@lemmy.world · 19 hours ago

It’s a lot. But here are my most recent notes.

Note that the pipe & lock locations different by distro, as does the nut username. So these files won’t drop in and work anywhere.

The dummy driver will need to be installed and configured.

Mine also have several layers of timers on battery an online:

NUT server: In ONBATT I use sleep commands to time subsequent commands. These will always run.
NUT server: Onbatt5 is a upssched timer I set to 300s and is cancelled if the UPS returns to line power.
NUT client: Onbatt30s starts on the client when it gets an ONBATT status from the server.

Most configs would have the client side cancel their timers if power returns. In my case I don’t want that because I prefer more certainty of state so that I know it’s safe to power cycle an outlet to bring a device back up. My NUT server runs on an old pi 3b with a read-only file system. So it never needs a safe shutdown. It’s the last man standing to bring everything else back up if the UPS returns to line power before the battery is exhausted. That’s why there is no local shutdown on the NUT server config.

Each snmpset... is a truncated version of an outlet power command.

Also, the wall commands were a debug aid. Uncomment to use, but test a command first. Wall doesn’t work on some distros.

NUT Server

NUT.CONF:

	MODE=netserver

UPS.CONF:

	[ups]
	desc = My UPS
	driver = dummy-ups
	port = /opt/ups/ups.seq

UPSD.CONF:

	LISTEN 0.0.0.0 3493
	## all interfaces
	## STATEPATH /var/run/nut

UPSD.USERS:

	[upsuser]
	  password = <pw>
	  actions = set
	  upsmon master
	[upssrv]
	  password = <pw>
	  actions = set
	  upsmon slave
	[upspi]
	  password = <pw>
	  actions = set
	  upsmon slave

UPSMON.CONF:

	RUN_AS_USER nut
	#RUN_AS_USER root
	MONITOR ups@localhost 1 upsuser <pw> master
	NOTIFYMSG ONLINE "UPS %s on line power"
	NOTIFYMSG ONBATT "UPS %s on battery"
	NOTIFYMSG LOWBATT "UPS %s battery is low"
	NOTIFYMSG FSD "UPS %s: forced shutdown in progress"
	NOTIFYMSG COMMOK "Communications with UPS %s established"
	NOTIFYMSG COMMBAD "Communications with UPS %s lost"
	NOTIFYMSG SHUTDOWN "Auto logout and shutdown proceeding"
	NOTIFYMSG REPLBATT "UPS %s battery needs to be replaced"
	NOTIFYMSG NOCOMM "UPS %s is unavailable"
	NOTIFYMSG NOPARENT "upsmon parent process died - shutdown impossible"
	
	NOTIFYFLAG ONLINE WALL+EXEC+SYSLOG
	NOTIFYFLAG ONBATT WALL+EXEC+SYSLOG
	NOTIFYFLAG LOWBATT SYSLOG+WALL
	NOTIFYFLAG FSD SYSLOG+WALL
	NOTIFYFLAG COMMOK SYSLOG+WALL
	NOTIFYFLAG COMMBAD SYSLOG+WALL
	NOTIFYFLAG SHUTDOWN SYSLOG+WALL
	NOTIFYFLAG REPLBATT SYSLOG+WALL
	NOTIFYFLAG NOCOMM SYSLOG+WALL
	NOTIFYFLAG NOPARENT SYSLOG+WALL
	
	MINSUPPLIES 1
	SHUTDOWNCMD "/sbin/shutdown -P +0"
	NOTIFYCMD "/usr/sbin/upssched"
	POLLFREQ 5
	POLLFREQALERT 5
	HOSTSYNC 15
	DEADTIME 15
	# POWERDOWNFLAG /var/run/killpower
	RBWARNTIME 43200
	NOCOMMWARNTIME 300
	FINALDELAY 5
	CERTVERIFY 0
	FORCESSL 0
	
	UPSSCHED.CONF:
	CMDSCRIPT /etc/nut/upssched-cmd
	PIPEFN /var/run/nut/upssched.pipe
	LOCKFN /var/run/nut/upssched.lock
	
	AT ONBATT ups@localhost CANCEL-TIMER online3
	AT ONBATT ups@localhost CANCEL-TIMER online10
	AT ONBATT ups@localhost START-TIMER onbatt5 300
	AT ONBATT ups@localhost EXECUTE onbattnotify
	
	AT ONLINE ups@localhost CANCEL-TIMER onbatt5
	AT ONLINE ups@localhost START-TIMER online3 180
	AT ONLINE ups@localhost START-TIMER online10 600
	AT ONLINE ups@localhost EXECUTE onlinenotify

UPSSCHED-CMD

	#! /bin/bash
	#
	# This script should be called by upssched via the CMDSCRIPT directive
	# The first argument passed to your CMDSCRIPT is the name of the timer
	# from your AT lines.
	logger -t upssched-cmd "Calling upssched-cmd $1"
	wall "Calling upssched-cmd $1"
	UPS="ups@localhost"
	HOST=$( uname -n)
	TIMESTAMP=$(date +%Y%m%d%H%M%S)
	STATUS=$( upsc $UPS ups.status )
	CHARGE=$( upsc $UPS battery.charge )
	CHMSG="UPS[STATUS]:$CHARGE%"
	case $1 in
			onbattnotify)
					# srv and pi are upsmon clients and will shut down on their own after a 30s delay.
					# Log the Notification.
					logger -t upssched-cmd "UPS on Battery Power. Servers will shutdown in 30 seconds. Switch shutdown in 5 minutes. $CHMSG $TIMESTAMP"
					wall "upssched-cmd onbattnotify: UPS on Battery Power. Servers will shutdown in 30 seconds. Switch shutdown in 5 minutes. $CHMSG"
	#                wall "upssched-cmd: sleeping to give servers time to poweroff"
					sleep 120s
					# Power off PDU outlets to save power.
					logger -t upssched-cmd "UPS on Battery Power. PDU outlets 4-srv-drives, 5-srv, 7-pi and 8-homeassistant power down NOW. $CHMSG $TIMESTAMP"
					snmpset ...
					snmpset ...
					snmpset ...
					snmpset ...
	#                wall "End upssched-cmd $1"
					;;
			onbatt5)
					# Shut down servers
					logger -t upssched-cmd "UPS on Battery Power. PDU outlets 3-switch and 8-homeassistant power down NOW. $CHMSG $TIMESTAMP"
					snmpset ...
					snmpset ...
	#                wall "End upssched-cmd $1"
					;;
			onlinenotify)
					# Log the notification.
					logger -t upssched-cmd "UPS back on Line Power. Bringing up network in 3 minutes, servers in 10 minutes. $CHMSG $TIMESTAMP"
	#                wall "End upssched-cmd $1"
					;;
			online3)
					# Power on PDU outlets for main switch to prep for server startup.
					logger -t upssched-cmd "UPS back on Line Power. Bringing up PDU outlet 3-switch NOW. $CHMSG $TIMESTAMP"
					snmpset ...
	#                wall "End upssched-cmd $1"
					;;
			online10)
					# Power on PDU outlets for servers now.
					logger -t upssched-cmd "UPS on Line Power. Bringing up PDU outlets 5-srv, 7-pi, 8-homeassistant NOW. $CHMSG $TIMESTAMP"
					snmpset ...
					snmpset ...
					snmpset ...
	#                wall "End upssched-cmd $1"
					;;
			upsgone)
					logger -t upssched-cmd "UPS communication has been gone awhile on $HOST. $TIMESTAMP"
					;;
			)
					logger -t upssched-cmd "Unrecognized command: $1"
					;;
	esac

NUT Client

NUT.CONF

MODE=netclient

UPS.CONF

maxretry = 3

UPSD.CONF

# empty

UPSD.USERS

# empty

UPSMON.CONF

RUN_AS_USER nut  
  
MONITOR ups@192.168.0.77 1 upspi <pw> slave  
NOTIFYCMD /usr/sbin/upssched  
SHUTDOWNCMD "sudo /sbin/shutdown -P +0"  
  
NOTIFYMSG ONLINE "UPS %s on line power"  
NOTIFYMSG ONBATT "UPS %s on battery"  
NOTIFYMSG LOWBATT "UPS %s battery is low"  
NOTIFYMSG FSD "UPS %s: forced shutdown in progress"  
NOTIFYMSG COMMOK "Communications with UPS %s established"  
NOTIFYMSG COMMBAD "Communications with UPS %s lost"  
NOTIFYMSG SHUTDOWN "Auto logout and shutdown proceeding"  
NOTIFYMSG REPLBATT "UPS %s battery needs to be replaced"  
NOTIFYMSG NOCOMM "UPS %s is unavailable"  
NOTIFYMSG NOPARENT "upsmon parent process died - shutdown impossible"  
  
NOTIFYFLAG ONLINE WALL+EXEC+SYSLOG  
#+SYSLOG+WALL  
NOTIFYFLAG ONBATT WALL+EXEC+SYSLOG  
#+SYSLOG+WALL  
NOTIFYFLAG LOWBATT SYSLOG+WALL  
NOTIFYFLAG FSD SYSLOG+WALL  
NOTIFYFLAG COMMOK SYSLOG+WALL  
NOTIFYFLAG COMMBAD SYSLOG+WALL  
NOTIFYFLAG SHUTDOWN SYSLOG+WALL  
NOTIFYFLAG REPLBATT SYSLOG+WALL  
NOTIFYFLAG NOCOMM SYSLOG+WALL  
NOTIFYFLAG NOPARENT SYSLOG+WALL  

MINSUPPLIES 1  
POLLFREQ 10  
POLLFREQALERT 5  
HOSTSYNC 15  
DEADTIME 15  
RBWARNTIME 43200  
NOCOMMWARNTIME 300  
FINALDELAY 5

UPSSCHED.CONF

CMDSCRIPT /etc/nut/upssched-cmd  
PIPEFN /run/nut/upssched.pipe  
LOCKFN /run/nut/upssched.lock  
  
AT ONBATT ups@192.168.0.77 EXECUTE onbattnotify  
AT ONBATT ups@192.168.0.77 START-TIMER onbatt30s 30

# Disabled the online cancel actions in favor of having certainty on the state.
# Servers will go down on an OB event, but that's fine. When Nutpi sees it back
# OL it will bring up the servers
#AT ONLINE ups@192.168.0.77 CANCEL-TIMER onbatt30s  
#AT ONLINE ups@192.168.0.77 EXECUTE onlinenotify

UPSSCHED-CMD

#! /bin/sh  
#  
# This script should be called by upssched via the CMDSCRIPT directive.  
#  
# The first argument passed to your CMDSCRIPT is the name of the timer  
# from your AT lines.  
  
logger -i -t upssched-cmd "Calling upssched-cmd $1"  
UPS="ups@192.168.0.77"  
HOST=$( uname -n)  
STATUS=$( upsc $UPS ups.status 2>&1 | grep -v '^Init SSL' )  
CHARGE=$( upsc $UPS battery.charge 2>&1 | grep -v '^Init SSL' )  
CHMSG="$UPS[$STATUS]:$CHARGE%"  
  
case $1 in  
onbattnotify)  
# Log the Notification.  
logger -t upssched-cmd "UPS on Battery Power. Shutting down $HOST in 30 seconds. $CHMSG"  
wall "upssched-cmd: UPS on Battery Power. Shutting down $HOST in 30 seconds. $CHMSG"  
;;  
onbatt30s)  
# Shut down servers  
logger -t upssched-cmd "UPS on Battery Power. $HOST shutting down NOW. $CHMSG"  
wall "upssched-cmd: UPS on Battery Power. $HOST shutting down NOW. $CHMSG"  
sudo shutdown -P +0  
;;  
# onlinenotify)  
# # Log the notification.  
# logger -t upssched-cmd "UPS back on Line Power. $HOST is already online. $CHMSG"  
# wall "upssched-cmd: UPS back on Line Power. $HOST is already online. $CHMSG"  
# ;;  
upsgone)  
logger -t upssched-cmd "UPS communication has been gone awhile on $HOST."  
;;  
*)  
logger -t upssched-cmd "Unrecognized command: $1"  
;;  
  
esac