* How to recover from s6-rc broken pipe? @ 2020-12-16 17:56 Joshua Ismael Haase Hernández 2020-12-16 20:11 ` Laurent Bercot 0 siblings, 1 reply; 3+ messages in thread From: Joshua Ismael Haase Hernández @ 2020-12-16 17:56 UTC (permalink / raw) To: supervision [-- Attachment #1: Type: text/plain, Size: 540 bytes --] I'm using s6-rc to manage services and have been changing databases. For some unknown reason sometimes the update fails with the error: s6-rc-update: fatal: unable to read /run/s6-rc/state: Broken pipe When that happens, I cannot use s6-rc anymore: /run/s6-rc does not exists, but s6-rc declares it as if it does: sudo s6-rc-init /run/s6 s6-rc-init: fatal: unable to supervise service directories in /run/s6-rc/servicedirs: File exists Creating the s6-rc symlink does not improve the situation. How should I recover from this error? ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: How to recover from s6-rc broken pipe? 2020-12-16 17:56 How to recover from s6-rc broken pipe? Joshua Ismael Haase Hernández @ 2020-12-16 20:11 ` Laurent Bercot [not found] ` <CA+ypG2Y2ogrRxESR+r-_RkN9gUovRbk_vhbXr9DWQEd3kW0ZzQ@mail.gmail.com> 0 siblings, 1 reply; 3+ messages in thread From: Laurent Bercot @ 2020-12-16 20:11 UTC (permalink / raw) To: Joshua Ismael Haase Hernández, supervision >I'm using s6-rc to manage services and have been changing databases. > >For some unknown reason sometimes the update fails with the error: > >s6-rc-update: fatal: unable to read /run/s6-rc/state: Broken pipe That should definitely not happen. Have your databases been built with the same version of s6-rc as the one you're using? (Normally they're compatible, but there has been an incompatible change between 0.3 and 0.4) >When that happens, I cannot use s6-rc anymore: > >/run/s6-rc does not exists, but s6-rc declares it as if it does: s6-rc-update removes the /run/s6-rc symlinks when it fails? If it does, it's a bug that I will fix for the next release. >s6-rc-init: fatal: unable to supervise service directories in >/run/s6-rc/servicedirs: File exists > >Creating the s6-rc symlink does not improve the situation. > >How should I recover from this error? It is possible that you have a bunch of dangling symlinks in /run/service, that were pointing to your old live directory, are not valid anymore, but are still preventing s6-rc from doing its job. Generally speaking, s6-rc-update failing is bad news, because it is difficult to do the proper cleanups (either automatically when s6-rc-update fails, because some operations cannot be rolled back, or manually afterwards), so yeah, your scandir may be in an ugly state and you may need to remove all the symlinks there, delete all the /run/s6-rc* directories and/or symlinks, and restart from scratch. Depending on the changes your oneshots made on your system, you may get error when running them again, too, so in the worst case, the only good option might be to reboot. Sorry. But really, the original cause of the problem should not happen, and s6-rc-update should not be failing like this. If it does not happen all the time, is something overwriting your state file? or removing the /run/s6-rc symlink? If your databases are compatible, then there is definitely some external interference here. -- Laurent ^ permalink raw reply [flat|nested] 3+ messages in thread
[parent not found: <CA+ypG2Y2ogrRxESR+r-_RkN9gUovRbk_vhbXr9DWQEd3kW0ZzQ@mail.gmail.com>]
* Re: How to recover from s6-rc broken pipe? [not found] ` <CA+ypG2Y2ogrRxESR+r-_RkN9gUovRbk_vhbXr9DWQEd3kW0ZzQ@mail.gmail.com> @ 2020-12-16 21:34 ` Laurent Bercot 0 siblings, 0 replies; 3+ messages in thread From: Laurent Bercot @ 2020-12-16 21:34 UTC (permalink / raw) To: Joshua Ismael Haase Hernández, supervision > sudo rsync -avr --delete compiled/ /etc/s6-rc/compiled/ > sudo s6-rc-update /etc/s6-rc/compiled/ There's your problem. You're overwriting the live database; that will throw a wrench into everything. s6-rc-update needs to run while the live database is still the old one. Only after s6-rc-update has completed can you delete the old database. Do something like this instead: stamp=`echo | s6-tai64n` s6-rc-compile /etc/s6-rc/compiled-$stamp sources... s6-rc-update /etc/s6-rc/compiled-$stamp old=`readlink /etc/s6-rc/compiled` ln -sf compiled-$stamp /etc/s6-rc/compiled.new rename /etc/s6-rc/compiled.new /etc/s6-rc/compiled rm -rf /etc/s6-rc/$old -- Laurent ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-12-16 21:34 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-12-16 17:56 How to recover from s6-rc broken pipe? Joshua Ismael Haase Hernández 2020-12-16 20:11 ` Laurent Bercot [not found] ` <CA+ypG2Y2ogrRxESR+r-_RkN9gUovRbk_vhbXr9DWQEd3kW0ZzQ@mail.gmail.com> 2020-12-16 21:34 ` Laurent Bercot
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).